Meta, the parent company of Facebook, is taking another leap in the Artificial Intelligence (AI) arena. The company has unveiled a new dataset called FACET (FAirness in Computer Vision EvaluaTion) to help researchers and developers evaluate the fairness of their AI models.
The dataset consists of 32,000 images “containing 50,000 people, labelled by expert human annotators for demographic attributes (e.g., perceived gender presentation, perceived age group), additional physical attributes (e.g., perceived skin tone, hairstyle) and person-related classes (e.g., basketball player, doctor). FACET also contains person, hair, and clothing labels for 69,000 masks from SA-1B.”
ALSO READ l ‘While GenAI is booming, it can be very costly’: UiPath India MD says specialised AI ensures high-value outcomes
Meta says that FACET is designed for research evaluation purposes only and cannot be used for training and is designed to address the challenges of benchmarking fairness in computer vision, which is often hampered by potential mislabelling and demographic biases. The company hopes that FACET will become a standard fairness evaluation benchmark for computer vision models and help researchers evaluate fairness and robustness across a more inclusive set of demographic attributes.
Meta hired expert reviewers for every image in FACET to manually annotate person-related demographic attributes. These annotation cover attributes like perceived gender presentation, perceived age group as well as correlating visual features like skin tone, hair type, and accessories. The reviewers also “defined bounding boxes for the people in the image and labeled fine-grained classes related to occupations and activities, such as doctor, disc jockey, or guitarist.”
In addition to the dataset, Meta has also released a dataset explorer that allows users to explore the FACET data and generate reports on fairness metrics. The dataset explorer is available at https://facet.metademolab.com/.
