On 30 May 2024, the National Institute of Standards and Technology (NIST) published a report evaluating software that estimates a person’s age from facial photos. This report, the first of its kind in ten years, assessed six algorithms to understand their capabilities and limitations. The study is part of NIST’s broader effort to support age assurance technologies, which are becoming essential in age-restricted activities and online safety regulations.
Key Findings
The report, titled “Face Analysis Technology Evaluation: Age Estimation and Verification” (NIST IR 8525), revealed that none of the evaluated algorithms clearly outperformed the others. NIST found significant variation in age estimates based on facial expressions, eyeglasses, and other factors. For instance, age estimates could differ when the same person wore glasses versus when they did not, and different facial expressions also impacted the accuracy.
Methodology
The evaluation used a diverse set of 11.5 million photos from four U.S. government databases, including visa applications, FBI mugshots, webcam images from border crossings, and immigration application photos. This comprehensive dataset ensured a variety of ages, genders, and regions of origin were represented. NIST maintained the anonymity of the data and conducted the study under rigorous privacy standards.
Performance and Demographics
The study highlighted several points of interest:
- Performance Over Time: Compared to a similar study in 2014, age estimation accuracy has improved. The mean absolute error (MAE) decreased from 4.3 years to 3.1 years.
- Gender Disparities: Error rates were higher for female faces than for male faces, consistent with the findings from the 2014 study.
- Demographic Sensitivity: The accuracy of algorithms varied significantly across different demographic groups, with some algorithms performing better on specific groups than others.
Implications
The findings are essential for developers, policymakers, and users of age estimation technology. As age assurance becomes more embedded in legislation and regulations, especially for protecting children online, the continuous improvement and evaluation of these technologies are crucial.
NIST aims to provide a reliable benchmark for the performance of age estimation software, ensuring that these technologies can meet the evolving demands of privacy and accuracy in age-restricted applications.
My take
While it’s encouraging to see advancements in age estimation technologies, the recent NIST report underscores their current unreliability, as no single algorithm outperformed the others and significant variations in accuracy were noted based on factors like facial expressions and the presence of eyeglasses.
👉 Find the press release here and the report here.
♻️ Share this if you found it useful.
💥 Follow me on Linkedin for updates and discussions on privacy education.
📍 Subscribe to my newsletter for weekly updates and insights – subscribers get an integrated view of the week and more information than on the blog.
