Is the AI model powering your favorite vibecoding app actually usable by everyone, or just a privileged few? A new open-source offering from the Global Accessibility Awareness Day (GAAD) Foundation and ServiceNow aims to help developers assess how effectively coding-focused large language models (LLMs) generate accessible code. Called the AI Model Accessibility Checker (AIMAC), it provides benchmarks that can evaluate their models’ output to see if it’s truly inclusive.
“Accessibility must be a foundational requirement as AI reshapes our digital future,” Joe Devon, GAAD co-founder, says in a statement. “With AI adoption accelerating, there’s a risk of the industry becoming a ‘winner takes all’ space dominated by a handful of companies. If accessibility isn’t prioritized, people with disabilities risk being systemically excluded from AI’s transformative potential.”
AIMAC is available for download on GitHub and operates as an extensible evaluation framework. It sends prompts to AI models and then reviews the returned HTML code to see if it meets accessibility standards. GAAD claims AIMAC features customizable prompts, allowing developers to use the framework for any use case from design to layout and semantic structure. A comparative score is then generated to help developers identify which LLM can produce accessible code.
Subscribe to The AI Economy
“Accessibility should never be an afterthought. It must be embedded into every phase of the product development lifecycle,” Eamon McErlean, ServiceNow’s vice president and global head of accessibility, remarks. “While the technology industry has made progress, accessibility was an afterthought for far too long. We can’t let history repeat itself with AI.”
The launch of AIMAC is the result of Devon and McErlean’s professional connection. The two individuals have spent the past several years collaborating on accessibility. They’re also hosts of the “Accessibility and Gen AI” podcast. According to a press release, the two pursued AIMAC because, while there are other accessibility checkers in the market today, they claim none are open-source, LLM-agnostic performance checkers “with the potential for widespread impact.”
When asked which models had already been tested with AIMAC, a spokesperson hinted at “a few surprises.” However, the results won’t be released for a few more days, as the team is still preparing a write-up to provide proper context. That being said, it’s commendable that this tool exists, but will it convince developers to swap out their preferred models for one that is better from an accessibility standpoint?
However, what will the model makers’ reaction be when they start to see these evaluations? Will the data incentivize them to adjust their training to improve their LLM?
The AI Model Accessibility Checker is poised to have a significant impact on ensuring AI-powered applications are accessible to all users, including those with disabilities. By providing a comprehensive framework to evaluate the accessibility of code generated by large language models, AIMAC empowers developers to prioritize inclusivity from the ground up. As the technology industry continues to embrace AI, this open-source tool will play a crucial role in shaping a more equitable digital future, where no one is left behind.
Embedding accessibility at the model level ensures it is part of the software’s foundation and not something programmers try to bandage in later. As tech giants like Microsoft, Google, and Meta lean more on AI to generate code, tools like AIMAC offer a much-needed check: Are LLMs producing inclusive and accessible outputs? By providing a standardized way to evaluate accessibility, AIMAC empowers developers to make informed decisions about the models they use, driving progress towards a more equitable digital landscape.
Featured Image: A diverse group of individuals interacts with a futuristic AI interface, showcasing adjustable settings and alternative input methods, symbolizing the inclusive and adaptable nature of accessible AI. Credit: Google Gemini
Leave a Reply
You must be logged in to post a comment.