This is a follow-up to my previous post that described my second structured assessment of cognitive Web accessibility. The work’s progression can be seen via this blog’s Category of Cognitive Web Accessibility Assessments.
Assessment Scoring System Needed Revision
The result of the second structured assessment was that the Web site was inaccessible to people with developmental disabilities. Had I followed my original assessment plan, the result would have been the opposite. This is because of the plan’s scoring system.
I had intended to score one point if any Web site feature met even one guideline of each of the sections of WebAIM’s Cognitive Web Accessibility Checklist. While following this system during my second structured assessment, I realized it was too generous. Points were adding up although it was obvious to me the site was likely inaccessible by people with developmental disabilities.
Consequently, I decided to average the number of guidelines that were met (successes) with those that were not (failures) for each checklist section. Arbitrarily but within reason, I also decided an average success of 80% or higher would score one point. I will apply this standard to future assessments unless I learn it is unworkable.
Assessments Require Significant Effort
When I developed my original assessment plan for 100 sites, I had been hoping the work could be performed quickly. That was naive. I now recognize much more work is needed. Every relevant guideline in all of the checklist sections must be evaluated to portray a site’s cognitive Web accessibility as well as possible. Indeed, because I evaluated all the relevant guidelines in the last (and only) structured assessments, comprehensive portrayals of the sites’ cognitive Web accessibility were produced.
Assessments Should Be Performed By Users
The cognitive accessibility of Web sites would be best assessed by users with cognitive disabilities. I was reminded of this by Joe Chidzik after I posted my second structured assessment. Specifically, his message was, “A site may be *likely* to cause accessibility issues, but to claim it does so without user testing isn’t helpful”. Point taken.
Yet this assessment work, in part, is an attempt to find a uniform way for developers to help determine the cognitive accessibility of Web sites. Of the many automated tools that help assess general Web accessibility, none are focused on cognitive Web accessibility. WebAIM is pursuing funding to incorporate such assessment into its WAVE Web accessibility evaluation tool. My work is an unofficial precursor to that. I hope it will be helpful.
That said, I would like to include people with cognitive disabilities in this work. Honestly though, I don’t know how to do it simply and economically. (This project’s work is performed primarily on my own time and is unfunded.) I am open to constructive suggestions. Please post a comment with one.