Celebrating Global Accessibility Awareness Day One Immersive Testing Experience At A Time
The GAAD and Lansing Area Software Testers logos.
Today, May 20, is the third Thursday of May, which means that it is the day that we celebrate Global Accessibility Awareness Day 2021.
As it so happens, today is also the day that the Lansing Area Software Testers Group invited me to present my Experience Report about exploring the accessibility of products/processes as part of their regularly scheduled monthly online meetup. Well, I couldn’t think of any better way to celebrate Global Accessibility Awareness Day than to have this great opportunity to talk to a group of local testers about my own immersive testing experience with the Sakai Project.
In case you weren’t able to attend today’s online meetup, the following is a summary of my presentation. Since I prepared the outline for my talk using the Experience Report format, I thought I would follow that same basic structure for this blog post.
Many thanks to the Lansing Area Software Testers User Group for providing me with this platform for discussing some accessibility testing best practices, and an extra special thank you to Jess Lancaster, QA/Software Testing Practice Manager for TechSmith, who serves as one of the moderators of the group, and helped me to prepare for today’s presentation.
Part 1 - Person(s) completing the work in the context of a real-life organization
The purpose of this talk was to highlight one of Accessiversity Labs’ ongoing projects with the Sakai LMS. Last fall, as part of a long-term retainer contract with Learning Experiences, I began doing accessibility testing and Quality Assurance work for the Sakai Project, an open-source Learning Management System.
Part 2 - Practical goal of work (i.e. identification and/or analysis of existing problem, finding solutions or implementing them in practice)
The goal of inviting me into the Sakai community and having me embed myself with the Sakai QA team was not just to help improve the accessibility of the Sakai LMS product, but to fully assimilate me into the community, to begin assessing every facet of the organization, from the top down, bottom up and inside out, in an effort to make the Sakai community more accessible.
Part 3 - Business/social context of the work
Like most open-source projects, the Sakai LMS is maintained by a distributed community of developers, QA testers and other practitioners sourced from Sakai partner institutions, as well as commercial affiliates like Learning Experiences and Longsight. This is the case for the Sakai QA team as well, which is primarily made up of part-time testers (mostly students from some of the Sakai partner institutions) who rely on a series of online collaboration tools and weekly (virtual) stand-ups/check-ins which are used to measure progress/steer the work of the group.
Part 4 - Methods/theories used to achieve the practical goal of the project
As I have already eluded to, our intent was for this to be an immersive testing experience. So I guess you can say we were using the sink or swim method.
With the goal of becoming a productive member of the QA team as soon as possible, I jumped right in and got to work assessing the tools, systems, and processes being used by the Sakai QA team, starting with the on-boarding document used to orient new QA testers to the Sakai LMS.
As I was introduced to these new tools, systems, and processes, I realized that a new, or at least a modified approach, would need to be developed for someone like me, a blind/low-vision user who relies on assistive technology to perform this work.
For example, I took the GoogleSheets version of the test script that the other QA testers work off of and developed a comparable Excel-based “Accessibility Regression Test Script” version since most blind/low vision users, myself included, are much more comfortable working in Excel than with Google docs. Plus, accessibility testing required that an entire new version of the test cases would need to be written with a focus on keyboard navigation/screen reader use/testing.
Communication/Collaboration Tools
There are several communication tools being used by the Sakai community that I had to figure out how to use with my assistive technology.
Most of the community meetings are conducted on an open-source web conferencing tool called BigBlueButton which has some quirky features that took some getting used to.
Sakai also uses an open source, web-based collaborative real-time editor called Etherpad for meeting notes, agendas, and for attendees to sign-in. Unfortunately, Etherpad is almost completely inaccessible for people who use the JAWS screen reader, so every meeting, I have to close out of JAWS, open another screen reader called NVDA (which stands for Non-Visual Desktop Access,) open a separate browsing session in Firefox, so that I can sign-in, check out the agenda, before closing out of Etherpad, NVDA and Firefox, and relaunching JAWS to continue with the virtual meeting being hosted over on BigBlueButton.
Other tools like Slack have proved to be a little more accessible/user-friendly, while something like Jira (which Sakai uses for submitting/tracking tickets) has plug-ins like “Unstoppable” to supposedly make it more accessible, although these sorts of custom plug-ins are not always very intuitive, requiring the user to essentially learn this whole new additional system with its own rules and ways of doing things.
Documentation, Documentation, Documentation
Of course, another unintended consequence of having gone through this process, is that all of these lessons learned and temporary workarounds that I was able to figure out prompted me to thoroughly document everything I was doing, which I have started to incorporate into “A Screen Reader User’s Guide to Sakai QA Testing” version of the on-boarding manual.
Part 5 - Lessons learned/Conclusions
Early on, this immersive approach started to show promise.
As you would expect, the more I worked in, and was able to familiarize myself with the Sakai LMS, the more I was able to assess the accessibility of its various tools/features.
This doesn’t mean that every issue was black and white, accessible or not accessible – oftentimes, I found that I could usually figure out a workaround to most every problematic tool/feature, even if it wasn’t ideal or optimal for use with my JAWS screen reader.
This was important for a couple of different reasons…
The Gray Area Between Functional Verification & Technical Compliance
In terms of a VPAT (Voluntary Product Accessibility Template) or any other sort of assessment, it’s much better to say that a user is able to complete a task, even if that task is not completely accessible or the method is less than ideal, than to say that something is completely inaccessible.
Leaving A Trail Of Breadcrumbs
During the course of my testing, I made a habit of documenting everything. Not only was this helpful for me when I would need to go back and remember how I completed a particular task, but I knew that my test script would serve as a “how to guide” for any blind/low vision testers/screen reader users who would be following in my footsteps.
The Good, The Bad, & The Ugly
The more I got to use Sakai and its various tools/features, the more I could provide feedback to the Dev team about potential accessibility issues, but also those parts of the Sakai LMS that worked well, sometimes even suggesting that they consider modeling a problematic element on one part of the site after an element on another part of the site which seemed to be configured better for use with my screen reader.
In other words, part of my role morphed into that of translator/interpreter with the goal of improving the feedback loop while decreasing the time it would take for the Dev team to act on accessibility-related issues identified by the QA team.
Interactive Test Sessions Over Zoom
Another development that has occurred in the past couple of months is a weekly interactive test session that I conduct with the person who is the lead for the Jira Triage meetings/ticketing process.
Originally pitched as a “pre-Jira triage meeting,” this Thursday afternoon session over Zoom is an opportunity for me to demonstrate certain issues encountered during the course of my testing that week. I am able to use the screenshare feature in Zoom so that my colleague can see my screen and hear my JAWS screen reader as I go out to some part of the Sakai site to try and recreate the issue. This continues to be an invaluable source of learning—both from the perspective of me having an experienced Sakai user be able to point out certain aspects of the web site to me, as well as an opportunity for members of the Sakai community to acquire a deeper knowledge of what the experience is like for an assistive technology user so they can begin incorporating this into their thinking when designing various aspects of the LMS.
Vision Aid Pilot
The other exciting development that has come about, is a new partnership with an organization called Vision Aid. Vision Aid (who’s U.S. headquarters are in Boston, MA) is an India-based organization which provides IT-related training for blind individuals in India.
In January, I started overseeing a pilot between Learning Experiences and Vision Aid which involves having a couple of blind testers at one of the Vision Aid facilities in India go through and complete testing on Sakai using the Excel-based test script that I had developed.
Why is this important?
First, it serves as further evidence that the adaptive test script could work for blind/low vision testers who use screen readers and need to rely on keyboard navigation. Second, expanding testing to include the work performed through this Vision Aid pilot means that we can have additional blind/low vision testers testing Sakai with different combinations of screen readers and web browsers, helping to paint a more detailed picture of what is going on in Sakai when it comes to the accessibility of its tools/features.
Acceptance Into The Sakai Community
The third thing that has started to happen, which is probably worth mentioning here, is that other members of the Sakai community have become more proactive when it comes to approaching me about potential accessibility issues.
This is maybe the most telling sign that we have succeeded with our goal of my becoming a fully contributing member of the Sakai community, as others within the community recognize me as a subject-matter expert and valued member of the team, and maybe more important, an integral part of the development life cycle.
Key Takeaways
Whenever I do one of these talks, one of my goals is to have the audience walk away with some actionable next steps for how they can start, expand, and/or improve upon their own accessibility testing practices/strategies.
This could include using automated test tools, as well as having developers download free versions of screen readers like NVDA to use during different stages of the software development life cycle.
Of course, being a proponent for “live” (manual) testing, specifically, utilizing individuals with disabilities to perform testing to provide for an “authentic” accessibility experience, at the end of the day, I hope organizations will reach out to Accessiversity to take advantage of the services we are able to provide.
While it’s great whenever an organization commits to doing more in the realm of accessibility, whatever that may be, sighted developers, testers, QA folks—they will never be able to 100% replicate the authentic experience of a blind user, so I would encourage them to think about where they can insert individuals with disabilities into their existing teams/processes.
Headshot of Jess Lancaster.
As Jess Lancaster, QA/Software Testing Practice Manager for TechSmith explains, “As software testers we endeavor to improve the experiences our customers have when it comes to the software teams on which we work. Having Chris present to TechSmith and the Lansing Area Software Testers regarding his experiences using assistive technology, engaging in the accessibility of test processes, and sharing his first-hand knowledge of accessibility is an invaluable opportunity. Sharing those experiences, Chris leads you to understand that improving accessibility is so much more than compliance and automated tools. Chris helped us engage, with empathy, in the real challenges of building more accessible software.”
I know many organizations who put off doing this important work because they feel overwhelmed or because they’re not sure where to start. But accessibility doesn’t have to be this overly complicated, expensive endeavor. Accessiversity offers a number of cost-effective solutions/services, for example, as part of a Phase I assessment, we could focus on those two or three use cases that your organization has deemed most important. Getting started on addressing your accessibility needs can be that simple – so I really want to debunk this myth that accessibility has to be this costly, complex thing that you have to somehow pile onto everything else you are doing.
Another one of my pet peeves, if I am being completely honest, is that many organizations have historically undervalued the experiences of users with disabilities and their contributions to this type of work, which needs to start changing.
My friend Kate Snyder said it best during a recent blog post that we worked on together – we need to start normalizing accommodations, a big part of which is how organizations should approach accessibility/usability testing.
For those of you who have been paying attention to recent A11Y trends, government agencies and others wanting VPATS and similar assessments are starting to ask whether individuals with disabilities have been involved in the testing, so this is not an issue that is likely to go away. That is why Accessiversity is here to stay, and why you’ll be hearing more about the types of immersive accessibility testing described in this Experience Report.