Accessibility of Command Line Interfaces

2.1.1 Understanding needs of developers with visual impairments. Research into understanding the needs of DWVI has sought to identify the issues developers face and the strategies and workarounds they use while programming [1, 19].

In interviews with eight software developers, Mealin et al. [19] found that DWVI use a variety of assistive technologies in their work, rely heavily on API documentation to get an overview of the code structure, and feel they excel at tasks that do not require vision like algorithm design. They also note the lack of development tool usage by DWVI - most of them used text editors instead of IDEs, printf debugging instead of debuggers, used temporary text buffers for editing code outside the IDE, and preferred languages like Python primarily due to the availability of the interpreter. Albusays et al. [1] conducted a survey of 69 blind developers and a follow-up study with more in-depth observations [2]. The most frequent issues found were around IDE accessibility, debugging, and code navigation. Common workarounds included printf debugging and usage of text editors instead of complex IDEs.

It is interesting to note that many of the workarounds used, such as text editors, printf debugging, and using a Python interpreter, all require developers to interact with a command-line interface.

2.1.2 Building accessible developer tools. Writing, comprehending, and debugging code are canonical developer tasks [29]. There have been multiple efforts in literature to make these experiences accessible to DWVI by building accessible developer tools.

Making code structure accessible: Many cues about the structure of source code like indentation and nesting hierarchies are encoded visually, making it difficult for DWVI to comprehend and navigate code with screen readers efficiently. SructJumper [3], is an Eclipse plugin that creates a tree-based representation based on the hierarchical structure of source code. Developers can use this tree view to comprehend the code's high-level structure and switch to a text editor to get the details. Schanzer et al. [27] extend this approach of making the code structure accessible for multiple programming languages with a language-independent, cloud-based toolkit. The toolkit generates a block editor and audio descriptions for the code structure. Codetalk [25], is a Visual Studio plugin that aims to make multiple facets of IDE and thus software development accessible like code comprehension, debugging, and working in a team. In their formative studies, the authors found that DWVI use CLIs and a text editor as an alternative to using IDEs.

Multimodal cues for encoding structure : A related approach to improving source code comprehension is to encode signals about code structure in the auditory and tactile modalities. Tactile Code Skimmer (TCS) [8] uses haptic cues to convey the shape of code with the intent of reducing the hearing load on developers who already use screen readers. Stefik et al. [32] evaluated auditory representation of a program using ’lexical scoping cues’ that convey the nesting structure of code. They also introduce the concept of ’artifact encoding’ to measure the auditory comprehensibility of auditory program representations. Hutchinson et al. [14] investigated the effectiveness of speech, non-speech, and spearcons (rapid speech-based auditory icons) to convey code structure. They found non-speech sounds to be useful in identifying code constructs. Ludi et al. [18] investigated the role of auditory cues in improving visual programming languages like Scratch. They found that spearcons resulted in the fastest time-to-completion, speech resulted in the most accurate understanding of the code.

Accessible Debugging : Sodbeans [31] is a plugin for NetBeans IDE that uses auditory cues to make debugging accessible. In motivating the work, the authors state command-line based tools are more usable for DWVI than visual programming tools. WAD [30] is an auditory debugger for Visual Studio IDE. It uses auditory cues to improve comprehension during dynamic behaviors in debugging, like tracking program flow and change in variable values.

Research into improving accessibility of developer tools has been concentrated on writing, comprehending, and debugging code. However, in the real world, developers use multiple tools outside their IDEs like documentation, command-line interfaces, source code repositories, code review tools, etc. There is a lack of research into understanding and improving the accessibility of these tools. In this paper, we concentrate on one of these tools - command-line interfaces.

3.1.1 Participants. In this formative study, we interviewed six software developers. The inclusion criteria were that participants self-identified as - a) software developers, b) using screen readers, and c) using command-line interfaces as part of their development workflow. Two of the participants reported using a braille display. Five participants had some form of visual impairment. One participant did not have a visual impairment, but extensively used screen readers as part of their work. The participant details and their assistive technology usage are available in Table. 1. Informed consent was obtained from all participants. Each participant received $175 for their time.

3.1.2 Procedure. The interviews were semi-structured. We asked participants about - a) their workspace and assistive technology usage, b) code development practices, c) the role CLIs played in their developmental workflow, d) the issues they faced with using CLIs, and e) what would be their ideal experience of using CLIs with screen readers.

Four interviews happened remotely over Google Meet. Two participants preferred to be interviewed over the phone. Each interview session lasted approximately 90 minutes. All the interview sessions were recorded and transcribed.

3.2.1 Participants. Six developers took part in this study. The inclusion criteria were identical to the first study - software developers who used CLIs and screen readers as part of their development workflow. All participants in the study reported having some form of visual impairment. All participants self-identified as legally blind. One participant had some light perception. The participant details are available in Table. 2. Each participant received $450 for their time.

3.2.2 Procedure. In the second study, six developers remotely completed four tasks with the gcloud CLI [24] of Google Cloud Platform - a) Install and setup the CLI , b) create a virtual machine in the cloud, c) recognize text from a given image using a public machine learning service and d) deploy a docker container to the cloud infrastructure. All participants chose to use JAWS screen reader on a Windows machine to complete their tasks.

The second study consisted of three parts: a) an onboarding interview explaining the tasks, a brief interview of their code development workflows, and provision of credentials and access to a cloud platform to complete the tasks, b) the participants spent two weeks working on the tasks and recorded their experiences, and c) a final interview reflecting on their experiences of working with CLIs and their accessibility. All communications with the participants used plain text email. Participant P2.1 dropped out of the study after the onboarding session, so their entries are not reported here.

3.2.3 Task Entries. Participants were provided the following structure for recording their entries - a) Give a brief overview of the steps you used to perform this task, b) Did you complete this task?, c) How long did you spend on this task?, d) What accessibility or usability issues did you encounter on this task?, and e) What resources or documentation did you use for this task?

They also recorded their perceived workload in completing each task using a modified NASA TLX scale. The NASA Task Load Index [11] measures subjective mental workload on five 7-point scales with 21 gradations. For this study, we used three NASA TLX scales: performance (success), effort, and frustration. The scales used were 7-point Likert scales, with one being lowest and seven highest. Participants reported their TLX metrics after the completion of each task.

We did not include the physical demand (e.g., physical activity like pulling, pushing) scale from the NASA TLX as this construct was not very relevant to our tasks. We also did not include the temporal demand scale. Instead, we explained the four tasks to the participants during the onboarding process and asked them to estimate how long they anticipated each task would take them to complete. We asked participants to keep track of the time they started and stopped working on each task. After completing the tasks, participants recorded the actual time the task took them. They reflected on the difference between the estimated and reflected time during the off-boarding interview.

4.1.1 Discussion. While the task success rates in Study 2 were high as shown in Table. 3, the effort (shown in Table. 4) and frustration (shown in Table. 5) to achieve the success was also relatively high. This is also reflected in the difference in estimated and actual time in Fig. 1. The relatively high success rate concurs with the perception that CLIs are accessible. However, the high effort and high frustration show that the user experience of using a CLI with a screen reader is sub par. The analysis of open-ended interview responses from Study 1 and Study 2 discussed in Section shed light into the accessibility issues participants encountered.

Graphs for participants P2.2 through P2.6 show that the actual time spent on tasks was dramatically higher than the time they expected the task to take.

Graphs for participants B through F show that the actual time spent on tasks was dramatically higher than the time they expected the task to take.

5.2.1 Navigation in terminal. Participants in both Study 1 and Study 2 consistently noted the difficulty in navigating the terminal with a screen reader. P1.1 and P1.2 noted in particular that simple tasks like scrolling could be painful. P2.2 explained that it is almost like trying to simulate a mouse with a keyboard and often does not work reliably.

P1.1 : Scrolling in terminal is generally a pain

P1.2 : Sometimes the screen reader can't, it's almost as if you can't scroll past a certain boundary. So you have this buffer and you're not able you can't scroll up past a certain point, or you can't scroll down past a certain point.

P2.2 : Because sometimes the limitation of a JAWS cursor, and I think that's the biggest limitation is like, it's using the mouse simulator so sometimes it gets stuck at one point on the screen and then you cannot navigate very well. It kind of sees what it's seeing and then it can only go so far, it doesn't move very well. It's almost like, what we call a mouse cursor getting locked, almost. So, I couldn't navigate everything like I can on the web page.

Given that even simple navigation tasks like scrolling are difficult in a terminal with screen readers, one of the common accessibility issues mentioned was the difficulty in consuming unstructured text that often appears as output of commands.

5.2.2 Man Pages. Man (short for manual) pages are a form of in-terminal documentation for CLI. They have the advantage of being available from within the terminal without having to context switch. They are usually invoked by man [command] or [command name] –help.

Participants in Study 1 called out man pages as a canonical example of unstructured text in a CLI. Fig. 2 shows the man page for the ls command that lists the contents of a directory. While the content in Fig. 2 appears formatted, it is simply not possible to navigate this content like a web page equivalent. For instance, users can not jump to headings or skip sections to reach the location they want. They would have to painstakingly read this content linearly multiple times to get the information they are seeking.

Terminal window showing the man page for the list directory of ”ls” command. The page has a section for the name of the command, the syntax for its arguments under synopsis, and a description section that describes the command and all its possible flags. The page demonstrates visual hierarchy through capitalizing headings, bolding command and flags names and indenting descriptions.

Workarounds : The workarounds participants use for man pages were diverse depending on their background. P1.1 mentioned having to write their own script to render man pages in a browser while P1.3 mentioned they never use man pages and rely only on web-based documentation.

P1.1 : I wrote a script to render man pages in browser

P1.3 : I just fall back to the web if I need to read documentation, I do a web search and go through it that way because man pages are very unstructured in terms of navigating with a screen reader, so it gets very cumbersome to navigate man pages.

This theme emerged primarily from Study 1, since the CLI used in Study 2 offered html version of all reference documentation. So users did not have to rely on just man pages.

5.2.3 Tables.

A cropped terminal window shows a command-line table with 12 columns with numbers and text data. Number data is left-aligned and text data is right-aligned to make them easier to read. The table headers are shown in inverse color to the rest of the cells, making them easy to spot visually.

This theme emerged in both Study 1 and Study 2.

Tables are a common form of unstructured text in CLIs. Many CLI commands present their output in a tabular format. An example is shown in Fig. 3 . While this appears structured with a title, row, and columns, this is just visual formatting, and there is no inherent structure here that is available to screen readers. Users can not easily tab through to a desired row and column or have a row heading announced. They would have to parse linearly and rely on their memory to contextualize the current element at their cursor location.

Most participants’ common workaround was to try to memorize the column names and hypothesize which column an element belongs to as they parse the table linearly.

P1.1: Usually, I just try to keep track of it all in my head because I haven't been able to figure out a good way to navigate tables. So I'd just have to memorize the column names and then manually navigate between each piece of information and try to remember what column it's associated with.

P1.5: A good example is the top command where it has the six or seven columns going across the screen to show the PID and the memory percentage and the CPU percentage and all that, that, those are a little bit challenging to read because, since a screen reader won't be able to interpret that as a table, you have to memorize which numbers go with which columns and which order they will appear in. So, anything that's tabular like that is hard

Some participants mentioned that this process becomes easier as they repeatedly use a command, but this could be an arduous task if the number of columns is large or the first time they encounter the table.

P1.4 on relying on their past experience to read the content in Fig. 3 : you always know the process line that comes first, you know the process then comes next, you know the amount of CPU usage of the next column, memory is the next one

5.2.4 Ideal Experience. When asked what the ideal experience would look like, participants wished that the CLIs would be flexible enough to convert tables into a format more suitable for consumption via screen readers. Popular choices included the ability to transform to flattened tables for easier parsing or being able to export these tables as html or comma-separated files so that they could read these in a browser or spreadsheet software.

P1.4 : I've done stuff like that before where I've written a full script, to just take the output of something that was tabular in nature and literally make it into a table by wrapping it in the table and some <td>s and <tr>s and calling it a day.

P2.5 : I love flattened tables. I absolutely do because thatś just the way that JAWS, interacts in a non-tabular form most of the time. If that exported as a CSV, would pop straight up into [spreadsheet software] and be on my way.

5.2.5 Long output text. This theme also emerged in both Study 1 and Study 2.

Many CLI commands provide long output texts as the response. Due to the difficulties navigating in the terminal, users reported copying the terminal contents to a notepad to review the content. This workaround makes it easier to perform tasks like searching for a keyword.

P1.5: I will have it display all of the text, so even if it's four or five screens ... I will select the entire scrollback buffer and copy it and paste it in Notepad, and then I'll review it in Notepad - it's easier to do that in a Notepad editor window. Or if I want to copy and paste text, that's much easier to do from the editor.

P1.4 : Let's say I'm doing a Port Scan of a network and I have really long output, then what I would do is I would output it to a file

Multiple utilities would allow users to search for a piece of text in a long output from within a CLI (e.g. grep). For instance, in the output snippet illustrated in Fig. 4, the field ”description” contains the most useful information. Using a utility like grep (e.g., [command] | grep ’description’) would make it easy to parse just that line of information.

However, in our study, participants did not report using these utilities. This was because they did not have a mental model of the output they are looking at until they review the content at least once.

Unlike APIs, most CLI commands do not document their output contract. So this means a user has no idea what to expect out of a command until they actually execute it. They do not have a schema in their mind ahead of time for the output they are parsing, which makes it very difficult to comprehend the output and, in particular, search for a specific value.

For instance, PC described his experience of obtaining the output in Fig. 4 like this :

P2.3 : But the results were pretty scrambled, I mean, they, I couldn't make much sense out of it other than the fact that they were, something to do with a graph, Y-axis with an X-axis, coordinates. They were like, Y, left brace, right brace, 500, then XX, then like, it was weird, I thought it was a bit weird.

Terminal window showing 23 lines of coordinates in an object-structure that has been output from a command.

5.3.1 Lack of Status and Progress Indication. Linux has the convention where a silent return without any errors is considered success [26]. This convention translates to many commands, not providing explicit progress and status indication. Unsurprisingly, participants found this paradigm difficult with screen readers. For instance, since the screen readers did not vocalize any status, P2.4 resorted to observing any brightness changes on their screen.

P2.4: Yeah, it might have been something on the screen, but the screen reader didn't say anything. Because I can tell there's much that when I was doing there, the screen would get a little bit darker. That would tell me something is on there, but I don't know, I can't see enough, so I don't know what. But then when I moved my focus away from it the screen would brighten up normal, so there's probably something happening, but I can't tell what's going on.

P1.1 observed this lack of status indication as a general issue when working with CLIs.

P1.1 : sometimes it'll not report status information, and so I've had to guess as to whether it was, in the middle of compiling something then sometimes it would hang. And I wouldn't tell if it had really froze or whether it was still doing something.

P2.6 stated that they ended up creating multiple resources because there was no status message.

P2.6: No it did not tell me. It did not, it didn't, it wouldn't, it didn't tell me and so I thought maybe I hadn't created it. So I created a second, not realising.

5.3.2 Lack of screen reader friendly progress indication. Sometimes CLI commands do provide progress indication. In these cases, they try to simulate GUI-like progress indication mechanisms like progress bars (Fig. 5) and spinners (Fig. 6). However, screen readers do not recognize them as progress indicators and vocalize the underlying Unicode characters.

P2.5 described their experience with the spinner like progress indicator in Fig. 6 as

P2.5 : Little dots submit, little blah, blah, blah, blah, completed with status failure.

A cropped terminal window shows the words ”Uploading files” followed by 4 characters that appear to be solid rectangles and 6 characters that appear to be dots inside unbounded rectangles. This indicates the command is 40% complete.

A cropped terminal window shows the words ”Updating service [default] working” followed by braille dots that function visually rather than semantically.