GNOME Gets Formal, Public Usability Testing

Off the Beat: Bruce Byfield's Blog
By definition, usability testing is difficult in free software. The reason is obvious: usability testing typically requires face to face observation of users, which is hard to arrange when most developers are interacting remotely. That's why Aakanksha Gaur's recent blogs about GNOME 3 usability caught my attention -- to the best of my knowledge, the last major usability study of GNOME took place twelve years ago, although small, informal studies have been done since.
Gaur is graduate student studying interface design at the National Institute of Design in Bangalore, India. She is also currently an intern in the Outreach Program for Women, mentoring under Allan Day. With Day's assistance, Gaur has decided to focus her internship on answering two questions: "What are the usability issues encountered by new and existing users of GNOME 3?" and "How is GNOME 3 perceived by new and existing users?"
Cynics might remark that these questions should have been asked several releases ago (in fact, in the comments on her blog, they already have). Yet in all fairness, GNOME's relative neglect of usability testing is mirrored in most free software projects, and is less extreme than many.
Day describes the previous usability test for GNOME 3 as "ad hoc." For instance, he himself did some testing when GNOME 3.0 was released "with friends and family," and recently conducted a "small study" on the lock screen design that "led to a number of design changes." Other designers, he adds, have done similar work. Usually, though, like most projects, GNOME mostly depends on developer's reactions, bug reports, and user comments to improve usability.
By contrast, what Gaur is doing rarely occurs in GNOME or most other projects. As Day comments, "While small-scale testing has been done on GNOME 3, this will be one of the first opportunities we have had to do an extended research study."
The importance of publishing
An intermittent GNOME user herself, Gaur characterizes free software as "a developer-driven world for very valid reasons," but one with "a huge potential to include usability and user-centric design." However, she sees some progress in GNOME's use of "design wikis" and the fact that volunteers "are asked to follow specific design exercises." She also notes that both Mozilla and Wikimedia Foundation have shown interest in design recently.
Gaur hopes that her study will assist some of these trends -- not just because it is being done, but because she is making her results public so that others can benefit from them. "I have continuous discussions with [Day] about pouring my data back into the community and making my research actionable," she emailed me recently. "This will happen in the form of bug reports, infographics, and blog posts. Reports of my project, including transcripts of user interviews, will also be made available for anyone who is interested."
These comments bring up another point: despite free software's tradition of openness, the little usability testing that has been done at various projects has generally not been publicized or published -- an omission that has made the development of a user-centric development model in free software much harder. If carried out, Gaur's publication of her data means that other projects, not just GNOME, can potentially benefit from her research.
The early stages
So far, Gaur has reported her study in some detail. She has studied a number of applications, including some on the Mac, focusing particularly on To Do list utilities. Her latest blog entries explain the advantages of taking on the task, as well as describing her research into the metaphor of the desktop and written critiques of GNOME 3 that describe users' experiences.
Gaur has also detailed her failures. Talking about an attempt to study how people used software by asking them constant questions as they work, she writes self-mockingly in one blog, "I was under the confident assumption that I shall take long interviews of users and magically they will reveal the design mistakes which we shall fix and hence, rule the world."
Instead, what her example shows is that the feedback she received "turned out to be very vague and very unfocused. I ended up putting words in the mouth of the interviewee and eventually the whole exercise did not yield good insights. By insights, I mean, data that challenged my existing beliefs about the system in any way."
By contrast, in the one piece of research she has described so far she has chosen to assign three users of varying ability a simple task in GNOME Character Map, and to observe how they go about it. From these observations, she produced a list of goals for the redesign, as well as a list of use cases to take into account, and did some sketches for a redesign.
Gaur is currently deciding on the tasks and applications for the similar tests she plans over the next few months, and plans to continue blogging about them.
Beyond that, I suppose, lies the task of synthesizing her findings to answer her two general questions -- something that, considering some of the recent animosity to GNOME, is almost sure to be greeted by discussion and controversy.
Yet I suspect that her individual studies and openness may prove at least as important as her conclusions in the long run. If Gaur continues as she started, she will provide a professional example of how to conduct usability testing, and that is something that free software developers could use every bit as much as GNOME could use her conclusions.
comments powered by DisqusIssue 269/2023
Buy this issue as a PDF
Subscribe to our Linux Newsletters
Find Linux and Open Source Jobs
Subscribe to our ADMIN Newsletters
Find SysAdmin Jobs
News
-
Kubuntu Focus Announces XE Gen 2 Linux Laptop
Another Kubuntu-based laptop has arrived to be your next ultra-portable powerhouse with a Linux heart.
-
MNT Seeks Financial Backing for New Seven-Inch Linux Laptop
MNT Pocket Reform is a tiny laptop that is modular, upgradable, recyclable, reusable, and ships with Debian Linux.
-
Ubuntu Flatpak Remix Adds Flatpak Support Preinstalled
If you're looking for a version of Ubuntu that includes Flatpak support out of the box, there's one clear option.
-
Gnome 44 Release Candidate Now Available
The Gnome 44 release candidate has officially arrived and adds a few changes into the mix.
-
Flathub Vying to Become the Standard Linux App Store
If the Flathub team has any say in the matter, their product will become the default tool for installing Linux apps in 2023.
-
Debian 12 to Ship with KDE Plasma 5.27
The Debian development team has shifted to the latest version of KDE for their testing branch.
-
Planet Computers Launches ARM-based Linux Desktop PCs
The firm that originally released a line of mobile keyboards has taken a different direction and has developed a new line of out-of-the-box mini Linux desktop computers.
-
Ubuntu No Longer Shipping with Flatpak
In a move that probably won’t come as a shock to many, Ubuntu and all of its official spins will no longer ship with Flatpak installed.
-
openSUSE Leap 15.5 Beta Now Available
The final version of the Leap 15 series of openSUSE is available for beta testing and offers only new software versions.
-
Linux Kernel 6.2 Released with New Hardware Support
Find out what's new in the most recent release from Linus Torvalds and the Linux kernel team.