GNOME Gets Formal, Public Usability Testing

Off the Beat: Bruce Byfield's Blog
By definition, usability testing is difficult in free software. The reason is obvious: usability testing typically requires face to face observation of users, which is hard to arrange when most developers are interacting remotely. That's why Aakanksha Gaur's recent blogs about GNOME 3 usability caught my attention -- to the best of my knowledge, the last major usability study of GNOME took place twelve years ago, although small, informal studies have been done since.
Gaur is graduate student studying interface design at the National Institute of Design in Bangalore, India. She is also currently an intern in the Outreach Program for Women, mentoring under Allan Day. With Day's assistance, Gaur has decided to focus her internship on answering two questions: "What are the usability issues encountered by new and existing users of GNOME 3?" and "How is GNOME 3 perceived by new and existing users?"
Cynics might remark that these questions should have been asked several releases ago (in fact, in the comments on her blog, they already have). Yet in all fairness, GNOME's relative neglect of usability testing is mirrored in most free software projects, and is less extreme than many.
Day describes the previous usability test for GNOME 3 as "ad hoc." For instance, he himself did some testing when GNOME 3.0 was released "with friends and family," and recently conducted a "small study" on the lock screen design that "led to a number of design changes." Other designers, he adds, have done similar work. Usually, though, like most projects, GNOME mostly depends on developer's reactions, bug reports, and user comments to improve usability.
By contrast, what Gaur is doing rarely occurs in GNOME or most other projects. As Day comments, "While small-scale testing has been done on GNOME 3, this will be one of the first opportunities we have had to do an extended research study."
The importance of publishing
An intermittent GNOME user herself, Gaur characterizes free software as "a developer-driven world for very valid reasons," but one with "a huge potential to include usability and user-centric design." However, she sees some progress in GNOME's use of "design wikis" and the fact that volunteers "are asked to follow specific design exercises." She also notes that both Mozilla and Wikimedia Foundation have shown interest in design recently.
Gaur hopes that her study will assist some of these trends -- not just because it is being done, but because she is making her results public so that others can benefit from them. "I have continuous discussions with [Day] about pouring my data back into the community and making my research actionable," she emailed me recently. "This will happen in the form of bug reports, infographics, and blog posts. Reports of my project, including transcripts of user interviews, will also be made available for anyone who is interested."
These comments bring up another point: despite free software's tradition of openness, the little usability testing that has been done at various projects has generally not been publicized or published -- an omission that has made the development of a user-centric development model in free software much harder. If carried out, Gaur's publication of her data means that other projects, not just GNOME, can potentially benefit from her research.
The early stages
So far, Gaur has reported her study in some detail. She has studied a number of applications, including some on the Mac, focusing particularly on To Do list utilities. Her latest blog entries explain the advantages of taking on the task, as well as describing her research into the metaphor of the desktop and written critiques of GNOME 3 that describe users' experiences.
Gaur has also detailed her failures. Talking about an attempt to study how people used software by asking them constant questions as they work, she writes self-mockingly in one blog, "I was under the confident assumption that I shall take long interviews of users and magically they will reveal the design mistakes which we shall fix and hence, rule the world."
Instead, what her example shows is that the feedback she received "turned out to be very vague and very unfocused. I ended up putting words in the mouth of the interviewee and eventually the whole exercise did not yield good insights. By insights, I mean, data that challenged my existing beliefs about the system in any way."
By contrast, in the one piece of research she has described so far she has chosen to assign three users of varying ability a simple task in GNOME Character Map, and to observe how they go about it. From these observations, she produced a list of goals for the redesign, as well as a list of use cases to take into account, and did some sketches for a redesign.
Gaur is currently deciding on the tasks and applications for the similar tests she plans over the next few months, and plans to continue blogging about them.
Beyond that, I suppose, lies the task of synthesizing her findings to answer her two general questions -- something that, considering some of the recent animosity to GNOME, is almost sure to be greeted by discussion and controversy.
Yet I suspect that her individual studies and openness may prove at least as important as her conclusions in the long run. If Gaur continues as she started, she will provide a professional example of how to conduct usability testing, and that is something that free software developers could use every bit as much as GNOME could use her conclusions.
comments powered by DisqusSubscribe to our Linux Newsletters
Find Linux and Open Source Jobs
Subscribe to our ADMIN Newsletters
Support Our Work
Linux Magazine content is made possible with support from readers like you. Please consider contributing when you've found an article to be beneficial.
News
-
The GNU Project Celebrates Its 40th Birthday
September 27 marks the 40th anniversary of the GNU Project, and it was celebrated with a hacker meeting in Biel/Bienne, Switzerland.
-
Linux Kernel Reducing Long-Term Support
LTS support for the Linux kernel is about to undergo some serious changes that will have a considerable impact on the future.
-
Fedora 39 Beta Now Available for Testing
For fans and users of Fedora Linux, the first beta of release 39 is now available, which is a minor upgrade but does include GNOME 45.
-
Fedora Linux 40 to Drop X11 for KDE Plasma
When Fedora 40 arrives in 2024, there will be a few big changes coming, especially for the KDE Plasma option.
-
Real-Time Ubuntu Available in AWS Marketplace
Anyone looking for a Linux distribution for real-time processing could do a whole lot worse than Real-Time Ubuntu.
-
KSMBD Finally Reaches a Stable State
For those who've been looking forward to the first release of KSMBD, after two years it's no longer considered experimental.
-
Nitrux 3.0.0 Has Been Released
The latest version of Nitrux brings plenty of innovation and fresh apps to the table.
-
Linux From Scratch 12.0 Now Available
If you're looking to roll your own Linux distribution, the latest version of Linux From Scratch is now available with plenty of updates.
-
Linux Kernel 6.5 Has Been Released
The newest Linux kernel, version 6.5, now includes initial support for two very exciting features.
-
UbuntuDDE 23.04 Now Available
A new version of the UbuntuDDE remix has finally arrived with all the updates from the Deepin desktop and everything that comes with the Ubuntu 23.04 base.