Off the Beat: Bruce Byfield's Blog

User testing is often limited in free software. However, long-time advocate Deb Nicholson is developing a simple but effective way around the limitations: getting developers and users together and calling the result SpinachCon. 

The idea of SpinachCon came through Nicholson's work with OpenHatch, a non-profit with the goal of encouraging new contributors to join projects suited to their expertise and interest. Nicholson wanted to apply some of OpenHatch's ideas about receiving user feedback to MediaGoblin, another project with which she is involved. "But it's not worth getting people in for twenty minute at a time," she says. "So I thought, what if you could do a bunch of projects at the same time?"

At first Nicholson named the idea ComplainCon, but "people thought that sounded really negative. So we developed the metaphor of what happens when you have a bit of spinach between your teeth, and you need a friend to let you know. We assume that the intent is good, but there are just a few problems. SpinachCon sets up a kind of camaraderie."

The metaphor immediately suggests a low-key, non-confrontational approach with a purpose that is clear to everybody -- especially when Nicholson modified the black and white image of Gnash's GNU by adding a bright green speck between its teeth.

How it works
Basically, the user-testing equivalent of a hackathon or a bug-stomping party, SpinachCon had its first test-run on March 21 (https://www.fsf.org/blogs/community/the-0th-spinachcon). Four projects participated: GNU Mailman, GNU MediaGoblin, LibreOffice, and Inkscape.

The format is simple. Each project had its own table, with computers ready to use, and developers sitting by to help. Users could also set up on their own laptops.

The 25 users were mostly 20-55, and about 40% women. Many were either local social activists or attendees of the Free Software Foundation's annual LibrePlanet Convention. Greeted by Nicholson, the users received a brief orientation, and were then invited to fill any vacancies at the tables. If no vacancies were available, users were invited to socialize while snacking on pizza and (naturally) "a big batch of spinach salad." When vacancies opened, Nicholson announced them to users not currently testing.

At each project, users were given a list of tasks to attempt. At the end of their efforts, they were asked to grade the software they used on ten point scales for aesthetics, intuition, user-friendliness, and professionalism. Additional questions asked how the software compared with proprietary equivalents, and what might help the software to become more widely used.

According to Nicholson, most users tried at least two projects. The degree of completion might have been higher, except that there were sometimes waits of 15-20 minutes for a vacancy, and some users came too late to do more testing.

Next
When I talked to Nicholson, participating projects were still evaluating their results. However, she did mention some of MediaGoblin's results: an endorsement of the black background on the project site, the observation that no confirmation dialogue appeared when uploading a file, and various navigation problems.

In addition, MediaGoblin's lead programmer Chris Webber apparently had a typical initial reaction that, as Nicholson phrased the situation, "People looked at it in ways I never even thought of." Like most developers, Nicholson notes, "because he's so deep in the code, usability's kind of invisible to him."

Conversely, Nicholson suggests that meeting developers face to face helps users overcome their pre-conceptions of developers. "The way that computer experts are portrayed in the media is really a hackneyed trope: a big guy, in a black T-shirt, with a big bag of Cheetos who lives in his parents' basement and has had a date like never. It's not an incentive to participate in the development of free software if you think that's who you're going to be hanging out with."

 The trial run did have some minor problems. Most users, Nicholson found preferred a mouse to the touchpad on a laptop, so the next SpinachCon will have more mouses available. In addition, users at the next SpinachCon will be encouraged to bring their own laptops to help reduce waiting times."
 

Generally, though, the enthusiasm for the idea of SpinachCon ran high. "I got a lot of people saying, 'This is fun. When are you going to do it again?'" says Nicholson. At LibrePlanet over the next couple of days, OpenHatch made SpinachCon an official project, which should help refine it. In addition, Nicholson met two people who already wanted to put on their own SpinachCons.

However, perhaps the best indication that SpinachCon is an idea whose time has come occurred during the event itself. When talking about some of the feedback, a representative of GNU Mailman and its archiver HyperKitty, remarked, "Oh, that's spinach we know about." With the language already being shaped by its central metaphor, you can be sure that SpinachCon is an idea whose time has come.