Adam Hyde

https://tampub.uta.fi/bitstream/handle/10024/81866/gradu04510.pdf

by Finnish student Olli Savolainen

Some quotes (indented). My notes follow some quotes:

Usability work is problematic in open source software development on many levels. There is often no documentation or explicit research of user goals to drive design, no user experience vision to direct development, no usability testing, and typically no resources allocated for usability work. No usability practitioners are usually employed, and doing usability work is hard due to the community value system being focused on concrete results. An attitude of “code is cheap” leads to a tendency to skip user research, design and validation before implementation. However, developers in open source communities do get feedback from actual users.

---

Usability work in open source software projects is challenging. Pemberton (2004) states the problem being as follows: either open source developers need to become interested in ordinary users, or the users’ needs have to be brought to developers by mediators. Indeed, it seems that not only open source communities have a lack of systematic thinking about users, but it is also challenging for usability practitioners to approach open source communities.

I have a small problem with the direction here, and there is something of a tension in the entire thesis with regard to this point: is the ‘fix’ one of tacking on usability to the end of a process (ie. solve it in the last mile). Or is the fix deeper – in the way products are designed in OS? I think the latter, which is why the Cabbage Tree Method is as it is. I don’t mean to mischaracterise the document as there is quite a bit of examination of these two approaches (and some solutions that appear in-between these two poles). However, it is worth noting that ‘usability’ is quite a different problem to how things are designed and how solutions are imagined.

---

Projects are often oriented in a straight-to-the-action manner: the user research and design phases are largely skipped

---

Defining what the software should be like happens iteratively, as development progresses, and replaces the initial planning, requirements analysis, and design phases of conventional development. As such, there may not be an explicit system design, a project plan, a schedule, a list of deliverables, or defined processes (Warsta & Abrahamsson, 2003). The idea of a user interface getting explicitly designed is thus in a sense antithetical to the idea of open source. “Code is cheap” is a general attitude, and the real action happens in programming code. Satisfaction comes from practical problem solving, not from applying abstract usability concepts (such as scenarios or personas). Users cannot be reduced to points in a to do list, so they, too remain abstract (Nielsen & Bødker, 2007) and distant.

I take issue with ‘code is cheap’. I would rather characterise it as code being ‘gold’ in the OS world. It is the only thing of value. UX and design culture (etc) does not have a value and hence does not appear in OS processes or thinking in general.

I don’t disagree with the point being made above, just how it is expressed.

---

It is said that developers care about the quality of their work more, because they only work on things for which they have a real passion (Raymond, 2001). An open source project is typically a meritocracy (Trudelle, 2002) where status (and trust) in the community is gained by tackling problems considered challenging. There is a bias for improvements in functionality instead of usability: Nichols et al. (2001) state that “hard” algorithmic problems may have a greater perceived value in the “reputation market” than “soft” usability related issues, being also easier to specify, evaluate and modularise than usability issues.

I’m also not entirely convinced by this. I think, with the development of libs like Bootstrap, developers think they can also be designers ie good UX is just a matter of putting the right things in the right place.

But once again, this misplaces the real problem. The problem is in how we create (design) solutions and how communities can be formed that have a diversity of the many skills required to develop kick ass software.

---

Still, developer communities can appear inward-looking to users and the community hacker culture closed and idiosyncratic (Nichols et al., 2001). Depending on the application domain, there can be a language barrier between developers and end users. Current systems for reporting software issues problems are reported to be difficult to use and biased towards bugs that can easily be expressed textually or with single screenshots. Usability problems and their interrelations are often hard to express in such issue trackers. In addition, analysis of complex usability problems is difficult with these tools and meaning is often lost (Nichols & Twidale, 2003; Viorres et al., 2007). Users can also be seen as co-developers (Raymond, 2001) – if a user has a usability issue, they may be encouraged to fix the application themselves. Alternatively, developers may find it more convenient to encourage the user to read the documentation in order to work around an issue, perceived to be a problem of the user instead of one of the software.

I, like Jonas, rally against requiring users to become programmers. See http://blog.jonasoberg.net/how-free-software-is-failing-the-users/

---

Twidale and Nichols (2003, 2005) differentiate between objective and subjective usability bugs. It is easy to agree on the severity of objective usability bugs, such as a button not fitting a window in a given situation (rendering the application dysfunctional). On the other hand, developers may be tempted to tag subjective usability bugs as “works for me” in the bug tracking system.

Yes. ‘Works for me’. Says it all really. I want that on a t-shirt.

---

In May 2007, I proceeded to present my prototype to the community in the forum of the Quiz module 25 . I explained the problems with the then-current UI and offered justifications for the new solutions.

The response I got in the forum thread was mostly negative. A considerable number of community members were not even convinced there were any problems with the UI. Instead of changing the UI, many community members saw improving the documentation a more viable way to approach the difficulties users were facing. Tim Hunt, the maintainer of the Quiz module, was worried about the consistency of my proposition with the rest of the Moodle UI.

I was dumbfounded. I spent days trying to find an appropriate reaction to what they were saying. I had spent some weeks creating a prototype and testing it, and I felt was being treated like I had attacked the community. Where not ignored, I thought I had received little relevant feedback on the actual work presented. From the first responses I could read virtually no understanding or seemingly not even an attempt to understand what I was trying to express. There was no counter argumentation to the actual design, just brief comments bypassing my effort as uncalled for. I used quite a bit of personal support from a colleague, friends and family before going back and continuing the discussion to see just how bad the misunderstandings were – either on my side, or theirs.

Not an uncommon story. Developers being hostile to ‘non-coders’. See this saddening discussion: https://ask.slashdot.org/story/14/12/02/007206/ask-slashdot-non-coders-why-arent-you-contributing-to-open-source

---

Hunt’s initial response to me in Spring 2008 was that I did not have sufficient experience in the Moodle project to carry out such big changes. In other words, he referred to the fact that as an OSS project, Moodle is a meritocracy where previous evidence of quality code determines whether one is trusted to work further. I sidestepped the meritocracy by stating that the contribution I had to make was not one of programming, but one of usability work. Even if I had had a history of quality code, that would not have proven my skills in usability work. Hunt also proposed making a group of smaller fixes at first, instead of the complete redesign that I was proposing, which exemplifies the fact that in OSS, iteration appears in actual small steps in source code, whereas in UCD it is usually seen that iteration is best done before any implementation. Suggesting small, incremental changes might also be seen connected with thinking that usability can be added to a UI after development.

I find the above paragraph particularly interesting because it shows the problems of trying to introduce new metrics into an established meritocracy. When doing this you essentially have no currency in this environment by definition, which makes cultural processes very hard to change.

---

Essentially Moodle is still closed source in terms of design (like its proprietary rivals). None of the higher-level thinking about the design of the system is readily available in a manner that would make it possible for anyone to further develop the UI. Such documentation would be critical so that one could know that when doing a redesign, the user needs that were taken into account in while creating the original UI, are still supported. Documenting any of user goals, conceptual models, workflows, personas and scenarios could provide such constraints to base design on.

The above is also interesting in that it suggests that design docs, which usually exist in the head of the BDFL should exist in the open. Else it is a closed, gated, and inaccessible process (the design process). Interesting point.

---

The methods need to be reconceptualized to fit the OSS culture of practice and the corresponding value system, fostering and taking advantage of the OSS social relationships, since these ”motivate attention to usability issues in day-to-day development” (Terry et al., 2010). UCD being unfit as a process for OSS seems a likely reason why usability practitioners often have hard time gaining trust in OSS communities.

---

Thus, the notion of a solution encouraging users to further participate in the design instead of being observed is a very tempting one.

You bet.

However, Iivari (2009) states that in the OSS project she studied, users do not have actual decision-making power regarding the OSS but are left in a consultative role. This seems to be true of most OSS projects. Ultimate decision-making is left to developers only, to whom a very limited amount of understanding about the users of the software is available in the forums of the community (Iivari, 2009). In OSS projects, user involvement is thin from the perspectives of both UCD (no usability practitioners to represent users) and participatory design (the few users who do participate have no true power over the eventual software).

Solved!

See also Ollis recent article on the topic:

https://medium.com/interaction-reimagined/user-experience-design-inviting-the-users-25ccd00711c7