Coko Team Meet 2017, Kenya

Increments

To investigate tools that could enable collaborative design of a product, I have been searching around for ways to display designs, discuss, and iterate amongst a distributed team.

There are some closed source products and services out there, but my I feel kind of dirty looking at them. like I always feel the need to go home and have a shower afterwards.

There don’t seem to be any open source tools out there to do it. I could be wrong, so if I am missing something please email me (adam@booksprints.net).

In part, I think the lack of open source tools for this kind of thing is in itself telling since open source projects itch their own scratch and many don’t see product design as very important.

It would be great if there was something out there that could live alongside GitLab (or the closed source GitHub) and there was a equal two-way integration of tools and processes, coupling technical development with product design.

So, I am making a first attempt with Julien Taquet (a designer) and Juan Gutierrez (a developer) to make a simple tool. We are calling it Increment and it is just a prototype in HTML at the moment. It lives here https://gitlab.coko.foundation/adam/increment

The idea is to develop a simple interface to present mocks (or links to prototypes, text descriptions, videos, etc) and then enable comments around the artefact. Then the discussion can be frozen, a new version posted and the comments can start again but refering to the new artefact.

Here is a screen shot of the ‘home’ page:

increment1-1

When clicking on an item you might see something lke this:

increment2

The idea being that if you click on a version on the left, you view that version and the associated discussion.

Of course, this blog post could really use Increment right now to show you this very concept and get your thoughts! 😉

For now, email me thoughts. We will hopefully have a basic prototype in a week (if you download the repo version and run under localhost then you can see something very basic in action). Later, if there is real utility, we will make a proper project out of it. For now this is a proof of concept.

Extra thoughts from Julien below:

incre-j1

incre-j2
incre-j3

Collaborative Product Development

v 0.9 Mar 22 added some cursory Extreme Programming notes
v 0.8 Mar 20 started reading Diffusion of Innovations (5th ed)
v 0.7 Mar 16 finished reading Xtreme Programming Explained (2nd ed)
v 0.6 Feb 15 “added extreme programming notes”
v 0.5 Feb 13 “notes about ba”
v 0.4.1 Feb 12 “add more Agile notes”
v 0.4 – Feb 11 “added Agile notes”
v 0.3 – Feb 9 “added Waterfall, SDLC, OSS notes”
v 0.2 – Feb 8 “added Radical Application Design (RAD) notes”
v 0.1 – Feb 6So, I am pondering more how to work with product development methods to improve the process for products that are of the ‘open source’ genre. My first step in this process was to write this post:
https://www.adamhyde.net/leadership-diversity-and-sector-change-in-open-source/This has lead to the realisation that open source is really a culture-methodology. While I believe in open source licenses, I think the default culture-method leaves a lot to be desired. So, I am investigating what are the key foundations, processes, and tools that would enable a different kind of product development. At this early stage in my pondering, I think there are several key ingredients:

  • a diverse set of stakeholders
  • facilitation
  • a distributed team
  • open licenses

In addition, I have the following goals:

  • make excellent products
  • enable widespread adoption

The goals are as important as the ingredients. After all, if you have don’t have lemons, you shouldn’t be trying to make lemonade.

I am thinking this exploration could have a working title ‘Collaborative Product Development’.

My first step is to look at historical methods for software product development and see what their approaches have been. The intent being to learn more about how software development processes are targeted at specific objectives and operational contexts. I think this will help develop an understanding of why certain methods are effective for this specific case, which in turn will help me understand which bits can be discarded.

I’ll keep this particular page in my Ghost blog as the notepad for my evolving thoughts.

Initial Thoughts, Learnings, Extractions
  1. There are a number of methods for building software products. Agile, lean, feature-driven method, XP, JAD etc. I would also include Open Source as a default methodology for product development and, even though open source projects can employ these other methods there is a kind of default modus operandi for open source which defines how things are done. Hence I think it can be considered to be a methodology.
  2. I have, this week, been involved in collaborative product design processes. I facilitated a small group (the second session in a series) in the beginnings of the process. The group consisted of those that started the project (owners?), a project manager, and people who knew the workflow intimately and would end up using the software. We started with the whiteboards, and pens. The opening question in the first session, which seemed to open the door to a great conversation, was “how would you design a system if the workflow would be transferred to the browser”. The second session went through the same process to get validation and to expand the group slightly (this was important). We now have a baseline understanding of what we are building. Each session was about 1.5 hrs.
  3. In collaborative design sessions, I think it is important to not rush to implementations too fast. Stay on the whiteboard for as long as necessary – it feels more organic and the output obviously is created by all in the room (as opposed to presenting well thought out ‘finished’ mocks).
  4. move slowly to more sophisticated artefacts (mocks) and get signoff as a group at every step. Give plenty of room for revising issues. Identify big problems but do not rush to solve them. It is ok to push some stuff down the road as long as it is agreed by all and the issue is clearly noted. Document all outputs of collaborative design sessions.
  5. it is ok to keep the documentation a little rough. it communicates that it is a WIP and things can be revised. The quality of the artefacts is secondary to the quality of the discussions about them.
research notes (scratch pad):
  1. http://www.itinfo.am/eng/software-development-methodologies/#chapter3
  2. https://en.wikipedia.org/wiki/Jointapplication design
    “Arnie Lind’s idea was simple: rather than have application developers learn about people’s jobs, why not teach the people doing the work how to write an application? Arnie pitched the concept to IBM Canada’s Vice President Carl Corcoran (later President of IBM Canada), and Carl approved a pilot project. Arnie and Carl together named the methodology JAD, an acronym for Joint Application Design, after Carl Corcoran rejected the acronym JAL, or Joint Application Logistics, upon realizing that Arnie Lind’s initials were JAL (John Arnold Lind).
    The pilot project was an emergency room project for the Saskatchewan Government. Arnie developed the JAD methodology, and put together a one-week seminar, involving primarily nurses and administrators from the emergency room, but also including some application development personnel. The project was a huge success, as the one-week seminar produced a detailed application framework, which was then coded and implemented in less than one month, versus an average of 18 months for traditional application development. And because the users themselves designed the system, they immediately adopted and liked the application. “
  3. learning through collaboration http://www.commonwealthclub.org/events/archive/podcast/joseph-henrich-secret-our-success
    “high fidelity copying from each other” is a secret to our success as a species and because “we are interconnected we can learn stuff drom each other and combine it”. (yesyes!). The ability to innovate “depends on the size and connectivity of the population”. The ability to adapt like this “drove genetic evolution”. Brings up interesting questions on who to learn from. “the size of the population and the degree to which people are interacting and sharing information actually affects how fancy the technology is going to get and how many tools you can produce….also if the the group should get cut off for some reason and the population is reduced, the group will suffer a loss of technology”. “Genetic evolution gives rise to cultural evolution” which in turn drives genetic evolution. “think of our learning abilities as adaptation” -> “cultural driven genetic evolution”. “A lot of creativity is the recombination of different ideas”. “If you want to be creative out yourself at the nexus of very different and divergent information”. “cultural learners” (people who learn from each other). “good cultural learners”.
  4. From the book by Joseph Henrich
    “natural selection favoured genes for building brains with abilities to learn from others.” p53
  5. crystal method considered to be a subset of Agile
    https://en.wikiversity.org/wiki/Crystal_Methods
    Feels very west coast. A bit too hippy dippy in its method names.
  6. https://medium.com/@nayafia/what-success-really-looks-like-in-open-source-2dd1facaf91c
  7. http://www.sersc.org/journals/IJSEIA/vol8no32014/38.pdf
    http://docplayer.net/9016010-A-review-of-open-source-software-development-life-cycle-models.html
    ” The development of OSS gains popularity due to the wide availability of the internet facility to each and every region of the world, parallel development, peer review, parallel debugging, expert developers, and feedback” “Many different developers, user, or co-developers can participate in the development of the OSS.” “The development of OSS is always initiated by single developer or a single group, who starts the development of software for its own “personal itch” “
  8. http://www.oreilly.com/openbook/opensources/book/vixie.html
  9. http://opendesign.foundation/resources/philosophy/why-you-should-design-for-open-source/
  10. http://www.ijcaonline.org/volume21/number1/pxc3873327.pdf More OSS SDLC (OSDLC)..this one a little bit too praisey of open source
  11. http://opendesignnow.org/
  12. http://openusability.org/
  13. http://opendesign.foundation/
  14. https://www.smashingmagazine.com/2010/09/the-case-for-open-source-design-can-design-by-committee-work/
  15. https://github.com/opensourcedesign/resources
  16. http://opensourcedesign.net/
  17. “Extreme Programming (XP) is about social change. It is about letting go of habits and patterns that were adaptive in the past, but now get in the way of us doing our best work. It is about giving up the defenses that protect us but interfere with our productivity. It may leave us feeling exposed. ” Kent Beck, Extreme Programming Explained
  18. amazing talk by Kent Beck https://www.youtube.com/watch?v=aApmOZwdPqA
  19. Agile is Dead https://www.youtube.com/watch?v=a-BOSpxYJ9M
  20. TDD is Dead https://www.youtube.com/watch?v=z9quxZsLcfo
  21. discovering ba
    “Nonaka told his interviewer that they were creating ba, a Japanese term that describes a field or space where people freely and openly share what they know in the service of creating something new. Ba resembles the concept of “flow” as set forth by psychologist Mihaly Csikszentmihalyi: It is the mental state that occurs when a person is fully immersed in whatever he or she is doing. But unlike flow, ba is never solitary; it exists among two or more people. As Nonaka says, “In ba, there is no you or me, there is only us, sharing a here-and-now relationship.” Ba can occur in a work group, a project team, an ad hoc meeting, a virtual e-mail list, or at the frontline point of contact with customers. It serves as a petri dish in which shared in­sights are cultivated and grown.
    Companies can foster ba by designing processes that encourage people to think together.”
    http://www.strategy-business.com/article/08407?gko=a1212
  22. “A phronetic leader mobilizes timely judgment in others by building a culture that is strong, nurturing, and sustained by informal connections.”
    http://www.strategy-business.com/article/08407?gko=a1212
  23. https://hbr.org/1986/01/the-new-new-product-development-game
  24. “Cross-fertilization. A project team consisting of members with varying functional specializations, thought processes, and behavior patterns carries out new product development. The Honda team, for example, consisted of hand-picked members from R&D, production, and sales. The company went a step further by placing a wide variety of personalities on the team. Such diversity fostered new ideas and concepts.”
    https://hbr.org/1986/01/the-new-new-product-development-game
  25. “All of us, as far as I can remember, thought waterfalling of a huge project was rather stupid, or at least ignorant of the realities
 I think what the waterfall description did for us was make us realize that we were doing something else, something unnamed except for “software development.””
    –GM Weinberg http://www.craiglarman.com/wiki/downloads/misc/history-of-iterative-larman-and-basili-ieee-computer.pdf
  26. “The conclusion that stands out most clearly from our field study observations is that the process of developing large software systems must be treated, at least in part, as a learning and communication process.”  Bill Curtis quoted in http://www.craiglarman.com/wiki/downloads/misc/history-of-iterative-larman-and-basili-ieee-computer.pdf

Development Methods

I think I need to cover at least the following questions for each:

  1. what problem was the methodology a response to
  2. what are its key goals
  3. what application is it intended for

In addition, I need enough information to succinctly describe the methodology and to provide some praise and critique.

SDLC

“I think what the waterfall description did for us was make us realize that we were doing something else, something unnamed except for “software development.” – Gerald M Weinberg

The history of Software Development Lifecycles is a history of experiment, description, and rejection.

Systems Development Life Cycle is what I refer to here as ‘method’ or ‘methodology’. It describes any methodology used as a framework for developing software. While all the methods described below can be used in a pure form, often SDLCs are combined and hybridised. Sometime for better, sometimes for worse. However, it is interesting to note that the basic framework address the following needs one way or another:

  1. conception
  2. requirements gathering
  3. design
  4. development
  5. use
  6. evaluation

Some SDLCs are cyclical, some incremental, some strict, others loose, linear etc

The interesting question to me is what can we learn from these if we want to develop a methodology (SDLC) for developing products with free licences and the principles of open source.

Also interesting is that there have been attempts to define OSS as a SDLC. See http://www.sersc.org/journals/IJSEIA/vol8no32014/38.pdf
http://docplayer.net/9016010-A-review-of-open-source-software-development-life-cycle-models.html

Somethings that I notice:

  1. SDLCs are methods that feel like they are to be deployed
  2. you become a subject of a SDLC
  3. OSS feels like a culture you want to inhabit
  4. you become a citizen of OSS

So, I think part of the framing is that OSS ‘SDLCs’ are about creating the right culture and employing tools to shape that. SDLCs (in the traditional) form are about employing a structured method and making sure everyone adheres to it. So, I would prefer to think of the kind of methodology i am looking for to be framed as a ‘Product Development Culture’, not a Software Development Life Cycle. The former is a culture, the later a structured process. The former can be shaped, the later employed. Best to pick the right conditions for success from the beginning 😉 To improve OSS PDC we need the right tools to shape culture – this is the culture-method I am after.

The interesting thing here is that the history of SDLC highlights the problem with history. It is easy to tell a story of a timeline and certain events happening in sequence, almost like building blocks in time. However that isn’t how things happen. We can, for example, set up a nice story about how ‘Waterfall’ came from a pre-software engineering era, and it was simply applied to software development, and then in the 1980s, people started rebelling against this legacy process and developed more iterative methods. However, while there is some level of truth in that story, it also hides the complex paralled, unfloding, emergent processes and histories that is the ‘full telling of this story’. For example, iterative development in software is recorded as happening as early as 1957 (and certainly in the 60s for Project Mercury), but the process evolved out of non-software projects as early as the 1930s. As Larman and Victor Basili noted, “We are mindful that the idea of IID [Interactive and Incremental Development] came independently from countless unnamed projects and the contributions of thousands and that this list is merely representative. We do not meanthis article to diminish the unsung importance of other IID contributors.”
http://www.craiglarman.com/wiki/downloads/misc/history-of-iterative-larman-and-basili-ieee-computer.pdf

While Waterfall is often said to have been born in 1970 (– Royce, Winston (1970), “Managing the Development of Large Software Systems“,) So before we even had waterfall for programming, we had iterative development as this article so nicely highlights: http://www.craiglarman.com/wiki/downloads/misc/history-of-iterative-larman-and-basili-ieee-computer.pdf

What this highlights is that these ideas and methods have evolved in a many faceted, evolutionary manner. There is no simple star wars narrative here of good vs evil although that is often how the history of SDLC is told.

Waterfall

“In my experience, the simpler method has never worked on large software development efforts
” – Royce Winston

Waterfall in software dev was actually developed in the 1970s, which is late for computing. It was first described by Winston Royce in 1970 (“Managing the Development of Large Software Systems“,)
http://ig.obsglobal.com/2013/01/software-history-waterfall-the-process-that-wasnt-meant-to-be/

The steps included:
1. Design comes first
2. Document the design
3. Do it twice
4. Plan, control and monitor testing
5. involve the customer

Interestingly, it has nothing to do with how open source generally works 🙂 except! involve the customer is often also at the end of OSS product development.

Even more interesting…Waterfall as we know it now is really what Royve was criticising when he introduced the term. ie a very strict, sequential process. Conception, initiation, analysis, design, construction, testing, implementation, and maintenance. It is one way process, no going back to an earlier step at any stage. This is known as ‘single pass’ or ‘single step’ Waterall and differentiates it from Royce’s method that advocated building twice (2 pass).

A foundation of Waterfall, as we know it now, is the argument that the more work done gathering requirements and designing, the more time you save and risks you avoid when it comes to building.

Rational Unified Process (RUP)

Adaptive Software Development

Pragmatic Programming

SCRUM

Mid 80s, origins outside of software, namely the 1986 article “The new new product development game” – placed the notion of rugby as a metaphor for product development dynamics and specifically called out the term ‘Scrum’ as a description of small mobile teams.

Joint Application Development

Targetted at the first stage of the SDLC (requirements gathering and design). Before the development of JAD business requirements were gathered from interviewing stakeholders. JAD suggested the solution could be jointly designed. It is a very prescriptive methodology developed by Chuck Morris and Tony Crawford (IBM) in 1977. JAD is a rather strict methodology. The goal of the methodology is to develop business requirements ie. to accelerate the design phase and to increase project ownership. JAD has very specific roles:

  • Executive Sponsor – set the vision, have the final say, can be strategically trusted to help steward the project within the organisation
  • Facilitator – employ the method, plan the workshops
  • Users – comprise 75% of the group, the people that have a need for the system
  • IT – typically the people that will build the system
  • Observer – simply put, they observe, it seems these are people from within the org who have some stake in the process but do not wish to be part of the design phase
  • Scribe – participate only to get clarity, document the design

All participants have equal say.

There are 8 key steps in JAD:

  1. Identify objectives and limitations
  2. Identify success criteria
  3. Define deliverables
  4. Schedule the workshops
  5. Select participants
  6. Organise workshops and lead up prep
  7. Prep the participants
  8. coordinate workshop logistics

The workshop is the centerpoint for this methodology. JAD turns meetings into production orientated workshops (common also in agile etc). The basic idea being that by combining the users and IT staff a better product is designed.

Development lifecycle includes:

  1. conception
  2. requirements
  3. design
  4. implementation
  5. integration
  6. test
  7. maintenance

Method started with all in one room but used more commonly now with remote softwares. It puts emphasis on a diverse team, representing all stakeholders.

Earlier version of “design thinking” or Theory (Japanese)?

“It is critical that the facilitator be impartial and have no vested interest in the outcome of the session.”

“the focus of attention should always be on the JAD process itself, not the individual facilitator.”

“There is a school of thought that trained facilitators can successfully facilitate meetings regardless of the subject matter or their familiarity with it. This does not apply, however, to facilitating meetings to build information systems. A successful IS facilitator needs to know how and when to ask the right questions, and be able to identify when something does not sound right.”
http://www.umsl.edu/~sauterv/analysis/488f01papers/rottman.htm

“Many of the principles originating from the JAD process have been incorporated into the more recent Agile development process. ”

“Where JAD emphasises producing a requirements- specification and optionally a prototype, Agile focusses on iterative development, where both requirements and the end software product evolve through regular collaboration between self-organizing cross-functional teams.”
http://www.chambers.com.au/glossary/jointapplicationdesign.php

Rapid Application Development

Initially developed by Barry Boehm. A response to Waterfall. RAD was a general approach (category of methods) as an alternative to Waterfall. In 1986, Boehm created the articulated Spiral methodology in 1986 which is considered as the first RAD method. Confusingly another RAD method evolved later (which also subscribes to the general RAD approach) by James Martin and named RAD by him.

Waterfall was developed out of pure engineering where strict requirements for a known problem lead to an inflexible design and build. Software however, could radically alter the landscape and hence, in a sense, change the problem space. So the process had to be more flexible, reflective etc. Hence RAD recognised that information gathered during the life of a project had to be fed back into the process to inform the solution.

RAD is generally used for the development of user interfaces and placed a lot of focus on the development of prototypes over rigorous and rigid specifications. Prototypes were considered as a method to reduce risk as they could be developed at anytime to test out certain parts of the application. The emphasis being to find out problems sooner rather than later. Additionally, users are better at giving feedback when using a tool rather than thinking about using the tool, and prototypes can evolve into working code and be used in the final product. The later could lead to users gettting useful functionality in the product sooner.

RAD is considered a ‘risk driven process’ ie. it identifies risk in the hopes to reduce it.

The addition of functionality incrementally is considered an advantage in user facing products as they can access certain functionality sooner. This iterative approach also produces some of the software earlier and hence there is something to show sooner.

Some disadvantages could be a ‘ad-hoc’ design, and the continual involvement of users might actually be difficult to achieve and may either slow down the production or offside stakeholders.

– SPIRAL RAD

This method had four critical steps which were used repeatedly. Hence the project progressed in increments.

  1. determine requirements – gather requirements, determine objectives
  2. identify and resolve risks – identify risks and a solution for them. develop prototypes around the risk-solution.
  3. development – prototype leads to more detailed design and build
  4. plan the next iteration – the output is evaluated and the next iteration is planned

The spiral notion refers to the cyclical process PLUS as the project goes forward more knowledge is gained about the project. For a good description of this dynamic see https://www.youtube.com/watch?v=mp22SDTnsQQ

– RAD RAD

In 1991 James Martin took these ideas and turned RAD processes into another methodology, confuslingly called RAD.

There are four phases in the James Martin-defined RAD methodology:

  1. requirements gathering -t featuring a diversity of stakeholders. Essentially the objectvies, vision and scope is identified and a go-ahead given by those with the capacity to do so. Similar to (1) above.
  2. user design – groups of users and other stakeholders gather to start designing the product. It may, or may not, employ JAD. Prototypes are designed, built and tried where possible.
  3. construction – build starts, users are continously involved hence steps 3 and 4 are cyclical.
  4. cutover – hand over of the project to users, training etc

Dynamic Systems Development Method

Developed out of the RAD group.

Rational Unified Process

Agile

Agile manifesto – 2001
“The basic idea behind iterative enhancement is to develop a software system incrementally, allowing the developer to take advantage of what was being learned during the development of earlier, incremental, deliverable versions of the system. Learning comes from both the development and use of the system, where possible. Key steps in the process were to start with a simple implementation of a subset of the software requirements and iteratively enhance the evolving sequence of versions until the full system is implemented. At each iteration, design modifications are made along with adding new functional capabilities. ” Vic Basili and Joe Turner, 1975
http://www.craiglarman.com/wiki/downloads/misc/history-of-iterative-larman-and-basili-ieee-computer.pdf

Agile is interesting because it is not a method in itself, it is a proposal for a certain kind of development culture, under which various methodologies have come to find a home. Agile, in other words, is a set of values or principles.

Dave Thomas has made a good presentation on the problems with turning Agile into a complex method when actually it was intended as a set of values. https://www.youtube.com/watch?v=a-BOSpxYJ9M

…and his blog post: http://pragdave.me/blog/2014/03/04/time-to-kill-agile/

Many of the famous methodologies for Agile – XP, Crystal, Adaptive Software Development, and SCRUM all point to specific parts of the SDLC and all existed before the term agile was applied to them. These early ‘Agile Methodologies’ were originally refered to as ‘light’ or ‘lightweight’ methodolgies which was really a way to label them in contrast to much more strict, linear and formal methodologies like Waterfall.
http://agilemanifesto.org/history.html

In the first edition of “Extreme Programming Explained” for example, lightweight is used to describe this kind of SDLC
“XP is a lightweight methodology for small-to-medium-sized teams developing software in the face of vague or rapidly changing requirements.” (Kent Beck)

Its also interesting to note that this ‘Agile’ shift in approach to development came around the first .com bubble and reflects the shifting business environment where speed to market was seen as a factor in business success – hence the need for SLDC methods that supported faster product development life cycles.

So, proving once again that one of the most difficult thing for programmers to do is to name things, a small group gathered in 2001 to replace this informal naming with something a little more descriptive and positive (programmers didnt like saying they employed ‘light’ or ‘lightweight’ methods as this implied a lack of concern or vigour over the quality of their work and ability to do it). The group came up with the term Agile to retro-name an existing category and also identified the core characteristics of those methodologies that would conform to the Agile way. This became known as the Agile manifesto and it is as follows:

We are uncovering better ways of developing software by doing it and helping others do it.
Through this work we have come to value:
Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan.
http://www.agilemanifesto.org/

So, in a sense, Agile is a framework, not a methodology. Methodologies conformed to it or, later, evolved from it. It is also interesting as the manifesto not only describes what it is, but what it is not. While this helps understand each value, it also shows that the manifesto very much defined itself as a reaction or remedy to the dominant culture. In some sense Agile represents a determined cultural fork within SDLC.

This is pretty interesting as:

  1. Free software existed as the default state in the 50s and 60s and was only became a movement self identifying as “Free Software” in the early 80s as an attempt to re-state the utility and re-enstate the cultural value of sharing code freely. Hence Free Software was also named after it existed.
  2. Perhaps OSS needs something like a stated manifesto on methodology or method-values. There have been political manifestos but not statements about process (apart from individual project contributing guidelines). Some attempts at a OSS manifesto include the GNU Manifesto (more accurately described as a Free Software Manifesto), and the famous Hacker Manifesto which is more of a ‘Catcher in the Rye’ approach to manifesto writing, and the follow up co-opting of the Hacker Manifesto by McKenzie Walk which is something of a detached (from hackers) marxist-academic articulation of what hacking might be, and the Debian Social Contract. However I think none of these speak much about the methodogies used to evolve a product. Other statements on method in OSS are more like anthropological reflections on culture, or odd attempts to encapsulate OSS in ill-fitting formal SDLC terms.

OSS

Wikipedia, perhaps wrongly, perhaps not, lists Open Source as a SDLC
https://en.wikipedia.org/wiki/Systemsdevelopmentlife_cycle

Open source as a culture-method is good for developing infrastructure / backend solutions.

Open-source software has tended to be slanted towards the infrastructural/back-end side of the software spectrum represented here. There are several reasons for this:

  • End-user applications are hard to write, not only because a programmer has to deal with a graphical, windowed environment which is constantly changing, nonstandard, and buggy simply because of its complexity, but also because most programmers are not good graphical interface designers, with notable exceptions.
  • Culturally, open-source software has been conducted in the networking code and operating system space for years.
  • Open-source tends to thrive where incremental change is rewarded, and historically that has meant back-end systems more than front-ends.
  • Much open-source software was written by engineers to solve a task they had to do while developing commercial software or services; so the primary audience was, early on, other engineers.

This is why we see solid open-source offerings in the operating system and network services space, but very few offerings in the desktop application space.
http://www.oreilly.com/openbook/opensources/book/brian.html

“The real challenge for open-source software is not whether it will replace Microsoft in dominating the desktop, but rather whether it can craft a business model that will help it to become the “Intel Inside” of the next generation of computer applications.”
http://www.oreilly.com/openbook/opensources/book/tim.html

Stages (Vixie)
http://www.oreilly.com/openbook/opensources/book/vixie.htmlb
1. inception (developer) – “The far more common case of an open-source project is one where the people involved are having fun and want their work to be as widely used as possible ”
2. requirements assertion (developer) – “Open Source folks tend to build the tools they need or wish they had.”
3. no design phase 1 – “There usually just is no system-level design”
4. no design phase 2 – “Another casualty of being unfunded and wanting to have fun is a detailed design…Detailed design ends up being a side effect of the implementation. ”
5. implementation (developer) – “This is the fun part. Implementation is what programmers love most; it’s what keeps them up late hacking when they could be sleeping. ”
6. integration (developer) – writing manpages and READMEs
7. field testing – forums, issues in trackers, lists, irc, chat

My thoughts,:
1. requirements assertion (itch) – 1 or more developers have an idea and assert the requirements
2. build (scratch) – building begins, might be preceeded with some simple system design but more likely to proceed towards a proof of concept
3. use – users, might be developers only, use or try the software
4. interaction – interaction between intial devs and users and/or developers. May lead to use or contributions. This is the iterative growth of ‘the community’. Contributions might be code, documentation, marketing etc…

part 4 is all about developing culture.

Process might terminate at step 2, or cycle steps 1 & 2 only, or cycle 1,2,3 only. Successful projects generally repeat items 1-4 in a parrallel and cyclical fashion.

So…as to why this model is good for adoption…i think it is the openness of the process that does this. That is, it is the inclusiveness that makes the possibility of wide scale adoption possible. Companies that have product teams do not necessarily achieve this via this means since the ‘ownership’ of the product is contained within its own walls. Abele broke this model, extending the team into customer space.

Notable issues:

  • autonomy is an extremely important cultural condition https://hbr.org/1986/01/the-new-new-product-development-game
  • often developer initiated, developer lead
  • requirements, after the initial assertion, driven by user needs
  • adoption strategy, if considered at all, occurs late in the process
  • no design phase to speak of
  • good for technical building blocks, not good for products
  • very mono-cultural to the possible point of exclusion of all roles except developers
  • non-developer users possibly rejected by the culture
  • techno-meritocracy
  • success of the project often depends on whether people want to use it, and how well the established ‘governing’ group manages stage 4

Extreme Programming (XP)

Founded by Kent Beck in the mid 80s. Interestingly, pre .com there was a general need to accelerate the design and producton of products. Speed to market was becoming an increasingly competitive factor. Extreme Programming was born from these conditions.

Seems Extreme Programming is really the cannonical start to agile values. Many practices are outlined in the book but the most common is ‘trust yourelf’. In a way you could narrate the evolution as being a move of early computer science as a practice of researchers and gov agencies, to corporate dev and markets in the 60s, then a reclaiming of programmer power in the 80s… Kent Beck’s writing is almost like a reclaiming of rights and power… like a workers movement… much of the language is centered around trusting yourself as a programmer.

The Extreme Programming tools and tricks are all about optimising quality of code and programmer satisfaction.

Diffusion of Innovation

Everett Rogers’ book. An apparent classic about how innovations are adopted.

Rogers lists the following components are critical to diffusion processes:
1. the nature of the innovation
2. communication
3. time
4. social system

nature of the innovation

Important things to consider regarding the nature of the innovation:
1. Relative Advantage of the innovation
2. Compatability of innovation with existing values and experiences
3. Complexity – the simpler the better
4. Trialability – the more an innovation can be experimented with the better
5. Observability – the degree to hich the results of the innovation are visible to others.

In addition – Rogers believes an innovation diffuses more rapidly when “it can be re-invented”…this re-invention also helps ensure that “its adoption is more likely to be sustained”.

communication

On top of this, communication is vital. Rogers states that “the heart of the diffusion process consists of the modelling and imitation by potential adopters of their network partners who have previously adopted”.

Interestingly, Rogers also states that while communication is easiest amongst those that have similar beliefs (homophilous), “one of the most distinctive problems in the diffusion of innovations is that the participants are usually heterophilous”…the nature of diffusion requires that “at least some degree of heterophily be present”….”ideally the individuals would be homophilous on all other variables […] even though they are henerophilous regarding the innovation. Usually, however, the two individuals are heterophilous on all of these variables”…hence….facilitation!

time

steps that one is likely to go through when adopting an innovation:
1. knowledge (of the innovation) eg. comms through website etc
2. persuasion (usually interpersonal comm)
3. decision
4. implementation (and re-invention)
5. confirmation

Interesting to understand how long it takes people to go through this process.

Those likely to adopt include:
1. innovators
2. early adopters
3. early majority
4. late majority
5. laggards

“the measure of innovativeness and the classification of a systems members into adopter categories are based upon the relative time at which an innovation is adopted”.

What is the expected and actual s-curve adoption rate for your innovation?

social system

diffusion occurs within a social system. the social system defines the boundaries.

Leadership, diversity and sector change in open source

Recently I have been pondering how sector change is possible. Sector change, specifically change in the publishing sector, is a key goal of mine. I don’t want to be involved in just improving sector publishing, I want to change it completely – so even the language we use to describe what was once ‘publishing’ is shifted into another domain, that of knowledge production.But how to do such a thing? How can you shift an entire sector sideways and rewrite processes, tools, value systems, expectations, career paths, language: the whole works?

Examples of sector change

Community building begins with convincing people who don’t need to work together that they should.
– John Abele

Recently I have found some clues to these questions in the writings of John Abele (founder of Boston Scientific), particularly “[Bringing Minds Together]”(https://hbr.org/2011/07/bringing-minds-together). His writings have inspired me to look again at how I see Open Source.

Early in his career, John was involved with some technologies for non-invasive surgery. Today, such matters are handled day-to-day. Back then, however, surgery was invasive by definition. Back then, talk of non-invasive instruments for surgery would be like talking about screen-less phones today. Imagine trying to sell that.

So he had a hard time trying to sell a new idea that he knew could transform the medical sector. As he writes, “We were developing new approaches that had huge potential value for customers and society but required that well-trained practitioners change their behaviour. … Despite the clear logic behind the products we invented, markets for them didn’t exist. We had to create them in the face of considerable resistance from players invested in the old way and threatened with a loss of power, prestige, and money.”

That’s more-or-less what we face with Coko (Collaborative Knowledge Foundation) at the moment. Smart people that are under the painful burden of outdated workflows and technology often resist systemic change because it requires them to alter their normal ways of working. Their resistance to change is a huge problem if you wish to modernise the way knowledge is produced and reduce that burden on them.

In John’s case, he drew on some insights he had gathered early on in his career from Jack Whitehead, CEO of Technicon, a small company that had the patent for a new medical device. When trying to bring this product to market, Technicon had the odds stacked against them. No one, from the lab technicians through to the professional societies and manufacturers, wanted anything to do with it. So Jack drummed up some interest from early adopter types and came up with a surprising next step. He “told all interested buyers that they’d have to spend a week at his factory learning about it.”

That sounds like an odd sales pitch now, but back then (early 60s) apparently it sounded a whole lot more crazy. Nevertheless, Jack convinced a handful of excited early adopters to seize that day, and brought them into his factory.

During that week, the early adopters were not treated like customers but like partners. They were part of the team. They came to know each other, they worked together, they shaped the product. They became the team. As John says:

“When the week ended, those relationships endured and a vibrant community began to emerge around the innovation. The scientist-customers fixed one another’s machines. They developed new applications. They published papers. They came up with new product ideas. They gave talks at scientific meetings. They recruited new customers. In time, they developed standards, training programs, new business models, and even a specialised language to describe their new field.”

This meeting of once potential customers, now team members, not only contributed to the design of the technology but then took it out into the world and fueled adoption and interest in the product. What had humble roots with a group of early adopters was on its way to creating large-scale change.

John witnessed this process and realised this was strategy, not whimsy: “[Jack] was launching a new field that could be created only by collaboration — and collaboration among people who had previously seen no need to work together.”

John went on to form Boston Scientific and refined this strategy further with Andreas Gruentzig when introducing the balloon catheter to a hostile and uninterested market. Again he was successful in catalysing large-scale change.

Astonishing. But you could have told the same story about any number of successful Open Source projects. Indeed, John reflects on the parallels to Open Source :

“Just as Torvalds helped spawn the Open Source movement, and Jimmy Wales spearheaded the Wiki phenomenon, Andreas [Gruentzig] created a community of change agents who carried his ideas forward far more efficiently than he could have done on his own.”

The strategy in both cases started with a simple idea – to create a community of change agents.

Leadership

Leaders need to cede control, not vigorously exert it.
– John Abele

The creation of a community where collaboration can flourish is a product of a particular kind of leadership – what John calls ‘collaborative leadership’. I have also seen this in many places including Open Source, Unconferences, and Book Sprints (which I founded). In the latter two cases, we call the collaborative leader a facilitator.

There are many modes of facilitation and choosing the right characteristics, the right mode, to create the right environment, is critical. John gives some interesting tips on what this mode looks like. The characteristics that I have gleaned from his article include the ability to:

  • foster “collaboration among people who had previously seen no need to work together”
  • manage egos
  • earn trust
  • respect stakeholder interests
  • be authentic
  • “listen, share, and ask good questions”
  • share the credit
  • use “we” not “I”
  • support the choice of ‘alternative methods’
  • create rituals
  • create non-traditional spaces and experiences to come together
  • ensure people “feel like partners”
  • be willing to take direction from the ‘customer-participants’
  • build trust over control
  • “point out the deficiencies in the tools” you are making
  • present ideas “in ways that [cry] out for input”

A good facilitator knows this language and these modes and can apply them but there are different requirements for facilitation depending on what you are trying to achieve. Book Sprint facilitation, for example, is much more product- orientated (books) than Unconference facilitation and requires a different set of strategies and tools. Most of the issues above I would categorise as being general rules of facilitation, however, those that modes that hint at enabling a group to create llarge-scalechange are the most interesting here. I think the last four points are telling:

  • be willing to take direction from the ‘customer-participants’
  • build trust over control
  • “point out the deficiencies in the tools” you are making
  • present ideas “in ways that [cry] out for input”

The most important issue at stake here is for the facilitator to create the space so others will come up with the product solution. This kind of facilitation creates vacuums, spaces where solutions don’t yet exist. This kind of facilitation is focused on a diverse range of stakeholders to design the product. This kind of facilitator doesn’t have all the answers but instead highlights problems and brings people together to create the solutions. Typical facilitation modes are necessary to enable an environment of trust and sharing, but enabling the participants to own the problems and create the productized solution requires a very specific mode of facilitation.

Can Open Source do this?

Open source software is about people.
– Sacha Greif

While these examples are about proprietary technology. It is not the proprietary-ness at stake here, it is the process that is important – and good process is the product of good facilitation. Often there is facilitation active in Open Source projects, but not necessarily, and where it does exist it is not typically with the kind of framing John has pioneered. Sure projects want people to use their software, they want to be sustainable, they don’t want to do all the work, they want community. However, Open Source projects are typically interested in bringing in more developers and not necessarily on fueling wide scale adoption (as odd as that may sound), and this informs the kind of people involved and the facilitation strategy employed.

There is a great article by Telescope founder Sacha Greif that illustrates this quite well. Sacha writes – “So why did I decide to make Telescope open source in the first place? It’d be easy to answer with typical clichĂ©s about “giving back to the community” or “believing in free software”….But the truth is a lot more pragmatic and selfish: I thought it would be a good way to get other people to help me accomplish my own goals, for free.”

Sacha goes on something of a journey with Telescope, undertaking some course corrections, and concludes “when it comes to Open Source software, software actually plays a fairly minor role. Open Source software is about people.”

I think this illustrates quite nicely where the mindset of open source projects generally are. Sacha went past the ideal of ‘build it and they will come’, a very tech-centric way of seeing things, through to understanding that the secret to success for an Open Source project lies with people, not technology. Sacha transitioned from developer to facilitator.

That is very laudable. However, facilitating developers to build your technology, is very different from facilitating a diverse set of stakeholders to develop a product, fuel adoption, changing user behaviors and putting your project on the path to creating wide scale change. The later requires involving different stakeholders and it involves a different mode of facilitation.

This is where Open Source is getting in its own way. Open source is about employing the right tools, facilitation included, to expand your developer base. That’s fine if you are building the plumbing – Open Source libraries, under-the- hood services, backend functionality etc can all be built well using this kind of method. However, if you want to be part of a team that creates wide scale change through a user-facing product, then I would suggest that open source as a culture-methodology is not the way to do it. Open Source is, at its heart, a techno-meritocratic culture. Great for building under- the-hood tools (builders building for other builders), terrible for producing products.

This shouldn’t be surprising. Somehow, however, it is. But then again when we look closely it sort of makes sense…Even the term ‘Open Source’ expresses just how much the code and those that make it form the core orientation and value metrics. Additionally, in a typical Open Source project, the person at the top isthe person with all, or most of, the technical answers (in a techno-meritocratic culture you can only not know so much before you ain’t the boss anymore). The famed benevolent dictator. And they are the culture keepers, the arbitrators, the facilitators. This sets the scene for a particular kind of facilitation that privileges the involvement of those with technical skills over anyone else. This all points to a certain kind of mono-cultural community.

To see a good example of this have a look at this post about Octocat. The entire emphasis of this article is to find ways to celebrate ‘non-code’ contributions. The term ‘non-code’ in itself is an indicator that anything other than code only has value by association (imagine a manager in any industry thanking a group of workers for their ‘non-management contributions’). Octocat was developed to measure ‘non-code contributions’ and to be effective ‘non-coders’ must log their activities in the projects issue tracker. Hence all contributors must use the same tools that the developers use. Octocat is a good project but it highlights just how focused the entire culture of Open Source is on technical development and needs everyone to fit into that paradigm. I believe this is at the cost of the product, adoption, and the potential change the product could make if managed differently.

Diversity

“Diversity creates better projects. So, why aren’t designers participating in open source?” – Una Kravets

So, what must open source change to enable a project to create massive change? Open source requires a different kind of culture. Open source must diversify away from technocratic meritocracy to become diverse communities of stakeholders. Unfortunately just the opposite is happening – ‘non-coders’ are being chased away in droves.

This is not new news. Open source has sort of known there was something wrong with its culture for sometime. The wholesale migration to closed source desktops (Apple) by the open source development world highlights this. Why did we see this enormous migration? Because OSX is ‘nicer to use’. That tells us something. Open Source has generally failed on user experience (I must note that I do not believe that to be the case now for open source desktops) and it has failed because Open Source culture is not inclusive of designers. The mono-culture has very firm boundaries for what it can do well, and it cannot provide good UX unless it diversifies.

Una Kravets has done a very nice presentation about these issues. In this presentation Una goes into detail of why designers are not involved in open source and how to enable the conditions to improve participation.

The point here is that it is not just a question about representation (and the shockingly low numbers of women involved), it is also a question of the health of a product. As Una has argued well ‘diversity creates better projects’. I would like to take that one step further and suggest that diversity not only creates better projects, it creates better conditions for wide scale adoption and change. To develop a successful product you need to have the customers designing the solution, you need UX designers to finesse the interactions, designers to make it look great, evangelists to present it at conferences, you need developers to do the plumbing, you need users for the feedback, and you need facilitators that can bring all these people together and enable them to function as a passionate, problem-solving, engaged whole. To do that, all those people need to be drawn from a diversity of contexts, valued equally, and they must all have a say in what it is that is being created. You need what John Abele calls ‘a community of change agents,’ not a community of developers.

Involving and privileging only developers is not the right kind of culture to achieve success with a product. We must do away with the techno-meritocratic culture of Open Source and replace it with a more change-centric paradigm – the community of change agents. The root of this new kind of product development culture requires a specific kind of facilitation (‘collaborative leadership’), and the passionate involvement of a diverse set of stakeholders.

The writings of John Abele have led me through a curious journey. While I was always troubled by the technocratic nature of Open Source, I figured it was a transformational movement and I would figure out a way to exist within in it and assist where I could. However, I am now seeing very clearly that, while there are individual exceptions, Open Source as a movement has its own hard coded limits. Unfortunately, these are limits that stunt effectiveness and the ability to create wide scale change, especially with the user-facing products. Unfortunately, the Open Source movement itself is barely cognizant of these limitations, why they exist, and how they could be removed.

Coda for the trolls

It probably won’t help moderate your responses but just in case… a bit of a background of where I am coming from. I love Open Source, I think it has done amazing things. I am a free license and free culture advocate. I started communities of free documentation writers to support Open Source, have initiated many software projects – all Open Source – and contributed in humble ways to others. The Open Source projects I work in now use entirely open source tools. I have supported free culture campaigns with my own time and money including (most recently) The Cost of Freedom. I have lived and breathed Open Source for about the last 20 years. I haven’t used any other desktop since 2000 or so, and have belonged to, and worked with, many hacker collectives. I don’t intend to give up on Open Source any day soon, but I do intend to do what I can to make it better, however small that contribution might be.

*original article and history here https://www.adamhyde.net/open-source-is-not-about-people-or-technology-its-about-change/ *

With thanks to Andrew Rens for great feedback, and Steve Song for some inspiring asides.

Open Source is not about technology, or people, it’s about change

NB Now updated, renamed, and moved to here
https://www.adamhyde.net/leadership-diversity-and-sector-change-in-open-source/


v 0.5.1 – Feb 3 “clean up”
v 0.5 – Feb 2 “rewrite, thanks to Andrew Rens”
v 0.4 – Jan 31 “add facilitation mode on product dev”
v 0.3 – Jan 30 “added more detail on good collaboration”
v 0.2 – Jan 29 “some lessons for open source added”
v 0.1 – Jan 28

needs more content, fact checking, structure. Be patient with me! Will improve it in the next days

[new working title]

Leadership, diversity and sector change in open source

Recently I have been pondering how sector change is possible. Sector change, specifically to change the publishing sector, is a key goal of mine. I don’t want to be involved in just improving it, I want to change it completely – so even the language we use to describe what was once “publishing” is shifted into another domain, namely knowledge production.

But how to do such a thing? How can you shift an entire sector sideways and rewrite processes, tools, value systems, expectations, career paths, language? The whole works.

Examples of sector change
Community building begins with convincing people who don’t need to work together that they should. - John Abele

Recently I have found some clues to these questions in the writings of John Abele (founder of Boston Scientific), particularly “Bringing Minds Together“. His writings have inspired me to look again at how I see open source.

Early in his career, John was involved with some technologies for non-invasive surgery. Today the stuff is day-to-day. Back then, surgery was invasive by definition. Back then, talk of non-invasive instruments for surgery would be like talking about screen-less phones today. Imagine trying to sell that.

So, he had a hard time trying to sell a new idea that he knew could transform the medical sector. As he writes, “We were developing new approaches that had huge potential value for customers and society but required that well-trained practitioners change their behaviour. … Despite the clear logic behind the products we invented, markets for them didn’t exist. We had to create them in the face of considerable resistance from players invested in the old way and threatened with a loss of power, prestige, and money. ”

This is what we face with Coko (Collaborative Knowledge Foundation) at the moment. Smart people that are under the painful burden of outdated workflows and technology are often resistant to change because it requires them to change their behaviours. It is a huge problem if you wish to modernise the way knowledge is produced and reduce that burden on them.

In John’s case, he drew on some insights he had gathered early on in his career from Jack Whitehead, CEO of Technicon, a small company that had the patent for a new medical device. When trying to bring this product to market, Technicon had the odds stacked against it. No one from the lab technicians, through to the professional societies and manufacturers, wanted anything to do with it. So Jack drummed up some interest from early adopter types and came up with a surprising next step, he “told all interested buyers that they’d have to spend a week at his factory learning about it.”

That sounds like an odd sales pitch now, but back then (early 60s), apparently it sounded a whole lot more crazy. However, Jack convinced a handful of excited early adopters to take that step, and brought them in.

In this week, the early adopters were not treated like customers but like partners. They were part of the team. They came to know each other, they worked together, they shaped the product. They became the team. As John says:

“When the week ended, those relationships endured and a vibrant community began to emerge around the innovation. The scientist-customers fixed one another’s machines. They developed new applications. They published papers. They came up with new product ideas. They gave talks at scientific meetings. They recruited new customers. In time, they developed standards, training programs, new business models, and even a specialized language to describe their new field.”

This meeting of once potential customers, now team members, not only contributed to the design of the technology but then took it out into the world and fueled adoption and interest in the product. What had humble roots as a group of early adopters, was on its way to creating large-scale change.

John witnessed this process and realized this was strategy not whimsy: “[Jack] was launching a new field that could be created only by collaboration—and collaboration among people who had previously seen no need to work together.”

John went on to form Boston Scientific and refined this strategy further with Andreas Gruentzig when introducing the balloon catheter to a hostile and uninterested market. Again he was successful in catalyzing this kind of large-scale change.

Astonishing. You could have told the same story about any number of successful open source projects. Indeed, John reflects on the parallels to open source:

“Just as Torvalds helped spawn the open source movement, and Jimmy Wales spearheaded the Wiki phenomenon, Andreas [Gruentzig] had created a community of change agents who carried his ideas forward far more efficiently than he could have done on his own.”

The strategy in both cases started with a simple idea – to create a community of change agents.

Leadership
Leaders need to cede control, not vigorously exert it. - John Abele

The creation of a community where collaboration can flourish is a product of a particular kind of leadership – what John calls ‘collaborative leadership’. I have also seen this in many places including open source, unconferences, and Book Sprints (which I founded). In the later two cases we call the collaborative leader a facilitator.

There are many modes of facilitation and choosing the right characteristics, the right mode, to create the right environment is critical. John gives some interesting tips on what this mode look like. The characteristics that I glean from his article includes the ability to:

  • foster “collaboration among people who had previously seen no need to work together”
  • manage egos
  • earn trust
  • respect stakeholder interests
  • be authentic
  • “listen, share, and ask good questions”
  • share the credit
  • use “we” not “I”
  • support the choice of ‘alternative methods’
  • create rituals
  • create non-traditional spaces and experiences to come together
  • ensure people “feel like partners”
    • be willing to take direction from the ‘customer – participants’
  • build trust over control
  • “point out the deficiencies in the tools” you are making
  • present ideas “in ways that [cry] out for input”

A good facilitator knows this language and these modes and can apply them but there are different requirements for facilitation depending on what you are trying to achieve. Book Sprint facilitation, for example, is much more product-orientated (books) than Unconference facilitation and requires a different set of strategies and tools. Most of the issues above I would categorize as being general rules of facilitation, however those modes that hint at enabling a group to create large-scale change are the most interesting here. I think the last four points are telling:

  • be willing to take direction from the ‘customer – participants’
  • build trust over control
  • “point out the deficiencies in the tools” you are making
  • present ideas “in ways that [cry] out for input”

The most important issue at stake here for the facilitator is to create the space so others will come up with the product solution. This kind of facilitation creates vacuums, spaces where solutions don’t yet exist. This kind of facilitation is focused on a diverse range of stakeholders to design the product. This kind of facilitator doesn’t have all the answers but instead highlights problems and brings people together to create the solutions. To do this, typical facilitation modes are necessary to enable an environment of trust and sharing while enabling the participants to own the problems and create the productized solution. This comprises a very specific mode of facilitation.

Can Open Source do this?
Open source software is about people.
 - Sacha Greif

While these examples are about proprietary technology, it is not the proprietariness at stake here, it is the process that is important – and good process is the product of good facilitation. While facilitation is active in many open source projects, it is not necessarily so, and where it does exist it is not typically with the kind of framing John has pioneered. Sure projects want people to use their software, they want to be sustainable, they don’t want to do all the work, they want community. However, open source projects are typically interested in bringing in more developers yet not necessarily on fueling wide-scale adoption (as odd as that may sound) and this informs the kind of people involved and the facilitation strategy employed.

There is a great article by Telescope founder Sacha Greif that illustrates this quite well. Sacha writes – “So why did I decide to make Telescope open source in the first place? It’d be easy to answer with typical clichĂ©s about “giving back to the community” or “believing in free software”….But the truth is a lot more pragmatic and selfish: I thought it would be a good way to get other people to help me accomplish my own goals, for free.”

Sacha goes on something of a journey with Telescope, undertaking some course corrections, and concludes “when it comes to open source software, software actually plays a fairly minor role. Open source software is about people.”

I think this illustrates quite nicely where the mindset of open source projects generally are. Sacha went past the ideal of ‘build it and they will come’, a very tech-centric way of seeing things, through to understanding that the secret to success for an open source project lies with people, not technology. Sacha transitioned from developer to facilitator.

That is very laudable. However, facilitating developers to build your technology is very different from facilitating a diverse set of stakeholders to develop a product, fuel adoption, change user behaviours and put your project on the path to creating wide- scale change. The later requires involving different stakeholders and it involves a different mode of facilitation.

This is where open source is getting in its own way. Open source is about employing the right tools, facilitation included, to expand your developer base. That’s fine if you are building the plumbing – open source libraries, under the hood services, back end functionality, and so on can all be built well using this kind of method. However, if you want to be part of a team that creates wide scale change through a user-facing product, then I would suggest that open source as a culture-methodology is not the way to do it. Open source is, at its heart, a techno-meritocratic culture. Great for building under the hood tools (builders building for other builders), terrible for producing products.

This shouldn’t be surprising. Somehow however it is. But then again when we look closely it sort of makes sense…Even the term “Open Source” expresses just how much the code and those that make it form the core orientation and value metrics. Additionally, in a typical open source project, the person at the top is the person with all, or most of, the technical answers (in a techno-meritocratic culture you can only not know so much before you ain’t the boss anymore- and become the famed benevolent dictator. And they are the culture keepers, the arbitrators, the facilitators. This sets the scene for a particular kind of facilitation that privileges the involvement of those with technical skills over anyone else. This all points to a certain kind of mono-cultural community.

To see a good example of this, have a look at this post about Octocat. The entire emphasis of this article is to find ways to celebrate ‘non-code’ contributions. The term ‘non-code’ in itself is an indicator that anything other than code only has value by association (imagine a manager in any industry thanking a group of workers for their ‘non-management contributions’). Octocat was developed to measure ‘non-code contributions,’ and to be effective ‘non-coders’ must log their activities in the projects issue tracker. Hence all contributors must use the same tools that the developers use. Octocat is a good project but it highlights just how focused the entire culture of open source is on technical development and wants everyone to fit into that paradigm. I believe this is at the cost of the product, adoption, and the potential change the product could make if managed differently.

Diversity
"Diversity creates better projects. So, why aren't designers participating in open source?" - Una Kravets

So, what must open source change to enable a project to create massive change? Open source requires a different kind of culture. Open source must diversify away from technocratic meritocracy to become diverse communities of stakeholders. Unfortunately just the opposite is happening – ‘non-coders’ are being chased away in droves.

This is not new news. Open source has sort of known there was something wrong with its culture for sometime. The wholesale migration to closed source desktops (Apple) by the open source development world highlights this. Why did we see this enormous migration? Because OSX is ‘nicer to use’. That tells us something. Open source has generally failed on user experience (I must note that I do not believe that to be the case now for open source desktops) and it has failed because open source culture is not inclusive of designers. The mono-culture has very firm boundaries for what it can do well, and it cannot provide good UX unless it diversifies.

Una Kravets has done a very nice presentation on these issues. In this presentation, Una goes into detail why designers are not involved in open source and how to enable the conditions to improve participation.

The point here is that it is not just a question about representation (and the shockingly low numbers of women involved), which is an effort we should all support in its own right, it is also a question of the health of a product. As Una has argued well, ‘diversity creates better projects’. I would like to take that one step further and suggest that diversity not only creates better projects, it creates better conditions for wide-scale adoption and change. To develop a successful product, you need to have the customers designing the solution, you need UX designers to finesse the interactions, designers to make it look great, evangelists to present it at conferences; you need developers to do the plumbing, you need users for the feedback, and you need facilitators that can bring all these people together and enable them to function as a passionate, problem-solving, engaged whole. To do that , all these people need to be drawn from a diversity of contexts, valued equally, and they must all have a say in what it is that you are creating. You need what John Abele calls ‘a community of change agents,’ not a community of developers.

Involving and privileging only developers is not the right kind of culture to achieve success with a product. We must do away with the techno-meritocratic culture of open source and replace it with a more change-centric paradigm – the community of change agents. The root of this new kind of product development culture requires a specific kind of facilitation (‘collaborative leadership’), and the passionate involvement of a diverse set of stakeholders.

Summary

I love open source, I think it has done amazing things. I am a huge free license and free culture advocate. I started communities of documentation writers to support open source, have initiated many software projects – all open source – and contributed in humble ways to others. However, the writings of John Abele have lead me on a curious journey. While I was always troubled by the technocratic nature of open source, I figured it was a transformational movement and I would figure out a way to exist within in and assist where I can. However, I am not seeing very clearly, that, while there are individual exceptions, open source as a movement has its own hard-coded limits. Unfortunately, these are limits that stunt its effectiveness and its ability to create wide-scale change, especially with user-facing products. Unfortunately, the open source movement itself is barely cognizant of these limitations, why they exist, and how they could be removed.

tbc

older version:

Recently I have been pondering how sector change is possible. Sector change, specifically to change the publishing sector, is a key goal of mine. I don’t want to be involved in just improving it, I want to change it completely – so even the language we use to describe what was once ‘publishing’ is shifted into another domain, namely knowledge production.

But how to do such a thing? How can you shift an entire sector sideways and rewrite processes, tools, value systems, expectations, career paths, language. The whole works.

It seems to me that the critical methodologies for wide-scale change like this are ‘almost native’ to open source. I’ll explain the ‘almost’ in a bit. But as an opener, open source has the ability to produce massive change. I don’t mean that open source can change the way that software is produced, we all know that. I mean the strategy for running a successful open source project is the same strategy you should adopt if you are trying to enable change, or massive change – of a sector, for example. I’m kind of surprised to arrive at this conclusion. I always loved open source but I am starting to realise that I have loved it for the wrong reasons.

Over the last 20 years or so of being involved in open source I have felt myself in a kind of continual falling forward into open source values. Who can’t love open source when you get down to it? The idea that software should be available to anyone that wants it, to do with it as they please. Its a great idea. It is about empowerment and that is something powerfully valuable it itself.

Empowerment through open source occurs because people have access to free tools and for this to happen we need generous tool makers. That highlights more good values active in open source, namely sharing. The fact that the technically gifted have brought these tools into our lives and generously shared them for free is something we should celebrate.

Of course, the sharing has its own reward system and isn’t all altruistic. Paid programmers building everything from Facebook to the technology that runs your dentist’s office systems (a reference to this ridiculous article in case you missed it) could not do their work without all the open source libraries they can handily import for free into their projects. Also, to ensure they don’t have to do the same work again in other projects, possibly for different clients or employers, they also share what they build (where possible). By reusing your own work and that of others like this, the total workload is then also decreased and people can get further faster. Progress can be made. Programmers can also enhance their reputations if their work is re-used by other programmers. This becomes value in the market place. Hence this sharing has turned into an economy. It’s a good one, who couldn’t love an economy that encourages sharing? This economy also makes many open source software projects sustainable so the whole system has a beautiful internal logic.

The openness of all this is also something to be valued. Exposing the code to all eyes is a famous strategy for reducing bugs. It’s also touted as a strategy for increasing security. It also means programmers can learn from code developed by others (both the good and the bad). Openness and transparency also have a lot of utility in this world, and open source has proven this utility time and time again. Openness is not only good for the code, but it makes us feel good (if we are not too tainted by the proprietary way of life).

Then there is community. You can’t go far into a conversation about open source without mentioning community. Community is the metric for the well-being of any open source project and seeing a healthy open source community is a sight to behold. It is astonishing. All these people somehow, impossibly, co-ordinating to pull in (more or less) the same direction. Good governance plays a role in this after a certain threshold of participants has been exceeded and good governance in itself is quite miraculous to consider. How do the guardians make the wise decisions necessary to balance participation and technical development, uptake, funding, systems, protocols, agreements…the whole works. To consider the day-to-day decisions, project leads must need to make and the context in which they operate is quite dizzying.

I love all of this.

These are many of the reasons I have loved open source. I mean, who couldn’t love sharing, transparency and community? Well, there are actually many, many other values! But I want to think of the open source developers as very cold-hearted. I still believe in good guys and bad guys – the world is more fun that way, more energising. However, as much as I love open source, I think I have been in love with it for the wrong reasons.

How did I get to this realisation?

Recently I have been reading the writings of John Abele (founder of Boston Scientific). This has lead me down another path and provided me with fresh lens through which to value open source. Strange for me as I am allergic to proprietary-ness and John Abele is a good proprietary citizen. Ranked high on the Forbes rich list – net worth 600 million I believe – John has made a lot of money from Intellectual Property, specifically via the sale of medical instruments.

So, why did this entrepreneur, with whom I would initially assume I would share few values, inspire me to look again at how I valued open source? Well…it all comes down to the fact that John did achieve wide scale, sector-wide, change and he did it in a way that is radical to proprietary culture, but I would consider more native, almost native perhaps, to open source.

Early in his career, John came up with some technology for non-invasive surgery. Today the stuff is day to day. Back then surgery was invasive by definition. Talk of non-invasive instruments for surgery back then would be like talking about screen-less phones today. Imagine trying to sell that.

So, he had a hard time trying to sell a new idea that he knew could transform the medical sector. He did transform a sector, so how did he do it?

John has written quite a bit on how he was able to do this, in particular he writes a lot about collaboration and even brought a conference center specifically to change it into a center for collaboration. But it is his ideas on change that most interest me. His article in this HBR article “Bringing Minds Together” gives some good insights. In this article John speaks of how openness can accelerate adoption where ‘walled’ processes have failed:

https://hbr.org/2011/07/bringing-minds-together

I really encourage you to read that article, I summarise it in part below but the full article is well worth reading.

As he writes:
“We were developing new approaches that had huge potential value for customers and society but required that well-trained practitioners change their behavior. … Despite the clear logic behind the products we invented, markets for them didn’t exist. We had to create them in the face of considerable resistance from players invested in the old way and threatened with a loss of power, prestige, and money. ”

So to overcome this John drew on some insights he had gathered early on in his career from Jack Whitehead, CEO of Technicon, a small company that had the patent for a new medical device. When trying to bring this product to market Technicon had the odds stacked against them. No one from the lab technicians, through to the professional societies and manufacturers wanted anything to do with it. So Jack drummed up some interest from early adopter types and came up with a surprising next step, he “told all interested buyers that they’d have to spend a week at his factory learning about it—and that payment was required in advance.”

That sounds like an odd sales pitch now, but back then (early 60s) apparently it sounded a whole lot more crazy. However, Jack found this handful of excited early adopters and brought them in.

In this week the early adopters were not treated like customers but like partners. They were part of the team. They came to know each other, they worked together, they shaped the product. They became the team. As John says:

“When the week ended, those relationships endured and a vibrant community began to emerge around the innovation. The scientist-customers fixed one another’s machines. They developed new applications. They published papers. They came up with new product ideas. They gave talks at scientific meetings. They recruited new customers. In time, they developed standards, training programs, new business models, and even a specialized language to describe their new field.”

They were on the way to creating large-scale change.

John witnessed this and realized that this was no one-off but “something much bigger was actually going on. [Jack] was launching a new field that could be created only by collaboration—and collaboration among people who had previously seen no need to work together. ”

Astonishing. You could have told the same story about any number of successful open source projects. Indeed after applying this strategy with Andreas Gruentzig some years later with Boston Scientific, introducing the balloon catheter to a hostile and uninterested market, John reflects on the parallels to open source:

“Just as Torvalds helped spawn the open source movement, and Jimmy Wales spearheaded the Wiki phenomenon, Andreas [Gruentzig] had created a community of change agents who carried his ideas forward far more efficiently than he could have done on his own.”

The strategy in both cases was simple. To create a community of change agents.

The change agent meeting became something of a user group, which in turn became a regular symposium, which then grew in size and prestige that it rivaled the established legacy events.

I feel compelled to note that the kind of product development we are talking about here is qualitatively different from developing a product for an organisation. Product development for a sector and creating large scale change is what this post is investigating.

There are many factors that play into the success of this approach. John Abele doesn’t go into detail what these are but he gives some hints, notably the creation of a culture of what he calls ‘leader – followers’ and a specific kind of ‘collaborative leadership’.

I think by ‘leader – followers’ he is referring a culture where everyone has the opportunity to be heard and to participate in the development of the solutions at stake. This kind of environment is what I would describe as strong collaboration and the interesting thing about this kind of approach is that it creates a strong feeling of shared ownership of the output. Who it is that owns the product, who are the best kinds of candidates to make up this group, so that they can go forward and take the product into the world, is a very interesting question and there might be some lessons to be learned here for open source.

John Abele doesn’t provide much detail in his article as to the desired make-up of this group. Beyond stating that they are potential customers there is very little detail on who these people are. It is interesting that they are customers however as this reflects a lot of the ‘itch to scratch’ mantra of the open source world – get people that have a problem to create the solution. However there might be something of an unintended but interesting critique here for open source. The open source world, it might be argued, leans too heavily on programmer involvement and does not necessarily directly involve the people that have the problem – the users – to design the solution.

The creation of a collaborative environment, in turn, is a product of collaborative leadership, which I have also seen in many places including open source and in Book Sprints (which I founded). In the later case we call the collaborative leader a facilitator.

Facilitation is a good art to place at the center of this strategy to create change and I far prefer this framing (‘collaborative leadership’ is a fine term but I think not many people know what this means, whereas I think ‘facilitator’ has much more currency) than those that exist in the open source world ie. ‘cat herder’ or ‘benevolent dictator’. These roles in open source are applied to co-ordinating programmers, and this highlights the techno-meritocratic approach that open source projects tend to subscribe to. Once again, the open source world could be criticized as too closely marrying this leadership to technical expertise. In the cases John outlines, the leaders don’t seem to necessarily have deep technical skills (although they appear to have a hand in the technical development), but they do have strong facilitation skills.

There are many modes of facilitation. Choosing the right characteristics, the right mode, to create the right environment is critical. John gives some interesting tips on what this might look like. The characteristics that I glean from his article includes the ability to:

  • foster “collaboration among people who had previously seen no need to work together”
  • manage egos
  • earn trust
  • respect stakeholder interests
  • be authentic
  • “listen, share, and ask good questions”
  • share the credit
  • use “we” not “I”
  • be willing to take direction from the ‘customer – participants’
  • support the choice of ‘alternative methods’
  • build trust over control
  • create rituals
  • create non-traditional spaces and experiences to come together
  • make people “feel like partners”
  • “point out the deficiencies in the tools” you are making
  • present ideas “in ways that [cry] out for input”

A good facilitator knows this language and these modes and can apply them but it is good to recognise that there are different requirements for facilitation depending on what you are trying to achieve. Book Sprint facilitation, for example, is much more knowledge product (books) orientated than Unconference facilitation and requires a different set of strategies and tools. Most of the issues above I would categorise as being general rules of facilitation, however those that are specific to the kind of product-orientated process designed to enable large scale adoption and change are the most interesting. I think the last two points are pointing in this direction:

  • “point out the deficiencies in the tools” you are making
  • present ideas “in ways that [cry] out for input”

The most important issue at stake here for the facilitator of this kind of project is to create the space so others will come up with the product solution. Don’t have all the answers, instead identify problems and bring people together to create the solutions. To get to that point the typical facilitation modes are necessary to enable an environment of trust and sharing but enabling the participants to own the problems of a product and create the solutions is slightly different from other facilitation modes.

This is the key ingredient, I believe, to the kind of processes that John and colleagues have pioneered and others have made the same observation. The quote below is taken from here:
http://www.kornferry.com/institute/709-growth-through-collaboration-john-abele-s-vision

“…Abele’s success at Boston Scientific was built upon his ability to bring together extremely competitive intellects and create an environment where participants could meet and be candid about what they were working on, including the new techniques, the risks, the benefits and the strategies they employed…’It was about really sharing what they were learning rather than lecturing about what they were teaching’ Abele said. ‘That was powerful because it really accelerated the development of these new technologies in health care — such as revolutionary steerable catheters — and I believed there must be more opportunities to apply this collaboration in a lot of different areas.'”

While these examples are about proprietary technology. It is not the proprietariness at stake here, it is the process that is important – the methods you use to enable early engagement with a wide diversity of stakeholders, bring them into the team, and enable them to be part of the product development. As the passage above states this is a default almost native methodology for open source but also found in the proprietary world. I say ‘almost native’ because I don’t think that this particular framing is the modus operandi for most open source projects, especially those starting out. Sure projects want people to use their software, they want to be sustainable, they don’t want to do all the work, they want community. Most open source projects start off thinking this way but few develop their thinking beyond this starting point, as this beautiful article by Telescope founder Sacha Greif states well:
https://medium.com/@sachagreif/open-source-lessons-learned-two-years-of-telescope-be4ed955b39

Sacha writes – “So why did I decide to make Telescope open source in the first place? It’d be easy to answer with typical clichĂ©s about “giving back to the community” or “believing in free software”….But the truth is a lot more pragmatic and selfish: I thought it would be a good way to get other people to help me accomplish my own goals, for free.”

Sacha goes on something of a journey with Telescope, undertaking some course corrections, and concludes “when it comes to open source software, software actually plays a fairly minor role. Open source software is about people.”

I think this illustrates quite nicely where the mindset of open source projects generally are. Sacha went past the ideal of ‘build it and they will come’, a very tech-centric way of seeing things, through to understanding that the real secret to success for an open source project lies with people, not technology.

That is very laudable. However, thinking about how to get people involved to build your technology, is very different from thinking about what you do as a methodology for creating change. I would say one particular issue at stake here is who you involve. If you want to create a technical product, you try and engage developers. If you are trying to create change, you must engage a much wider group of stakeholders.

Open source is a strategy, and a proven one, for product development. All the standard lessons have been learned for this, if not universally shared. Openness, sharing, community can lead to a healthy open source project. However these methods are not only about how the product is developed and who is developing it, but they are the basic tools necessary to create change. When wielded right, these strategies can, and have, lead to large-scale change as John Abele has already proven more than once in his life time and as we have seen with some open source projects.

Although I’m very much in love with the many values of open source I am more interested in their ability to create change which I think is an under considered effect. Its not that creating change is the only thing that should be considered but rather it is a lens that must be considered. It is as important to look at your open source project as a strategy for creating change (which, whether you own it or not, you are trying to do) as much as it is important to see it as being about technology and people.

This is a new realization for me. It opens a lot of doors. It perhaps explains, for example, why many open source projects do not succeed. The famous ‘build it and throw it over the wall’ strategy, for example, that so often leads to a lack of adoption and eventual failure. It perhaps presents an argument that the success of a project aimed at large-scale adoption and change is reliant on the involvement of a diverse range of stakeholders – changemakers not just programmers. It explains why open source is good at developing libraries and tools for other programmers but not necessarily very good at producing platforms for non-programmers. It also explains why other projects succeed even if they are not conscious of this framing.

This is, as if it had to be said, extremely important for publishing right now, or ‘knowledge production’ as I would prefer to talk about it, since sector change is what we need as workflows and outputs are still stuck in (what one may call) ‘post-paper’ processes and thinking. Open source can change this sector if viewed as a strategy for creating change. Massive change. However, if you want change to be native to your open source project, you have to own it. And that’s what I intend to do from now on.

Poking around in the corners of the web

I just was doing a little researchy stuff. Whimsical browsing. And I came across the Digital Publishing Toolkits epub2trailer script  https://github.com/DigitalPublishingToolkit/epubtrailer.py

These folks were a research unit which I believe has since finished. They were looking at how digital tools might transform publishing. One of the outcomes was the scripts in their repo https://github.com/DigitalPublishingToolkit/

Of these, I tried the epubtrailer python script. Its intention is to autogenerate animated gifs from epub files. I’d say it’s a bit of whimsy but a nice idea. So I ran it and here are some outcomes generated from some Project Gutenberg and Archive.org epubs:

From https://www.gutenberg.org/ebooks/13476:

tesla_real

From https://www.gutenberg.org/ebooks/20165:

wonderfulwizardo00baumiala_0005

From https://archive.org/details/wonderfulwizardo00baumiala :

uncletomscabinor00stowiala_0007

From https://archive.org/details/uncletomscabinor00stowiala:

If you do want to try it then you will need to pip install images2gif and then pip install pillow. Also apply this fix to images2gif.py http://stackoverflow.com/questions/19149643/error-in-images2gif-py-with-globalpalette … although it might not be on line 200 as suggested.