Some stuff I found. More information coming.
It is a rampant meme – ‘non-code’. Used to describe contributions to open source projects that are not code.
This kind of language creates insider-outsider boundaries and will signal to anyone that isn’t a programmer that they are secondary citizens and that their contributions will be valued accordingly. Not very motivating.
To investigate tools that could enable collaborative design of a product, I have been searching around for ways to display designs, discuss, and iterate amongst a distributed team.
There are some closed source products and services out there, but my I feel kind of dirty looking at them. like I always feel the need to go home and have a shower afterwards.
There don’t seem to be any open source tools out there to do it. I could be wrong, so if I am missing something please email me (firstname.lastname@example.org).
In part, I think the lack of open source tools for this kind of thing is in itself telling since open source projects itch their own scratch and many don’t see product design as very important.
It would be great if there was something out there that could live alongside GitLab (or the closed source GitHub) and there was a equal two-way integration of tools and processes, coupling technical development with product design.
So, I am making a first attempt with Julien Taquet (a designer) and Juan Gutierrez (a developer) to make a simple tool. We are calling it Increment and it is just a prototype in HTML at the moment. It lives here https://gitlab.coko.foundation/adam/increment
The idea is to develop a simple interface to present mocks (or links to prototypes, text descriptions, videos, etc) and then enable comments around the artefact. Then the discussion can be frozen, a new version posted and the comments can start again but refering to the new artefact.
Here is a screen shot of the ‘home’ page:
When clicking on an item you might see something lke this:
The idea being that if you click on a version on the left, you view that version and the associated discussion.
Of course, this blog post could really use Increment right now to show you this very concept and get your thoughts! 😉
For now, email me thoughts. We will hopefully have a basic prototype in a week (if you download the repo version and run under localhost then you can see something very basic in action). Later, if there is real utility, we will make a proper project out of it. For now this is a proof of concept.
Extra thoughts from Julien below:
v 0.8 Mar 20 started reading Diffusion of Innovations (5th ed)
v 0.7 Mar 16 finished reading Xtreme Programming Explained (2nd ed)
v 0.6 Feb 15 “added extreme programming notes”
v 0.5 Feb 13 “notes about ba”
v 0.4.1 Feb 12 “add more Agile notes”
v 0.4 – Feb 11 “added Agile notes”
v 0.3 – Feb 9 “added Waterfall, SDLC, OSS notes”
v 0.2 – Feb 8 “added Radical Application Design (RAD) notes”
v 0.1 – Feb 6So, I am pondering more how to work with product development methods to improve the process for products that are of the ‘open source’ genre. My first step in this process was to write this post:
https://www.adamhyde.net/leadership-diversity-and-sector-change-in-open-source/This has lead to the realisation that open source is really a culture-methodology. While I believe in open source licenses, I think the default culture-method leaves a lot to be desired. So, I am investigating what are the key foundations, processes, and tools that would enable a different kind of product development. At this early stage in my pondering, I think there are several key ingredients:
- a diverse set of stakeholders
- a distributed team
- open licenses
In addition, I have the following goals:
- make excellent products
- enable widespread adoption
The goals are as important as the ingredients. After all, if you have don’t have lemons, you shouldn’t be trying to make lemonade.
I am thinking this exploration could have a working title ‘Collaborative Product Development’.
My first step is to look at historical methods for software product development and see what their approaches have been. The intent being to learn more about how software development processes are targeted at specific objectives and operational contexts. I think this will help develop an understanding of why certain methods are effective for this specific case, which in turn will help me understand which bits can be discarded.
I’ll keep this particular page in my Ghost blog as the notepad for my evolving thoughts.
Initial Thoughts, Learnings, Extractions
- There are a number of methods for building software products. Agile, lean, feature-driven method, XP, JAD etc. I would also include Open Source as a default methodology for product development and, even though open source projects can employ these other methods there is a kind of default modus operandi for open source which defines how things are done. Hence I think it can be considered to be a methodology.
- I have, this week, been involved in collaborative product design processes. I facilitated a small group (the second session in a series) in the beginnings of the process. The group consisted of those that started the project (owners?), a project manager, and people who knew the workflow intimately and would end up using the software. We started with the whiteboards, and pens. The opening question in the first session, which seemed to open the door to a great conversation, was “how would you design a system if the workflow would be transferred to the browser”. The second session went through the same process to get validation and to expand the group slightly (this was important). We now have a baseline understanding of what we are building. Each session was about 1.5 hrs.
- In collaborative design sessions, I think it is important to not rush to implementations too fast. Stay on the whiteboard for as long as necessary – it feels more organic and the output obviously is created by all in the room (as opposed to presenting well thought out ‘finished’ mocks).
- move slowly to more sophisticated artefacts (mocks) and get signoff as a group at every step. Give plenty of room for revising issues. Identify big problems but do not rush to solve them. It is ok to push some stuff down the road as long as it is agreed by all and the issue is clearly noted. Document all outputs of collaborative design sessions.
- it is ok to keep the documentation a little rough. it communicates that it is a WIP and things can be revised. The quality of the artefacts is secondary to the quality of the discussions about them.
research notes (scratch pad):
- https://en.wikipedia.org/wiki/Jointapplication design
“Arnie Lind’s idea was simple: rather than have application developers learn about people’s jobs, why not teach the people doing the work how to write an application? Arnie pitched the concept to IBM Canada’s Vice President Carl Corcoran (later President of IBM Canada), and Carl approved a pilot project. Arnie and Carl together named the methodology JAD, an acronym for Joint Application Design, after Carl Corcoran rejected the acronym JAL, or Joint Application Logistics, upon realizing that Arnie Lind’s initials were JAL (John Arnold Lind).
The pilot project was an emergency room project for the Saskatchewan Government. Arnie developed the JAD methodology, and put together a one-week seminar, involving primarily nurses and administrators from the emergency room, but also including some application development personnel. The project was a huge success, as the one-week seminar produced a detailed application framework, which was then coded and implemented in less than one month, versus an average of 18 months for traditional application development. And because the users themselves designed the system, they immediately adopted and liked the application. “
- learning through collaboration http://www.commonwealthclub.org/events/archive/podcast/joseph-henrich-secret-our-success
“high fidelity copying from each other” is a secret to our success as a species and because “we are interconnected we can learn stuff drom each other and combine it”. (yesyes!). The ability to innovate “depends on the size and connectivity of the population”. The ability to adapt like this “drove genetic evolution”. Brings up interesting questions on who to learn from. “the size of the population and the degree to which people are interacting and sharing information actually affects how fancy the technology is going to get and how many tools you can produce….also if the the group should get cut off for some reason and the population is reduced, the group will suffer a loss of technology”. “Genetic evolution gives rise to cultural evolution” which in turn drives genetic evolution. “think of our learning abilities as adaptation” -> “cultural driven genetic evolution”. “A lot of creativity is the recombination of different ideas”. “If you want to be creative out yourself at the nexus of very different and divergent information”. “cultural learners” (people who learn from each other). “good cultural learners”.
- From the book by Joseph Henrich
“natural selection favoured genes for building brains with abilities to learn from others.” p53
- crystal method considered to be a subset of Agile
Feels very west coast. A bit too hippy dippy in its method names.
” The development of OSS gains popularity due to the wide availability of the internet facility to each and every region of the world, parallel development, peer review, parallel debugging, expert developers, and feedback” “Many different developers, user, or co-developers can participate in the development of the OSS.” “The development of OSS is always initiated by single developer or a single group, who starts the development of software for its own “personal itch” “
- http://www.ijcaonline.org/volume21/number1/pxc3873327.pdf More OSS SDLC (OSDLC)..this one a little bit too praisey of open source
- “Extreme Programming (XP) is about social change. It is about letting go of habits and patterns that were adaptive in the past, but now get in the way of us doing our best work. It is about giving up the defenses that protect us but interfere with our productivity. It may leave us feeling exposed. ” Kent Beck, Extreme Programming Explained
- amazing talk by Kent Beck https://www.youtube.com/watch?v=aApmOZwdPqA
- Agile is Dead https://www.youtube.com/watch?v=a-BOSpxYJ9M
- TDD is Dead https://www.youtube.com/watch?v=z9quxZsLcfo
- discovering ba
“Nonaka told his interviewer that they were creating ba, a Japanese term that describes a field or space where people freely and openly share what they know in the service of creating something new. Ba resembles the concept of “flow” as set forth by psychologist Mihaly Csikszentmihalyi: It is the mental state that occurs when a person is fully immersed in whatever he or she is doing. But unlike flow, ba is never solitary; it exists among two or more people. As Nonaka says, “In ba, there is no you or me, there is only us, sharing a here-and-now relationship.” Ba can occur in a work group, a project team, an ad hoc meeting, a virtual e-mail list, or at the frontline point of contact with customers. It serves as a petri dish in which shared insights are cultivated and grown.
Companies can foster ba by designing processes that encourage people to think together.”
- “A phronetic leader mobilizes timely judgment in others by building a culture that is strong, nurturing, and sustained by informal connections.”
- “Cross-fertilization. A project team consisting of members with varying functional specializations, thought processes, and behavior patterns carries out new product development. The Honda team, for example, consisted of hand-picked members from R&D, production, and sales. The company went a step further by placing a wide variety of personalities on the team. Such diversity fostered new ideas and concepts.”
- “All of us, as far as I can remember, thought waterfalling of a huge project was rather stupid, or at least ignorant of the realities… I think what the waterfall description did for us was make us realize that we were doing something else, something unnamed except for “software development.””
–GM Weinberg http://www.craiglarman.com/wiki/downloads/misc/history-of-iterative-larman-and-basili-ieee-computer.pdf
- “The conclusion that stands out most clearly from our field study observations is that the process of developing large software systems must be treated, at least in part, as a learning and communication process.” Bill Curtis quoted in http://www.craiglarman.com/wiki/downloads/misc/history-of-iterative-larman-and-basili-ieee-computer.pdf
I think I need to cover at least the following questions for each:
- what problem was the methodology a response to
- what are its key goals
- what application is it intended for
In addition, I need enough information to succinctly describe the methodology and to provide some praise and critique.
“I think what the waterfall description did for us was make us realize that we were doing something else, something unnamed except for “software development.” – Gerald M Weinberg
The history of Software Development Lifecycles is a history of experiment, description, and rejection.
Systems Development Life Cycle is what I refer to here as ‘method’ or ‘methodology’. It describes any methodology used as a framework for developing software. While all the methods described below can be used in a pure form, often SDLCs are combined and hybridised. Sometime for better, sometimes for worse. However, it is interesting to note that the basic framework address the following needs one way or another:
- requirements gathering
Some SDLCs are cyclical, some incremental, some strict, others loose, linear etc
The interesting question to me is what can we learn from these if we want to develop a methodology (SDLC) for developing products with free licences and the principles of open source.
Also interesting is that there have been attempts to define OSS as a SDLC. See http://www.sersc.org/journals/IJSEIA/vol8no32014/38.pdf
Somethings that I notice:
- SDLCs are methods that feel like they are to be deployed
- you become a subject of a SDLC
- OSS feels like a culture you want to inhabit
- you become a citizen of OSS
So, I think part of the framing is that OSS ‘SDLCs’ are about creating the right culture and employing tools to shape that. SDLCs (in the traditional) form are about employing a structured method and making sure everyone adheres to it. So, I would prefer to think of the kind of methodology i am looking for to be framed as a ‘Product Development Culture’, not a Software Development Life Cycle. The former is a culture, the later a structured process. The former can be shaped, the later employed. Best to pick the right conditions for success from the beginning 😉 To improve OSS PDC we need the right tools to shape culture – this is the culture-method I am after.
The interesting thing here is that the history of SDLC highlights the problem with history. It is easy to tell a story of a timeline and certain events happening in sequence, almost like building blocks in time. However that isn’t how things happen. We can, for example, set up a nice story about how ‘Waterfall’ came from a pre-software engineering era, and it was simply applied to software development, and then in the 1980s, people started rebelling against this legacy process and developed more iterative methods. However, while there is some level of truth in that story, it also hides the complex paralled, unfloding, emergent processes and histories that is the ‘full telling of this story’. For example, iterative development in software is recorded as happening as early as 1957 (and certainly in the 60s for Project Mercury), but the process evolved out of non-software projects as early as the 1930s. As Larman and Victor Basili noted, “We are mindful that the idea of IID [Interactive and Incremental Development] came independently from countless unnamed projects and the contributions of thousands and that this list is merely representative. We do not meanthis article to diminish the unsung importance of other IID contributors.”
While Waterfall is often said to have been born in 1970 (– Royce, Winston (1970), “Managing the Development of Large Software Systems“,) So before we even had waterfall for programming, we had iterative development as this article so nicely highlights: http://www.craiglarman.com/wiki/downloads/misc/history-of-iterative-larman-and-basili-ieee-computer.pdf
What this highlights is that these ideas and methods have evolved in a many faceted, evolutionary manner. There is no simple star wars narrative here of good vs evil although that is often how the history of SDLC is told.
“In my experience, the simpler method has never worked on large software development efforts…” – Royce Winston
Waterfall in software dev was actually developed in the 1970s, which is late for computing. It was first described by Winston Royce in 1970 (“Managing the Development of Large Software Systems“,)
The steps included:
1. Design comes first
2. Document the design
3. Do it twice
4. Plan, control and monitor testing
5. involve the customer
Interestingly, it has nothing to do with how open source generally works 🙂 except! involve the customer is often also at the end of OSS product development.
Even more interesting…Waterfall as we know it now is really what Royve was criticising when he introduced the term. ie a very strict, sequential process. Conception, initiation, analysis, design, construction, testing, implementation, and maintenance. It is one way process, no going back to an earlier step at any stage. This is known as ‘single pass’ or ‘single step’ Waterall and differentiates it from Royce’s method that advocated building twice (2 pass).
A foundation of Waterfall, as we know it now, is the argument that the more work done gathering requirements and designing, the more time you save and risks you avoid when it comes to building.
Rational Unified Process (RUP)
Adaptive Software Development
Mid 80s, origins outside of software, namely the 1986 article “The new new product development game” – placed the notion of rugby as a metaphor for product development dynamics and specifically called out the term ‘Scrum’ as a description of small mobile teams.
Joint Application Development
Targetted at the first stage of the SDLC (requirements gathering and design). Before the development of JAD business requirements were gathered from interviewing stakeholders. JAD suggested the solution could be jointly designed. It is a very prescriptive methodology developed by Chuck Morris and Tony Crawford (IBM) in 1977. JAD is a rather strict methodology. The goal of the methodology is to develop business requirements ie. to accelerate the design phase and to increase project ownership. JAD has very specific roles:
- Executive Sponsor – set the vision, have the final say, can be strategically trusted to help steward the project within the organisation
- Facilitator – employ the method, plan the workshops
- Users – comprise 75% of the group, the people that have a need for the system
- IT – typically the people that will build the system
- Observer – simply put, they observe, it seems these are people from within the org who have some stake in the process but do not wish to be part of the design phase
- Scribe – participate only to get clarity, document the design
All participants have equal say.
There are 8 key steps in JAD:
- Identify objectives and limitations
- Identify success criteria
- Define deliverables
- Schedule the workshops
- Select participants
- Organise workshops and lead up prep
- Prep the participants
- coordinate workshop logistics
The workshop is the centerpoint for this methodology. JAD turns meetings into production orientated workshops (common also in agile etc). The basic idea being that by combining the users and IT staff a better product is designed.
Development lifecycle includes:
Method started with all in one room but used more commonly now with remote softwares. It puts emphasis on a diverse team, representing all stakeholders.
Earlier version of “design thinking” or Theory (Japanese)?
“It is critical that the facilitator be impartial and have no vested interest in the outcome of the session.”
“the focus of attention should always be on the JAD process itself, not the individual facilitator.”
“There is a school of thought that trained facilitators can successfully facilitate meetings regardless of the subject matter or their familiarity with it. This does not apply, however, to facilitating meetings to build information systems. A successful IS facilitator needs to know how and when to ask the right questions, and be able to identify when something does not sound right.”
“Many of the principles originating from the JAD process have been incorporated into the more recent Agile development process. ”
“Where JAD emphasises producing a requirements- specification and optionally a prototype, Agile focusses on iterative development, where both requirements and the end software product evolve through regular collaboration between self-organizing cross-functional teams.”
Rapid Application Development
Initially developed by Barry Boehm. A response to Waterfall. RAD was a general approach (category of methods) as an alternative to Waterfall. In 1986, Boehm created the articulated Spiral methodology in 1986 which is considered as the first RAD method. Confusingly another RAD method evolved later (which also subscribes to the general RAD approach) by James Martin and named RAD by him.
Waterfall was developed out of pure engineering where strict requirements for a known problem lead to an inflexible design and build. Software however, could radically alter the landscape and hence, in a sense, change the problem space. So the process had to be more flexible, reflective etc. Hence RAD recognised that information gathered during the life of a project had to be fed back into the process to inform the solution.
RAD is generally used for the development of user interfaces and placed a lot of focus on the development of prototypes over rigorous and rigid specifications. Prototypes were considered as a method to reduce risk as they could be developed at anytime to test out certain parts of the application. The emphasis being to find out problems sooner rather than later. Additionally, users are better at giving feedback when using a tool rather than thinking about using the tool, and prototypes can evolve into working code and be used in the final product. The later could lead to users gettting useful functionality in the product sooner.
RAD is considered a ‘risk driven process’ ie. it identifies risk in the hopes to reduce it.
The addition of functionality incrementally is considered an advantage in user facing products as they can access certain functionality sooner. This iterative approach also produces some of the software earlier and hence there is something to show sooner.
Some disadvantages could be a ‘ad-hoc’ design, and the continual involvement of users might actually be difficult to achieve and may either slow down the production or offside stakeholders.
– SPIRAL RAD
This method had four critical steps which were used repeatedly. Hence the project progressed in increments.
- determine requirements – gather requirements, determine objectives
- identify and resolve risks – identify risks and a solution for them. develop prototypes around the risk-solution.
- development – prototype leads to more detailed design and build
- plan the next iteration – the output is evaluated and the next iteration is planned
The spiral notion refers to the cyclical process PLUS as the project goes forward more knowledge is gained about the project. For a good description of this dynamic see https://www.youtube.com/watch?v=mp22SDTnsQQ
– RAD RAD
In 1991 James Martin took these ideas and turned RAD processes into another methodology, confuslingly called RAD.
There are four phases in the James Martin-defined RAD methodology:
- requirements gathering -t featuring a diversity of stakeholders. Essentially the objectvies, vision and scope is identified and a go-ahead given by those with the capacity to do so. Similar to (1) above.
- user design – groups of users and other stakeholders gather to start designing the product. It may, or may not, employ JAD. Prototypes are designed, built and tried where possible.
- construction – build starts, users are continously involved hence steps 3 and 4 are cyclical.
- cutover – hand over of the project to users, training etc
Dynamic Systems Development Method
Developed out of the RAD group.
Rational Unified Process
Agile manifesto – 2001
“The basic idea behind iterative enhancement is to develop a software system incrementally, allowing the developer to take advantage of what was being learned during the development of earlier, incremental, deliverable versions of the system. Learning comes from both the development and use of the system, where possible. Key steps in the process were to start with a simple implementation of a subset of the software requirements and iteratively enhance the evolving sequence of versions until the full system is implemented. At each iteration, design modifications are made along with adding new functional capabilities. ” Vic Basili and Joe Turner, 1975
Agile is interesting because it is not a method in itself, it is a proposal for a certain kind of development culture, under which various methodologies have come to find a home. Agile, in other words, is a set of values or principles.
Dave Thomas has made a good presentation on the problems with turning Agile into a complex method when actually it was intended as a set of values. https://www.youtube.com/watch?v=a-BOSpxYJ9M
…and his blog post: http://pragdave.me/blog/2014/03/04/time-to-kill-agile/
Many of the famous methodologies for Agile – XP, Crystal, Adaptive Software Development, and SCRUM all point to specific parts of the SDLC and all existed before the term agile was applied to them. These early ‘Agile Methodologies’ were originally refered to as ‘light’ or ‘lightweight’ methodolgies which was really a way to label them in contrast to much more strict, linear and formal methodologies like Waterfall.
In the first edition of “Extreme Programming Explained” for example, lightweight is used to describe this kind of SDLC
“XP is a lightweight methodology for small-to-medium-sized teams developing software in the face of vague or rapidly changing requirements.” (Kent Beck)
Its also interesting to note that this ‘Agile’ shift in approach to development came around the first .com bubble and reflects the shifting business environment where speed to market was seen as a factor in business success – hence the need for SLDC methods that supported faster product development life cycles.
So, proving once again that one of the most difficult thing for programmers to do is to name things, a small group gathered in 2001 to replace this informal naming with something a little more descriptive and positive (programmers didnt like saying they employed ‘light’ or ‘lightweight’ methods as this implied a lack of concern or vigour over the quality of their work and ability to do it). The group came up with the term Agile to retro-name an existing category and also identified the core characteristics of those methodologies that would conform to the Agile way. This became known as the Agile manifesto and it is as follows:
We are uncovering better ways of developing software by doing it and helping others do it.
Through this work we have come to value:
Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan.
So, in a sense, Agile is a framework, not a methodology. Methodologies conformed to it or, later, evolved from it. It is also interesting as the manifesto not only describes what it is, but what it is not. While this helps understand each value, it also shows that the manifesto very much defined itself as a reaction or remedy to the dominant culture. In some sense Agile represents a determined cultural fork within SDLC.
This is pretty interesting as:
- Free software existed as the default state in the 50s and 60s and was only became a movement self identifying as “Free Software” in the early 80s as an attempt to re-state the utility and re-enstate the cultural value of sharing code freely. Hence Free Software was also named after it existed.
- Perhaps OSS needs something like a stated manifesto on methodology or method-values. There have been political manifestos but not statements about process (apart from individual project contributing guidelines). Some attempts at a OSS manifesto include the GNU Manifesto (more accurately described as a Free Software Manifesto), and the famous Hacker Manifesto which is more of a ‘Catcher in the Rye’ approach to manifesto writing, and the follow up co-opting of the Hacker Manifesto by McKenzie Walk which is something of a detached (from hackers) marxist-academic articulation of what hacking might be, and the Debian Social Contract. However I think none of these speak much about the methodogies used to evolve a product. Other statements on method in OSS are more like anthropological reflections on culture, or odd attempts to encapsulate OSS in ill-fitting formal SDLC terms.
Wikipedia, perhaps wrongly, perhaps not, lists Open Source as a SDLC
Open source as a culture-method is good for developing infrastructure / backend solutions.
Open-source software has tended to be slanted towards the infrastructural/back-end side of the software spectrum represented here. There are several reasons for this:
- End-user applications are hard to write, not only because a programmer has to deal with a graphical, windowed environment which is constantly changing, nonstandard, and buggy simply because of its complexity, but also because most programmers are not good graphical interface designers, with notable exceptions.
- Culturally, open-source software has been conducted in the networking code and operating system space for years.
- Open-source tends to thrive where incremental change is rewarded, and historically that has meant back-end systems more than front-ends.
- Much open-source software was written by engineers to solve a task they had to do while developing commercial software or services; so the primary audience was, early on, other engineers.
This is why we see solid open-source offerings in the operating system and network services space, but very few offerings in the desktop application space.
“The real challenge for open-source software is not whether it will replace Microsoft in dominating the desktop, but rather whether it can craft a business model that will help it to become the “Intel Inside” of the next generation of computer applications.”
1. inception (developer) – “The far more common case of an open-source project is one where the people involved are having fun and want their work to be as widely used as possible ”
2. requirements assertion (developer) – “Open Source folks tend to build the tools they need or wish they had.”
3. no design phase 1 – “There usually just is no system-level design”
4. no design phase 2 – “Another casualty of being unfunded and wanting to have fun is a detailed design…Detailed design ends up being a side effect of the implementation. ”
5. implementation (developer) – “This is the fun part. Implementation is what programmers love most; it’s what keeps them up late hacking when they could be sleeping. ”
6. integration (developer) – writing manpages and READMEs
7. field testing – forums, issues in trackers, lists, irc, chat
1. requirements assertion (itch) – 1 or more developers have an idea and assert the requirements
2. build (scratch) – building begins, might be preceeded with some simple system design but more likely to proceed towards a proof of concept
3. use – users, might be developers only, use or try the software
4. interaction – interaction between intial devs and users and/or developers. May lead to use or contributions. This is the iterative growth of ‘the community’. Contributions might be code, documentation, marketing etc…
part 4 is all about developing culture.
Process might terminate at step 2, or cycle steps 1 & 2 only, or cycle 1,2,3 only. Successful projects generally repeat items 1-4 in a parrallel and cyclical fashion.
So…as to why this model is good for adoption…i think it is the openness of the process that does this. That is, it is the inclusiveness that makes the possibility of wide scale adoption possible. Companies that have product teams do not necessarily achieve this via this means since the ‘ownership’ of the product is contained within its own walls. Abele broke this model, extending the team into customer space.
- autonomy is an extremely important cultural condition https://hbr.org/1986/01/the-new-new-product-development-game
- often developer initiated, developer lead
- requirements, after the initial assertion, driven by user needs
- adoption strategy, if considered at all, occurs late in the process
- no design phase to speak of
- good for technical building blocks, not good for products
- very mono-cultural to the possible point of exclusion of all roles except developers
- non-developer users possibly rejected by the culture
- success of the project often depends on whether people want to use it, and how well the established ‘governing’ group manages stage 4
Extreme Programming (XP)
Founded by Kent Beck in the mid 80s. Interestingly, pre .com there was a general need to accelerate the design and producton of products. Speed to market was becoming an increasingly competitive factor. Extreme Programming was born from these conditions.
Seems Extreme Programming is really the cannonical start to agile values. Many practices are outlined in the book but the most common is ‘trust yourelf’. In a way you could narrate the evolution as being a move of early computer science as a practice of researchers and gov agencies, to corporate dev and markets in the 60s, then a reclaiming of programmer power in the 80s… Kent Beck’s writing is almost like a reclaiming of rights and power… like a workers movement… much of the language is centered around trusting yourself as a programmer.
The Extreme Programming tools and tricks are all about optimising quality of code and programmer satisfaction.
Diffusion of Innovation
Everett Rogers’ book. An apparent classic about how innovations are adopted.
Rogers lists the following components are critical to diffusion processes:
1. the nature of the innovation
4. social system
nature of the innovation
Important things to consider regarding the nature of the innovation:
1. Relative Advantage of the innovation
2. Compatability of innovation with existing values and experiences
3. Complexity – the simpler the better
4. Trialability – the more an innovation can be experimented with the better
5. Observability – the degree to hich the results of the innovation are visible to others.
In addition – Rogers believes an innovation diffuses more rapidly when “it can be re-invented”…this re-invention also helps ensure that “its adoption is more likely to be sustained”.
On top of this, communication is vital. Rogers states that “the heart of the diffusion process consists of the modelling and imitation by potential adopters of their network partners who have previously adopted”.
Interestingly, Rogers also states that while communication is easiest amongst those that have similar beliefs (homophilous), “one of the most distinctive problems in the diffusion of innovations is that the participants are usually heterophilous”…the nature of diffusion requires that “at least some degree of heterophily be present”….”ideally the individuals would be homophilous on all other variables […] even though they are henerophilous regarding the innovation. Usually, however, the two individuals are heterophilous on all of these variables”…hence….facilitation!
steps that one is likely to go through when adopting an innovation:
1. knowledge (of the innovation) eg. comms through website etc
2. persuasion (usually interpersonal comm)
4. implementation (and re-invention)
Interesting to understand how long it takes people to go through this process.
Those likely to adopt include:
2. early adopters
3. early majority
4. late majority
“the measure of innovativeness and the classification of a systems members into adopter categories are based upon the relative time at which an innovation is adopted”.
What is the expected and actual s-curve adoption rate for your innovation?
diffusion occurs within a social system. the social system defines the boundaries.
Examples of sector change
Community building begins with convincing people who don’t need to work together that they should.
– John Abele
Recently I have found some clues to these questions in the writings of John Abele (founder of Boston Scientific), particularly “[Bringing Minds Together]”(https://hbr.org/2011/07/bringing-minds-together). His writings have inspired me to look again at how I see Open Source.
Early in his career, John was involved with some technologies for non-invasive surgery. Today, such matters are handled day-to-day. Back then, however, surgery was invasive by definition. Back then, talk of non-invasive instruments for surgery would be like talking about screen-less phones today. Imagine trying to sell that.
So he had a hard time trying to sell a new idea that he knew could transform the medical sector. As he writes, “We were developing new approaches that had huge potential value for customers and society but required that well-trained practitioners change their behaviour. … Despite the clear logic behind the products we invented, markets for them didn’t exist. We had to create them in the face of considerable resistance from players invested in the old way and threatened with a loss of power, prestige, and money.”
That’s more-or-less what we face with Coko (Collaborative Knowledge Foundation) at the moment. Smart people that are under the painful burden of outdated workflows and technology often resist systemic change because it requires them to alter their normal ways of working. Their resistance to change is a huge problem if you wish to modernise the way knowledge is produced and reduce that burden on them.
In John’s case, he drew on some insights he had gathered early on in his career from Jack Whitehead, CEO of Technicon, a small company that had the patent for a new medical device. When trying to bring this product to market, Technicon had the odds stacked against them. No one, from the lab technicians through to the professional societies and manufacturers, wanted anything to do with it. So Jack drummed up some interest from early adopter types and came up with a surprising next step. He “told all interested buyers that they’d have to spend a week at his factory learning about it.”
That sounds like an odd sales pitch now, but back then (early 60s) apparently it sounded a whole lot more crazy. Nevertheless, Jack convinced a handful of excited early adopters to seize that day, and brought them into his factory.
During that week, the early adopters were not treated like customers but like partners. They were part of the team. They came to know each other, they worked together, they shaped the product. They became the team. As John says:
“When the week ended, those relationships endured and a vibrant community began to emerge around the innovation. The scientist-customers fixed one another’s machines. They developed new applications. They published papers. They came up with new product ideas. They gave talks at scientific meetings. They recruited new customers. In time, they developed standards, training programs, new business models, and even a specialised language to describe their new field.”
This meeting of once potential customers, now team members, not only contributed to the design of the technology but then took it out into the world and fueled adoption and interest in the product. What had humble roots with a group of early adopters was on its way to creating large-scale change.
John witnessed this process and realised this was strategy, not whimsy: “[Jack] was launching a new field that could be created only by collaboration — and collaboration among people who had previously seen no need to work together.”
John went on to form Boston Scientific and refined this strategy further with Andreas Gruentzig when introducing the balloon catheter to a hostile and uninterested market. Again he was successful in catalysing large-scale change.
Astonishing. But you could have told the same story about any number of successful Open Source projects. Indeed, John reflects on the parallels to Open Source :
“Just as Torvalds helped spawn the Open Source movement, and Jimmy Wales spearheaded the Wiki phenomenon, Andreas [Gruentzig] created a community of change agents who carried his ideas forward far more efficiently than he could have done on his own.”
The strategy in both cases started with a simple idea – to create a community of change agents.
Leaders need to cede control, not vigorously exert it.
– John Abele
The creation of a community where collaboration can flourish is a product of a particular kind of leadership – what John calls ‘collaborative leadership’. I have also seen this in many places including Open Source, Unconferences, and Book Sprints (which I founded). In the latter two cases, we call the collaborative leader a facilitator.
There are many modes of facilitation and choosing the right characteristics, the right mode, to create the right environment, is critical. John gives some interesting tips on what this mode looks like. The characteristics that I have gleaned from his article include the ability to:
- foster “collaboration among people who had previously seen no need to work together”
- manage egos
- earn trust
- respect stakeholder interests
- be authentic
- “listen, share, and ask good questions”
- share the credit
- use “we” not “I”
- support the choice of ‘alternative methods’
- create rituals
- create non-traditional spaces and experiences to come together
- ensure people “feel like partners”
- be willing to take direction from the ‘customer-participants’
- build trust over control
- “point out the deficiencies in the tools” you are making
- present ideas “in ways that [cry] out for input”
A good facilitator knows this language and these modes and can apply them but there are different requirements for facilitation depending on what you are trying to achieve. Book Sprint facilitation, for example, is much more product- orientated (books) than Unconference facilitation and requires a different set of strategies and tools. Most of the issues above I would categorise as being general rules of facilitation, however, those that modes that hint at enabling a group to create llarge-scalechange are the most interesting here. I think the last four points are telling:
- be willing to take direction from the ‘customer-participants’
- build trust over control
- “point out the deficiencies in the tools” you are making
- present ideas “in ways that [cry] out for input”
The most important issue at stake here is for the facilitator to create the space so others will come up with the product solution. This kind of facilitation creates vacuums, spaces where solutions don’t yet exist. This kind of facilitation is focused on a diverse range of stakeholders to design the product. This kind of facilitator doesn’t have all the answers but instead highlights problems and brings people together to create the solutions. Typical facilitation modes are necessary to enable an environment of trust and sharing, but enabling the participants to own the problems and create the productized solution requires a very specific mode of facilitation.
Can Open Source do this?
Open source software is about people.
– Sacha Greif
While these examples are about proprietary technology. It is not the proprietary-ness at stake here, it is the process that is important – and good process is the product of good facilitation. Often there is facilitation active in Open Source projects, but not necessarily, and where it does exist it is not typically with the kind of framing John has pioneered. Sure projects want people to use their software, they want to be sustainable, they don’t want to do all the work, they want community. However, Open Source projects are typically interested in bringing in more developers and not necessarily on fueling wide scale adoption (as odd as that may sound), and this informs the kind of people involved and the facilitation strategy employed.
There is a great article by Telescope founder Sacha Greif that illustrates this quite well. Sacha writes – “So why did I decide to make Telescope open source in the first place? It’d be easy to answer with typical clichés about “giving back to the community” or “believing in free software”….But the truth is a lot more pragmatic and selfish: I thought it would be a good way to get other people to help me accomplish my own goals, for free.”
Sacha goes on something of a journey with Telescope, undertaking some course corrections, and concludes “when it comes to Open Source software, software actually plays a fairly minor role. Open Source software is about people.”
I think this illustrates quite nicely where the mindset of open source projects generally are. Sacha went past the ideal of ‘build it and they will come’, a very tech-centric way of seeing things, through to understanding that the secret to success for an Open Source project lies with people, not technology. Sacha transitioned from developer to facilitator.
That is very laudable. However, facilitating developers to build your technology, is very different from facilitating a diverse set of stakeholders to develop a product, fuel adoption, changing user behaviors and putting your project on the path to creating wide scale change. The later requires involving different stakeholders and it involves a different mode of facilitation.
This is where Open Source is getting in its own way. Open source is about employing the right tools, facilitation included, to expand your developer base. That’s fine if you are building the plumbing – Open Source libraries, under-the- hood services, backend functionality etc can all be built well using this kind of method. However, if you want to be part of a team that creates wide scale change through a user-facing product, then I would suggest that open source as a culture-methodology is not the way to do it. Open Source is, at its heart, a techno-meritocratic culture. Great for building under- the-hood tools (builders building for other builders), terrible for producing products.
This shouldn’t be surprising. Somehow, however, it is. But then again when we look closely it sort of makes sense…Even the term ‘Open Source’ expresses just how much the code and those that make it form the core orientation and value metrics. Additionally, in a typical Open Source project, the person at the top isthe person with all, or most of, the technical answers (in a techno-meritocratic culture you can only not know so much before you ain’t the boss anymore). The famed benevolent dictator. And they are the culture keepers, the arbitrators, the facilitators. This sets the scene for a particular kind of facilitation that privileges the involvement of those with technical skills over anyone else. This all points to a certain kind of mono-cultural community.
To see a good example of this have a look at this post about Octocat. The entire emphasis of this article is to find ways to celebrate ‘non-code’ contributions. The term ‘non-code’ in itself is an indicator that anything other than code only has value by association (imagine a manager in any industry thanking a group of workers for their ‘non-management contributions’). Octocat was developed to measure ‘non-code contributions’ and to be effective ‘non-coders’ must log their activities in the projects issue tracker. Hence all contributors must use the same tools that the developers use. Octocat is a good project but it highlights just how focused the entire culture of Open Source is on technical development and needs everyone to fit into that paradigm. I believe this is at the cost of the product, adoption, and the potential change the product could make if managed differently.
“Diversity creates better projects. So, why aren’t designers participating in open source?” – Una Kravets
So, what must open source change to enable a project to create massive change? Open source requires a different kind of culture. Open source must diversify away from technocratic meritocracy to become diverse communities of stakeholders. Unfortunately just the opposite is happening – ‘non-coders’ are being chased away in droves.
This is not new news. Open source has sort of known there was something wrong with its culture for sometime. The wholesale migration to closed source desktops (Apple) by the open source development world highlights this. Why did we see this enormous migration? Because OSX is ‘nicer to use’. That tells us something. Open Source has generally failed on user experience (I must note that I do not believe that to be the case now for open source desktops) and it has failed because Open Source culture is not inclusive of designers. The mono-culture has very firm boundaries for what it can do well, and it cannot provide good UX unless it diversifies.
Una Kravets has done a very nice presentation about these issues. In this presentation Una goes into detail of why designers are not involved in open source and how to enable the conditions to improve participation.
The point here is that it is not just a question about representation (and the shockingly low numbers of women involved), it is also a question of the health of a product. As Una has argued well ‘diversity creates better projects’. I would like to take that one step further and suggest that diversity not only creates better projects, it creates better conditions for wide scale adoption and change. To develop a successful product you need to have the customers designing the solution, you need UX designers to finesse the interactions, designers to make it look great, evangelists to present it at conferences, you need developers to do the plumbing, you need users for the feedback, and you need facilitators that can bring all these people together and enable them to function as a passionate, problem-solving, engaged whole. To do that, all those people need to be drawn from a diversity of contexts, valued equally, and they must all have a say in what it is that is being created. You need what John Abele calls ‘a community of change agents,’ not a community of developers.
Involving and privileging only developers is not the right kind of culture to achieve success with a product. We must do away with the techno-meritocratic culture of Open Source and replace it with a more change-centric paradigm – the community of change agents. The root of this new kind of product development culture requires a specific kind of facilitation (‘collaborative leadership’), and the passionate involvement of a diverse set of stakeholders.
The writings of John Abele have led me through a curious journey. While I was always troubled by the technocratic nature of Open Source, I figured it was a transformational movement and I would figure out a way to exist within in it and assist where I could. However, I am now seeing very clearly that, while there are individual exceptions, Open Source as a movement has its own hard coded limits. Unfortunately, these are limits that stunt effectiveness and the ability to create wide scale change, especially with the user-facing products. Unfortunately, the Open Source movement itself is barely cognizant of these limitations, why they exist, and how they could be removed.
Coda for the trolls
It probably won’t help moderate your responses but just in case… a bit of a background of where I am coming from. I love Open Source, I think it has done amazing things. I am a free license and free culture advocate. I started communities of free documentation writers to support Open Source, have initiated many software projects – all Open Source – and contributed in humble ways to others. The Open Source projects I work in now use entirely open source tools. I have supported free culture campaigns with my own time and money including (most recently) The Cost of Freedom. I have lived and breathed Open Source for about the last 20 years. I haven’t used any other desktop since 2000 or so, and have belonged to, and worked with, many hacker collectives. I don’t intend to give up on Open Source any day soon, but I do intend to do what I can to make it better, however small that contribution might be.
*original article and history here https://www.adamhyde.net/open-source-is-not-about-people-or-technology-its-about-change/ *
With thanks to Andrew Rens for great feedback, and Steve Song for some inspiring asides.