What I’m Pondering Now

I am slowly working out a pretty clear picture for the argument that open source needs to start developing its ‘own way’ to develop products. The issue is complex and interesting. While, for example, the Cabbage Tree Method solved some of these issues, it also opened up many more questions. How, for example, do we build flourishing open source product development communities… this is the next level of problems I’m pondering and I’m developing a narrative for it to help think through the issues as well as engage others in the discussion.

On this path, there are some issues that I need to get better at breaking down and explaining and I may have to evolve the language I use quite a bit to hit the right spot. For example, ‘product’ is hard to define. I started by saying ‘user-facing products’ etc but I have received some push back on this. Mainly the argument is that ‘developers and users are often the same’. I agree with that, but my point is very specifically that I’m interested in cases where this is not the case; I’m interested in cases, which I consider to be very typical product development scenarios, where the developers and the users are not the same people. This is exactly the area I’m targeting.

So, I’d like to define a product as any software where there are people involved in creating the solution that are not the primary stakeholders with the problem. This distinguishes the situation from the typical ‘itch to scratch’ model where the people creating the solution are the same as the people with the problem, in fact, you could say that I’m interested in developing models which mean that, to some understandable degree, we are ‘scratching someone else’s itch’.

Now…this might not be a suitable framing either. This is because I believe that ‘scratching someone else’s itch’ is a good way to describe how current open source culture attempts to solve this very tricky product development problem ie. developers creating solutions for ‘users’. However, if we are to develop a healthy open source product development culture, we actually all have to have a stake in the problem. Otherwise, why would we be interested in being involved? This is, in fact, at the very heart of the issue. If we are to “scratch someone else’s itch” we actually have to reframe what we do. We have to reframe the open source project mission at a higher level to attract people of all kinds to participate in solving a higher value problem.

For example, we need to reframe the mission from something like ‘building a browser’ to ‘building the open web’ (to borrow from a contemporary example). In my world, it is a reframing of ‘building better publishing platforms’ to ‘improving the sharing of scholarly research’. If we can reformulate what we are trying to achieve then we can attract a broader range of stakeholders and place the user (as a use-case specialist) at the core of the culture alongside others that have a stake in this mission (including developers, UX people, system architects, marketing people etc). We are, in effect, reformulating Eric Raymonds ‘itch to scratch’ model from

Every good work of software starts by scratching a developer’s personal itch.

to something like

Every good open source product starts by scratching a shared itch.

I think this is super interesting because it retains the essential feature of open source culture that has made it so valuable and effective – that those participating in creating the solution are ‘internal’ to the problem. That is not often true of typical proprietary solutions models where ‘the user’ is, in essence, treated as a ‘research object’. But at the same time, this statement up-ends the current discussion and directs open source away from a developer-centric solutions model towards a culture that must embrace collaboration by a diversity of stakeholders to succeed.

Historical Progression

I am also slowly structuring the historical narrative. This comes from my discussions with Tony Wasserman. In essence, the argument can be made that open source has indeed solved many problems. The initial set of problems addressed were infrastructural –  this can be clearly linked to the development of the Linux kernel where Linus Torvalds wanted a free and open OS (the itch) and started the development (the scratch). He then reached out to others that were interested to communally scratch this shared itch. It’s a good framing story, well known and understood and not likely to be too controversial. The interesting point of the story, is that this foundational moment was in the pursuit of building infrastructure (an OS) which is primarily a technical endeavor.

To take the narrative further, I think it can be said that the ‘next set of problems’ (an important framing too), was developer tools ie. tools developers needed to do what they need to do. This would include things like basic email clients (eg pine/mutt etc), command line tools, IDEs, and reusable libraries. This may need to be refined a little when describing what this literally refers to, so as to avoid push back. The general arc of the narrative, however, is a story of progression. This aids in setting the scene for the ‘next set of problems’ ie. solving the problem of producing products.

I like this progression story because it also illustrates that open source culture has always been evolutionary. What we need to do now is take the culture forward to solve the next big problems it faces. It is a positive framing of a march forwards and a march towards solving the perplexing problem of open source product development.

The Open Source Way

In addition, I think I am getting some clarity on what I’m asking. We know how to develop products, that’s been going on for years. You can look at any number of amazing products and look at how they were developed. The big issue here, is how we do this ‘the open source way’.

I would frame this as largely a question about the bazaar vs cathedral, to borrow from Eric Raymond’s famous framing. We can, in open source projects, borrow methodologies like ‘Agile’ to build products (I have problems with notions of Agile as a method which mostly are inherited from this presentation by Pragmatic Dave, but will refer to Agile as a methodology here for efficiency’s sake). But these methods are pretty strongly managed and break two important foundational principles of the bazaar:

  1. they are formal processes – which puts their whole modus operandi in question as to its appropriateness in the informal bazaar. This makes community a casualty and community is, IMHO, what we are after.
  2. they assume the solution provider is external to the problem – ie Agile (etc) is a typical “scratch someone else’s itch” model. In the way that the cathedral is not with the people like a bazaar is, but for the people, Agile processes are cathedral-like. They lose all the power of ‘being of people’ which is central to the open source community model – the essential value being, in my opinion, derived by placing the ‘people with the problem’ at the center of the solutions culture.

So, how do we do this ‘the open source way’? Well, I think we need to rethink our community models. We need to develop communities that solve these problems in a cross-disciplinary way where use-case specialists, developers, UX people marketeers etc all work together such that all ‘roles’ are playing on an egalitarian playing field. This is the heart of the matter. How do we scratch this ‘higher level’ itch together?

In addition, we need to keep software development being a fun, productive, conversation. Voluntary working environments are opt-in processes. So, if they are to work, they need to hold to a notion of informality and they have to be fun. In my mind, this means that conversation is key. Productive conversation is both the driver and a key metric for a flourishing community.

When we know how to develop crossdisciplinary conversations where all parties are valued equally, we will be well on our way to forming communities that produce amazing open source products.

Just How Radical is This?

I guess this is the big challenge. When I think about an open source project culture that performs like this I think the fundamental principles of open source remain the same – we retain, celebrate and build community as a primary activity, we hold collaboration high in the culture, we celebrate individual contributions towards a collective effort, we continue to work in ways that are antithetical to intuitive proprietary-commercial culture by working out modes for collaborating with competitors (etc), we develop processes that appeal to both intrinsic and extrinsic motivations, and continue to carefully cultivate notions and practices of stewardship and governance, while at the same time upholding the full spectrum values of openness and freedom so deeply embedded in open source culture.

However, operationally, things would look quite different. In a community where there are a diverse set of equally valued roles at play to produce open source solutions, we would expect language, tools, working processes, and value metrics, as well as a whole swag of culture practices, to evolve in directions that, over time, would mark a clear departure from how things are done now.

That is the interesting part to all of this. The principles of open source remain the same, but the practice will look quite different.

Even though I am still working all this out ‘out in the open’ in an incremental and evolutionary way, I am convinced we need to do this. The big questions that are starting to surface to me revolve around how we do this. How do we evolve communities to experiment with these ideas and work out the path ahead. It is going to require a lot of empathy, bravery, trial and error to discover ways forward that can improve upon what we are doing now. Which is why I’m spending quite a bit of time talking to people, evolving the narrative, and thinking through strategies with others that can help us move forward.

Collaborative Product Development

v 0.9 Mar 22 added some cursory Extreme Programming notes
v 0.8 Mar 20 started reading Diffusion of Innovations (5th ed)
v 0.7 Mar 16 finished reading Xtreme Programming Explained (2nd ed)
v 0.6 Feb 15 “added extreme programming notes”
v 0.5 Feb 13 “notes about ba”
v 0.4.1 Feb 12 “add more Agile notes”
v 0.4 – Feb 11 “added Agile notes”
v 0.3 – Feb 9 “added Waterfall, SDLC, OSS notes”
v 0.2 – Feb 8 “added Radical Application Design (RAD) notes”
v 0.1 – Feb 6So, I am pondering more how to work with product development methods to improve the process for products that are of the ‘open source’ genre. My first step in this process was to write this post:
https://www.adamhyde.net/leadership-diversity-and-sector-change-in-open-source/This has lead to the realisation that open source is really a culture-methodology. While I believe in open source licenses, I think the default culture-method leaves a lot to be desired. So, I am investigating what are the key foundations, processes, and tools that would enable a different kind of product development. At this early stage in my pondering, I think there are several key ingredients:

  • a diverse set of stakeholders
  • facilitation
  • a distributed team
  • open licenses

In addition, I have the following goals:

  • make excellent products
  • enable widespread adoption

The goals are as important as the ingredients. After all, if you have don’t have lemons, you shouldn’t be trying to make lemonade.

I am thinking this exploration could have a working title ‘Collaborative Product Development’.

My first step is to look at historical methods for software product development and see what their approaches have been. The intent being to learn more about how software development processes are targeted at specific objectives and operational contexts. I think this will help develop an understanding of why certain methods are effective for this specific case, which in turn will help me understand which bits can be discarded.

I’ll keep this particular page in my Ghost blog as the notepad for my evolving thoughts.

Initial Thoughts, Learnings, Extractions
  1. There are a number of methods for building software products. Agile, lean, feature-driven method, XP, JAD etc. I would also include Open Source as a default methodology for product development and, even though open source projects can employ these other methods there is a kind of default modus operandi for open source which defines how things are done. Hence I think it can be considered to be a methodology.
  2. I have, this week, been involved in collaborative product design processes. I facilitated a small group (the second session in a series) in the beginnings of the process. The group consisted of those that started the project (owners?), a project manager, and people who knew the workflow intimately and would end up using the software. We started with the whiteboards, and pens. The opening question in the first session, which seemed to open the door to a great conversation, was “how would you design a system if the workflow would be transferred to the browser”. The second session went through the same process to get validation and to expand the group slightly (this was important). We now have a baseline understanding of what we are building. Each session was about 1.5 hrs.
  3. In collaborative design sessions, I think it is important to not rush to implementations too fast. Stay on the whiteboard for as long as necessary – it feels more organic and the output obviously is created by all in the room (as opposed to presenting well thought out ‘finished’ mocks).
  4. move slowly to more sophisticated artefacts (mocks) and get signoff as a group at every step. Give plenty of room for revising issues. Identify big problems but do not rush to solve them. It is ok to push some stuff down the road as long as it is agreed by all and the issue is clearly noted. Document all outputs of collaborative design sessions.
  5. it is ok to keep the documentation a little rough. it communicates that it is a WIP and things can be revised. The quality of the artefacts is secondary to the quality of the discussions about them.
research notes (scratch pad):
  1. http://www.itinfo.am/eng/software-development-methodologies/#chapter3
  2. https://en.wikipedia.org/wiki/Jointapplication design
    “Arnie Lind’s idea was simple: rather than have application developers learn about people’s jobs, why not teach the people doing the work how to write an application? Arnie pitched the concept to IBM Canada’s Vice President Carl Corcoran (later President of IBM Canada), and Carl approved a pilot project. Arnie and Carl together named the methodology JAD, an acronym for Joint Application Design, after Carl Corcoran rejected the acronym JAL, or Joint Application Logistics, upon realizing that Arnie Lind’s initials were JAL (John Arnold Lind).
    The pilot project was an emergency room project for the Saskatchewan Government. Arnie developed the JAD methodology, and put together a one-week seminar, involving primarily nurses and administrators from the emergency room, but also including some application development personnel. The project was a huge success, as the one-week seminar produced a detailed application framework, which was then coded and implemented in less than one month, versus an average of 18 months for traditional application development. And because the users themselves designed the system, they immediately adopted and liked the application. “
  3. learning through collaboration http://www.commonwealthclub.org/events/archive/podcast/joseph-henrich-secret-our-success
    “high fidelity copying from each other” is a secret to our success as a species and because “we are interconnected we can learn stuff drom each other and combine it”. (yesyes!). The ability to innovate “depends on the size and connectivity of the population”. The ability to adapt like this “drove genetic evolution”. Brings up interesting questions on who to learn from. “the size of the population and the degree to which people are interacting and sharing information actually affects how fancy the technology is going to get and how many tools you can produce….also if the the group should get cut off for some reason and the population is reduced, the group will suffer a loss of technology”. “Genetic evolution gives rise to cultural evolution” which in turn drives genetic evolution. “think of our learning abilities as adaptation” -> “cultural driven genetic evolution”. “A lot of creativity is the recombination of different ideas”. “If you want to be creative out yourself at the nexus of very different and divergent information”. “cultural learners” (people who learn from each other). “good cultural learners”.
  4. From the book by Joseph Henrich
    “natural selection favoured genes for building brains with abilities to learn from others.” p53
  5. crystal method considered to be a subset of Agile
    Feels very west coast. A bit too hippy dippy in its method names.
  6. https://medium.com/@nayafia/what-success-really-looks-like-in-open-source-2dd1facaf91c
  7. http://www.sersc.org/journals/IJSEIA/vol8no32014/38.pdf
    ” The development of OSS gains popularity due to the wide availability of the internet facility to each and every region of the world, parallel development, peer review, parallel debugging, expert developers, and feedback” “Many different developers, user, or co-developers can participate in the development of the OSS.” “The development of OSS is always initiated by single developer or a single group, who starts the development of software for its own “personal itch” “
  8. http://www.oreilly.com/openbook/opensources/book/vixie.html
  9. http://opendesign.foundation/resources/philosophy/why-you-should-design-for-open-source/
  10. http://www.ijcaonline.org/volume21/number1/pxc3873327.pdf More OSS SDLC (OSDLC)..this one a little bit too praisey of open source
  11. http://opendesignnow.org/
  12. http://openusability.org/
  13. http://opendesign.foundation/
  14. https://www.smashingmagazine.com/2010/09/the-case-for-open-source-design-can-design-by-committee-work/
  15. https://github.com/opensourcedesign/resources
  16. http://opensourcedesign.net/
  17. “Extreme Programming (XP) is about social change. It is about letting go of habits and patterns that were adaptive in the past, but now get in the way of us doing our best work. It is about giving up the defenses that protect us but interfere with our productivity. It may leave us feeling exposed. ” Kent Beck, Extreme Programming Explained
  18. amazing talk by Kent Beck https://www.youtube.com/watch?v=aApmOZwdPqA
  19. Agile is Dead https://www.youtube.com/watch?v=a-BOSpxYJ9M
  20. TDD is Dead https://www.youtube.com/watch?v=z9quxZsLcfo
  21. discovering ba
    “Nonaka told his interviewer that they were creating ba, a Japanese term that describes a field or space where people freely and openly share what they know in the service of creating something new. Ba resembles the concept of “flow” as set forth by psychologist Mihaly Csikszentmihalyi: It is the mental state that occurs when a person is fully immersed in whatever he or she is doing. But unlike flow, ba is never solitary; it exists among two or more people. As Nonaka says, “In ba, there is no you or me, there is only us, sharing a here-and-now relationship.” Ba can occur in a work group, a project team, an ad hoc meeting, a virtual e-mail list, or at the frontline point of contact with customers. It serves as a petri dish in which shared in­sights are cultivated and grown.
    Companies can foster ba by designing processes that encourage people to think together.”
  22. “A phronetic leader mobilizes timely judgment in others by building a culture that is strong, nurturing, and sustained by informal connections.”
  23. https://hbr.org/1986/01/the-new-new-product-development-game
  24. “Cross-fertilization. A project team consisting of members with varying functional specializations, thought processes, and behavior patterns carries out new product development. The Honda team, for example, consisted of hand-picked members from R&D, production, and sales. The company went a step further by placing a wide variety of personalities on the team. Such diversity fostered new ideas and concepts.”
  25. “All of us, as far as I can remember, thought waterfalling of a huge project was rather stupid, or at least ignorant of the realities… I think what the waterfall description did for us was make us realize that we were doing something else, something unnamed except for “software development.””
    –GM Weinberg http://www.craiglarman.com/wiki/downloads/misc/history-of-iterative-larman-and-basili-ieee-computer.pdf
  26. “The conclusion that stands out most clearly from our field study observations is that the process of developing large software systems must be treated, at least in part, as a learning and communication process.”  Bill Curtis quoted in http://www.craiglarman.com/wiki/downloads/misc/history-of-iterative-larman-and-basili-ieee-computer.pdf

Development Methods

I think I need to cover at least the following questions for each:

  1. what problem was the methodology a response to
  2. what are its key goals
  3. what application is it intended for

In addition, I need enough information to succinctly describe the methodology and to provide some praise and critique.


“I think what the waterfall description did for us was make us realize that we were doing something else, something unnamed except for “software development.” – Gerald M Weinberg

The history of Software Development Lifecycles is a history of experiment, description, and rejection.

Systems Development Life Cycle is what I refer to here as ‘method’ or ‘methodology’. It describes any methodology used as a framework for developing software. While all the methods described below can be used in a pure form, often SDLCs are combined and hybridised. Sometime for better, sometimes for worse. However, it is interesting to note that the basic framework address the following needs one way or another:

  1. conception
  2. requirements gathering
  3. design
  4. development
  5. use
  6. evaluation

Some SDLCs are cyclical, some incremental, some strict, others loose, linear etc

The interesting question to me is what can we learn from these if we want to develop a methodology (SDLC) for developing products with free licences and the principles of open source.

Also interesting is that there have been attempts to define OSS as a SDLC. See http://www.sersc.org/journals/IJSEIA/vol8no32014/38.pdf

Somethings that I notice:

  1. SDLCs are methods that feel like they are to be deployed
  2. you become a subject of a SDLC
  3. OSS feels like a culture you want to inhabit
  4. you become a citizen of OSS

So, I think part of the framing is that OSS ‘SDLCs’ are about creating the right culture and employing tools to shape that. SDLCs (in the traditional) form are about employing a structured method and making sure everyone adheres to it. So, I would prefer to think of the kind of methodology i am looking for to be framed as a ‘Product Development Culture’, not a Software Development Life Cycle. The former is a culture, the later a structured process. The former can be shaped, the later employed. Best to pick the right conditions for success from the beginning 😉 To improve OSS PDC we need the right tools to shape culture – this is the culture-method I am after.

The interesting thing here is that the history of SDLC highlights the problem with history. It is easy to tell a story of a timeline and certain events happening in sequence, almost like building blocks in time. However that isn’t how things happen. We can, for example, set up a nice story about how ‘Waterfall’ came from a pre-software engineering era, and it was simply applied to software development, and then in the 1980s, people started rebelling against this legacy process and developed more iterative methods. However, while there is some level of truth in that story, it also hides the complex paralled, unfloding, emergent processes and histories that is the ‘full telling of this story’. For example, iterative development in software is recorded as happening as early as 1957 (and certainly in the 60s for Project Mercury), but the process evolved out of non-software projects as early as the 1930s. As Larman and Victor Basili noted, “We are mindful that the idea of IID [Interactive and Incremental Development] came independently from countless unnamed projects and the contributions of thousands and that this list is merely representative. We do not meanthis article to diminish the unsung importance of other IID contributors.”

While Waterfall is often said to have been born in 1970 (– Royce, Winston (1970), “Managing the Development of Large Software Systems“,) So before we even had waterfall for programming, we had iterative development as this article so nicely highlights: http://www.craiglarman.com/wiki/downloads/misc/history-of-iterative-larman-and-basili-ieee-computer.pdf

What this highlights is that these ideas and methods have evolved in a many faceted, evolutionary manner. There is no simple star wars narrative here of good vs evil although that is often how the history of SDLC is told.


“In my experience, the simpler method has never worked on large software development efforts…” – Royce Winston

Waterfall in software dev was actually developed in the 1970s, which is late for computing. It was first described by Winston Royce in 1970 (“Managing the Development of Large Software Systems“,)

The steps included:
1. Design comes first
2. Document the design
3. Do it twice
4. Plan, control and monitor testing
5. involve the customer

Interestingly, it has nothing to do with how open source generally works 🙂 except! involve the customer is often also at the end of OSS product development.

Even more interesting…Waterfall as we know it now is really what Royve was criticising when he introduced the term. ie a very strict, sequential process. Conception, initiation, analysis, design, construction, testing, implementation, and maintenance. It is one way process, no going back to an earlier step at any stage. This is known as ‘single pass’ or ‘single step’ Waterall and differentiates it from Royce’s method that advocated building twice (2 pass).

A foundation of Waterfall, as we know it now, is the argument that the more work done gathering requirements and designing, the more time you save and risks you avoid when it comes to building.

Rational Unified Process (RUP)

Adaptive Software Development

Pragmatic Programming


Mid 80s, origins outside of software, namely the 1986 article “The new new product development game” – placed the notion of rugby as a metaphor for product development dynamics and specifically called out the term ‘Scrum’ as a description of small mobile teams.

Joint Application Development

Targetted at the first stage of the SDLC (requirements gathering and design). Before the development of JAD business requirements were gathered from interviewing stakeholders. JAD suggested the solution could be jointly designed. It is a very prescriptive methodology developed by Chuck Morris and Tony Crawford (IBM) in 1977. JAD is a rather strict methodology. The goal of the methodology is to develop business requirements ie. to accelerate the design phase and to increase project ownership. JAD has very specific roles:

  • Executive Sponsor – set the vision, have the final say, can be strategically trusted to help steward the project within the organisation
  • Facilitator – employ the method, plan the workshops
  • Users – comprise 75% of the group, the people that have a need for the system
  • IT – typically the people that will build the system
  • Observer – simply put, they observe, it seems these are people from within the org who have some stake in the process but do not wish to be part of the design phase
  • Scribe – participate only to get clarity, document the design

All participants have equal say.

There are 8 key steps in JAD:

  1. Identify objectives and limitations
  2. Identify success criteria
  3. Define deliverables
  4. Schedule the workshops
  5. Select participants
  6. Organise workshops and lead up prep
  7. Prep the participants
  8. coordinate workshop logistics

The workshop is the centerpoint for this methodology. JAD turns meetings into production orientated workshops (common also in agile etc). The basic idea being that by combining the users and IT staff a better product is designed.

Development lifecycle includes:

  1. conception
  2. requirements
  3. design
  4. implementation
  5. integration
  6. test
  7. maintenance

Method started with all in one room but used more commonly now with remote softwares. It puts emphasis on a diverse team, representing all stakeholders.

Earlier version of “design thinking” or Theory (Japanese)?

“It is critical that the facilitator be impartial and have no vested interest in the outcome of the session.”

“the focus of attention should always be on the JAD process itself, not the individual facilitator.”

“There is a school of thought that trained facilitators can successfully facilitate meetings regardless of the subject matter or their familiarity with it. This does not apply, however, to facilitating meetings to build information systems. A successful IS facilitator needs to know how and when to ask the right questions, and be able to identify when something does not sound right.”

“Many of the principles originating from the JAD process have been incorporated into the more recent Agile development process. ”

“Where JAD emphasises producing a requirements- specification and optionally a prototype, Agile focusses on iterative development, where both requirements and the end software product evolve through regular collaboration between self-organizing cross-functional teams.”

Rapid Application Development

Initially developed by Barry Boehm. A response to Waterfall. RAD was a general approach (category of methods) as an alternative to Waterfall. In 1986, Boehm created the articulated Spiral methodology in 1986 which is considered as the first RAD method. Confusingly another RAD method evolved later (which also subscribes to the general RAD approach) by James Martin and named RAD by him.

Waterfall was developed out of pure engineering where strict requirements for a known problem lead to an inflexible design and build. Software however, could radically alter the landscape and hence, in a sense, change the problem space. So the process had to be more flexible, reflective etc. Hence RAD recognised that information gathered during the life of a project had to be fed back into the process to inform the solution.

RAD is generally used for the development of user interfaces and placed a lot of focus on the development of prototypes over rigorous and rigid specifications. Prototypes were considered as a method to reduce risk as they could be developed at anytime to test out certain parts of the application. The emphasis being to find out problems sooner rather than later. Additionally, users are better at giving feedback when using a tool rather than thinking about using the tool, and prototypes can evolve into working code and be used in the final product. The later could lead to users gettting useful functionality in the product sooner.

RAD is considered a ‘risk driven process’ ie. it identifies risk in the hopes to reduce it.

The addition of functionality incrementally is considered an advantage in user facing products as they can access certain functionality sooner. This iterative approach also produces some of the software earlier and hence there is something to show sooner.

Some disadvantages could be a ‘ad-hoc’ design, and the continual involvement of users might actually be difficult to achieve and may either slow down the production or offside stakeholders.


This method had four critical steps which were used repeatedly. Hence the project progressed in increments.

  1. determine requirements – gather requirements, determine objectives
  2. identify and resolve risks – identify risks and a solution for them. develop prototypes around the risk-solution.
  3. development – prototype leads to more detailed design and build
  4. plan the next iteration – the output is evaluated and the next iteration is planned

The spiral notion refers to the cyclical process PLUS as the project goes forward more knowledge is gained about the project. For a good description of this dynamic see https://www.youtube.com/watch?v=mp22SDTnsQQ


In 1991 James Martin took these ideas and turned RAD processes into another methodology, confuslingly called RAD.

There are four phases in the James Martin-defined RAD methodology:

  1. requirements gathering -t featuring a diversity of stakeholders. Essentially the objectvies, vision and scope is identified and a go-ahead given by those with the capacity to do so. Similar to (1) above.
  2. user design – groups of users and other stakeholders gather to start designing the product. It may, or may not, employ JAD. Prototypes are designed, built and tried where possible.
  3. construction – build starts, users are continously involved hence steps 3 and 4 are cyclical.
  4. cutover – hand over of the project to users, training etc

Dynamic Systems Development Method

Developed out of the RAD group.

Rational Unified Process


Agile manifesto – 2001
“The basic idea behind iterative enhancement is to develop a software system incrementally, allowing the developer to take advantage of what was being learned during the development of earlier, incremental, deliverable versions of the system. Learning comes from both the development and use of the system, where possible. Key steps in the process were to start with a simple implementation of a subset of the software requirements and iteratively enhance the evolving sequence of versions until the full system is implemented. At each iteration, design modifications are made along with adding new functional capabilities. ” Vic Basili and Joe Turner, 1975

Agile is interesting because it is not a method in itself, it is a proposal for a certain kind of development culture, under which various methodologies have come to find a home. Agile, in other words, is a set of values or principles.

Dave Thomas has made a good presentation on the problems with turning Agile into a complex method when actually it was intended as a set of values. https://www.youtube.com/watch?v=a-BOSpxYJ9M

…and his blog post: http://pragdave.me/blog/2014/03/04/time-to-kill-agile/

Many of the famous methodologies for Agile – XP, Crystal, Adaptive Software Development, and SCRUM all point to specific parts of the SDLC and all existed before the term agile was applied to them. These early ‘Agile Methodologies’ were originally refered to as ‘light’ or ‘lightweight’ methodolgies which was really a way to label them in contrast to much more strict, linear and formal methodologies like Waterfall.

In the first edition of “Extreme Programming Explained” for example, lightweight is used to describe this kind of SDLC
“XP is a lightweight methodology for small-to-medium-sized teams developing software in the face of vague or rapidly changing requirements.” (Kent Beck)

Its also interesting to note that this ‘Agile’ shift in approach to development came around the first .com bubble and reflects the shifting business environment where speed to market was seen as a factor in business success – hence the need for SLDC methods that supported faster product development life cycles.

So, proving once again that one of the most difficult thing for programmers to do is to name things, a small group gathered in 2001 to replace this informal naming with something a little more descriptive and positive (programmers didnt like saying they employed ‘light’ or ‘lightweight’ methods as this implied a lack of concern or vigour over the quality of their work and ability to do it). The group came up with the term Agile to retro-name an existing category and also identified the core characteristics of those methodologies that would conform to the Agile way. This became known as the Agile manifesto and it is as follows:

We are uncovering better ways of developing software by doing it and helping others do it.
Through this work we have come to value:
Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan.

So, in a sense, Agile is a framework, not a methodology. Methodologies conformed to it or, later, evolved from it. It is also interesting as the manifesto not only describes what it is, but what it is not. While this helps understand each value, it also shows that the manifesto very much defined itself as a reaction or remedy to the dominant culture. In some sense Agile represents a determined cultural fork within SDLC.

This is pretty interesting as:

  1. Free software existed as the default state in the 50s and 60s and was only became a movement self identifying as “Free Software” in the early 80s as an attempt to re-state the utility and re-enstate the cultural value of sharing code freely. Hence Free Software was also named after it existed.
  2. Perhaps OSS needs something like a stated manifesto on methodology or method-values. There have been political manifestos but not statements about process (apart from individual project contributing guidelines). Some attempts at a OSS manifesto include the GNU Manifesto (more accurately described as a Free Software Manifesto), and the famous Hacker Manifesto which is more of a ‘Catcher in the Rye’ approach to manifesto writing, and the follow up co-opting of the Hacker Manifesto by McKenzie Walk which is something of a detached (from hackers) marxist-academic articulation of what hacking might be, and the Debian Social Contract. However I think none of these speak much about the methodogies used to evolve a product. Other statements on method in OSS are more like anthropological reflections on culture, or odd attempts to encapsulate OSS in ill-fitting formal SDLC terms.


Wikipedia, perhaps wrongly, perhaps not, lists Open Source as a SDLC

Open source as a culture-method is good for developing infrastructure / backend solutions.

Open-source software has tended to be slanted towards the infrastructural/back-end side of the software spectrum represented here. There are several reasons for this:

  • End-user applications are hard to write, not only because a programmer has to deal with a graphical, windowed environment which is constantly changing, nonstandard, and buggy simply because of its complexity, but also because most programmers are not good graphical interface designers, with notable exceptions.
  • Culturally, open-source software has been conducted in the networking code and operating system space for years.
  • Open-source tends to thrive where incremental change is rewarded, and historically that has meant back-end systems more than front-ends.
  • Much open-source software was written by engineers to solve a task they had to do while developing commercial software or services; so the primary audience was, early on, other engineers.

This is why we see solid open-source offerings in the operating system and network services space, but very few offerings in the desktop application space.

“The real challenge for open-source software is not whether it will replace Microsoft in dominating the desktop, but rather whether it can craft a business model that will help it to become the “Intel Inside” of the next generation of computer applications.”

Stages (Vixie)
1. inception (developer) – “The far more common case of an open-source project is one where the people involved are having fun and want their work to be as widely used as possible ”
2. requirements assertion (developer) – “Open Source folks tend to build the tools they need or wish they had.”
3. no design phase 1 – “There usually just is no system-level design”
4. no design phase 2 – “Another casualty of being unfunded and wanting to have fun is a detailed design…Detailed design ends up being a side effect of the implementation. ”
5. implementation (developer) – “This is the fun part. Implementation is what programmers love most; it’s what keeps them up late hacking when they could be sleeping. ”
6. integration (developer) – writing manpages and READMEs
7. field testing – forums, issues in trackers, lists, irc, chat

My thoughts,:
1. requirements assertion (itch) – 1 or more developers have an idea and assert the requirements
2. build (scratch) – building begins, might be preceeded with some simple system design but more likely to proceed towards a proof of concept
3. use – users, might be developers only, use or try the software
4. interaction – interaction between intial devs and users and/or developers. May lead to use or contributions. This is the iterative growth of ‘the community’. Contributions might be code, documentation, marketing etc…

part 4 is all about developing culture.

Process might terminate at step 2, or cycle steps 1 & 2 only, or cycle 1,2,3 only. Successful projects generally repeat items 1-4 in a parrallel and cyclical fashion.

So…as to why this model is good for adoption…i think it is the openness of the process that does this. That is, it is the inclusiveness that makes the possibility of wide scale adoption possible. Companies that have product teams do not necessarily achieve this via this means since the ‘ownership’ of the product is contained within its own walls. Abele broke this model, extending the team into customer space.

Notable issues:

  • autonomy is an extremely important cultural condition https://hbr.org/1986/01/the-new-new-product-development-game
  • often developer initiated, developer lead
  • requirements, after the initial assertion, driven by user needs
  • adoption strategy, if considered at all, occurs late in the process
  • no design phase to speak of
  • good for technical building blocks, not good for products
  • very mono-cultural to the possible point of exclusion of all roles except developers
  • non-developer users possibly rejected by the culture
  • techno-meritocracy
  • success of the project often depends on whether people want to use it, and how well the established ‘governing’ group manages stage 4

Extreme Programming (XP)

Founded by Kent Beck in the mid 80s. Interestingly, pre .com there was a general need to accelerate the design and producton of products. Speed to market was becoming an increasingly competitive factor. Extreme Programming was born from these conditions.

Seems Extreme Programming is really the cannonical start to agile values. Many practices are outlined in the book but the most common is ‘trust yourelf’. In a way you could narrate the evolution as being a move of early computer science as a practice of researchers and gov agencies, to corporate dev and markets in the 60s, then a reclaiming of programmer power in the 80s… Kent Beck’s writing is almost like a reclaiming of rights and power… like a workers movement… much of the language is centered around trusting yourself as a programmer.

The Extreme Programming tools and tricks are all about optimising quality of code and programmer satisfaction.

Diffusion of Innovation

Everett Rogers’ book. An apparent classic about how innovations are adopted.

Rogers lists the following components are critical to diffusion processes:
1. the nature of the innovation
2. communication
3. time
4. social system

nature of the innovation

Important things to consider regarding the nature of the innovation:
1. Relative Advantage of the innovation
2. Compatability of innovation with existing values and experiences
3. Complexity – the simpler the better
4. Trialability – the more an innovation can be experimented with the better
5. Observability – the degree to hich the results of the innovation are visible to others.

In addition – Rogers believes an innovation diffuses more rapidly when “it can be re-invented”…this re-invention also helps ensure that “its adoption is more likely to be sustained”.


On top of this, communication is vital. Rogers states that “the heart of the diffusion process consists of the modelling and imitation by potential adopters of their network partners who have previously adopted”.

Interestingly, Rogers also states that while communication is easiest amongst those that have similar beliefs (homophilous), “one of the most distinctive problems in the diffusion of innovations is that the participants are usually heterophilous”…the nature of diffusion requires that “at least some degree of heterophily be present”….”ideally the individuals would be homophilous on all other variables […] even though they are henerophilous regarding the innovation. Usually, however, the two individuals are heterophilous on all of these variables”…hence….facilitation!


steps that one is likely to go through when adopting an innovation:
1. knowledge (of the innovation) eg. comms through website etc
2. persuasion (usually interpersonal comm)
3. decision
4. implementation (and re-invention)
5. confirmation

Interesting to understand how long it takes people to go through this process.

Those likely to adopt include:
1. innovators
2. early adopters
3. early majority
4. late majority
5. laggards

“the measure of innovativeness and the classification of a systems members into adopter categories are based upon the relative time at which an innovation is adopted”.

What is the expected and actual s-curve adoption rate for your innovation?

social system

diffusion occurs within a social system. the social system defines the boundaries.