XSweet 1.0!

We have 1.0 of the docx -> HTML transformation tool XSweet out today! It also has a new site:

http://xsweet.coko.foundation/

XSweet is a finely crafted tool. It takes docx files, those horrible mangy MS Word files, and translates them into clean, lovely, HTML. XSweet is open source, modular, and very nicely done.

A huge tip of the hat to Wendell Piez (XML guru), and to Alex Theg. As geeky as it sounds, I loved watching these two chat about the issues they encountered making this software. The attention to detail was really unbelievable. Amazing work. XSweet is a finely crafted tool.

More info on Coko https://coko.foundation/announcing-xsweet-1-0/

If you want to do a deep dive into why I think this is important, I wrote this some time ago – https://www.adamhyde.net/typescript-redistributing-labor/

OLPC

img_3558

So, you may never have heard of OLPC, but it was quite a thing. OLPC = One Laptop per Child. A project initiated by MIT. Its mission was to change the world – essentially to educate millions of kids that did not have much in the way of educational resources. The basic idea was to make really really cheap laptops and then get them to kids that needed them. The Laptop was pretty innovative at the time as there was no such thing as a ‘small factor’ laptop back then. You just had big, expensive, laptops. OLPC tried to get the price down by innovating in form factor and the attendant technologies like screens…

olpc-x0-unboxed

It was a pretty cool thing.

Anyways… there was an interesting article that a friend passed to me about it. I just landed in NZ, so I’m pretty knackered and can’t quite write what I want to write about this now. But here is the link:

https://www.theverge.com/2018/4/16/17233946/olpcs-100-laptop-education-where-is-it-now

I’m writing about it here as I was pretty close to the project. In fact, I facilitated all the documentation and worked closely with Walter Bender and the crew. We (FLOSS Manuals) did a few Book Sprints to create the docs and as it happens I found this old, old vid online from that time:

I found the OLPC project deeply flawed. It was a movement without proper resourcing and an untethered ambitious aim. But I liked Walter and many of the people involved. It was interesting to watch this whole thing unpack slowly infront of me. Anyways… will read the article more thoroughly when I’m more present and make some more comments from my own experience of the project.

continued…. actually, I slurped up a coffee and read the article more closely. It’s pretty accurate. I realised things were topsy with the OLPC when I discovered the reason why we were doing docs (apart from them not having any in the first place) was because the laptops were selling at (something like) $180 a unit, but costing $186 (0r something) per unit in support costs alone. They were making a loss on each machine purely on support costs. It wasn’t a surprise to me as many people needed a manual just to work out how to open it up…

1 2

 

 

But I have to say, Walter Bender was the real deal. Super smart and humble as pie. He had his heart and vision in the right place, and if he had been supported the OLPC project would not have lived and died as a hardware project. His vision was much greater and worthy but, as the article discussed, didn’t get the traction over the sexy hardware sell.

Anyway… some of the manuals we made are still online 🙂

http://booki.flossmanuals.net/xo/

It was even translated to Farsi and a heap of other languages that are only to be found now in the Internet Archive (eg Greek, Arabic and others). The docs were also available in book form, electronic book, and on the laptops themselves.

117 one-laptop-per-child-nicolas-negroponte-9 software_activities_11 2952057847_42de996872_b img_3342

Things I think Open Source Needs to Address

Following on from my previous post… here are some issues I think the Open Source movement needs to address:

  1. Where is Open Source Failing?
    I believe a discussion is needed about why there is a failure of Open Source to capture the desktop and web platform space. We have some successes, notably WordPress and Chrome, Firefox, but generally speaking these two categories of software are a big fail for Open Source. How come?
  2. Are we past the era of licenses and code?
    Open Source has been preoccupied with notions of code and licensing since it started. That’s fair enough, given that the first couple of decades have been about establishing fundamentals. But aren’t we past this moment? Can’t we count issues such as licensing pretty much as solved and move on? Is it now time to put other issues at the center of the culture?
  3. Becoming a full stack culture…
    Open Source, the culture, the values, the toolset, the lexicon, all value code and developing code above all else. When will Open Source embrace all roles in software development and value them equally… what does that kind of project look like? What does a network of different skill sets need to thrive?
  4. Diversity/Gender
    This is being addressed to some degree in the last years at various open source conferences and get togethers… however, it would be good to face up to the fact that (as far as I can tell) women make up < 10% (10% seems to be the high end of estimations) of all developers in Open Source… this isn’t something just for discussion, this is an issue which required urgent action. We also need to ask ourselves why do less women participate in Open Source than in proprietary software culture?
  5. The Open Source Cultural Method
    Open Source is a culture/method for solving problems. Where can this be improved? What can we borrow from existing Software Development Life Cycles that may improve our game?
  6. Open Source tools…
    What do we need to change in existing workflow tools (eg gitlab/github) to enable Open Source culture to improve? What innovations can we bring to beat the proprietary projects?
  7. Solidarity
    What does it mean to show solidarity for other open source projects? Do we need to do it? Do we expect this from other projects towards what we are working on?
  8. Challenging cultural community tropes
    What are the failings of community staples such as the Benevolent Dictator for Life? Is open source really any different from other types of collaboration? Why do we think it’s special? What can we learn from other community efforts of shared production? What tropes hold value and which are unhealthy?
  9. Encouraging Criticism
    Why aren’t there active critical voices encouraging healthy criticism? What does that say about the culture?

This is just a starter…I am sure there are many other issues that need to be put front and center. These are my starting 9…

Where are the Open Source Critics?

I hear a lot of stupid arguments about open source. For example, this article is particularly flawed. Silly comments like “Not getting paid while seeing major corporations make hay from your volunteer labor both erodes good will.” is full of so many strange assumptions I don’t really know where to start unpacking it.

But… leaving all this aside, I do believe open source needs criticism. Well-founded, reasoned, criticism. Criticism on points of culture, on diversity, on where the open source cultural method is failing, on what open source doesn’t do well. Any movement needs this kind of refreshing of the dialog so that it can improve and, as it matures, help it tackle the next generation of issues it needs to solve.

However, for many years I believe we have seen a failure of informed  criticism of open source. I can find many who herald open source as a wonderful movement, but I do not see healthy criticism. Without it, open source risks being stuck and not moving on, which can very well be not only damaging but eventually terminal for the movement.

Open Source Successes

Believe it or not there is a bit of an active campaign out there to discredit Open Source in the publishing world. I’m pretty sure this is because the nay sayers are feeling a little threatened, else why comment on it at all?

So, just for the record…here are some examples of successful open source projects…

1. The Internet

Yup. The underlying infrastructure for the internet is all open…TCP and IP, also known as TCP/IP, the governing communications protocols among all computers on the Internet, are both open. Don’t forget also that the Berkeley Internet Name Domain, otherwise known as BIND, is open source, and is by far the most common way to publish and resolve Domain Name Service (DNS) queries. Without it, you would type ‘http://google.com’ into your browser location bar and nothing would happen…

2. The Web

Let’s not forget that the internet and the web are not the same things. The internet is that hidden substructure that enables information to ferret around all over the world. The web is what surfaces that info up to you to consume in the browser and other mechanisms. It’s no surprise that the vast market share of browsers are open source now that IE (closed) has fallen to a wee 12-15% or so of the market and Chrome (open) has risen to hold the majority share (somewhere around 60%). Firefox is coming in around 12-15% also. I got my stats from here – https://www.netmarketshare.com/browser-market-share.aspx

Why, do you wonder, did IE fall so far from grace? Could it be that Open Source enables a faster adoption rate?…cause…y’know …it’s free …maybe…

3. Your Computer

Whats that? I hear you say…”I don’t use Linux!”…. yeahyeah…  but do you use Apple? The under-the-hood OS that enables the pretty bells and whistles is the BSD Kernel (the core of the operating system) and that is licensed under BSD, one of the most permissible and liberal Open Source licenses. In fact, Open Source now beats Windows (the only major proprietary OS) in total user-facing operating system market share (including mobile devices, see below) (https://en.wikipedia.org/wiki/Usage_share_of_operating_systems).

4. Your Phone

And your tablet! In the world of mobile devices, Open Source is just killing closed source operating systems hands down (https://en.wikipedia.org/wiki/Mobile_operating_system).

5. Your Website

Yep. over 25% of all websites in the world are run by…WordPress…that humble, non-threatening, simple blogging engine is now an internet monster. It is also by far the most popular CMS in the world. See https://om4.com.au/wordpress/market-share/

Also, don’t forget that your WordPress site is probably delivered to the world by an Apache web server which is open source and holds 50% of the market share (http://www.perfectleads.com/marketshare/apache) which, together with nginx, means open source holds about 65% of the web server market.

And more…

Well, so much more…. Publishers have been skeptical of open source for a very long time but they don’t seem to realise their businesses are built on top of it… They literally could not operate without open source…  It’s like a homeowner complaining that they don’t think plumbing, as an approach to delivering water, works. Or that houses are a bad idea. I’m at a bit of a loss to what to say to such folks…

Designing for All

At Coko we are tackling the long unsolved problem of design in Open Source. I have written a bit about this elsewhere, and it is a hard problem to solve, mainly because it is not solved anywhere – there are no good community models for this and very little discourse about the issue.

Design, of course, covers many things. I am chiefly talking about UX, although we have also solved the problem of designing for users by getting users (I prefer to call them ‘use case specialists’) to design their own systems. But this is not about that. I want to talk just about community processes for UX that will yield common best practices, while also enabling each organisation to realise their desired aesthetics ie achieve the look and feel they want that best reflects their organisation. As reflected in the following screenshots of xpub-Hindawi, xpub-eLife, and xpub-collabra:

b

c

a

There are several mechanisms that we have employed to achieve this. The first is that we have ‘elevated’ design to the same level of value in the community as code. I say ‘elevated,’ not because we had to raise its value in the Coko community, but rather the estimation of the value of design in the open source community at large is very low. So, against this, as a baseline, at Coko we have chosen to value design more than in your typical open source project. Also, I don’t want that to imply there are other disciplines in the software development process that do not rank as highly as these two – documentation, feedback, testing and so on… these are all equally valued alongside design and code, but for now I want to focus just on UX design.

To ‘elevate’ design, it took just the simple step of stating it to be so. As the co-founder of this community I have some sway over the direction and form of the culture. So I cashed out some of this capital to make a very simple statement when I was facilitating a community meeting that we must value design as much as we value code. It was this simple statement that set a placeholder for building processes that put design in the value-center of the community.

Next, when there was a call for more processes to assist communication across the various orgs, I made sure we had two work groups which each consisted of one person from each collaborating organisation. These two foundational WorkGroups were the Dev WorkGroup and the Design WorkGroup.

The Design WorkGroup then went ahead and started discussing how we might contribute to a platform where each org could have the look and feel they wanted. It was a process of first understanding the problem and then working through possible solutions. It was necessarily like this, a process of discovery before implementation, because there are not very many established or discussed processes like this in the open source world that we could follow (as mentioned above). So we have to discover the problem, and then invent a solution.

The problem boils down to small levels of abstraction. Interfaces are, after all, made up of many small parts. The problem therefore resided in how to account for different stylistic approaches to, what we now call, atoms (see Brad Frost’s article on Atomic Design), and how atoms fit together to form ‘molecules’ ie. we needed a level of abstraction that would allow any org to have the look and feel  they wanted for an (eg) button, and also how that button was situated with other elements.

Julien Taquet (Coko) and Nick Duffield (eLife) met with Yannis Barlas (Coko) in Athens for three days to thrash this out.

image-2

They came up with an agreed common set of standardised CSS attributes per UI element, and associated reusable CSS variables that could be used across any UX element.

You can see these lists here…

  1. Color Variables
    https://gitlab.coko.foundation/pubsweet/design/issues/1
  2. Typography Variables
    https://gitlab.coko.foundation/pubsweet/design/issues/2
  3. Form Variables
    https://gitlab.coko.foundation/pubsweet/design/issues/3
  4. Spacing Variables
    https://gitlab.coko.foundation/pubsweet/design/issues/4

That is the simple starting point. In effect, this means that if any organisation designs a feature using UI elements (eg a login form) they can use these shared style names and populate them with their own values. This means another org’s stylesheet will apply equally well to inherited/reusable components as it does to the ones they build themselves.

Which seems simple: it is… but understanding the problem space, and where the commonality lies, was half the battle.

The group has now also identified a common set of UI elements that can be reused and are iterating through this list, designing them as they go. See https://gitlab.coko.foundation/pubsweet/design/issues/21

This is required because even the simplest UI element may have several contexts it has to account for. Take the example of a ‘simple’ button, one of the simplest UI elements of them all…. however, there are actually 5 different button states that must be accounted for – the default state, active, hover, pressed and disabled. In each case, the style variables must be applied appropriately. See here for more information – https://gitlab.coko.foundation/pubsweet/design/issues/19

So, it’s not quite as easy as it might first seem.. the good news is that the teams are very committed to working through these issues, as without doing this groundwork, components are really not reusable.

dsc05093

The result is you can grab components that another org has made, be it a page-based component (eg. Dashboard), or a molecule (eg a Staff Assignment dropdown) and plug it into your system and it will look like it is part of your system.

We are making good progress on all of this, including now integrating common best practices for accessibility into the design process. Also, since this groundwork has come quite far, new orgs joining the Coko community can benefit a lot from the work already done and pick up components from the shared/common UX stylebook and integrate them… pretty cool….

Collaboration on UX

So, I have a pet thesis…. it goes something like this… Open Source, as we know it out there in the wild, is a code-centric pursuit. Its roots are in code, the culture is all about developers solving problems, the tooling is code-centric, and the culture values code above all else. That is not a very controversial thesis so far. However, I have experienced a lot of kickback when I get to the next bit… and that is, open source has both succeeded and failed because of the these characteristics.  It has succeeded to produce a lot of code, and a lot of tools and libraries that developers need, but it has failed in any category of software where the primary beneficiaries of the software are not developers.

To me it makes sense. But bringing it up has produced so much blowback, notably from long-time open source practitioners, that it only reinforces to me the truth in the mini thesis. There is a huge blind spot in open source culture that does not recognise where it has failed. It is a pity because I believe the first step in succeeding in these areas is to recognise why open source has failed. Only then can you fix it.

I believe it will take a long time to change this and I once had aspirations to be part of the fix-it movement, but I think it’s too long a game so I have elected instead to play a part in addressing these issues in realms where I know I can have an immediate effect. Hence, in Coko,  a not for profit I co-founded,  we are spending a lot of time to see how we can create an open source project that values all contributions as much as traditional open source values code contributions.

Part of this is making way for UX design. It is pretty much the high-value role, when it comes to conquering the most obvious limitations in open source, since it is where the rubber hits the road when it comes to ‘user’ meets software.

In the Coko community, Julian Taquet and Nick Duffield (eLife) are putting a lot of time into this with the able assistance of Yannis Barlas (there has also been a lot of excellent input from Sam and Tam from YLD and others). I’ve shepherded the process from a distance – setting the scene and making the space for the right people to do the right work and making sure this work has the right value accorded to it in the Coko culture.

So, in essence, we have realised that collaboration in UX comes down to three things:

  • identifying the common ground
  • tooling
  • process

Common Ground

Identifying the common ground actually took some discussion. We initially thought the common ground – think of it as UX space shared across projects – was on the page-level. We thought, for example, one org would need a dashboard and so they make it and others can use it. While this is true for a limited number of specific page level components it soon became obvious that there was a higher opportunity for reuse should we break the page-level components down into smaller components. We then had a short period of lexicon confusions (“duh. what sort of component is a login?”) until we settled on Brad Frost’s atoms and molecules concepts and lexicon.

After that, we could make faster progress as we had identified, and could talk about, a new level of component that had infinitely more opportunities for reuse across projects.

That was the highest level common ground we identified.

Tools

Next, we moved onto tooling… there had been a lot of discussion about this. The trick was to get the designers to experiment with and understand the options. It also highlighted the fact that in each collaborating org there was a different workflow that might play into some discussions and not into others. For example, Julian from Coko does as much of the tweaking of CSS variables and values in the code, whereas Nick from eLife does the design and then hands these designs to others to implement. So, in many ways, the questions about tooling are informed by these workflows ;and different people, even if identified as having the same kind of role, have very different questions and needs. This is important to take into account and we will need to keep this very much in focus as we go forward. One easy way to keep issues of this in focus is to always insist that any discussion, workflow change or feature that affects design workflow must include the designers in that conversation. You get better results and people are much happier! Not to mention that it saves a lot of time as there is more informed discussion as you progress and fewer possibilities for major rollbacks because someone wasn’t looped in.

This conversation on tooling took quite a few weeks; there were many options on the table and we wanted to make sure the right people were in the right conversations. It came to a close, for at least the foundational stage, when Nick and Julien met with Yannis in Athens for a 3 day UX meet and nailed down the final agreements on tooling (amongst other things). This highlights to me also the need for periodic in person meets if you can manage it, as required. You can clear out a lot of ‘hanging issues’ in one swoop if you meet in person for short focused bursts.

Below are some pics from this very important meet in Athens showing Nick, Julien, and Yannis at work on the whiteboard in our Athens office.

image-1

file

image-2

We now have general agreement on the use of CSS styled components, as well as an understanding of what a basic atom or molecule would look like, a high-level list of agreed design principles, an approach to ‘plain vanilla’ theme with org-specific overrides, and a prescribed set of common CSS variables.

You can see the embryonic documentation about design decisions here – https://gitlab.coko.foundation/pubsweet/design

So, the crew nailed down the tooling with a few things left to discuss. There are many tools in the design/UX workflow. Unfortunately, there are not many good open source tools to support open source design workflows. That is because of the limited scope of open source projects to involve designers as I mentioned above. So design has not been seen as a priority and, consequently, the tooling is not there. You can see this in GitHub and GitLab – where are the tools that support designer workflow?

Process

Which brings me to the final item – process. We are still working this out, but essentially each org will design components as needed, and then scope these to common established CSS variables, and then ask for feedback through Mattermost. When agreed, the component will be built and committed to the common styleguide for reuse. When the flow is established it should be a pretty fast way of working. The idea being, in essence, that atoms and molecules are developed for a target, common, ‘plain vanilla’ theme, and then each org can have their own theme that will use those common components and apply their own CSS values to the common variables.

After writing the above I asked Julien if it all looked ok, he wanted to make the following additional point about tooling and sharing design ideas and mocks:

For now, we’ve stopped the conversation at ‘let’s share svg through syncyng folders and see how it goes’.

The only things that will stay in the library of components, shared for all Pubsweet apps (from Coko and others), is the code. Therefore, since there is no easy way to test mockups with different themes (which is the thing that we would need), we will end up sharing png and discussions (for which, the Increment project could be helpful: https://gitlab.coko.foundation/adam/increment).

So for now, I don’t think we can say more, specifically if we don’t want to force the user on a specific tool.

In other words, the atoms and molecules will go into the shared component library, but the mocks and discussions leading up to the creation of the components will occur elsewhere. This is because the current open source software development tools don’t support these processes (collaboration around iterative design in a live environment).  Julien also makes the point that the mocks will also be shared as SVG since that allows each org to decide for themselves which environment (design software) they will use to create the mocks, so SVG, in a way, acts as an interface between the collaborating designers.

It sounds simple, but it takes time to work out simple solutions. We are also finding that there are no established models for collaborating on open source UX that we know of that we can follow… so discovery always comes with an overhead but it’s also exciting to be leading, in some small way, with creating a demonstrable real ‘in the wild’ example of how to collaborate across orgs on UX design in an open source project.  That comes with its own challenges, and with its own sense of satisfaction.

Commits

One interesting metric for the health of an open source community is the number of commits – how many, and how many orgs/individuals represented. Below is a screenshot of activity on Coko’s gitlab  repository (https://gitlab.coko.foundation). It shows the activity over one day. Looks pretty good! Note – this is just activity for developers…  it’s not a good metric for activity of other members of the community, especially for designers… I’ll post a bit of an update about our design activity in a bit as it’s pretty interesting what’s going on….

One days commits. Showing a healthy early stage in an Open Source project.

 

Release now…

I come across a lot of projects (especially in the academic realm) that don’t like releasing ‘open source’ code until the code is all nice and pretty. Some also want to get governance structures etc in place before doing anything…

It is almost once or twice a month that I find myself in discussion with a project about this. They are usually very nice people, well meaning, but don’t really have a good handle on how open source works.

Firstly, there is a well-known mantra in software development that is true in general, but particularly true for open source:

Release early, release often

In the open source world, there are very special reasons why this is a best practice and baseline premise.  First, it tells people that are watching, the people that will want to use and/or contribute to your project, that you are serious about open source. If instead, you hold back the code, it sends the signal that you ‘don’t really get it’. I can’t recall a single conversation with an open source advocate that argued for holding back the code until it’s all nice and neat.  So, you’re sending out a signal that you don’t really understand how open source works, and that is a bad look.

This is especially true if there is anyone among your potential target collaborators/partners that have been around the open source block a few times as they will be extremely wary of anyone saying ‘we will release it’…or (worse) ‘it is open source, but we haven’t released it yet’… you might be stating this because you ‘know’ it to be true… but from the ears of the listener (especially and old hand) there is nothing different between what you are saying (which you consider fact) and a promise of sorts. You are asking people to trust you to do this sometime in the future – and people like me, who have heard this a lot, will automatically tend not to believe you. Not because we don’t think you believe this to be true, not because we are inherently distrustful people, but because we have heard so many, many, people say this that have not gone ahead and done it.

If you say it is open source, prove it by handing out the repo URL. Otherwise, don’t expect anyone to believe you or trust you  – and trust, as it happens, is the most important ingredient in successful open source communities. If you wrong foot it at the start, you have just created yourself an unnecessary uphill battle to rebuild trust when (if) you finally do release the code…

Secondly, open source models are all about adoption…. that is the entire market-killing model of open source. Adoption. Open source can kill proprietary products just simply because the threshold for adoption is lower (ie, free to try, free to install, free to use etc). If you wait until everything is in place, then you have just killed one of the most important moments to build interest and adoption – early stage development. Interested orgs/individuals can download the code and see what is about as soon as they hear about it… that way they can see where you are going, and if it is the right direction for them, they may decide to adopt the product (even in early stages) and/or contribute to the project. It is very good if this happens as these early adopters will be the product’s main advocates, drawing in the next layer of interested parties… they become the project’s salespeople. They will be especially good at doing this because they have been in there from the beginning, following and (hopefully) participating in all the discussions and decisions, and so they understand the project in detail and can talk to it with authority. That is invaluable.. .why wait? Don’t wait…if you do, the threshold for getting involved is going to be a lot higher (since there is more to understand) and the burden of helping people understand the project, and ‘selling’ it,  will fall entirely on your shoulders rather than being nicely distributed upon the shoulders of the early adopters…

Lastly… all open source projects grow organically and respond to the needs of the moment. So don’t wait to build governance structures etc before putting the code out there. This is not only bad for the reasons discussed above, but in the early phases of the project, there is nothing to govern…  It is just you (you, literally, or your org)… so first build the community and then look at what infrastructure you need to put in place to run that community. That is what governance is all about ..running the community. So, why not wait and see who shows up, and then decide what the governance structure (etc) should look like.  Also, as a last word, make sure your community is involved in discussing and deciding the shape of the governance…

anyways…just a few thoughts on this from Rarawa beach!