publishing workflows – Page 2

Rethinking Workflow

I spent an awesome session at UCP with Dan Morgan and Alison O’Connell discussing workflow. We were working through ideas of Journal workflows and how they would be encapsulated in a single system, namely xpub-collabra. It was a super interesting discussion and I think we have some solid concepts that rethinks how systems can capture multiple workflows in such a way they avoid endless configuration options and hardcoded prescriptive paths. We will be creating some narrative around this to get out here for people to think on and feedback…

What is xPub?

So, there are a lot of product names at Coko – PubSweet, Editoria, XSweet, INK, xPub etc etc etc… so becoming tricky to track, but I wanted it seems there are quite a few people interested in xPub right now.

xPub is not really a product as such, it’s more a group of products – each of which relates to Journal workflows. The names for each product indicyae those relationships: xpub-collabra, xpub-elife, xpub-faraday (Hindawi) and xpub-epmc

Each of these is a platform, so yip, you read it right – there are actually no less than 4 journal platforms being developed in the Coko community, not including the Micropublications platforms (of which there are two – one developed with Wormbase, and the other with the Organisation for Human Brain Mapping).

So, right from the get go our ‘offer’ is not standard. We don’t offer one platform to rule them all – there are many journal platforms in production. All of these are built on top of PubSweet… PubSweet is a kind of ‘headless’ publishing platform….it’s more or less the backend brains in that it is a kind of framework that ‘thinks like a publishing system’ but has not determined a workflow for you. So, each of the xpub-* platforms are actually publishing platforms built to meet a specific workflow and built on top of PubSweet…

Here is a crappy diagram (drawn in haste) to get the basic idea across:

In the above, you see a PubSweet platform (eg. xpub-elife). PubSweet itself is the whole app – it sort of ‘encapsulates’ everything. Really though, you get the PubSweet core and then extend it with front-end components to meet your workflow needs. A typical front-end component might be a login screen, or a dash, a submission info page, a reviewer form etc.

You can also extend PubSweet on the back end as well (eg to enable integration with external services etc). The following is a more accurate but slightly techy architectural overview (you can skip this image and the following paragraph if you want to avoid the slightly techie part of this article).

ps2

In case you are wondering – everything is written in Javascript. Why did we choose JS? It was a deliberate choice to go with a language that was prolific. Almost every dev around these days needs to know some Javascript (it’s the most popular language by far on Github), this makes finding a developer to work on your project is as easy as we could possibly make it. JS is also a phenomenal language these days. Fast, sophisticated and more than capable of supporting large-scale publishing requirements. I mean, if it’s good enough for Paypal, Netflix, Linkedin, Uber and ebay, then it is good enough for you.

So each PubSweet platform has its own collection of front and back end components to meet the workflow of someone’s dreams… the idea is that to achieve the platform of your dreams you can reuse what others have already built and then just build what isn’t already available. In a sense, you can ‘assemble’ your platform from existing parts.

c1e22edf10cd018c9d1a697c29fbd891

The nice thing about this is that each of the organisations building Journal platforms are sharing the following:

the same back end/framework (PubSweet)
various front (and back) end components
lessons learned…

Each organisation has a vision of their ideal Journal workflow. They then design and build this on top of PubSweet, but as they do, they build various components (either page-based components such as a dashboard, or smaller UI components we call atoms and molecules) and they share these components with everyone. Hence, you should check the list of components before you start building in case the component you need already exists.

We have various agreed-upon ways to build and share components (see this as an example). These best practices are continuously evolving but you can read some of the latest discussions about this approach here – https://gitlab.coko.foundation/pubsweet/pubsweet/issues/408

Of course, all code is reusable in the Coko community because it’s all open source. The best practices are there to make the code easy to reuse.

All agreements as per above are made by consensus by the community. It is actually a pretty snappy process – don’t believe every crappy thing you read about how open source is built. Open source community processes can be elegant and fast, and the resulting code can beautiful. Coko is a good example of this – a community of professionals collaborating together to make fantastic open source software.

So… back to the xPub story. To make these decisions on how to share components etc, we have regular meetings with various workgroups (we keep the numbers in each group small so we can move fast), and we also have quite a few in-person meetings. Not only do we have PubSweet community meetings where all of these organisations meet, but we have various get-togethers on various topics if required. For example, we met in Cambridge recently to discuss Libero (the eLife delivery product), before that EBI there was an onboarding session in Athens, next month we have a special designers workgroup in-person meet, anbd so on. All this helps us keep in contact with each other, which helps build trust, but also turns out to lift energy levels and boost production. These meetings are fun.

Also at these meetings, we learn from each other. One of the big problems in this sector, and one of the reasons people ask ‘why are publishing platforms so hard to build?‘ (after a number of high profile failures), is that there has not been a focused effort to share experiences on how to build publishing platforms. So, that is what we are doing – at each meeting we talk about what we have learned, how we are thinking about things, and show each other what we have done. The last meeting, for example, each xPub platform gave a deep dive to everyone in the group in a process known as speed geeking (it wasn’t so speedy as each table had 25-30 mins to go through their platform).

So, xPub isn’t a single platform, it’s more the community that is building journal platforms on top of PubSweet and sharing learnings and components. It is a very cool thing.

As a community, we also produced a book entitled ‘PubSweet – how to build a publishing platform’. You can get this for free here – https://coko.foundation/books/

I can also send you a print copy (they look great!) if you send me a postal address.

The book covers many things – a whole lot of technical documentation for how to build and share components etc, as well as some information about to think about workflow and how to map this into a PubSweet system. Personally, I don’t think the technology is very hard when it comes to building platforms – we knew what we wanted from the beginning with PubSweet and went about and built it (not to downplay the extraordinary job Jure Triglav has done in leading this effort). But the real hard stuff is actually thinking about workflow – not many publishing orgs have had the opportunity to think about designing their workflow to meet exactly what they want (rather than shoe-horning it into an existing system), and so we have had to beat this track for other to follow. It has been quite a journey but the book pretty effectively outlines how to think about workflow (which of course is a process that can be accelerated using Workflow Sprints which is a process I designed to facilitate a publisher to design their own workflow and PubSweet platform in one day).

0e7a0345

In that book you can see some high level ‘architectures’ of each of the xPub platforms such as the following (xpub-collabora):

02-adam-workflow-v3

Or this (xpub-epmc):

ebi

So, you might be asking…. at what stage of development are each of these platforms? … I’ll leave it to each of the teams to say exactly, but it has been shocking to see how fast things have come along. I mean, the xPub community only really got together for the first time mid-last year, and building started for most late last year and early this year. It looks like we will have a lot of platforms landing by the end of this year. In this sector that is lightning fast. EBI are particularly impressive – they started two months ago and are almost ready to go – if it weren’t for the fact we are all replacing the under-the-hood data model we designed together at the last PubSweet meet, they would be good to go already. They can achieve such speed because they are reusing a lot of the components that the other teams built before them (and because EBI are just very good 🙂 ).

I can speak a little more about xpub-collabra since that is the platform Coko is building for and with the Collabra Psychology Journal. It’s looking pretty nice. We have some work tidying up the UI, replacing the data model and a few other things, but it’s looking rather good. We are also putting some time into making it a generic platform since the Collabra workflow follows a fairly ‘plain vanilla’ model. So we are building in some management interfaces and various bits and pieces to make it more widely useful – for example, Giannis just built in a Submission Info builder – enabling a Journal admin to build their own submission forms. It requires some hand-holding to use right now, but we’ll shape it to be very usable by your general journal adminy-type. We also have to integrate ORCID and DOIs, extend the range of submission file types etc… but it’s pretty close.

Below is a video showing xpub-collabra in action. It is a a version from some months ago, but you can see the workflow pretty well in this demo.

I think most of the xpub community is going to offer their new platforms to the market in various ways. This will be interesting as there are very different approaches at play. Hindawi, for example, is looking to make their platform a multi-tenant platform, eLife will put JATS at the centre of the workflow etc… So look for more news on that also. For our part, Coko will offer xPub through a partnership with a hosting provider – probably with the same organisations we will work with for Editoria hosting services. Since everything we do is open source, we will also be supplying all the Docker, Helm and Kubernetes scripts so that you can set up your own commercial hosting service if that is your cup of tea (or you can extend your offering should you already be a hosting provider). Coko is pretty close to getting our first hosting partnership set up, so look for news of that soon!

One last thing – because of the modular nature of any platform built on top of PubSweet, it is possible to take any of the xPub platforms and customise it to meet your needs. No need to start building from zero. Additionally, the modularity means you can extend the systems with your own interesting new innovation – finally a place for innovation to call home in the publishing platform world….

So… there is a lot going on in the xPub world….we look quite different to the rest of the market because we are not building one platform – we have instead focused on building a community to support the development of multiple platform solutions for journal workflows. You can pick and choose which one you want, or build something else, reusing as much of the other systems as you can to reduce your development costs (we also spend a lot of time onboarding new folks and supporting them as much as we can).

But it’s not just about improving the game today – supporting the optimisation of workflows is one part of what we are trying to do, the other is to support future innovations.

If you want to know more, feel free to jump into the Coko community channel and chat – https://mattermost.coko.foundation

Come and join the party. We are happy to support you and happy to learn from you…. not-for-profit or commercial we don’t care, build a better journal platform on PubSweet than the rest of us and we’d be happy! We are in it for the mission – come to talk to us! no preciousness here 🙂

Its all about workflow…

In the last 2 years, I have been very privileged to see inside the workflows of many publishers (book, journal, micropubs), aggregators, and preprint servers.

As a system designer, or one that facilitates others to design their own systems, this provides incredible insights into common struggles all of these organisations have. Recently this has given me pause to think about the system design process. So below is a little emergent theory of the importance of mental models in the process of workflow design….

Who better to ask about an organisation’s workflow than the publishing staff themselves?
There isn’t anyone better! Publishing staff are experts in what they do and they know every nook cranny of their workflow. They know the broken parts *intimately* since they live the pain on a day to day basis. They also know what works and what cannot be sacrificed.
Publishing staff are the best placed to know how to improve their workflow (if given the chance)
In order to be able to improve a workflow you need to first know it intimately. As above, there is no one better to do that than the publishing staff themselves. If you want to solve the problems, publishing staff need to be at the core of the design of solutions. By publishing staff I mean the actual people doing the publishing work.
Publishing staff don’t design their own solutions
This may seem like an oxymoron, but hear me out. In order to design an improved workflow you need a *perfect* mental model of the current workflow and what needs to be done. You need the first so you make sure you aren’t missing anything, you need the later so you know exactly what you can miss (what can be left out, what needs to stay)… Often I have seen one person given this job, and that person comes in from outside the org, or they are from within the organisation but have never done the publishing work directly. That leads to an imperfect mental model and pure speculative theory as to where and how the flow can be improved. It simply doesn’t work but it is the way it is done everywhere.

So… the heart of what I am trying to get at here are these three questions that lead on from the above which I am currently asking myself – why it is we don’t involve the publishing staff intimately in the design of better workflows? What is this mental model stuff and why is it so important? Why do I have this growing gut feeling that it’s all about workflow?

So… as a first, rather rough and quick response to myself on the above three issues…

why it is we don’t involve the publishing staff intimately in the design of better workflows?
Ok… I might actually sound rather nutty when I say this out loud. I believe the problem lies in the history of software development… First of all, these days we conflate workflow with software. They are not the same thing, but we make the mistake of equating the solution to workflow as software. This is far from the truth… but it is what it is…so, this is where the problem starts. To compound the problem, I’m going to go out on a limb and say that I think software dev today, including ‘Agile’ and Lean etc… repeat the same mistakes as older methodologies like, for example, ‘Waterfall’ and Xtreme Programming etc… That mistake is simple – the solutions providers are outsiders to the problem. I have at times, to be sure, been that person in the past. So I’m not pointing any sticks here…well at least, if I am its also at a past me. Waterfall treated the ‘users’ as research objects. That will only give you an outsider’s solution. Agile (etc) make me a little crazy because they try and cross that divide with tricks like ’empathy maps’ and creating avatars (which is, literally, a process of drawing pictures of the ‘users’ and guessing what they want). It’s no good. Why use avatars when you can actually just ask a real, living, experienced, member of the publishing staff? I have a lot more to say on this…but I’ll leave it here for now, suffice to say these problems situate the ‘end user’ at the end of the process ie. when the software is designed and / or delivered.
What is this mental model stuff and why is it so important?
This is a new insight for me. In a former life, I was a Book Sprint facilitator, and I own Book Sprints Ltd (though I have little to do with the operations). Recently the staff there have been pondering how to communicate the value of Book Sprints and interviewing people that love the process. One such interviewee hit so many deep points I was astounded (I’ll ask if I can share her name and credit her here). Her core point was this – Book Sprints, as a highly immersive collaborative model for book production where the experts (usually programmers) write the book together — [in this paarticular case we are talking about corporate documentation of highly complex software systems], gets to a shared mental model of the complex system exceedingly fast – less than 2-3 days. This is shared as a book so others can ‘get inside’ that mental model. If the same company hires an experienced tech writer to do the same thing, they will take at least 6 – 8 months to arrive at the same level of understanding. It will take them that long to develop the same mental model of the system. That’s the point. (Also, the collaborative effort will simply pull in more nuance, at a deeper level of understanding, that the lone contractor can, arguably, ever do).That is quite an insight, and when I heard it, I recognised the truth in it right away. The same is true for our workflow design processes. If you want the best workflow design you first need a perfect mental model of the problem. It is critical. So, get the staff involved, and if you want to save money and get the quickest design process possible, get them involved early.
Why do I have this growing gut feeling that it’s all about workflow?
Well… because publishing is not about technology, nor is it about formats, nor is it about a confused lexicon that we all think is stable (how many different types of Editor roles are there? what do you mean by peer review?). Publishing is about none of this. It is about how you get a document, how you improve it, and how you get it to others. That’s it. And that is all about workflow…

Pondering A Publishing Event

I used to do lots of events back in the day. The last big one I was really involved in was called net.congestion when I lived in Amsterdam around 2000… it was about the use of streaming media in art and activism. It actually turned about to be a huge, and timely, event.

So… now I’m pondering maybe putting something together around publishing…in the last year or so I have been fascinated by workflow in publishing. It seems to me to be a rich topic and pretty much publishing is all about workflow… if you would like to collaborate to make it happen let me know…its in the ponder stage at the moment…

Aperta Halted

A part of my personal history is a platform called Aperta which was previously called Tahi. It was a PLoS project and I was hired to design and build it. I quit when the PLoS Board decided to close the repositories, effectively making it a closed project. The repos remained closed, and as far as I know, are still closed today. Ironically after I left, they renamed the project ‘Aperta’ – Italian for ‘open’. A really silly marketing move to reassure everyone that despite what they may have heard, the project was still open…that was perhaps true, albeit (ironically and literally) in name only.

Now, it seems, the platform dev has been halted. Feels good to me. From what I heard (and I didn’t hear much), PLoS didn’t take the project in a good technical direction and generated a significant amount of bad faith and market confusion while trying to develop it behind closed doors.

To quote the new CEO Allison Mudditt (who I respect very much, Coko worked with Alison when she was at UCP):

Part of this initiative will involve changes around the workflow system – Aperta™ – we set out to develop several years ago with the goal to streamline manuscript submission and handling. At the time we began, there was very little available that would create the end-to-end workflow we envisioned as the key to opening research on multiple fronts. But the development process has proved more challenging than expected and as a result, we’ve made the difficult decision to halt development of Aperta. This will enable us to more sharply focus on internal processes that can have more immediate benefit for the communities we serve and the authors who choose to publish with us. The progress made with Aperta will not be wasted effort: we are currently exploring how to best leverage its unique strengths and capabilities to support core PLOS priorities like preprints and innovation in peer review. This will be part of our planning for 2018.

http://blogs.plos.org/plos/2017/12/ceo-letter-to-the-community-mudditt/?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+plos%2FBlog+%28Blogs+-+PLOS%29

I hope that PLoS releases the technologies that have been developed for Aperta (there was a lot more than just the submission system) into the open… with both open repositories and open licenses AND, more importantly, an open heart. Collaboration and openness is more to do with how people are than what open license they choose and several of the practices, including asking potential collaborators to sign Non-Disclosure Agreements (NDAs) before getting a demo of the system, were ridiculous and ungenerous.

Having said that, it would be awesome to see all that work released into the open, in open repos with open licenses, and no more blurring of the word ‘open’. Afterall the systems developed that included Tahi were all paid for by researchers. The PLoS Article Processing Charges fuels PLoS and they committed some of this revenue to the development of Tahi. When I was there, no external funding was secured for developing the system. Pedro Mendes made a good point in response to the announcement:

There is some merit to this, but I do applaud PLoS for being adventurous, and if it had worked then the result would have been APCs could be lowered, not just for PLoS, but for any Journal out there reliant on expensive and dysfunctional Manuscript Submission Systems. Allison also notes this in a discussion below the post mentioned above:

…the original idea was that Aperta would allow us to eliminate or speed up the slowest steps between a finished work and its publication in order to reduce the cost of our publishing services

That is true, and it was an admirable goal. However, whatever the journey was between then and now, the project should have always have been out in the open as a public asset. Open for science, open for access, open for source, open for all – and the fact researchers paid for it but it was turned into closed project mid-flight is reprehensible and in the end it worked against PLoS, in particular, it severely weakened PLoS claims to supporting all things open. What a mess.

But, it can’t be ignored that Tahi is about 5 years old now, which is old in software years. A entire generation of technologies that are better suited to solving these problems has arisen in that time. The system is now not much more than a still (just) relevant but outdated approach. That is the risk you take when you develop things behind closed doors. By the time you release it (or don’t in this case), it is out of date. That said, it would still be good to release it, but there are better technologies and approaches out there now.

So I look on with interest to see what will happen next. I sincerely hope PLoS can return to cutting a path through publishing and exploring and enabling a viable Open Access model that others can follow. With Allison at the helm I am betting things are going to take a much needed turn for the better, not just with this project, but on all counts.

As for me, I learned a lot from designing Aperta (I prefer to call it Tahi). The design process was an introduction to scientific journal publishing for me. I learned a great deal. Tahi gave me, at the time, an unencumbered dream time to imagine something new. It had a lot of interesting innovative approaches and if I had stayed with the project it would have ended up close to where PubSweet is now as I wanted to completely decouple the ‘spaces’ (a concept important to Tahi). It would not have been as good as PubSweet at doing this as a complete ‘decouple’ really has to be imagined from the start, and isn’t as clean if retrofitted. Still, the system would have been a lot more flexible and reusable.

But that wasn’t to be. Don’t get me wrong – I don’t think Tahi was the perfect platform, but it was a pretty good starting point with some significant innovations. At the time, I was looking forward to shaping Tahi with use and to mature it into an excellent system. The good news is, the next platform you design is always better. I took a lot of what I learned (I have now been involved in instigating around a dozen publishing systems) to my next development, and worked hard to re-conceptualise a new system that avoided some of the mistakes I made with Tahi, and took some of the good parts a whole lot further. That new project is PubSweet and it is looking awesome, and leverages modern technologies and approaches to the max – mainly thanks to the bunch of amazing folks working on it within the Coko team (particularly Jure Triglav) and also now, increasingly, from the collaborators we work with (at this stage mainly eLife, YLD, Hindawi and ThinSlices). Also a huge thanks to the Shuttleworth for backing me, especially because it was at a time (I had just quit PLoS) when it was very much needed. Their backing meant Coko was possible, and consequently, PubSweet and everything else we have done.

Anyways… it was past time PLoS moved on too from Aperta and congratulations to Allison for making the right call, especially given that it would have been a difficult one given the cultural forces at play inside of PLoS.

Getting Design and User Experience Right in Open Source

So, I’ve thought about Open Source and design… I’ve even written some articles on opensource.com (https://opensource.com/users/adam-hyde) about the subject, and created a methodology for bringing the use-case specialist (user) into the center of the process, along with designers and everyone else…

I’ve also brought this subject up a number of times in Shuttleworth Foundation meetings and received some invaluable advice and insights from fellow Fellows and Shuttleworth staff… many of whom have heard my whacky ideas several times over now and are still patiently listening and offering advice! Forever grateful to y’all… especially Helen, Sean, Arthur, and Andrew for ideas and feedback.

But what I didn’t expect, is that I’d be part of a wider community where these ideas could form the basis of the culture. This is what I saw happen this week in London as part of the PubSweet Global meet Coko hosted (& I facilitated).

There were about 25 of us coming together to discuss all things PubSweet with particular emphasis on building Journals. Present were many folks from eLife, Hindawi, Ubiquity, and Coko. We got to the topic of ‘Technical Council’ and I tabled the idea that we need some kind of process in place so that all stakeholders feel they are getting a say in, and are being heard, the future of PubSweet – since it is their technology too.

CoFounder of Coko, Kristen Ratan, kicking off the meeting

When I tabled the idea that we need some form of technical council, Catriona MacCallum, who I used to work with at PLOS, asked the very salient question – and what about the users?

I’m very grateful to Catriona for that question as it gave me the opportunity to open up the concept and talk about how there are very few communities in open source that treat software development as anything other than just a technical problem, and further that we should take this opportunity to experiment in making this community strong on solving ‘user needs’ and design… it was a great discussion and I’m also grateful to eLife’s head of product – Giuliano Maciocci – for having a strong voice in favor of this and really stepping into, what looks to be, an emergent leadership role with regard to design in the community.

As a result, we formed a Dev Council and a Design Council. These are oversight/communication groups of 5 people each. So, they don’t ‘govern’ but the choice by the community to form these two groups is a testament to how seriously the community is to making beautiful products that solve real problems in publishing for real people…

Whiteboard from the session showing our decisions. Dev/Design council structure at the bottom, also showing their relationship to community (supporting) and Coko (facilitated by).

All in all, a pretty fantastic 3 days.