Workflow Cost vs Pain

Today I talked with Lisa Gutermuth about workflow and software. We explored what avenues are available for finding the right software for your workflow. It is a common pastime and I suggested a simple taxonomy of solutions. It comes down to three simple categories:

  1. Just use anything –  a low cost, high pain strategy
  2. Find something useful – medium pain, medium cost
  3. Build a custom solution – high cost, low pain

Just use anything – this is where many organisations start. Essentially they grab ‘whatever is out there’ and cobble together a process to ‘make it work’. It might be that everyone has a word processor, for example, so they simply use spreadsheets and email them around. Or they may grab some wikis, use Google docs and sheets, rely on etherpad when needed (etc) and use these tools. This approach actually gets orgs quite far. The problem comes when your volume increases, or your operations diversify, or your staff increases etc. Over time, these types of tools can cause a lot of organisational pain and the inefficiencies created can force you to think about moving up the stack in the solutions taxonomy.

Find something useful – looking around your sector, seeing what others use and bringing these tools into your organisation, is often the next step. There are some good things and some bad things with this approach. Firstly, unless there is a startlingly obvious solution out there, you can spend a long time looking for the right tool. This can actually be harder than you think since software categories do not have a stable taxonomy. You can’t go anywhere, look up a table and understand what kind of software you need. So searching for the right software may take a long time.  Secondly, ‘off the shelf’ solutions will (most likely) only approximate your needs. That might be enough to get going. Bring these tools on board and start work. You might then, over time, need to hack it a little which might be cheap or it might be (if it is proprietary of if you get a bad vendor/developer), very very expensive. Or you could ‘Just grab anything’ and augment the tool with ‘whatever is out here’.

Sooner or later though, you are probably spending increasing amounts of money on the solution, and it doesn’t quite meet your needs so it is causing some amount of pain. So, while above options suggested this is a medium cost, medium pain approach, it can also turn out to be a high cost, high pain choice. I believe this is the position for many publishers today using expensive proprietary solutions that do not meet their needs.

The high pain – high cost effect takes place when the org ‘grows around’ the sore point (dysfunctional software). It is like the hiker learning to limp to cope with the pain of a stone in their shoe. Orgs will employ all sorts of tools to make up for the deficiencies and even employ staff to cope with the broken workflow. Best not to learn to limp as it can have long lasting organisational effects that are hard to dig out.

Build a custom solution –  the (seemingly) deluxe approach is to build the tool you need. This can be expensive if you take on all the costs yourself. The advantage is that you get what you need and if you do it well you build tools that help you improve your workflow into the future. Savings come in efficiencies and possibly, savings on staffing costs.

As you probably know, I am CoFounder of the Collaborative Knowledge Foundation. Our approach to the above is to design open source custom solutions for organisations but in such a way that they are easy to tweak and customise for similar orgs. Hence we are aiming to get the sectors we work with into the custom solution space and capture that elusive last category – low pain, low cost.

Down in Mississippi all I ever did was die

I’m a bit of an old school blues fan and 2 years ago I decided I would go check out the Mississippi. It was kind of an odd set up. I was in Columbia, Ohio for work and I figured…hell… this is about as close to the Mississippi as I ever got… so I should go check it out!

As it happened, that very weekend was a festival in honor of one of my blues heroes – Mississippi John Hurt. Amazing timing.

So, I hired a car and away I went. How far could it be? As it happens it could be over 800 miles. A drive that would also take me through Kentucky and Tennessee. This was all new territory for me and I was up for the adventure.

On the way I had some interesting stops. First stop was in Kentucky at a Saturday morning community fair. Awesome… I love fairs… cupcakes, maybe an espresso truck, second hand goodies…


…fluffy fairies in goldfish tanks…


…large men selling guns…


…hand guns…


…hand guns, rifles and gospel CDs…


It wasn’t the kind of country fair from back home where the most malicious offering is an old scrabble set with some of the pieces missing. Instead I was surrounded with firearms in great quantities, casually sold to whoever wanted them.

I felt out of my depth. So I headed south again and watched as Kentucky faded away in the rear view mirror. I was in a bit of a hurry. I knew I wouldn’t make it to Avalon, where the festival was, until the next day but I had to get somewhere to sleep. My choice was Tupelo – a famous place for me as my favorite John Lee Hooker song is about Tupelo

The first few lines being

Did ya read about the flood?
Happened long time ago
In Tupelo, Mississippi
There were thousands o’ lives

It rained, it rained
Both night and day
The poor people was worried
Didn’t have no place to go

The thing is… as I came up upon Tupelo it started to rain. I was still some miles from the city line and as I came up to the city boundary the rain got harder and harder. It wasn’t long before I couldn’t see more than a few metres, forcing me to slow the car to walking speed. It was the hardest rain I have ever experienced in my life. I began to have the feeling that this was something more than just another road trip…

I got to Tupelo and stayed the night at a crappy soulless hotel. Getting up early I discovered that Tupelo is actually very famous as it was where Elvis was born. I stopped in to see the humble house, worked out which store sold him his first guitar and then headed out towards Avalon.

As I drove, the roads were long and narrow. The towns small. I saw unhappy posters taped to power poles calling for information about a missing young local woman. A short stop for gas allowed me to overhear the attendants agreeing that gas should be free for anyone in (army) uniform. I passed farms with large homesteads that I imagined were once plantations of old… reality was starting to agree with my imagination. I drove onward…

Avalon is famous in the blues world. It is where MJH grew up, and Avalon Blues is one of Mississippi John Hurt’s most well-known songs and the title of his first album. Avalon is also where he was rediscovered many years later when blues fan Tom Hoskins went on a legendary journey to look for him, after refusing to believe, as most did, that he had been dead for many years.

The thing about Avalon is, it doesn’t exist. At least, it doesn’t exist now. The spot where I thought I would find a small town and a festival was a wasteland of empty shacks and potholes. It was dusty, weird, and full of ghosts. Further, it had no connectivity so finding my way to the festival was going to be tricky.

I drove around a bit. I went down a long road which came to a dead end with a sign saying the road was closed. I turned back but a car, the only one I had seen for a while, passed me and continued up past the sign. I followed them but lost them. The road turned bumpy. Somewhere it became heavy duty road works, heaped dirt and the impressions of giant graders.


Everything looked abandoned.


I drove on, turned down a narrow road that got narrower. The trees seemed to hang closer to the road and slowly obscure more and more of the sky… I passed a home with abandoned cars in the front and a family sitting outside staring at me as I drove slowly past.

I finally turned a corner and came into a clearing. There was a brick house on a small open lawn. Two cars were parked by the house but I couldn’t see anyone. I parked up and walked over to the house. Behind it I found half a dozen friendly faces looking at me…. one of them walked up to me. She looked weirdly like Mississippi John Hurt. It was Mary Hurt, his granddaughter. She had a large smile and shook my hand. It was like history just reached out and grabbed me.

We sat around and talked and played blues on the lawn. There is even a video of me hanging with the gang, playing guitar. I didn’t bring my guitar so Mary gave me one to play given to her by John Sebastian from the Loving Spoonful. I felt overwhelmed.

We played a bit and then Mary took us down to visit her grandfather’s grave. We drove a bit then got out in a deeply wooded forest. On one side of the road was a reasonably maintained cemetery. It was where the white people were buried. On the other side of the road, throughout the forest and in unmarked graves was where the black people were buried. Mary reminded us to be careful where we stepped…

We came to a small clearing.


It was the grave of Mississippi John Hurt. So simple and alone in this beautiful forest.


Mary told us stories about her grandfather. About Mississippi itself. How she hated it for how hard it is. How mean it has been. I wondered how I could think of the blues so romantically until now. How I hadn’t understood the sadness. I wondered about the history of rock and roll. How one of the giants was right here in front of me. How humble the scene was, how humbling it was.


We stood around and listened. We played some of his songs.


It was a very moving experience and I emerged from the forest with some new friends.


Oh manno

So, I realised I’m getting a little sick of talking about publishing. I love it so, sort of. But I never thought I would ever be ‘in publishing’, I kinda just fell into it. Or maybe more accurately, I fell, then woke up, and slowly came to realise I was in publishing.

But actually I’m not in publishing. I’m in a fascinating world and I kinda want to start talking about some of that. Isn’t that what blogging is meant to be about anyho? So…First up, baths. Yes…one of my favorite things. Infact I just built a bath platform…oh…maybe I’m grabbing too much credit…I didn’t actually grab the hammer and wood and stuff…hohoho….anyways…it looks like this:


It is at my cottage in NZ. I will be going there in December and can’t wait! My bath will fit on the platform and I can stare at the stars and the sky. Its going to be awesome.

The view from the bath should be pretty good…looks something like this:


…don’t expect me to hurry back 😉

Typescript and publishing systems

Wendell Piez and I just co-wrote a post for the Coko website about the quandary of going from MS Word to HTML:

A Typescript for the Web

The point being that publishers take badly structured Word documents and process them. Adding structure etc and then ‘throwing them over the wall’ to outsourced vendors to convert into other formats. When publishers add structure to documents, they often do this with MS Word and custom-built extensions. They simply click on part of the text, choose the right semantic tag, move to the next. Just imagine… how many publishers have built these custom macros (it is very common) and also imagine that each publisher must tweak the macro code with new releases of MS Word. Tricky and expensive!

So, the point is, why not do that in the browser using web-based editors? It not only brings the content into an environment that enables new efficiencies in workflow but it also means publishers don’t have to keep upgrading these macros all the time. Further, if the tools for doing this in the browser are Open Source…well… you get the picture – share the burden, share the love.

So the article is a small semantic manoeuvre to get the conversation away from the rather opinionated, but dominant position, that MS Word-to-HTML conversion is terrible because you can’t infer structure during the conversion process… The implication is that HTML isn’t ‘good enough’. Our point is, you don’t need to infer the structure because it wasn’t there in the first place. Plus, HTML is an excellent format for progressively adding structure since it is very forgiving – you can have as much, or as little, structure as you like with HTML. Hence we can look to shared efforts to build shared browser-based tools for processing documents rather than creating and maintaining one off macros.

Remix and Reshuffle Revisited

FLOSS Manuals Remix
FLOSS Manuals Remix

A few years ago, I wrote a brief post on Remix vs Shuffle. At the time, the Open Educational Resources (OER) movement was struggling to work out how existing teaching materials could be remixed and reused. No one had really cracked it. At the same time, we built remix into FLOSS Manuals. The primary use case was for workshop leaders to be able to compile their own workshop manual from existing resources. We had a large enough repository of works, so it was a question of how we went about enabling remix.

Recently, I have been in two separate conversations about remix (after not having thought or talked about it for some time). One conversation was in the context of OER, the other in the context of remixing many journal articles into a Collection. So, I’m revisiting some earlier thoughts on the topic and updating etc…

At the time, we expected the FLOSS Manuals remix to be used a lot. I was a workshop leader myself and thought I could benefit from the feature. However, remixing wasn’t used very much by me (I did get some very useful manuals from using it, but didn’t use it often) or anyone else. Hence I wrote the reflection on remix (linked above). My position is outlined by the following quotes from that article:

I have come to the understanding the ‘remix’ as such has only a limited use when it comes to constructing books from multiple sources.

And the following, where I liken book remix to remixing of music to illustrate the shortcomings:

Text requires the same kind of shaping. If you take a chapter from one book and then put it next to another chapter from another book, you do not have a book – you have two adjacent chapters. You need to work to make them fit together. Working material like this is not just a matter of cross-fading from one to the other by smoothing out the requisite intros and outros (although this makes a big difference in itself), but there are other aspects to consider – tone, tempo, texture, language used, point of view, voice etc as well as some more mundane mechanical issues. What, for example, do you do with a chapter that makes reference to other chapters in the book it originated from? You need to change these references and other mechanics as well as take care of the more tonal components of the text.

I think these are valid points, but I think, revising this, there is one nuance I would like to add. Sometimes ‘shuffling’ is adequate where you are compiling an anthology which is, as it happens, the case when you are putting multiple journal articles into a collection. Building tools to enable this kind of ‘reshuffle’ is very useful but still I would question the usefulness in certain contexts. It is a use case that, from my experience, would be great as a tool used by, for example, a publisher or curator. I’m not sure of its usefulness in a more generic ‘user space’. Journal publishers do, in fact, make collections where several articles are compiled together to form one ‘bound’ work (often a PDF). In this space, such a tool could make life much easier. Whether members of the research community, for example, would want, need, or use such a tool is still an open question to me.

For information on how FLOSS Manuals Remix worked see here:

It is still working here:

Here is a video (ogg vorbis) demo of it in action, with the resulting PDF linked below.

my_pdf (note, the colored text is because, as shown in the demo above, I edited the styles via CSS to make the body text red).

video made with recordmydesktop.