A good friend, Enric Senabre, together with Ricard Espelt (who I don’t know) wrote and published an interesting article on designing for platform cooperativism. They set out to define “platform cooperativism UX”, which seems to be to be a very concrete task on one level (UX is nothing if not concrete) for a general state, approach, or category of ‘platform’.
I’m not going to go into detail here about Platform Cooperativism, because I don’t really get it. I do know Trebor Sholz and figure whatever he does is probably right and makes sense. So I’m buying the book to find out more.
Enric and Ricard are approaching software design by intersecting strategies to overcome technically disenfranchising stakeholders while ‘learning as you go’. These are laudable aims, especially in the NFP sector where there is a great need to develop solutions that actually solve problems. As I read somewhere recently, the technology community has no shortage of solutions, what they are in need of are solutions that solve problems. Zara Rahman has also pointed this out recently and is conducting research into just this area.
The issue here is that often technologists see all problems through the eyes of code. Further, they are prone to see the intended beneficiaries of their work as avatars. There are, in fact, many strategies to turn real people with real needs into avatars.
If you try to solve real world problems with code, and your participants are avatars, you are really setting yourself up to be a great game developer. You are possibly not in a good position to solve real problems.
So, Enric and Ricard are starting off with the right premise and in the article they document their experiment, exploring fundamentals to come up with a new facilitation methodology for this context.
we began with a reflection on which specific functionalities and features (other than those available on existing online platforms and social web interfaces in general), if any, could be explored
They seem to have given themselves an extremely difficult task – designing for an open-ended ‘imaginative’ state. Although they couch this as ‘specific functionalities,’ I take it they are trying to define specific functionalities for a generalized approach to platforms. That is tricky. I admire that they took this on. Innovating with design methodologies takes some gusto, and it is a vital process for defining and refining tools for a new method. However, in my experience, open-ended problems like this seldom lead anywhere useful. You need to start smaller, with real, concrete problems. These might add up to, and constitute, larger issues, but the road to those issues is from the bottom, not the top.
It seems that the result was still productive, but perhaps not in the way expected. They appear to have elevated the group’s awareness to issues of trust in the social web. A hot topic at the moment. As such, the process is successful as a barometer of the times, identifying issues that concern people here and now. Enric and Ricard appear to have understood this too and continued on to refine this starting point, moving it towards actual UX design with the well-known method of user stories.
However, user stories are best deployed as a function of software design and I don’t think their process was there yet. User stories require a concrete problem. They are intended to drive people toward designing a concrete solution. Bringing this framework to a general question of reputation is confusing methods and will cause cognitive drag and a mixed understanding of intended results. It would be better to keep this part of the process outside of software design paradigms, and instead, employ general ‘sense making’ methods.
It seems that Enric and Ricard diverted from the goal to produce concrete UX and ended up driving towards requirements. I would say this is a better direction. However, requirements-gathering for a general issue is not well placed to lead to much of use to software development other than a ‘general direction’ – which is what they seem to have achieved but not what they set out to achieve.
As such, I think the results of Enric and Ricard’s experiment are interesting, but the results are not interesting in the way they outline. In the summary they state:
The next steps in addressing “platform cooperativism UX” should continue along these lines: new user stories that generate both potential platform coop requirements and design-driven research outputs.
This overstates the value of their findings as generated by the participants. The real value of this session is that they tried to assemble a methodology for an ambitious context – in essence, they are actually trying to help the ‘platform cooperative’ community to understand itself, to understand the implications of their philosophy. I think that is really interesting and admirable. What they need to do, however, is not to override this aim with the pretense of generating actual user stories, software requirement, and UX for platforms. They need to name and design a method that starts in another place – a place where the articulation of values is the outcome, not the construction of code.
Booktype is still going very well and has also spawned the very interesting Omnibook service. Due to the recent interest in this project, I revisited this old video which documents some of the exploratory thinking I had when leading the Booktype team at Sourcefabric. It was recorded May 2012 at #dev8ed in Birmingham, UK. At the time I was leading a small team, having just migrated Booki (FLOSS Manuals) to Booktype (at Sourcefabric).
I found the video really interesting as it covers my thinking at the time, (developed over many years of experimenting in this area) over many issues, including rendering books in the browser and using the browser as a design environment for books. There are some nice quotes which accurately reflect how I was thinking then which are interesting:
“there is no one taking responsibility for designing environments where you can target both flowable text as an output like Kindle or EPUBS, and at the same time, target fixed page outputs like paper books. So we are trying to work this out at the moment. How do you deal with this? .[…] We are trying to work out how can you possibly find a paradigm that fits both flow-based, and fixed page, design” [36min 25s]
and
“what we want to see [in the browser] is when you are outputting to book-formatted PDF, we want to see like you see in Google Docs – exactly the page dimensions that you are going to get when you output the PDF. Google Docs does some sort of magic where that is possible, we haven’t yet cracked it ourselves, but for fixed page design we think it is quite important that what you see in the HTML page is what you would eventually get in the PDF. [41min 37s]
…
“…how do you actually render one to one representation of a book-formatted PDF in a browser?” [49min 49s]
…
“…we can have JavaScript playing a role in rendering elements of pages for book-formatted PDF.” [16min 58s]
…
“…we take the Booktype content as HTML, HTML as the base format, and Objavi formats that into one long HTML page for which we have specific CSS rules to structure the book in a specific way. Then we run WKHTML over the top of it, and a number of other tools, and we assemble a book out of it, book-formatted PDF” [18min 38s]
…
“Thats because WKHTMLTOPDF is webkit, the browsing engine behind Chrome and Safari, … so you can use CSS, and JavaScript and everything from webkit, and turn it into a PDF” [19min 50s]
…
“…the advantage of using webkit as part of the rendering environment, as webkit is a browser, [is that] if you design in the browser you have a one to one co-relation between content creation environment and output environment” [33 min 49sec]
To be clear, we were already using browser engines to make books for quite some time, and Douglas Bagnall, a friend who also worked with me at FLOSS Manuals, even investigated collaborating with the Gecko (Mozilla layout engine) developers to add widows and orphans controls and the CSS page-break control (which we needed for books), in 2010 or so. Actually, it was pretty cool because Douglas, myself and Robert O’Callahan (Mozilla layout engine dev) were all New Zealanders. But FLOSS Manuals had been making books for many years with browser engines since Behdad Esfahbod advised me to explore this, many years earlier. We knew browsers could be used for producing book-formatted PDF and we had been doing it for years.
However, as I have learned over the years, there is an important role for vision, experimentation, and theoretical exploration prior to developing good software. Hence, I was now exploring how you could take these positions further to design books in the browser client. Rendering PDF was one part of the story, the other was working out the tools to take book design to the browser. This was what Adobe was also after, I believe, when they implemented CSS Regions in webkit and started on their Adobe Edge Reflow line of products that leveraged the browser as a ‘design surface’. They were interesting times.
While BookJS didn’t quite get to be the design environment I was (and still am) after, it was still a good tool. In an attempt to get to a design and rendering solution in the browser, we later took the Booktype Designer (demonstrated in the video) ideas to a JavaScript prototype called StyleJS for integrating with BookJS but, unfortunately, it didn’t make it to production. StyleJS enabled a kind of ‘WYSIWYG’ tool for styling a page live. Which is an interesting prototype for future in browser book production exploration.
Work continued on BookJS and it has had a useful life despite some quirky turns in the road. During this time, the Booktype team worked with several people on the development of BookJS and received good advice and contributions from Mihai Balan (from the Adobe CSS Regions team), Phil Schatz (from Connexions), Maria Fraser (University College London) and others. As with many software projects, contributions like this deserve a lot of credit, as I have written elsewhere, since these contributions are not always preserved in the code.
In the Booktype world, Juan Gutierrez (who worked on BookJS at Sourcefabric, and now works with me at Coko) extended BookJS to support the CSS Regions polyfil. It is still in use now with Book Sprints for rendering books. Consequently, we are still very grateful that Booktype and Sourcefabric kept the BookJS product AGPL after I left the project so we could extend it. Hurray for Open Source!
It is good to see Booktype going strong, Sourcefabric still invested in Open Source, and a growing interest around Omnibook. I know the team there, Micz Flor (co-founder of Sourcefabric and Managing Director of Booktype) being an old friend, and Julian Sorge also makes a great Booktype Managing Director. They have brought their own vision to the Booktype products, pushing them in new directions, and it is really great to see. I’m hoping they will continue to go from strength to strength.
In summary, these were interesting, productive times. Sourcefabric provided the opportunity for Booktype to grow, and I experimented a lot, as I had done at FLOSS Manuals (and continue to do now), with new ideas and approaches. There was some great software, books, and ideas that came out of that period. Some of the books we made I have even kept with me through my travels. In the video, for example, I demonstrate the Booktype Designer. We built the Designer before and during the Sandberg Institute workshop I led in Amsterdam and used it in the same month as I did the presentation to create this wonderful artist’s book. I carried it with me all over the world and still have it on my bookshelf now!
A few weeks ago Dave Cramer and I started a new website – Paged Media. The website’s purpose is to promote the use of HTML, CSS and JS to make books, whether the books are displayed in the browser, or in e-readers, on mobile devices, or in print. The site is coming along nicely with blog posts and links to valuable resources to do with the production of books for reading or display on screen. Soon we will introduce podcasts to the site, some weekly how-to posts, and items about the future and past of this very important approach to making books.
While copying over my archives from everywhere, Raewyn Whyte, who is helping me with the copying and cleanup, carried over all the posts from my 2006-07 summer trip to Antarctica. You can find them here:
The original site doesn’t exist anymore so we had to fish them out of the good ole Wayback Machine.
It was an amazing journey to Antarctica. I have some definite highlights and lowlights. Lows included the death of one of our crew on the first day at the base due to very unfortunate circumstances. That was unbelievably horrible. Also, I think I felt quite compromised being in Antarctica in the first place. I almost didn’t go but didn’t want to let down the buddies I had made a commitment to.
I felt uncomfortable because the science bases on Antarctica are strategic placements… waiting, waiting, until the pressure gets too much and the moratorium on extractive industries comes off… I can’t speak for the entire continent but my direct experience was that there was very little real science going on, and a lot of resources were being spent in the name of science while maintaining strategic positions for bases.
The highs were that it was an amazing experience and I felt privileged to be there. I met great people, had good times, even beat the chopper captain in the New Year’s darts finals! We did some amazing stuff too. Building antenna, setting up a radio station and a very low-fi radio talkback system, among other things.
But mostly it was the really unbelievable beauty of the place.
Our mission was to start the build of an autonomous mobile base for art-science collaborations. We did the first phase, others came later. I don’t know if it was ever completed, I lost track of it.
Anyway.. .it was a great experience. I visited the SA Agulhas (the icebreaker we travelled down in) a few years ago at the Capetown wharf. Beautiful boat.
The new Collaborative Knowledge Foundation website is now live. It was designed by the fabulous Dutch design company the van Leeuwen Brothers. I have known designer Henrik van Leeuwen for many years. He’s a good friend, talented designer, and great artist. Henrik has done all the illustrations for Book Sprint books for many years now and does almost all the Book Sprint cover designs. He developed the Book Sprint and Coko logos and now the Coko website. Henrik and his brother Kresten are huge talents and if you are looking around for good designers I highly recommend them.
Editoria is a software platform Coko has been working on in collaboration with the University of California Press and the California Digital Library. It is the first product that we have taken through the Collaborative Product Development process and it is looking pretty good!
Editoria is built on top of PubSweet, which means there are separate frontend components integrated with the PubSweet core application. These frontend components are available as open source (MIT) in the Coko gitlab repositores. They have not yet been nicely bundled for distribution but this will be available soon as we have done with many other PubSweet components. You can see these if you search the NPM repository online:
PubSweet components on NPM
The frontend components enable 3 workspaces, a book dashboard, the book builder, and an editor. This follows the general pattern of book production platforms I have been involved in since FLOSS Manuals in 2006 or so. This kind of design places the structure of the book itself as a central workspace which gives not only an overview of the structure of the book but of all operations concerning the book. I think this is a good general strategy as I have previously written about here and others, including Micz Flor of Booktype, have also written about.
In Editoria, this central workspace is known as the Book Builder, and the following is a screenshot as of Nov 19, 2016.
Editoria Book Builder
It contains a lot of punch but is pretty simple to understand which is the ideal since you really want to get an understanding of the status of a book at a glance.
Before looking at this in detail, first a little disambiguation – we are following the Chicago Manual of Style for the naming of items. In this case, each of the front matter, body and back matter sections are known as divisions. Items within each of these, be they a table of contents, preface, glossary, appendix or chapter (etc) are known as components. A part is a collection of chapters (technically a part is also a type of component).
In the Book Builder we have the following:
Front matter, body, and back matter divisions
A list all book components (chapters/parts)
Status (4 status markers for each component, each with 3 states) – from the Book Builder you can set a status for each component. These are currently hard-coded (but may become configurable at a later date) to the following 4 items with 3 states each: To Style, Styling, Styled To Edit, Editing, Edited To Review, Reviewing, Reviewed To Clean, Cleaning, Cleaned Clicking on a status item moves it to the next state. These states are both indicators of the status of a component (what needs to be done, is being done, or is done) and they are connected to access permissions. When, for example, a chapter is marked To Review, in this case those that are Authors (see Team Manager below) can access the chapter in edit mode and review changes and make alterations as requested by the Copy Editor.
Upload word (converts to HTML, yet to be wired in) – we are working on the MS Word-to-HTML converter which will shortly be integrated. This will enable the upload of MS Word files into the system which is critical since all books come to UCP and CDL from the author(s) to Acquisitions to Production in MS Word format. It is the Production department that uses Editoria.
Tools to add, rename, and delete components – it is possible to add new components dynamically, rename them and delete them. It is also possible to drag and drop these components to reorder them.
Pagination markers – these indicate whether the component should be left or right breaking when paginated for paper book production. A click on the left or right pagination boxes changes the state. This state will later be important for exporting to PDF via the open source Vivliostyle HTML-to- PDF renderer.
In addition to the above, there is a team manager interface available from the Book Builder:
Editoria Team Manager
Through this interface, you can add team members to the book. This affects access permissions. The team manager, in this case, has three roles – Production Editors, Authors, and Copy Editors. These are all in themselves configurable within the admin side of the system so it is possible to have different types of roles for different organisations (in my experience role names and duties differ quite a bit between publishers).
In addition, you can press edit next to any of the components and it takes you through to the editor for that component:
Editoria Editor
This editor has been built with the open source Substance libraries. To build this, we have first conducted an audit of the element types CDL and UCP require within a book. By this I mean we looked at the types of content UCP and CDL already has within chapters (components). For example, they require (as most books do) headings of various levels – Heading 1, Heading 2, Heading 3 etc. In addition, custom elements are required like an Extract, Block Quote or Dialogue etc. Our mission is to capture all these different content types and custom build these into the editor so that you can highlight a part of the text and apply the correct element type. This is extremely important when it comes to outputting the book since we need to know how to style each of these elements to the chosen design.
We also have all the regular features in the editor such as the ability to edit the document, add and remove bold, italics etc. We also have a number of additional features that were designed by the UCP Production Staff during our Collaborative Design Sessions. These features include:
Notes management – it is possible to add, edit, and remove notes. These notes correspond to end, book, or foot notes in books. In the editor, they are displayed at the bottom of the page while in the output they may appear at the end of the page, chapter, or book so we refer to them more generally as just notes in this environment. You can add notes and edit them through an overlay so that this can be done in situ (without having to scroll to the bottom of the page).
Comments – sometimes referred to as annotations. It is possible to select a portion of the text and add a comment. These appear in the margin. It is then possible to reply to, and resolve, these comments. This is intended to be used by Copy Editors and Authors for discussing and resolving issues in the text. For those that are interested, it took about 3 weeks to develop the annotation feature which is pretty quick.
Chapter Structure – the structure of the chapter is currently displayed at the far right of the page. This lists all headings in a nested fashion. Clicking on any heading will scroll the page to that position. It is intended to give a quick overview of the structure of a chapter and to enable fast navigation. It can also be used to check the headings follow the require d nesting conventions.
We are building in a few additional features in the editor before we trial books through the system. These are:
Images – the ability to add and remove images in the book
Spell check
Track changes – the ability to turn on and off a ‘track changes’ type function
We are also integrating the MS Word-to-HTML conversion and the PDF rendering features. MS Word-to-HTML conversion is done through a set of conversion scripts we are writing called XSweet. These convert MS Word to HTML in a series of steps. We have deliberately designed it this way so that we can customise various stages of the conversion for different publishers since content types vary enormously between different organisations (or different departments within the same publisher). The conversion will be executed using an additional software we are building called INK. More about INK can be found on the Coko website.
The HTML-to-PDF conversion will be done using Vivliostyle. This is a great open source software that uses the browser to paginate HTML into a book form that can be output from the browser to PDF. It requires the development of custom CSS for styling the outputs and this takes a little time but so far the results look good.
In time we will layer on more functionality but, for now, this is the overview of the system. The aim is to get it functional for production tests within a couple more months, then trial it, fix issues, and start adding on more functionality. It has been an interesting road so far. Working with the Coko team and the UCP and CDL teams has been a real pleasure and I think we are building an amazing open source system for the production and maintenance of scholarly monographs.
To watch the ongoing development of the system please see the Editoria website. Also, consider reaching out to UCP/CDL via the Editoria mailing list of you are interested in learning more.
I have been poking around in my own archives recently. It’s pretty cool to remember things that I had completely forgotten. Anyway, the following is a post I found about the first Book Sprint. It was about Pure Data and held on the little island of Korčula, Croatia. A buddy, Darko Fritz, let us use his apartment in the center of the old town. I was to join two friends, Derek Holzer and Luka Prinčič, at the apartment for 3-ish days. However, on the way I had a health freak out and had to participate remotely. It was our first attempt at a Book Sprint and looks almost nothing like how Book Sprints are done now, but it was a starting point and gave me a lot to think about.
The following is the report written by Luka for the FLOSS Manuals site. I love the two images in the post. They are the only images Luka and Derek sent me of this mysterious event.
So, Adam sent us (he was supposed to come too, but couldn't at the end) - he sent Derek Holzer and myself to this Croatian island Korcula, into the little (but very very cute) town named the same (Korcula) so that we Sprint-it up and write a Pure Data manual for FLOSS Manuals. I took my little old red car and drove down from Ljubljana over Rijeka and under scary stoney Velebit entering the warm sunny region of Dalmatia and driving further towards Split. Now this was a fun ride, especially under Velebit was very scenic. My cup of tea, as I hate the boredom of motorways. And my clutch was going to hell too, so my fingers were crossed. In split I caught this superfast catamaran-pedestrian-ferry that took me in under three hours to Korcula where Derek was waiting for me. And for phone line. And for internet connection.
In this lovely little town with incredible feel of Venetian architecture, tiny little streets, all set in stone, we waited for ADSLine to appear, which it did right next morning. The evening before we discussed over a plate of Italian pasta and a beer what would be good to work on in the coming days. After the internet magically appeared (yeah right!) we could also discuss with Adam, who was in Amsterdam, over Skype how to work and what to work on. Derek was somewhat the main captain although each of us worked on our own chapter. Adam was going after installation instructions for various platforms, Derek was rewriting and expanding tutorial on how to build a simple synth, and I was going deep into 'Dataflow' tutorials.
On Monday when I arrived, the sea was quite wild as there was a lot of wind. Tuesday was all gray and rainy. We had some talk about doing some hiking, as Derek had already explored some terrain, but Tuesday was a bit too wet, so we stayed in and worked on bits and pieces trying to see how much work there is and how writing feels. It seems from the after-perspective that this was the day to slowly get the grip on how the writing will go.
On Wednesday, we took a bus in the morning to a nearby village by the coast and with a little map from the tourist office tried to find that little path that would correspond to a tiny dashed black line that goes up the hill towards another village about 500m higher. We searched for more than an hour until finally a communicative small but wide man told us that old paths were overridden with a new tarmacced road. "I used to go up those little paths, when I was young..." he said, "but now there's a road so why would one bother, but if you really want there's a little path there, you can find it...." We walked a path for a while and than decended slightly to find a road and walked the road for an hour, maybe two, under not-too-hot spring-ish sun. It was lovely. After icecream in a local shop (those local shops are always something!) we caught a bus and went back 'home'. We started to work immediately, had some lunch and worked quite late into the night. We talked to Adam over Skype and kept working each on our own part.
Interestingly, and maybe because we were supposed to leave on Saturday morning (Derek had a plane ticket), next two days somewhat turned into a bit of frenzied work. As we knew we had just two days left, we tried to finish our parts. In consequence, at the end of Friday, we realized we hadn't gone out much. A real Sprint! One of us would go to the shop now and then, couple of times a day, and cook something (mostly pasta! - but I found some local home-made from Istria, and also Korcula olive oil!) The weather was nice, not cold, with many bits of sun. The voices of kids, sometimes horny cats, and other inhabitants were protruding through an open door and were making a special kind of atmosphere. It was lovely to feel this Venetian-medieval town around you. Our flat was on the third, top floor, we could see the sea from it and the roofs around us.
On the friday evening we found ourselves with two huge finished chapters, both quite satisfied with it. We were tired but happy we made something. We searched the town for fish and found an empty but slick-looking restaurant with a young owner-cook who made us some fish and vegetables. I guess we could have made them ourselves at home, but none of us spotted a fishery and had much time and energy left. I must say it has also been very educational for me to listen to Derek's peculiar playlist which included a lot of metal, old rock (Black Sabbath, etc) and African (Angolan) music from 70s. Luckily, Derek had a near perfect (or at least compatible) feel for sound levels, so it was always very enjoyable to become aquanted with a genre I haven't explored yet in my life much.
In the aftermath of the Sprint of two-and-a-half days, we both felt we wouldn't be able to go like this for another day. If anything, we would need a day, or ideally two days of real break from this and than we could come back to the book with passion. I was thinking this would be great to do three or four times. In waves. Either that, or steadily work for 8, maybe 10 hours per day and then have a break. But I think this also depends on the writer and her/his ability and fitness to write.
At the last moment I decided to stay for another two days (as our 'landlord' told us we could stay for us long as we liked, and Adam agreed). So Derek left on Saturday morning (6 am, ugh) and I worked on some of my music and had couple of walks around town and surroundings, really enjoying it immensely.
My car was still where I left it in Split, and the clutch was still functioning (barely), so I drove back, under lovely Velebit of course. It was almost like summer all the way to the border with Slovenia, where heavy rain was waiting for me.
Korčula image by PJL (Own work) CC-BY-SA-3.0 (http://creativecommons.org/licenses/by-sa/3.0/)
Many years ago I founded the Book Sprint methodology. It was a fascinating path from idea to method which I documented here.
Since 3 years, Books Sprints Ltd exists and we have a full team of nine amazing people working on Book Sprints. I held the title of CEO until recently, although it was a name-only title really. However, as of last week, I have completely stepped away and appointed Barbara Rühling as the CEO and Katerina Michailidi as the COO.
I have known both Barbara and Kat for a long time. Barbara has been the lead facilitator for the last 2 years and she is an amazing facilitator. Kat I have known since I moved to Berlin many years ago. Kat has been a very trustworthy and hardworking manager of Book Sprints for the last 2 years.
They are an amazing team and have a lot of Book Sprints experience between them. I am expecting we will see Book Sprints continue to go from strength to strength, continuing to do the good work it currently does while also exploring new ideas and opportunities. I’m very much looking forward to finding out what this next phase of the Book Sprints journey entails!
(I will remain on the board as the sole director and will be advising when needed).
Wow…I was browsing some old archives to update this new version of my site. I found the most incredible stuff in the Internet Archives Wayback Machine including the outline of a description of Booki (2010) many years before it became Booktype. Amazing! I didn’t think I had the product manager in me but it seems once upon a time I was really focused on this kind of acute detail for product management. I had forgotten!
I’m so astonished how much of my thinking recorded in this email carries through to the way we are approaching product development for Coko now. The statement:
You might have noticed that I prefer to take the easy road for features, leaving as much open as possible, and then refine according to use. That is because,from experience, I have learned that when designing software it is better to be led by the user rather than force them into an imagined work flow.
Might as well be out of the Collaborative Product Design manifesto.
I’ts kind of incredible. The email documents so much of how we were thinking at the time, including using HTML and CSS to create paginated books using browser engines:
* Objavi utilises Webkit for PDF generation. Later Gecko will be added.
…and later in the product description…
3.2.2 CSS Book Design
Status: High Priority, Implemented
Function: The default PDF rendering engine for Booki is now Webkit and will eventually be Mozilla Firefox hence there is full CSS support for creating book formatted PDF in Booki. This changes the language of design from Indesign to CSS - which means any web native can control the design of the book.
Pretty interesting, if only to me! Anyway, the email is below, it documents some features we built on commission for Source Fabric before they eventually took over the project. Thank you for indulging me 🙂
From adam at flossmanuals.net Wed Jul 28 09:11:21 2010
From: adam at flossmanuals.net (adam hyde)
Date: Wed, 28 Jul 2010 18:11:21 +0200
Subject: [Booki-dev] notes to meeting
Message-ID: <1280333481.1582.143.camel@esetera>
hi Frank,
It was good to meet you and I'm glad Source Fabric is considering working with us and you to develop features they and we need (Aco is also keen for this).
I have sent this email to the dev list and to you and Micz. It might be good for you both to consider joining the list.
http://lists.flossmanuals.net/listinfo.cgi/booki-dev-flossmanuals.net
Below the content of this email is a very basic requirements doc. It does not outline the notes tab, so I thought I would make some notes here for your (and Micz's) consideration should Source Fabric decide they wish to commission all or part of this development.
In essence, I think that the notes tab could nest the following:
1. To do list
2. Book notes
3. Style guide
These could be hidden via a dropdown or accordian style interface. Our plan is to keep everything as simple as possible so I would imagine a page with three headings and clicking on each reveals the information behind it.
Some ideas:
1. To do list
The basic form could be a Jquery to do as we looked at today:
http://demo.tutorialzine.com/2010/03/ajax-todo-list-jquery-php-mysql-css/demo.php
If this is the format, it would be good enough as it is. The good news is that this is done using Jquery so I imagine this is a very easy implementation. What you would need to work out, however, is how Aco implements the dynamic updates so that when a to do is altered everyone has that info updated.
If there was room to take this development a step further, I would recommend considering adding the following fields:
* assigned to
* due date
* priority
I am not married to those ideas though as I think we need to insure that the interface does not have too many things going on. So I would actually recommend we start with the basic implementation and move on. When users have tried it then we can consider extending it with these items.
2. Book Notes
Something like etherpad would be good but too complex (see.
http://piratepad.net/ )
I would suggest considering either a) the same interface as we have now in the notes pad except with a very very simple WYSIWYG or b) a threaded comment system. I think the best would again be to do the easiest and simplest - what we have now with a WYSIWYG interface (and no need to press 'save'). Then when users use it we extend according to demand for most-needed features.
3. Style Guide
This is pretty much the same as (2) except it would be used for storing the Style Guide. A style guide is optional but many people request it in FLOSS Manuals and some go out of their way to create one so I think this would be a very good feature to anticipate based on our user experience so far.
I think all of the 3 above are simple and I think Source Fabric's working process (especially for the forthcoming Sprints) would benefit a lot from them.
You might have noticed that I prefer to take the easy road for features, leaving as much open as possible, and then refine according to use. That is because from experience I have learned that when designing software it is better to be led by the user rather than force them into an imagined workflow.
It has worked well for us so far - everything you now see in Booki is pretty much that way because we have tried similar ideas in FLOSS Manuals and seen their effect. I would prefer to continue to work this way with Booki.
So...there was one more feature we discussed - Chapter Level notes. I think this would be extremely useful for Source Fabric (but Micz needs to comment on this) but we need to be careful that we get it right because it is not so obvious how this might work.
I think the notes have to be associated with the chapter page when you edit it - however there is very little space there. One possibility is to build this into the WYSIWYG editor - Xinha - as a 'notes server' or some such. ie. it opens from the WYSIWYG editor but stores the content (chapter notes) in the booki db. The risk here is that people will not know that the notes are there...so we need to consider this. Another possibility is to build this into a 'sliding tab' as Micz suggested. I think that might be ok but it would have to be done carefully as it might look too much like a gimmick.
The other issue with chapter level notes is that I strongly believe that an overview of all chapter notes for a book should be able to be seen somewhere, in one place. Otherwise it would mean checking each chapter which would be a tedious job (books easily have 30+ chapters). So if you consider Chapter notes then you must also consider how to do this.
So on this I am not so clear what would work well for Chapter level notes and because of this I think it's not such a good feature for our first adventure working together. I would recommend instead the first three to be done all together - however this is up to Micz.
My feeling is that the first 3 are an extremely quick development, first however you need to know how it all fits together so i would suggest emailing this list when you have questions and I am sure Aco will answer your questions...
Also, Aco is currently working on the Booki site update so I expect the GIT repo is not updated but will be within the next days once the booki www is updated....
also you should meet Doug - doug is on this list and he is the Objavi (PDF generator) developer....doug - frank, frank - doug
also, meet John who does the Booki manual and other essential tasks intro intro :)
:)
adam
1 INTRODUCTION
1.1 Description
Booki is designed to help you produce books, either by yourself or collaboratively. A book in this context is a "comprehensive text" which can be output to book-formatted PDF (for book production), epub, odt, screen readable PDF, templated HTML and other formats.
Booki supports the rapid development of content. Booki has tools to support the development of content in 'Book Sprints'. Book Sprints are intensive collaborative events where collaborators in real and remote space focus on writing a book together in 3-5 days.
While you can use Booki to support very traditional book production processes, the feature set matches the rapid pace of publishing possible in the era of print on demand and electronic readers. Booki can output content immediately to multiple electronic formats. Print ready source (book formatted PDF) can be immediately generated, and then uploaded to your favorite Print on Demand (PoD) service, taken to a local printer, or delivered to a publisher.
1.2 Purpose
Booki embraces social and collaborative networked environments as the new production spaces for comprehensive (book) content.
1.3 Scope
Booki is available online as a networked service (http://www.booki.cc) for free. This service is a production tool for the creation of free content and not a publishing/hosting service. Content produced within Booki.cc is intended to be published elsewhere, either under another domain, in paper form (ie. books), distributed in electronic formats, or re-used in other content.
Booki can be installed by anyone wishing to utilise this software under their own domain or within private or local networks.
2 OVERALL DESCRIPTION
2.1 Product Perspective
Booki takes what was learned from building the FLOSS Manuals tool set and posits these lessons within a more suitable architecture.
Booki is the name of the collaborative production environment, however there are 2 associated softwares that provide all the services required :
Booki - production environment
Objavi - import and export engine
This document refers to Booki 1.5 and Objavi 2.2
2.2 Booki Functions
* User account creation requiring minimal information
* One click book creation
* Drag and drop Table of Contents creation
* One click editing of chapters
* Chapter level locks
* Live chat on a book and group level
* Live book status reports (editing, saving, chapter creation) delivered
to the chat window
* Drop down chapter status markers
* One click to join a group
* One click to add a book to a group
* One click exporting to epub, screen pdf, book formatted pdf, odt, html with default templates
* Easily accessible advanced styling options for export (CSS controlled)
* User profile control (status, image, bio)
* One click group creation
* Easy importing of book content from Archive.org, Mediawiki, other Booki installations
* Option to upload content to Archive.org
2.3 User Characteristics
2.3.1 Contributor
The majority of users will be contributors to an existing project. They may contribute to one or more project and may produce text and/or images, provide feedback or encouragement, proof, spell check, or edit content. These are the primary users and the tool set should first meet their needs.
2.3.2 Maintainer
These are advanced users that create their own books or have been elevated to maintainer status for a book by group admins. Maintainers have associated administrative tools for the books they maintain which are not available to other users.
2.3.3 Group admin
These are advanced users that wish to establish and administrate their own group. They have maintenance tools for every book in their group plus additional group admin tools.
2.4 Operating Environment
Booki is designed primarily for standards-based Open Source browser comparability but is tested against other browsers.
2.5 General Constraints
* Booki and Objavi are Python-based.
* Booki is built with the (bare) Django framework.
* Booki uses Jquery for dynamic user interface elements.
* Booki uses Postgres as the database but sqlite3 can also be used
* Redis is used by Booki for persistent data storage to mediate dynamic data delivery to the user interface
* Objavi utilises Webkit for PDF generation. Later Gecko will be added.
* Rendering of .odt by Objavi requires OpenOffice to be installed with unoconv.
* The Booki Web/IRC gateway may eventually (and optionally) require a dedicated standalone IRC service hosted on domain.
* Content editing in Booki is done by default with the Xinha WYSIWYG editor
* XHTML is the file format for content.
* Content will be ultimately be stored in GIT.
* Localisation in Booki is managed with Portable Object files (.po).
* The code repository for both projects is GIT with a dedicated Trac for bug reporting and milestone tracking :
http://booki-dev.flossmanuals.net
* A Dev mailing list is maintained here:
http://lists.flossmanuals.net/listinfo.cgi/booki-dev-flossmanuals.net
* Developers can be reached in IRC (freenode, #flossmanuals)
* Each release will be as source. Beta and later releases will also be available as Debian .deb packages.
* User and API Documentation will be maintained in the FLOSS Manuals
Booki Group.
* For development we use Apache2 for http delivery
* The license is GPL2+ for all softwares
2.5 User Documentation
Maintained here : http://www.booki.cc/booki-user-guide/
3 SYSTEM FEATURES
3.1 Booki Features
3.1.1 Booki-zip (Internal File Format)
Status: High Priority, Implemented
Function: A Booki-specific file structure for describing books
Interface: Used for internal data exchange between Booki and Objavi.
Notes: booki-zip definition maintained here :
http://booki-dev.flossmanuals.net/git?p=objavi2.git;a=blob_plain;f=htdocs/booki-zip-standard.txt
3.1.2 Account Creation
Status: High Priority, Partially Implemented
Function: Quick access to a registration form from the front page for account creation
Interface: Requires only username, password, email and real name (required for attribution). Email is sent to the user with autogenerated link for verification
Notes: email confirmation mechanism missing
3.1.3 Sign in
Status: High Priority, Implemented
Function: Quick access to a sign-in form from the front page
Interface: Username and Password form and submit button. Username and
pass remembered.
3.1.4 Profile Control
Status: Medium Priority, Implemented
Function: When logged in the user can access a profile settings page to set personal details (email, name, bio, image). Personal details can be browsed by other users
Interface: "My Settings" link in user-specific menu on left gives access to a form for changing the details.
3.1.5 Book Creation
Status: High Priority, Implemented
Function: Users can create a book from their homepage ("My Profile").
Interface: User can click on "My Profile" link from the user-specific menu on the left. On the Profile page a text field for the name of the book, and a license drop down menu (license *must* be set) is presented.
Clicking on "Create" adds the empty book with edit button to the listing of the users books on the same page.
3.1.6 Archive.org Book Import
Status: Medium Priority, Implemented
Function: Users can import books from Archive.org
Interface: "My Books" link in the user-specific menu on the left presents the user with a field for inputting the ID of any book from
Archive.org. The book is then imported when the user clicks "Import".
Notes : Interface is through Booki but Objavi does the importing and returns Booki zip to Booki. Relies on Archive.org successfully delivering epub for each book but this is not always happening. Needs error catching and user friendly progress/error messages.
3.1.7 Wikibooks Book Import
Status: Medium Priority, Implemented
Function: Users can import books from Wikibooks
(http://en.wikibooks.org)
Interface: "My Books" link in the user-specific menu on the left presents the user with a field for inputting the URL of any book from Wikibooks. The book is then imported when the user clicks "Import".
Notes : Interface is through Booki but Objavi does the importing and returns Booki zip to Booki. Needs thorough testing as it is sometimes failing possibly due to time-outs. Needs error catching and user friendly progress/error messages. Should be extended to be a "mediawiki import" tool, not just for Wikibooks.
3.1.8 Epub Book Import
Status: Medium Priority, Implemented
Function: Users can import any epub available online
Interface: "My Books" link in the user-specific menu on the left presents the user with a field for inputting the URL of any epub. The book is then imported when the user clicks "Import".
Notes : Interface is through Booki but Objavi does the importing and returns Booki zip to Booki. Needs thorough testing as it is sometimes failing possibly due to time-outs. Needs error catching and user friendly progress/error messages.
3.1.9 Group Creation
Status: High Priority, Implemented
Function: Users can create groups.
Interface: "My Groups" link in the user-specific menu on the left presents user with 2 text fields - group name, and description. When a name for a group is entered and "Create" is clicked then the group is created.
Notes: Group admin features missing.
3.1.10 Joining Groups
Status: High Priority, Implemented
Function: Users can join groups with one click.
Interface: "Groups" link in the general menu on the left presents a list of all Groups, by clicking on link the user is transported to the homepage for that group. At the bottom of the page the user can click "Join this group" and they are subscribed.
3.1.11 Adding Books to Groups
Status: High Priority, Implemented
Function: Users can add their own books to groups they belong to.
Interface: While on a Group page that the user is subscribed to the user can add their own books to the group.
Notes: When Group Admin features are in place we will change this so that Group Admins set who can and cannot add books to groups. At present a book can only belong to one group.
3.1.12 Readable Book Display
Status: High Priority, Implemented
Function: Users can read stable content in Booki without the need to log-in.
Interface: Upon clicking on the "Books" link in the general menu on the left a page listing all books is presented. Clicking on any of these presents a basic readable version of the stable content. Alternatively users can link to a book on the url http://[booki install domain]/[book name]
3.1.13 Edit Page
Status: High Priority, Implemented
Function: Page for editing content.
Interface: The edit page is accessed by clicking on "edit" next to the name of a book in "My Books" or "Books" (all books) listings. The user is then presented with a page with tabs for : editing, notes, exporting, history
3.1.14 Edit Tab
Status: High Priority, Implemented
Function: Edit interface for chapters.
Interface: Clicking ?edit? on a chapter title will open the Xinha WYSIWYG editor with the content in place.
3.1.15 Notes Tab
Status: High Priority, Implemented
Function: A place for contributors to keep notes on the development of the book
Interface: User clicks on the Notes tab for a book and is presented with a shared notepad for recording issues or discussing the development.
Notes : Implemented but future direction TBD
3.1.16 History Tab
Status: High Priority, Implemented
Function: Shows edit history of the book
Interface: User clicks on the history tab and can see the edit history with edit notes.
Notes: Implemented but unreadable. Users should also be able to access diffs here.
3.1.17 Export Tab
Status: High Priority, Implemented
Function: Export content to various formats
Interface: User clicks on the Export tab and is presented with a form for choosing export options. Clicking "Export" returns the desired output for download.
3.1.18 Version Tab
Status: High priority, Not Implemented
Function: can easily freeze content at stable stages while work continues on the unstable version.
Interface: From the Edit Page a maintainer sees an extra tab "Version".
>From here a maintainer can click "create stable version" - the last stable version is archived recorded and the current version becomes the new stable version.
3.1.19 Subscribe to edit notifications
Status: High Priority, Not Implemented
Function: Users can subscribe to edit notifications
Interface: User clicks "Subscribe to edit notifications" from the Edit Page for a book. If there are edits made a synopsis is emailed with basic edit information (time, chapter, person who made the change, version numbers) and a link to the diff.
3.1.20 Chat
Status: High priority, Implemented
Function: A real time chat (web / IRC gateway).
Interface: Persistent on the edit page for any book.
3.1.21 Localisation
Status: High priority, Not Implemented
Function: Booki needs to be available in any language where it is needed. Hence we may integrate the Pootle code base into Booki to enable localisation of the environment.
Interface: TBD
3.1.22 Translation
Status: High priority, Not Implemented
Function: Content can be forked and marked for translation. A
translation version of a book will provide link backs to the original
material, be placed in a translation work flow, and edited in a
side-by-side view where the translator can also see the original
source.
Interface: TBD
3.1.23 Copyright Tracking (Attribution)
Status: High Priority, Implemented
Function: Any user contributions are recorded and attributed.
Interface: All attributions are automated in Booki. Book level attribution is output in any chapter that contains the string "##AUTHORS##"
Note: should be a syntax for producing Attribution notes on a per-chapter basis eg. "##CHAPTER-AUTHORS##"
3.2 Objavi Features
3.2.1 Book-Formatted PDF Output
Status: High Priority, Implemented
Function: the server side creation of Book Formatted PDF is a pivotal feature. This is managed by Objavi which runs as a separate service. The book formatted PDF supports Unicode, bi-directional text, and reverse binding for printing right-to-left texts on a left-to-right press and vice versa. The formatting engine outputs customisable sizes including split column PDF suitable for printing on large scale newsprint.
Interface: This feature is managed by Objavi, an API is functional and feature rich but not well documented at present. Objavi also presents a web interface for those wanting more nuanced control (see http://objavi.flossmanuals.net/).
3.2.2 CSS Book Design
Status: High Priority, Implemented
Function: The default PDF rendering engine for Booki is now Webkit and will eventually be Mozilla Firefox hence there is full CSS support for creating book-formatted PDF in Booki. This changes the language of design from Indesign to CSS - which means any web native can control the design of the book.
3.2.3 Export Formats
Status: High Priority, Implemented
Function: Users also can export to self contained templated (tar.gz) HTML, to .odt (OpenOffice rich text format), epub, and screen readable PDF. Other XML output options can be developed as required.
I guess I can never claim to not having project management experience again. Darn it.
Attribution in technical projects is a fascinating topic. Fascinating, important and very occasionally controversial. Who should get credit for a work? That developers get attributed for the code they create is generally non-controversial. One, fairly typical, way to do this is to document these contributions in the copyright and contributions file of an open source project. You can also, of course, look at any given repository and see who has added what over the course of a project. It all seems pretty clear cut.
But what of the other people involved? Those that make wireframes for example? High level systems architects? Project or product managers? Ideas people? Founders? Designers? These people do not contribute code, so where do we place attribution for their work?
Generally speaking, the acknowledgement of these contributions occurs in places other than the code. How should we tell the story of the other people involved and what are the appropriate tools to do this?
These stories don’t often get told. When they do, they are in narrative form, on websites or blogs, telling the personal or organisational journey that saw a software from idea to reality. But unlike the contributions files and history records in code repositories, blogs and websites have a much shorter lifecycle and are bound, sooner or later, to be lost. Those stories tend to fade, leaving just the record of the code to tell the story.
It is a pity, because many of these contributions are critical in the life cycle of any software. Where would any successful desktop software be without the contributions of user testers? Many software solutions would not get a second look without a designer’s touch or feedback. Some projects would never have been born if it wasn’t for the inspiration of some energetic soul who managed to convince others of the value of an idea.
I’ll use my own work as an example but the phenomenon is bigger and there are far too many folk whose contributions go unacknowledged.
In a career, I’ve been fortunate to work on many successful and interesting technical projects over almost 2 decades. I have often been the ideas guy, not a developer, not a designer. I suck at QA, but I major in musing on the possibilities that technology can offer and I have become better and better at opening up people’s minds, from developers to clients, to funders, to what is possible. It is a process of inspiring and convincing others, of finding ways to gather momentum and resources to propel people down a promising path.
Sometimes I am fortunate to be credited for this work and I cherish those few moments where people have recognised this contribution and credit me. It doesn’t happen often, but it does happen in the occasional blog. Martin P Eve for example, a good friend and all round solid guy, called me out recently in a post about his JavaScript Typesetting Engine CaSSius:
It was very generous of him to do so. Without this, there would be no other record of my affect on his thinking and practice. I was humbled by his generosity.
When these things happen, while rare, it motivates people like me. It puts some fuel in our engines to keep moving on and continuing to work the way we do.
Sometimes, however, it goes the other way. Occasionally, and once again (thankfully this time) it is rare, there are those that believe we should not be credited. Having managed many projects, I actually understand why some people do not see the value of this kind of contribution, as Lao Tzu is credited with saying:
“When the best leader’s work is done the people say, ‘We did it ourselves.'”
Management and leadership are strange arts, and often lead to strange paradoxes. I have often found myself invisible in credits while generously crediting others. Once or twice someone has been outraged when I casually mention my involvement in a project that I managed, initiated or inspired.
These moments are rare and curious, especially when they involve small projects that represent not much more than a line item in my career over several decades involving many innovative approaches to many interesting problems. I guess over time I have come to understand that for someone the line item is the source of great pride. This may be the only innovative project he or she has ever worked on. While I’ve begun to understand it, this kind of proprietariness doesn’t make a whole lot of sense in the open source world – it plays better in an all-rights-reserved world.
But even when things are this weird, I will continue to shine the light on their work and attribute them, despite the lack of reciprocity. Credit where credit is due, even if attribution is, unfortunately, a one-way street at times.
The good news is that many open source projects do attribute many types of work. Mozilla, for example, has a single global credits list where names are recorded of people that have made “a significant investment of time, with useful results, into Mozilla project-governed activities.” Audacity is a favorite project of mine that credits pretty widely, categorizing contributions to include code, documentation, translation, QA, administration etc. These are great examples of software projects recognising and honouring a variety of work. We should all follow these examples, Open Source can only benefit from the generosity of attribution illustrated by these projects.
As a coda, and on reflection, I think we need to think carefully about how work is valued and attributed in technical projects. Architects often proudly say “I built that house.” Did they actually lift the hammer and cut the wood? Probably not, but I think they have a right to proudly state their contribution as much as user testers, designers, developers, managers, QA folk, and ideas people have a right to state and be recognised for the contribution they made towards a software. Not only that, we should all celebrate and recognise the large variety of contributions that goes into making software and find effective, lasting, ways to tell those stories.
In this lovely little town with incredible feel of Venetian architecture, tiny little streets, all set in stone, we waited for ADSLine to appear, which it did right next morning. The evening before we discussed over a plate of Italian pasta and a beer what would be good to work on in the coming days. After the internet magically appeared (yeah right!) we could also discuss with Adam, who was in Amsterdam, over Skype how to work and what to work on. Derek was somewhat the main captain although each of us worked on our own chapter. Adam was going after installation instructions for various platforms, Derek was rewriting and expanding tutorial on how to build a simple synth, and I was going deep into 'Dataflow' tutorials.
On Monday when I arrived, the sea was quite wild as there was a lot of wind. Tuesday was all gray and rainy. We had some talk about doing some hiking, as Derek had already explored some terrain, but Tuesday was a bit too wet, so we stayed in and worked on bits and pieces trying to see how much work there is and how writing feels. It seems from the after-perspective that this was the day to slowly get the grip on how the writing will go.
On Wednesday, we took a bus in the morning to a nearby village by the coast and with a little map from the tourist office tried to find that little path that would correspond to a tiny dashed black line that goes up the hill towards another village about 500m higher. We searched for more than an hour until finally a communicative small but wide man told us that old paths were overridden with a new tarmacced road. "I used to go up those little paths, when I was young..." he said, "but now there's a road so why would one bother, but if you really want there's a little path there, you can find it...." We walked a path for a while and than decended slightly to find a road and walked the road for an hour, maybe two, under not-too-hot spring-ish sun. It was lovely. After icecream in a local shop (those local shops are always something!) we caught a bus and went back 'home'. We started to work immediately, had some lunch and worked quite late into the night. We talked to Adam over Skype and kept working each on our own part.
Interestingly, and maybe because we were supposed to leave on Saturday morning (Derek had a plane ticket), next two days somewhat turned into a bit of frenzied work. As we knew we had just two days left, we tried to finish our parts. In consequence, at the end of Friday, we realized we hadn't gone out much. A real Sprint! One of us would go to the shop now and then, couple of times a day, and cook something (mostly pasta! - but I found some local home-made from Istria, and also Korcula olive oil!) The weather was nice, not cold, with many bits of sun. The voices of kids, sometimes horny cats, and other inhabitants were protruding through an open door and were making a special kind of atmosphere. It was lovely to feel this Venetian-medieval town around you. Our flat was on the third, top floor, we could see the sea from it and the roofs around us.
On the friday evening we found ourselves with two huge finished chapters, both quite satisfied with it. We were tired but happy we made something. We searched the town for fish and found an empty but slick-looking restaurant with a young owner-cook who made us some fish and vegetables. I guess we could have made them ourselves at home, but none of us spotted a fishery and had much time and energy left. I must say it has also been very educational for me to listen to Derek's peculiar playlist which included a lot of metal, old rock (Black Sabbath, etc) and African (Angolan) music from 70s. Luckily, Derek had a near perfect (or at least compatible) feel for sound levels, so it was always very enjoyable to become aquanted with a genre I haven't explored yet in my life much.
In the aftermath of the Sprint of two-and-a-half days, we both felt we wouldn't be able to go like this for another day. If anything, we would need a day, or ideally two days of real break from this and than we could come back to the book with passion. I was thinking this would be great to do three or four times. In waves. Either that, or steadily work for 8, maybe 10 hours per day and then have a break. But I think this also depends on the writer and her/his ability and fitness to write.
At the last moment I decided to stay for another two days (as our 'landlord' told us we could stay for us long as we liked, and Adam agreed). So Derek left on Saturday morning (6 am, ugh) and I worked on some of my music and had couple of walks around town and surroundings, really enjoying it immensely.
My car was still where I left it in Split, and the clutch was still functioning (barely), so I drove back, under lovely Velebit of course. It was almost like summer all the way to the border with Slovenia, where heavy rain was waiting for me.
