Building Book Production Platforms p3

The editor

This series is based on HTML as a source file format for book production platforms. I have looked at many HTML editors over the years and can remember when the first in-browser editors appeared…it was a shock. Prior to that, all HTML creation was done by writing directly in HTML code, then came fully-featured environments like Front Page and Dreamweaver which allowed you to create HTML in a desktop app, then came wiki mark-up to liberate us all from the tedium of writing HTML, and then finally…the browser-based WYSIWYG editor…

It’s worth noting that the Wiki markup and WYSIWYG solutions were a different category to the previous solutions in that they weren’t designed for creating web pages, rather they were designed to enable the production of wikis and content management systems.

What-You-See-Is-What-You-Get at that time was a refreshing and liberating idea, a newcomer to this scene (although WYSIWYG as a concept and approach to document creation predates the web, with the first true WYSIWYG editor being a word processing program called Bravo, invented by Charles Simonyi at the Xerox Palo Alto Research Center in the 1970s, the basis for MS Word and Excel). Many WYSIWYG strategies have been explored, and many weaknesses unearthed, including the very important critique that What-You-See-Isn’t-What-You-Get, because the HTML created by these editors is unreliable, but more on this later…

As far as I can tell, the first HTML-based WYSIWYG editor was Amaya World, first released in 1996. I don’t know what WYISWYG editor was the first to be embedded in a browser (if you know, please email me). However, I remember TinyMCE like it was a revelation. According to the Sourceforge page, they started building it around 2004 to solve the need to produce HTML in content management systems. It was, and is a great product. The strategy at the time was pretty much to emulate rendered HTML within an HTML text field. TinyMCE (and the others that followed until contenteditable came along) used a heap of JavaScript to turn a simple editable text field into a window onto the browser’s layout engine.

alt.typesetting

From this point, a number of plugins were developed for use with WYSIWYG editors like TinyMCE to extend the functionality.

Some of these plugins ventured into the ever-important area of typesetting. TinyMCE even tried at times to make up for the lack of browser functionality in this area – for example, there were some early and workable attempts to bring equation editing into TinyMCE. I can’t remember when it was, but surely around 2006/2007 that IMathAs had an experimental jab at this. I thought it was pure genius at the time as there was no other solution (I searched! a lot!). As I can remember they used a very clever round-tripping to achieve the result… essentially, since browsers didn’t then support Math, IMathAs supported inline equation writing using ASCIIMath syntax. When the user clicked out of the field, the editor sent the equation markup to the server, and the server returned the rendered equation as either a bitmap (PNG, JPG etc) or as vector graphics (SVG). It was genius and I built it into the workflow for FLOSS Manuals around 2010 because we wanted to write books with equations for software like CSound (produced in 2010/11). It worked great – the equations always looked a bit ‘bit-mappy’ but we could write and print books with equations using in-browser editors and HTML as source (the HTML produced included equations as images so we could render PDF direct from the HTML). Awesome.

It’s also worth noting that these days math typesetting has largely moved to the client side with the evolution of fantastic libraries like MathJax  and KateX. These are JavaScript typesetting libraries designed to be included in web pages and render math from markup on the client side. There are one or two tools that still use server-side rendering, notably Mathoid, and this is often used to reduce the burden on the client’s browser, however, they have possible additional bandwidth costs as the client and server must remain in communication with each other, otherwise nothing will be rendered or displayed.

Mature solutions for math and other typesetting issues are only just starting to come online – no surprise to historians who inform us that notations such as math and music were the last to come online for the printing press as well. The first book to contain music notation post-Gutenberg, the Mainz Psalter, was printed with moveable type, and the music notation was added manually by scribe. It seems the first thing to get right is the printing of text, all other notations come later in print systems. These solutions are slowly evolving – even music notation has its champions. However what is really surprising, is that Google, a company priding itself on being built from ground up by math-heads, seems to struggle to bring native math typesetting to their own browser . I would say that is embarrassing.

Contenteditable

Moving on from typesetting… The initial WYSIWYG editors proved an admirable solution for many content management systems. The name persisted but the background technology fundamentally changed in when the first implementations of the  W3C contenteditable specification for HTML5 was brought to the browser. Contenteditable is an attribute that you can add to a number of HTML container elements (like ‘P’ or ‘div’) that make their contents editable. So, in essence, you are directly editing the content in the browser rather than through some JavaScript text field trickery. This strategy might be called WYSI (What-you-see-IS). This strategy also spawned a whole new generation of editors leveraging this new native browser functionality. Aloha Editor was one of the first to grab the spotlight but there were many many others to follow. Additionally, the big legacy WYSIWYG editors such as TinyMCE and CKEditor added support for contenteditable although they were a little slow to the party.

Contenteditable at first promised a lot… native editing of the browser … phew … that certainly lowers the technology burden and opens the door to innovation and experimentation. Additionally, the idea that this is a read-write web suddenly comes more keenly into focus when you can just edit the web page right there and inherit all the same JavaScript and CSS that operates on the element you are editing. It’s good stuff.

Inevitably, though, some problems soon emerged, first some wobbly things about not being able to place a caret (your mouse pointer) between block elements (eg between two divs) was really a problem, but later a more serious issue was identified – contenteditable does not produce stable results across different browsers, such that if you edit one page in browser A, the resulting HTML could look different if you edited the same page in browser B. That might not affect many people – if you just want some text with bold and italics and simple things, then it doesn’t really matter… the HTML created will render results that will look pretty much the same across any browser. However there are use cases where this is a problem.

In the world I work in at the moment – scholarly publishing – we don’t want a manuscript that contains inconsistent HTML depending on the browser it was edited in … it hurts us down the road when we want to translate that HTML into different formats (eg JATS) or if we want to render that HTML directly to PDF and get consistent results.

So, unfortunately, editors like CKEditor (used by many book production platforms including Atlas), and TinyMCE (used by Press Books AKA WordPress), or Aloha (used by Booktype 2) have to use a lot of JS magic to produce consistent HTML to overcome the problems with contenteditable, and this doesn’t always succeed. I would recommend reading this article from the Guardian tech team about these issues. You also may wish to look at this video from the Wikimedia Foundation Visual Editor core devs for the comments on contenteditable (audio is lousy, jump to 1.14.00) (readable subtitles can be found here .

A better way

So…what can you do? The answer is kind of threefold.

First choice: decide not to care – an entirely legitimate approach. You can still do huge amounts with these editors, and if you need to tweak the HTML now and then, so what? I can clean up the HTML by hand for a 300-page book in an hour, not too tough really and it enables me to cash in on all the other enormous gains to be had from a single-source HTML environment.

Second choice: provide client-side and server-side cleanup tools. Most editors have these built in, but it’s also good to implement backend clean up tools to ‘consistify’ the HTML at save-time (or at least at pre-render time).

Third choice: find an editor that is designed to produce consistent HTML.

In my opinion, the third choice is the best long term option and the ‘right way’ to do things. Being able to produce reliable results with ease, and without having to do things twice, will make everyone’s life easier.

Thankfully there is a new editor on the scene that is designed to do just this – the Wikimedia Foundations Visual Editor. This editor was developed to help the Wikimedia Foundation solve an uptake problem … essentially there are not enough people these days prepared to sit around learning Wiki markup (which is pretty much a complicated scripting language these days). The resulting need to drop the threshold on the foundation’s contributions environment has resulted in the development of the Visual Editor (VE). New contributors can use an easy WYSIWYG-like environment instead of having to learn markup.

Obviously, the entire Wikimedia universe is already stored as wiki markup, so the editor needs to be able to translate between HTML and wiki markup on-the-fly (interestingly, it is actually part of much larger plan to store all Wikimedia Foundation content in HTML. To do this there is a back end called Parsoid that converts markup to HTML and vice versa. Also, the HTML produced by the editor obviously needs to be tightly controlled, otherwise the results are going to be a mess when converting back to wiki markup. VE does this by replicating the content in its own internal (JSON) model and displaying the results in a contenteditable region. When the content is edited, the edits are strictly controlled by the VE internal rules, and then rendered to display. The result… consistent HTML is produced across any edit session regardless of the browser used…

That’s pretty good news. This is one reason amongst many that the platform I am working on for the Public Library of Science has adopted VE software (we were the first to use it outside of the Wikimedia Foundation) and we are extending it considerably and contributing the results upstream to the VE repos. So far we have added table, equation, and citation plugins – all of which are in an early alpha state. If you want a peek, you can see some of the work here.

I highly recommend to anyone building a book platform, or any other kind of knowledge production platform, that you examine VE more closely. It is a sophisticated software and has been carefully thought through. It is still relatively immature, and development is happening at an incredible pace, which can make testing new plugins against an unstable API a little arduous … still, it is a great solution. VE also approaches content editing in a way that will open the door to concurrent editing via operational transformations in HTML, which is a hard problem and currently only solved by Google and Wikidocs (recently acquired by Atlassian.)

If you are in the process of choosing an editor, choose VE and contribute to the effort to make it not just the best Open Source solution to editing in the browser, but the best solution, full stop.

Many thanks to Raewyn Whyte for improving this article.

Math Typesetting

Typesetting math in HTML was for a long time one of those ‘I can’t believe this hasn’t been solved by now!’ issues. It seemed a bit wrong – wasn’t the Internet more or less invented by math geeks? Did they give up using the web back in 1996 because it didn’t support math? (That would explain the aesthetic of many ‘home pages’ for math professors.)

MathML is the W3C-recommended standard markup for equations – it’s like ‘HTML tags for math.’ While MathML has a long history and has been established in XML workflows for quite some time, it was only with HTML5 that mathematics finally entered the web as a first-class citizen. This will hopefully lead to some interesting developments as more users explore MathML and actually use it as creatively as they’ve used ‘plain’ text.

However, the support in browsers has been sketchy. Internet Explorer does not support MathML natively, Opera only supports a subset, and Konqueror does not support it. Webkit’s partial support only recently made it into Chrome 24 and was held back on Safari due to a font bug. Firefox wins the math geek trophy hands down with early (since FF 3) and best (though not yet complete) support for MathML.

What’s the hold-up?

It seems that equation support is underwhelmingly resourced in the browser development world. Webkit’s great improvements this year (including a complete re-write + the effort to get it through Chrome’s code review) have been due to a single volunteer, Dave Barton. Frédéric Wang plays the same heroic (and unpaid) role for Firefox. The question is: why is this critical feature being left to part-time voluntary developers? aren’t there enough people and organisations out there that would need math support in the browser (think of all the university math departments just for starters…) to pay for development?

As a result, we have no browser that fully supports MathML. While you cannot overstate the accomplishments of the volunteer work, it’s important to tell the full story. Recent cries that Chrome and Safari support MathML can give the wrong impression of the potential for MathML when users encounter bad rendering of basic constructs. The combination of native MathML support, font support, and authoring tools, [ see this more recent post for more detail] makes mathematical content one of the most complex situations for web typesetting.

There have been some valiant attempts to fix this lack of equation support with third-party math typesetting code (JavaScript), notably Asciimath by Peter Jipsen with its human-readable syntax and image fallbacks, jqmath by Dave Barton (same as above), and MathQuill.

However, by far the most comprehensive solution is the MathJax libraries (JavaScript) released as Open Source. While MathJax is developed by a small team (including Frédéric Wang above) they are bigger than most projects, with a growing community and sponsors. MathJax renders beautiful equations as HTML-CSS (using web fonts) or as SVG. The markup used can be either MathML, Asciimath or TeX. That means authors can write (ugly) markup like this:

J\alpha(x) = \sum\limits{m=0}^\infty \frac{(-1)^m}{m! , \Gamma(m + \alpha + 1)}{\left({\frac{x}{2}}\right)}^{2 m + \alpha}

into an HTML page and it will be rendered into a lovely looking scalable, copy-and-paste-able equation. You can see it in action here. 

So why is this really interesting to publishing?

Well… while EPUB3 specifications support MathML, this has not made it very far into e-reader devices. However, MathJax is being utilised in e-reader software that has JavaScript support. Since many ebook readers are built upon Webkit (notably Android and iOS apps, including iBooks) this strategy can work reasonably effectively. Image rendering of equations in ebook format is another strategy, and possibly the only guaranteed strategy at present, however, you can’t copy, edit, annotate or scale the equations and you cannot expect proper accessibility with images.

The only real solution is full equation support in all browsers and e-readers either through native MathML support or through the inclusion of libraries like MathJax. We can’t do anything to help the proprietary reader developments get this functionality other than by lobbying them to support EPUB3 (a worthwhile effort) but since more and more reading devices are using the Open Source WebKit, we can influence that directly.

The publishing industry should really be asking itself why it is leaving such an important issue to under-resourced volunteers and small organisations like MathJax to solve. Are there no large publishers who need equation support in their electronic books that could step forward and put some money on the table to assist WebKit support of equations? Couldn’t more publishers be as enlightened as Springer and support the development of MathJax? You don’t even have to be a large publisher to help – any coding or financial support would be extremely useful.

It does seem that this is a case of an important technological issue central to the development of the publishing industry ‘being left to others.’ I have the impression that it is because the industry as a whole doesn’t actually understand what technologies are at the centre of their business. Bold statement as it is, I just don’t see how such important developments could go otherwise untouched by publishers. It could all be helped enormously with relatively small amounts of cash. Hire a coder and dedicate them to assist these developments. Contact Dave Barton or MathJax and ask them how your publishing company can help move this along and back that offer up with cash or coding time. It’s actually that simple.

Many thanks to Peter Krautzberger for fact checking and technical clarifications.

Published on O’Reilly, 26 November 2012 http://toc.oreilly.com/2012/11/math-typesetting.html