...The Tech's Blog
A blog by Paul (Jack) Daniels, of all things technical being an independent publisher and some things not.
Thu, 03 Mar 2011 01:34:07 +1100 441 views

Those of you who already know me will probably also know that I don't "do" Microsoft Office, not even OpenOffice (or LibreOffice) for the production of print and eBooks. Many of you are further likely to be wondering just why I'd be so crazy is to cut against the norms in such a wild manner, surely it would leave us (Elita and myself) in a world of obscurity, left without a support network and endless troubles trying to get our books published, fortunately it doesn't... so read on :).

Back in the crazy days of PCs when PC-DOS, MS-DOS and DR-DOS roamed the savanna, WordPerfect v5.1 was king. You were a nobody if you didn't use WP51, it was your big spanner into the world of business computing (along with Lotus 1-2-3). There were not many fancy graphics consoles, everyone generally worked from plain text screens, usually 80x24 characters (some of us crazy people used 132x40). To use WP51 you had to have quite a bit of training, else it was just a blank screen. Despite the huge learning curve that WP51 presented it was also one of the best PC word processors and a skilled operator could generate documents very quickly, consistently and with good quality output...

... then Microsoft gave us MSOffice...

Office bought a lot of people into the computing world, it was fairly easy to use, produced reasonable results and cost a reasonable amount of money. The whole "reasonable", or "good enough" mantra is something Microsoft did very well and it was a winning mix for sales. With the shiny appeal of WYSIWYG, content and formatting became merged and unfortunately it didn't turn out pretty all the time. People without appropriate training hacking their way through an appointed task any way they could, finding amazing pathways to the end result, some that still baffled me to this day (I'm sure many a tech has uttered the phrase "How the HELL did you manage to do that?").

I'm sure we can all recall at least a couple of documents we've handled where the person had used spaces or tabs to meticulously format the text, or finding that they had to manually insert titles into the table of contents, chapter numbers or even manually managing index entries, forcing a few spaces in here or there just to get a picture to "stay put" where you thought you wanted it. Let's not remind ourselves what happens to these hand-painted documents when they move to another version of Word, or even just another computer with different fonts installed. These bad crutches frequently a proper methods to achieve the desired result but it was just too easy to hack it instead. The damage is done on society, we now spend too much time worrying about the look of the document, rather than what is in it.

Finally I get to the point of this blog entry, we use LyX for our document processing. LyX takes away from you all the pretty toys that you're not meant to really be playing with when trying to get your ideas down. It forces you to get back to writing content and leave the typesetting to the back-end engine (TeX). It takes away the distraction, the urge to fiddle, and believe me, at first you do try VERY hard to fiddle, even trying to add double-linebreaks, tsk tsk. LyX forces you to assign a style to each block of text (paragraph), such as Title, Chapter, Section, Paragraph, etc but you don't get to fiddle with the font size, family or position on an individual basis, that would break you back into bad habits. The look of the output is determined by the particular style you've selected (in my case, I use 'Book - Memoirs'). By doing this, we've finally separated content from format and we can once more get back to writing.

LyX does not offer a WYSIWYG front end, instead it is a more akin to WP51 where you see the text and nothing more other than some fundamental buttons. The on-screen appearance of the content does not convey what the rendered output will look like. Now you might well ask... "why take away the WYSIWYG ability?", two reasons;

1) It's distracting (because you'll likely try and massage the look of it)

2) ... but more importantly, it's computationally slow to generate the output.

Typesetting well is not a simple task. LyX produces LaTeX format files. LaTeX files are plain text, you can actually read them and many people write them directly, like HTML pages. LaTeX files are processed by the TeX typesetting engine, a formidable beast, mature, big and exceptionally versed with the wisdom of Typesetting.

On first glances the output from a LyX generated PDF compared to that of MS-Word or OpenOffice might seem without difference, however there are a multitude of subtleties in the spacing of letters, the determination of where to break a page, the choice of where to break a word versus leaving a ragged edge, and all these decisions have a rippling effect all the way through your document. The number of things that the TeX engine takes into consideration when typesetting your document is staggering and the rules it applies have been honed since the first printing press. The TeX engine is also stable, you will always get consistent results from it (unlike the pot-luck of Office as you move between versions).

Qualitatively, printed documents (PS, PDF) generated on the back of the TeX engine have almost no equal, they are superior. You can look at two books, one from Office, one from a TeX source (LyX etc) and while you might not be able to directly discern the differences, your mind and eyes can feel it. LaTeX is still the preferred document format for most academic institutions, especially if you're writing up math papers with complex formulas. The more you come to learn about typography the more you see these differences and you start to find yourself being able to pick which program generated the text you're viewing. I don't recommend picking up that 'superpower', it drives you mad (Graphics artists and colour experts endure a similar insanity).

Unfortunately when it comes to eBooks, many of those beautiful gains from using the TeX engine are gone, because we fall back to relying on the rendering engine of the eReader (Kindle, NOOK etc). LyX still however has a great place to play and still produces excellent output. The exported HTML version of the manuscript from LyX imports directly into Calibre and produces an ePub/mobi with fully functioning ToC and chapters, consider that next time you find you cannot skip back/forth between chapters while reading an eBook. The HTML output from LyX is also very clean and doesn't require exotic recipes in Calibre to 'get right', this also includes things like drop-caps and scene breaks.

There are downsides to using LyX if you're not already using a great Linux distro like Ubuntu, installing LyX on Windows is a royal ... pain, because of all the associated dependencies (the whole TeX engine for a start). If you're on Ubuntu, you're good to go. There's another ugly downside for using LyX at the moment, especially if you're using Smashwords, LyX doesn't generate ms-doc format files. To create a ms-doc from LyX you have to export as RTF or HTML and then pull it into OpenOffice or MS-Office, unfortunately all your formatting has basically become useless. This is a real deal breaker for many writers and I can't blame them, however there is good news on the horizon with Smashwords looking to allow direct ePub uploads for some publishers (I'll write more about me vs Smashwords in a future post).

LyX does take time to get used to, it's not the prettiest thing to look at when you're hacking away at the keyboard, it feels a bit antiquated and to be honest, the dictionary and grammar checker are at this point rather useless compared to the excellent progress made in OpenOffice/MSOffice (fortunately the LyX team are now making adjustments to let you use the OpenOffice spell/grammar engines, opensource at work!). With LyX, the time investment is at the start, it's the steeper learning curve but the payback is massive. I'm sorry to say but I shake my head at the number of threads I read on KindleBoards and Mobilereads about people having to spend hours on hours with every book trying to get it to work right, no thanks, I'll stick with LyX and its boring look but consistent production results.

I don't personally recommend LyX to people who aren't at least already into the world of Linux/Ubuntu, simply due to the setup pains (There's an OSX version too), however because Linux is my backyard it was a natural choice. If you are still interested, despite my hesitations, then you're welcome to catch up with me and ask some further questions. For everyone else, stick with what you know but know also that there is a long-term easier path if you can invest the up front time.