• Print

What Publishers Need to Learn from Software Developers

There was a great exchange on the O’Reilly editors’ backchannel the other day, so illuminating that I thought I should share it with the rest of you. We’ve been discussing the fast-track development we’re using to produce The Twitter Book. (We’re basically authoring the book as a presentation, after I realized how much more quickly I am able to put together a slide deck to make my points than I am a normal book. Twitter is also such a fast-moving topic that we need to be able to update the book every time we reprint it.)

Sarah Milstein wrote:

Apropos of everything, the NYT on publishers’ speeding up the production
process, especially with eBooks

“If this book had gone through the normal publishing procedures,” Mr.
Kiyosaki said, “it wouldn’t be worth writing.”

Andrew Savikas replied:

The more I think about it the more obvious it’s becoming to me that the next generation of authoring/production tools will have much more in common with today’s software development tools than with today’s word processors.

Software developers spend enormous amounts of time creatively writing with text, editing, revising, refining multiple interconnected textual works — and often doing so in a highly distributed way with many collaborators. Few writers or editors spend as much time as developers with text, and it only makes sense to apply the lessons developers have learned about managing collaborative writing and editing projects at scale.

‘Nuff said. I await said next generation of authoring/production tools.

tags: , ,

Comments: 29

  1. Tim,

    You may have been posting this at the precise moment I tweeted my frustration over the slide deck obsession in government reporting:


    I’m curious about how the two can be reconciled.

    Your post here also seems to have something in common with the Google Chrome Comic:


  2. Cool stuff. I think Andy & Dave over at the Pragmatic Programmers already figured this out. Interestingly you can read about working with them as an author and everyone raves about their publishing toolchain. Dave posted a podcast about it, I think it’s awesome.

    Cool guys those two.

  3. To this writer, Andrew’s comment was spot on: “…spend enormous amounts of time creatively writing with text, editing, revising, refining multiple interconnected textual works.”

    That’s how I see my online project, “A Novel of America,” which follows the exact plan James Michener and I used in crafting our epic novels, with a key difference of letting these multilayered tasks interconnect and unfold on the Web.

    As I go along, I share my working notes and research links, plotting, image and map files, and so on — all part of a long and challenging transition toward the book of the future.


  4. Pretty fascinating point. I don’t think it’s just authoring/production tools that need to take lessons from software development, either. Focused, measured and tight iteration loops can fit more evolutionary generations into smaller time periods than [ever?] before, and I think that many, many more industries face unknown solutions than we or they will necessarily admit. Manufacturing? Architecture, real estate development, urban planning? Biotech? Everything? Lean and agile practices carry value far beyond software.

  5. At the risk of appearing flattering, Tim, I want to point out to programmers reading that your engineering skills are showing there. You have this book that needs to be out quickly and inexpensively AND after deployment it will need upgrades. You picked the “presentation format” because it lowers up-front costs (quick to deploy) AND because it modularizes the book in the right way to make the future upgrade process smooth. To the editor of a periodical (e.g., magazine or newspaper) that kind of component-based, modularity-oriented thinking is normal. In your tech. pub. biz that kind of thinking is probably just in your blood by this point and instinctive. But my sense is that a lot of working programmers generally don’t appreciate that those engineering patterns apply to how a lot of business gets done – not just the business of writing new programs. (ok, I’m putting away my highlight marker, now.)

    Yes, much like coders “you guys” (your industry) wants, I think, a distributed and decentralized content authoring system with revision control features and good support for modularity and for release engineering – a kind of “IDE” if you will, for multi-media content. Within the database structures of such a beast there can be an exploration of the new reality in which on-line, interactive and printed traditional media co-mingle – consider the experiments in printed copies of Wikipedia content as the evolutionary precursor here. There are sweet spots yet unrealized in that corner of the design space.


  6. Tim, be careful.

    As an ancient and out-of-date software developer I’d point out that the size of the components and their structure is quite different – books vs. code. This is clearly less true for reference document style books, like your Twitter project. But can you imagine a novel written by a team in small chunks that are checked-in and checked-out with revision control and code review?

    I have no doubt that a reference manual or document can be much more easily managed with some sort of source code control system. Just avoid trying to write a novel with such a thing.



  7. I like it. Agile and XP practices are applicable to almost every industry (cough, cough, auto industry).

    Evolve or die.

  8. @Beau — I’d agree with you that writing a novel as a group effort probably wouldn’t turn out all that well. But I suspect most individual authors would appreciate the comfort of an infinite revision history, as well as a very easy way to compare one of their revisions against changes made by an editor or copy editor. Nearly anything I write that’s longer than a blog post goes into SVN (including presentations) regardless of whether or not anyone else will every touch the file. Version control gives me permission to fiddle without fear of not being able to turn back. I’m no novelist, but that sounds helpful (and imagine scholarly access to novelist’s commit logs!)

  9. Makes sense to me. If you can process all foods with a food processor, you should be able to use a single machine to process all words.

  10. Nick,

    Huh? What do you mean?

    When you (if that’s you) write a book there’s a certain mechanical aspect wherein you shuffle manuscript and mark-up back and forth with yonder publisher, right? The software tools alluded to here would support that neatly. The thing about Tim’s particular twitter book being modularized and presentation format is specific to that particular work – nobody is saying “all” in that sense. It’s just that the underlying, low-level, how-do-manuscript-versions/parts-get-named and how-are-the-accessed/created/etc. technology is just some comfortable extensions to things like a word processor. E.g.: automate keeping track of different versions of a work as part of the “save” command; make it easy to compare versions; make it easy to pass versions between people (an “email this document” link not on a web page but in your word processor, say); that kind of thing. I’d make a plausible guess that when you work on a book you are careful about making backups, about setting aside “snapshots” as you hit various milestones, and so on. You probably do such things “by hand”. For a lot of software developers, analogous features are built into the programs they use to edit source code and the suggestion here is just to build similar tools for other media besides software source code.

    There is socio-economic subtlety to it. The new tools make new trade patterns possible. But they aren’t that radical a departure from what you are probably already doing (by external appearances).


  11. Interesting.

    Have you ever authored books in wiki? Or looked at adapting a wiki for authors?

  12. @Bradly: Funny you say that. When I was an editor at O’Reilly, I got interested in Agile, and my group experimented with a few XP techniques to try to improve and speed up the book development process. Although it didn’t quite work on the tactical level, it did get us thinking about more appropriate strategies for the fast-to-market series we were publishing.

  13. I personally believe that it´s the creative minds that create excellent software. Their instructions and ideas. Some programmers are very creative, but many are not thinking in terms of simplicity.

    Mike Dammann

  14. As an O’Reilly author, I’d like to point out that many authors *do* write books using the same tools that software developers use to write programs.

    Let me give you an example.

    When I (and my co-authors) sat down to write Asterisk: The Future of Telephony (the second edition), we used Subversion to keep track of revisions. I used vim as my editor. When I wanted to build a PDF of a chapter to see how it looked on paper, I typed “make pdf” and let my Makefile compile the PDF for me.

    I guess to me it was the obvious way to do the project. Maybe I’ve just taken it for granted all this time!

  15. omigawd, i am actually in agreement with nicholas carr! oh no!

    what’s the world coming to! i feel slimy. must go take shower…


  16. Bowerbird

    No, just sit in that stink for a while. You might find, as I have, you come to like it. “He’s right you know. About everything.” – Mo Sizlack.


  17. Bowerbird: Try using capital letters occasionally. They’re great.

    T. Lord: No, really, that’s not how I work at all. In the course of writing a book, which takes, say, a year or two, I’ll probably revise each chapter a few hundred, maybe a thousand, times. I’ll mainly do that using a word processor, though there are also times I’ll print stuff out and edit on paper (mainly to get a different “feel” for the text). Out of all those editing cycles, I’ll maybe back up a half dozen versions, usually when I’ve decided to try a very different tack (and know I might want to go back at some point and lift some passages from the old version). I concentrate pretty intensively when I’m writing, so most “old versions” are stored in my long-term memory, which is a far better place to store them than on a hard drive. They’re inert on a hard drive; they’re active, at least subconsciously, in long-term memory. It’s actually exceedingly unusual for me to go back and look at any of the old versions stored on my hard drive (though it’s nice to know they’re there).

    When I have a complete, draft manuscript, I send it off to my editor at my publisher. He prints the whole thing out and marks it up in pencil and sends it back to me, along with a letter laying out general comments. Sitting back at my computer, I go through the marked-up manuscript page by page, think about each suggestion, and revise the text. (If my editor did a “track changes” edit, it would be much less conducive to deep thinking about the text. It would, in fact, be intolerable.) There may be a couple of more rounds of this kind of editing, and then it goes to a copyeditor, who also prints out the manuscript and marks it up manually.

    Eventually, it’s done, and I’m left with a big stack of pages on the floor of my office, which I eventually throw out. The idea that some kind of more efficient edit-tracking system, based on a software-development system, would lead to a better book is utterly absurd.

    Now, I’m sure my writing and editing process is idiosyncratic, and I’m sure other writers have very different systems. Some may find a software-development-like process very useful. And that’s great. But the most ridiculous thing I can imagine is to attempt to automate and standardize creativity.


  18. I’m currently finishing up work on Programming the Semantic Web for O’Reilly, and I’ve been feeling like the whole writing – editing – production process is akin to waterfall software development. I wonder if the next book that I write will look more like a well-edited blog.

  19. (Just to be clear: by “thanks” I mean “that’s really interesting I don’t have anything more clever to add than to acknowledge that. It was generous of you write that.”)


  20. Stephen Henderson

    Good grief- apparently journalists, politicians, marketers, carmakers, and now writers all have to become more like bloggers/software developers/twitterers/etc… or DIE!

    Is there anything you insufferable know-it-alls could learn from someone else?

  21. An interesting posting I must say. However I do not think that publishers need to take any lesson from software developers.

  22. Nick,

    I’m sure you found it difficult to go from longhand to typewriter as well, and from typewriter to word processor was a really steep climb!

    Yes, tools change the way we write. I’ve done an awful lot of writing and editing on paper, and an awful lot with online tools. The process is certainly different. But I think it’s short-sighted to say that the old way is “better,” or that sustained thought isn’t possible when writing with online tools.

    I remember switching back and forth from one to the other in the early years, and it always took me a while to adapt. But in each case, I don’t think what I wrote or what I edited was better or more thoughtful because of the tools I used. Thoughtfulness is the province of the writer, not of the pen, the word processor, or the revision control system.

    As for your initial comment, I’m sure you are aware that it’s just long enough for a tweet. Dorothy Parker & Oscar Wilde would have flourished in the age of twitter. But they must not be real writers because of that!

  23. The discussion is making me think of a collaborative writing tool called Mixed Ink (www.mixedink.com) that I came across a while ago. It allows large groups of people to work on content simultaneously in some crazily democratic fashion.

    Whilst it would be great to speed up time to market on any product, with publishing you are dependent on the thinking & writing speed of your author as much as the efficiency of publishing processes. That’s not to say that innovations can’t be found, but it’s never going to be one size fits all.

    Quality vs Time is pretty diffcult to overlook when you’re talking about the cafeful crafting of 1000s of words.

  24. While I believe publishing can learn a lot from software dev practices, it is at a higher level than code production. Both software and books are improved when they’re approached not as a collection of words but as a crafting of user experience.

    Looking to software tools to help us (authors) generate and revise words can help the process of generating and deploying those words, but will it help us tell better stories?

    I have always believed a big problem with many tech books is the same problem we find in a lot of software–too much emphasis on code production, too little on UX design. As both a software developer and tech book author, I think BOTH domains have a lot to learn from those who create memorable, lasting, engaging user experiences.

    Maybe we can look to successful Hollywood writing teams, for example, or improv theater groups. And if we’re going to look to software dev for models, I’d look at some of the great game producers. Beginning with referring to the project manager as “producer”, and referring to the programmers as “the talent” ; )

  25. “Dorothy Parker & Oscar Wilde would have flourished in the age of twitter.”

    That’s your April Fools joke, right?

  26. nick said:
    > Bowerbird: Try using capital letters occasionally. They’re great.

    actually, they just give me indigestion… :+)

    as a one-off, this was a semi-interesting blog entry, tim —
    hey, collaborative writing could benefit from version control!

    well, yeah, maybe, but most writers don’t write collaboratively.

    and, as nick has pointed out, even the editing process is one
    better handled by sequential hand-off, and not simultaneously.

    if anyone out there does want to explore collaborative writing,
    i’d suggest you go exploring over at the if:book blog, where
    bob stein has been researching this idea for quite a while now.
    i can’t say he’s accomplished much. i think it’s just a dead end.

    i’m a poet, and we’re probably the exemplar of the lone writer,
    but i still don’t see much interest for group writing in the wild.

    if anything, the rise of blogs seems to have convinced everyone
    that they have the unique voice that’s required by being a writer,
    so i see an increased interest in people hearing their own voice
    — often unhampered even by spell-check, let alone editors —
    rather than selflessly blending in with one or more other people.

    oh, and cory had a post recently over on boing-boing where he discussed using version-control to track changes in his work…

    i wrote a program a few decades ago that saved all keystrokes
    while you were writing something (and even the timing of ’em),
    so you could go back and watch yourself writing the piece again.
    i called it “process-recorder”, and it was really quite fascinating.
    it’s much more interesting than doing a diff on static versions,
    because you get to see the false-starts, edits, and reworkings.
    it’s also a lot more economical than saving all of your versions,
    because there’s far less redundancy. i never released the app,
    but you can do something similar by using the macro capability
    found in many word-processors to give yourself the feel of it…


  27. This reminded me that I recently heard Cory Doctorow on This Week in Tech talking about using GIT version control for his current work and how having diff reports is helping, etc. He’s got some other contextual metadata getting rolled in with commits as well. http://craphound.com/?p=2171

  28. Whilst it would be great to speed up time to market on any product, with publishing you are dependent on the thinking & writing speed of your author as much as the efficiency of publishing processes. That’s not to say that innovations can’t be found, but it’s never going to be one size fits all.