Release Early, Release Often: Agile Software Development in Publishing

“How do Web startups release three or four new versions of a product in the time it takes publishers to launch just one new feature on their online platforms?”

This question framed “The Agile IT Organization,” a lively and well-informed discussion at the recent Society for Scholarly Publishing annual conference in Boston. As a software engineer, I’ve used both agile and traditional product development methodologies and I was interested to hear the perspectives of other programmers as well as publishers who’ve gone through the process.

Geoffrey Bilder of CrossRef provided an introduction to agile development practices, which are concisely summarized in plain English by a core set of principles.

Summarizing even further, agile development means:

  1. Minimal up-front specification. A project has high-level goals (e.g. “make our back catalog searchable and available for print-on-demand purchase”), but is not fully described before
    development begins.
  2. Frequent, short-cycle releases. A project is broken up into mini-projects, each with a small set of features that take only a few weeks to implement. Every release (“iteration”) has a specification,
    development and testing phase. This means that every couple of weeks the software is fully usable, although it may have very few features at the start.
  3. Change to the product design is accommodated and even expected. Market conditions, corporate re-organization or user demands may mean that new features are added or old ones are re-worked. Changes are treated as just another iteration.

The panel at SSP focused on two approaches: internal, IT-driven products, and those developed by a third-party vendor. Larry Belmont, manager of online development at the American Institute of Physics, gave an excellent presentation on the in-house approach. His organization ran its first agile project with a timeline measured in days rather than weeks or months.

Leigh Dodds, CTO of Ingenta, provided the vendor perspective, and described the principles of a formal type of agile development known as Scrum.

The panel was, to their credit, enthusiastic about the approach, but agile development requires commitment and is not right for every
organization or project. Some caveats that need to be emphasized:

  • Short development cycles come with a price: you will be asked to review and comment on small pieces of the larger project, and be involved on an almost daily basis. Many publishers need vendors they can treat like plumbers: “I want a new sink put here, it should look like this, call me when it’s done.” If someone in your organization isn’t prepared to think very hard every day about copper pipe fittings, agile isn’t right for you.
  • Project managers must be empowered to make decisions. Whether the project is in-house or vendor-driven, every day the PM will be asked to make calls without appealing to higher powers. When editorial buy-in is required, or when the product needs a larger review, consider a hybrid approach: appoint a single decision-maker with deep editorial knowledge to work on evaluating, testing and approving each iteration, but use a more traditional alpha/beta/gold release process for the wider group.
  • Product features may change, but time and budget should be invariant. Hard deadlines might seem to be antithetical to the free-wheeling, change-friendly agile approach, but in my experience they’re critical. They focus the entire team: key decision-makers cannot spend weeks in committee, IT personnel don’t fear the “death march” project with no end in sight, and it’s more difficult to introduce budget overruns that cause friction with management and vendors. If an agile project
    does run out of time, you will still have a launchable product that’s been thoroughly tested and reviewed all the way down the line, not something just out of beta with weeks of QA ahead. Many agile methodologies use the hard deadline, or timebox, as the primary method of structuring the project.

“Release early, release often” can sound a lot like “throw whatever we’ve got out the door.” This is one reason why the iterative approach has been so embraced by Web startups: each small release has been thoroughly tested and evaluated, and there’s never a moment where the software doesn’t work. It’s possible to to go live with a project that might not be “finished” according to the original master plan, but might otherwise be caught up in insurmountable technical hurdles or tied up in editorial review.

If publishers are going to be ready for an “iPod moment,” this kind of flexibility and responsiveness is critical.

tags: , , , ,