I've told these stories many times in person but not on NYCB (on the other hand, in NYCB you can see the stories develop as they happened). A while after working on Beginning Python my agent approached me and wanted me to meet Michael Loukides, an O'Reilly editor who was looking for someone to write a Ruby Cookbook. This was almost exactly 3 years ago. Mike was in San Francisco for a geolocation conference. I agreed to do the project despite not really knowing Ruby at the time. A major new O'Reilly book is not the kind of opportunity an up-and-coming writer passes up.
The pitch!
In March 2006, while Ruby Cookbook was in the editing stage, I had some phone conversations with Michael about doing another O'Reilly book. He tried to get me interested in various projects that existed in potentia, crystallized from some neural net in Tim O'Reilly's brain. However I'd already done such a project and I wanted the next one to be my own idea.
Lots of people have ideas for technical books they want to write (I've heard many pitches myself), but the animal-cover part of O'Reilly is pretty conservative and won't do a book project unless there's a good-sized market for what the book is talking about. At one point I suggested an in-depth book on ncurses programming, but that never happened because it would have sold about four copies.
Then I came up with the idea for RESTful Web Services, which was better, but in March 2006 still kind of a hard sell. Michael's initial response was: "I agree it's a book we need; I don't think it's something we need quite as much as Scriptaculous and Prototype". I argued for starting the book ahead of the curve, but see above re conservatism. Ultimately Sam and I got the book deal, as you know, but I think that's the reason underlying the drama we had in November with some factions wanting to stick "with Ruby" on the book's name.
Time management
Near the start of Ruby Cookbook I went from full-time at CollabNet to working three days a week. I was really bored with my job but I didn't have the confidence or the money to just quit. Three days a week at the job worked out well for my writing, and to this day 25 hours a week is the maximum amount of time I prefer to spend in the corporate world.
My one piece of advice, if you'll only listen to one, is to rearrange your job so that you have one or more days off every week to work on your book. The writing will go faster and you'll take some of the pressure off your evenings.
Later in 2005, Sumana moved in with me and eventually I felt like I could quit and work on Ruby Cookbook full-time. That's also how I did RESTful Web Services, and it's by far the best arrangement if you can manage it. Writing a book is approximately a full-time job, so treating it as a full-time job lets you live a normal life.
Outlining
As a book with 350 short "chapters", Ruby Cookbook had a pretty detailed outline from the start. My daily goal was two or three recipes, depending on how many I needed to meet the next deadline. Longtime readers may remember recipes going green on my webpage for the book as they came in. Because the project was so clearly specced out, I sometimes had the luxury of being done by 2 in the afternoon and being able to take the rest of the day off. Ruby Cookbook is the only book project where that happened.
Baron suggests approaching the prose of your book by outlining in more and more detail, rather than jumping into prose that you'll always be mentally editing. That's a good approach, but my outline for RESTful Web Services never got that detailed. old_outline.txt is 5000 words. Some parts (Introduction, Chapter 1, Chapter 8) are very detailed and the rest is pretty skeletal. To keep a steady flow, if I felt myself unable to express some thought, I would just stick a TODO in the text, like when coding.
External dependencies
I was able to get a small budget for paying Ruby Cookbook recipe contributors, I think by giving up part of my advance. Contributors were my major external block. I started recruiting contributors as soon as I announced the project and my goal was to get all the contributions in before the second-to-last deadline, giving me a buffer time of two weeks to write any recipes that contributors couldn't deliver. This did not work completely, but it did keep the last-minute scrambling down to one or two instances.
For RWS we had some coauthors in the last chapter talking about Django and Restlet. I don't remember the details but there was some last-minute scrambling to get one of those sections in under deadline. I'm going to go ahead and call this a general rule.
Revision
Baron writes that he did a lot of revision and that every editing pass found a bunch of errors. I can second most of what he writes, but this section in particular rings true for me; I know too well the seemingly endless TODO list. However, my books didn't go through as many editing passes as his did, leading me to think that O'Reilly will skimp on copyediting to get the books out in time for whatever conference they've scheduled the release for. (I found this conference-centrism strange, but it shows up in Baron's entry as well: "[D]on't feel bad if the book doesn’t come to the [MySQL] conference: it will be at Velocity which is directly related, and at OSCon.")
The result is that the first printings of my books have lots of typos, as the more unkind Amazon reviewers have noticed. I've spent a full day plus a cross-country plane trip going through my copy of RWS finding missing words and other minor errata. I haven't seen the second printing yet, but it should be a lot cleaner.
Unlike with Wrox, with my O'Reilly books I had to find technical reviewers on my own. I actually think this was a net benefit for both books, because the reviewers I found were domain experts who were interested in the book, and willing to respond to requests for clarification. I wrote and rewrote whole sections of RWS because a reviewer said "you forgot this" or "this is wrong, you idiot", things I don't think a hired gun would have caught, and RWS is a much better book because of it.
The downside is that I had to coordinate the reviewers myself and there was no money to pay them (they got free copies of the book, which didn't cost O'Reilly much). There were a lot of technical reviewers who just found typos, and while that wasn't the best use of their time, I'm not going to say it was a waste of time, given the number of typos that got past everybody.
Formatting
Ruby Cookbook was written using RedCloth-like wiki markup, and kept on an experimental internal wiki called Aardvark (which doesn't exist anymore). I describe the doctest-like way I tested the recipes in Unit Testing Your Documentation. This was a very convenient format for me, but it's not the format used for the final edit or to typeset the book, which is going to cause big headaches when it comes time to do the second edition.
RWS started out being written in wiki markup and then I converted it to Docbook about a third of the way through. Docbook was great. It's like writing HTML, except the tag names are longer. Plus, Docbook is what O'Reilly uses internally, so if you're writing a book for O'Reilly and someone tells you you have to use Word, make a fuss.
Docbook did have a couple shortcomings. Although I could do a cross-reference to another chapter fine, doing cross-references to a section within another chapter resulted in just the section name with no indication of what chapter that section was in. (This was a problem with O'Reilly's stylesheet; you can see a couple instances of this problem in the first printing of RWS). And I apparently use too many footnotes. But the major problem was the code samples.
Code starts to decay as soon as it's taken out of a file that can be executed as code. That's why I wrote the doctest-like program that executed Ruby Cookbook entries. But Ruby Cookbook looked like a set of unit tests: a bunch of self-contained little demonstrations of what you can do with code. The code in RWS looks like integration tests, full of interacting parts. There's a whole Rails application in there. And Docbook is less flexible about inserting code snippets than the wiki markup was.
So instead of putting the code in the text, I left the code alone and wrote a preprocessor that folded it into the text as necessary. Here's a random example, from my version of chapter 7 (implementation.xml.in):
<para>
At this point I know enough about the dataset to create the
database schema (see <xref linkend="bookmark-schema" />). I
wrote this file as
<filename>db/migrate/001_initial_schema.rb</filename>, created
my <literal>bookmarks_development</literal> database in MySQL,
and ran <command>rake migrate</command> to create the database
schema.
</para>
##ruby/bookmarks/app/db/migrate/001_initial_schema.rb|The bookmark database schema as a Rails migration|bookmark-schema
<para>
Now I can create the database schema by running this command:
</para>
(You can see one of those problematic cross-references in there, though that one's OK because it links elsewhere in the chapter.)
My preprocessor goes through a .in file and replaces that double-hashed line with a Docbook example:
<example id="bookmark-schema">
<title>The bookmark database schema as a Rails migration</title>
<programlisting>class InitialSchema < ActiveRecord::Migration
...
</programlisting>
</example>
Some files are explained in multiple sections. If you look at the RWS sample code you'll see some lines that just have a double hash.
example 1
more example 1
##
example 2
more example 2
The preprocessor stops the example at the double hash, and stores the location within the file for the next time the .in file asks for an example from that file.
Once the preprocessor runs, I've got an implementation.xml file that unifies text and code, and my book.xml file sews all the chapters together. Not as clumsy or random as a Word doc; an elegant toolchain for a more civilized time. The files did get a little out of sync near the end, in the final editing stage, which again will cause headaches come second edition time. But it won't be nearly as bad as Ruby Cookbook, and honestly most of the code in RWS will need to be replaced anyway.
That sordid subject, money
I'm not comfortable going all John Scalzi on you and showing you my royalty statements, so let's talk in generalities. I earned out my advance for both Ruby Cookbook and RWS in the initial buy. (The technical term for this is "my advance was too small".) That's Amazon buying a bunch, and Borders and B&N stocking a copy in each of their stores. And Waldenbooks, I dunno. The initial buys are probably the biggest sales I'll ever make, and they're almost entirely due to the O'Reilly brand name.
My advances were in the single-digit thousands of dollars. Subsequent to earning them out I've earned single-digit thousands of dollars for each book, quarterly. I'm the primary author on both books and I get the majority of the royalties. If I let someone else do the second edition, my share of the royalties will go down.
It's problematic for a number of reasons to try to convert book royalties into an hourly wage. One big reason is that much of the income hasn't come in yet, so who knows how much it's going to be and the time value of money etc. Each of my books took about a year of my life. My estimated earnings are more than the sub-minimum-wage figure commonly thrown around, but they're a lot less less than what I'd have earned working for those two years as a programmer. In fact, it's less than I earned at my first real programming job back in 2000.
Now, these are incredibly successful books. RWS is a year old and has an Amazon sales rank between 3k-5k. Ruby Cookbook's rank is between 20k-40k at two years old; a year ago it was 4k-10k. Beginning Python probably had just as many man-hours devoted to it, but it never cracked 14k. The fundamental author's fallacy is to make a connection between Amazon sales rank and number of sales, but think about what sales rank really means: that's the number of books that are doing better than mine on Amazon. That's approximately the number of people in the United States who are making more money from their books than I am.
So, as I said in in another context, writing is not the most cost-effective use of your time. But unlike writing science fiction, writing technical nonfiction can help your career in other ways that Baron and others have covered ably.
The wall
I want to write something about the feeling I get halfway through a book where I'm just sick of writing, but I've been writing this big weblog entry and I'm... sick of writing. So I'll just mention it. There's a point where you think "what the hell, why am I doing this, this is killing me", but at that point you've signed a contract and how are you going to feel if you flake out. I imagine some people actually do flake out at this point. To get through this I find it helps to get into the submarine mentality, and to not have a job that's killing you on top of the book project that's killing you.
![[No comments]](/nb//themes/Default/no-comments.png "No comments")
Fri Jul 04 2008 19:01:
Excellent! The car from "One Piece at a Time" was actually built as a publicity gimmick, by a guy at this company. However, according to a Wikipedia talk page, surely one of the least reliable sources imaginable, "the car pictured never ran". 
![[Comments]](/nb//themes/Default/comments.png "2 comments")
Happy solstice. My plan was to show you a picture of this can of Malaysian shaving cream we have. Sumana got it from a friend who was moving back to Singapore, thinking I might like it. I don't like it because it's lemon-lime scented. It's shaving cream that smells like something you'd eat. I'm pretty sure they eat limes in Malaysia, so it's not a cultural thing. What the hell, folks. 
