A Conversation with Joel Spolsky
What it takes to build a good software company
Joel Spolsky has never been one to hide his opinions. Since 2000, he has developed a loyal following for his insightful, tell-it-like-it-is essays on software development and management on his popular Weblog “Joel on Software” (http://www.joelonsoftware.com). The prolific essayist has also published four books and started a successful software company, Fog Creek, in New York City, a place he feels is sorely lacking in product-oriented software development houses.
Spolsky started Fog Creek not with a specific product in mind, but rather to create a kind of software developers’ utopia, where “programmers and software developers are the stars and everything else serves only to make them productive and happy.” So far, he has succeeded. The company has maintained a 100 percent employee retention rate while shipping several profitable software products. Its latest release, Fogbugz, is a comprehensive, Web-based project-management system that uses a technique called EBS (evidence-based scheduling) to help software developers better predict their release dates.
For this month’s interview, Queue editorial board member Ben Fried from Morgan Stanley sat down with Spolsky in Fog Creek’s snazzy Manhattan offices and discussed, among other things, how EBS works, the challenges of Web 2.0, and why the military doesn’t breed good software managers.
BEN FRIED You did a lot of things before you started Fog Creek. What brought you to this point?
JOEL SPOLSKY I started my career at Microsoft and really enjoyed it. I would have stayed there for a long time except that I didn’t like living in Seattle. I had a lot more friends in New York, and life is more exciting and more dynamic here.
I came back to New York and sort of bounced around to various places, all of which were somewhat interesting, but I was frustrated by the lack of a real technology product company in New York. There are definitely lots of places to do technology in New York. Most of them are banks. The next large category is probably the entertainment industry. And then there were dot-coms for a while. But for a software developer, there’s nothing more exciting than working at a software company, and I couldn’t find a company that was close enough to being a software company, that was organized around making software developers happy. So, sort of lacking any place to go to work, I started one. I think if Google had gotten to New York a few years earlier, I might have just gone there.
BF You’ve spoken eloquently about how you view it’s your mission to create a utopia for software developers and that the things that come out of it will naturally be profitable. That sounds a bit like Google.
JS From what I’ve heard, Google does a really good job at some of the things that I preach. That said, there are some things that Google has forgotten. For example, I think Google is a little bit too diversified, that it has too many product ideas going at once. That 20-percent-time thing—it’s a nice idea in theory, but it generates too many ideas for one company to follow up on. And it’s hiring way too fast. The people at Google probably don’t think this, but they are certainly hiring engineers too fast to keep a single, unified corporate culture and shared values in a way that would be productive.
BF Weren’t you a sergeant in the IDF (Israel Defense Forces)? There are those who might say that being a paratrooper would help with being the CEO of a software company.
JS Occasionally I have thought back to the way in which the army creates cohesion and leadership in military circumstances and considered whether it might be applicable in the civilian world. But I discovered recently that it just wouldn’t. The military has a problem that other institutions don’t have in leadership: specifically, it needs to be able to give people commands to do things that are suicidal and to have those commands instantly obeyed. With that in mind, it has a very specific form of discipline and a very specific form of training. If you tried to do that in a software company, everybody would just leave and go to another company.
BF I think a lot of people are interested in understanding the difference between big and small companies, and in particular, between a software product company and other kinds of companies. You’ve done big, you’ve done small, and you’ve created your own company, so how would you differentiate them?
JS I really feel that for somebody who loves software and is a good developer, there’s a lot of benefit to working in a software company, mainly because the things that you know about and are good at are directly aligned with the things that the business is good at and the kinds of things that you get promoted for. But, actually, banking is sort of interesting, because when it comes right down to it, 95 percent of what you’re doing actually is information technology in a certain way. Even a derivative is a form of code in some ways, although there’s a trading aspect to it as well. So it’s actually conceivable in the banking industry that somebody who started out as a developer might be promoted to the point of actually running a bank—and that has happened. So, you don’t feel completely misaligned.
On the other hand, if you go to an entertainment company as a software developer, such as when I was at Viacom working on MTV, or any other kinds of companies such as insurance, you’re really a subcontractor/service provider. The things you think about every day are not the same things that the CEO thinks about writ small; they’re actually a very different kind of thing.
BF You’re not doing the thing that makes the company money.
JS Right. You’re usually in a cost center, not a profit center. Again, there’s a slight difference with banking, because with a bank, if you can calculate something faster than everybody else can calculate it, you become a part of the profit center. But I find that, inevitably, as a software developer, you are just going to be happier in a software company.
BF What about big versus small? When you worked at Microsoft, it had 8,000 or so people.
JS I think when I first got to Microsoft as an intern it was 5,000, and it was up to about 10,000 when I became full-time. Now, there must be 3½ billion or something. There’s a very different flavor to the place, and it’s really weird to me. At the time I appreciated the benefits of a big company. It felt like there were more interesting things to do. You could always jump to a new job and stay within the same company. If you were bored with what you were doing, there were lots of other jobs that you could move into.
On the other hand, there’s something really weird about being a little gear in a very large project and not really being able to describe to anybody what you do, not really being able to influence or affect the way things happen. Pretty much 95 percent of everything you do at Microsoft is in some way feeding the beast. You’re working on Microsoft, not your product. You’re not directly causing your product to become better. You’re trying to create the meetings that will cause the things to happen that will cause at some point some part of some product to be better.
BF I read someone’s blog about how many people and how long it took to design one element of the shutdown menu.
JS There’s usually a very large number of people meeting constantly to argue about some tiny thing, and then they get it wrong because it was all designed by a committee.
Basically a small company has a flavor to it, whereas a big company is sort of like checking into the Bellagio in Las Vegas. It’s a nice hotel but it has 5,000 rooms, so don’t expect anybody to remember your name. A small company is more like a bed and breakfast. You’re going to have a great time because you get along with people and it’s a much friendlier experience. You don’t really mind that the bathroom is down the hall because the people made a special vegetarian meal for you and then showed you around town. On the other hand, you might be at a bed and breakfast where they have weird leather implements and lots of cats.
BF Tell us about the experience of growing your company. How has what you do changed over time as Fog Creek has grown?
JS Basically, what I do at any given time is whatever new thing Fog Creek suddenly has to do as we get larger, and nobody is around to do it. The difference between a small company and a large company is that at a small company, you still have to do all the same things, but you have to do them less. So you need somebody to do, let’s say, marketing, but it’s really only .03 percent of a person. You need somebody to take out the garbage. You need somebody to do the bookkeeping, but you don’t really have a full-time bookkeeping need yet; even to this day at Fog Creek, bookkeeping is about one day a week for one person. So you can’t really hire a bookkeeper at that point. For a while we didn’t have an office manager or anybody to answer the phones.
As you get larger, those tasks start to hit the point at which you can hire full-time people to do them, and the CEO stops doing them. Now, on any given day, we actually have at least two full-time people who will answer customer e-mail, answer the phones, talk to customers, follow up with customers, that kind of stuff.
For a long time, one of my activities was public relations—marketing, advertising, and raising awareness of Fog Creek. I did that by writing “Joel on Software.” Now, that wasn’t really the goal of “Joel on Software,” which, indeed, preceded Fog Creek. The classic “Joel on Software” articles were all from the summer of 2000 before I started Fog Creek.
I’ll keep that kind of ticking when I have time. Hopefully, if I’m lucky, I’ll be able to make a few more key hires and get all the day-to-day work covered so that most of what I’m doing is building things.
BF Speaking of hiring, something that you touched on about recruiting has resonated with a lot of people: “Smart people who get things done” has become kind of a mantra for a lot of people who hire developers. What specific qualities or aptitudes do you look for when recruiting for Fog Creek?
JS I look for lots of things. For example, I don’t care if you’ve never used pointers or you’ve never used recursion or you never needed to know pointers or recursion. I have just found that being good at those things happens to correlate with being good at all kinds of more substantial problems.
My theory for why that is, is that in order to do them well and understand them, you’re required to think about a problem at different levels of abstraction simultaneously. That is an aptitude that most people don’t have, and lacking that aptitude, they’re not going to be good programmers.
It’s a good thing to learn, a good part of your brain to exercise. And it may well be the case that these are things that can be taught to people even without that aptitude if they’re willing to work hard at it.
BF But it’s not a critical thing in itself?
JS As a programmer, I feel it’s great that we don’t have to think about pointers anymore, and the types of algorithms that we have to write actually rarely use recursion that much anymore. But it makes it a little bit hard for me recruiting because I don’t have that capability that used to exist that would weed people out of computer science if they couldn’t make it through learning how to program in Scheme and doing data structures with pointers in Pascal. Now when I ask somebody to demonstrate knowledge of recursion in an interview, they may fail to be able to do that, not because they’re dumb or because they can’t do recursion, but because they’ve never been taught recursion.
BF What are the things that developers need in someone to manage them effectively?
JS That’s a good question. I always think of it as more of a support role—like moving furniture out of the way so they can get things done—than an actual leadership role. Some of the support that developers may need is, for example: “We need to resolve the text of this error message.” This is not something that computer programmers are necessarily good at, so just give them the error message and let them move on to the next thing.
To the extent that managers can actually make decisions for programmers, they are very low-level, mundane decisions that allow the programmer to move on to the next task. In terms of decisions about algorithms, about the big picture, structure, code, it’s probably a very bad idea for management to make those decisions. I see two roles for a manager of developers: number one, keeping a useful and assorted queue of things for the developers to be working on, so that they can pick things off the top of the queue and do them next; and, second, basically being there to answer developers’ questions.
BF You’ve written a few books in the past few years, and I see many books piled up in your office. What are you reading now? Anything to recommend?
JS There aren’t many books on the state of the art in software development and software management right now, which are the kinds of things that I like to write and read about. Unfortunately, we haven’t moved beyond the anecdote phase, and attempts to move beyond the anecdote phase are usually just anecdotes with statistics.
Part of the problem is there really isn’t a science going on here, which is very frustrating. The same applies to an awful lot of business writing. It’s very easy to write a book called, for example, The Starbucks Principle or The Dell Way, and just bring up a whole bunch of random anecdotes and somehow tie them together thematically and pretend that this is a real thing that you can do and be successful at.
And, lo and behold, another moron then is going to try to attempt the same thing in his or her own company. It doesn’t work because it doesn’t apply or because it didn’t work in the original company, either—it’s just an anecdote that somebody pulled out of thin air. That’s one of the problems that this particular field has been suffering from.
What we do have are anecdotes from the elderly, people like me, or even the truly elderly, the Gerald Weinbergs of the world, writing brilliant things. Timothy Lister, Tom DeMarco, Ed Yourdon—these are people writing, “I’ve been here for a long time. I know I’m a curmudgeon, but let me tell you young folk what it’s like, da da da da da da da. Here are some examples. What this showed was da da da.” If you read a bunch of those anecdotes, you actually may learn something. That’s sort of the oral knowledge of our field.
On the other hand, we don’t really have enough statistics. For example, I’m a strong advocate of private offices with doors that close, and I can give you two reasons why. One, it’s easier to hire people. All else being equal, the good developer would prefer the private office. Number two, fewer interruptions mean the programmers are able actually to get work done.
BF But there is actually a lot of math to support that—statistics, or let’s call it pseudoscience.
JS Pseudoscience, right. But if you try to find an article where somebody actually used a team with doors closed and a team with doors not closed, there’s only one example: a weekend coding experiment done by Lister and DeMarco before they wrote Peopleware, where they actually tried both.
BF I think if you look at the field of knowledge work, there has been a lot of work done in interruption management and attention management with no interruptions for flow and so forth.
JS I’m not really sure how scientific it is because there are all kinds of effects to consider. There’s the Hawthorne effect, which makes it impossible ever to measure any of these things properly. There’s also enormous variation in different people in software development. Sometimes one person may be able to accomplish something in three weeks that somebody else would take a year doing. But that other person might be able to do a different thing in three weeks that the first person would have taken a year to do.
Not only are there enormous differences in productivity, but there are enormous differences in personal skill sets and in personal aptitude for certain things. And the work we’re doing is just very weird and not easily controlled in an experiment. To do controlled experiments is almost impossible.
There’s not a whole lot of data out there. The only evidence there seems to be is that it’s very easy to find orders of 10 differences in productivity, and nobody knows why, and being faster or slower doesn’t necessarily mean you do a better or worse job.
BF You’ve been talking about something called evidence-based scheduling in Fogbugz. Is that related?
JS Not exactly. It’s actually more like the risk management that you guys do in the banking industry. In evidence-based scheduling, you come up with a schedule, and a bunch of people create estimates. And then, instead of adding up their estimates—instead of taking them on faith—you do a little Monte Carlo simulation where you look at what speeds developers had in the past, vis-à-vis their estimates. You use that same distribution of probabilities, as we call them, that you had in the past and run a simulation of all of your futures. What you get, instead of a date, is a probability distribution curve that shows the probability that the product will ship on such-and-such a date.
The interesting thing that I discovered is that people often underestimate. But you might think they overestimate, too, so if you have 10 things to do, just add up all those estimates and some of them will be late and some of them will be early, and you should still end up with the same date. You’ll make up for the late ones by doing other things early.
But when you think about software tasks, when things go wrong, they take three or four times longer than expected. If I told you I’ve got an eight-hour feature, it’s about eight hours of work. Now, could it take eight hours? Yes. Could it take six hours? Sure. Could it take two days? Definitely. Could it take a week? Yes, probably with a 10 percent probability because you’ve discovered some huge problem, and it’s a new thing you have to write code for, and you just completely forgot about it and it’s going to take you a week.
Now, could it take zero? No, that’s impossible. Could it take negative 32 hours? It could never take negative 32 hours. You can go over 32 hours; you just can’t go under by 32 hours because that would cause you to go backwards in time.
BF It’s almost all downside, no upside, right?
JS To the extent that it’s true that eight hours is the average, it’s got a long tail to the right.
BF It’s infinite, or virtually infinite, probability in some sense, right?
JS Yes, there are some eight-hour features that could take six years with some probability. But it does have that very long tail to the right, and that means that actually if you add up two features, eight hours with 50 percent probability plus another feature that’s eight hours with 50 percent probability, you can do it in 16 hours, but not with 50 percent probability. Probably with only 30 percent probability. The most important observation here is that even if you’re very good at estimating the 50 percent point, it is mathematically incorrect to add two estimates to get the estimate of those combined features. You have to do something like evidence-based scheduling, where you use historical data—and this is actually risk analysis.
BF Since you’re passionate about how people interact with computers, I’d like to get your thoughts on the current buzz about Web 2.0 user interface technologies. Has this stuff made our lives better or worse?
JS It has made life a lot harder for programmers, but it has probably benefited users when the programmers do it well. When the user interface works in the way that it’s expected to, and the user model corresponds to the program model, then it’s great for users.
BF You’ve written that Windows desktop development is pretty much dead. Do you still think the Web is good enough for most of what people want to do?
JS The Web is good enough and getting better. It’s just really unusual to see people writing GUI apps anymore.
BF What about the world that Adobe now inhabits with Flash, and now Adobe’s Apollo and Microsoft’s Silverlight? These are ways of writing something that runs in a rectangle in your Web browser in some slightly better way than HTML does, right?
JS Or outside of your Web browser.
BF Yes, Silverlight lets it be in your Web browser, and Apollo seems to take the thing that in Flash would have been in your Web browser and puts it on your desktop. Should they have just left us alone with the tools that we had before, or is this actually better technology?
BF To wrap things up, I’d like to talk about what you’re perhaps best known for: “Joel on Software.” Although you don’t agree with the term blogger, you’re widely considered to be a thought-leading blogger who helped define what blogging is in much of the tech world. Does it help you think better? Does it help you run Fog Creek better?
JS In general, I don’t think that blogging or writing personal essays is necessarily the right thing for any CEO to do. On the one hand, I’ve done it and it has benefited Fog Creek in the sense that we have gotten a lot of awareness of our products out there. On the other hand, an awful lot of people have said, “That’s a good idea. I’m going to do the same thing.” I don’t want to say that I necessarily inspired them, but they seem to be attempting to do the same thing as I did writing about their own software, and they’re just not getting that same audience. I don’t know why that is, but they’re not, and it’s not working for them so well. A lot of times they just give up.
There’s nothing wrong with a million blogs for your friends and family, and there’s nothing wrong with one in a million blogs occasionally saying something interesting and getting its 15 minutes of fame. But we’re not going to have a million software blogs out there that are actually read by a million people a month. Not that there aren’t better bloggers than I am out there right now, languishing in obscurity.
Originally published in Queue vol. 5, no. 5—
see this item in the ACM Digital Library