Intro
David: That is the pleasure and privilege of working with the web. No one knows what you built it in. You could build it in Basic, you can build it in Ocaml, you can build it in Haskell. You can build it in whatever, Ruby. No one is going to be none the wiser. You get to choose. You want to write for the web? I mean literally every programming language that’s ever been invented and known to humankind is serving a web page somewhere.
Adam: Today’s interview is with David Heinemeier Hansson, DHH, from Ruby on Rails fame. It was a really fun interview to do. Some of the things we talked about include let’s say you like an obscure programming language, how do you make it popular? How do you use it to get work done? It doesn’t have the libraries you need, how do you handle that situation?
Also talk about current best practices that seem crazy, ways of building things that seem complex. David is definitely of the school that you shouldn’t play along, you should call things out if you think that they are overly complex or not productive.
If you are familiar with David, you probably already know that he is very passionate. He’s also not afraid to drop some F bombs, so there is some swearing in this episode. Also a lot of really fun, controversial, and contrarian ideas. Enjoy the interview.
Hey, how’s it going?
David: All right. I’m all set, ready to roll.
Why Did Rails Succeed?
Adam: So I think Ruby on Rails, it changed the world in some ways, at least my very small world of software development. I recall years ago going through some tutorial that I think was like Build a Blog in Ruby on Rails, and I was a C Sharp developer at the time. It was kind of mind-blowing just how quickly you could build things. Why did Rails succeed?
David: That’s a good question. I think a lot of it was just the right time. There were just a bunch of things that had gone before the stuff that I’d made that essentially provided the foundation, but that I felt was just trapped. So I drew my main experience and influence for Rails from two camps, one was Java and the other was PHP.
And I thought that Java was simply chalk full of clever people who have clever ideas working in an absolutely terrible development environment. So if you could take those clever ideas and as I like to call it liberate them from the environment, you’d be left with some really clever ideas. And they would be much more appealing. They would be much more broader distributed. They were just very inaccessible for people who weren’t already deeply enmeshed in that world.
So that was where I drew the other inspiration, which was from PHP, where you literally could do a one line thing that said, “Print hello world,” and it would show a web page. It would show Hello World on a web page. You just drop that file into the correct folder, and you were on the web. I mean just such an incredible I think to this day still unsurpassed ease of Hello World.
So I thought, “Hey, if we could take the great ideas around frameworks and how to structure larger software applications, and we could infuse them with that ease from PHP, there’d be a really interesting combo here. And that was really what I did, right? And then the conduit for that was Ruby. That was the sort of third missing element, the third secret spice to the dish here, was that Ruby’s simply an incredible programming language that enables you to have the best of both worlds, that immediacy then ease of introduction, to the ease of use. The print Hello World is almost as easy, not quite because it’s not targeted just at the web yet, it’s a language capable of carrying the ideas from something like Java.
So that’s really all I did. I took those three elements, put them in a pot, and stirred it real good. I was building base camp at the time. And I needed practical things, right? And I think that that approach to it, the builder was also the tool maker, gave it a unique feel that wasn’t true in all other environments. Especially in the Java world, there were at the time a lot of this architecture thinking, that people would be architects, and all they’d be doing, they’d be doing frameworks and they’d be doing deep thoughts about how you should build applications. They wouldn’t actually necessarily be building those applications.
And the outcome was sort of what [inaudible 00:04:40] used to call architect astronauts, that they would get so high up in the stratosphere that you’d essentially lose all oxygen and lose your mind. So my way of tethering myself to earth was to actually build with what I was constructing.
So Ruby on Rails turned out to be this really practical thing that was using influences from these three different environments. And then I had the blessing and the gift of being a newbie. I was new to Ruby when I started On Rails, Rails was basically my first real sort of project, so I didn’t know what you couldn’t do. And that went both for building something in Ruby, it went for making open source software. It went for a lot of things that I simply just didn’t know what it entailed or what the challenges were or what I was supposed to do and not to do.
So I just felt like this is a thing you can do, of course you can release a framework that does everything and is full stacked in 2,000 lines of codes, and you should show it off to the world. So I was influenced certainly by the open source community at large, but I was also repelled by it in some instances. It was very intellectual when you were pitching your software like, “Here are the trade-offs, here are the objective measures for why we’ve done it that way,” where I try to convey an emotion. What does it feel like to program in this environment? What does it look like? What does it taste like? What does it smell like? Invoke all the senses, not just your prefrontal cortex. Go wider, dig deeper into all the elements of being a human because I think if anything, Ruby’s greatest insight was the programmers aren’t just programmers, they’re also human. And I try to follow in Ruby’s footsteps of Appealing to the whole person to sort of a deeper humanity in programmers.
And for reasons greater than just let’s make software, right? Like let’s make software that we really enjoy making, where the art of creating the software is a pleasurable experience in and of itself that can supersede and be more important than all these other attributes we keep talking about, whether that’s execution speed or memory usage or all these sort of objective terms that feel very concrete and very real. Ruby was incredibly daring I think by going out and saying no, the most important thing, the most important design criteria for our programming language, is going to be programmer happiness. I mean what a hippie concept.
I showed up here and it felt like Ruby heard me, Ruby saw me, Ruby was sort of speaking to a deeper aspect of my being, almost communicating with my soul. Now it’s really getting hippie. And then Ruby sort of took my hand and showed me just how deep the rabbit hole could go. And by the end of it, I came out on the other side loving programming, loving the creation, loving the dance with the syntax and the subtleties of the language and going like, “This is what I want to do with the rest of my life.”
Why Ruby?
Adam: Out of your three components, Ruby definitely seems like the most unusual. I had never heard of it before Rails, I don’t know if I would’ve heard of it if not for Rails. How did you come across it?
David: Yeah, so when I first got introduced to Ruby, it was absolutely this esoteric, very lightly-used programming language, at least in the West. In I think it was 2003, I picked up a couple of these programming magazines, I think it was one from IEEE and maybe one from ACM or something else like that.
And one of them had an article by Martin Fowler, where he was explaining some pattern. And he picked Ruby to explain this programming [inaudible 00:08:31]. And he said something in the intro akin to, “Hey, I don’t usually get to programming Ruby professionally, but I’m just explaining a concept here. Ruby is very close to pseudo code, so even if you don’t know Ruby, you’ll be able to follow along.”
And then he showed his idea and I thought like wow what a beautiful language, what an interesting … this looks very different from what I’ve done before. And then I think it was Dave Thomas who had another article in the other magazine where he was also talking about some programming concept or another, but he also picked to use Ruby. And I just got inspired by the fact that here are two of the great heroes of the programming world. When they are free to pick, they pick Ruby. So I thought to myself when I had an opportunity to pick whatever programming language I wanted to use in, I should probably listen to these guys, they sound pretty smart.
So I think that initial interest, or that initial exposure, just sparked something in me, and I quickly tried to learn everything I could about Ruby. And shortly thereafter, I think maybe 2004, I went to … it was called the Third International Ruby Conference I think, which was essentially an American Ruby conference. And I think there were about 42 people there, and I was giving a talk on Rails, at that point I had worked on it for a bit and I was giving a preview of it.
And at one point I asked, “So how many people in this room get to work with Ruby professionally?” And I raised my hand because I was actually getting paid to work on Ruby, and I think there was one other person who raised their hand. So this was like 2004, the main Ruby conference in the West has like 42 people there, and like two people get to raise their hand how many work with it commercially. And I think just a few years later, we’d be like 2,500 people at the Rails conference in whatever city that was.
So it was kind of a wild ride, and it was fun, and I look back now I’ve worked with Ruby for what is it, like 17 years now, and I’m as excited as I was when I discovered Ruby 17 years ago. It is just truly uniquely beautiful, engaging, rewarding programming language, and I feel just utterly blessed to be able to still work with it. It’s just a treat.
The Benefits of Obscure Technology
Adam: So you went to this conference, and these people were interested in the language but weren’t using it professionally, but were excited about it. Do you think that there’s a broader lesson there? Like if I picked a language today that people are very excited but aren’t using in their work, that that’s a sign that there’s something there?
David: Yeah, I think it’s always a great signal when you have people who are so enthralled by a technique or a tool or a language or whatever that they’re willing to just do it for the fun of it. That was really what the other 40 people in that room did. They did it for the fun of it, for the intellectual stimulation, for the excitement of getting to work in programming with Ruby. That’s a very strong signal that there’s something there.
Now that doesn’t mean that that’s going to be the next big thing. There are countless, tons, of both frameworks and programming language and whatever that over time have had a very passionate small core group of people. Like you can speak to Haskell programmers or OCaml programmers or whatever who are like, “This is the best, greatest thing ever,” and it is for them. It doesn’t mean that it’s going to be a mass movement in the way that Ruby on Rails is because truly very few things become a mass movement.
It doesn’t necessarily matter. I mean I’d be doing Ruby on Rails still if we were … I mean maybe not 42, but 200 people … Well maybe not 200, but 2,000 or 10,000 people. It doesn’t have to be a million person movement for it to be a place where you can live and breathe and work. I mean I created Base Camp when Ruby had nothing, right, in terms of the tooling that I wanted and I needed to create web applications. I just created all that stuff for myself by hand, I mean it’s totally doable. That is the pleasure and privilege of working with the web, no one knows what you built it in. You could build it in Basic, you can build it in OCaml, you can build it in Haskell, you can build it in whatever, Ruby, no one is going to be none the wiser. You get to choose.
Which is a really unique advantage of the web as a distribution platform that is very seldomly cherished enough in my opinion. You look at all these other platforms. You want to write an Android application? It’s going to be Java. Well, maybe you can write it in [inaudible 00:12:56], but it’s going to compile down to Java code. You want to write a thing for iOS, well you’re going to write it in Swift. It’s just such a mono-language approach. There are a few people who do something on the edges, but not in the broad sense right? You want to write for the web? I mean literally every programming language that’s ever been invented and known to humankind is serving a web page somewhere.
I’m sure there’s all sorts of applications out there that those programming languages don’t have good access to other programming platforms, but they have access to the web.
Adam: Yeah, yeah, that’s an interesting kind of meta point, like if I find the language that’s my Ruby, I might not be able to build Rails and have it be such a big success. But I can still make a framework, I can still build my stuff in it. I could still live in that world.
David: And you really don’t need that many people for that to happen. I mean I think when we look at the modern web tool chain, it’s very easy to get overwhelmed and think oh my God I can never build that. Just look at Rails, look at the number of lines of code that’s in Rails, and you stare at that today and you think let me rebuild that in OCaml and and you’re going to go, “Yeah, that’s just not going to happen.” There’s hundreds of programming years into that.
But remember, Rails didn’t start like that. Rails was serving real business traffic on 2,000 lines of code. That’s what Base Camp launched on. Ruby on Rails I think was 2,000, maybe it was only 1,000 lines of code actually. I should know this, but it was a very small package created by one person in their spare time, and that was enough to get on the web. You don’t need to recreate the entire tool chain. In fact, in many ways it’s better to start from scratch because this is your opportunity to jettison all the preconceived wisdom of what you’re supposed to have, what you need to have. No you don’t. Can you print a [inaudible 00:14:45]? Can you send back a 200 OK? There you go, off to the races.
Fighting the Monoculture
Adam: Yeah, yeah. And you said business value, which is interesting as well, right, because I guess if I build my thing in OCaml, if it has business value and people end up using it, there’s going to be some longevity there.
David: Yes, and I think that that’s sort of the magic of programming, right? Like we get to build these things out of basically the spit of our imagination. And again with the web, it’s just you can do it in anything and no one is going to judge you in terms of at least users using it as long as it’s a good application. It really can be done in anything. And then you get to have that longevity, at least if you’re the ones working on it, right?
I mean Ruby had this argument used against it in the early days, which there’s some validity to. But for example, let’s say you build a large application in OCaml. Perhaps your talent pool for just hiring someone who could plop into that project and be productive right away is a little smaller than it otherwise been, but it’s certainly doable. And I think you should do it. I love this idea that the web is being served by hundreds of different programming languages, that I could be using this web app I really like and it’s actually written in Ocaml or Basic or Algo 624 or whatever esoteric language it could be.
There’s just something heartwarming in that, this idea of the mono-culture that this is all just a battle to the death, and there’s going to be one framework and there’s going to be one programming language that is left standing. Programmers are really drawn into that horse race. So much of their technology choices seem to be predicated on is this popular, is this going to be popular next year? Do you know what? Fuck that. Not all, not in all places, but plenty of them who have the freedom to just pick something because they want to do it. And I say to them please do, I want to see what that OCaml next framework is going to be.
I think actually one example of that that I love following from afar is Elixir, is here you have this sort of obscure for a lot of programmers at least, programming language in [inaudible 00:16:53], and then along comes Jose and says this is actually great technology, there’s some core ideas here that are really important. Let me try to shape those a little bit and make them more appealing to a wider audience. And here we go, now there’s a great programming platform and language. I look at that and just go like man that’s amazing, I love that.
Adam: Yeah, I agree. There is a certain weight to popularity, right? Like there is a certain fullness of libraries or whatever that is an advantage when you’re-
David: It is, it is, but I find that that advantage is often overstated. And the reason I say that is because that was literally the number one argument that I kept hearing when I created Rails back in the day. “Well, I mean there’s so few programmers in it.” Yeah, everything starts from a seed. If you expect that every new thing is going to be this thousand year old redwood, nothing new is ever going to happen. You have to have the seeds. You have to have the people who are willing to nurture those seeds, and accept the fact they’re not standing with the redwood right now. You can’t drive through it. It’s a small little thing, and you have to protect it and you have to nurture it. And that’s also really satisfying. There are people who really get a kick out of that, me included, that seeing my life’s work in terms of Ruby on Rails grow from this tiny little seed that I planted in the ground, the fertile ground of Ruby, but ground nonetheless and seed nonetheless, grow into it not a redwood, at least a very strong solid tree.
It’s just magic, and there are people who are attracted to that, and they’re not all the same people. Some people, “Yeah, I just want to use the thing that’s popular because I don’t care as much or I’m not influenced as much or my boss is yelling at me,” or whatever it is. There’s all these practical, reasonable reasons for why someone would just go like, “Just give me the most popular thing. That’s got to be the best thing.” And that’s fine, it’s great, it’s okay. Like Ruby on Rails is in some ways a beneficiary of that same thinking, that once we tipped over and became this mass phenomenon, there were a lot of people who suddenly had permission to use it.
And that was for a long time that driving force for me working on Rails was to make Rails popular enough that it gave people who didn’t have otherwise a choice the permission to use Ruby. That felt just like a gift to the world, and a gift that I essentially owed the world given the fact that I had found Ruby as a gift to me in my programming journey. So to return that favor to a broader audience was incredibly rewarding, and I think still one of the crowning achievements of all the work I’ve done on Rails. It’s not just the Rails, it’s the fact that so many people got to work with Ruby, and perhaps they otherwise wouldn’t have had. Maybe they would’ve, maybe someone else would’ve come along six months later and they’d created bails or whatever, something else, and it would’ve gone on to be just as popular.
But there’s certainly plenty of examples where that didn’t happen, right, like programming languages that had something to it where people saw a sparkle in it, didn’t go on to be a mass phenomenon for a variety of reasons of either timing or fit or whatever else have you. So the fact that Ruby did, and ended up being this mass language, like this mainstream language actually in many ways. It’s been used at all these internet companies that everyone have heard of. It runs Skid Hub, it runs Shopify, it runs Square. It was the foundation of Twitter, it was all these things. People go like, “Oh yeah, I know that.”
And all these companies are now, many of them, big billion dollar companies that hire tons of programmers all the time, and those programmers get to use Ruby and then there’s these entire ecosystem, what a wonderful thing.
Adam: It’s true, but I’m going to give you the negative spin on it, which is when you created Rails, there were people who were forced to do Java who didn’t want to. There’s people who are doing Rails now who don’t want to.
David: Yes, and I think that is a real thing. But I also think that’s a sort of objective truth that I absolutely recognize, and then at the same time if you were to do in on the grand scale of like the percentage of people forced to use Ruby who actively hate it. I am certain in my soul that that is a far smaller number than the percentage of people who are forced to use Java who hate it. Again, certainly there are people who will be exposed to Ruby and go like, “What the hell?”
One of the things for example is that Ruby is loosely and dynamically typed, and some people simply have their brain wired in such a way that that is an offense to their sensibilities right? And I respect that because I feel the same way about statically typed languages. I just don’t like them, and I find a lot of those people have been sort of forced into a bad situation from the start, like they joined some company that had a truly terrifying code base because some people just wrote some bad code, and they associate that with Ruby on Rails and they go like, “This is where I hate Ruby on Rails because I have to work in this shitty code base every day and it’s killing me.”
Adam: I think that’s right, yeah. Oftentimes it’s actually just dealing with a horrible code base where everything might fall over, and yeah there’s just no fun there.
I definitely find types really useful, but I do like Ruby as well. When I first encountered it, it has some things that the aesthetics of it are unusual, right, or maybe less unusual now but certainly at the time it seemed like a lot of things seemed the inverse as I would expect. Did it look strange to you, or was that just me?
Ruby VS OCAML
David: I think for whatever reason, Ruby was the programming language I’d been searching for my whole life. And when I saw it, and I saw just the layout of the statements and how it all fit together, it was just one of those eureka moments where how my brain thought about programming was expressed in this programming language. And I know that that’s A, people have that about all sorts of programming language. They go like, “Yes.” Again, we use this example over and over again, and the funny thing here is like I actually don’t even know what OCaml looks like, like I don’t even have a mental picture of it.
But I’m sure there’s some people comfortable with OCaml and just think like, “Yes, this is me. This fits my brain.” And then there are other people who sort of acquire a taste for something, that they didn’t like Ruby in the beginning and then they work with it for a while and then they do, or they didn’t like Java in the beginning and then they work with it for a while and then they do. For me, it was just this romantic moment of love at almost first sight, and then just this discussion right from the get-go of like this is how I think. It’s like if you were to do sort of the source map of my brain, it would map to Ruby.
Adam: Nice. That’s interesting, yeah. It’s funny, you just made me look up some OCaml examples. I think that you might like it because it is strongly typed, but it’s like inferred type so there’s not actually any types, and it kind of … Please stand by. I see your screen, there’s lots of lets.
David: Yeah. It’s funny because I don’t know for whatever reason I had sort of a Lisp look in my head when I said, “OCaml, this obviously does not look anything like Lisp. It doesn’t speak to my sort of heart right away, but at this point I’m perhaps also beyond the point of no return with my love affair with Ruby, I will absolutely accept that that’s a possibility.
But yeah, this is actually not bad. That looks pretty cool. Yeah, then I look at this, although this is sort of … This is one of the reasons why for a long time I thought I wasn’t going to be a programmer. The code we’re looking at now is very mathematical, like it’s about math problems. I was never interested in math problems.
Adam: For the listeners, I’ve pulled up the N Queens problem, which I think is a very math-y type problem.
David: Yeah. This is exactly why I literally thought I wasn’t going to be a programmer because I just didn’t have an interest in math problems. I don’t have any interest in algorithms beyond their utility. What I do have a deep, deep affection for is sort of the main modeling in the Eric Evans sense of the word domain-driven design. I love noodling with a business domain. I love finding just the right words. I love breaking the main model apart and all that stuff, which is all sort of logical, sort of semantical, approaches to it. It is not algorithmic, it is not scientific, it’s not math.
And I think that this was one for the longest time why I thought programming is not for me because growing up I knew quite a few programmers, and they all did the math type programming. They were demo coders or game programmers or whatever, and everything was vector this and vector that. And I would look at the code and I’d just go like, “Yep, no interest. Absolutely zero interest in that.”
And then I started working with the web and I started working with business applications and sort of IT in the literal sense of information technology and so on. And I thought, “Oh, can programming be this too? This is what I like, programming of this sort, that’s what speaks to me. That’s what I’m interested in.” And then I kind of went like, “Let me double down on that.” And then Ruby just felt like that was a perfect fit for that, right? We looked at that piece of math-y code, and the OCaml actually looked more clean somehow, it didn’t feel like Ruby is the natural language for doing math like programming.
Coding as Writing, Not Math
Adam: Yeah. So I had somebody on as a guest, and he was talking about how some people really think about coding as math, but he thinks of it as like literature. What’s your take on that?
David: Yeah, I’m in that camp. I had a keynote at Rails Com I think about five years ago where I framed all of the approach, how we think about programming, rather than thinking about like construction projects or thinking about math problems, I think about it as writing problems. It’s about being a good writer. How can you be clear? How can you be succinct? How can you structure your paragraphs into cohesive arguments that make sense to a human reader? That’s the part I enjoy, the writing part. And the rewriting part, like sort of the draft and the edits and boiling things down into logically clearer components, fitting those things together, is just like there’s a division of labor here is that I have no interest in the zeros or the one or the simpler language above it or the C code above that.
I join the stage by the time we get to the high level programming language of Ruby. And then someone else with interests different than mine can do the work beneath it.
Enterprise Java People
Adam: I think at the same time I became familiar with Rails, there was a lot of excitement about it, but also people who really didn’t like it. And maybe that’s kind of where the division was. There was people who maybe … I don’t know what they were doing, building databases or distributed systems, or they thought they were, and they were like, “This isn’t real coding.” I’m not sure, this is my impression. Did that kind of pushback happen, or am I imagining it?
David: Oh hugely, hugely. No, absolutely, it’s happening today. I’ve still literally listened to people say … Well, what do they call it? Ruby is a scripting language, and they say it in this sort of derisive tone, like it’s not a real programming language. And people like, “Well, you can make prototypes.” This is what I hear all the time, “You can make your prototype in Ruby on Rails, but once you make a real application, you’re going to have to rewrite it in something else.” And you just go like, “Dude, where have you been for 20 years? Look at the internet, you stooge. A lot of it is running off Ruby on Rails. You think Shopify is not a real application? You think Get Hub is not a real application?”
It’s just the blinders of sort of ideology, and I can see where they’re coming from because I’m a fan of ideology in many ways. I’m a fan about having values and practices and beliefs that underpin your path through life. But you got to recognize the fact that when you adopt an ideology, the trap is that it puts blinders on your head, right, and you start thinking that the values that you hold dear are these universal truths that are true for everyone, rather than just your personal truths. I’m very comfortable accepting just the personal truths of Ruby for me. I don’t need everyone in the world to believe those same truths. They’re true for me.
Ruby is the perfect programming language for the kind of work that I do using the brain that I have. That’s about as personal of a statement as they come, but there are a lot of people who sort of adopt an ideology and then they can’t square other people having a different one, right? And then the cognitive dissonance of other people being successful with a different ideology or different toolkit, like just blows their minds. They literally can’t hold that truth in their head without going crazy, so they just sort of dismiss it.
Adam: I guess their complaints have to do with trade-offs that Rails and Ruby have, right?
David: I think the vast majority of this is not based in some logical, empirical, objective evaluation of different languages. I think it’s absolutely a contrast of ideology. And if you’re a kind of person who goes like, “You cannot write high quality software without static typing,” your head will simply not accept the fact that people are writing high quality software using a dynamically typed language like Ruby. And then you need to contort your arguments to support that conclusion. Then you come up with all this other stuff.
There’s a bunch of good books about how the logical mind is actually not the one in charge. It’s our primal mind, the one that operates on emotions and so forth, that’s in charge, and that mind will come to a conclusion and then it will enlist your logical mind to come up with the justifications that support that conclusion.
And I’m as susceptible to that as anyone, but I try to sort of limit those justifications just to underpin my own personal truths. And it doesn’t always succeed, but yes.
Does Speed Matter?
Adam: You think the enterprise Java guy looks at Ruby and Rails, and he’s just like primarily angry about it, and then backwards he works and says like, “Well, it’s not as fast as what I’ve written.”?
David: Right, or that that speed matters for that application I’m working on. That is the biggest joke of all. Back in the day when the enterprise was the bar, like is this language enterprise ready, we don’t talk about these terms anymore because they’ve been ridiculed to death because the enterprise usually means nothing of significance. Like these days, it’s web scale. There’s no enterprise application that runs in a Fortune 500 company that does more traffic than say Facebook pushes through PHP or that Shopify pushes through Ruby or that any of these mega companies on the web deal with. The enterprise is just small potatoes. It didn’t used to be. It used to be that there was some sort of unique scaling challenges that the enterprise was facing, but that’s not been true for a very long time.
Stop Copying Google
Adam: I guess you’re saying performance is not a trade-off that’s being made because it doesn’t matter.
David: I mean not categorically, right? There are these cases, like there are cases where you truly hit a hot spot or a bottleneck and you want to replace that with something. It’s just that for the vast, vast, vast majority of applications, that point is so far out into the future or so imagined as it might as well be a fairy tell.
It isn’t a fairy tail, there literally are people who deal with web scale. And when you’re dealing with, I don’t know, a million requests per second, yeah you need to do different things. You can’t use off the stock software. But that was always true. Even in the Java days, you can’t use standardized tools to deal with extraordinary circumstances. You need extraordinary tools.
This is why if you look at everyone from Facebook to Google to whatever, they built their entire tool chain internally. Google built a goddamned programming language. Facebook built a new interpreter for PHP. All these web scale companies eventually end up in a place where they build their own tooling because they need to. They’re literally at the vanguard of sort of pushing things to the maximum. They’re building their own damn hardware for Christ’s sake. Facebook even is doing hardware design because that just makes sense once you have 200 data centers that have millions of computers and shit in them.
These are not the problems that you and I face, and that 99.999999% of everyone else making software today face. I’m interested in making software for the 99%. And you know what? The 1%? They can go off and make their own software, they always have.
Adam: One thing that I think is unique to the software industry is that 1%, they’re really held in some reverence, so we’re all watching what they do and trying to learn from-
David: To our detriment.
Adam: Yeah. I guess if I watched somebody build a skyscraper and then I tried to build my house using like I-beams or something, that would be a mistake.
David: Perfect example. Yes, there is a reverence, I think thankfully it is fading I think both in society at large and also in programming circles where everything that comes out of say Facebook or Google is not automatically just thought to be the greatest thing ever, or in service of the noblest aims of goal ever, I think finally.
But also just on the practical sense, trying to learn from the people who have the 1% problems and apply those to your 99% concerns, is often exactly the opposite of what you should be doing. The constraints and challenges that Google face at web scale are just utterly irrelevant to what the 99% are dealing with. Not only are they irrelevant, the solutions are literally the opposite. Simply, you almost couldn’t go worse if you tried by looking at how’s the Google search engine implemented? What kind of setup does it have? What kind of distribution does it have? And then you try to apply those lessons to how you run your little web app like Base Camp, and I mean you die before you even made it to Hello World because just the construction of that whole rigamarole would kill you as a business.
So that reverence is better thought of as like sort of a distant appreciation, not actually as a fucking guidebook. It’s actually the opposite. And then worry about the 1% problems when you’re in the 1%. It is all these embarrassed millionaires that are wondering about how to outfit their yacht while they have 30 bucks in their checking account, that I just think, “You know what? Maybe you have more pressing concerns than what marble to use on the fourth floor of your yacht than worrying about that. You should be worry about about how can I make something happen that people like and how can I get 100 people to like it, how can I get 1,000 people to like it. Worrying about what will happen once you have a billion users, I think you’re wasting your time here.”
Micro Services Are a Mistake
Adam: Yeah, but it’s hard to find where the line is I guess, I don’t know. So let me give you some examples. A company called Amazon I think kind of pioneered micro services, and my experiences with it have been that it can be a pretty useful way to do things. What do you think?
David: Is this a trap? Do you know I have a 15 minute rant on micro services sort of always at the ready in my gun holster? Yeah, I think micro services and the hype around it is probably one of the most damaging trends that has hit web development in the last 10 years. I think very few things have done more damage to sort of the integrity and the productivity of software development teams than the premature application of microservices.
I am a staunch and proud supporter of the majestic monolith, this idea that you have a single application that a single person can fully grasp, comprehend, understand, deploy, operate, and that is far, far preferable to this idea of having a fleet of microservices built in 100 different toolkits and languages that no one knows how to operate and go on.
Microservices is a great example of an [inaudible 00:37:30] tech pattern. It’s not actually a programming tech pattern. Microservices is what you do when you have teams so large that they essentially need dominion over their own domain. They need to control their own boundaries and so on. And when you have 50,000 programmers, yeah it’s a completely reasonable pattern and it makes total sense. Trying to reply while you have two, five, 20, 50 programmers? Jesus, no. Just no.
Adam: So I would agree at five, I don’t know 50. 50 is a lot of people. It can be helpful to break people up into teams, right? And then it can be helpful if those teams have a clear area that they own with clear interfaces.
David: Yeah, I think that just the boundaries are much higher. I don’t think it happens at 50. I think it happens much later in the equation and that the benefits of having a single application that everyone sort of operates on are vastly underrated, and that this appeal of the microservices approach neglects all the tremendous downsides to that approach, including the whole idea that a method call is now a network call and just the complications that come with trying to orchestrate different applications together.
And I don’t say that just out of sort of a high-minded, theoretical concern. I say it out of as a practical concern of someone who did microservices back in 2006. Base Camp actually had a number of microservices before microservices were called service oriented architectures, and it’s sort of the same thing depending on how coarsely you grain things. But we broke Base Camp up into a number of sub services, and I learned on my own body just how different a method call from a network service call.
And it’s funny because in some ways we’re traveling back in time here. When I got started with Ruby on Rails, the big thing in Java was EJ Beats and J2E, which had these remote and local services and which were essentially doing the same thing, turning method calls into network service calls. And it was a shit-show then too, and I think it’s exactly as we talked about. People looking at the big shops, “Oh, Netflix has 126 services to run their thing. Of course we should have 23.” And you just go, “No you shouldn’t. You should have one.”
React And Single Page Apps Are a Worse Mistake
Adam: That’s funny. What about another example I can think of I guess is single page applications or react, like I think Rails is in the world primarily of emitting HTML, but a lot of things are moving towards maybe a more freestanding front end.
David: First, I think tons of Rails applications are feeding Jason into react applications, and all peace be with that. That’s not how I want to build applications. That is another instance of distributed applications, where you end up having one application on the front end that talks over an API to an application on the back end, and you end up with a lot of reimplementation back and forth and a lot of contortions to make that happen. And I’m not a fan at all.
And at Base Camp, we try to not do that, and all of our approaches to dealing with the front end is in an essence ways to get out of that while delivering the same high-quality user experience. And there are, not surprisingly, tons of other alternative techniques that people can use, but I will absolutely say that react has been an astounding popularity success. I mean it’s really quite something to watch, just how sufficiently it swept the world. I think it’s more of a tragedy or a comedy than it is a documentary of how to do things well, especially once you mix it … Not so much just React in its core beautiful form of blow away the world and rerender, I actually kind of like that. It’s more once you get into all the contortions needed to make a complete application, and Redux and all these other things.
When I look at a fair amount of that code, I look at some Redux routing code and I go, “You know what? J2E was simpler than this, even at its height of complexity, even at the height of the WS Death Star for … it was simpler than this.” How did we end up here? Holy shit.
So even if I think the single page application front end is a horrifically overused pattern, far more so than even microservices, and I think the crimes against programming humanity that have been done in service of single page applications are far worse than the ones that have been at the service of microservices.
But then of course as it is, lots of people combine the two, so it’s a fleet of microservices serving a single page application, and that’s just where I go like ka-blam, my head explodes with I would rather retire and fucking make weaved baskets than deal with that shit. If I had to work in assembler or even see, I’d go like, “Do you know what? I could do something else with my life. It doesn’t have to be programming.” And if someone told me, “Do you know what? You have to build a fleet of microservices and it has to work with a single page application front end,” I could go like, “Do you know what? Farming sounds nice. Give me a rake.”
So we are different, right? It doesn’t mean that … That’s just my personal truth, as we were talking about. It doesn’t mean that that’s right for everyone, and clearly there are people who like this thing, and to each their own in some regard. Like I will continue to advocate for the fact that I think that this is bad, but that’s just my advocacy, and you have to discount that with my biases and so on and so forth. But you can use that as a counter melody to play in your head for reconsideration when you think, “Oh, let me just go with what’s popular in the industry norm. Let me startup with my micro services. Let me go for my React, Redux monstrosity,” and that’s best practices, right?
You’ll have me as a little canary singing, “No, it’s not,” or like, “That’s a dead end,” like a canary in the coal mine laying with my tongue out just going like, “Don’t go so deep. You’re going to die.”
Adam: Do you think we make things too complicated, like just in general?
David: I would even go stronger than think in that statement. I would say people make things too complicated period, and it’s tragic period, and I wish they wouldn’t period, and I’m trying to focus most of my advocacy on helping them unlearn the things that they have learned that have led them down this tragic path.
So yes, I think that programmers in particular are susceptible to the siren song of complexity, and they get this kick out of being able to master that complexity. And to wield that complexity, whether it’s appropriate or not, it feels just like a sense of intellectual accomplishment that you can figure out the most gnarly shit on the universe, like a multi-microservices-backed application serving a single page front end application, you can go like, “I am victorious. I beat the computer,” and you can feel good about that.
And I just go like, “Okay. I mean we all have our kicks. That’s not my kick, and let me tell you that there is a different way here.”
Test Driven Development Is Oversold
Adam: How about unit testing and test-driven development?
David: You’re hitting all the highlight hot buttons here. Yeah, I gave a talk, it was actually that same talk, the one we just talked about software development being writing, 2014, you can look it up on YouTube, where I call TDD the greatest diet fad that the software development world has ever seen. And I kind of labeled that from the perspective of sort of this approach of pseudoscience. TDD presenting itself as this scientific method of creating better software, and I just thought it was bullshit.
And I lived under it for a long time where I thought yes the gospel is true, TDD is truly better, and I’m a bad programmer and a bad person because I don’t get it. I wrote TDD, I wrote a lot of TDD, test-driven development. I wrote a lot of tests first, and then I wrote my code, and I didn’t like it. I thought it was not a way to fit with my flow of my brain. Like most of the time what I will do is I will explore my programming first, just sort of exploratory, I’ll figure out how it works and then I’ll write my tests afterwards.
I’m a huge believer in automated testing. It often serves the proponents of TDD to conflate those two things like, “Oh, you’re against TDD, therefore you’re against automated testing.” No you’re not. What the fuck are you talking about? I love automated testing. What I don’t love is to drive my development through my tests. I don’t live writing my tests first and then writing my code afterwards. I don’t love having my tests dictate the inner workings of my classes and of my methods to serve some sort of testability purpose.
There’s TDD proponents who simply believe that if you write code, it’s easy to test, it means it’s better code. Bullshit, no it’s not. I did a whole series together with Martin Fowler and Kent Beck on this topic back from that 2014 talk when I provocatively pronounced TDD to be dead, and I pronounced it to be dead in sort of that Nietzsche sense of the word that God is dead. Like it’s not that God doesn’t exist, it’s that he doesn’t hold the central focal point in our universe anymore. And that’s how I felt about TDD, that TDD for a while held that central focus point in a programming universe, and I wanted to kill that, or I wanted to see that die.
I just got fucking tired of listening to people tell me I just didn’t get it. No I get it, I just don’t agree. Those are two separate things, right?
Adam: The TDD advocates or whatever, they always seem a bit too keyed up or something, like it solves everything. What about just like unit tests? I believe I watched something where you were saying unit tests not as valuable as people think.
David: We usually have about half as much test code as we have production code, and that’s a ratio that’s been surprisingly stable across all the applications that we’ve done at Base Camp. Not because we designed it that way, but that seemed to just be a level of test that we sort of like to have the confidence for our code. And unit test is one part of it, but I think that oftentimes … Actually, I’ve stopped calling it unit test. What we call it now, we call it model tests because some purists took offense to the fact that when we do unit tests in Rails in particular, we often end up talking to the database, which is a big no-no when it comes to unit tests. You’re supposed to isolate all your dependencies and test only in isolation both for speed and for repeatability and blah blah blah.
And I just went like, “Do you know what? I’m not interested in that. Those kinds of tests do not reveal anything interesting for me. I want to be able to test all the way down to the database layer when I test my models because that’s where I often find the bugs or the issues or whatever, and I don’t really like mock or stuffed data, I find that it often introduces all sorts of bugs and then I need to deal with those.”
We write some model tests, and I think model tests are great, and then we also write some sort of controller tests which happen at the layer above, and those controls also do not stop anything out. They actually call the real models, and those real models call the real database calls and so on. And then above that, we write call system tests that flex the system of the [inaudible 00:48:29] and driving a browser and so on and so forth.
Sometimes you have the same thing tested in your model test as you do in your control test as you do in your system test, and other times you just go like, “You know what? The system test coverage here is fine. The control test coverage here is fine. I don’t need to replicate everything down to a unit test level because I have the level of confidence that I need or want in my code, so we’re done.”
Drop the Daily Standups
Adam: And there you go. Your book had some interesting things that were kind of like more organizational, but that I found equally shocking. Like in my work, we meet daily. We have a standup meeting where we kind of all say how things have been going, if anything’s in the way. Do you think that’s a good use of time?
David: In the broad sense? No. I think the requirement for standup, which often have these connotations of in-person, that they’re better when they happen sort of face to face. They’re not worth their price, and the price, especially for in-person standups, are synchronization of time that everyone shows up at the same time, synchronization of space that they’re all in the same location at the same time. These are dramatic compromises and costs that I could never subject our organization to. And the value you get out of that synchronized link up, to me seems very small in comparison.
The advantages of remote working, we have 56 people at Base Camp, and I think the city that has the most people in one city is like Chicago and there’s like 10, we could not operate our business like that if it was for an in-person standup. Now people do these standups, they do them over video chat or whatever. I also find that just to a lot of time be a waste of time. We don’t need a synchronous meeting that’s on the calendar every day to give status reports. In fact writing status down is a far better use of time so people can show up when they show up, they can read it at their leisure when it fits into their schedule of the day, and then you can have this asynchronous communication back and forth if there’s blockers that need to happen.
Now that does not mean that there is never a time where you should sync up synchronously and talk things through. When you hit real deep blockers where you need to do collaborative design work together, by all means jump on a video chat. I do that all the time. It just doesn’t happen every day, it barely happens every week. Maybe it happens every few weeks that we have a thing where we go like, “Oh, this is a real hard problem. We don’t want to wait for the asynchronous back and forth.” Most of our other design problems we solve through pull requests and comments, and someone will chime in on that request when they’re ready, and someone will read those comments when they’re ready, and it’s plenty good enough, and you don’t need these synchronized gears that kind of grind the whole organization to a halt unless they line up perfectly.
Adam: So I work remotely from here. I have for some time. And my team, so we all kind of do our standup on Zoom as we’re talking now. One thing that I get from seeing people’s face, like maybe not everybody’s as vocal as you are, so if I’m saying we need this new calendar thing so I’m going to rebuild this section using React in a single page app, maybe I’ll see somebody wince and then they can explain.
David: I agree. It’s important. Humans are important. Looking at humans as they move and picking up subtle cues, important. Everyday important? No. We do these synchronizations at Base Camp every six to eight weeks. So we run these six to eight week cycles, which also comes into a whole critique, I don’t know if we have time for that too, but I think sprints and the whole sprint language is absolutely a travesty and a harm to the software development community at large. This idea that you should be constantly sprinting and that sprints should only be a couple of weeks long, I think has caused immense damage and harm to software development.
Now it came as a reaction to something that I totally understood why that reaction came. If you’re used to projects that run on for years and years, you go like, “Holy shit, this feedback loop is totally broken. Let’s do the counter do that, and let’s just do it every two weeks.” I just found that doing that every two weeks is its own kind of harm.
And where we ended up at Base Camp was with this methodology we call Shape Up. We just published a web book about that. It’s basecamp.com/shapeup, the details, our full methodology, and how we approach the software development process. And that process includes that every six to eight weeks, you have essentially some cool down time where you consider what you do next, and it is in that time when we sync up and we have a chat face to face, Zoom to Zoom or video chat to video chat, where you go over what you should build next.
If you are in the middle of developing things and you just come up with, “Hey, we should build a calender in React,” your suffered decision making process is broken. If you can just on any given Wednesday come up with a whole new branch of feature, something is wrong.
Adam: Yeah, I guess my example’s bad, but sometimes people thrash, right? Like they just start working on something and they get stuck, and I don’t know. Yeah.
David: Totally, and you should followup on that, and you can. If you have someone observing the work and following along and doing [inaudible 00:53:38], you could see when people get stuck. And then you can do an intervention when that happens. But overprescribing that intervention to be something that has to be applied every single day for every single one, is to me a complete misapplication and misintervention and actually a disruption and one that comes at great cost.
Slack Is Toxic
Adam: How about just I use Slack and often pinging people on my team to ask them questions.
David: My condolences.
Adam: Not useful?
David: Harmful. I think actually the introduction of chat as a main source of company communication has probably been the most hurtful introduction in the history of interactive communication tools. And I say that as someone who built essentially a chat tool just like Slack in 2006 called Campfire, and for many years essentially ran a company on that philosophy. It’s a terrible way of working, and you don’t realize it until you take a step back and look at how your day is spent and how it’s broken up.
And the cognitive harm that happens through interruptions, that if you can’t seem to get two, three, four hours of uninterrupted time without someone pinging you, interrupting you, it’s not good. We have a whole big writeup on this that Jason my business partner just recently finished called Group Chat Stress. Again, not that chat is always bad, but that chat as a main mode of communication is a horrible interruption and intrusion on people’s productivity, cohesion, and sanity.
So you have this partial attention syndrome going on all the time because of chat, versus sort of the reaction that chat came to, which was email. Email has, in it’s sort of most naïve implication, a lot of [inaudible 00:55:34] issues. I’m not saying that email as in its base form is wonderful. But do you know what is wonderful? Asynchronous writeups of cohesive full thoughts, people using actual goddamned fucking paragraphs to describe ideas and proposals, and they put those paragraphs together into form entire cohesive thoughts, and then letting someone take that in, read those several paragraphs, sit back for more than fucking five minutes, ponder that, and then respond.
That is all lost once you move your collaboration to chat. Every person now thinks on a line by line stacodo basis. And you know what? The quality of that thinking is low. It is poor, and it has all sorts of cognitive consequences to interrupt people constantly during the day to get them to reply to this or reply to that. And I think chat has really done tremendous harm in that regards, and in some ways we’re still healing our organization from the harm that chat has influenced because once you pick up the habit of chat, once you pick up the habit of pinging someone, it is so fucking addictive. You have the slightest issue, you’re stuck for 30 seconds, “Let me ping Sam because Sam knows the answer,” with no consideration about whether Sam is in the zone trying to solve a real hard problem.
You just ping Sam. “Hey Sam, what’s up with this? Can you solve it for me?” You have no consideration about what the cost of that interruption is. And often the cost of that interruption is incredibly dear because Sam wasn’t just pinged by you, he was also pinged by Susan and Jack and a bunch of other people that you didn’t even know. And before you know it, Sam has spent his entire day just answering other people’s questions trying to get into his groove, into his loop, and he has nothing to show for it.
Absolute travesty, absolutely a contributor to burnout and all these other things. Chat is not the sort of devil in all regards, and I think especially the social components of chat can be quite helpful and useful to create cohesion in a remote team, and they’re good and you should do some of them. You just shouldn’t drip it throughout the whole day and require everyone to pay attention through the whole day and require them to be immediately responsive to someone pinging you on chat.
We have a concept for example called Office Hours. Let’s say you have someone like Sam who’s the expert in some domain, and a lot of people have questions they want to pose of Sam. Sam decides that Thursdays from 10:00 to noon, you can reserve time with Sam to ask him questions about his expertise. And you don’t pepper those questions over the whole week, you simply hold them until it’s fucking Thursday.
And do you know what? Most of the time you’ll solve your own problem sooner and you’ll learn more and you’ll be better. You have taught yourself how to fish before it’s Thursday and it’s not relevant. Or it gets to be Thursday and you get some real dedicated time where you’re not an imposition where everyone has agreed upon that this is now a good time to do it and Sam has set time aside to be available for you. And that works well.
Adam: It makes sense, but at the same time because of the way that I feel like I’m working currently, it sounds insane that I would wait until Thursday to talk to Sam.
David: Totally because we’ve all gotten addicted to right here, right now, ASAP, and that is a terrible addiction that is very hard to break. Once you’ve become addicted to ASAP, almost anything else seems insane, I could totally see that. But actually, the insanity is the [inaudible 00:58:56] that we’re in.
I love this commencement speech by David Foster Wallace that goes, “This is water,” you can look it up on YouTube, which basically goes to say that you become blind to your insane environment very quickly. To me, it’s an absolute insanity that anyone can interrupt anyone else in an entire company at their leisure with no warning, no consent, and just like because I want to. Fuck that. It is such a selfish view of the world, and I think it’s counterproductive and when it happens back to you in return, you don’t like it. People like helping other people, so this is why they always say yes. This is why you never get, “Hey, can you help me with this?”, and people say, “No, come back on Thursday.”
You need to set up procedures and protocols for this to be effective, and if you don’t you just get this selfish imposition of people stealing other people’s attention in the slight inconveniences just because it’s good for me. This is what I want right now. I want someone to help me this second. It can’t wait 10 minutes, it can’t wait an hour. I can’t write it up as a post for Sam to ponder later. I just need this right now. No, no, that’s insanity, but your reaction is a perfect reflection of how most people respond to a lot of the ideas we have about how to work. You go like, “Well this is not how I work, that seems crazy.” That’s literally the title of our latest book, It Doesn’t Have to Be Crazy at Work.
Adam: Yeah, so we’re running out of time, but yeah I really like the book. I think it’s short and pithy. And I guess the big takeaway I got was something like that, like we’re all really busy but we’re not getting anything done, maybe we should stop that.
David: Yes. Isn’t that insane? The fact that people can’t get work done at work, that so many people think, “I have to show up early in the morning, stay late at night, or work weekends, because that’s the only time I can get my actual work done.” If I’m just there in the office from 9:00 to 5:00, what will happen to my time is I spend half of it paying attention to some rolling conveyor belt of chat, then I have to spend the other half with people pinging me, and then another half, and now we’re up to three halves, that’s intentional, being pulled into all sorts of meetings for status reports. Holy shit, just shoot me now.
Adam: So I recommend everybody check out the book. Thank you so much for your time. This has been a lot of fun.
David: Absolutely, thank you for having me on and pushing all my right buttons.
Adam: That’s what I’m here for. Awesome, thanks man.
David: All right.
Ending Thoughts
Adam: That was the interview. I hoped you liked it. What did you think? Let me know. I thought that he was super interesting, and I kind of wished I had pushed back on him a little bit more about types because I really like types and I find them super valuable, and I really just think that I can convince him that they can be very unobtrusive and helpful. But I do still find microservices a valuable way to cut things up sometimes. So yeah, I don’t know.
Anyways, that was the show. Let me know what you think on these topics. Maybe you disagree with him, maybe you agree. I know a couple people who think more like him and that we’ve kind of gone into some architecture astronaut mode in the industry recently with all of this [inaudible 01:01:55] and microservices and front end frameworks. But I’m not sure, what do you think?
Sticher
Overcast
Nov 04, 2024
52
min