Learning to Think With Andy Hunt - CoRecursive Podcast

Introduction

Adam: Hello. This is Adam Gordon Bell. Join me as I learn about building software. This is CoRecursive.

Andy: As a species, we suck at estimating. First of all, let’s call it what it really is. It’s fortune-telling. Estimating is fortune-telling. You are trying to predict the future. Now, there are some occasions where you can predict the future pretty well. If it’s a very short timescale, if there’s not a lot of volatility, not a lot of unknowns, you can do okay at it.

But the farther the time horizon goes, the more volatility, the more changes that are in the environment, the more chaotic it is, it drops off pretty quick. And if you’re trying to come up with some sort of detailed project plan that goes six months, a year into the future? It’s fantasy. It’s a fantasy. There’s no way around it.

And so, get better at that? No, I don’t think that’s really the way to go. I think it’s a better approach to figure out, how can we create software at a reasonable cost and in a reasonable amount of time, without having to waste time making up estimates that no one really has any faith in, in the first place?

Adam: That was Andy Hunt. He is the co-author of The Pragmatic Programmer, and he also wrote this book called Pragmatic Thinking and Learning, which is kind of a survey of different ways you can get better at thinking and communicating and coming up with ideas, and we’re going to talk about it today.

I’ve been trying to avoid, in this podcast, using the word “agile” for personal reasons, I guess. My aversion to process. But Andy’s one of the authors of the Agile Manifesto, so it’s pretty interesting, actually, to hear his thoughts on what the vision was for Agile and where we are today. That’s only a small aspect of what we talk on.

We also talk about brainstorming, note-taking, becoming a better developer, all kinds of interesting stuff. Even spaceships. If you like this podcast, then subscribe to it. You can help me out by telling people who you think might enjoy it about it.

Today, my guest is Andy Hunt. Welcome to CoRecursive, Andy.

Andy: Hi, thanks so much for having me.

Adam: Yeah, it’s great that you’re here. You have quite a long biography. Most notable to me is, you co-authored The Pragmatic Programmer, which has got to be one of the more influential programming books in the past.

Andy: Yeah, we were very surprised by how that’s turned out over the years. When Dave Thomas and I first sat down to write that book in about 1998 or so, we were just going to write a little white paper for our consulting clients at the time.

So, that was why we put in, it’s like, “Okay, try to think of things this way. Stop doing the stupid stuff.” This kind of advice. And we were really shocked when the book came out. It was on the Amazon top 10 bestseller list, and not of computer books. I mean, the Amazon top 10 bestseller list.

So, yeah. It was right next to … I think it was like a Harry Potter, and a John Grisham kind of … I mean, it didn’t stay there long. Don’t get me wrong. But that it was there at all was remarkable. And yeah, it still shows up on things like IEEE or ACM’s top 10 most influential books, all these sorts of top five, top 10 lists.

Which is really, it’s kind of amazing. And it’ll go some years before I … I don’t read it or reread it that often. It’s just kind of in there, stuck in the neurons. But every so often, I’ll go back and read it and be like, “Wow. This is still a thing.” It’s 20 years later now, and these are still problems. These are still issues. These are still things people aren’t doing as well as they could.

Adam: Common sense is uncommon, maybe? Is that …

Andy: Yeah, it is. Common sense ain’t.

Adam: So, besides that, you … I have a little bio here. You were one of the co-authors of the Agile Manifesto, and you’re writing a science fiction book. You sound like a busy guy.

Andy: Yeah, I’ve actually got two science fiction novels out, and I’m supposed to be working on the third, but I’m actually puttering on some other projects at the moment. But the first two of the trilogy are out there, Conglommora and Conglommora Found.

Adam: Sounds very cool. So, today, I wanted to talk to you a bit about this book you wrote that I really like, called Pragmatic Thinking and Learning. So, as a software developer, why should I learn more about both thinking and learning?

Pragmatic Thinking and Learning

Andy: Well, that’s kind of what the game’s all about. A lot of folks think, “Oh, I’m a coder. I’m a programmer. I’m just here to write code.” But that’s not really the job. We’re really problem solvers. And now, most often, we write code to solve problems, but not always.

And so, as a problem solver, kind of the two key things that that boils down to is, you have to be skilled at communications, and you have to be skilled at learning, because we’re learning from everything all the time. And not just the new tech stack or … We’re recording this on a Thursday. It’s about 3:00 pm my time, so there’s been 37 new JavaScript frameworks that just came out today, right?

Not even counting the new barrage of new tech, but just you’re learning from your fellow team members. How do they work? How do you interact with them? How do you work and interact with the users? How do you interact with the evolving system under test, as it’s growing and building?

These are all areas that you have to learn from, and learn to work with as they’re moving and changing. So, that’s really what the whole game is about. And of course, there’s the new tech that comes along constantly. Somebody pointed out a while back that you think the pace of change is really frantic right now. All these new technologies, new languages, new frameworks, new security threats, all this stuff happening.

But you have to consider, on the long curve of history, today is the slowest the pace of change will ever be going forwards. It is just going to get faster. So, yeah. It’s hard now, and just literally not going to get any easier. So, we have to learn all the time.

Adam: Yeah, that’s a frightening thought, that it’ll just … So, a year from now, JavaScript framework pace will be 2X.

Andy: Yeah, I’m thinking it’s just going to go up geometrically, at least.

Learning To Learn

Adam: So, which is a better investment? Learning in a new piece of technology, or learning at this meta level about learning?

Andy: Well, you have to do both, obviously. And if you haven’t learned how to learn well, you need to start with that. That needs to be your starting point. Yeah, I’ve always figured that was the advantage of college or university education. Theoretically, they should teach you how to study and how to learn.

Now, unfortunately, of course, it doesn’t tend to work out that way in at least modern times. I’m not sure if it ever did. But yeah, you have to learn how to learn. And for me, that’s taking a lot of notes, physically writing out longhand, because there’s a difference, cognitively, whether you’re writing out notes or typing them or recording them on audio and then listening later.

Those all sort of hit different areas of the brain. And for some people, some are going to work better than others on how you can remember that and go back and retrieve it later. But in general, whatever you’re learning, you really have to work with the information.

Write it down, make yourself cheat sheets. One of the first things I do any time I’m learning a new language is start with a cheat sheet, however basic. “Okay, this is what assignment looks like in this language. Oh, they don’t have assignment in this language. Okay, well, I’ll make a slightly different note. It’s immutable forever. Okay, well, that’s different. Great.”

But whatever it is, you start making the notes. Get onto newsgroups or meetups, talk about with other people. But really, it comes down to practice, to actually trying to do it. I swear, this happens to me every time. I’ll read some book on a new language, and I’m reading through it and it’s like, “Okay, I understand what they’re … Oh, this is like this other language. Okay. This is cool. This is great.”

And then you sit down and try to write even “hello world.” You just try to write something very simple. It’s like, “Now, wait. What was that again? How do I do that thing?” And you’re flipping back through the book, trying to find the example and you type the example in exactly as they have it in the book, and of course it doesn’t work and it throws some error that you have no idea.

It could be talking in Sanskrit or some … What are they on about? And then you get experience with that language and realize that you put the docstring in the wrong place, if it was in Clojure. You left a semicolon if it was a C-derivative language, or whatever it might be.

And so you get used to that, but you have to be in there working it, to get that kind of experience, and to get that sort of deliberate learning. And that’s really the path to mastery. How to become an expert in anything, 10,000 hours or not, is this concept of deliberate practice.

So, not just doing the same thing over and over again, but really honing in on, “These are the parts that aren’t easy for me. This is the part I don’t understand. This is where I’m weak in this topic.” That’s what you really dive in and learn more about it, read up, try it, write a couple samples, and now you’re no longer weak in it.

And you just keep going like that. If you don’t quite get what the buzz is about functional programming, dive into something like Clojure or Elixir or Rust, and try and program in that style in several different languages. Because of course, none of them are the same. They all have a slightly different take on it, but you’ll learn something from each one.

And even back in the original Pragmatic Programmer book, back in the late ’90s, we suggested that you should learn a new programming language every year for exactly that reason, because each one will teach you something different about problem solving.

So, you might play with and learn Elixir, say, and you might never use that in your day job or even as a hobby. But just the act of learning it and seeing how they solve some of the common problems, and some of the features they offer, now you will design your programs in other languages a little differently, armed with that knowledge. So, even if it’s something that you’re not going to use directly, it still has incredible value for you to learn.

Adam: So, it seems like there’s a difference between learning and skills. You were talking about reading a book, and you think you have the knowledge, but how do you develop the actual skills?

Andy: Practice. It’s that old joke. “How do you get to Carnegie Hall?” Practice, man. Practice. That’s really what it comes down to. There’s no shortcut. The best way to do it is to do it. And unfortunately, there’s no shortcut for that.

Adam: You mentioned taking notes. So, how do you take notes?

Taking Notes

Andy: I’ve done a variety of things over the years. I’ve taken paper notes in a small Moleskine-style notebook. I’ve typed on the computer. I’ve used programs such as Evernote or things of that nature. And I keep coming back to, my personal and maybe idiosyncratic favorite is an editor-based wiki.

So, I rig my editor, whatever it might be: Vim, TextMate, Sublime over the years, I rig it up so that if you put in a camel case word and you just hit “enter” or “alt+enter,” it takes you to that file and that directory. And so, you’ve got something like an online wiki, like Wikipedia or a MediaWiki or whatnot, just in plain text, right there local on your machine.

It’s blindingly fast. You can bop in between different pages as fast as it takes to load an editor buffer on your machine, which is pretty instantaneous. Works offline. The actual text files are then … I usually have them backed up over a version control system.

When I first started doing this, no joke, it was RCS and then CVS, and then I swear Subversion was in there somewhere. It’s Git today, but it’s kind of treating it like source code. Here’s plain text files. It’s in version control, but I can link between them for different ideas, and then depending on what’s sort of hot in my brain space at the moment, I’ll have a home or a to-do sort of base page, with the most important topics up at the top.

And then very quickly, the five or six things I’m working on, because it’s always at least five or six things, right? You can just dive in and start taking notes, coming up with ideas, putting up sketches of articles or blog posts or whatever you’re working on.

So, that’s what I do for that sort of thing. But really, whatever works. And let me … I want to add one footnote to that. So, you have sort of like your main system for taking notes and working with data like that, but sometimes, you’re not ready for that stage yet, especially when you’re reading a book on an unfamiliar topic.

I may not be ready to really understand all the different concepts and how they fit together, or what parts I’m even interested in, or what parts I’m not. And for something like that, I’ll start off with a mind map instead. So, I’ll get a big old piece of paper, at least a pen or a pencil, maybe some colored pencils if I’m feeling particularly fancy, but probably not at first, and just start doodling on the mind map.

Start drawing little circles of, “Okay, this is an interesting topic. And wow, okay, this is new and different. I hadn’t considered that before. Hey, look. This connects to this other thing.” And so, you just do a bit of a dump on the mind map, which of course gets very messy because you’re scribbling on it and you’re drawing lines and you’re cross out lines, and “Oh, I thought that was connected. It’s not.”

But I find that it’s a really good first step. And then, you can take that, maybe now you’ve got better understanding. You’ve finished the book. You’ve tried some stuff, whatever. And then, transfer that over into notes on the computer or a wiki on the computer or something else that’s a little bit firmer.

But I like to start off with the looser tools for the looser thoughts, and then you work your way up. And also, as it says in the Pragmatic Thinking and Learning book, because of the way our brain works, a lot of really good ideas will come to you when you’re not near the computer. Especially when you’re not near the computer.

So, pro tip, right? If you’re stuck on a problem, the best thing you can do for creativity is step away from the keyboard. Just walk away. And people will say, “Oh, I was stuck on this bug forever or this design problem or whatnot, and I left to go home, walked down the hall, and halfway down the hallway, guess what happened? Bang! It popped right into my head.”

Sure. Happens all the time. That’s how we’re wired. So, one of the things that I always recommend to people is to carry something with you, you can jot notes down on, whether it’s a tiny notebook or a larger notebook or your phone, or whatever you’ve got. You’re going to be somewhere where it’s not convenient, and you get this thought. “Wow, okay. Now I know why that’s broken, or I really should test this,” or whatever the idea is.

You have to jot it down, because if you think, “I’ll remember this later,” odds are you won’t. You’ll forget about it. It’s like, middle of the night, you wake up and it’s like, “Oh, I got this great idea.” And then you wake up in the morning. “Uh. Let’s see now, what was that again?”

Personal Wiki

Adam: So, it sounds like you travel around with your little notebook, and then you also have big sheets of paper for mind mapping, and then this whole thing ends up in this wiki. How big is this wiki? Have you been building it for years, or is it a …

Andy: I have been building it for years, and occasionally I go in and prune it. There’s stuff in there about the proper spark plugs and oil change to do on a tractor that I haven’t owned for five years. There’s some of that kind of stuff in there, that if I think about it, I prune. And sometimes I don’t, but yeah, it’s probably some five, 10, 20 megabytes by now, maybe, of text.

Adam: Well, it sounds cool, but it also sounds like there’s some upkeep involved. Where is the value that you get from these systems?

Andy: The biggest value I find is, it acts as an exocortex. It’s an additional part of my brain and my memory, especially for things you don’t do very often, like changing the oil on the tractor, or what was the proper filter you needed for your HVAC system? Or little household-y things like that.

Or, “Here’s this language I haven’t used for a couple years. I had a cheat sheet on that once. Where was that? Oh, bang. It’s right here. Oh, that’s right. This one works this way. Here’s the things I need to remember, that I’m likely to forget, that I stumbled on once before, and now the notes are right there.” It’s like, “Okay, that’s right.”

So, it becomes a very easy way to organize and get to these sort of refreshers on different topics, on different details of things that you need to know. That sort of thing.

Cheatsheet

Adam: And so, if I need a cheat sheet, I might just google it. Is that as good as having one of my own?

Andy: It depends. The advantage of having your own is, you’re going to highlight the things that are most troublesome for you. One of the problems I find with a lot of cheat sheets in any discipline is, there’s maybe a quarter of it I desperately need, and maybe a quarter to a third of it that’s like, “Well, duh. Of course. Everyone knows that. You’re wasting space.” And then maybe a third of, “What the hell are they talking about? I haven’t gotten to that yet.” Or whatever.

So, it’s hard to get a tailored experience from that, whereas with someone you’ve made yourself, these are the problems you’ve run into and fixed. These are the mistakes you’re likely to make from your background in some other language, or whatever it might be. So, I find it very helpful just from that aspect, that it’s tailored to my particular experiences, and my particular biases of what I’m going to trip up and what I’m not.

Adam: And probably just the act of making the cheat sheet is a learning process?

Andy: Oh, absolutely. Absolutely. And that was going back to this idea of why it can be helpful to hand-write notes. The act of writing it, even just of capturing it, goes through different plumbing in your brain, and it helps cement those ideas, helps cement those memories a little bit better.

The more you walk to put it in, and then try to recall what’s in there, that retrieval is really key. That’s the absolute secret to memorizing something, is in the retrieval, or even doing spaced retrieval or those sorts of games. But it’s that trying to come up with it, is what really cements it in your memory.

So, when cramming, you don’t want to reread the material over and over again to try to memorize it. Maybe do that once, and then start quizzing yourself. Starting asking yourself. That’s why, if you’re studying a programming language, going to an empty editor buffer and trying to do it from scratch, that’s why that’s such a valuable exercise.

Because now you’re trying to recall and retrieve that information of, “Oh, how do I declare … I want to write some meta-programming here, or how do I even declare a variable? How do I get to the file system?” Whatever the thing is. You have to retrieve that information, and it’s that act of retrieval that really cements it in your mind. So, the more and more that you do, the easier it becomes. And now bang, you’ve actually learned it.

Flashcards

Adam: Yeah. I did do flashcards for a little while to help learn things. And it’s based on that, right? The idea that …

Andy: Yeah. And you can get fancy with that too, if you do the sort of … There’s various schemes of doing spaced repetition, so that for items that you miss, you put those back in, so you practice on those more frequently. And then the ones that you do get and get cold, you don’t put away forever. You just space them out a lot farther, but you still put them in the cycle so you don’t forget about them just because you got lucky once.

But I mean, that’s a whole … There’s programs that do that, and there’s a whole field of research on that kind of spaced repetition, of what’s the ideal sort of forgetting curve, and how do you put material in that curve to really maximize your ability to memorize it?

Agile Manifesto

Adam: One of the ideas of the Agile Manifesto was kind of to downplay documentation and writing things down. And you were involved in creating it, so how do these two things …

Andy: Well, they’re different things. When the Agile Manifesto downplays documentation, what they’re really saying is more along the lines of today’s thinking in Lean of eliminating waste. So, it goes back to, “What’s the intent, and what’s the purpose?”

If you’re, for instance, interviewing users and you’re writing down what they’re saying to help you learn and understand what they’re doing, that’s great. Nothing wrong with that. That’s perfectly good. If you’re filling out 1000 pages of documentation for the sole purpose of having it bound to sit on a shelf somewhere, that’s waste.

And it is an unfortunate thing that many large enterprise organizations have a requirement that you have to have detailed design documents, or you have to have this or you have to have that. You have to have some massive amount of writing for absolutely no good reason, other than someone thought it would be a good idea once.

And it’s not used. It doesn’t provide value. It literally sits on a shelf somewhere. And that’s waste. You’re wasting time. You’re wasting money. You’re wasting energy on something that’s not needed. Now, a lot of folks have misunderstood this as, “Oh, Agile’s about not documenting anything.” No, that’s not true. I mean, there’s plenty of occasions where documentation at different levels is absolutely required.

If you’re in a regulated industry, for example, you’ve got to provide some reports for the FDA for a medical device, or the SEC or whatever. Yeah, you have to do it. Absolutely. And there is value to that, if only in that, if you don’t do it, you’re going to get fined $100,000. So, yes. There is value in doing this. No problem, no problem. But the real argument there in the manifesto is to avoid producing documentation for documentation’s sake.

No Estimates

Adam: Yeah, that makes sense. I watched a talk you gave related to this subject, and you talked about black swans and estimating. Do you think that estimating is something that we should get better at, as a software community?

Andy: No, I think it’s something that we should not do. I think we should avoid it. I’ll tell you, we should avoid it because we suck at it. And by “we,” I don’t mean you and me personally.

Adam: Oh, I suck at it. Don’t worry.

Andy: Anyway, yeah. Well, as a species. I mean, this is the problem. As a species, we suck at estimating. First of all, let’s call it what it really is. It’s fortune-telling. Estimating is fortune-telling. You are trying to predict the future. Now, there are some occasions where you can predict the future pretty well. If it’s a very short timescale, if there’s not a lot of volatility, not a lot of unknowns, you can do okay at it.

Adam: So, how do we do that?

Andy: Well, that is the question, isn’t it? There’s a lot of answers to that. There’s the whole “no estimates” movement, which is big into debating the fine points of that question. But if you go back to Agile basics or Lean basics, the real answer is, don’t consider the traditional, “Here’s this year-long budget or year-long timeframe, or even six-month timeframe.”

It’s like, “What can we do right now?” Your budget is the team. You’ve got this many people. Okay. What value can we get out of them? Well, here’s this thing we absolutely need the most desperately. Can we get that done in a couple days? Let’s just do that. Now, let’s just do this other thing.

The real key is to continually and consistently produce small bits of value that just accrete and build up, and go over time. So, you got to get away from the thinking of “a project,” of having a project lifestyle, that it begins, it forms and everybody does it and they build it and then they all leave and go away.

That’s not really what the world is like anymore. It’s much more continuous than that. A lot of folks have this mental idea that, “Well, once the project’s done, it’s done. I move on, and we do something else.” Let me ask you. For the apps on your phone, how often do they come out with updates? Like, all the time?

Adam: Yeah.

Andy: Right, right? In that world, updates are continuous. And these aren’t just little updates. These are new areas of functionality, entirely new markets sometimes, a large pivot if the tech company’s unlucky. It’s an ongoing process. It’s not a project. It’s more of a mission. It’s whatever that company’s corporate mission is.

And the software is just an ongoing process, so to look at doing annual budgets or annual project plans doesn’t really make sense in that kind of context. You’re going to build something. You’re going to get feedback on it from the audience, and then you’re going to have to make some changes.

So, if you’ve just spent a couple months’ worth of meetings laying out a year-long project plan, and then two weeks after your app is out in the world, you realize, “Oh, we got to throw all that away. We have to do something slightly different,” that’s more likely.

Adam: Yeah. If you can react to feedback, then your plan’s out the window but you’re actually doing better, right?

Andy: Yeah. And reacting to feedback, that is the definition of Agile. And I find a great many people get really confused as to the difference between Scrum or Lean or Agile, and unfortunately, I think too many people believe that the beginning and end of Agile is this one small set of practices called “Scrum.”

And they don’t understand that Scrum is a set of lightweight management practices for a team in a particular context. And there’s absolutely nothing wrong with that. I mean, I’m not going to bash that. That’s a very useful thing for that topic area in this kind of a situation.

The thing is, that’s maybe 20% of what you need. You need all these technical practices underneath it. You need continuous integration, continuous delivery. You need maybe TDD. You need to actually work with the users, right?

Agile says, “Well, you’ve got to get requirements straight from the users and work with them and talk with them. Have we ever told them how? Do we have any practices that train users on how to work with us? We should.” Right? And so, I’ve been puttering around with this idea of the GROWS Method, at growsmethod.com.

And that’s one of the things I’m trying to get into there. It’s like, “Okay. Well, we have this expectation to work with executives, to work with users. Well, shouldn’t we give them a clue how that actually is going to work, what we actually want out of that?” So, yeah. There’s a lot of missing pieces to the puzzle there, that Scrum’s only one part of, XP is one part of. And that’s great, but we need more than that.

No Maintenance

Adam: Yeah, I totally agree. Here’s a quote that I have from you, while we’re on the topic of software processes. It says, “Efforts to make software maintainable or extendable or reusable are a waste.” What did you mean by that?

Andy: Right, so the idea there is … and I’m glad you asked that, because that’s exactly in the same vein of talking about fortune-telling, that we suck at. So, I’m writing this piece of software and I have been told I have to make it maintainable. I have to make it extensible, all these sorts of good things that people want us to do.

And I’m looking at it, and I have no idea. Because to make it maintainable, I need to know what’s going to change in the future. To make it extensible, I need to know what new requirements are going to come down that we have no idea about, that we’re going to get blindsided by.

Because if we knew them, we would just write to handle that. Okay, fair enough. But the point is, we don’t know what’s going to happen. So, we’re guessing. We’re fortune-telling. We’re trying to figure out what wild scenario, what might possibly happen that I should code against or prepare for.

And you can make some good, commonsense things. “Well, there’s going to be different codecs, so we’ll make that a plugin, and we can just stick in whatever.” Okay, that’s vaguely reasonable. So, to a point, you can do that. But really, at the end of the day, you’re trying to guess what’s going to happen in an unknown future.

So, it’s probably a better use of resources, a better use of your energy, rather than trying to fantasize about what’s going to have to be extended or maintained, say, “Well, what if I just throw it out? How can I architect this and design this so that, worse comes to worst and this is not doing its job anymore, I just want to rip it the hell out and replace it with something more suitable.”

Now, you’ve got better coupling, better cohesion, better modularization, whatever … all those sort of classic aspects of software. You’ve achieved all of that, and you don’t have to waste time with fortune-telling. It’s just, “Well, I don’t know what’s going to happen, but it’s going to be bad. And worst case, I’m just going to have rip this thing the hell out, so let’s make sure it doesn’t leave any dangly bits behind when you rip it out.”

And that goes back to your first comment about the Black Swan book by Taleb. And he points out that, in the history of history, all these sweeping changes that have happened blindsided everyone. No one saw it coming, and kind of for obvious reasons. Again, if you knew it was coming, you’d be prepared for it.

So, the things that do come and do cause sweeping changes, we’re blindsided by. No one saw that happening. I’ve told this story a bunch of times now, but it cracks me up every time. I was cleaning the back, back reaches of my office a couple years back, and there was this old, dusty stack of magazines. You remember magazines? You know, people used to print the internet and bind it, right? In color.

Adam: Yes.

Andy: And there were all these tech magazines from late ’80s, early ’90s, I guess, kind of timeframe. And years and years’ worth of arguments and ink spilled on, “Who’s going to win the GUI wars? Is it going to be Motif, or Open Look?” And people look at me going, “The what or the what?”

Okay, for the kids in the audience. So, Motif and Open Look were different GUI libraries that ran on top of X Windows. X Windows being the underlying graphics library architecture for all the Unix workstations at the time.

And the big question was, “Which one of these two competing standards was going to win out? Would it be Open Look or Motif?” And the answer was, “The web.” Right? None of the above, right? X Windows died. It all became HTTP and web clients, and it was a completely different thing than what everyone was arguing about.

In a similar vein, and I think roughly that same time period, maybe a little bit later, there was all these arguments about what networking, what middleware technology would take over, CORBA or RMI?

Adam: I remember CORBA.

Andy: And the answer was, “The web.” We were asking the wrong question. And that’s what happens. So, yeah. In that kind of environment, when you’re trying to consider, “How do I make this software last 20 years? How do I make it maintainable? How do I make it extensible?” Don’t.

Make it replaceable, because that gives you all those advantages you’re looking for. You don’t have your bits spread all over the system. It’s nicely contained. It’s got high cohesion, low coupling, everything’s all tied together. If it is easy to replace, then it’s well-designed software.

Adam: And when people try to make something endlessly extensible, there’s a trade-off, I think, between structure and how many something can extend, right? Like the ultimate extendable software is some sort of vowel function that just runs whatever’s in a database row or something, right? So you can update whatever it does at any time.

Andy: Or it’s something that becomes sentient, which you always have to watch out for. Yeah. I mean, there’s always common sense in that kind of thing, but a lot of the times, I see folks waste so much time arguing over what’s going to happen, A or B, when the answer is none of the above.

So, even in that case, right? It’s like, “Well, okay.” And then, three years later, they decide to go to a no-SQL database and it’s a completely different architecture, or whatever. We all end up plugging into the great brain on the squishy port on the side of the thing, and it’s a liquid AI with quantum gates, and you know, whatever. It’s not going to be what you think it is. So, I’m very conscious of not wasting time trying to imagine a future that’s not actually going to happen.

Get Away From Your Computer

Adam: That’s very pragmatic of you.

Andy: I try.

Adam: Earlier, you talked about getting away from your computer to solve a problem. And in your book, you described a sort of dual core processor of the brain. What did you mean by that?

Andy: So, this is going to be a long one. Sit back. Have some coffee. The brain is a really amazing piece of squishy lump of hardware. And one of the hardest tasks it has is understanding itself. It’s very complex. A lot of simplistic assumptions that we’ve made over the last 50, 100 years have been overturned in the last four or five, as we get better imaging, better understanding.

But it’s a really, amazingly complicated bit of stuff. And one of the things is, your brain will actually use different strategies for different kinds of processing. So, this notion of it being sort of a dual CPU, there’s actually several, but for now, let’s just consider there’s sort of two major problem solving modes or activation modes that your brain uses.

And when this was first observed, the fellow, Sperry if I remember right, got a Nobel Prize for it, and came up with the idea of left brain versus right brain, the idea that you had the sort of plodding, one step after the other, sort of von Neumann processor kind of model.

And then you had this magical digital processor thing that just worked asynchronously in the background, like a background task and would just grab stuff and synthesize it and put it together and throw it over the fence at you, at some random moment when you’re walking down the hall or in the shower, or just waking up, or these sorts of less-than-conscious moments.

So, this entered the popular vocabulary, right brain thinking and left brain thinking. And of course, the brain being the brain, it’s not nearly that simple. The way it actually works is, you’ve got these different regions of the brain that activate in tandem with each other, so it’s sort of a question of network activation regions.

And they’re not exclusively in either hemisphere. They’re evenly balanced between the two, but these observed properties still exist. You have this mode of thinking that is more creative, more out-of-the-box, more problem solving. And you have this more focused mode that’s sort of linear and step-wise.

Bimodal Brain

Adam: So, what are the advantages of these different modes? Do they have advantages?

Andy: Oh, absolutely. And presumably, they have evolutionary advantages. Otherwise, why would we bother having them both? But what happens is, we’re used to the sort of … I’ll use the term “left brain,” sort of synchronous, plodding through stuff, and that is how we perform tasks. That’s how we get through the day and do one after the other.

And that’s great, but for creativity, for ingenuity, for invention, for these sorts of things, you have to give that one a rest, and let the other activation mode kick in, the so-called “right brain” or … I had a better word for that in the book, and it’s completely gone out of my head right now as I’m sitting here. “Right mode,” I think I called it. So, instead of being-

Adam: Yeah, yeah.

Andy: It’s like, “right mode and left mode” instead of “right brain,” because it’s not hemispherically divided. So, we’ll just call them “modes” for the sake of convenience. But the right mode is asynchronous. I can ask a trivia question and it’ll sit there and work on it in the background, if you will, like a background job, and at one of these random points, it will produce the answer and pop it out to you.

Now, this gets kind of fun because you might be in the shower or in the car or something, and it pops into your head. Or, under some circumstances, it doesn’t come to you in your awake, conscious state. It comes to you in a dream, and this is where you get these experiences of having some dream that could be as wild as all get out, but there’s some fundamental core to it that is the answer to the problem you’ve been working on.

And one of the famous stories is the fellow who invented the sewing machine, the electric sewing machine with the needle that goes down through the material and back up over and over again. He couldn’t figure out how to get the thread to loop and tie knots properly with this arrangement.

And so, he had a nightmare one night about headhunters, the kinds with spears. Not the scary kind. Headhunters chasing him through the jungle. And he’s describing the dream to his wife and said, “You know, these spears were really weird-looking. They had holes up near the tip.” And bang, the light flashed, because what makes a sewing machine work is the holes are at the tip by the needle, not at the back like in a hand-sewing needle.

So, that was the big revelation that he needed to finish his invention and file his patent, and it came to him in this, one would think, completely unrelated dream about headhunters running around with spears. But it was that imagery of where the hole was, that was the salient point, and that came to him.

And as I describe in the book, quite a few famous inventors were aware of this phenomenon of trying to tap into these hypnagogic states and access that sort of imagery that your brain is generating. So Edison, Thomas Edison, who had what? 1000 or more patents? He used to take a nap in his study holding a cup of ball bearings or marbles, as the story goes, in his hand.

And the idea was, he would nod off to sleep, drop the marbles, and the clatter would wake him up, and he would write down whatever was in his head at the moment. And I got to think his housekeepers just loved that action. “Mr. Edison!” Just picture 1000 ball bearings all over the floor again.

But yeah, it is reasonable to put a little notepad by the bed or wherever you sleep, because when you wake up with those first thoughts, it’s very often this right mode set of activation regions in the brain, generating imagery, generating solutions to something you’ve been pondering, that it’s trying to express.

Taking a Walk

Adam: I haven’t experienced this dream version of it, but definitely the shower. I feel like I’ve had a lot of great breakthroughs there.

Andy: Yeah.

Adam: How do you, personally, take advantage of this? Do you load up your brain with facts, and then go for a stroll?

Andy: Yeah, almost literally. And it’s funny. I don’t really consciously do that, so much as I’ll just be jamming stuff in for a day or a couple days of something I’m working on, and then I’m just tired of sitting there, and I’ll go out and do something.

And yeah, as soon as you’re aware from the keyboard, ideas pop in. I actually had that happen with the novel I’m working on currently. It kind of works that way. It’s like, I’ll be … Writing is always a hard thing, whether it’s a tech book or a fiction book. You get to a point and you’re just stuck.

You’re looking at the blank page. It’s a blank page. You go, “I don’t know what to do next.” And you walk away and you got nothing, and as you say, you’re in the shower or you’re doing dishes, or you’re out doing yard work, or you’re walking the dog or whatever, and then boom.

It’s not just one, but I’ll get like 12. “Oh my God, I can do this. And then that ties in … Oh, and this is perfect. It ties back to this other thing.” And then it’s a scramble to try to capture all that before it evaporates again, before you forget it.

So, some years back, I discovered they actually make waterproof notepads that you can put in the shower. And they actually made a specific version for the shower. I don’t think it’s manufactured anymore, but you can go to like a camping store and they make journals and special pencils for hikers or campers, that are designed to get wet and work in a wet environment. And yeah, I’ve literally written whole chapters of books, standing there in the shower, jotting down furiously a couple notes that just came to me so I don’t lose it.

Meditation

Adam: That’s fantastic.

Andy: When it works.

Adam: Yeah. In your book, you talk about meditation even. Can meditation make me a better software developer?

Andy: It seems to. There’s a large body of studies out there that cite the cognitive benefits to meditation, and you read through it. It sort of seems that this is kind of the best way to reboot your brain, maybe defrag your mental hard disk, sort of clean things up.

So, they’ve cited studies with school age children, of improved ability to memorize, improved ability to analyze, improved ability to write, study. All these sort of things you would look at in a population of schoolchildren. I’m not aware of any particular study that’s been done with software developers, but all of the general studies showing improved cognition, improved mental abilities, hell yeah. Sign me up. I need all the help I can get. So, absolutely. And it’s a nice way to get away from email and Twitter for a little bit.

Adam: Do you meditate?

Andy: I have for significant periods of time, for years at a stretch. I’m not just at the moment as we’re recording this. I’ve gotten out of the habit lately, but every time I give one of these interviews, it’s like, “I need to get back to that.” So, yeah. I’m going to put that back on my to-do list, because it is helpful.

Adam: There’s no pressure. I’m just curious.

Andy: Well, it’s … The cobbler’s children have no shoes, right? All the stuff that I talk about and that I preach, I have done for significant amount of time. I don’t do it all at once, because I wouldn’t have time to do anything else, so it’s kind of pick and choose, depending on your needs.

Adam: That makes sense. What do you think about working in a high-pressure environment? So, I feel like a lot of these ideas, like taking a stroll and stuff, imply sort of a relaxing work world. They don’t imply like whatever, the production server’s gone down and nobody can figure out what’s going on.

High Pressure

Andy: So, interesting point there. Cognitively speaking, that is a terrible environment. So the first thing your brain does, the first thing your body does when it realizes there’s pressure, “Oh my God, the production server’s gone down. The database is gone. There’s hackers from some country I can’t even pronounce, it’s got too many consonants in it,” whatever. Some crisis has happened.

Your body is trained, from however many million years of evolution to say, “Okay. There’s a crisis. Step one, shut off the brain. We don’t need to waste any blood flow or oxygen going there, so let’s shut that down. Funnel everything to the legs, empty the stomach just in case we have to run.” You know, the flight or fight response, right?

So, your legs get all jiggly. You might feel a little nauseous, and your brain is shut off. The higher-order thinking is shut off, because your body’s like, “There’s a tiger coming after us!” Or whatever. Now, of course in this day and age, that ain’t the problem. We actually need it go the other way and say, “Okay, make my stomach stop rumbling because I’m going to miss lunch, and let’s get all the extra glucose and O2 into the brain, because that’s where the action is.”

Unfortunately, I don’t have a switch for that. Most people don’t. But they’ve done studies that show that it’s actually even worse than that. Just knowing you have a deadline can pressure the mind into failure, which is really sort of daunting. So, you look at some of these stereotypical high-pressure environments, and look at their failure rates.

It’s like, “Well, okay. We’re wired for that. That’s sort of how it works.” So, one of the best things you can do, and this kind of goes back to the mediation question, when you’re in a panic situation like that, literally the first thing you want to do is take a deep breath.

Nope, you did it wrong. Most westerners, it turns out, myself included, apparently we breathe wrong. When you say, “take a deep breath,” you first inclination is to go … like that. Right?

Adam: Yeah.

Andy: Wrong. That’s a shallow, upper chest breath. The first thing you need to do is exhale the crap, stale air. You want to push all the bad air out, then take a very deep breath from the lowest point of your diaphragm, filling up all of the chest, out to the sides, out past your arms, up into the top of your chest, and up into your throat. You want to fill from the bottom up, after you empty it first.

So, okay. First of all, we breathe wrong. It’s all downhill from there. But yeah, you take a deep breath. That has the practical advantage of helping to re-oxygenate your brain, because your body’s going, “Oh, crap. We have to run, because you know, the tiger.”

You take a deep breath, and … I mean, this sounds really cliché, but you basically make it calm. Everyone can be running around you like their hair’s on fire, and the production server’s on fire. But if you buy into that and go along with them, then your brain’s going to be shut off, and that’s not going to work.

So, you literally have to take a deep breath, focus on the evidence, look at what’s actually happening. Don’t listen to the person nattering next to you going, “Oh my God, it’s hackers!” It’s like, well, let’s look at the log. Let’s trace the packets. Let’s look at the stack trace.

Whatever the stuff is you got to look at, just start looking at the evidence and work your way through it. And I know it’s hard, but you want to try and shut off that emotional response that is our first and our default response of going, “Ah!” And screaming and running around.

Adam: Yeah. I feel like we could do better with making these less stress situations. I don’t know. If something goes wrong, I get paged every five minutes just make sure that I know it’s still going wrong.

Andy: Well, there’s a lot of reasons for that, I think. The first is, software in general is a very difficult area, because it’s so completely intangible. It’s not a factory floor. The bosses, your coworkers can’t see you out there with the hammer, actively banging on the thing, trying to get it back into shape.

It’s intangible, and because of that, it makes … Trying to manage software operations and development, is a very difficult thing anyway, but you couple that with, at this point in time, most of the organizational structures in place were not designed to handle software companies.

They were designed to handle railroads. They were designed to handle manufacturing. And so, you’ve got these decades upon decades of “best practices,” he says with disdain, that are not designed for the sort of work that we do. We have accounting structures, governance structures that aren’t designed for software. And round peg, square hole.

Adam: Totally. So, it’s 20 years since The Pragmatic Programmer, approximately. What do you think is missing, or what would you update?

20 Years of Pragmatism

Andy: Wow. Boy, there’s a question. I’m going from memory from here. So, the last time I read Pragmatic Programmer, it struck me how many things had changed in 20 years, and of course, how many things hadn’t changed. So, anything where we’re talking about philosophy or human nature, or how we react to things? Dead solid, perfect, don’t need to change a word. People haven’t changed. 20 years, on the timescale of a species evolution, is not even nothing, right? We are exactly the same. So, that’s all fine.

A lot of the technology, obviously, has changed. We had a lot of examples in C or C++ or CORBA or Eiffel. Hot, experimental languages at the time, that now no one’s heard of. So, I would be tempted to go in and talk more about Elixir or Clojure or Rust.

And in 20 years, some of those may be popular and some won’t, and people then will be looking and going, “What the hell was Clojure or Rust or whatever?” Because the technology, it does. It comes and it goes. And we sort of expected that at the time, so we figured the languages themselves and the frameworks and stuff probably wouldn’t last.

But what we didn’t really foresee was the pervasiveness of the internet. So, it’s a little embarrassing, but circa 1998 or so, we spoke of “the Internet” with a capital I, and how people were starting to use this email thing, and it might really take off.

And this was obviously, at the time, it was very much a different world from the world of today. I remember there was one thing about this fancy idea of unit testing, and I think we actually advocated writing your own unit testing framework, since there weren’t any out there, or weren’t enough out there. And now it’s like, “Oh, wow. No. No, no. Not that.”

I don’t think we even envisioned the idea of having CI/CD and the cloud, and being able to push and do builds in the cloud kind of environment. That just wasn’t what was done back then. That was the age of CD-ROMs and America Online carpet bombing people with CDs in the mail.

So, yeah. It really is a different world. The tech landscape is different, but the errors that we make, the opportunities we don’t pursue, that’s really all the same. And that’s why the book still remains on the classic list, is we got so much of that right and it is still useful. It’s like going back and reading Fred Brooks’ Mythical Man-Month. He’s talking about building the IBM 360 mainframe, which is horribly dated.

But literally, if you go through and replace that … If you just did a search and replace, and replaced it with “Samsung phone hardware” or “an iPhone” or something contemporary, it would read exactly the same. The management struggles are the same. The misunderstandings are the same. The problems we face are exactly the same, just the tech changes.

Writing Science Fiction

Adam: So, my very last question is, what made you decide to write science fiction?

Andy: Well, that’s a really good question. Well, okay. Yeah. What made me decide to write science fiction? Well, funny enough, and this comes on the hell of the hypnagogic responses question, the core of the plot, sort of the scene, the environment came to me in a dream.

And it was a very brief dream. It was more sort of like the scenery. This is what it would look like, the idea of where the plot takes place, the sort of … All these spaceships decided was Earth was dead. They couldn’t find another useful planet to set up on, so they just stuck all their ships together out at the edge of space, not going anywhere, and called it home.

And this sort of vision of what that culture would look like, what living there would be like, what threat comes to them and how they have to deal with it, it just came to in this brief dream. I was like, “That’s kind of a cool idea. I should write that down so I don’t forget it.”

So, I wrote it down. And literally, it was a page, maybe? Half a page? This wasn’t any grand treatment. It was just, “Well, here’s a sort of nifty idea.” Then I sort of looked at it. It’s like, “Well, let me just kind of extend that into a proper short story.” And I did, and then I kind of went from there. And now, two books are out and the third one will come after the current novel.

Adam: I haven’t read the books. I looked at Amazon, but I saw the keywords in the review, “homebrew spaceship.” And I’m like, “I don’t know what that is, but it sounds cool.”

Andy: It’s fun. I mean, the fun thing about science fiction is, it’s easy enough to decide, “Okay, I’ve got this great new technology. I’ve got warp drive. I’ve got transporters. I’ve got gene editing. I’ve got AI, whatever. I got blockchain.” Got to get blockchain in there somewhere. “I got this thing.” Okay, that’s the easy part.

Now, the hard part is, what is it like to live in that world? What would that look like? What are the implications? What are the consequences? And that’s why I think writing science fiction really is like engineering, because engineering is not about right and wrong. It’s about trade-offs and consequences.

“Which is the best JavaScript framework?” Best for what? Under what circumstances? What are the trade-offs? What are the consequences of it? So, that kind of line of thinking, I think fits in very naturally with that sort of science fiction.

It’s like, “Okay, here’s the setup. What are the consequences? How is society going to change because of that? How are our characters going to react because of that? What are their opportunities? What are their challenges? How do I make that exciting?”

Adam: Sounds great. So, I think that’s a great place to end it, get everybody excited about your book. So, thank you so much for joining me, Andy.

Andy: Oh, thanks for having me.

Adam: All right. That was my talk with Andy. I sent him a couple questions about what we would talk about, and I’m not even sure that we got to them, but so much interesting stuff going on in that talk. I’m really interested by his idea of using a text editor as kind of a simple wiki. And yeah, he was a fun guy to talk to. Also, no estimates. That’s a movement I can get behind. All right. Until next time, thank you for listening.

CORECURSIVE #029

Learning to Think

With Andy Hunt

Transcript