Software in Context With Zach Tellman

Intro

Zach: If we say something is overengineered, what we mean is it’s too complex or it’s too robust or it handles a bunch of situations or scenarios that are not relevant to how we’re using it. It’s okay for us to create narrow things. It’s okay for us to create power shells instead of bash sort of environments because that narrowness gives us the ability to go and do things you might not otherwise be able to do.

Adam: Hello. This is Adam Gordon Bell. Join me as I learn about building software. This is CoRecursive. Zach Tellman wrote a book about designing software called the Elements of Clojure. I love the interview I did with Runar, Episode 27 on abstraction. Zach also has a lot of things to say about abstraction, but they’re very different. They’re not incompatible with Runar’s thoughts, just totally different.

One big takeaway I have from talking to Zach is that we need to consider the context in which software is built, something can be over engineered or it can be a hacky solution, only in the context in which built and in fact, it can be both over engineered and hacky in different dimensions and in different contexts.

Also, Zach just looks at software as being closer to literature, which is a, it’s a foreign perspective for me. It’s not how I think about things, so very interesting talk. I hope you enjoy it.

Elements of Clojure

Adam: Yeah. Thanks for joining me, Zach. I bought your book, Elements of Clojure. I don’t know a lot about Clojure, actually but the kind of latter parts of the book, I felt were kind of more generally about building software. I was curious why you decided to write this book.

Zach: I called it Elements of Clojure because it was meant to be this callback to Strunk and White, The Elements of Style, which is a flawed book in a number of ways but still I think served its purpose, right? It helped people sort of stop overthinking things and allowed them to just sort of get into the process of writing.

With that goal in mind, I sort of started out writing a book about names and sort of what was a good policy for naming. I very quickly realized that I, the writer, could not go and give hard and fast rules for that because that required some understanding of the domain that people were trying to find names for but I did around that same time that I finished first chapter, quit my job, so I could focus on it full time.

I thought that was going to be a three-month project, right? If I just put my head down and really worked at it. Then, I was going to go and work on a whole bunch of other projects that I’ve been thinking about. It turned out that it was a 16-month process of writing a book because I kept on trying to do more or I kept on coming up against this thing where I felt like I understood what I would do in a certain situation but I didn’t understand it well enough to explain why.

There’s a concept that I actually encountered while I was doing research for the book called tacit knowledge, which was first coined by this guy, Michael Polanyi, who was a chemist and then later a philosopher and historian of science. He talked about sort of why it was that hypotheses turned out to be right so often in science, right?

Adam: Mm-hmm (affirmative).

Zach: I mean, there are infinite many hypotheses. Most of them are wrong and practice, but we get it right a lot more often than we should. It’s clear that upstream of what you would kind of call the scientific process, where you have a hypothesis and you test it and then you try to invalidate it and if you can’t, then it gets adopted. There is this much more ineffable kind of thing going on there, which is a thing that we know but can’t explain.

We name things constantly but we don’t ever actually study the act of naming. We constantly draw lines between things but we don’t actually focus on that process. It is this thing that we kind of assume that if we just do it long enough, we’ll stop being bad at it. It’s a little peculiar, right? We regularly say or possibly joke that naming is one of the two hard problems in software. Nowhere in my undergrad program, did anyone actually talk about how to name things. I think the only guidance I ever got was like single letter names are not good.

That seems strange, right? It seems like if this is a hard problem, then we ought to go and actually talk about it rather than just kind of treat it as this thing, which is sort of beyond explanation or beyond refinement. That was kind of where I ended up. I was trying to go and put words to this intuition that I had.

The book that I created, I think, does an okay job with it. I know that this is probably going to take more than a lifetime to do sort of in a way that I would actually find satisfying but it was a good first cut. I don’t know, I mean, it was a very fulfilling process, I think because I do feel like I’m better able to explain why I feel like something is good or something is bad.

Indirection vs Absrtaction

Adam: Yeah, there is a lot of things that we have strong opinions about but can’t explain. I guess. It’s interesting. I think that starting with names maybe that seemed easy to you before you got into it. That’s like the hard area. What did you find or what are your recommendations now that you’ve spent this time on names?

Zach: I mean, if we were to just kind of jump into it, I guess, there are these very closely related concepts, which we often use the words for, but I think very rarely bothered to define, which are indirection and abstraction. Just as a disclaimer here, I’m going to define these words, not in a way that I think everyone listening to this will immediately agree with. I think that they represent sort of the plurality of usage in the industry but I think that there are also cases where they’re using ways that I find to be sort of misleading.

Make of that what you will, I guess, but for me, indirection I think is best described as the separation between what we’re doing and how we’re doing it. The simplest explanation or sort of example of this would be a power button. We press the power button, something happens and like our computer turns on or some other piece of electronics turns on.

The key thing there is that if we press the button and nothing happens or the wrong thing happens, we don’t understand what is going on underneath the covers well enough to be able to actually debug that, right? This is why we constantly turn things off and turn them on again, right? We’re just returning to the one thing that we know how to do and hope that we do it repeatedly, like things will finally align with our expected outcome.

This is both the power and the sort of danger of indirection, right? We are purposefully blinding ourselves to something because we either don’t want to or simply can’t understand what’s going on underneath there. That gives us a lot of power, but it makes our sort of ability to understand the world and to interact with it more fragile.

Adam: I think of indirection, I mean, the first thing that comes to mind is a magician, kind of doing something over here while they do something that you’re not supposed to see on the other side.

Zach: Well, I guess, typically, you’d call that misdirection…

Adam: Oh misdirection.

Zach: … as trying to actively mislead somebody. I think indirection is just saying, if you want to go and understand this, you’re going to have to go and reach this point, this sort of line that we’ve drawn and you’re going to have to move past it.

The power button does not trying to, at least in an ideal case, completely make opaque what’s going on with the covers is just an opportunity to stop asking, and then what? It’s a chance for us to be incurious. That’s not to say that people won’t, and often do, I think, have sort of a tendency to say, “And then what,” or “Why, or to actually try to go and peel back that layer, but you want to go and have that sort of invitation be there because some people want to stop digging down and they want to start building up.

You need to have the ability to go and stop because even in a fairly short amount of time, I think that we’ve built enough from the sort of foundations of like the computer hardware that you could spend a lifetime asking, “And then what,” before you ever get a chance to actually start building up in the other direction.

Adam: Yeah, it’s like, a black box or an abstraction barrier or something?

Zach: Well, yeah. Let’s talk about abstraction, because I think that these terms are often used interchangeably. I think that there is a subtle difference there. Let’s consider another example. Let’s say that you have the code print line, “Hello world,” right?

Adam: Mm-hmm (affirmative).

Zach: There is indirection going on here because we are telling the system, please print this text. We are unaware of how that actually occurs. We’re not even sure where it’s being written to, right? It might be getting written to a console or to a log file or to a browser or anywhere that our texts could possibly be streamed, it might be getting streamed there or to all of them simultaneously. There is a separation between the what and the how and for us, all of those different mechanisms are fundamentally collapsed into something which is the same. That is how I define abstraction.

Abstraction is treating things which are different as if they’re the same. It’s critical to recognize that this is happening on both sides, because this thing that is actually doing the printing, it is treating all possible text as interchangeable. We did not have a special, “Hello, world” printing handler. We go and we take the text and we’re able to go and we’re able to deal with this. There is this generalization, sort of on either side of this interface, on the one hand with respect to implementation, on the other hand, with respect to the sort of entire input domain.

The reason that it’s important to distinguish between indirection and abstraction is that abstraction occurs via indirection but it also occurs in a lot of other situations that have nothing to do with indirection or at least not something that we would sort of recognize naturally as indirection.

In semiotics, there’s this idea of semiosis, which is basically collapsing something down to representation and then sort of resolving the representation back into something in the real world. This is the thing that we do unconsciously, right? This is medieval argue what it is to be intelligent is to be able to reason via representation.

I think everyone who is a successful software engineer has a powerful, innate sensibility about abstraction. The people who have succeeded in this industry have that, which is not to say that people who haven’t succeeded don’t have this. I think that if we were to go and talk about software, the hard problems of software being about sort of modeling and reasoning by analogy and all these other things, I think that would open it up to a much wider group of people but there’s a unspoken kind of shared sensibility that I think that many of the people that we work with on our sort of day to day all share.

When we look at something we say, “Yeah, that’s not a very good abstraction,” there’s something going on there. There’s a shared understanding that’s going on there and I think that this is, in some way, innate to almost everyone, and it’s just sort of hyper developed in a lot of people who have sort of found their way to success in our industry.

Unix Pipes vs Power Shell

Adam: I’m having a hard time connecting that to software development. If I were to pick an abstraction, okay, let’s say Unix pipes, right? I can take several programs and take the input of one and pipe it to the other, is that an indirection? Is it an indirection and an abstraction? Is it an effective one? How does it fit in your various definitions?

Zach: Sure, I mean, this is a classic one. I mean, it is clearly indirection, right? We have this channel by which we’re going in or passing bytes, which are, by convention, presumed to be like ASCII text, in most cases, but not all. Obviously, there is attraction going on there, because cat doesn’t go and presumed to know what sort of upstream of it, right?

Adam: Mm-hmm (affirmative).

Zach: Nor does grep nor do any of these other sorts of things. They have a certain assumption there. In the case of grep, it is something that is [grepple 00:11:49], but there is a sort of mix and match quality there. There is abstract and high interaction going on there but there are some assumptions that are kind of going into the domain, notably, that the input is largely unstructured.

I mean, there are tools like jq and other sorts of things that sort of temporarily lift the input from this sort of unstructured text domain into this sort of JSON domain but that doesn’t sort of follow along the other way. It goes in it sort of inside of it will go and do the decoding, and then the operation, and then this encoding, because everything drops down to this sort of lowest common denominator of data representation, which is this sort of unstructured text.

In that sort of case, we’re treating all things as the same and we’re saying and, the sort of lowest common denominator that allows us to go and make that assumption is that there is no structured data encoding that is sort of common across all these things.

If you go and look at something like PowerShell, which is a Microsoft sort of dotnet version, there’s an assumption that everything is fundamentally a dotnet runtime, which is going and is emitting sort of very specific sort of encoding, which is a far less general assumption as much narrow domain but you get something out of that. By going and not trying to be all things to all processes, you get a certain amount of power because you can now begin to sort of reason about a much more limited set of cases that you need to handle.

Adam: I really like this TypeScript versus Unix pipe example, by the way, because I think that there is some pluses and minuses of both sides, right? I mean, maybe not the dotnet runtime but actually, if each process in a pipe needs to reserialize and deserialize it, there’s a certain cost to that, right?

Zach: There’s a certain cost and there’s also certain things that are not representable, right? I mean, in a very sort of trivial and not very interesting way, JSON is representable as a flat stream of characters but there might be things that are not representable inside of dotnet. Notably, like if I have objects that have been encoded via some other sort of standard, that’s just like a totally foreign language for this.

Like I said, there’s a generality here, but it means that fundamentally, we are reduced to this sort of Pidgin language when we’re dealing with bash utilities, often, which is, well, we have lines of text. It’s like, well, are they structured? Well, maybe, but not in a way that we can really deal with. Can we do greps that allow us to go and sort of say, only in these substructures should we go in doing this match? Well, no, not at all, unless we go and use some sort of special tool like jq, which is itself only composable with other jq things, right? This is not something where we can do this.

I mean, notably, what the bash utilities don’t have, which is maybe not related to pipes, but there is no higher order bash operator. LS does allow us to pass in a different bash process that does its sorting. This is why you have this kind of this explosion of parameters for a lot of these things, because there’s no ability to go and say, “Go and do this and I want to go and sort by time,” that’s not a general purpose operator that bash gives us because again, there’s no sort of concept of structure or a particular field or other sort of thing like that.

Every process has to kind of exist within its own self-enclosed universe where it gets unstructured text, it interprets that text. It applies semantics to that text and is able to do operations on the basis of the semantics that it’s described. Then, once it’s out the other side, that’s gone. That’s not passed along. That interpretation is only understandable in terms of the raw text that it admitted. There is a loss there, right? I mean, there’s a reason that we don’t go and write large, complex applications using just shell scripts.

Adam: I never thought about higher order functions. Yeah, being able to instead of adding a flag that you should be able to say, “No, I’m going to give you the thing that you would use to sort this.”

Zach: But yeah, how would that work because, again, the only contract we have between them is I’m going to give you some texts, and you give me back some text. How do we go and say, “I want the text that is the time,” or something like that. You can imagine something that would do that but then you would have a much more kind of narrow and complex set of data descriptions that are going on there. You end up in something that is much closer to, I think, PowerShell or any other modern programming language where we actually have data types.

Adam: Are abstractions good or bad? Which is better bash or PowerShell?

Zach: It depends. I mean, it’s the total pop out answer, I guess, right? I mean, something that I talked about in my book is, what does it mean for something to be over engineered? If we say something is over engineered, what we mean is it’s too complex or it’s too robust, or it handles a bunch of situations or scenarios that are not relevant to how we’re using it.

If someone gives you a watch, and they say, “It can tell perfect time down to 1000 meters,” and you’re like, “Great. It’s very heavy and I don’t plan on doing any deep dives anytime soon.” I probably won’t use it, right? This is an over engineered watch. Unless we really enjoy that aesthetic, it’s probably less useful than a watch which is more tied to the situations in which we plan to wear it.

When we talk about is an abstraction good or something that I find even more kind of problematic, which is like, is this abstraction correct? I mean, what does that mean, right? I mean, you can say that the code that we’ve written is self-consistent, right? It has a set of invariants that it’s defined and those invariants are maintained in the sort of closed over all the operations we can do but to say that an abstraction is correct, is to imply that for all possible scenarios, this is the right abstraction. I think that that is impossible because we’re ignoring things. We are going and we are collapsing distinctions into something which is the same. We can easily imagine a situation in which those distinctions matter, right?

Adam: Mm-hmm (affirmative).

Zach: Now, those scenarios, for instance, we might go and say, “Hey,” well, this clock that you’ve come up with. An example I give in the book is I’m talking about how long it took for people to make maritime clocks, clocks, which keep accurate time on a ship. This is actually a longstanding prize that was sort of outstanding from the British Empire because they really needed to be able to keep accurate time on ships, even though ships sort of pitched around a lot, there was wildly varying temperatures, and gravity varies by about half a percent between the equator and the pole.

There’s a massive number of things that you need to sort of shield the inner workings of the clock from in order for it to keep time therein and it turns out that this is possible but if someone gave you a maritime clock and all you did is put it on your mantelpiece, it’s not very useful.

Software in Context

Zach: When you’re kind of considering whether or not something is useful, you have to first say, “Well, how do I plan to use it?”

When you say an abstraction is correct, you’re asserting this is useful no matter what, which I think is only true, if you have failed to consider all the possible scenarios that are there or you are more commonly asserting a bunch of implicit assumptions along with what you’re saying in terms of how you expect it to be used, right?

This clock I’ve made is a very good clock. I assume that you will never bother to take it on to a boat but that part is often left unsaid, because oftentimes, people don’t have a rich enough understanding of the world or of their users or of like the sheer possibilities that exist out there to have anticipated the ways in which people might knowingly or unknowingly misuse the thing that they just created.

I think that there’s also a bit of a stigma attached to this idea that you built this thing, this open source library, this framework, whatever, and I tried to use it for something and it didn’t work very well. I think that there’s a conversation that happens a lot on the internet, where people are like, I use it for whatever it is that I’ve been working on lately. It didn’t work very well. It’s a failure. That’s a bad abstraction. You can say the abstracting did a poor job of explaining how it’s meant to be used, but I think that you can with a very straight face, say like you’re holding the abstraction wrong.

You’re bringing it into a situation which is, was not meant to be used in and that’s okay. It’s okay for us to create narrow things. It’s okay for us to create PowerShells instead of bash sort of environments, because that narrowness gives us the ability to go and do things you might not otherwise be able to do.

Assumptions Implied in Software

Adam: Can you think of an example of a time that something was misapplied so that something had a certain abstraction that fit a used case like in terms of software, and then was used somewhere where the abstraction didn’t fit?

Zach: Well, I don’t know. I tend to always kind of focus on sort of the most trivial example I can think of, because otherwise, you just spend all this time explaining all the contexts but let’s talk about the SQL concatenation thing I was taught.

Adam: Yeah.

Zach: If we have something as an internal tool, where we’re just trying to go and create like a little bit of sort of macrocompression of common manual tasks that people are doing, it’s meant for people who are already privileged users of the system and everything like that, trying to harden the system against malicious input is probably a waste of our time. If they want to be malicious, they have other sort of means to do. Who cares if we’re doing things in kind of the dumb, fragile way?

Now, that’s perfectly valid up until the point where someone’s like, “Oh, well, this actually is pretty useful and we’re going to go and open it up to a bunch of third parties,” or this team that was internal, now we’re spinning it out into some sort of contractor, team, whatever. Very subtly, our threat model has shifted. Very subtly, all these sorts of assumptions that we were making about who was that using this has been invalidated.

Now, the degree to which we recognize that in the moment depends entirely on whether or not we were very vocal about the assumptions that we’re making early on. This is an internal tool. It is fragile. It is insecure. It is broken, except in these very sort of narrow set of circumstances.

When I say that it’s about its fussing over these nuances, I think that what I really mean by that, I guess, is to say that we are talking about the intended use of this thing that we built, and the intended scope of changes that it’s meant to sort of encompass and by sort of explicit or implicit, just kind of being absent from how we talked about our tool, the things that it’s not meant to deal with, the situations that is not meant to go and be robust to the face of.

Adam: That’s very good point. Something I don’t think I’ve heard people talk about explicitly before, that happens constantly, right? I can think of a really old example at a company I worked for and just a checkbox that was in some sort of UI, not a checkbox, like a drop down or something and generally, there was like about five options, but they were configurable, but then some client joined who they had about 3,000 options that needed to go in this, right? All of a sudden they’re like, “This is crap. This doesn’t work. How can we navigate through this list?” There’s so many of them, I guess, of these kind of assumptions.

Zach: I mean, it’s hard and I don’t think that this is a question that has like a closed form answer. I think that there are frameworks that people have put around this sort of stuff. There’s the domain-driven school of this. I mean, I don’t know what you did in the case of the drop down, the 3,000 items, but like, you could just say, “Oops, our bad. We probably shouldn’t have sold this unit in the first place.” That is the classic sort of startup move there.

I mean, I think that, ideally, you can come up with these sorts of metaphors or analogies, which are reasonable are well-understood by a nontechnical or quasi technical sort of audience where those limitations are, if not obvious, at least not surprising. You go and you can kind of talk about these things. You can go and talk about these sorts of used cases that you’re imagining. Then, hopefully, someone who has this crazy complex sort of taxonomy will go and be like, these all seem substantively different than what we do.

Then, they can go and they can ask, and they can say, “Well, let’s go and talk in more details about what we’re trying to do.” Is this a good fit? But that intuition that it’s a bad fit has to be coming from the person who is the domain expert, not from you, the engineer, who is the system expert. You have to come up with something that sort of survives that translation.

Again, this comes down to metaphors and analogies and things where, again, this is this attempt to communicate by this very reductive set of signals that we’re getting, these very reductive set of analogies or metaphors, or very small amounts of texts or something, and you hope that the analogies you’ve drawn are rich enough that they can be sort of understood. But that also has to go the other way, right? If you have this analogy they’re using to talk to the customer that has to drive the decisions that you’re making in your own system, so that those two things are aligned and they’re not sort of suddenly drifting over time.

Adam: It makes me think of that there’s some article about 20 things that people think are true about names that aren’t, it’s like people’s names that are not representable in Unicode are people’s names that are things that most systems don’t support, right?

Zach: Right.

Adam: My wife has an apostrophe in her last name, which for most of the ’90s caused people who tried to get her onto a loyalty card system to kind of freak out when their computer crashes.

Zach: Right and I mean, we have, I think, as an industry become better about this, right? We’re not just kind of assuming that 255 characters is good enough for everybody, right?

Adam: Yes.

Zach: There is maturity that exists there but I think that also part of that is we have people who have run into these problems over and over again. In my book, I assert that, “What makes someone a senior engineer is not their expertise with the technology. It’s their ability to predict what the world is going to throw at that technology.” To be able to easily and upfront at the cheapest point in the design process, say, “Oh, but what if this happens?” Then, people can decide in that moment whether or not that is a case that they want to cover or not, right? Whether or not that sort of falls within or out outside of the scope of the problem that they’re trying to solve.

That’s the real value, I think, of having someone who wears all those scars from all those mistakes, all those things where they found very late in the game, “Oh, actually, people do have apostrophes in their name,” or “Oh, people actually do send an emoji,” where we thought there was only just kind of plain text or what have you.

Adam: What you’re saying about senior engineers sounds like the key to building the right abstractions is building up your intuition about what the assumptions might be? Is that the only way or is there a way to build better abstractions even without kind of having an intuitive sense of who your customers might be?

Zach: Here’s what makes an abstraction useful, is that its assumptions match the context in which it exists, right?

Adam: Yeah.

Zach: The things that it doesn’t represent are things which are not relevant to like the thing that is trying to do. How do you go and build good attractions? Well, one thing is that you become fairly expert in the environments that you are kind of writing your software for. Depending on where you are in the system, that might mean like, as a distributed systems engineer, you are expert at all of the weird stuff that can happen in a distributed system, right?

Adam: Mm-hmm (affirmative).

Zach: It also might mean that if you’re writing sort of like, if you’re a developer, productivity engineer, or something like that, you’re expert at what your customers. The other engineers within your organization, will and [inaudible 00:27:25], which may just mean that they’re very used to some technology and so you have to write something that very closely resembles that technology.

Then, as you sort of get further and further out towards the periphery of the product, you have to become an expert in your customers in terms of what they’re trying to do and what sort of new customers you might want to go and try to target in the future. Is this thing that you’re doing is this only going to work for the current verticals that you’re targeting or for the next three verticals you might want to target.

The point is not, like as an engineer, you have to become an expert in all things but you do need to go and sort of understand a few of those concentric circles outside of the specific thing that you’re working on because the specific thing that you’re working on, can’t tell you anything about what’s outside of it.

Someone Is Wrong on the Internet

Adam: Yeah, PostgresSQL has this new feature, computed rows and I was reading online, people were arguing about it. It seems relevant to this because somebody was like, “Oh, I just used this to speed up my system because this value that we were calculating on read time, now the database calculates it every insert.”

Then, somebody else was like, “This is a known anti pattern. You only have one database. Clearly, you should do this at the level above it where you can kind of scale horizontally.” It strikes me that that’s kind of what you’re talking about. Those people have two very different verticals that they’re in or set of requirements in one, maybe the database isn’t so overloaded that you need to take everything out of it possible.

Zach: Right. I mean, really, what it comes down to is how many writes do you have and how many reads do you have, right?

Adam: Yeah.

Zach: If you have thousands of times more reads than you ever do writes, going in caching that sort of derived value might make a ton of sense.

Adam: Yeah.

Zach: On the other side, if you don’t, and then that doesn’t even get into, well, what if you want to go and change what that sort of derived value looks like. Is it easier for you to go and do a database migration or to do these other sorts of things or do you want to have that sort of be part of the code? Then, also what teams own the database and what teams own the actual logic there. Are they close to each other or are they only sort of incidentally attached?

You want to go and make sure that on either sides of these interfaces, there is the ability to be mostly incurious, right? As I said, you can’t be entirely incurious. You can’t, every time that you do something and it doesn’t behave as expected, you have to have at least some sort of sense of what to do next other than just like turn it off and turn it on again, right? There’s so many different sort of things that come into whether or not something is a “good decision.”

Then, of course, all those circumstances that informed that decision are subject to change, right? People leave. New people are responsible for them. Institutional knowledge disappears or bit rots or other sorts of things like that, but suffice it to say it’s a big trash fire and we’re just trying to go and do what we can to keep it contained. I think that the discourse around like that is a good thing or that is an anti-pattern, or this is great or bad or something like this, it’s hugely distracting at best and I think harmful in a more realistic kind of reading of the whole situation.

Adam: Yeah, no, I totally agree. It gives a lot of food for thought in discussions, right? If people could be labeled with some sort of metadata and I could see the guy proposing the one idea, you know he has a startup and they’re selling to small businesses and the guy who’s arguing with him, he’s targeting Fortune Five companies. They’re not wrong. They just very much have different worlds.

Zach: Right. They’re just talking past each other. Evan Czaplicki, who created the Elm language, gave an interesting talk a few years back called the Hard Part of Open Source, where he talked about sort of a similar problem in a slightly different vein, which is people debating, “Well, in his case, you know the design of the Elm language, but in any other sort of technical discussion and he spends a lot of time sort of laying out what he sees of the problem. Towards the end, he made some really interesting proposals about how you might solve this.

One of them is, in addition to writing what you have to write, you have to go and sort of state your assumptions, right? There is actually like a required form there, where you’re kind of talking about this or saying, in the context of talking about this, I really care about the performance language or I really care about the expressivity of the language, whatever that means or I really care about this particular domain or something like that.

To have that be a required set of, as you said, metadata around what these people are arguing, would be hugely illuminating, and what I think make it easier for people to understand why they’re talking past each other or at least be able to infer the worldview that kind of made someone say this thing that they the other person so strongly disagrees with.

Adam: We don’t even consider our assumptions like the person who’s building very high performance software for a very large data used cases, they don’t even think about the context of somebody who’s building a Rails app or something that’s just going to serve so many people per server, I think.

Zach: I mean, I think that’s true but then people can always talk about the fairly small number of examples of like Twitter, built on top of Ruby, because that was a reasonable thing and then it stopped being the reasonable thing. Again, you have this kind of, I don’t know, I call it Hacker News induction, which is like, “Well, I built this thing and then I built this other thing, which is almost exactly the same thing and it worked or it didn’t work and therefore, I think that this must generalize across all possible applications of this thing.”

I tried Rails and it was great. It was awful and therefore it is great or awful in all situations, and then people go and say, “Okay, well, Twitter used Ruby and then they stopped using Ruby and they went on to the JVM. Shouldn’t they have started with the JVM?” Right?

Adam: Yeah.

Zach: Shouldn’t everyone go and do this or shouldn’t we tried to go and make something which is all seeing and all dancing, and it’s both expressive and fast and scalable. That’s, it’s an understandable impulse, I guess. We are problem solvers. We see a problem. We’re like, “Okay, maybe there’s a technological solution to this.” I think Rust is a very good example of that sort of possibility.

I’m not saying that things can’t be better. I just think that the idea that we are monotonically sort of approaching this kind of Ur language er technology or Ur approach that is based upon all of the failures and has surpassed them. It’s a weird perspective on how progress works, I guess.

I actually really, really dislike the why don’t we build software the way that we build bridges, right? This sort of software, aesthetic engineering metaphor because I think that civil engineering is a static solution to a largely static problem.

Software as Cities

Zach: I think that if we’re going and looking for a metaphor for software, it’s much better to look at something which is a larger scope, which is like a city, right? Cities are constantly changing. Notably, if your city is successful, if it’s growing, people are going to be discontent with you just kind of keeping things the same, right?

Adam: Mm-hmm (affirmative).

Zach: If you are successful, people will demand more from you. The decisions that you made that were appropriate at a certain sort of scale or for a certain kind of population or whatever, will become inappropriate at some point and you will have to go and deal with that and that’s not an indictment of the past choices that you’ve made. It’s just you have to acknowledge that circumstances have changed and you need to go and understand which of your assumptions have survived that transition and which ones haven’t.

Adam: I’ve definitely been involved with software that fits the city metaphor because you need a historian to walk you through and say like, “Though this is the old part of the city and things here are done a little bit differently.” There is a brief modernist area that swept through this zone. That’s why.

Zach: Right. Maybe that’s not desirable but it’s like if you go and you look at, I mean, as part of the research for my book because I was quite swept up in this metaphor for a while, I’ve read a bunch about city planning, right? Planned cities are often bad cities.

Adam: Oh, interesting.

Zach: Because a lot of the planned cities, which were birthed out of this sort of modernist movement, was this sort of idea that they said, “A house is a machine for living.”

Adam: Oh, yeah.

Zach: We’re just going to make it very efficient. You can get these economies of scale, if instead of having people live in smaller buildings, will create mega blocks, where just all the people live. Then, all the living will happen in this part of the city and all the working will happen to this other part of the city, and then the recreating will be in the, yet, still a third part of the city and that’s not how people live, right?

Adam: Upfront design architecture, bad. That’s the lesson I’m intuiting here. You said before, in the Clojure community, there was this idea that there is a right way to do things. I mean, I think Python takes this very seriously. That seems like counter to what you’re saying. That sounds like central planning. It’s like this is the way to work with a list.

Zach: Well, yeah, that’s an interesting thing because I should be more clear in terms of how I characterize this because I think that Python is a well, purports to be a one way to do it language but I think that that’s not really true. I mean, you can go and you can do list comprehensions or you can do the other thing, right? I mean, you can go and say like this thing is idiomatic in this circumstance, but that’s not a, there’s one right way to do it, not there’s one obvious way to do it.

I think that Clojure does a pretty good job of actually of sort of putting people into a wellborn sort of groove in a number of different ways but I think that there are other things that it completely pumps on in terms of having sort of a normative strategies.

I don’t think that Clojure is a language that necessarily has such strict guidelines, that it’s clear what the right thing to do is. Rather, I think, that it is a language that puts a lot of importance on having done the right thing, because I don’t know if you’ve seen Rich Hickey’s talks at all but he’s a very thoughtful person. He is a very talented communicator about sort of the various aspects of software design. I think that people aspire to be as thoughtful as he is. They attribute their own failures to have like a clear understanding of what goes in the same namespace or doesn’t as insufficient thoughtfulness.

Clojure’s Looseness

Adam: Yeah, that’s an interesting dichotomy. I really like his talks. He clearly spends a lot of time thinking things through in depth, but my outsider perspective of Clojure is it’s more… Let me ask you, actually. Is day-to-day Clojure programming closer to bash or is it closer to some Haskell programming with very rigid constraints on interfaces?

Zach: There are a lot of ways I could slice that question. I mean, notably, Clojure has structured data.

Adam: Yeah.

Zach: It is or at least was when it first came out, I guess you could also kind of lump elixir in with this a little bit but it is a dynamic language that has immutable data structures. I think that that’s a very novel niche, right? Immutable data structures were the domain of Haskell and sort of the Haskell era and I think that the insight that this is useful, that dynamic languages aren’t just as sort of like Leroy Jenkins, kind of like dash into let’s just slap it together until it works, that there is a principled way to use dynamic types, was a novel insight on the part of rich.

I think that you spend a lot of time in Clojure dealing with things which are structured, but aren’t necessarily named. If I go and I’m building up a or I’m taking like an input and I’m trying to like build up some sort of derived data representation of that, I’m going and I’m composing together a bunch of these operations, typically using these sort of arrow combinator, which goes and takes like a thing.

Then, instead of having the sort of the typical list inside out structure of the code, which is I think, for most people very, very difficult to read, you just have a left to right thing. I have this and I want to apply this transformation. I want to apply the next transformation. You can just read it various sort of left to right or top down. In terms of step by step, here’s what I’m doing.

What you often don’t have, though, is a clear name for the data types that exists at every point within every intermediate point about composed operation. There’s a looseness there, where you’re kind of doing the Wile E. Coyote thing where you’re going, you’re kind of walking off of this cliff, which is like a named entity in your code base and you’re dashing across the empty space hoping that you’ll land and get another thing, which is like a named part of your codebase, these actual distinct entities, which are well understood. Then, between, you’re just going in sort of doing a bunch of fancy footwork to get from A to B.

I think that that’s actually a really reasonable thing to do because the one thing that I learned about names when I was writing the first chapter was, if you cannot give something a name, go for it. Naming is hard.

Adam: Yeah. Yeah.

Zach: Naming is a really easy way to go and miscommunicate about what your code is doing. If you can just go and say, “I don’t know, it’s like a map of this one thing that you have a name for, this other thing I’ve named for, relationship between them and sort of implied or whatever,” but really, at the end of the day, if you just go and focus on the terminus or the termini of the two different things, where did I come from, and where did I eventually end up.

Clojure is something where you can have a very pluggable degree of specificity. You can be very loose in places where looseness is valuable because you don’t have a good understanding of the domain yet, you’re just kind of trying stuff out or maybe the domain is something which is highly fluid, right?

Oftentimes, when you’re going and you’re working on a new product or with like a new kind of customer, there’s a lot of premature calcification, from my perspective, when you go and you have a much more strictly-typed sort of language because you’re often going and having to name these things or provide some sort of rigidity, which is not reflective of the degree that you actually understand your domain.

When you find that you’ve been wrong, you have to go and unwind that, in a way, that’s much more expensive than when you’re just like, “I don’t know, some stuff, right?” As I’m leaping from one cliff face to another, there’s some things going on there. If I have to change it, so be it.

Notably, the job that I recently took is a Scala [inaudible 00:41:36]. I’ve been working with Scala and really more than just kind of like, in having a passing familiarity with how it works. That is reflective of what I’m saying. I mean, by that same token, if you have a very rigid domain or a domain that is very well understood, the degree of specificity that is possible in a language like Scala far exceeds what Clojure is able to do, because it’s very easy for you to be like, I don’t know, some stuff happens, even though you want it to be quite narrow, right?

Adam: Mm-hmm (affirmative).

Zach: That’s a tradeoff there. I think that it’s quite possible that you can’t have a simple language that spans all of that, is able to be wholly fluid and then completely rigid. There have been some interesting projects on sort of adding gradual typing to Clojure but I’m unconvinced that that necessarily goes in like really gives you that whole range. I think it might actually be the sort of step function where it’s like, you have something which is largely dynamic, and something which is largely static. It’s very hard to go and find the happiest medium between those two.

Adam: That’s super interesting. There’s like the with open macro, and Clojure?

Zach: Mm-hmm (affirmative).

Adam: It’s pretty understandable, pretty simple, right? It’s kind of like, “Hey, shove some code inside of this, try, catch finally, and then make sure you kind of open and close things. There’s a Scala equivalent. I mean, there’s a couple of, the bracket is kind of something that people use, but I just found one on Google that was called with resources. It did almost the exact same thing.

It was much more verbose, mainly because it’s not a macro. It has to talk a lot about types. That’s to say, like, “Okay, I’m going to take in something that can be opened and closed and then I’m going to take in the returns type of whatever. Then, I’m going to take in a function that takes that type of whatever, and then turns it into a type. The body of it ends up the same, but the types are not something that a beginner like Scala would probably come up with is a bit complex.

Then, there’s trade-offs, right? The Clojure one is very understandable, I think, because you’re just like, “Oh, this code expands to this,” but then like, the Scala one can tell you like, “Hey, you pass something in that can’t be closed,” for instance.

Zach: I think that that is absolutely a trade-off but I think that whether people prefer one thing versus the other, obviously, there’s more to it and obviously, there are very concrete benefits and drawbacks to both approaches but I think that where you land and that because there is such a multifaceted thing is just like, what is your mental model for this? I’ve joked for a while that there are two kinds of programmers. People think that software is fundamentally math. People think that software is fundamentally literature, right?

Adam: Yeah.

Zach: I think that if you were to go and sort of draw a line through the sort of access of Haskell, Scala versus Clojure, Ruby, what have you, that would be actually probably fairly predictive but not wholly, right? Because again, it’s much more complex than I’m giving a credit for here.

Adam: It’s an interesting dynamic, right? Yeah. I feel like both of it fringe things. It’s like, “Hey, you guys both say you do functional programming. You guys should have a lot in common.” You’re like, “Well, it’s more complicated than that.”

Zach: Sure. Right. I mean, if it were simple, then it probably wouldn’t be even worth discussing, right?

Adam: Yeah.

Zach: But it is something where, for all the reasons we discussed earlier, I don’t think that the conversations that are being had about this are at all productive, right? I think that that’s kind of the shame that we have here, which is that it becomes this kind of tribal thing rather than an attempt to actually introspect and understand what is it that makes me feel that this is the right way to do this?

I find it really exciting when I find something that’s there. To just think that the rest of the world has nothing to teach you because software didn’t exist until quite recently is, I don’t know, it’s a sad way to live.

What Type Nerds Miss

Adam: Yeah, I agree with that. I mean, type systems came about before software existed, interesting fact from a sort of logical, philosophical tradition. One question I want to ask you, before we wrap up, what do the kind of math, I assume you put yourself in the literature side, what’s the math type level Haskell, et cetera, people get wrong or miss about abstraction and design?

Zach: Well, I mean, I think that on the math side of it, you’ll find great many more people who are willing to go and talk about an abstraction being correct. I think I talked about in the book and in the talk that you mentioned on abstraction is sort of the distinction between church numerals and cons cells. Both of them are fairly similar, right? They represent sort of the accumulation of a thing by adding a thing that refers to what came before it, right. Church numerals are just a bunch of functions wrapped around functions. Cons cells are linked list that you prepend to.

The point that I raised is like church numerals are timeless because math is timeless, or is at least is extremely slowly moving. Something that’s useful to construct a proof will likely be so for your lifetime, give or take, right? Whereas cons cells are something which are much more contextual, because notably between when the first Lisp implementations existed now, the ratio of clock cycle to memory latency has changed drastically, right?

It used to be that you could go and read from a memory in like a clock cycle or two and now it’s hundreds of clock cycles, to actually reach all the way out in the main memory. You want to have more coherency in terms of your data. You want to actually not be jumping all around the place is going on there. Notably in Clojure, it doesn’t really use cons cells for much of anything. It uses these chunk sequences that have 32 contiguous elements before it goes, it has like a reference to the rest of it.

That’s just reflection of where we are. Computers are not superfast moving. There’s still sort of these like von Neumann things and they still work in much the same way from a sufficiently abstractive point of view but it’s changing fast enough that we can’t just go and say, “There’s something timeless about Lisp.” From a proof perspective, it is, but to actually have a useful list, you have to have a bunch of things that are very tangled up in when was it made? What was it made for?

There’s a lot of history around this kind of idea of like, correct abstractions, right. I mean, the way that I go and I sort of deal with this in the book is just say, “Don’t call them abstractions. I call them module and talk about your model having invariants and talk about these sorts of things. If you go back to the original mention of abstraction in the CS literature, this is a paper called proof of correctness of data representations by Tony Hoare back in ‘72. This is what he talks about. He says, like, “Look, you can go and you can prove that your data type is correct by saying here are the invariants in the model and you can see that all the operations on the model maintain that invariant, and therefore that invariant is closed over all different compositions of those operations,” which is great.

It’s a very clean, clever way to go and sort of constructive proof but that doesn’t talk about whether or not it’s useful. I think that after much kind of talking about it, let me the answer your question simply, I think that there is a focus on the sort of, this ineffable kind of always out of reach sense of a correct abstraction, rather than just saying like, “Well, is it useful given this particular set of circumstances?”

I think that to the extent that there are these sorts of things, even in say, Haskell, there’s a lot of having to hold the compiler right to make sure that it can go and optimize things properly, to go and do what you sort of expected to do and there’s a lot of expertise required to do that. They say, “Well, okay, but that’s not intrinsic to Haskell, the language. That’s just its implementation. If we have the much vaunted sufficiently smart compiler, then this will be a problem. This is not a critique of Haskell just its current imperfect form.

What that presumes is imperfection is fleeting, right? We will escape that imperfection or will become an asymptotically reduced over time, as opposed to we’re just going to go and sort of bounce around a sort of like pinball machine of circumstances that is constantly changing and constantly making us seem like a year ago, we were complete fools for having thought that this could never change or whatever.

I much more view towards that. Change is constant. Things will fall out from underneath our feet and the trick is to go and be able to see just far enough into the future that we’re not constantly falling flat on our face, but also having a plan for when we do. It’s a much more pessimistic view of what it is that we can control about our lives. That’s not to say that there aren’t domains where someone is doing like a linear Algebra library, right?

Adam: Mm-hmm (affirmative).

Zach: Last time that the main LAPACK sort of Fortran set of subroutines changes when people switched off of like the Cray vector supercomputer model onto the Von Neumann, the more typical kind of feared cache kind of mechanism that exists today and then required them to rewrite it and it’s been pretty much constant ever since, there’s probably like a new set of ones for the GPU now, right, which is sort of actually closer to the Cray model but that’s a very stable domain, and you may work in a very stable domain and the sort of biases that I mentioned, may be completely sound within that domain or sound enough that it doesn’t really matter.

I’m not trying to say that everyone needs to see the world, the way that I see it. I think that you do, however, need to acknowledge that there is no such thing as timeless software in a general sense, right? All there is, is software that hasn’t existed long enough to be useless or a software that exist within an environment, which we are very carefully shielding it from the changing of the world.

There’s still mainframes running COBOL and I guarantee you, they’re just layers of calcified corporate structure around that just trying to make that not fall over. This is like the skyscraper that’s slowly sinking into the ground. We can’t do anything to it but we can try to go and do something around it. We have to try, right?

I just think that if we go and we kind of take that as a given, as a given that software can only be talked about in terms of contextual utility, as opposed to being right or being wrong or being correct or incorrect or whatever, then we can have much more nuanced conversations. We can also, I think, begin to go and apply the insights of a much wider set of softer studies out there, sort of the humanities, because the humanities is fundamentally focused on these sorts of ideas of contexts and how it applies and how it makes things sort of fall apart.

We can learn the lessons there and not have this kind of very blinkered sense that like, the only problem is that we haven’t imbibed enough math or category theory or enough or created a powerful of type systems like this. Those were all very useful. I don’t mean to go into sort of denigrate those. It’s just they are a tool. They’re not a solution.

Ending

Adam: Yeah, I think that’s a great place to end it. Thank you so much, Zach. This has been great.

Zach: Yeah. Thanks for having me.

Adam: All right, that was the interview. I hope you enjoyed it. I think Zach put his finger on a lot of problems with our online developer culture, lots of people talking past each other because they work in different contexts. It was also just a lot of fun to me to talk to somebody from the Clojure community, and kind of compare notes about Clojure versus Scala and just the different ways that he thinks about things. He really comes at things from a literature and philosophical perspective.

I hope you enjoyed the episode. Let me know if you didn’t or if you did, or just that you listened. Reach out, tweet, join the slack, et cetera. Until next time, thank you so much for listening.