Introduction
Adam: Hi, this is CoRecursive and I’m Adam Gordon Bell. Each episode is the story of a piece of software being built.
And today is all about something I’ve never used before, but that has shaped so much of how software development is done today, it’s almost hard to imagine a world without it. Can you guess what it is? It was first announced to a private group on August 1st 1995, 27 years ago. And eventually, the ideas embodied in it, spread to nearly all programming language communities. I’m talking about what led to our modern world of open source software development and to the world of building things by gluing packages together.
I’m talking about CPAN, The Comprehensive Perl Archive Network. Maybe you’ve heard of it, maybe you haven’t, but CPAN was the first open source software module repository. CPAN inspired everything that would follow. NPM, Maven, Cargo, NuGet, package, RubyGems, Python pypi, and so on and so forth. There’s so many.
If you’re building things today by pulling in various packages from various open source places and really who isn’t, then the history of how this world came to be is important. Let me say it again. CPAN should be something you know about. It was the original, it changed software development, and it has lessons to teach us about community and about collaboration.
So today, I want to share the early history of package management and how it shaped the world of software development and to discuss that, I have my former colleague and all around car guy, Don Mckay.
Don: Hello. I’m not that much of a car guy.
Adam: Listen, if you’ve after market modified your car, you’re a car guy to me.
Don: Maybe half car guy.
Adam: And also, I have my [inaudible 00:01:57] friend Krystal Maughan.
Krystal: Yay. Thank you so much for inviting me. And I’m really happy to be here today.
First There Was AWK
Adam: The cool thing about Perl is that it’s kind of like the story about the fun side of computing. Like people playing around with things.
If you go back to 1977, Unix existed, Anna had the shell, which meant that you could just get things done interactively by typing commands and piping them in. You didn’t need to write big programs in C. You could do like interactive stuff, which was a big deal at the time. And Brian Kernighan apparently used to do demos of doing spell check at the command line. Back in the 70s, somebody would take a document, a text file. And he would at the command line like chain together, splitting it up into words, and then joining those words against dictionary and returning all the words that weren’t in the dictionary. I made a spell check, just like typed it out in a couple of commands.
And so that was scripting. Like people now talk about, scripting languages versus non-scripting languages. And it seems pretty murky. But back then it was a big difference.
So, Brian and his colleagues, Alfred Aho and Peter Weinberger, they wanted to extend what you could do in the scripting mode and make computers more expressive. So, they created this language AWK where it was easy to parse text and where you didn’t have to declare variables ahead of time. And you used like dictionaries, like key values instead of arrays. It was programming that looked a lot modern like Python or Ruby instead of the C programming that was more common in Unix at that time.
So, that’s 1977, but then you fast forward, not many people had computers in 77, but you go to 87, 10 years forward when I was actually alive, although quite small. And Larry Wall announced Perl. And so he wanted to follow in this AWK tradition. He wanted to make something so that it was easy to do these little interactive computer tasks that people had to do.
Krystal: I did a mock interview once in AWK. It’s like Ops people use it quite a bit.
Perl and IRC
Adam: So, Larry Wall, he wanted to follow in that tradition. And so he announced Perl and he described it this way:
Perl is an interpreted language optimized for scanning arbitrary text files, extracting information from those text files and printing reports based on that information, it’s also a good language for many system management tasks.
Adam: So, it was 1987. And the one thing that was different from 1977 was that the early internet culture was starting to become a thing on ARPANET. So, Perl became not just a language that was a replacement for AWK. Although it did become that. It also became like a community. Computers were becoming more and more common. And so more people had jobs where they needed to like maintain computers or do things with them. And also one year later in 88, IRC was created, which became like also an important part of the Perl community.
Don: I spent a lot of time in IRC.
Krystal: Were you a nice person on IRC or were you?
Don: Nobody’s a nice person on IRC.
We used it for a purpose, you had a certain group that you were hanging out with, and it was just a way of to connect with people that were not in your local area, but yet shared your same interests. Because like growing up in like a smaller town, like not like a village or anything, but not like a big town. There weren’t a lot of people that were interested in video games at the time that I was going to like high school. It was mainly all sports. So, you had like a few friends around, but if there was no bigger community until IRC, and then there were channels for that kind of thing, for people that were fans of certain things, and you could connect with those people and talk with them. And yeah, I mean, anytime you introduced anonymity to the situation, you’re going to get a lot of people that commit questionable behavior.
Adam: What IRC channels were you on?
Don: So, it was mainly around Quake. And then there was one for D&D, I used to play D&D over IRC.
Adam: So, IRC much like it became an important part, of Don’s Quake community. It became an important part of the Perl community, but this was 1988 instead. And also news groups were a thing. It could maybe troll each other on IRC or share code, answer questions and stuff on the news group. Did you guys ever use news groups?
Krystal: I don’t recall using news groups at all.
Adam: Like very early, when we got the internet, I think the email program that we used, you could sign up for news groups and then I did. And then I just remember that it would take forever when you went to check your mail because it would download all these like threaded conversations of all these people, like talking in depth and I don’t know what they were talking about. And so I think I bailed out of it after probably using up a lot of our bandwidth, just like downloading old, like long threaded conversations.
Perl the Community
This was before the World Wide Web existed, like ARPANET existed. But there was there wasn’t websites. Mainly people at universities or research institutions, like using the internet for email and news groups and I guess like FTP existed I had for a long time. So, people would talk and argue on IRC, but then the web came along and websites came along and so people started sharing, these bits of Perl code that they were exchanging like on their websites or on the news groups.
It was sort of a new thing that people could meet with others like this and talk about a language remotely. And this isn’t like commercial software. This is just like people trying to get things done and sharing stuff.
And this went on for a while with more people hanging out online, more computers coming online, people talking and sharing code. And then in 1993 is when Marc Andreessen created the Mosaic web browser. And then the web started to really be a thing. And then there was the birth of CGI scripts. Do you guys know what CGI scripts are?
Enter the Web
Don: It was like the beginning of web applications.
Adam: Yeah. Because like before then websites were like, you returned this static document.
Don: Yeah. Like an HTML page, right? Like you just wrote the HTML and that’s what you, but like the CGI scripts would be like the way of using like an application, like code to like generate a response. It was revolutionary at the time. It was like a different way of thinking about things.
Adam: It was proposed on a news group as well. And originally it was writing programs in C, but then quickly people started using Perl. They started making these Perl scripts, these CGI scripts, you could hook it up to the web browser, up to your web server. And now you had dynamic content and this was really cool, webpages before then had been static. But now someone could share like a Perl script, which returned an image of a counter. And then this became like the web counter. Like when you went to a webpage, it would fire off this Perl script that would return an image and say, “Your user 35 to Adam’s awesome website.”
Don: This many hits, right. It was all hits.
Krystal: And they have the map where it’s from all over the world. Mostly from the US.
Adam: And so someone made a Perl script that was like a contact us form. So, you could fill out a form on the webpage. And then it could send off an email to the owner or I don’t know if you remember guest books, do you remember that was the thing?
Krystal: Yes. Oh my, I was not always a very good guest though.
Adam: What did you say?
Krystal: If their website was kind of crappy, like you want to be like, this sucks.
Adam: That’s awesome.
Don: You were trolling them. You’re like this sucks.
Adam: Yeah. Guest books were a bit before comments and articles. It wasn’t like, “I read this and I liked it.” It was more like about your whole website. I was browsing the World Wide Web and I came across your website. And you mentioned that you went to St. Mary’s school. Did you happen to go to school with my cousin Roger? Say hi to Roger.” It was just like a weird…
Don: Say hi to Roger.
Adam: Like a weird world, right?
Krystal: Yeah. I’ve always felt like it suited people who collect weird things, like somebody’s into birds or weird stickers or stamps or something. And they’re like, “This is my website.” Where I just show all the things I have and talk about infinitely about things that most people don’t care about. And some other person would be like, “I’m aware to about those things too.” And they would leave. They would sign the person’s guestbook.
Adam: You do needle point of old beer labels, I do that too. We’re the only two people in the world. Did you have a early website Krystal?
Krystal: I think I had a MySpace. I don’t even know.
Adam: How about Don? Did you have a early website?
Don: Yeah, I did. I used Macromedia Dreamweaver.
Krystal: Oh no.
Adam: Nice.
Don: I had it for my D&D game. So, I had a D&D group and I used to like post it up, not for like people on the internet, but from like our friends group, you can go on to the website and you can, if you miss a game, I would take all the notes that I took from the game and then type it up as a story of what happened in the last game. And then I would post it up and so that everyone could read it so that it became just one long story, and I had a website for that.
The Pack Rats
Adam: That’s awesome. So, what happens, right? People are sharing these scripts, like here’s how you make a guest book or here’s how you make whatever. People start sharing those and then other people start collecting them. This is valuable information.
I remember when I first got Napster and high speed internet and I was just like, I need to get all of the music, that’s just what I need to do. And I think it’s like a common thing, where you’re like, “This thing has value, let me collect it.” So, there was this mailing list that got set up for Perl people who were collecting code and they called themselves the Pack Rats. So, there was a mailing list. I don’t know their original motives, but they were collecting, various bits of Perl code. And they had FTP sites and they could share it around.
They said, we should pool our resources, we could mirror each other’s Perl scripts on each other’s FTP sites. Okay, so how do we do this? How do we divide the various scripts? How do we organize it? Should it even be FTP? The web is a thing now, maybe we should use that. There was gopher, which I think I never used, but I think was from before that time. So they decided to call this CPAN after CTAN, which was the TeX archive network for TeX and LaTeX. Krystal, what is TeX?
Krystal: So if you’re trying to write like the sum over and starting from one, it’s hard to write as in a document, like it renders the mathematical notation really easily. I actually use it like every day and I love it.
Building CPAN
Adam: So in this private email group, there’s like an email where somebody said, let’s do it, let’s do this CPAN thing. Let’s find a way to organize all these piles of code that we have.
And around this time also Perl 5 came out and it had this feature called modules. So you could organize code into packages. Like it didn’t just have to be like, here’s a guest book. You could say, make like a package that was like, okay, this does whatever: It figures out how old people are if you give it your birthday.
So they put together this master list. So the master list was a list of modules and where you could find them online. So like Don’s birthday calculator, and then here’s the link to Don’s FTP site. And then you have Krystal’s token ring module that’s on Krystal’s FTP site. And so on.
You could imagine building a script that goes through all these grabs, all these things and puts them on your FTP site. And then you’re running a CPAN mirror.
So they came up with this idea that they would create a place for people to put their modules. So this they eventually called PAUSE, The Perl Author’s Upload Server.
There was a guy named Andreas J. König and Jarkko Hietaniemi. So they, they were the big people. There was like several people on the list. But these two people kind of did a lot of the development and a lot of the early effort. And a guy named Tim Buns, he made this original module list.
Refining CPAN
But so building all this took time, they were going to build PAUSE and which is Perl author upload server. And you could like register and reserve a name. And then once you had a name, you could just upload your module code there. So Don could upload his token ring stuff. And Krystal, her birthday thing.
Don: Yeah, that’s the other way around.
Adam: And then, CPAN could mirror all this. Like it would all be in one spot.
And then one of them got this idea. Why don’t we just, instead of just listing it by name, that’s not really the way people use software. They’re not like, Hey, what’s great new software by Don. I should check out. Like if you’re trying to find a module to do something, you should categorize it. And so they came up with this idea, like, let’s do categories.
Categories
So they built a tree of categories that went from just being a old FTP thing with people’s names and files in it to them saying, oh, here’s a section for all the code that talks to databases. Here’s a section for all the code that talks to user interfaces and here’s file handling. And here’s the world wide web and all these collections of software. And it was still stored under people’s user names, but it’s just kind of, they like mirrored it under the different category names. So it was easier for people to find.
But yeah, it’s interesting how simple it is. Like if you just list by username, it’s sort of like GitHub. You go on GitHub, you can look under people’s usernames, but once they start organizing it by like modules, this is something cool.
Krystal: I feel like you could go full bore with the whole pack rat thing. And they had like a logo with like a one-eyed pirate rat and who was like in a box and they had t-shirts and a whole thing going.
I don’t know, I feel like that’s what drives a community as like online communities as well is like, let’s do a thing together.
Adam: No, I totally agree.
Announcing CPAN
Adam: So yeah, on August 1st, 1994, they announced this they’re like “we built it!” They, they told some other Perl people. And then on August 16th, the first package was uploaded. It took off. They let it have some time to bake and then on October 26th, that same year, 1994, they announced it publicly to all the Perl people on the Perl news group.
And then things changed because all these people could upload modules.
And yeah, like you said, Krystal, I love this idea that there’s these bunch of SysAdmins or whatever and they’re working at various places. It’s still early in computer history. So it probably in charge of the servers and between keeping everything running right on their nights and weekends, they were building this thing. They were building this thing that was: here’s a way that we can collect all of this Perl code.
And it wasn’t even until 1997, like, what is that five? No, my math is not so good. Three years later, it wasn’t until 1997, like three years later, O’Reilly had a Perl conference. They brought together all the Perl users. And that’s when, especially these two people, they first met each other. They were just strangers building this community, this little mailing list. And then they got to meet each other in person years later.
So this little group and this little mailing list, but really these two people, they built this thing that really changed things.
Here is user melling:
CPAN was Perl’s claim to fame. It’s what gave it an edge over Python and Ruby.
And here is user ceronman:
Thanks to the Perl community for building CPAN and having an easy to access repository of reusable libraries. It’s given us more power to develop cool stuff than any specific language features. That’s why nowadays every important programming language has some sort of CPAN clone.
CPAN Testers
And so CPAN and Perl, they were a community and people kept contributing to CPAN. And so by 99, there were around 200 packages a month being released on CPAN. And by 2001, this crossed to 500 releases per month. And it kept on building.
Sometime in 2004, there was a thousand monthly package releases. And some of these might have just been like minor upgrades of existing packages, but like a thousand a month is a lot, especially for 2004. And one thing that happens when you’re early like this, like you build this community and it’s early is you hit problems that nobody else has had. Nobody’s hit them before.
And one of the first problems they hit was like, how do you know if one of these modules is good? If it even works. You have thousand of them, thousand releases a month. So they created something to deal with this. Perl users had, they had various versions of Perl. They all had various OSS and various computers. So they had something in the Perl modules. You could run, make tests to run the test. People would just download modules and test them and report like: “Hey, I’m on this operating system, I’m on this OS version and it worked” or it failed. But that obviously that doesn’t scale very far, if you have like a thousand releases a month. So they had to come up with an automated option.
So people would volunteer machines. And then on that machine, it would download modules and run tests and send back to this website called CPAN testers like how things worked.
So it was in fact, a global CI continuous integration server for packages and CPAN, but it was distributed because people could volunteer their computers to just download all these packages and test it on all these versions. And then you could go on CPAN testers and see like, oh, this works on this and this and this. It must be good.
It’s kind of wild that they built that back then. CI is standard now but I don’t think any package repositories out there are like testing every release across all these various versions. Now these are commercial entities. But these cpantesters are just bunch of volunteers, just a distributed system of volunteer computers, testing all this code.
And then they added other things on top to deal with the scale. They added ratings for packages. They added a website that would index them.
And so you could search. So if you’re searching for whatever you needed to connect to, ICQ via Perl probably somebody has a module like that and you could search and find it.
Krystal: Were you on like ICQ and stuff back then?
Adam: Uh Oh!
Krystal: What?
Adam: Oh, you don’t know those ICQ noise.
Krystal: Oh, oh, the noise. Yeah. I, oh, sorry. Yeah.
CPAN’s Impact
Adam: Yeah. So it’s such a cool small community working towards a common goal and they’re all coming together and building this stuff and it really became Perls advantage. As other languages appeared, they wouldn’t have CPAN. So if you needed to do something, there was already a way to do it in Perl. And it was probably tested, and it probably worked.
If you want a random Chuck Norris saying. If you want to read some odd file format then probably CPAN already had something that handled this. And we take this for granted right now. I feel like every other language community that you can expect this now, but everybody got there by copying CPAN.
This was something magical that they came up with. And I’m sure at the time they were like, I think we’re onto something here.
They must have felt like: “look at what we’ve accrued all, these packages that do all this stuff”.
Adam’s Story
So in my world, right in my personal timeline after I went to university and we played with lots of languages, but really a lot of it was a Java. And I remember building stuff in Enterprise Java Beans (EJBs) and it was like a lot of XML and it was very crufty, but, but I thought it was so cool.
Enterprise Java Beans was like a open source thing that you had to download APACHE Tomcat. And for my final project, I built this EJB point of sale thing for a video store, a video rental store, which apparently the video rental store concept is more dated than the EJBs at this point. It wasn’t really like CPAN. It was more complicated. You had to get Apache Tomcat and like get it running.
And so I leave university and I get a job. And at the job we’re doing C# development and C# is really new at the time. And in Microsoft world, just like in Java, there was all these cool open source efforts. And a lot of them were just porting Java code over to .NET because .NET was new and Java existed. So there was Hibernate, was like a big persistence framework in Java. And then they brought it to .net. It was called NHibernate. And there was JUnit, was like a testing framework. And so then there was NUnit and there was ANT, which was Java building. And then, so there was NAnt.
Don, did you use C# back in these, this like C# 1.0 or something? I don’t know if you were involved back in those days.
Don: Yeah, no, I learned C# in college. There was a class, it was a sixth semester class about C#.
Good Bye Open Source
Adam: Yeah. So, I got this job, we were doing C# stuff and I was learning C# and I loved it. Visual studio was so much better than all the Java things I would use. And there was all these bloggers and they would be like, yeah, I created Nunit and isn’t an awesome, or I’d learn how to use NANT to build things.
But when I mastered it and I would learn like, oh, here’s how NANT works. And there’s all the community people explaining and it felt cool that I was involved in it. Then Microsoft was like, oh, we came up with this thing called MS Build. It’s exactly like NANT, but it’s better or something. And then everybody would switch. They would abandon this open source and the mailing list would die. Who’s going to use NANT now we’ve all moved to MS Build.
Don: I didn’t get to use NANT, but I am very familiar with MS Build.
Krystal: Yeah. I’ve heard of MS Build as well. That’s so frustrating.
Adam:
So then there was Nunit, which was the unit testing framework. And then there was this one that was called MBUnit and the guy who built it was like brilliant. And it could do all these different kind of testing things. And I thought MBUnit was so cool.
Then this guy, he got a job working for Microsoft. And then Microsoft came out with this thing called MSTest. And it was a lot like NUnit, but didn’t have all the cool features of MBUnit. It was just like a standard way to run tests. And the cool thing about all the open source ones, to me was, there was these people with blogs and they were describing how it worked or there’d be mailing list and people chatting about it and this community.
An Act of War
Adam: And slowly in my time, in early .NET, it just seemed like an act of war. Microsoft was crushing all these open source projects. There’d be a very cool, very talented open source programmer, building something. And a community would build up against them. And then Microsoft would be like, oh, we’ve built a version of that. And then everybody would just switch to that because, well, it was Microsoft.
You were already bought into Microsoft if you were using C# and .NET. So if Microsoft comes in with a product, you’re like, well, I’m not going to use that open source one by just some guy on this blog, I’m going to switch to that.
And I struggled with this. I got mad about this. I started working with you, Don, but I was building some of my own stuff on the side.
ALT.NET
Adam: And a lot of these open source .NET people, they formed this community that they called ALT .NET. Like it was supposed to be like alternative. They were like feeling what I felt, but probably even stronger because they built these tools.
You know, we want to be .NET developers, but we want to use the things we built. We want to use our open source tools and we want to support them. And we’re like an alternative world.
It was like this CPAN world that CPAN started was trying to blossom in the .NET community. Intentionally or unintentionally, it was being crushed. It was being squashed. You can’t build up a community around Microsoft releases MS Test and put some documentation online. That’s not a community.
Don: And then in 2014 they make .net core open source.
Adam: Yeah. I mean only a decade later or something.
And you couldn’t even see the source of the code you were using and the .net framework. You couldn’t go into the source code for the methods you were calling and see how it worked. You couldn’t learn from it. You could just consume it.
Don: It was a black box. You send things in, you get things out and that’s in.
A Horrible Alternate World
Adam: Yeah. And like Microsoft has changed. As Don said, they’ve now like embraced this model. I mean, I’m not part of the .net world anymore, but they’ve changed very much.
And the reason I bring this up is because I feel like there’s an alternate world. There’s a different timeline where CPAN didn’t happen. And this model that Microsoft had of like handing everything down, we could live in this world where Microsoft or Google, they build all the software packages. And everybody who builds software is just like a consumer of these black boxes and how software works is handed down to you. And you don’t get to understand how it works. And I’m glad we don’t live in that world.
Don: Yeah. There was a big push for proprietary solutions like early on. There was a lot of pushback I had from within Microsoft as well by prominent developers.
Krystal: Even though they worked for these companies, they’re part of those communities too. They’re bringing back the information to Microsoft about, Hey, this is cool package or this cool library that this person just made. And I don’t know. I find that to be incredibly frustrating.
Adam: Oh, well just wait, it’s going to get worse. I’m not against people making money. But like they were against people, freely sharing software. The thing I didn’t like is it separated people into hierarchies. There’s the people who are building the important libraries who work at Microsoft – if you’re in the .NET world – and then there’s the consumers of it. You’re either part of that, or you’re not.
Mort, Elvis and Einstein
Adam: This is the awesome part.
Microsoft used to have these personas and they publicly published this, which seems like a horrible idea. So they had built marketing personas of the developers who used their software. They had three separate personas. All of them were dudes. That’s actually not the bad part. So they’re all men, but they were called Einstein, Elvis and Mort.
So Einstein was a genius to their description. And he built everything himself and he cared a lot about performance.
Elvis whose pitcher was literally of Elvis Presley was a pragmatic developer.
And then Mort, Mort was dumb. You could just tell like, I feel bad for people named Mortimore, but they’re like, Mort just does whatever he needs to do to get his job done and then goes home.
It’s very demeaning to all these people who are using the .NET software kind of what they were saying is like, oh, there’s the C++ people who really get into the performance and stuff and understand how computers work. And then there’s Mort. It’s like VB.net, C# devs, that’s Mort.
Krystal: Yeah.
Adam: Like we’re not that dumb.
Krystal: I mean, I feel like it also paints people as like this static kind of persona too. It’s like people can start off not being super knowledgeable about a thing and eventually become experts in things. So I don’t yeah. Just painting somebody as a Mort is, that’s terrible.
Adam: So they did receive a backlash from this. Here is a Scott Bellware talking about it.
The Microsoft developer personas that include Mort, Elvis and Einstein are ultimately an ethically bankrupt mechanism to pigeonhole software developers into overly simplified categories that typical marketing staffers comfortable with. It appears to be a bid by developers to rid themselves of the capacity for rational thought in favor of tribal identification with corporate brands and rockstars.
Adam: It’s developers building these products and they’re like, Hey, make sure you build this for Mort. Don’t give any of those fancy options that might confuse.
I mean, this is the alternative world that CPAN helped us avoid where you could be Elvis or a Mort, or if you’re smart enough, you might be in Einstein. And then you might work for one of these big companies. You might work for Google, or you might work for Microsoft, or you might just bounce out of the field because the field doesn’t reward deep thinking.
If everything is targeted at people with this assumption that they’re not smart, then what does that say? People will just leave. The CPAN model is different. Everyone can come. Everyone’s part of the community. Everyone can contribute.
I really loved C#. And I really liked, especially they had the LINQ style, concise syntax that you could write, but the .NET community back then it was stifling. It seemed like they didn’t want like makers and people who wanted to create stuff. They wanted consumers and fanboy of whatever the latest thing they were putting out. And they put out a lot of cool stuff. But yeah, to me, it felt like a problematic scene.
And I don’t want to blame Microsoft for all this. Microsoft is a big place like Don was saying, there’s lots of people there. And I’m sure that there was people in the community, there was people at Microsoft who wanted things open source.
I mean, I feel like I’m a little bit bitter about this. I don’t know if I’m too bitter. It’s too better guys?
Krystal: No. I don’t think.
Don: There’s always going to be a struggle within the company because there’s going to be people who don’t develop, who don’t really appreciate open source communities. And they’re just looking for the way to increase the bottom line. Because that’s what a corporation does. And those people are at a constant, like kind of tug of war. Yeah.
It was October 23rd and 2021. There was an article on the verge where there was backlash against Microsoft because they removed a key feature from the .NET 6.0 release that allowed hot reload. So you can modify your source code for your app while it’s running. And they removed it and locked it to visual studio 2022, which is their mostly paid product.
And everyone was upset because you couldn’t get it for free anymore. And, but it’s like a crucial feature that if you’re a programmer, you like to be able to modify your code while it’s running, it’s just very convenient. And they reverse that decision. But like Adam was saying, Microsoft is a very large company full of a lot of people and there’s a lot of differing opinions on what that company should do. And you see that every once in a while.
Microsoft’s Perspective
Adam: And I’m sure from Microsoft’s perspective, they’re like, Hey, if somebody builds an important library that does something, they’re like, oh, there’s market demand for that. But from the outside, it just there’s no community. It’s just killing it. There’s no room for outside voices.
Krystal: If you’re within Microsoft, you’re still part of the community. But what is your responsibility now? Like do you think that you’re still, let’s say you were making an open source package and you get hired by Microsoft and they’re like, we want to close source and make money from this package check. How much do you as an individual owe to uphold and kind of speak for the community that you came from?
Adam: It’s super tricky. And I think it has to do with how much you feel like you’re part of that community. I think the reason CPAN probably did so well in those early days is because well, there was no commercial interest. It was just this Larry Guy built Perl. But also the community all felt pretty tight and they weren’t selling commercial software. They were all just people trying to solve problems.
And like, I’m not against capitalism. I like to get paid, but it’s hard to mix with this concept of these people kind of contributing and bringing things together.
If I’m a dev in one community and I can produce PDFs by running like a single command and pulling in someone else’s code. And then in another community, I have to go, talk to somebody and we have to buy a license to a software package for PDFs and add in some DLLs and do the licensing and stuff. Not that the people who built the PDFs don’t need to get paid, but the guy using CPAN who can just pull it down, he can get more done. Might not be as good as like the paid one, but they can move faster.
Communities Grow Developers
Adam: But I think the cool thing is what happens next. If they find a problem with that PDF generating thing, they reach out to whoever created it and they suggest a fix or they submit a patch or they just file a bug. It might not be easy for them to fix it. It might be a huge challenge, but all of a sudden he, or she isn’t just left as a Mort. They’re not this Microsoft, they just consume packages. That person is now learning how to contribute. She can contribute to the community. She can join.
This was the big mistake Microsoft made or any commercial company. They don’t see that the community can grow a developer from one level to another. People aren’t born great developers. They’re created, they need community to nurture them. And I think this is what these communities like CPAN or wherever got right. The community Springs up around these packages. And then people try to build it together. And that’s how you learn.
You can look at the code, you can suggest things. And that brings you to that next level where you can build packages like this. So like the more that’s built on this community, the easier it is to accomplish things, because you’re sort of training more and more people how to build the type of software that needs to get built. There’s just so much software out there that needs to get built.
Maybe at some point that’ll stop being the case, but nobody’s at in danger right now of not having anything to do if they don’t monetize like every single thing. So CPAN was the first with this idea. But, it spread from there. So Python has pypi and Ruby, which was supposed to be a direct improvement upon Perl – it has Ruby gems.
And all these communities, they would have people in them that came from the Perl community who would make noise if things were less easy to use. They would say like, this isn’t as good as CPAN. We need to improve that. It’s like a cultural idea that spread. It’s like COVID maybe to use a bad example. Like once COVID infected enough people in a community and then they would spread to other communities and this thing would keep spreading.
Don: So the idea that we can all work together as a virus.
Krystal: Maybe not the best, yeah, analogy.
Third Party Packages for Everyone
Adam: It’s like a mind virus. Anyways, here is in 2009, Guido Van Rossume, the Python creator on a mailing list:
People want CPAN. I just found this comment left on my block. People want CTAN. People have told me this in person too. So I believe it’s a real pain, but I don’t know how to improve the world. Do we need more than pypi?
And so pypi is just the package ecosystem for Python. People from the Perl community would join and Python, like in this mail thread, they broke out into huge discussion. You need to improve this. You need to improve that. The table stakes had been raised. Like everybody needed to be this good. So that was 2009.
Another thing that happened in 2009 was Microsoft changed. So in 2009, something appeared called The Outercurve Foundation. And if you looked into it, it was just like a nonprofit that was opened by Microsoft and Microsoft employees worked for it. All the open source people were like, thought this was some big conspiracy to undermine open source, but it actually wasn’t that. They were actually embracing the world of open source. Probably some people within Microsoft got this idea going. But the world had changed. Microsoft was funding a open source nonprofit. And then they built NuGet, which was .net’s answer to CPAN.
They built a package manager, open source where people could upload packages for .net and Microsoft put all their packages in that too. So the world had changed. CPAN – using my infectious disease metaphor, right – had spread out into the world. And now it had even gotten into Microsoft.
Krystal: That’s interesting. I can understand now why there was such a big uproar when that whole GitHub…
Adam: Oh yeah.
Krystal: Acquisition thing. There’s so many developers, I knew who immediately were like, I’m moving to GitLab, but having heard all of this from you now, it’s in that’s interesting.
Adam: And the thing is guy was a fanboy, like C# and Visual Studio and .Net were really nice. Microsoft was good and continues to always be good at building developer tools, visual studio code. Super good. But yeah, it took them a while to figure out this open source thing.
Developer Burn Out
Krystal: That whole Mort thing too, like the whole Mort idea. I don’t think that’s necessarily something that Microsoft came up with. I don’t think anybody believes that all developers are equal.
Don: There’s the opposite that I see perpetuated by a lot of actual developers, which is like, you have to live, breathe, eat, sleep code 24 7, or you’re no good.
Krystal: Yeah.
Don: And that’s not true either. And there’s like two extremes. It’s like, no, if you’re not like crunching code, when you go home pumping out some kind of cool package on your own time, then you’re not a good developer. There’s like people who can be really interested in coding and can like apply themselves at work. But when they clock out, they can do other things. Like that’s possible. People have lives.
Krystal: Burn out is a real thing. Like if you’re a central maintainer of a package, like,
Adam: Oh yeah.
Krystal: You can burn out too. Like just having some kind of policy to, for what if someone just doesn’t want to deal with maintaining a thing anymore.
Adam: Yeah. How many tech things, or programming languages have I been excited about? And then like a year later I’m doing something completely different. Like, thank God I didn’t build something really important in one of those. And I’d be like, oh my God, I don’t remember how Idris works. Like I got to maintain this thing.
Don: You have to go back to Scala Adam. What would you do? You can’t, you don’t remember?
Adam: I think I remember some things.
Krystal: Print. Hello world.
NPM and Scaling a Community
Adam: So also in 2009, NPM was created by Isaac Schlueter. He created it after saying that he had seen module packing done horribly and he wanted to do it well. And he wanted to take inspiration from projects like CPAN.
When it comes to talking about packages and people sharing them like NPM is where people start rolling their eyes and saying like, well, that community is all messed up. But I think it has a bad reputation.
So in March 11th, 2016, there was this guy named Azure. And he got an email from this other person named Bob Stranton. And Bob said like, Hey, you have this package named kik. It’s like, K-I-K on NPM. And I work for a company called kik and we want to use that name. So you should give us that. And Azure said, no, that’s my open source project. No, thanks. And then Bob said the following:
We don’t mean to be a dick about it, but it’s a registered trademark in most countries around the world. And if you actually release an open source project called kik, our trademark lawyers are going to be banging on your door, taking down your accounts and stuff like that. We’d have no choice, but to do that because you have to enforce your trademarks.
So this is the best part, because what Azure says back is “you’re actually being a dick so fuck you.”
Krystal: Yes. That’s the first thing I thought was like, why say we don’t mean to be a dick about.
Adam: Like, correct me if I’m wrong. But if he had the name first, even if they had a trademark, he could prove that his name was there first.
Krystal: Yep.
Adam: Yeah. I don’t even know how works, but here’s where it goes sideways.
KIK Wins, Everyone Loses
Adam: So Bob goes to NPM who are a commercial entity. It’s not quite like CPAN, they’re a company. They charge some corporations money. And they say the same thing to NPM I assume we’re going to sue you. So NPM takes it. They take it away from Azure and they give it to this Bob guy and KIK.
And Azure says, listen, you have hundreds of packages on NPM that overlap with trademarks. And he was right. I mean, WeWork trademarked the word We. Square, the payment company, they try to own square.
Don: Well, trademarks are also … they’re confined within the framing of, you have to mistake that for something else that’s in that particular industry. So if they trademark square, you could have like a shoe company called square. No, one’s going to mistake your shoe company for the payment processor. Even though they’re named the same thing, but if you have something that’s technology adjacent, that’s called square, then square be like, I don’t know. People will think that we’re affiliated with you or that we make that product as well.
LeftPad
Adam: I mean, NPM probably gets scared of the talk of lawyers. I don’t know. But anyways, what happens is Azure gets mad because they, they sided with kik. And so he deletes all his packages on NPM. It turns out he has 273 of them. Like he’s actually a pretty big contributor to NPM and like not the most popular of his packages, but turns out one of the most important ones is leftpad.
So leftpad was a NPM library that just can like pad out a number or a string. And when he removes that, it basically breaks the internet because guess what uses left pad: React. And everybody used React. Everybody still uses React.
Don: Yeah. I remember that.
Adam: And so thousands of popular packages stopped working. I don’t know how many websites, like they just couldn’t build it because they get an error, because it would try to pull down this library that didn’t exist and they could no longer build their site.
Don: That’s kind of a scorched earth policy. You’re hurting a whole bunch of people that aren’t even involved at all.
Adam: I agree. He was probably pissed and probably NPM should have cached these dependencies so that like everybody’s website didn’t break the moment he pulled it. But yeah.
So what happens is NPM. They freak out and so they go in and they unpublished his code. They republished his leftpad thing to like fix everything. They put it back up. I mean it’s only 11 lines of JavaScript. And people don’t talk about this but I think they broke the expectations of what this package manager was. Like they sided with like an outsider corporate entity rather than like the users of the community.
Protect the Community
Adam: And then the story all became about how NPM is a mess and jokes about how like JavaScript developers have too many packages and whatever. But I think the main thing that was missed was that community part.
NPM is big, much bigger than CPAN. And it’s hard when you get to a certain size to maintain the sort of like community spirit, but it’s especially hard when you feel like these decisions are going against you, like you’re building things for this faceless group and then you’re getting demands that somebody’s going to sue you because of some like little package you made. Imagine you just throw up some software somewhere and some lawyers contacting you and saying like, we’re going to sue you because of this.
Don: It puts up a barrier to entry. Like I don’t think I really want to be part of that.
Adam: Exactly. Right.
Don: I mean, this problem isn’t even limited to development. I mean, content creators face this problem all the time with YouTube. Like they’re the ones creating the content. But they’re also the ones living in fear from like DCMA takedowns and stuff like that.
Adam: Yeah. And like Kik is like a second-rate WhatsApp clone that’s being investigated by the SEC because they issued their own like crypto coin because they ran out of money and whatever. And like there’s allegations of child predation. Like they were never going to be the heroes of the story.
But this is just a problem of communities. It’s hard to scale communities to a certain size. And when it becomes like a faceless NPM organization versus some person building something, that’s just what happens.
And so since then, like all kinds of other problems have happened. Like a lot of them in the NPM world, just because it’s the biggest, I think simple ones are just messages asking for sponsorship. People saying like, Hey, you’re using my code. Guess what, I don’t have a job. Please send me money. Which is just a little bit annoying, I suppose.
And then there’s open source, supply chain attacks. You could pay a popular package owner to add a back-door into their code or people just upload typos of popular packages and hope somebody makes a spelling mistake and then pull in a Bitcoin minor.
Or there’s like log4shell. log4J, the Java logging, thing like it’s just complicated enough that people found an exploit and could take over people’s servers.
The very latest one is protestware. So to protest Russia’s invasion of the Ukraine, this person who ran the node-ipc package, he briefly put in code that geo located IP, and if you were inside Russia, it would delete all the files on your hard drive, which was a really bad idea. Some US charity in Russia, like had their files wiped out by this. And then he quickly like backtracked and changed it. So it’s just, if you were in Russia, it wouldn’t run and would just produce like a heart. Like that wasn’t going to fix the war against Russia aggressing on the Ukraine.
Don: Could definitely ruin that charity though.
Dump Trucks of Money
Adam: Yeah. Right. We just live in a divisive world. I think that you’re going to keep hearing stories like this, the world of package managers, it shouldn’t just turn into this seedy world of trademark fights and paid sponsorships and people begging for money.
So the question is how do we fix that? Like it started as a small group of people in the Perl world, but now these packages have become big things. How do we prevent it from going bad?
Krystal: That’s a deep question, Adam.
Don: All of them will eventually be bought by some kind of corporation.
Krystal: No.
Don: Yeah. Like new get or NPM. NPM is actually owned by Microsoft.
Adam: Is it?
Don: Well, they own GitHub that closed the deal to buy NPM in 2020.
Adam: If I put on my Microsoft’s Evil Hat, Microsoft is slowly buying up all of the developer stuff. So they can slowly return us to the world of proprietary only software. I don’t think that’s what’s happening.
Don: Well, no, I don’t think that’s what’s happening, but like a lot of these, a lot of these software solutions are being like eyeballed by corporate interest.
Adam: Oh yeah.
Don: And they usually come up with a dump truck full of money. I mean, who’s going to say no to that. I think that there’s still a lot of open source repos out there. That aren’t controlled by corporate interest, but they’re a dying breed.
So you’re always going to have the corporate consolidation where everything just kind of eventually becomes part of Micro-Bucks-Mart. And they’re like the only Corp and they own everything. And that’s the natural endpoint of the capitalist society that they’ve built. It’s up to us to try and keep our packages out of that system. And it’s going to be up to the authors of those content packages to stop using things like NPM, if they get too corporate or if they start restricting access to people with paid memberships or they stop doing open source, they’re going to have to go somewhere else and make a new system.
Adam: You two .. I have you on usually to counterbalance me, like when I go off talking on this rant about something and you guys be like, why is that important?
Don: Well, what side am I going to take the side of like corporate? It’s like, oh no, everything should be like proprietary.
Adam: That’s the problem today though. I don’t have you two to counterbalance me. You guys are both like, yeah, no submit to socialism. It’s the way forward. They’re like, no, I actually, I’m not against capitalism at all. Like I just think that the package world, like this is something cool. We shouldn’t let this fade away. This like community of people building awesome things.
Don: But do you think it’s fading though? Because like I see open source communities all over the place.
Scaling a Community Is Hard
Adam: Oh, I don’t think it’s fading at all. But like I worry, like the bigger you get, the harder it is to keep a community intact. I see backlash of people against open source users. Like there was recently this Python guy, this guy made a Python package and it was considered critical because so many people used it and they asked him to set up two factor authentication to post updates and he was like, fuck you. And he deleted it
Don: And he deleted it. Yeah.
Adam: What’s happened is that guy doesn’t feel like a part of that community. He feels like they’re taking from him. He’s having to maintain the software and he’s not getting anything back. He’s not feeling the love from that community.
Don: But isn’t that a little unreasonable though, like put two factor in your thing and he’s like, fuck you. And he like deletes it. Like don’t you think that’s a little extreme?
Adam: Oh no, I do.
Krystal: I mean, some people are just jerks too. I mean some people make things, make packages because it’s for themselves.
Don: That’s the con of the community. The community’s full of people and people are flawed. So you’re going to have people that are just like, nah, and they can delete their package. Oh, well, whoops.
A Reasonable Solution
Adam: So I’ll tell you what I think my solution is.
The original CPAN was built by a community, it was built by a community for a community and that’s the way we fix things. You just have to lean in to the community aspect. And this is like a super vague plan to save the world of packages that I just kind of made up: but like we celebrate package creators.
Like in the CPAN world one thing they had was let module advents like an advent calendar, where for the month of December leading up to Christmas, people would like share cool packages like on their blog or whatever. And this is spread to other communities. And I think celebrating the authors of these packages is super important. People are often not doing this for the money, or if they are the amount of money that like you could give them, probably wouldn’t make a difference.
Anyway, they’re often doing it because of this ownership as part of this community. Package repositories and open source, they seem like code, but they’re not really about code. They’re really communities. They’re like groups of people and you need to nurture that. Like treat them as people. And when I say you, I mean like specifically like Don, Krystal, me, Adam, the listeners, like we need to work to make the community better.
People are hard. Like people need care. If, if you build a community and it’s successful, like it’ll keep growing and it’ll get bigger. And then you’ll start hitting some of these problems like NPM.
So people shouldn’t laugh at NPM because of these problems. They’re just actually so big that it’s hard to keep the community together
And I guess this is more of a policy recommendation but at some size, communities need to have the community’s interest at heart. They should have like some sort of board of people who make recommendations against it. Their criteria should be what benefits the community. Because this is kind of like a weird collectivist communist thing.
Don: You sound like you want to create a union. That’s what it sounds like. You sound like you need a union of package creators so that they have a collective voice against.
Adam: But it’s not a voice against. Let me give you an example.
So if I create a package and then I just lose interest in the community and people are like, there needs to be updates in that. Like right now it’s not clear, like I own the package. And maybe I don’t want to open it. Maybe I’ll just like delete it or whatever. But a community kind of needs to protect the community’s interest.
At some point, if something’s important enough, it’s more important then that single author. People should be able to step in to make changes to it, to improve it. It’s putting the community’s interest at a certain point above the individual contributors. Like you need to celebrate the contributors, but at some point you need to protect the community. Which probably is what NPM was trying to do by undeleting that package.
Imagine if you said like, listen, when you push your package to whatever Adam’s repo, like you agree that there’s a process for taking it over. Like if we deem it to be important enough, like we’ll take it over.
Don: Too many people depend on it.
Adam: The CNCF, which is big in the Kubernetes world and the Apache foundation, they both work this way. You donate important software to them and you can still help make it work open source and whatever. But they have processes in place to benefit the users of it.
Because that’s how you build something collectivist. You keep the collectivist ideas in mind. If you build up Goodwill, if you try to have the community’s interests at heart and everybody tries to get along to the extent that they can, and then occasionally you take actions – taking a package from somebody – I think the community will have your back because they know that you have their best interests at heart. Let’s not go back to the world of commercial packages being handed down to us Morts to consume from on high.
Perl has fallen out Vogue. Maybe the reasons that it happened are valid, maybe they’re not. But everybody’s learned how CPAN works. And this idea has spread. CPAN’s releases are actually dwindling now, but they’re actually far above where they were back in 2004 because just the world of software is so much bigger now.
And this is when it gets hard. I think that there’s going to be more and more stories about like package stuff going awry. And people are going to say like, this is a mess and we need to do X and Y, but I think they’re wrong.
Like I think the thing we need to do is just come together and support each other and build cool software. If you want to live in this world of open source communities, probably you don’t maintain some major open source package yourself. Like statistically, most people are just consuming this stuff, not generating it, but you use it and you do issues and you ask for features, and so nurture it, thank the creators, follow them online, help them with their package, stick money in their pocket, buy them a coffee.
They’re the goose that lays the golden egg. That is our world of software development.
That was my long riot.
Don: That’s a rant.
Outro
Adam: That was the show. Thank you, Don and Krystal for being here. Thank you to Andreas J. König, Jarkko Hietaniemi, Tim Bunce, everyone who made CPAN, what it is, everybody who uploaded things and helped build that amazing community. And thanks also to the online CPAN timeline for laying out a lot of these details for me.
If you like the show, please support me on Patreon for access to tons of bonus content.
And until next time, thank you so much for listening.
Sticher
Overcast
Dec 02, 2024
42
min