Adam: Welcome to CoRecursive. Each episode is the story of a piece of software being built.
Help Adam Find His Next Role
Adam: Before we dive into today’s episode, I need your help. Like truly, I’m on the hunt for a new developer relations role after being laid off from Earthly. During my time at Earthly, I really honed my skills in combining coding with communication.
I’m talking about making tutorials, doing conference talks, YouTube videos, the whole shebang. My developer-focused content had a lot of impact at Earthly. And so now I’m looking for a new role where I can keep flexing my software development muscles and my communication powers.
And so I’m looking for a DevRel role. I know it’s a bit of a niche, but if you’re aware of one, that’s where I need your help, right? If you know of any dev tool companies or dev op outfits or app security tools, or AI ML places where they need someone who can speak engineer, who can communicate to developers, that’s me.
I’m your guy for explaining complex stuff in a way that’s catchy and fun and makes sense to software developers. So here’s how you can help. If you know of any roles like this, let me know. Who should I be talking to?
You can reach out to me, Adam@CoRecursive.com or on Twitter @AdamGordonBell. I’ll put a link to my LinkedIn in the show notes and a link to my calendar. If you want to set up a call and just tell me about an opportunity, then go for it. I’m excited to find a new role where I can keep doing what I think I’m pretty good at.
I think I’m good at developer communication and hey, maybe you even know about something cool that I haven’t even thought of. And if so, let me know. I’m all ears, but all right, that’s enough about me. Let me know if you can help me out. But now let’s get to the podcast
Intro
Adam: Today, we have the story of the creation of PowerShell, a tool that transformed Windows system administration forever.
And it’s a fascinating story because of all the challenges it took to get it built, especially because at the time that PowerShell was built, the culture at Microsoft was definitely going a different way.
Jeffrey: By the way, is it okay to swear?
Adam: Oh yeah. You could swear
Jeffrey: You know, I had executives say, ‘Jeffrey, exactly which part of fucking Windows is confusing you, Jeffrey?’
Two keynotes ago, Bill Gates got up there and said, ‘Look, here’s command.exe. It’s the last time you’re ever going to see it.’ And he types exit, carriage return, and it goes away.
Like, remember that? And then what the hell is this Snover guy talking about?
Comes in from the outside, doesn’t drive a freaking Porsche, so let’s, let’s get that in focus. Okay, not driving a Porsche, uh, and he’s saying we need to do command line interfaces. But over here, you look, I mean, I wrote this GUI, I got a promo, John wrote a GUI, he got a promo.
Like, who’s the person who wrote a command line interface and got a promo?
Adam: That’s Jeffrey Snover. He’s now a distinguished engineer at Google. And yeah, he is the creator along with his team of PowerShell.
Today, he’s going to share his insights on navigating large organizations and driving change.
How to achieve significant outcomes despite strong opposition, right? This is the essence of technical leadership. PowerShell, if you’re not familiar, was groundbreaking, not only influencing other CLIs but enabling Microsoft’s move to the cloud, and maybe most importantly, cultivating a vast network of skilled Windows system administrators.
In fact, Jeffrey got his start at Microsoft because, in 1999, Bill Gates, at the height of the relevance of Windows, well, Bill Gates was worried that Microsoft just didn’t understand the data center, didn’t understand the enterprise market. That’s where our story starts.
The Problem
Jeffrey: So this was kind of a fixation. Hey, how do we make sure we’re not complacent? That we’re not sitting there patting ourselves on the back and saying, Hey, this is great. So yeah, we’ve been wildly successful with the PC. But look at all this money being spent in the, in the enterprise software marketplace.
We want to be good at that. What do we have to take? Now, again, there’s the organization, then there’s the leadership. Like Bill understood that. And so then when the organization tried to do it, tried to do it, tried to do it, that’s where they had that famous, you know, meeting where he’s like, ‘Okay, I think we need some help.’
Let’s bring in some people who understand the problem better.
Selling Jeffrey
Adam: The problem was clear. They didn’t understand the server market. Their executives were skilled in the personal computer realm, but they lacked enterprise experience. That’s when they found Jeffrey.
Jeffrey: A Microsoft executive who I met a couple of times before, just an awesome guy, Dave Thompson, reached out and said, ‘Well, Jeffrey, I’d like to talk to you.’
I was like, ‘Well, okay, I’d like to talk to you, Dave.’
Anyway, at the end of that conversation, I ended up having my final interview with Jim Algen. He wanted to hire me and I said, ‘No, your software is crap.’ And he said, ‘Jeffrey, I know, and I need help.’ And think of it this way: if you do that, if you come here and help me fix this, think of the effect you’ll have on the world.
That was like a laser-guided missile to my psyche.
It’s like, ‘Yeah, I’m in.’
Beat Unix
Adam: The goal was clear to establish Microsoft NT based operating systems in data centers and to outcompete Unix vendors like Sun and IBM and HP.
Jeffrey: How do we do that? And the answer is, we do the same stuff, but with a lot lower cost, right? Same capability, a lot lower cost. One, we have an intrinsic price advantage because we were on Intel and most of the Unix vendors were using it as a mechanism to sell proprietary hardware. So we had Intel, but we also had this like open hardware ecosystem.
So that gave us a structural advantage. So now our software just had to be as good and boom, we win.
Adam: But the software wasn’t as good, especially for managing many servers. You are physically clicking a mouse and configuring things on every machine and the setup you need for each business might vary.
Jeffrey: A bank is different than an industrial control process, is different than a, you know, a scientific lab. Everybody’s different. And so basically, if the scenario doesn’t work, then what do you do? And the answer is IBM Professional Services.
Jeffrey: Or, Arthur Anderson or you know, systems integrators, right?
So now all of a sudden, I say, ‘My hardware costs 10, my software costs 2, and then my systems integrator costs me 40.’
Like, wait a second. In that equation, the hell, we’re right back with these guys. So the key thing was you can’t have systems integrators. Like if you have systems integrators, the whole price equation is broken, and it doesn’t work, and then they get all the value, and then they also own the customer.
So anyway, so you had to get rid of the systems integrators. And so the answer is, well, in-house systems integration, right?
That’s what the Unix model was. Unix guys aren’t going out there hiring IBM Professional Services, right? They were doing it themselves, right? Because they were programmer admins. So that’s what we’ve got to do. We’ve got to develop our own professional class of administrators, people who can do more than just click next. They become our systems integrators. They’re not going to cost what IBM is going to cost. It’s an upfront thing. It’s a salary thing. And then they can become heroes, and so how do we do that? And the answer is of course, the Unix composition model, right? Have a standard, tool chest of small tools that then these people can put together to solve unique problems, automate it, and then go solve the next one, the next one, the next one.
Adam: This plan is what Jeffrey calls a plausible theory of success. It’s not probable, right? But it’s plausible. There is a path where this could possibly work. And so Jeffrey joins at Microsoft’s Windows Server team to help with this mission.
Jeffrey: Like literally I faxed in my letter, fax, it was a while ago, faxed in my letter of acceptance. And 20 minutes later, I get a call from Dave. And I’m thinking, Oh, wow. I was like looking at the fax machine. That was pretty quick.
And no, it wasn’t. He called me up to say, Hey, Jeffrey, just heads up. There’s been a reorganization and, and, and you’re not going to be reporting to me. You’re going to be reporting to this other person. I was like, what? Like, oh no, I already faxed him my letter of acceptance and hit enter on my email to my boss saying adios.
So I was like, oh, holy schmoly. But the idea was that they were, he owned Windows Server and what they were doing was they were taking the management technology out of Windows Server and putting it, combining it with some management products. So it’s going to be a management division. And so he says, you’re going to be the chief architect over this management division.
I said ‘so is it, is this a good thing?’ And he says, ‘yeah, it’s a really good thing.’ And I think that was the only time Dave ever lied to me.
Cultural Challenges
Adam: This management division, right, they’re all about making Windows easier to manage. So it’s still really what he signed up for. But the problem is the group is made up of these people who’ve been succeeding by seeing the world through this lens of personal computing.
Jeffrey: Now, again, they had. business server, right? you know, an admin in his server, right? Boy and his dog, you know, you got one server, you go there and all that sort of sensibilities, you know, you walk up to it, you have a mouse, you have this little screen, like happy days, click, click, next, next, next.
And they have been very successful with that, right? So again, they were thinking, Hey, that’s the way to do it. This is all good stuff. And I was coming in saying, well, Yeah, but, uh, for the enterprise, you know, we’re going to have data centers, which are going to be large
Adam: You can’t be expecting people to walk up to each server and log into it in a data center.
Jeffrey: And they said, no, no, no, we got that problem solved. I said, okay, I didn’t know that. How do you solve that problem? Remote desktop. wait, but it’s still, you connect to one of those machines, right?
So yeah, you connect it. It’s like, that’s not going to work.
this is a server. Another example was when somebody was producing a, uh, service and I said, okay, well, uh, you tell me about the service that you have, the server and they’re like, blah, blah.
I said, okay, but how do you start it up? And they said, well, uh, you know, you start it up from the, you enter a command line to start up. I said, okay, but. How’s the user going to do that? He said, well, that’s how they’ll do it. I said, wait, so you’re saying every time they start the system, someone has to log in, enter a command line to start your process.
They said, yeah. So, okay, that’s not going to work. So, you know, just the whole sensibilities were off,
Scenarios for Enterprise
Adam: The Microsoft team, for their part, considered Jeffrey old fashioned, right? He was discussing how Unix and mainframe vendors were tackling issues. But the modern Microsoft way was a user interface for each problem. So why not just do that?
Jeffrey: And the answer is that’s not the model, right? It’s a toolkit approach. It’s the small tool approach, right? You have a tool chest of small tools. And then, as unique problems come up, an admin will compose these tools in different arrangements to solve that novel problem, and then it goes away, and then you iterate, iterate, iterate.
And so their mindset was, well, no, that’s a bad idea, because it requires, you know, a smart admin to do something. Instead, tell us what the We’ll put it into our queue. We’ll spend a couple of years developing it. Then we’ll ship it as a new product. Then if and when they adopt the new product, problem solved.
It’s like, but immediately after you sell that product, the scenario changes. So we’ll then tell us again and the next version will include it. It’s like, you’re just not getting it.
Windows isn’t Unix
Adam: initial solution for managing Windows was to use Windows services for Unix. This was something you could download, add to Windows, and get a Unix shell. And you’d have awk whatever else. And his theory was a skilled administrator could learn these tools and then use them to manage servers.
Jeffrey: Then turns out didn’t solve any of the problems. And here’s where I began to understand the true, where you are, uh, versus where you want it to be. There’s a core architectural difference between Unix and Windows. Unix, everything is, it’s a file oriented operating system, okay? So everything, if you can manipulate a file and restart processes, you can manage everything, right?
That’s why AWK/GREP/SED those are management tools, because they manipulate files. Okay. So then when I said, okay, great. Take AWK/GREP/SED and a shell and all that and put it on Windows. Happy days, right? No, because Windows is not a file oriented operating system. There are very few files, uh, in, in, in Windows.
Instead, everything’s behind an API. Right. So awk didn’t work against the registry, sed didn’t work against Active Directory, grep didn’t work against WMI, Windows was all APIs that call, and then you get structured data back. So none of that stuff worked.
Adam: The only thing close to a solution was WMI, Windows Management Instrumentation, and it allowed for some management tasks, but it was largely underused. Jeffrey’s group saw this as an opportunity. They could develop command line tools to manage everything. User setups, network configuration, application installation, all using this WMI.
Jeffrey: I remember the meeting when I was there and I finally convinced my exec that yes, this is what we had to do. And then she said, okay, get it. I totally get it. Which 10? I said, which 10 what? She said, well, which 10 command lines should we do?
I said, well, we need to do thousands. So yeah, yeah, yeah. But we’re probably going to get to 10 this release. So which 10 should we do? I was like, Oh my God, we’re doomed.
Adam: Is there really thousands of APIs sitting in Windows? Was this? Windows 2000. Windows xp. What was,
Jeffrey: Yeah. So it was XP and beyond.
Yeah.
I don’t know. Pulled that number out of my ear, but yeah, probably, you know, depends on what you call a command.
yeah, you need like lots of commands focused in on the objects in WMI. Okay. That’s why you probably need thousands.
Okay. So, so then the next step was, okay, I need more than 10. I was able to secure funding to get contract engineers to do this. , the existing engineers are like, well, the command lines, they’re beneath me. You know, you’re off talking about this crap, but nobody cares about it.
You’re wrong. And, uh, I’m not going to waste my career, you know, spending time on this shit.
Coding Window
Adam: But Jeffrey still had this vision clear in his mind, right? It was the reason he came to Microsoft, enable a system administrator to script and manage hundreds of headless Windows servers in a data center and Windows 2000 had just shipped. So there was an opportunity to integrate some of these commands into Windows XP.
Jeffrey: And they’re like, yeah, but the coding window is, uh, 10 weeks. 10 weeks? F me. What the hell are you talking about? 10 weeks? Wait, what can you do in 10 weeks? Oh, we can do a lot in 10 weeks. I don’t think so. Obviously, I misunderstood something. Um, I’ll have to wait till my next window of opportunity to figure things out and do something.
Well, it turned out when they said, “Our coding window is 10 weeks,” what they were doing was they’re just writing code for 10 weeks. And some of that code would be like a function. Uh, comment, fill in this function, check it in. And then, so you have 10 weeks worth of that. Then nothing works after those 10 weeks, nothing. And then they spend like a couple of years making it work. It’s like what the hell is that?
Adam: Yeah, why did they just check in like a stub of a function?
Jeffrey: Oh, because, because, uh, after 10 weeks, you couldn’t do anything new, but you could fix the bug. The bug is doesn’t work. I mean, how crazy is that?
Adam: So Jeffrey’s group pays contractors to write these command line interfaces that use WMI, they managed to implement 70 actions. But really that was barely making a dent in the needs of administering Windows servers. It was a start though, right? So they decided to seek more funding and continue this work.
Jeffrey: So now let’s bring in another weird aspect of, of Microsoft. Nothing goes out the door unless the test organization signs off on it. Okay, great. And I will tell you that there’s an incredible, There are mountains, mountains of awesome features. a dustbin Microsoft dustbins, because, uh, they developed this stuff and didn’t have the, the test organization’s bandwidth, to sign off on it.
And then I had some issues with them signing off on the 70 commands that I had done, but I got that right. Cause we spent a lot of money on this. It’s like you, you were going to find a way to fricking sign off on these things. I was like, okay, but if I do this again, this is gonna be a problem. So what I said was think about HTML, right? they test the browser, but then you don’t have the test organization test every HTML page.
Adam: This is clever. Instead of writing code for each command, you know, you have a format for setting up a command. Describe it in XML or whatever config format, how each command works. And then you make this generic command builder and the organizational trick is saying that any specific commands config is just metadata. No need to test them. Just test the tool. And bam, the, the testing. Organization’s bottleneck in the process has been sidestepped.
Jeffrey: People fought me against that. Like, that’s a stupid idea.
You know, it’s going to take forever. Nobody, they just didn’t see it. It’s like, okay, and here’s one of the life lessons. every now and again, you just need to know when to be a butthead. So this was one of those moments, right? If you’re always a butthead, , you’re never going to get very far, but if you’re never a butthead, it probably means that you’re not taking enough courageous changes and driving a reticent organization where it needs to be.
So this was definitely one of those times. They did not want to go down this path, but I was just adamant. It’s like, no, we must do this.
Adam: Jeffrey won the battle for his metadata driven solution, and over his Christmas vacation, he diligently produced metadata, and he got done 72 commands.
Jeffrey: And I was like, Oh my God, you know, I just spent basically 4 million at a long time to get these 70 commands I spent. I forget what it was. It was like 60, 000 to get this engine. And then I spent my Christmas vacation and I got more commands than them.
It’s like, wow, this is, this is gold. This is good stuff. And then I said, Hey, you know what? That engine is missing a number of features. Can I get some more money to improve it? And they said, Yeah, yeah, we can. And so we did that and we added a bunch of features, right? Filtering, formatting, et cetera. And here’s what the magic happened. We added it to the engine and all 72 of those commands got better.
It’s like, Oh, this is gold. This is gold. And what I had discovered was, with my architecture, I have a base investment. And then the incremental cost to write these things is really, really small, right? It’s really flat. Incremental cost to add new function. And I can invest in the engine and the value goes up. The functionality goes up for everything.
Adam: Now these commands aren’t PowerShell, right? This is WMIC, an earlier solution. But if you’re familiar with PowerShell, this might seem familiar, right? It’s generic, it’s data driven, things are coming together. But still, the problem of getting coverage for everything in Windows is an issue. And at this point, Bill Gates and NET enter the picture.
Jeffrey: Bill’s a, you know, an interesting guy, um, more of a business guy than a technology guy, in my opinion, as technology, he’s got opinions, etc. But, it comes to business, guy’s a freaking genius.
And so here was the challenge. we had shipped XP. XP was an excellent operating system. Excellent operating system, right? We got rid of the crap old, you know, sort of handcrafted kernel, brought in Dave Cutler, god of operating systems. You have the NT kernel now available for everybody. So we had this awesome operating system and we couldn’t get anybody to use it.
Jeffrey: They were all stuck on Windows 98. Windows 98 was demonstrably unfit for purpose and we couldn’t get people off of it onto XP.
Adam: What Bill had figured out is that if people won’t get off Windows 98 for something good like XP, Then once they get on XP, how will we ever get them to upgrade?
Jeffrey: And so he realized that he had a severe business problem. And so he said, Here’s the way we’re going to solve this business problem. We’re going to draw a line in the sand. And we’re going to engineer a Windows 95 moment.
A Windows 95 moment when the world changed. There was the world before Windows 95, and then there was the world after Windows 95. And this new, moment is going to be called Longhorn. Um, and it was going to have a new way to do develop code. net. It was going to have a new graphics model, WPF. It was going to have a new communications model, WCF, and it’s going to have a new, uh, storage model
Okay. Just a whole new way, you know, just a, a complete generational shift and no one was going to be allowed to invest in anything that wasn’t, uh, this. Okay, so I’m going to engineer this moment and transform the industry. It’s actually brilliant thinking. Didn’t work, but brilliant thinking.
Adam: Now Bill is a business genius, yes. But by this time Jeffrey also knew Bill was a force of nature.
Bill was famous for his Bill G reviews, where executives would pile into a boardroom and give him status updates on various projects.
Jeffrey: So I remember one it was so bad. It was so bad. It was so bad. So we’d been beaten up, beaten up, beaten up. And then. This one time, we’re going to have three topics.
Jeffrey: The first guy, , he’s going to come in and do exactly what Bill told him not to do in the last meeting. The second guy is coming in, has made no progress since our last Bill meeting. It’s like, oh my god, we’re going to get killed. And then the third guy has a new topic.
Talk about diagnostics and it’s like, okay, well, that’s actually some pretty good stuff. So we’re, we’re in good shape here. Anyway, so our exec was out that day. She was sick or something. And so we go in and it’s like, well, it’s just us. It’s like, this is going to be bad. first guy talks, what guy who did exactly what Bill told him not to do.
And Bill’s all smiles, like the hell. That’s weird. Second guy is okay. Here’s where it’s going to put, brace for impact. This guy’s going to get it. goes through it, builds smiles. Oh, this is great. Fantastic. Great progress. All this. I’m thinking, Oh, you know what? He hates our exec. That’s what it is. It turns out that’s not what it was.
What happened was, was that that morning, just before that meeting, he, as part of his, you know, other work, he had acquired a bunch of patents for malarial treatments. And so he’s riding high on that. Anyway, we didn’t know that. So we’re just like, what the hell?
Adam: Then Jeffrey’s up. He’s covering changes to diagnostics, for a person that’s out. It’s not his area, but the changes that are being covered make a lot of sense to him.
Jeffrey: I start to present, like the first slide, Bill’s face changes, like a bad Mexican meal just hit his intestines. And it’s just a matter of, like, minutes before he’s losing it, right?
And he’s screaming at me, ‘YOU FUCKED HIM! YOU FUCKED HIM! YOU FUCKED HIM!’ Spit. I’m not making this up. Spit is coming across the conference room table and landing on my glasses as he’s like, ‘YOU FUCKED HIM!’ I have no idea what the hell he’s talking about, but he’s so, and it’s like, wow, this other execs on the other side of the table is like, Bill, Bill, Bill, Jeffrey didn’t fuck those people.
I fucked those people. I’m like, what the hell is going on? And so, I just keep trying to go on. And, uh, what had happened was he got very excited about. Uh, diagnostics in the past and somebody had convinced him, oh, the key to diagnostics are these Bayesian networks, blah, blah, blah. And so, they had these things, these Bayesian networks troubleshooters and they shipped them in windows.
Bill just loved it, right? AI. He loved these things.
Adam: The problem is, these things didn’t work. And even getting them translated into all the various languages that Windows supports was going to cost a lot of money.
And so the decision was made to just stop shipping them.
Jeffrey: Bill thinks, ‘Oh, you guys fucked up. My favorite Bayesian network, uh, diagnostics people.’ So that’s why he was screaming at me and it was like, okay, well, can we get beyond that and just talk about this? The answer was no, but he didn’t say no.
and so I felt it was my responsibility. Like, hey, I got to speak truth to power here.
So let me just break it down. Bunk, bunk, bunk, bunk, bunk. beat the crap out of me. People were like, ‘Oh my God, you were like a fricking Weeble. He’d knock you down. You just get back up.’ He says, we were looking for a towel to throw. It was so bad. And it really was. It really was. So at the end of that meeting, right?
Concludes, didn’t conclude well. I ran out of that conference room and ran to the men’s room because I thought I was going to puke. I just hovered over the toilet for like, It was bad. It was really bad. I mean, no joke.
Adam: So Jeffrey made it through that tough review. But then he realized from these other Bill G meetings that Bill was really pushing everyone towards .NET, aiming to shape that specific outcome.
The .Net Wedge
Jeffrey: Right, because that was when he was beating us up, .NET, .NET, .NET. I was like, man, that’s just not going to help me. but okay, well, let’s find out what he’s talking about.
So, that’s where I began to say, ah, this can help me. Bill’s going to go get everybody to give me coverage. Cause I can’t get anybody to give me this Windows Management Instrumentation (WMI) coverage. Cause I, I did not generate that. loop where everybody said, oh, I can’t wait to write a WMI provider.
I get so much value from it. So it wasn’t getting coverage there, but he was going to get me coverage for .NET. And so, all I had to do was to write the utilities on top of it. So that’s, that’s, that’s how I got down that path.
Adam: .NET was you using Bill’s anger to get, to get you leverage interally?
Jeffrey: Exactly. The Bill Bully puppet.
Adam: This .NET insight gave Jeffrey an even stronger plausible theory of success. But of course, things wouldn’t stand still while he tried to execute on that. Because, at the same time, his org’s leadership was struggling to figure out priorities.
The ReOrg
Jeffrey: Windows Server kept saying, hey, I, I need you to do X, Y, and Z. You know, I’m willing to work with you, I understand what you’re doing, but I need the following things. And they weren’t getting what they needed from the organization.
You know, the execs like, nope, I’m going to do this. I’m not going to play ball. Uh, and then one day she woke up and half her organization was gone and it was moved to Windows Server. And so then it was remained, went into crisis, dysfunction, et cetera. And so because of that, because of my role, right, I’ve been in this role of like, okay, figuring out the products, et cetera.
And that’s effectively stopped, right? Stopped having staff meetings. There’s no all hands, there was no message. I mean, he just went into complete dysfunction. It’s like, oh fuck, what do I do? What do I do? And it’s like, okay, well this isn’t gonna last for long, right? Something’s got to change here, but I got nothing to do.
Adam: There are many ways to handle a reorg. One option might be to reconnect with the Windows Server Group and try to focus on this broader goal of helping them understand the data center and the enterprise. But despite his organization virtually disappearing, Jeffrey stuck to his plan. It was still plausible and he was excited about its potential.
But how could he keep it moving?
Shell Team
Jeffrey: Turned out there was another group in some other organization that were producing a shell.
And so, I said, ‘Hey, you guys, let me tell you how the right way to do a shell. Right. Here’s how you should do it.’ And they’re like, ‘I don’t know what you’re talking about. We’re just going to take K-shell and ported it.’ I said, yeah, but we should do better. Like, cause you’re, you’re going to run into the problem.
I ran into when I did the first thing, you’re going to have a shell and then nothing’s going to talk to it. So you’re not going to move the ball forward. And so I tried to explain it, tried to explain it, they didn’t get it. And so as much as I tried to explain the ideas, they didn’t get it. I said, you know, just go away. And I’m going to lock myself in a room and then produce a prototype of what became PowerShell.
And so I produced basically a 10, 000 line demo prototype of this that had all of the core architectural principles of PowerShell. And I invited them back. I said, ‘Let me show you something.’ And I was able to show them and they said, ‘Well, what about this?’
And I showed them. And they said, ‘What about that?’ And I showed them. And I said, ‘What about this?’ And I showed them. Their eyes just got big and they’re like, ‘This, this, this.’ And so I said, ‘Okay, great. Go do this.’ And then, eventually, they were going to go do that and I helped them get funding. And then I realized like, man, this is probably the best idea I’ve ever had in my life.
I should go work on this.
Demotion
Adam: Getting himself on that project though, that wasn’t easy.
Jeffrey: Like, ‘Hey Jeffrey, wait a second.’ We hired you in as this industry expert. You’re the chief architect for our products and our services, varied from like 700, uh, You know, 1, 200 people, and now you’re going to want to go work on something that’s got a team of like a couple people.
I said, yeah, but we need to fund it tomorrow. They said, okay, but it’s a couple people and it doesn’t matter. It’s a command line interface. It’s like, but this is important. Like it’s a command line interface. And I said, well, I know I want to work at this. And basically the answer was, ‘Well, okay, we’ll let you work on it, but you know, you can’t keep that job.’ We’re going to take that away from you.
And so, yeah, so I got demoted.
It was really painful actually. So I didn’t, you know, I didn’t tell any of my friends. I didn’t tell any of my family. It just was so painful. I’ve obviously I had to tell my wife because it had a big economic effect on us.
And because my focus was on having an impact on the world, that was far more valuable than, you know, a title. It’s far more valuable than, you know, money, et cetera. So I said, you know, I think I can change the world with this thing. And okay, I got demoted. That sucks. It’s gonna cost me a lot of money.
That sucks. But if I can pull this off, and get it in Windows, man, I could affect the lives of many millions of people.
The Monad Manifesto
Adam: Jeffrey’s new team named their project Monad and lacking staff. They outsourced some of the work to India. And so to align everyone with the project’s vision and this plausible path to success, Jeffrey created the Monad Manifesto.
And although the collaboration with the India team eventually dissolved, this document became a cornerstone of the project.
Jeffrey: Basically laid out the argument in very clear notion like, hey, what is the problem? What is the traditional approach? What is our new approach? Yeah. Why is this valuable? How does it differ from other things? And then at the end had its concrete, Jeffrey Moore, market relevant statements, which I found to just be incredibly hard to do.
But then once you have it, It provides incredible clarity about what you are, what’s important and what you must achieve and where it’s okay to not achieve things. And it basically takes the format of for user who qualifier offering value proposition unlike alternative offering differentiator and then repeat, repeat, repeat.
For, Admins who need to manage Windows Server. PowerShell provides a comprehensive, coherent, composable management solution, unlike command.exe, PowerShell provides a common parser, PowerShell provides consistent formatting, PowerShell provides a pipeline, blah, blah, blah, more details and all that.
Unlike Unix shells, PowerShell, da, da, da, da, da. And then for users, for providers, for, uh, development teams.
Adam: One thing Jeffrey did was outline all the stakeholders for this project and how it would benefit them. Because this whole thing had a huge bootstrapping problem.
Jeffrey: You know, at Microsoft, everybody’s hair’s on fire and they have 10 jobs and they got to decide which nine they’re going to fail and not, not get fired. Right? And the command line interface is something that, Not doing it would not get you fired
and doing it wasn’t going to get you promoted.
So I was like, okay, but my success depends on all of those teams giving me coverage. Like, how’s that going to work? So I had to articulate that I’m going to knock on their door and ask them for the code that they and only they could write. and only that code. And I’ll provide all the rest of the code. I provide the formatting.
I provide the sorting. I provide the filtering. I provide the parser. I provide the remoting. I provide the, you know, elevated privilege. I provide all of that. You don’t have to do all of that. Any of that. You just need to tell me how to manipulate your objects. Okay. Um, and then you don’t have to test all that.
Like I test all that.
Getting Product Teams on board
Adam: With the Monad Manifesto as his clearly articulated theory of success, Jeffrey now just needed to make it happen.
Convincing the product teams was crucial, and so he met with the Active Directory group, and attempted to win them over
Jeffrey: Knock on people’s door. They’re like, eh, you know, you’re one of the nine that I can fail and not do something.
It’s like, yeah, but come on. And at that point you have a little bit of personal credibility and you get people to try something like, can you just like, Have some people do this because I think you’re going to find it’s easy. you know, get people to spend a couple of weeks on it.
I’m trying to remember who had the weight with who, but we got them to, to invest a couple of weeks of some somebody’s work and we worked with them and they were able to develop a couple of cmdlets and then they had user groups, you know, you continually talking to the users, et cetera, and they showed it to them and they just got such a strong reaction that there’s like, Oh, okay.
And then the customer reaction is like, Oh my God.
My engineers tell me this is good. We should do it. So let’s do it.
And so now a rational product owner looks at this and said, Okay, that’s a good deal. Let’s, let’s finish that work. I liked how little it cost me and I liked the reaction of my customers. So let’s do it. And then they helped get the next one, which helped get the next one.
Adam: At this point, the capabilities of PowerShell were expanding, and although there was a big bump in the road still coming, the Shell team was very busy building things out.
Jeffrey: We would be in somebody’s office at a whiteboard like brainstorming. There’s a group of us and just like, hey, drawing pictures and what about this? And always challenging herself, like go big, go big like that. Don’t go for the easy solution. What’s the bigger answer?
When somebody had done something and everybody rushed to their office to see what they had just done.
Um, yeah. realized at some point, like, oh, yeah, this is, this is familiar. You know, computing used to be fun. And then it sort of wasn’t fun anymore, but this is fun again. And as I thought about that, I realized that, you know, that the mouse is antisocial, The GUI is antisocial, So what’s that mean? you have a problem to solve and you solve it with the GUI. What do you have? A problem solved. But when you solve it with a command line interface in a scripting environment, you have an artifact. And all of a sudden that artifact can be shared with someone. By the way, the way you did it can show cleverness.
I’ve never seen anybody use a GUI in a clever way. Ever. There’s no cleverness to it. No, like, Oh my God, you should see the way Adam clicked that mouse. Oh my God. Guys, guys, guys, guys, come on, check it out. Adam’s going to click the button. Oh my God. That’s amazing. It just doesn’t happen.
Scripting, you’re using a language, right? You’re communicating.
It’s like, Oh my God, did you see what, Proust did, that’s phenomenal. This guy’s a freaking genius. And then, Hey, give that to me. I’m going to steal that technique and apply it to my code. Or then I have this artifact and I publish it and people are using it. There’s a debt of gratitude. Like they owe me a beer.
Right? Or they’ll come back and they say, Hey, Jeffrey, uh, why’d you do it this way? Why didn’t you do it that way? And it’s like, Oh my God, that’s a better way to do it. Thank you. And so it’s a social environment. Anyway. It really was this moment, like, man, we are having a blast.
Like, and it hasn’t been that way in years.
Having Impact
Adam: For Jeffrey, this made the demotion feel worthwhile, right? It gave him a sense that there was potential impact here. For many at Microsoft, a Porsche or a promotion or the total compensation package were the important rewards. But Jeffrey was measuring things differently.
Jeffrey: It’s that, uh, effectively I’m an existentialist. Okay, so what’s that mean? That means I don’t believe in a life after this. I don’t believe in any grand narrative. I don’t believe any of that. I believe life is what you decide to make of it.
And that caused a lot of like when I went through that kind of discarding of, of, uh, comfortable beliefs and embracing of this. That was a very difficult period. But then after that I said, okay, well then, what do I want to make of this? And it really came down to, uh, I’ve got a certain number of amount of time and I want the world to be different and hopefully better, uh, because I existed.
Jeffrey: It’s that simple. So it’s really about having impact and, and, and mattering that made me say, yeah, I want to do this.
.Net Execution
Adam: With that conviction, PowerShell looked like it just might happen. After all, it’s attached to this great new .NET operating system initiative, Longhorn.
Jeffrey: So then began the boufforama, right? A bunch of engineers making terrible decisions about how and where to adopt .NET. And the reality was, many of these technologies like, Hey, if you had done it over the course of, you know, five to seven years, that brilliant idea. They were like, no, no, we just know, we just know.
You set a deadline and our engineers will rise to the moment. They’ll sleep here nights, weekends, they’ll get divorced, but the hell with that, they get the, they’ll be making enough money. They can afford a new family. I’m not making this up, but you know, people, people will rise to the occasion and when they tell us it’s not working, we’ll say, make it work and they’ll do it because they have, they’re a really amazing group of people.
Until they weren’t and it didn’t work this time and they couldn’t force people and it couldn’t, you couldn’t force reality. They couldn’t make it work.
Adam: Take Notepad, for example. It was native code, and it used around 15 kilobytes of memory.
Jeffrey: Well, somebody got the idea that said, Hey, you know what? We’re one way to make sure everybody’s using .NET is, uh, I’ll rewrite the common control dialog boxes using it, you know, use WCF. So now all of a sudden I got this native code notepad. I bring it up, starts instantaneously.
Everything’s fast, fast, fast. And then I say save as, and the dialog box comes up a minute and a half later. And I go from a 15 kilobyte working set to a 15 megabyte working set.
And then I can save my file. Terrible idea. Terrible idea.
Other people were looking to figure out, hey, how can I get .NET in the kernel?
I think it got to some point. I don’t remember the details, but it had been like maybe seven months that the nightly builds didn’t work.
And so there had been this, okay, no new code, only code to fix the builds and, uh, make things work. And then that wasn’t working. So they basically had to do a reset.
Getting Back into Windows
Adam: So the Windows people decided that .NET was the problem, when perhaps they had just bitten off more than they could chew.
But PowerShell got pulled out of Windows, like all the .NET code during this reset, and now Jeffrey has to find a way back in.
Jeffrey: So, that, that began like a multi-year process. Just hell as they, they began to really come after command line interfaces, anything that do with .NET.
So they just like on a regular basis would come at us and try and get us canceled.
First, Bill Gates always got it. Bill Gates got it. That helped me exactly zero. Helped me not one bit, right? Then I had the head of Windows Server. He got it.
He helped at like crucial moments, right? There were crucial moments. But like the day to day stuff, uh, you know, it just felt, it was a very hostile environment, uh, very unwelcoming environment. Lots of people throwing rocks at us and, and, uh, and we were just trying to, you know, You know, go underneath the covers and get our job done.
Adam: Doing the actual building was actually going really well.
Jeffrey: I mean, it was just so delightful. I mean, we would run into each other’s office. Like, Hey, have you seen what, what Bruce just did? And we all run into Bruce’s office and we’d see something did like our heads would explode.
Like, Oh my God, that’s fantastic. And, Oh, Hey, I heard Jim’s doing something. We’d all run into Jim’s office and he’d do something. And, uh, you know, you just see it coming together. And there was a reality to it that made you say, Hey, all these people are throwing rocks at me. They don’t know what the fuck they’re talking about.
They’re wrong. This is awesome. And we’re gonna, we’re gonna ship this thing.
I had partnered with exchange. , and they were a deep customer. So when they’d like have this meeting, like, okay, let’s kill them this time. And I just bring my exchange guys there and they just sit in the corner, they had a meeting.
Okay. So we’ve agreed, uh, we need to kill this Monad thing, the exchange guys would then perk up and said, no. I have a multi billion dollar business. I’m betting on this. I need this. You can’t kill it.
Adam: Even though the project survived, it had funding, it was still not part of Windows. It needed a way to get back in.
Jeffrey: WinArch, WinArch was the windows architects. They got together and said, okay, here are the seven things you have to do to get .NET into windows. And these were like incredible draconian set of things that, you know, some of them were based in good things, but they were just phrased in a way.
It was very clear, like, No, yeah, we’re going to keep .NET out of this. So again, everybody looked at that thing and everyone bailed except for two people, Scott Guthrie and myself. he, he cheated. He, he, he shipped outside of Windows.
I tried to figure out how to get back in.
Adam: So what’s the process to get into windows?
Like, is it,
Jeffrey: Yeah, well, you have to have a formal, uh, design change process, right?
What I said was, they’re going to go out of their way to find any, uh, you know, weakness in our argument.
So we can’t be green. We got to be greener than green. And so we did that for everything, everything, everything, finally got our story together. And we said, okay, we’ve sent the request. Like 20 minutes later, the executive sends back mail saying, withdraw this request. Like the head of windows, withdraw this request.
And so my program manager, Jeffrey, come here. Yeah. Look at this mail. I just got from Brian. Like, oh my God, what should I do? I said, I, I don’t know. He said, I know what I’ll do. His response, he says, no.
He says, you turn it down.
And so he says, okay, fine. I, I turned it down.
Adam: Turning things down formally, it triggered a review process. This process involved the Windows Server Organization, the group Jeffrey was originally supposed to work for.
Jeffrey: And the head of windows server had dialed in that this was critical to his business. And so he was a supporter and he basically said, Hey, this is critical to my business. I own this decision. If it meets the requirements, it’s going in and it did meet all the requirements.
And so, and the, and so then there’s this big battle between the executives. It was actually messy, but ultimately, uh, the Windows server executive prevailed and we were able to get back in.
Keep Shipping
Adam: So they shipped it. PowerShell 1 was released as part of Windows Vista. But for Jeffrey and his team, the work wasn’t done.
Jeffrey: Even before it shipped version one, people like, my manager’s like, okay, what’s the next thing you’re going to do? It’s like, I’m going to ship this. I say, yeah, yeah. But after you ship it, what are you going to do after that? And it’s, well, it’s not done.
I got to do the next version. Like, no, no, you’re a senior guy. You need to do something else. Otherwise, you’re going to get pigeonholed. And it’s like, no, I’m going to do this. And it’s like, well, it’s going to hurt your career. I say. Whatever. And so then that happened with version two, and then it happened with version three, and then it happened with version four,
Adam: Each version, each release, was executing towards the end goal of successful server administration that Jeffrey had laid out in his manifesto.
Jeffrey: Version one, we got this far. Version two, we filled out these things. Version three, these things, version four, we pretty much came to completion of the vision. So, you know, it just took that long to do. So that was me personally. And again, being warned all the time, “Hey, this is going to affect you financially. It’s going to affect your career, et cetera.” I was like, yeah, I don’t care. I’m just going to do this, man.
It’s the right thing to do. And eventually it worked out okay for me.
Adam: That sustained focus paid off. For those who invested in PowerShell, their skills grew more valuable over time. Instead of frequent disruptive changes, the core ideas remained consistent, and each release made their skills more valuable.
In big picture, what’s truly fascinating is how PowerShell bridged this gap between the administrator and the programming world. Windows administrators who never saw themselves as coders were suddenly writing scripts and automating complex tasks and PowerShell created this whole community of people helping each other out.
Server admins forming PowerShell user groups. People were sharing scripts back and forth. They were answering questions online. Conferences were coming together.
Jeffrey: Yeah, they become a hero. They’re asked to talk at conferences. A number of people have become, you know, professional speakers because of this. It’s amazing.
One woman who was so nervous about presenting at a conference, she’s in the speaker room, you know, doing shots.
Like, what are you doing? I can’t cope. She goes up there and she just nailed it. She was fantastic. And then she’s at all the circuits, just giving talks all the time now. So she’s a pro. Yeah. Transformed her career.
Azure
Adam: For Jeffrey, the success of PowerShell diminished the sting of his demotion.
But really, the true impact of this work, of this effort, is that Windows Server Administration changed. And that change to how Windows is administered actually enabled a lot.
Jeffrey: So it took me five years, but I finally got my stripes back became Distinguished Engineer and then eventually Technical Fellow.
At some point I went to work in office and the head of the office, we were having lunch, and he said, “You know, you realize that PowerShell is the reason why Microsoft’s in the cloud.”
What? How’s that? He says, “Well, Office led the way to the cloud. Like we were, you know, many years ahead of Azure.”
And he said, we never would have been able to get to the cloud without PowerShell. He said, before that when I went to provision a server, it’s click, click, click, click, click.
And you couldn’t do that multiple times. And then you, when you got it wrong, you couldn’t fix it. So having scripts allowed us to scale up when something went wrong, we could change the script. And, uh, he says that enabled us to get to the cloud and, office going to the cloud enabled Azure to go to the cloud.
Outro
Adam: That was the show!
Jeffrey. thank you so much for candidly sharing your story. I hope I did a good job of presenting it because it’s an amazing one.
I learned so much from you about the importance of a clear strategy for success. Thanks. And in the need to stay focused on these long term goals, despite the daily challenges thrown your way, you know, sometimes you have to be willing to, to take the demotion or face the criticism to keep your eye on that long term prize.
Adam: And listeners, Jeffrey is very active on Twitter at @jsnover, and he’s now working in the SRE group at Google. So give him a follow. If you’ve got a PowerShell story of your own or more relevant, if you’ve faced similar organizational challenges, driving change, uh, if you have a story to tell, I’d love to hear about it or let me know what you think of this episode.
Adam:
You can reach out to me on Twitter @AdamGordonBell or email me adam@corecursive.com. Or join the slack and tell me about it there or whatever. You can find me lots of places
And until next time, thank you so much for listening.
Sticher
Overcast
Nov 04, 2024
52
min