Hit Refresh: A Memoir by Microsoft’s CEO

The aging Rocky looks skyward. “What cloud? What cloud?” Rocky may not know about the cloud, but millions of others rely on it.

Microsoft is at the leading edge of today’s game-changing cloud-based technologies. But just a few years ago, that outcome seemed very doubtful.

By 2008, storm clouds were gathering over Microsoft. PC shipments, the financial lifeblood of Microsoft, had leveled off. Meanwhile sales of Apple and Google smartphones and tablets were on the rise, producing growing revenues from search and online advertising that Microsoft hadn’t matched. Meanwhile, Amazon had quietly launched Amazon Web Services (AWS), establishing itself for years to come as a leader in the lucrative, rapidly growing cloud services business.

The logic behind the advent of the cloud was simple and compelling. The PC Revolution of the 1980s, led by Microsoft, Intel, Apple, and others, had made computing accessible to homes and offices around the world. The 1990s had ushered in the client/server era to meet the needs of millions of users who wanted to share data over networks rather than on floppy disks. But the cost of maintaining servers in an ever-growing sea of data—and the advent of businesses like Amazon, Office 365, Google, and Facebook—simply outpaced the ability for servers to keep up. The emergence of cloud services fundamentally shifted the economics of computing. It standardized and pooled computing resources and automated maintenance tasks once done manually. It allowed for elastic scaling up or down on a self-service, pay-as-you-go basis. Cloud providers invested in enormous data centers around the world and then rented them out at a lower cost per user. This was the Cloud Revolution.

Amazon was one of the first to cash in with AWS. They figured out early on that the same cloud infrastructure they used to sell books, movies, and other retail items could be rented, like a time-share, to other businesses and startups at a much lower price than it would take for each company to build its own cloud. By June 2008, Amazon already had 180,000 developers building applications and services for their cloud platform. Microsoft did not yet have a commercially viable cloud platform.

All of this spelled trouble for Microsoft. Even before the Great Recession of 2008, our stock had begun a downward slide. In a long-planned move, Bill Gates left the company that year to focus on the Bill & Melinda Gates Foundation. But others were leaving, too. Among them, Kevin Johnson, president of the Windows and online services business, announced he would leave to become CEO of Juniper Networks. In their letter to shareholders that year, Bill and Steve Ballmer noted that Ray Ozzie, creator of Lotus Notes, had been named the company’s new Chief Software Architect (Bill’s old title), reflecting the fact that a new generation of leaders was stepping up in areas like online advertising and search.

There was no mention of the cloud in that year’s shareholder letter, but, to his credit, Steve had a game plan and a wider view of the playing field. Always a bold, courageous, and famously enthusiastic leader, Steve called me one day to say he had an idea. He wanted me to become head of engineering for the online search and advertising business that would later be relaunched as Bing, one of Microsoft’s first businesses born in the cloud.

For context, search engines generate revenue through a form of advertising known as an auction. Advertisers bid on search keywords that match their product or service; the winning bid gets an opportunity to display a relevant advertisement on the search results page. Search for a car and a car dealership has likely paid to be displayed prominently on your results page. Delivering that purchase experience both from the consumer and the advertiser perspective is computationally expensive and sophisticated. And while Microsoft was struggling with low market share in search, Steve had invested in it because it would require the company to compete in a sector beyond Windows and Office and build great technology—which he saw as the future of our industry. There was tremendous pressure for Microsoft to answer Amazon’s growing cloud business. This was the business he was inviting me to join.

“You should think about it, though,” Steve added. “This might be your last job at Microsoft, because if you fail there is no parachute. You may just crash with it.” I wondered at the time whether he meant it as a grim bit of humor or as a perfectly straightforward warning. I’m still not quite sure which it was.

Despite the warning, the job sounded intriguing. I was running an emerging new business within Microsoft Dynamics. I had taken over from Doug Burgum who later would become the governor of North Dakota. Doug was an inspirational leader who mentored me to become a more complete leader. He thought about business and work not in isolation but as part of a broader societal fabric and a core part of one’s life. Some of the lessons I learned from Doug are today an important part of who I am as a leader. Leading the Dynamics team was a dream job. For the first time, I was getting the chance to run a business end to end. I had spent nearly five years preparing for this job. I had all the relationships, inside and outside Microsoft, to drive the Dynamics business forward. But Steve’s offer was essentially pushing me out of my comfort zone. I’d never worked in a consumer-facing business and had not really tracked Microsoft’s search engine efforts or our early attempts to build cloud infrastructure. So one night, after a long day at work, I decided to drive over to Building 88, which housed the Internet search engineering team. I wanted to walk the hallways and see who these people were. How else could I empathize with the team I was being asked to lead? It was about 9 p.m., but the parking lot was packed. I’d expected to see a few stragglers finishing up their day but, no, the whole team was there working at their desks and eating take-out food. I didn’t really talk to anyone. But what I observed caused me to wonder: What gets people to work like this? Something important must be happening in Building 88.

Seeing the team that night, their commitment and dedication, clinched it for me. I told Steve, “Okay, I’m in.” What color was my parachute? I didn’t have one.

I was entering a new world, and the move proved to be fortuitous. Little did I know it would be my proving ground for future leadership and the future of the company.

Very quickly I realized we would need four essential skills to build an online, cloud-based business that would be accessed primarily from mobile phones rather than desktop computers.

First, I thought I knew a lot about distributed computing systems, but suddenly I realized I had to completely relearn these systems because of the cloud. A distributed system, simply put, is how software communicates and coordinates across networked computers. Imagine hundreds of thousands of people typing in search queries at the same time. If those queries landed in just one server somewhere in a room on the West Coast, it would break that server. But now imagine those queries being distributed evenly across a network of servers. The vast array of computing power would enable delivery of instant, relevant results to the consumer. And, if there’s more traffic, just add more servers. This elasticity is a core attribute of cloud computing architecture.

Second, we had to become great at consumer product design. We knew we needed great technology, but we also understood we needed a great experience, one you want to engage with time and again. Traditional software design mapped out what developers thought a product should look like in a year’s time, when it would finally go to market. Modern software design involves online products updated through continuous experimentation. Designers offer Web pages in “flights,” so an old version of Bing is delivered to some searchers while an untested new version reaches others. User scorecards determine which is the most effective. Sometimes, seemingly tiny differences can mean a lot. Something as simple as the color or size of a type font may profoundly impact the willingness of consumers to engage, triggering behavioral variations that may be worth tens of millions in revenue. Now Microsoft had to master this new approach to product design.

Third, we had to be great at understanding and building two-sided markets—the economics of a new online business. On one side are the consumers who go online for search results, and on the other side are the advertisers who want their businesses to be found. Both are needed to succeed. This creates the auction effect I was describing earlier. Both sides of the business are equally important, and designing the experience for both sides is crucial. Attracting more and more searchers obviously makes it easier to attract more and more advertisers. And showing the right advertisements is crucial to delivering relevant results. So, “bootstrapping” the online auction and improving search results’ relevance would prove to be a vital challenge.

Finally, we needed to be great at applied machine learning (ML). ML is a very rich form of data analytics that is foundational to artificial intelligence. We needed a sophisticated understanding of how to do two things at once—discern the intent of someone searching the Web and then match that intent with accurate knowledge gained from crawling the Web, ingesting and understanding information.

Ultimately, Bing would prove to be a great training ground for building the hyper-scale, cloud-first services that today permeate Microsoft. We weren’t just building Bing, we were building the foundational technologies that would fuel Microsoft’s future. Building Bing taught us about scale, experimentation-led design, applied ML, and auction-based pricing. These skills are not only mission critical at our company, but highly sought after throughout today’s technology universe.

But we started very much behind in search; we had yet to launch a product that could compete with Google. So I hit the road, meeting with executives from Facebook, Amazon, Yahoo, and Apple to evangelize our emerging search engine. I wanted to make deals, but I also wanted to learn more about how they engineered their products to stay fresh. I found that the key was agility, agility, agility. We needed to develop speed, nimbleness, and athleticism to get the consumer experience right, not just once but daily. We needed to set and repeatedly meet short-term goals, shipping code at a more modern, fast-paced cadence.

To accomplish this, we needed to periodically gather all of the decision makers in a war-room setting. In September 2008 I called together the search engineers for the first of these meetings, which we casually called Search Checkpoint #1. (Perhaps we should have been more creative with the name, because it has stuck and now we’re at a checkpoint in the many hundreds.) We had decided to launch Bing in June 2009—a new search engine and a new brand. I learned a lot about creating urgency and mobilizing leaders with different skills and backgrounds toward one common goal in what was a new area for Microsoft. I realized that in a successful company it is as important to unlearn some old habits as it is to learn new skills.

My learning during this time was greatly accelerated by the hiring of Dr. Qi Lu as head of all online services at Microsoft. Qi had been an executive at Yahoo and was intensely recruited throughout Silicon Valley. Steve, Harry Shum, today our head of AI and research, and I had gone down to the Bay Area to spend an afternoon talking to Qi. On the flight back Steve said to me, “We should get him, but if you don’t want to work for him, that will be a problem.” Having just met with Qi, I knew that he was someone from whom I could learn a lot and Microsoft could benefit. So, I did not hesitate in supporting the hiring of Qi to Microsoft, even though in some sense it was stalling my own promotion. I realized that my own professional growth would come from working for and learning from Qi during my time in our online business. Later Qi would become an important member of my senior leadership team during the first few years I was CEO. Qi eventually left the company, but he continues to be a trusted friend and advisor.

Over time, Yahoo integrated Bing as its search engine, and together we powered a quarter of all U.S. searches. The search engine that many had said should be shuttered in its early days of struggle continued to win an expanding share of the market, and today it is a profitable multi-billion-dollar business for Microsoft. Just as important, though, was how it helped to jump-start our move to the cloud.

As was so often the case at Microsoft, there were other experiments elsewhere in the company aimed at the same problem, leading to internal competition and even fiefdoms. Since 2008, Ray Ozzie had been incubating a highly secretive cloud infrastructure product with the code name Red Dog. A longtime Microsoft reporter, Mary Jo Foley, came across a job advertisement for a Red Dog engineer and wrote a piece speculating that this project must be Microsoft’s answer to Amazon’s AWS.

At some point during my time at Bing, I met with the Red Dog team to explore how we might work together. I quickly realized that Microsoft’s storied server and tools business (STB), where products like Windows Server and SQL Server had been invented and built and where Red Dog was housed, was worlds apart from Bing. STB was Microsoft’s third largest group by revenue after Office and Windows. They were the deep distributed systems experts. But when I contrasted STB with Bing a few things were apparent. They lacked the feedback loop that comes from running an at-scale cloud service. I realized that they were caught up in the local maxima of servicing their existing customer base and were not learning fast enough about the new world of cloud services. And the Red Dog team was a side effort that was ignored by the mainstream of the STB leadership and organization.

In late 2010, Ray Ozzie announced in a long internal memo that he was leaving Microsoft. He wrote in his departure email, “The one irrefutable truth is that in any large organization, any transformation that is to ‘stick’ must come from within.” While Red Dog was still in incubation and had booked little revenue, he was correct that the transformation of Microsoft would come from within. Steve had already proclaimed that the company was all-in on the cloud, having invested $8.7 billion in research and development, much of it focused on cloud technologies. But even though engineers were working on cloud-related technologies, a clear vision of a Microsoft cloud platform had not yet surfaced—to say nothing of a real-world revenue stream.

Right around that time, Steve asked that I lead STB, which today has evolved into Microsoft’s cloud and enterprise business. I was given this news of my new role not even a week before I got the job. Steve had a sense that we needed to move faster to the cloud. He had personally and aggressively driven the transformation of our Office business to the cloud. He wanted us to be equally bold when it came to cloud infrastructure. When I took over our fledgling cloud business in January 2011, analysts estimated that cloud revenues were already multi-billions of dollars with Amazon in the lead and Microsoft nowhere to be seen. Meanwhile, revenues from our cloud services could be counted in the millions, not the billions. Although Amazon did not report its AWS revenues in those days, they were the clear leader, building a huge business without any real challenge from Microsoft. In his annual letter to shareholders in April 2011, just as I was beginning my new role, Amazon CEO Jeff Bezos gleefully offered a short course on the computer science and economics underlying their burgeoning cloud enterprise. He wrote about Bayesian estimators, machine learning, pattern recognition, and probabilistic decision making. “The advances in data management developed by Amazon engineers have been the starting point for the architectures underneath the cloud storage and data management services offered by Amazon Web Services (AWS),” he wrote. Amazon was leading a revolution and we had not even mustered our troops. Years earlier I had left Sun Microsystems to help Microsoft capture the lead in the enterprise market, and here we were once again far behind.

As a company, we’d been very publicly missing the mobile revolution, but we were not about to miss the cloud. I would miss working with colleagues at Bing, but I was excited to lead what I sensed would be the biggest transformation of Microsoft in a generation—our journey to the cloud. I had spent three years, from 2008 to 2011, learning the cloud—pressure-testing its infrastructure, operations, and economics—but as a user, not as a provider of the cloud. That experience would enable me to execute with speed in my new role.

But it wouldn’t be easy. The server and tools business was at the peak of its commercial success and yet it was missing the future. The organization was deeply divided over the importance of the cloud business. There was constant tension between diverging forces. On the one hand, the division’s leaders would say, “Yes, there is this cloud thing,” and “Yes, we should incubate it,” but, on the other hand, they would quickly shift to warning, “Remember, we’ve got to focus on our server business.” The servers that had made STB a force within Microsoft and the industry, namely Windows Server and SQL Server, were now holding them back, discouraging them from innovating and growing with the times.

Shortly after I took over, the company issued this statement: “Nadella and his team are tasked with leading Microsoft’s enterprise transformation into the cloud and providing the technology roadmap and vision for the future of business computing.” Steve had said the transformation would not happen overnight, but we were running out of time.

I had a very good idea about where we needed to go, but I realized that my real task was to motivate the pride and desire in the STB leaders to go there with me. Sure, I had a point of view, but I also recognized this was a team that cared deeply about enterprises, those customers with exacting and sophisticated computing needs. I wanted to build on their institutional knowledge and so I set out first to learn from the team I was to lead, and, hopefully, to earn the team’s respect. Only then could we go boldly together to a new and better place.

Leadership means making choices and then rallying the team around those choices. One thing I had learned from my dad’s experience as a senior Indian government official was that few tasks are more difficult than building a lasting institution. The choice of leading through consensus versus fiat is a false one. Any institution-building comes from having a clear vision and culture that works to motivate progress both top-down and bottom-up.

In business school I had read Young Men and Fire, a book by Norman Maclean (best known for A River Runs Through It). It tells the story of a tragic forest fire that killed thirteen “smokejumpers” (parachuting firefighters) in 1949 and the investigation that followed. What I remembered was the lesson that went unheeded: the urgent need to build shared context, trust, and credibility with your team. The lead firefighter, who ultimately escaped the blaze, knew that he had to build a small fire in order to escape the bigger fire. But no one would follow him. He had the skills to get his men out of harm’s way, but he hadn’t built the shared context needed to make his leadership effective. His team paid the ultimate price.

I was determined not to make the same mistake.

Like that lead firefighter, I had to convince my team to adapt a counterintuitive strategy—to shift focus from the big server and tools business that paid everyone’s salary to the tiny cloud business with almost no revenue. To win their support, I needed to build shared context. I decided not to bring my old team from Bing with me. It was important that the transformation come from within, from the core. It’s the only way to make change sustainable.

The team I inherited was more like a group of individuals. The poet John Donne wrote, “No man is an island,” but he’d feel otherwise had he joined our meetings. Each leader in the group was, in essence, CEO of a self-sustaining business. Each lived and operated in a silo, and most had been doing so for a very long time. My portfolio had no center of gravity, and to make matters worse, many thought they should have gotten my job. Their attitude was one of frustration—they were making all this money and now this little squeaky thing called the cloud came along and they didn’t want to bother with it.

To break out of this impasse, I met with everyone on the STB leadership team individually, taking their pulse, asking questions and listening. Together we had to see that our North Star would be a cloud-first strategy. Our products and technologies would optimize for the cloud, not just for private servers that resided on an organization’s own premises. Though we would be cloud-first, our server strength would enable us to differentiate ourselves as the company that delivered a hybrid solution to customers who wanted both private, on-premise servers and access to the public cloud.

This new framework helped reshape the argument, breaking down the resistance to going all-in on the cloud. I began to notice a new openness to innovation and a search for creative ways to meet the needs of our commercial customers.

Unfortunately, Red Dog, which had become Windows Azure, was still struggling. They were trying to leapfrog with a new approach to cloud computing, but the market was clearly giving them feedback that they first needed to meet their current needs. Mark Russinovich, who was an early member of the Red Dog team and the current CTO of Azure, had a clear road map in mind to evolve Azure. We needed to infuse more resources into the team to execute on that road map.

It was time to move Azure into the mainstream of STB rather than have it be a side project. People, the human element of any enterprise, are ultimately the greatest asset, and so I set about assembling the right team, starting with Scott Guthrie, a very accomplished Microsoft engineer. He had spearheaded a number of successful company technologies focused on developers. I tapped him to lead engineering for Azure on its way to becoming Microsoft’s cloud platform—our answer to Amazon Web Services.

Over time, many others from both inside and outside the company joined our effort. Jason Zander, another key leader who built .Net and Visual Studio, joined to lead the core Azure infrastructure. We recruited the highly regarded Big Data researcher Raghu Ramakrishnan from Yahoo and James Phillips who had cofounded the database company Couchbase. We relied heavily on the expertise of Joy Chik and Brad Anderson to advance our device management solutions for the mobile world. Under their leadership we made our first major steps in providing business customers the technology they need to secure and manage Windows, iOS, and Android devices. Julia Liuson took over our Visual Studio developer tools, evolving it to be the tool of choice for any developer regardless of platform or app.

Complementing these world-class engineers was world-class business planning and modeling. Takeshi Numoto moved from the Office team to join STB. Takeshi had been an important member of the team that had strategized and executed the transformation of Office products to a cloud-based, subscription model. And in his role as business lead for STB, he set about building the new commercial model that was based on creating meters to measure consumption of cloud services and inventing new ways to package our products for customers.

One of the early decisions I made was to differentiate Azure with our data and AI capabilities. Raghu and team designed and built the data platform that could help store and process exabyte-scale data. Microsoft was developing machine learning and AI capability as part of our products such as Bing, Xbox Kinect, and Skype Translator. I wanted us to make this capability available to third-party developers as part of Azure.

A key hire for Azure was Joseph Sirosh, who I recruited from Amazon. Joseph had been passionately working in ML for all his professional career, and he brought that passion to his new role at Microsoft. Now our cloud not only could store and compute massive amounts of data, it could also analyze and learn from the data.

The practical value of ML is immense and incredibly varied. Take a Microsoft customer like ThyssenKrupp, a manufacturer in the elevator and escalator business. Using Azure and Azure ML, they can now predict in advance when an elevator or escalator will need maintenance, virtually eliminating outages and creating new value for its customers. Similarly, an insurer like MetLife can spin up our cloud with ML overnight to run enormous actuarial tables and have answers to its most crucial financial questions in the morning, making it possible for the company to adapt quickly to dramatic shifts in the insurance landscape—an unexpected flu epidemic, a more-violent-than-normal hurricane season.

Whether you are in Ethiopia or Evanston, Ohio, or if you hold a doctorate in data science or not, everyone should have that capability to learn from the data. With Azure, Microsoft would democratize machine learning just as it had done with personal computing back in the 1980s.

To me, meeting with customers and learning from both their articulated and unarticulated needs is key to any product innovation agenda. In my meetings with customers I would usually bring other leaders and engineers along so that we could learn together. On one trip to the Bay Area, we met with several startups. It became clear that we needed to support the Linux operating system, and we had already taken some rudimentary steps toward that with Azure. But as Scott Guthrie and our team walked out of those meetings that day, it was certain that we needed to make first-class support for Linux in Azure. We made that decision by the time we got to the parking lot.

This may sound like a purely technical dilemma, but it also posed a profound cultural challenge. Dogma at Microsoft had long held that the open-source software from Linux was the enemy. We couldn’t afford to cling to that attitude any longer. We had to meet the customers where they were and, more importantly, we needed to ensure that we viewed our opportunity not through a rearview mirror, but with a more future-oriented perspective. We changed the name of the product from Windows Azure to Microsoft Azure to make it clear that our cloud was not just about Windows.