Welcome to Part 8 of our “History of the Internet” series. In Part 7, we looked at some of the early search engines, including Lycos, AltaVista, and Yahoo!. By 1996, AltaVista and Yahoo! were by far the two most popular options for Internet users looking to discover new websites to visit.
Google was co-founded by Sergey Brin and Larry Page. Both of these inventors had highly intelligent parents and grew up with access to computers. Page’s parents bought a home computer in 1978, and Page soon amazed his teachers by handing in his homework as printed Word documents. He loved to experiment, and while in high school, he built a working inkjet printer out of LEGO bricks.
Brin is the son of a National Aeronautics and Space Administration (NASA) scientist and a math teacher. He loved computers beginning with the Commodore 64, and also shared his father’s interest and talent in math. He received an undergraduate degree in math and computer science from the University of Maryland at just 19 years old.
Page met Brin in 1995, while both were studying for a Ph.D. in computer science.
By 1996, they were hanging out together so much, they became known as the single word “LarryandSergey.” Later on, they would be known as the Google Guys.
Brin looked at some of the existing search engines with his professor Rajeev Motwani, but they found all of them lacking. Typically, a search returned hundreds of results in no discernible order, and it took much too much time to sift through them and find the valuable ones.
Page liked using AltaVista, but found it odd that they didn’t appear to be doing anything with the links beyond reporting on them. He thought that conducting a sophisticated analysis of these hyperlinks could lead to better searches. But first, he would need a really big database.
BackRub and PageRank
Page had the wild idea of downloading the entire Internet onto his computer, and stated that this could be done fairly easily. He wasn’t dissuaded by anybody’s skeptical responses to this, and got started right away in 1996.
Brin was drawn to the project, both to work with Page and also because of his interest in extracting useful information from large amounts of data. Brin was ideally suited for this, as he had recently started a research group called Mining Data at Stanford, or MIDAS.
Together, they wrote “BackRub” in Java and Python. They ran the program on Sun Ultras and Intel Pentium machines, including one made out of LEGO bricks. These computers were funded mainly by the National Science Foundation, the Defense Advanced Research Projects Agency, and NASA as part of the Digital Library project at Stanford.
The idea to name it BackRub came from the search engine’s focus on analyzing backlinks (websites linking to your website).
Page reasoned that the more websites are linking to a particular website, the more likely that website is the one that people find to be most valuable. He thought of web links in much the same way as citations in academic papers. “It turns out, people who win the Nobel Prize have citations from 10,000 different papers,” he said.
Not only are the most frequently cited papers held in the highest regard, the act of citing these papers is an important part of creating a strong argument in newer academic papers.
This is the basis for the technology filed for patent at the beginning of 1998, “PageRank,” named after its inventor Page.
They abandoned the BackRub name and began calling their technology “Google,” a common spelling of googol (10 to the power of 100), because they felt it fit well with their goal of building very large-scale technology.
The search engine was first made available internally at google.stanford.edu and rapidly became popular on the campus. Its simple and clean interface stood out in sharp contrast to Yahoo! and other commercial search engines.
Google Research Paper
Cornell professor Jon Kleinberg was heavily involved with search engine technology and developed the HITS algorithm for ranking web pages. He visited Page in the summer of 1997 so they could share findings with each other. Kleinberg had just completed an early draft of a groundbreaking research paper, “Authoritative Sources in a Hyperlinked Environment,” and encouraged Page to publish his own academic paper.
Page was initially wary of this idea, because it was possible that someone might steal his ideas. However, a few months after the patent was filed, Brin and Page published the paper, “The Anatomy of a Large-Scale Hypertextual Web Search Engine,” in 1998. In this paper, they recognize that the main technical challenge is how to scale up to deal with the hundreds of millions of queries per day that they predicted search engines would soon need to process.
In the research paper, the authors’ intelligence shines. They are acutely aware of the challenges, but also recognize the opportunities coming their way with falling hardware costs likely to be to their advantage.
They conclude by predicting a bright future for search, but are in no respects complacent about the many hurdles that need to be overcome.
To succeed, large problems needed to be solved, including how to crawl the entire web quickly enough to keep up with the rapid content changes, and how to store so much information efficiently.
Business success would not depend on just technical solutions, but on the ability to provide customers with a better experience than AltaVista, Yahoo!, and other competitors. They noted, “The biggest problem facing users of web search engines today is the quality of the results they get back” and gave an example of the query “Bill Clinton” returning a Bill Clinton Joke of the Day from April 14, 1997 as the top result.
Brin and Page’s strong focus on the consumer experience even initially kept them from taking money from advertisers: “We expect that advertising funded search engines will be inherently biased towards the advertisers and away from the needs of the consumers.”
They could not think of an objective way to assess the quality of the results returned, but felt that their engine was already returning significantly better results than those of their competitors. Most of the early feedback for Google came from users at Stanford University, and was mostly positive, despite regularly bringing down Stanford’s Internet connection!
By 1998, technology journalists were also enthused. PC Magazine reviewed their technology, which was labelled as Beta, as part of their guide to the top 100 websites, and concluded, “There’s much more to come at Google!, but even in its prototype form it’s a great search engine.” However, the magazine also declared, “AltaVista is still the best brute force Web search engine.”
Early Investment
Page and Brin first sought investment from the current market leader in search, AltaVista, in March 1998. They were confident that their technology could help improve their search results, and hoped to sell their patent pending technology for $1 million, allowing them to profit and then finish their studies at Stanford.
AltaVista architect Paul Flaherty thought, “they really had something with their link based approach to page ranking, which was AltaVista’s technical weakness at the time.” Flaherty agreed to speak with his colleagues and think it over. However, AltaVista’s parent company, Digital Equipment Corporation (DEC), didn’t like the idea.
DEC had a pending merger with Compaq Computer Corporation and improving their search engine was neither a focus nor a priority for them. A few weeks later, Flaherty passed back the sad news to Page and Brin.
Yahoo! rejected the same investment opportunity, partly because they wanted users to spend more time on their website. Google technology was aimed at letting users quickly find where they wanted to go and then move on.
David Filo, co-founder of Yahoo!, wanted to help them, however. He advised Brin and Page to take a leave of absence from Stanford to set up their own company, and even introduced them to his own investor, Michael Moritz. As the bigger company, Yahoo! had something to gain from Moritz being in touch with the Google Guys. If Google’s technology continued to improve enough to provoke a change of heart, Filo and Jerry Yang could work out a future deal with their investor.
Other search engine companies also turned them down. Many people in the industry felt that search was already good enough and that all that was needed to make more money was to sell more and more advertisements. With more and more people discovering the World Wide Web and using search engines for the first time, this get-rich-quick strategy was currently working for them.
Brin and Page were frustrated by the rejections and unsure of what to do. They decided to keep developing the technology further at Stanford for a few more months and address user feedback.
By late August 1998, they got their break. One of their school professors, David Cheriton, called a Silicon Valley angel and told him Brin and Page had a great idea to show him.
Sun Microsystems founder Andy Bechtolsheim, who enjoyed investing in technologies at an early stage, almost immediately saw enormous potential for the new Google technology.
He stopped off in Palo Alto, California on his way to work to meet Brin and Page at 8 a.m., saw a demo, and said, “This is the single best idea I have heard in years. I want to be part of this.”
He walked out toward his car to get his checkbook. Meanwhile, Brin and Page looked at each other excitedly and asked each other how much they should ask him for.
When Bechtolsheim arrived back at the house, they told him $50,000. He replied, “Oh, I don’t think that’s enough; I think it should be twice that much,” and wrote a $100,000 check out to Google Inc. Brin and Page went out to celebrate with breakfast at Burger King.
In addition to the $100,000, this endorsement gave them something even more valuable: credibility. Such was the regard for Bechtolsheim, Brin and Page now had a much easier sell to future investors. They were soon able to reach about $1 million from three investors: Jeff Bezos, Ron Conway, and Ram Shriram.
As Bechtolsheim drove away, he was confident in his investment, but he had no idea how profitable it would be. Years later, he said, “I didn’t know how big this could be at the time. Nobody knew.”
Stanford University Presentation
In the space of a few months, interest from investors had gone from freezing cold to red hot. The pressure was now on Brin and Page to deliver for them, and they didn’t want to let anyone down. They decided to take a leave of absence from Stanford, but before they left, they gave a presentation to other students in the William Gates Computer Science Building.
Their engine used much more computing power than its rivals in calculating the search results, taking account of many factors such as font size, distance between words and phrases, and capitalization. So, to make the project work, they were going to need powerful hardware.
Brin explained that they were putting their faith in Moore’s law: that hard disks and memory sticks would continue to get faster, cheaper, and higher-capacity, allowing Google to buy enough hardware to deliver fast results all over the world.
Page illustrated the issue of distance between words. Often, users search using more than one word. Other search engines counted the number of instances of those words on the page, but Google ranked sites higher according to how close those words were to each other.
Page spoke about the challenge of beating websites that try to fool search engines into giving them higher rankings, stating that it was a very serious problem, but that he was working on a solution.
Google Inc.
In September 1998, the Google Guys formally incorporated their company, Google, in the garage of their friend and marketing expert Susan Wojcicki in Menlo Park, California. Wojcicki also became their first landlord. They moved all their computers into the garage and several of the rooms in the house. Their first employee was fellow Stanford student Craig Silverstein.
Brin and Page knew that they had a long way to go before they could dream of people calling them the next Yahoo!, but they were at least starting off in the right direction.
Although they were on leave from Stanford, they kept in touch. The majority of Google users were Stanford students, and the growth of usage was due to their positive word of mouth.
After working in a garage for five months, the team moved out into new offices on University Avenue in downtown Palo Alto near San Francisco. To meet the increasing load on their free service, they began ordering new computers in batches of 21 units, and it wasn’t long before their first million had almost been spent.
Google gained Red Hat as their first customer, which ran the service on their internal and external networks. However, this gave Google only a small return and they found themselves in danger of going broke.
Google was lucky to have launched during the dot-com boom. Stock prices in Internet-based companies were mostly rocketing. As an example, Jeff Bezos originally invested $10,000 in Amazon at the price of one tenth of a penny per share. Amazon went public at $18 per share, giving Bezos an enormous 1.8 million percent return on his investment ($184 million). One year later, stocks had risen even higher to $105 per share, valuing the company at $5 billion, despite the company never having made any profit!
With huge success stories like these, investors were far more willing than usual to part with their cash for a slice of a company that could become the next big online name.
In 1999, Google took a second round of investments. Ditching the standard boring PowerPoint presentations in favor of real demonstrations of their cutting-edge technology, the Google Guys quickly impressed and received separate offers from two fiercely competitive investment companies, Sequoia Capital and Kleiner Perkins Caufield & Byers.
With the help of Ron Conway and Ram Shriram, the Google Guys were able to persuade both investment companies to take the deal together, along with several smaller investors, raising $25 million.
In 1999, nobody was sure how the market was going to change, and Yahoo! was now potentially interested in using Google to power their service. This deal gave Michael Moritz (of Sequoia Capital) a seat on the Google board. A major part of Moritz’s financial interest was using the seat to persuade them to do a deal that helped Yahoo!. He said, “When we looked at Google, the idea was that it would power a lot of other sites, most notably Yahoo!.”
This interest came to fruition on June 26, 2000, with Yahoo! signing a deal to use Google’s search engine to power its results. Hailed by Brin as a “significant milestone for Google,” this was easily Google’s most important customer to date.
Another win for the Google Guys was that the $25 million investment deal allowed them to remain in charge with majority control. But the investors insisted that they hire an experienced CEO to turn the company into a profitable business.
This was a much higher priority for the investors than the Google Guys, who were confident that they would find a way to make a profit with or without a new CEO. Part of this confidence was due to their fortunate position as a beneficiary of an extremely turbulent time in our industry.
The Dot-Com Crash
In earlier episodes, we’ve covered some of the aspects of extreme growth and speculative investment in the Internet during the late ‘90s. It was the electronic equivalent of a gold rush: an optimistic mood that investing in Internet companies was a sure-fire way to get rich. This became known as the dot-com bubble, and this metaphorical bubble inevitably burst.
One of the companies that felt the effects of the crash was Yahoo!. In May 2001, Terry Semel left Warner Bros. to join Yahoo! as chairman and CEO, and a few months later he announced a restructuring plan involving 400 job cuts—about 12 percent of their total workforce.
The company had expanded into so many different areas that they lacked focus. Their new business plan focused on just six of their 44 business categories: listings, access, commerce, communications, media and information, and enterprise. Semel called for Yahoo!’s free content to be complemented by fee-driven services, giving them more sources of income.
Google was one of only a few technology companies to benefit from the dot-com crash.
As the value of stocks in many tech companies fell, their employees started looking around for other opportunities, and Google was a clear one. While businesses all around them were laying off staff, Google was on a hiring spree and now the talent pool available to them was greater than ever. Not only were talented laid off staff interested in working for them, many Silicon Valley employees’ remuneration was largely based around stock options.
Another example of very fortunate timing for Google was that the Microsoft antitrust trial was happening at the same time. Microsoft has always recruited a lot of very talented engineers, but while the trial was unfolding, Microsoft suffered a ton of negative press reports. Microsoft’s loss here was Google’s gain.
The abundance of talent arriving at Google helped them to make important technical innovations. For example, in 2001, the engineers invented search spelling error detection and added a “Did you mean ___?” option at the top of the search results. From this point forward, Google’s technology started to feel as if it were smarter than the humans who were using it.
Don’t Be Evil
During the dot-com crash, Google also created their most famous, and controversial slogan: “Don’t Be Evil.” Many people love it for its simplicity and economy. Many others have taken a much harsher and moralistic view on it, arguing that Google cannot be an arbiter of what is considered to be good or evil, or suggesting that it has failed to live up to such a lofty goal.
Google angel investor and Amazon CEO Bezos argued, “You shouldn’t be evil, but then again, you shouldn’t have to brag about it either.”
To make sense of this slogan in the correct context, it is useful to understand how it came into existence.
The term was borne out of exasperation. A meeting led by Human Resources to create a list of Google’s core values produced many bland examples, such as “Be on Time for Meetings” and “Treat Everyone with Respect.”
Paul Buchheit grew increasingly frustrated as the meeting dragged on until he blurted out, “All of these things can be covered by just saying, ‘Don’t Be Evil.’” The other engineers in the room shared his affinity with efficiency, especially Amit Patel, who agreed so strongly he wrote this out on nearly every whiteboard across the whole campus.
The New CEO
Although Brin and Page verbally agreed to bring in a new CEO, they had a number of concerns over the type of person this should and should not be. They reviewed more than 75 candidates between 1999 and 2001, but were unimpressed with all of the interviewees who lacked a technical background.
Their idea of a model Google CEO was Amazon CEO Bezos. Although Bezos wasn’t available for the position, both Page and Brin admired him for his intelligence and his ability to motivate employees. Any serious candidate needed to strongly demonstrate these qualities.
Eric Schmidt was CEO of Novell and previously CTO at Sun Microsystems. His experience satisfied both the founders’ insistence of technical skills and the investors’ demand for leadership skills. However, he was not initially very interested in the role: “I thought search was not that interesting,” he recalled.
Schmidt was eventually talked into interviewing for the role by the highly successful venture capitalist John Doerr, who said, “Go look at Google, this is a little jewel that needs help in scaling it.” Schmidt wanted to stay on good terms with Doerr, so he reluctantly agreed.
The Google Guys weren’t expecting him to be much better than the many previous failed candidates. However, as Doerr was a major investor, the Google Guys knew they had to agree to this meeting.
The founders left a memorable impression on Schmidt, who found them to be arrogant, but smart: “They criticized every single technical point I made, and everything I was doing in my business,” he said.
Schmidt felt invigorated by this lengthy argument, which made him privately reassess some of the actions he was taking. He was later humbled when he checked on the substance of their arguments and found all of their criticisms were valid.
The Google Guys were pleased to see that Schmidt had a Ph.D. in computer science, and noted that he was someone willing to speak his mind. They were also impressed with his experience developing the Java programming language and his battle with Microsoft. They believed that this experience could help Google to avoid making similar tactical errors in the future.
Schmidt joined initially as chairman while finishing up as Novell CEO, and then replaced Page as CEO three months later. On the insistence of the Google Guys, he agreed to invest $1 million in Google stock as part of the deal.
Schmidt had the tough job of turning Google from a startup into a highly profitable business. This involved professionalizing many aspects of the young company’s operations. He was unimpressed to find Google was using Quicken as their financial recordkeeping and payroll system. But the Google Guys rejected his recommendation of replacing it with an enterprise solution from Oracle on the grounds of it being a waste of money. Schmidt described this as “a huge fight.”
Schmidt soon identified that Google was underperforming overseas, with 60 percent of searches but only 5 percent of advertiser revenue coming from outside the United States. He began addressing this by telling Omid Kordestani to get on a plane and set up a European sales operations department.
The Search for Profits
In the period where Google was looking to appoint their new CEO, the company had plenty of capital and was a darling of the technology press for their innovations. However, investors were increasingly concerned that Google was struggling to find a viable business model and was losing money every month. Schmidt played an important role in helping Google become profitable, but the lucrative business model was discovered by looking at one of their competitors.
But before we can understand the value of this business model, we need to understand a problem that was plaguing the early search engines.
The Spam Problem
Early search engines simply indexed the words on each web page, and then matched those words to the entered search terms. While this worked okay for a while, it didn’t take long for people to realize that this system could easily be gamed by adding hidden words onto a web page.
One simple way to do this was to add white text against a white background, but there were numerous other techniques for pushing a web page up the search rankings, even when the web page had nothing to do with the terms that the users were searching for. Users of search engines were increasingly disappointed with the number of junk results that came back.
Google’s PageRank system was more sophisticated than that of its competitors, but no system was perfect and new techniques for beating the system were being found all the time. At its worst, searching for cars on the Lycos engine brought up more porn sites than car sites on its first results page.
The entrepreneur Bill Gross saw this development like an escalating arms race that could never be beaten through technology alone. But most importantly, he was able to see a new business opportunity in this.
Many users were clicking on irrelevant links for a variety of reasons, and spam was one of them. These users had almost no chance of spending their money on any of these sites.
Gross figured that for generating profits, it was less about the number of users coming to visit a website, and more about the types of users and what their intentions were. Websites had good traffic—those that turned into paying customers or loyal users. They also had what Gross called undifferentiated traffic—traffic from users landing on a site that they are uninterested in.
So advertisers should be willing to pay a lot more money for attracting good traffic instead of undifferentiated traffic.
By charging businesses for listing them in the search results, this effectively eliminates spam results. Spammers would be shut out, because if they didn’t pay, they would not be listed.
This was the idea for GoTo.com—a pay-for-placement search service for users who were interested in buying products.
Gross also reasoned that some search terms were much more valuable for businesses than others, and the top rankings for these search terms could be auctioned to the highest bidder.
But, in order to succeed as a business, he needed to attract a large number of both users and advertisers to the GoTo.com site.
The Evolution of Advertising Models
The advent of online advertising gave businesses some hard numbers to work with.
Search engines such as Yahoo! used the cost per thousand model. Advertisers were charged for every 1,000 times their advertisement was displayed. Businesses could see if their reach was good enough, and invest more if needed.
But their tools for assessing the effectiveness of their advertising remained rudimentary. There was no guarantee that anyone would take much notice of any particular advertisements, but even if they were a total flop, the advertisers would still be charged by the search engine provider.
GoTo.com introduced the pay-per-click business model—a business gets charged only when the user reads the advertiser’s description and decides to click through to their website.
This removed a lot of risk for the advertisers, as they would be paying only pennies for high-quality traffic. GoTo.com described these clickthroughs as “paid introductions.”
GoTo.com initially charged only one cent per click, and with the first advertisement given away for free, it wasn’t long before many businesses were hooked.
To attract users, GoTo.com made deals with multiple other websites so that they redirected to their website. Although this cost between five and 10 times as much as Gross was initially charging, it wasn’t long before advertisers were competing with each other to be listed first.
Even though GoTo.com was a commercial success, broader opinion on it was divided because some viewed it as unethical, or at least against the free sharing spirit of the Internet.
Web users were already used to getting impartial results back from search engines, ranked on some quality metric unrelated to advertising dollars. Perhaps the most distasteful aspect of the business was the ambulance-chasing nature of the highest bid prices. The search term for a rare cancer caused by exposure to asbestos, mesothelioma, earned GoTo.com over $100 per click, because this disease often resulted in a highly profitable lawsuit for the winning advertiser.
“When somebody searches for ‘cancer’, should you put up the site that paid you, or the site that has better information?” – Sergey Brin
Defending the business model, Gross described it as the Yellow Pages for the Internet: those who pay for the larger advertisements win themselves the most business. And those who visit the site come with the explicit intention of buying a product or service.
GoTo.com changed its name to Overture in September 2001. Noticing that Google was gaining in market share and becoming a serious competitor, Gross met Brin and Page and suggested a merger. Brin and Page felt that associating with a company that mixed paid search with organic results could tarnish their image, and turned Gross down.
AdWords
Google’s business model was heavily influenced by the business success of GoTo.com. The initial release of Google’s AdWords was in 2000. Advertisers paid for the service monthly, and were charged according to the number of impressions the advertisements received.
In February 2002, it evolved into a pay-per-click business model, and two months later, Overture sued Google for patent infringement.
To Google’s credit, Brin and Page insisted on keeping the advertising results separate from their natural search results, making it clear to consumers which results were ranked on merit, and which were bought.
They also had the idea of ranking advertisements based on relevance as well as the editorial results. Google’s formula ranked according to both price paid by the advertiser and the number of clicks that an advertisement has had. With this system, the most popular and relevant advertisements are usually at the top, which better serves the user’s needs.
America Online and Ask Jeeves Deals
America Online (AOL) used Inktomi to provide its search results and Overture to provide search-related advertisements. However, their co-founder Steve Case found he was using Google more and more, and that Google provided faster results. He recommended that AOL executives try to reach a new deal with Google.
During negotiations, AOL demanded that Google give them a multi-million-dollar financial guarantee. The Google Guys agreed, but Schmidt was afraid that this deal would risk bankruptcy. Schmidt later recalled, “I understood that if you ran out of cash, you were done. They were more willing to take risk than me. They turned out to be right.”
In May 2002, AOL switched to using Google and their COO-elect, Robert Pittman, announced, “Google is the reigning champ of online search.” This was a big loss for Overture, whose stock price fell almost a third on this news. It was also bad news for Microsoft, whose MSN service represented the main rival for AOL.
Another big deal was with one of their competitors, Ask Jeeves (now Ask.com), which already had its own search engine technology. Ask Jeeves’ CEO, Steven Berkowitz, invited Google to complement those results with its search advertisers. Berkowitz believed that in business, you can both compete and cooperate at the same time, and he called this unconventional idea “coopetition.”
Schmidt agreed, saying that in a fast-growing market, it is better to work together to grow the market than to kill competitors and shrink the market. The Google deal helped Ask Jeeves to finally become profitable in the fourth quarter of 2002, and went on to make $107 million in revenue in 2003.
Google first became profitable in 2001, and in 2002 they hit the big league, making $440 million in sales and $100 million in profit. However, they decided to keep this huge success a secret to avoid tipping off Microsoft, Yahoo!, and other competitors as to how huge the search and advertising business really was.
Buyouts
The year 2003 was a busy one for buyouts in the search industry. Overture bought out AlltheWeb and AltaVista.
Although these purchases gave the impression that it was a serious competitor to Google, Overture was actually looking to sell to either Microsoft or Yahoo!.
By October 2003, Overture was acquired by Yahoo! for $1.63 billion. As part of this deal, Yahoo! now had ownership of AltaVista’s search technology.
Google Dominates
Yahoo! remained a major player in the search industry, but it was no longer number one. By 2003, Google had grown into a medium-sized company with over 1,000 employees and began opening up offices all around the world.
From the beginning of the century right up to this day, when people hear the word “search,” they instantly think of “Google.” It has become so successful that its name has become a verb.
By 2003, Google was the most popular search engine. It stood out to many people for its clean, minimalist design, fast load times, and relevant search results.
But it was still vulnerable to competitors because its only mechanism for ensuring customers kept coming back was that its technology was superior. If a better search engine came along, there was nothing to deter customers from rapidly abandoning them.
In later episodes in this series, we will look at their strategy for locking in customers so that they never wanted to leave, including the evolution of email and social media.
In September 2003, Page spoke to hundreds of students at a school in Tel Aviv. After telling the Google story, and stressing the importance of having a healthy disregard for the impossible, he summed it all up:
“Two kind of crazy kids have had a big impact on the world because of the power of the Internet, the power of the distribution, and the power of software and computers. And there are so many things like that out there. There are so many opportunities where you can have a huge impact on the world by using the leverage of science and technology. All of you are uniquely positioned, and you should be excited about that.”