What the Tech?! Search Engines

The internet was a huge advancement and streamlined search was essential in leveraging this.

The Start:
If you’re of the older generation, we’ve spoken before about how you probably remember landline telephones being used for both dial up and regular calls, along with the early days of the web where Usenet was king and Napster was where you found music.
In the early days of the internet things were vastly different and nowhere near as streamlined as we see today. Primarily the domain of the tech focused, the web struggled with consistency in providing relevant information and prior to 1993 the indexing system was even performed manually. Whilst this performed adequately in the late 80s as the internet exploded into mainstream use it became harder and harder to maintain this manually updated data pool as larger numbers of people entered the connected world.

The Boom:
Whilst it’s probably hard to imagine in today’s world where we are spoiled with choices for access, even things like how we browsed the web became a brand battlefield, with Netscape and Microsoft having it out for predominant market share of browsers with the outsider company Netscape providing many with their first taste of the modern internet via the Navigator browser.

It’s easy to think of this evolution as somewhat of a trivial thing however many are unaware of how much of this came to shape our modern connected world. Netscape who had the lion’s share of the market for the bulk of the 90’s had all but collapsed by the mid 2000’s due to the bundling of Internet Explorer with Windows equipped PC’s. This loss of market share that helped lead to the collapse of Netscape as a company was fundamental in one of the biggest trials in corporate America’s history, the 2001 United States v Microsoft Anti Trust trial. While the outcome and hearing of that trial is worth an article of it’s own, we’ve referenced this case as it sets the field for the dynamic of the era, with the dotcom boom of the time providing large amounts of capital and setting the stage for some of the internet giants we see today, like Amazon, Google and PayPal battling it out for their portion of market share. While these are billion dollar companies today, back then they were as new as everyone else with Amazon even being ran in the garage of Founder Jeff Bezos’ house in the early days.
We’ll quickly turn it over to ChatGPT now for a quick synopsis of what we might have seen as an internet user of the day for a bit more of an expansion of that picture.

Input: I’m an internet user in the late 90’s. I use Netscape Navigator and the internet still hasn’t figured out how to best perform search yet. Give me a quick synopsis on what my user experience might be like while i’m trying to search relevant information.

Output: As an internet user in the late 90’s, your user experience while trying to search for relevant information might be quite different compared to today’s standards. Here’s a quick synopsis of what you might experience:
1. Limited Search Engines: Search engines like AltaVista, Excite, and Lycos were popular during that time, but they were not as advanced as modern search engines. They primarily relied on keyword matching rather than complex algorithms to deliver results.
2. Basic Keyword Search: To perform a search, you would typically enter keywords related to the information you were looking for in a search box. The search engine would then return a list of web pages that contained those keywords.
3. Inaccurate Results: Due to the reliance on keyword matching, search results often lacked accuracy. You might encounter irrelevant or unrelated websites in your search results, making it difficult to find precisely what you were looking for.
4. Limited Web Content: The internet was still in its early stages of development, and the amount of content available was relatively small compared to today. This meant that finding specific or niche information could be quite challenging.
5. Slow Connection Speeds: Internet connection speeds were significantly slower compared to today’s broadband connections. This meant that loading web pages and navigating through search results could take a considerable amount of time.
6. Lack of Multimedia: The internet primarily consisted of text-based content, and multimedia elements like images, videos, and interactive features were not as prevalent as they are now. Finding visual or multimedia-rich content might have been difficult.
7. Manual Website Discovery: In addition to search engines, you might have relied on directories like Yahoo! Directory to discover websites related to your interests. These directories organized websites into categories and subcategories, allowing you to explore different topics manually.
8. Bookmarking: As a way to keep track of useful websites, you likely used browser bookmarks to save URLs of interesting pages. This was essential because revisiting websites or finding them again through search engines could be time-consuming.
Overall, searching for relevant information in the late 90s was a somewhat rudimentary experience. The search engines of that era were still evolving, and finding precise and accurate information required patience, persistence, and sometimes a bit of luck.
The Players:
When we look at how search engines evolved we can sense a pattern that we’ve seen occur in similar industries, namely social media companies. While this may be a controversial call for some, it merely takes a look through the history books to realize that some companies that were thought to be too big to fail at the time, were really anything but with market share evolving along with product development. This meant that companies that had large dominant portions of the market share were often bit players by the time of the new millennium, with many dotcom companies unable to survive the collapses of the time which were to come later on.
This meant that while we saw market leaders of the early 90s being search engines such as Archie, Alta Vista and Gopher, who provided basic text lists of search options these notable players were taken out of the market by later entrants such as Google and Yahoo who provided detailed search options with back linking and page ranking being fundamental in developing the search engines we use so effectively everyday in our modern world.

This 1998 ad for Yahoo at the peak of the dotcom bubble came at a time when Yahoo was far larger than it was today. Source: Youtube
The Dot Com Bubble:
If you weren’t around at the time, it’s hard to describe just how strange the era was back then. The excitement around new technology was infectious and in the early days many saw a new aspect to their lives. The digital component that came with businesses being online and the development of early era social network’s like ICQ and IRC being fundamental in connecting us and laying the path for the social media of the future.
Where as before the primary form of communication was either by telephone, post or if you needed it yesterday Facsimile, now finding a pen pal or nurturing a niche hobby became far simpler, with tools, forums and networks becoming available to those patient enough to spend the time looking.
However while we can paint a picture of the technology easily enough one thing that becomes more difficult to explain was the amount of money flying around at the time. With large market shares available to those willing to nurture risk and present a usable product to the market, the race was on to provide internet search in usable format that would allow the full power of the internet to be harnessed.
A good indicator of just how much money was involved at the time can be seen by analyzing a unique indicator of the market.
By looking at advertising types in the 2000 Superbowl XXXIV we see a slew of tech companies making the most of the moment with a large number of dotcom companies providing advertising entertainment at large expense to the ledger. We’ve included screenshots for E-trade below, a dot com company that paid over 2 millions dollars to air an ad featuring a dancing chimpanzee with the caption “Well we just wasted 2 million bucks”.
Etrade’s Superbowl XXXIV Spectaculor.
However a large number of these companies would fail to see out the year, with pets.com being one of the most notable to collapse being liquidated in November just 10 months later.
90s search was rudimentary at best. Source: Yahoo

Yaaaaaahhooooooooooo:
Today when we think of search, it’s easy to have one name spring to mind and that’s Google. And whilst Google most certainly was a key player in the developer of search, along with being the eventual victor there’s one other company that was a key competitor among this. It’s story is a prime example of just how dynamic the boom was along with how quickly things can change in just a few years which is why we’ve given it a special mention in this piece.
Formed in 1994, while Yahoo! Search was by no means the first, it was one of the first to face general acceptance and use by the mainstream.
Launched by a pair of Stanford students, Jerry Yang and David Filo, their program was a directory type format that despite not using a web crawler to provide accurate archiving was able to grow an develop by purchasing and refining technology throughout the boom. For a long time, Yahoo was the Google of the day and this was represented in its stock price, entering the market in 1996 and being valued at more than 600% of its IPO price just two years later. By this time in 1998, Yahoo served nearly three times as many requests as its closest competitor of the time, Excite.
However this market share and valuation was not to last. Despite surviving the Dotcom bubble, which was no small achievement Yahoo was wounded, loosing a significant value of its stock price during the bursting of the bubble. This was a secondary issue however as during the 2000 period Yahoo relied on Google to provide its search and indexing services. While it would take some time for this to play out in its entirety, by the late 2000’s the company was loosing market share, with Google increasing it’s footprint while providing services we know as essential today, starting with GMail in 2007 and setting the stage for it’s market dominance. As a result, Yahoo continued to flounder with notable incidents and deterioration in to the late 2010's.
During this same period Google continued to grow. Using an algorithm called PageRank, and by analyzing page linking as well as content, Google search became more efficient as well as heavily automated. With the launch of the Google Suite of products in the late 2000’s the battle for search was over with one clear victor sitting at the table. While many factors were responsible for the decline of Yahoo, it’s tale from a time when it was too big to fail remains a reminder to companies across the globe about the importance of evolving with technology and not taking things like market share for granted.
The development of language learning models means the evolution of search is anything but done. Source: openai

The Future:
It’s easy to get to this point in the article and feel like we’re dealing with a battle from long ago that is all but completed. However the reality is that as members of the internet community, we’re slowly facing one of the biggest changes in how we rank and use the internet since the collapse of the dot com bubble.
While it’s still early days, the development of internet enabled language learning models like Chat GPT gives us new tools in effectively managing the internet and implementing these tools within our day to day. We can see corroboration of this in the tools that we’ve seen launch over the last year or two, with BingChat a good example of how we may see these tools develop into the future, providing an integrated search experience that leverages both traditional search and SEO technology paired with a language learning model to increase efficiency and provide a better experience to the end user. Despite this taking time to develop, it’s reasonable to assume that the over the next decade, our search powers will continue to evolve meaning that even those of us with good Google-Fu will need to learn how to integrate these tools into our day to day experience.

Where to Find Us:
If you’d like to be paid in crypto for reading our content you can find us on publish0x.
If you’d like to support us on medium by becoming members, you can do so here.

For our content updates on all platforms which includes an early article drop for followers at least once a month as well as the occasional giveaway, follow us on telegram here.

We’re also making What the Tech?! into an official medium publication. If you’re a creator and you’d like to submit a story for publication then get in touch as we’d love to hear from you.

We’d also love if you follow our new publication and share it with your friends!