As the big three search engines (Google, MSN, and Yahoo) have been so active of late, with advancements and proposals and deals flying back and forth between them, news on other search technologies available might have been missed. We’re going to take the opportunity now to welcome Alexa and Dogpile to the search algorithm stakes.

Ask anyone involved with search engine optimization about the web traffic monitoring and rankings service Alexa (which is a division of Amazon) and you may have the answer preceded by an exasperated sigh. While the downloadable Alexa toolbar has been used by many Internet users, from the casual surfer to webmasters, the fact that the ranking site had based its statistical information from the toolbar users alone was frustrating. Some of the functionality that Alexa offered with the way it served up this data was good, but the fact that the data was, for lack of a better word, “skewed” was the cause of much frustration. It was Alexa’s heavy reliance on user data that led to the sighing and grinding of teeth.

But this is no more. Alexa recently announced that it had completely revamped its ranking protocol. Alexa said it would be moving away from releasing data based solely on users of the downloadable toolbar, to ranking sites based on info from multiple sources. The new Alexa rankings are now based on more sources of data. This provides a much better indicator of website popularity, which in turn leads to better rankings as site popularity is more accurately reflected in the new and improved algorithms. Additionally, Alexa has introduced an improved methodology to provide consistently accurate rankings for all countries.

Not to be left behind, Dogpile (owned by InfoSpace) has also made improvements. Dogpile has not only updated their updated search algorithm, but also their site design and they now offer new partner content.

Dogpile’s updates include improvements in the blending of results (the importance of blended results has really come to the fore as a result of Google’s Universal Search), the addition of tabbed search categories on the home page, the launch of a “SearchSpy” social networking widget for Netvibes or iGoogle, deep links (which are along the lines of Google site links), and integrated content from new partner agreements

While the big three search engines continue to one-up each other in terms of relevance and functionality, the smaller search engines and web ranking tools are doing the same. Alexa and Dogpile have shown that they are listening to their users and are serious about continuing to upgrade their offerings and providing their users improved features and technology. As Google, MSN, and Yahoo push one another to be bigger and better, their advancements also serve to inspire the other ranking and search technologies in the market. It will be interesting to see who will be the next entry to this stage of the race.

With Microsoft’s unsolicited bid to purchase Yahoo, how would such a merger affect the Internet, especially where search technology is concerned?

Microsoft’s long courtship of Yahoo has now entered a serious phase, one of not-so-gentle persuasion and has somehow managed to become even more complex. It’s no longer a matter of, “Software Company seeks Web Portal for friendly merger” – events of the past week have elevated Microsoft’s chase of Yahoo into a complicated mass of what-might-be.

Since we last wrote about the major three search engine companies – Microsoft, Yahoo, and Google – volleys have been lobbed back and forth from both sides over the possible “YaSoft/MicroHoo” union. This past week alone has seen not only an increased flurry of activity, but a more intense pursuit as well.

When Microsoft’s initial $44.6 billion dollar take-over bid was first made public, Yahoo vigorously rejected it. The weeks that followed the rebuff were filled with silence from both the Yahoo and Microsoft camps. Yahoo had made it quite clear that it felt Microsoft’s offer seriously undervalued its earnings potential and, in turn, Microsoft felt that it made a fair offer.

For weeks, it seemed that was that. Neither side had any more to say on the matter. Then reports surfaced that Yahoo was talking to Time-Warner about a merger with AOL. This led to speculation that Google might leverage its 5% stake in AOL by partnering with or even acquiring AOL before Yahoo could pursue them.

Given Yahoo’s pursuit of alternate suitors (News Corp Online and Time-Warner), which may be viewed as either a way to escape Microsoft’s bid or to raise it to perhaps a $34.00 a share asking price, Microsoft issued an ultimatum: the initial $31.00 a share bid is good until April 26th. After that, the offer price will drop and there will be a proxy fight to nominate a new board of directors.

Despite this ultimatum, Yahoo continued on with business as usual, immediately responding with an announcement that Google will power 3% of paid search ads on Yahoo on a test basis. This fueled the earlier speculation that Yahoo and AOL were in serious talks and that Microsoft and News Corp Online were partnering to bid for Yahoo.

Microsoft’s reply to the Yahoo-Google search deal was to raise the issue of antitrust concerns, concerns that had been noted by Google when news of the Microsoft bid for Yahoo first surfaced. In fact, Google and Yahoo combined account for at least 80% of the search market. A full-scale outsourcing deal would place Google in control of the vast majority of paid search. However, Microsoft’s antitrust concerns are more reflective of the magnitude of a full-scale merger, one which would combine paid search, organic search, and paid advertising, and not a search outsourcing deal.

Almost as if to battle this, Yahoo made several announcements, one of which detailed a three-year deal with Major League Baseball’s MLB.com. The deal involves video distribution and related advertising, with Yahoo offering a co-branded version of the MLB.TV media player on Yahoo Sports.

In the first year of the MLB.com deal, Yahoo will offer its clickable video ad format (allowing users to click through on video ads) along with standard pre- and post-roll ads. After this initial first year period, all video ad inventory will be powered by Yahoo’s new AMP ad platform.

The new advertising management platform called AMP will be rolled out later this year. The platform is intended to offer a single interface for buying search, display, local, mobile, and video ads across all of Yahoo’s properties. Additionally, AMP will also reach across Yahoo’s premium ad partners (among them AT&T, eBay, and Viacom) as well as the sites of more than 600 publishers in the Newspaper Consortium. Consortium founders include Belo, Hearst, Lee, MediaNews Group, and Scripps.

The AMP platform is scheduled to be available to the Newspaper Consortium in the third quarter of the year and to Yahoo’s other advertising partners after that.

In addition to striking a deal with MLB.com and rolling out the AMP platform, Yahoo also announced the acquisition of IndexTools , a maker of Web analytics software for online marketing.

Of all of the recent deals struck by Yahoo, it’s this acquisition that presents a risk for them. Although IndexTools is considered by many to be superior to the free and low-cost solutions offered by other services, time can negate any benefits. For example, Yahoo could take too long to integrate IndexTools, more hiccups could appear with the Microsoft bid, and Google Analytics could offer more dramatically up-market, competitive features.

Having spent some time looking at what Yahoo has been up to let’s take a look at Microsoft’s recent activities. The flurry created with Yahoo’s meeting with Time-Warner led to an unexpected move on Microsoft’s part – the software giant reached out to News Corp Online’s MySpace. Having MySpace as an ally would fill a major gap for Microsoft, as the software giant is currently missing a social networking tie-in. This tantalizing bit of news certainly influences thoughts of what may come of YaSoft/MicroHoo.

With recent developments in hand, and assuming that the Yahoo board of directors (who met April 11th), decide to entertain Microsoft’s offer, the potential of the YaSoft/MicroHoo union fairly explodes.

Let’s start with Google and MySpace. Google has a $900 million Internet ad search deal in place with MySpace. Microsoft recently paid $240 million for a 1.6% stake and a broader marketing relationship with Facebook. An alliance of Yahoo, Microsoft, MySpace and Facebook, ostensibly to compete directly with Google, is mind-boggling.

Despite this flurry, it is still likely that Microsoft will acquire Yahoo. In the end, all of the players involved will still be focused on competing with Google. After all, Google accounts for over 50% of Internet searches. The current scramble has Microsoft, Yahoo, and AOL working to expand their respective user bases and to develop more powerful advertising mechanisms. Microsoft’s courtship of Yahoo has morphed into a strategic tit-for-tat involving MySpace and News Corp Online, and AOL and Time-Warner. The irony to this development is that Google lacks its own social networking component, yet Microsoft and Yahoo are attempting to place themselves into a better competitive position with Google by acquiring social networking.

So, should Yahoo’s board of directors accept Microsoft’s offer, whether it be at $31.00 a share or slightly higher, there are still at least 18 months of integration, streamlining and cultural clashes. This would leave Google the same 18 months to launch improvements and to move into areas where Yahoo may hold a lead (specifically, Internet banner ads).

Once these hypothetical 18 months come to a close, likely point of impact may not necessarily be search technology (Microsoft and Yahoo combined have roughly 30% of online search) but web email and instant messaging (MSN Hotmail and Yahoo Mail, and MSN Messenger/Windows Live Messenger and Yahoo Messenger). A combination of these services (of which Yahoo has the stronger presence and technology) would make YaSoft/MicroHoo the most heavily trafficked portal on the Internet.

In the short term, search technologies from MSN/Windows Live Search and Yahoo Search are likely to remain the same as it will take a significant amount of time to get through the legal hurdles, let along the logistical, technological and cultural ones. This span of time may allow for Google to test and implement further improvements to their algorithm and to build out the functionality of their Universal Search. In the short term, things should be business as usual, with the search engines delivering what we come to expect in the way of improvements with their search technologies. It’s the long term that still remains in question.

With Microsoft’s unsolicited bid to purchase Yahoo, how would such a merger affect the Internet, especially where search technology is concerned?

Previously, we explored how the proposed Microsoft – Yahoo merger would affect our use of the Internet. In this installment, we’ll take an historical look at the ways in which Microsoft, Yahoo, and Google have enhanced our experience on the Internet.

Before we soothsay and gaze forward to what may be with “YaSoft” or “MicroHoo” and how Google will react to either possible incarnation, let’s take a look back to where the big three search engines began.

MSN Search/Windows Live Search/Live Search
MSN Search was introduced in the fall of 1998 using search results provided from Inktomi, a search company whose technology employed the widely-used HotBot search engine. In the spring of the following year, MSN Search launched an improved version of their search engine, one which displayed listings from LookSmart, an online advertising company, blended with results from Inktomi.

Subsequent upgrades to MSN Search provided it with its own Microsoft-built search engine results, offering an index of data that was updated on a weekly or even a daily basis. MSN Search also began providing its search results to other search engine portals in an effort to better compete in the market.

In 2006, MSN Search underwent the change to Windows Live Search and then to Live Search. The changes moved beyond the offering of a search engine, index, and web crawler and expanded its use to include the ability to search for specific types of information using search tabs for the Internet, news, images, music, computer desktop, local search, and Microsoft Encarta, the digital multimedia encyclopedia published by Microsoft.

Live Search offers innovative features, among them the ability to view additional search results on the same web page (instead of needing to click through to subsequent search result pages) and the ability to dynamically adjust the amount of information displayed for each search result. It also allows users to save searches and see them updated automatically on Live.com.

Yahoo!
Before Yahoo was Yahoo, it was “Jerry’s Guide to the World Wide Web”. Yahoo was originally developed over 13 years ago as a web-based directory of sites compiled by Jerry Yang and David Filo, then students at Stanford University. The original organization of Yahoo was a hierarchical structure as opposed to a searchable index of pages. Over time, “Jerry’s Guide to the World Wide Web” developed into “Yet Another Hierarchical Officious Oracle” (Yahoo), a full-fledged portal with a search interface.

In the early 1990s, “Yahoo Search” initially referred to the Yahoo-provided search interface. Much like the early stages of MSN Search, the results presented weren’t from Yahoo per se. The actual web crawling and storage and retrieval of data were powered by Inktomi in 2001 and later, in 2004, were powered by Google.

Wanting to provide its own search engine results, Yahoo bought Inktomi in 2002. In 2003, it purchased Overture Services, Inc., which owned the AllTheWeb and AltaVista search engines. It was also in 2003 that Yahoo! Search finally evolved into its own web crawler-based search engine called “Yahoo! Slurp”. Yahoo had taken the step of combining the capabilities of the search engine companies it had purchased and put those capabilities into a single search engine. In this way, not only could Yahoo serve up its own results, but it could also sell those results to other companies. It was at this time that Yahoo’s relationship with Google was concluded (as they were now direct competitors).

One of the more recent advancements made to Yahoo! Search technology-wise is the addition of Search Assist. With Search Assist, as search terms are typed, related search terms are automatically suggested, enhancing Yahoo’s user experience.

Google
Another innovation from Stanford University came in the form of a research project called “BackRub”. The 1996 research project was initially that of Larry Page, who was soon joined by Sergey Brin while both were Ph.D. students at Stanford University. BackRub ultimately became “Google”. The name originated from a misspelling of “googol,” which refers to the number zero followed by one-hundred zeros.

The search engine originally used the Stanford website with the domain google.stanford.edu. In 1997, the domain google.com was registered, and in the following year Larry Page and Sergey Brin formally incorporated their company, Google, Inc.

By the end of 1998 Google had amassed an index of approximately 60 million pages. Although the home page was still marked as an “alpha test”, reviewers were making the argument that Google’s search results were better than those of competitors like HotBot or Excite. Google was also garnering praise for being more technologically innovative than portal sites such as Yahoo, Excite, Lycos, AOL, Go, and MSN. One of the reasons that Google found early loyalty and interest was due to its simple design and the way in which it worked. Larry Page viewed the link structure on the World Wide Web as being a huge graph. He and Sergey Brin hypothesized that a search engine which analyzed the relationships between websites would produce a better ranking of results. Their search engine took into consideration the number and the nature of web pages that link to a given page, placing the value of importance on the nature of these “backlinks”. By analyzing the output – which, for a given URL, consisted of a list of backlinks ranked by importance –Google was able to produce better results than existing techniques. Search engines at the time essentially ranked results based on how many times the search term itself appeared on a web page.

And now, back to the present: Microsoft’s $44.6 billion bid for Yahoo (which is $31.00 per share) is at a stalemate. In simple terms, Yahoo believes it is worth more than $31 a share, but it can’t force Microsoft to raise the asking price since Yahoo hasn’t (yet) been able to find a so-called “white knight”. Similarly, Microsoft doesn’t want to move away from what it views as being a fair offer.

Among the white knights that Yahoo has met with are News Corporation Online (News Corp Online) and Time Warner. Yahoo was unable to make a deal with News Corp Online. The parent company, News Corporation (News Corp), is still digesting Dow Jones and New Corp’s chairman and managing director, Rupert Murdoch, reportedly does not want to get into a bidding war with Microsoft

Time Warner, who owns AOL, may be a stronger ally than News Corp Online. Yahoo and AOL are similar in many ways: both are quite strong in webmail, instant messaging, and online display advertising, and both are legacy companies, having come to prominence during the dot-com gold rush of the late 1990s. Additionally, Google owns 5% of AOL and Yahoo and Google have a similar “friendly” ethos and openly embrace open source.

However, one week after meeting with AOL, Yahoo met with Microsoft. For the moment, the stalemate continues. With one company whose strength lies in productization, another whose strength lies in amalgamation, and another whose strength is innovation, what happens next? What could happen to the current search technology should a YaSoft/MicroHoo union occur? What, if anything, does all of this mean?

Our next installment will try to answer that very question.