"The utopia of Internet search engines? That's gone," said Andrew
Wetzler, president of MoreVisibility.com, a search engine
optimization company. That's good for Wetzler and businesses like
his that help companies get better placement in search engines. But
for anyone who sees the web as more than just a place to buy an air
purifier from Sharper Image, it stinks.
Now, I like plenty of commercial stuff online, but the problem is
that there's no balance on search sites today. As with every
previous Internet marketing fad, search engine marketing, which is
hot right now, is being grossly abused. With most sites biasing
their own results with a number of paid offerings, the balance of
educational information and commercial content-in other words, what
made the web interesting in the mid-1990s-is being lost. You simply
can't find the non-commercial stuff. It's also clear that existing
search engines, using their current approaches, will never be able
to keep up with the Web's explosive growth-Google indexes about 3
billion pages, but conservative estimates place the Web's size at 10
billion pages-so a new idea is welcome in these quarters.
One hope: Grub (http://www.grub.org/). Grub is a project that wants
to index the entire Internet. Every day. It works much like the
SETI@Home project that's looking for intelligent life in the
universe by harnessing what's known as grid computing, accessing the
collective power of the online community's unused computing
resources through a screensaver you install. Grid computing has its
skeptics, and of course, having the X-Files crowd as its poster
child doesn't necessarily aid its credibility. But Grub needs only
perhaps 100,000 users to start to achieve its goal, and in just a
month, it already has about 10,000 active users running the
screensaver.
Grub started a couple of years ago as an independent venture by
the owner of an Oklahoma-based ISP and has recently been acquired
and launched by LookSmart, traditionally a second-tier player in
search and one of the more aggressive players in putting for-sale
signs on Net real estate. Grub's creator, Kord Campbell, and
LookSmart execs say in interviews that they want to create a "DNS
system for websites," meaning that they would create a central
registry for web pages. Once you know about every site out there,
the thinking goes, then you can share that information -- and do
some interesting things with it. Examples mentioned include true
interpretative search, using the power of the grid to determine
whether you meant soups, securities, or 17th-century punishment
devices when you typed in "stocks." Or you could create a database
that was totally free of dead links that clog up a lot of search
results.
Questions remain about the viability of Grub. Is there anything
worth finding in the pages not searched today, or like the early
shrimp boat scenes in "Forrest Gump," will Grub just bring back the
rusty tin cans and waterlogged boots of the Internet? There have to
be some uncaught prawns out there, because Google's searching
technique relies too much on popularity and influence, and as most
of us remember from high school, just because you're popular doesn't
mean you're interesting, and vice versa. Even if it is just muddy
Mountain Dew bottles that get caught in Grub's wide net, there will
always be someone who's searching for just that and will be happy.
Of course, there's still the issue of whether LookSmart, whose
track record of search integrity has more holes in it than 50 Cent,
is the best company to find the hidden web. The LookSmart execs I
spoke with seemed genuine in their desire to give back to the Net
community, knowing how much money they make from paid inclusion.
Their intent to double-check the veracity of the index they compile
and then make it available to anyone who wants to build an
application on top of it is the kind of web-friendly idea that you
don't hear much of anymore. And it may just mean that LookSmart, one
of the few search engines still out there that never was a one-hit
wonder, has the ideal business model that balances search engines'
commercial and consumer responsibilities.
So maybe the idea of a search utopia isn't dead yet.