why is weblog search so hard?

Posted by | January 09, 2006 | search engines | No Comments

Buried within the comments of Jermey Zawadny’s post about Feedster is this comment:

“I don’t recall Feedster ever being all that useful. But I also don’t find Technorati particularly useful. Why can’t someone just create a simple search engine for feeds/blogs?”

The truth is that it is very difficult to build a search engine with real-time updates, since search engines are optimized for retrieval and usually use batch indexing. In addition, the majority of weblogs are spam, further compounding the problem.

Blog search, which may once have seemed niche, will eventually be a standard part of search engines. At the moment, nobody, including Google, have a weblog search product that works.
If they did it would be very useful.

The real reason this is important is that it has nothing to do with weblogs, long term. There are only two things that matter in search – freshness and relevancy.

At the moment search engines like Google do not have a button that says order results by date – they will, eventually, and from that comes blog search or from blog search comes that.

Feedster Will Die in 2006 (by Jeremy Zawodny)