Powerset have a demo out and its interesting, technically proficient and built by a solid team, but winning requires questioning the premise: is better search a problem and is it solved by changing the way people are currently used to searching to the the way people naturally speak?
Google is a long term threat to Microsoft’s hegemony not by having built a better OS, but by owning Search. The web shifted the landscape of technology and a a once niche application, dominated by companies like Verity: full text search, became the ‘command line of the web’. Since Microsoft had always owned the command line, this made web search a strategic threat.
Powerset has some very bright people like Barney Pell behind it, and who am I to challenge it, but I have a nagging doubt, which is to do with my years spent in architecture rather than technology. In architecture the first thing you do is question the brief: if someone asks you for a building with a sloping facade, you ask why and you may have a good reason for doing something differently. If someone asks you for a better search engine, you would ask why. Here is my asking why.
If the value in building a better search engine is to beat Google, perhaps Google can only be beaten when something other than a search engine becomes a starting point for the web. It doesn’t take a stretch of the imagination to see that if Facebook became a truly monopolistic social network it would be a strategic threat to Google. If building a better search engine is the way to beat Google then Powerset is on the right track.
Is the way to build a better search engine based on the ability to answer questions the way they are spoken? If so, then natural language technology is the right approach and Powerset is on the right track. A few years ago this would definitely be the case, but these days, the ergonomics of the web have evolved in tandem with Google. People don’t tend to type question into search engines, but type a few salient words. This may not be the most elegant practice, but it is the de facto standard behavior and to try and change it might be like trying to change the QWERTY keyboard for a more rational one.
Assuming that there is a better search practice than currently used, how does Powerset stack up when natural language queries are typed into it. This would require very thorough testing, but I’ll give on example: ‘who was churchills father’ [sic]. Both sites return the correct answer, but Powerset requires adding an apostrophe: churchill’s, not a big deal for them to fix but a perfect example of how a simple grammatical rule dealt with by query parsing can sometimes get forgotten in the attempt to index perfectly.
Lastly, intelligent indexing comes at a cost – it may be slower to query, and it is definitely slower to index. Quick response time has always been a priority for search – and Powerset can possibly match. But the biggest change on in search in this second phase of the web, has been the rise of ubiquitous, news style (e.g. weblog) publishing systems and the importance of search by date. AltaVista’s last throw of PR success against Google was their news search which was pounded after 911, before Google News, let alone weblog search existed. Fast updates require fast indexing.
I wish Powerset every success, and think that this will come when something else is thrown into their mix.