Loading ...« If a Business isn’t in a Search Engine? | Home | National Advertisers in a Local World »
Local Search: Think Data Mining
By Matthew Berk | September 25, 2007
Local Search isn’t about search.
Bear with me….Four years ago, anyone wanting to strike out in the world of vertical search (think local, shopping, people search, etc.) really had two problems to solve: first, how to aggregate content only applicable to their intended domain, and second, how to make that content searchable. The first of these tasks was largely a mechanical proposition (directed crawling and/or feed management), while the second was where we loaded on the magic. I say magic largely because back then, search had not yet been commodified to the extent is has been today (granted, it’s still hard to make search work well, but back before packages like Lucene were widely adopted and stable, building keyword search from scratch was really like wrestling with angels).
Today, I’d argue, producing an effective product with a vertical (or local) focus carries an additional challenge, and one that’s especially tricky in the local space: data mining. After the mechanics of aggregation (really, noise filtering), and prior to applying any form of content retrieval metaphor, innovators are really forced to listen to the data, to interpret it, and to make sense out of it. While some of us knew this was our lot long ago, it’s especially critical today, as the volume of content available has skyrocketed (think UGC, local guides, competing local data sources, ever more prolific local publishers, etc.).
To get concrete, let me give an apt local example: the taxonomy. At the last Kelsey event, the panel I sat on was called “building a better database,” and had speakers representing three points on a spectrum:
- The advertiser-centric YP taxonomy. Rich, varied, and robust, the world of 20,000+ YP headings treats the blacksmith and the restaurant with equal taxonomic weight.
- The consumer-centric local taxonomy. A tighter remix of the taxonomies applied to local base data, this kind of organizational regime seeks to apply a focused, consumer-relevant structure to underlying base data by interpreting raw data and categories through the lens of what’s important to the typical consumer.
- The “we don’t need no structure because full-text and user generated tags is ideal” organizational regime, in which user-generated tag sets and full keyword search stands in for the structured organization of data.
Open List (www.openlist.com) has always been about the middle of these two paths: fielding as much data–categorical, taxonomic and other–as possible, and then undertaking to understand, interpret, normalize and reorganize it on behalf of the consumer audience. This is where data mining comes in: the work of local search is not merely the act of making local listings searchable, but of making sense of the data on behalf of the consumer.
As another example, consider what happens when the work of aggregation creates an “embarrassment of riches” scenario: business categories where individual merchants can have dozens (in some cases, well over a hundred) reviews or pieces of content associated with them. In the industry, we all tend to view UGC (user-generated content) as always positively tinctured: the more, the better. But past a volumetric threshold, we’re simply baffling the consumer. Enter data mining, and the Open View, a technology by which we mine content and construct an English-language summary programmatically:
The Four Seasons Hotel New York, in New York, is a 5 star luxury hotel. Those who recommend it say it’s perfect for “business”. Sleep connoisseurs described the bed as “great”. Guests found the service “excellent” and “exceptional”. What travelers said they loved: “the people”, “the place”, “the service”, “the staff”, and “the room”. The hotel is recommended by Gayot.com (it’s on their Top 10 list for Business Hotels), Fodor’s (it’s one of their hotel “Picks”), and seasoned travelers, who rate it 5 stars. Travel + Leisure named it one of the “World’s Best” and it made the prestigious T+L “Top 500″ list.
Data mining is hard work, and seldom the graceful, fully-automated, completely algorithmic enterprise many folks would have us believe; it’s painful and labor intensive, fraught with false starts and false positives but, we believe, where the kernel of real consumer value is generated. To put it in the obverse, working in the world of vertical search is not looking a hard problem in the eye and then tossing it back to your users to solve, nor is it the tidy sum of directed crawling plus Lucene; it’s messy, slow, difficult work and, if you’re very lucky and very persistent, all adds up to a new kind of magic: helping the data help consumers make better decisions.
Topics: Data, Local Search |
October 5th, 2007 at 2:15 pm
[…] Should My Mom Learn Geo-Targeting? OpenList.com is Live! National Advertisers in a Local World Local Search: Think Data Mining If a Business isn’t in a Search […]