Monday, November 10, 2008

Database Search vs. Text Search

Databases store their information neatly organized into fields, such as product name, category, description, price and so on. While databases can set up complex queries to find the search words in all applicable fields, this makes them slower to respond, requires more memory, and is more difficult to program. Text search engines store this information in a single index and can find words in any field for a record. Many high-end search engines can also store field information, so searches can be limited to a specific field as well.

Users are required to type in complex Boolean or SQL (Structured Query Language) commands for searching. Some databases, such as MySQL, are limited to "Or" searches, will return all records which match any of the search terms. Others default to exact phrase matching, so a search for fuzzy slippers would not find slippers, fuzzy. Search logs show clearly that most people will not do anything complicated while searching, they will give up if the search is too hard to use. Full-text search engines offer simple and flexible search options, with most providing an Advanced Search feature for power users.

Database search functions tend to look for exact matches in capitalization and characters. If someone searches for Safire, they won't find records with Safire in them. Many text search engines will automatically match lowercase searches with any text, and will adjust for extended and diacritical characters. Some also allow search administrators to set up synonyms for searching (doctor and physician), and automatically perform stemming (find octopi when searching for octopus).

Many databases are not designed for easy searching. For example in a wine database, it may be difficult to find wines from a certain region, such as the Napa valley of California, because the location is stored in a table far away from the wine names. Some text search engines, such as dtSearch, FAST Data Search, Ultraseek, and Verity K2, provide a rich data set for searching. Databases are optimized to search for exact words and phrases, and they tend to respond very slowly otherwise. So if a searcher wants to find sheepskin or shearling in the same search, databases will tend to do two searches and then merge the results. Full text search engines are designed to store these words in a single index, so they can perform these kinds of searches efficiently and return quickly.

Many database search engines will happily display all results on the same page, whether there are 8 or 8,000 records. Text search engines have a mechanism for dividing up the results and providing navigation from the first page to following pages (and back).

Database search functions sort by size, or price, or date, or the order in which the items were entered in the database. Text search engines sort by relevance, as determined by the number and location of matched words in the result page or record.

No comments: