Federated search
From Wikipedia, the free encyclopedia
Federated search is the simultaneous search of multiple online databases and is an emerging feature of automated, Web-based library and information retrieval systems. It is also often referred to as a portal, as opposed to simply a Web-based search engine.
Contents |
[edit] Purpose
As described by Peter Jacso (2004), federated searching consists of (1) transforming a query and broadcasting it to a group of disparate databases with the appropriate syntax, (2) merging the results collected from the databases, (3) presenting them in a succinct and unified format with minimal duplication, and (4) providing a means, performed either automatically or by the portal user, to sort the merged result set. In traditional search engines such as Google, only sources that have been indexed by the search engine’s crawler technology can be searched, retrieved and accessed. The large volume of documents housed in databases is not open to traditional Internet search engines because of limitations in crawler technology. Federated searching resolves this issue by the technique described above and makes these deep Web documents searchable without having to visit each database individually.
[edit] Process
Federated search computer programs allow users to search multiple data sources with a single query from a single user interface. The user enters a search query in the portal interface’s search box and the query is sent to every individual database in the portal or federated search list. Access details for the individual databases must be preset in the portal by its owner. Federated search systems either rely upon vendors to create commercial portal systems, or they rely upon government or other organizations to provide open access portals. How federated search is implemented depends upon which of the two types of organizations is providing the portal.
Federated search portals, either commercial or open access, generally search public access bibliographic databases, public access Web-based library catalogues (OPACs), Web-based search engines like Google and/or open-access, government-operated or corporate data collections. These individual data sources send back to the portal's interface a list of results from the search query. The user can review this hit list. Some portals will merely screen scrape the actual database results and not directly allow a user to enter the data source's application. More sophisticated ones will de-dupe the results list by merging and removing duplicates. There are additional features available in many portals, but the basic idea is the same: to improve the accuracy and relevance of individual searches as well as reduce the amount of time required to search for resources.
This process allows federated search some key advantages when compared with existing crawler-based search engines. Federated search need not place any requirements or burdens on owners of the individual data sources, other than handling increased traffic. Federated searches are inherently as current as the individual data sources, as they are searched in real time.
[edit] Implementation
One application of federated searching is the metasearch engine; however, this is not a complete solution as many documents are not currently indexed. This is known as the deep Web or invisible Web. Many more information sources are not yet stored in electronic form. Google Scholar is an example of a project trying to address this.
When the search vocabulary or data model of the search system is different from the data model of one or more of the foreign target systems the query must be translated into each of the foreign target systems. This can be done using simple data-element translation or may require semantic translation.
Another application Sesam running in both Norway and Sweden has been built on top of an open sourced platform specialised for federated search solutions. Sesat, an acronyn for Sesam Search Application Toolkit, is a platform that provides much of the framework and functionality required for handling parallel and pipelined searches and displaying them elegantly in a user interface, allowing engineers to focus on the index/database configuration tuning.
