Skip to Main Content
Research Guides

Review: The Deep Web

The Deep Web, also known as the Invisible Web, is a portion of the web not reached by standard search engines such as Google and Bing.  Less than 10% of the web is indexed by search engines with the remaining 90% of web content called the Deep Web.  It is estimated to be 2-500x bigger than the surface web. 

Content on the Deep Web is not found by most search engines because it is stored in a database which is not coded in HTML.  Google and Bing might lead us to a front door [a search interface], but it generally can't search the content of a databse.  It is up to you to search the database where the results of your search are loaded into a dynamically generated HTML page for viewing. 

Directories

Strategies for searching the Deep Web

Make the mental shift from find the content to find a doorway to the content. 

  1. Look for portals and directories - lists of links to information abuot a topic.  The best are constantly updated and have few broken links.  Try searching for a subject term and the word database. Examples:

    public records database
    crime database
    languages database
    toxic chemicals database

  2. Search for foundations, research institutes, even fan organizations.  Non-scholarly aficionados of a discipline will often create a list or portal of excellent resources.
  3. Universities that offer a program of study in a particular field will often feature excellent portals to information in that field.  For example, the Pacific Northwest has a rich labor history and the UW Libraries has a large collection of labor papers and artifacts.  To provide access to this history, we have the Labor Archives of Washington.
  4. Think globally, act locally.  The best information source is not always an ivory tower.  For example, one of the best sources of historical information about Japanese internment during World War II is the Densho Project: the Japanese American Legacy Project.  The Densho Project is digitizing their archive, but not everything is online.  The best source of historical information about Saratoga, CA is the tiny little museum (and the archivist inside) on Saratoga Avenue.  \
  5. Ask an expert.  Most of the information on this guide has been put on the web by experts.  Ask experts [librarians, professors, researchers, authors, archivists] to recommend portals or directories of specialized information.


Content from http://library.harker.org/DeepWebMining

Deep Web search engines

Here are a few ways to search for content on the Deep Web: