Firstly, we’re going to be talking about search engines.
Crawling & building an index is one of two main functions of a search engine. The search engines crawl the internet and stops at all the best links, through these links the search engines are able to reach all of the interconnected documents.
Every page found by the search engine has to be stored so that it can be used again for another search. For search engines to hold all these pages to be accessed again within a fraction of a second, they store them in data centres all over the world.
These data centres facilities are able to hold thousands of machines which will all be processing extreme amounts of information. The search engines are then able to access all the data at extreme speeds, if this is not done when a search is made it may cause some dissatisfaction.
Providing answers is the other main function of a search engines. The search engine is what provides you with as many possible answers it can when a search is made. The search engine makes sure that any answers it retrieves are relevant to the search and then it will then rank them in order to give the most useful and important first.
You may wonder how the search engines chooses which pages are more important than others. This is done by the popularity of the page; the more popular the page is for that search the more important the information is. The popularity of the page is determined with mathematical equations, this ensures that it is all correct and no errors can be made.