Difference between Crawling and Indexing?

Difference between Crawling and Indexing?

Crawling means if Google bot (softwares) visits a page in your site and reads the content inside it is called crawling.

Indexing refers to when Google saves your site address in its index (collection of webpages, content, anchor text).

What is NoIndex Meta-Tags:

The ‘Noindex’ allows search engines to read the pages but instructs them to remove it from memory that it was ever indexed. That means when the search engine comes to a page with noindex meta tag, it will continue to read the content inside it including the links (so link juice is passed) but will forget it after reading and will not index it.

What is NoFollow?

If a search engine finds a “no follow” backlink it will just ignore the link and will not get deeper into crawling and no link juice is passed.

What is DoFollow?

When Google bots are crawling a website they look for external links point to other websites. If this links is “do follow” then search engines will follows the link and so link juice gets passed.

How different search engines treat Nofollow?

Google: Google does not follow the nofollow link.

Yahoo: Yahoo follows that link for finding new content but link juice is not passed.

Ask: It does not follow nofollow tag and treats it as normal dofollow link.

Keep reading for more informative posts...

Working of Search Engines

Working of Search Engines - Chapter 2nd

In this chapter you will come to learn about the search engines and how Search Engine works. It is very important topic with SEO point of view to have a knowledge about the Search Engines, because if you have proper knowledge about search engines only then you can perform well on your projects and give desired results to client.

When we are talking about Search Engines we should know about how can we define Search Engine?

Search engine is a web based tool that search documents for specified keywords and returns a list of the documents where the keywords were found. The most popular search engines are Google, Yahoo, MSN and Bing.

Now, the next question that rise in your mind is how Search Engines work? So read the points below:-

1.) While working of Search Engines two software play an important role

a.) Spiders (also called robots, bots)

b.) Index or Indexer

2.) Search Engines work by sending out a Spider (also called robots, bots) to fetch as many documents as possible. In other words We can also say that these spiders (also called robots, bots) travel along the web, following links from page to page, site to site. The spider doesn't collect images, flash, logos or formatting - it is only interested in text and links AND the URL, (for example, http://www.example.com)

3.) Then another program, called an Indexer reads these documents or words or links or URL's and creates an index based on the words contained in each document. The Indexer makes sense of the mass of text, links and URLs using what is called an algorithm - a complex mathematical formula that indexes the words, the pairs of words and so on.

So now in brief the spider takes the information it has gathered about a web page and sends it to the index software where it is analyzed and stored.

3.) When someone types shoes into the query box on a search engine page (such as Google), then it's time for the query software to go to work.

The query software is what you see when you go to a search engine. The query software doesn't actually search the web - instead, it checks through all the records that have been created by its own index software. And those records are made possible by the text, links and URL material the spider software collects.

Types of Search Engines will be continued in next chapter. i.e 3rd Chapter...