What do you mean by indexing?
Indexing is a process where our server crawls through your website, fetches every page that it can find
and stores a list of keywords that are found on your site in our database. These keywords are then used
to find pages on your site when a user perform search operations.
How does indexing work?
When an account is created, a list of starting URLs are added for your website. Our indexer, sometimes also called
as a spider, is a software that programtically fetches the page referred by these starting URLs, extracts all HTML
links from those pages and then repeats the same process again until all links are exhausted.
IMPORTANT: The indexer cannot follow links that are dynamically generated using JavaScripts. Therefore, if there
are pages on your website that can only be accessed through scripting, you should add those pages as the starting URLs.
|
