Search Engine Tools

Bing Indexed Pages Checker
Find out how many pages are indexed in Bing for any website. The Bing Indexed Pages Checker will scan the Bing search engine and tell you how many pages are reported in their search index for a domain. You can check a full domain (mydomain.com) or subdomains specifically (sub.mydomain.com). What is Bing? Bing is Microsoft's search engine, formerly known as Windows Live Search or MSN Search. Bing is one of the newest major search engines, but is still based on years of indexes and technology developed by Microsoft for their MSN Web search engine. Bing is generally considered to be one of Google's major competitors, and Microsoft has given Google their first formidable competition with the release of Bing in 2009. What are Bing Indexed Pages? Bing constantly crawls the Web to discover new content, and to refresh their "cache" of content for existing websites. When Bing crawls your website with their search bots, they create an index that tracks all of the URLs on your website. Once a web page has been crawled and included in their index, it can be included in their search results. In order to be ranked in Bing results, a page must first be indexed by Bing.

Google Indexed Pages Checker
Quickly check how many pages you have indexed in Google with the Google Indexed Pages Checker. What Are Indexed Pages? Google is constantly scouring the Web for new content, new websites, and new pages on websites it already knows about. For each site, Google indexes each page that it crawls for future reference, and to be included in search results. In order for a page to be listed in Google search results, it must first be indexed! When Google's bots crawl a website they create a cached copy of each page, and then adjust their indexes. It is of course vitally important to have all of the content on your website indexed by Google in order to be included in their search results. This tool will tell you exactly how many pages Google currently has indexed for your website. You can check a full domain or subdomain, and see how many pages have been included by Google. If you have a large number of pages on your website, but Google is only reporting a few of them, it could signify a problem with your site that needs to be addresses. Ensuring that all of your content is properly crawled and indexed is absolutely vital to be listed in search results.

MSN Live Indexed Pages Checker
The MSN Live Index Pages Checker scans MSN Live Search for a website's indexed pages. What Are Indexed Pages? Indexed pages are pages from a website that have been crawled by search bots and added to a search engine's database of websites. In order to appear in search results on any search engine, each page must be included in their index of pages. Search bots are constantly crawling the Internet seeking new content and refreshing their cache of existing content. An index is basically a long list of individual pages for each domain or subdomain, and is used as a reference for keeping track of all the content in a search engine's database. A new website will sometimes take a while to get all of the content indexed, while an authority website might have new pages indexed daily or even hourly! What is MSN Live Search? MSN Live Search was Microsoft's official search engine for many years. Since 2009 MSN Live Search has been replaced by Bing, and MSN Live indexes have been officially taken over by Bing search. Since Bing has now taken over MSN Live Search, the MSN Live Indexed Pages Checker tool now gets its results directly from Bing.

Robots.txt Checker
Check any website for a robots.txt file and see what it contains. What is robots.txt? The robots.txt file is a plain text file that is placed on a website containing instructions for search engine bots. It is a loose standard with a specific format called the Robots Exclusion Protocol. When a search engine sends their bots crawling your website, they will check this file to determine any limitations or rules that the webmaster has set for the search engine. For example, a webmaster can disallow a search engine from crawling part of their website explicitly, or disallow all crawling of a website unless specifically allowed by the rules defined in robots.txt. How Reliable is robots.txt? Using the robots.txt file to provide instructions to search bots is in no sense a highly secure method of protecting content from being indexed. While it is generally understood that a well-behaving search bot should follow the rules in the robots.txt file, there is nothing in this protocol that actually prevents them from crawling your entire site. Robots.txt is not a form of security or authentication for your website, and some spider bots don't explicitly follow rules placed in this file. However, for webmasters that wish to leave content on their site without authentication, but do not want it to be indexed, robots.txt is the most suitable method of providing search engines with instructions for what to avoid.

Search Engine Listing Preview
Preview your website in search engines with the Search Engine Listing Preview tool. What is Included in a Search Engine Listing? Search results in the major search engines like Google, Yahoo and Bing, all tend to follow a similar format. After a search engine has crawled your website and extracted the title of each page, the content of the page, the URL, etc. it can include your website in search results. Search engines usually use the page title as the hyperlink for your website in their results. This is also a predominant factor in what users will check when looking through the results of a search. If your titles are clean and direct they may even get better results. Google and other search providers are including more and more customized information in their results, as well as personalized results, thumbnail images, extra links and even previews of a web site. But most results still follow the same traditional search listing format. How do I Preview a Listing? Enter the details of an example website (or copy them from your own) to get a preview of what your page is going to look like in search engine results. This tool uses Google as a standard model for the results.

Search Engine Saturation Checker
Check how your website is indexed in the major search engines with our Search Engine Saturation Checker. What is Search Engine Saturation? Search engine saturation describes how well your website has been "absorbed" by search engines into their site index. As search bots crawl your website and build up their index and cache, new pages of your website are included as they are crawled and verified by the engine's algorithms. The more pages that are indexed on your website, the more your content is included in searches, and the better chances that your site will come up in results. How do I Improve Search Engine Saturation? Content, content, content! Adding new content to your website is an essential property of expanding your presence in search engines. Indexed pages correspond directly to how much content is available on your website. Indexed pages do not accurately describe the quality or depth of the content on a website, it is simply a measure of how many pages have been included in a search engine's index. Some sites have a large number of pages but very little unique content for each page. On the other hand some sites have both, for example Wikipedia. For SEO, improving search saturation is important, but more important is focusing on the quality of content - not just quantity.

Spider Viewer
View your website like a search engine spider! The Spider Viewer tool is a quick and simple way to look at your website through the eyes of a search engine spider. What is a Search Engine Spider? Spiders, also called search bots, bots or spider bots, are software used by search engines to "crawl" a website and all of its content. A spider will fetch each page and follow the links from that page recursively, building up an index of pages which the search engine can then use to build its database. A spider bot doesn't view websites in a browser like we do. Instead, it fetches the raw HTML code for the site and then strips it down further into just the text and content. The spider viewer tool emulates what a search spider might do when viewing your page. Instead of rendering all the HTML code from a site, it strips out the code and images and provides a view of any web page in plain text. Search engine spiders all function somewhat differently, and they often view META tag information and how HTML tags are used. For a detailed view of information a spider bot can obtain you can also use our Website spider tool.

Website Spider Tool
Check details of your site as a search engine spider would! Our website spider tool crawls any URL and fetches details that a spider bot might use when crawling a page. Enter an URL and our tool will crawl the specified site, extracting details that a search engine spider would see. Get a report of the META tags and site content including the page title, META description, META keywords, the full source code and the full character length of the website's code. The spider viewer tool makes it simple to identify missing META tags, check on keywords, review the title and description of a page, or determine how long the source code is. It lets you analyze a web page much like a spider would, by extracting details from the HTML instead of being rendered in a browser. What is a Search Engine Spider? Spiders are software that search engines use to "crawl" web pages and retrieve the content, size, and other details for their website indexes. A search spider looks at the raw HTML and text of a web page instead of the final (rendered) version that we see with a browser. Search spiders will also look at META tags for a site, like the title and description text.

Yahoo Indexed Pages Checker
Find out how many pages are indexed in Yahoo for your website with the Yahoo Indexed Pages Checker. Indexed pages describe how many pages a search engine has included in their database for a website. As Yahoo spider bots crawl the Web they follow hyperlinks that show up - both to new domains and new pages on existing domains. As you add new articles, content, or blog posts to your website search engines should be able to pick up this new content quickly. An established website that is already included in Yahoo's site database might get checked regularly, and new articles added to the index within weeks, days, or even hours! A brand new website sometimes takes a while to get fully indexed, but if the site is never fully indexed in Yahoo, or it's taking many months, it could signify a problem with your website. Making sure Yahoo can regularly index your site and include new content in their database requires some simple, basic SEO such as properly formatted HTML, a server with reasonable load times, and preferably not a lot of links to "bad neighborhoods" - unsavory or spammy websites that might be avoided by Yahoo. As your website expands, you can keep track of how many pages are being indexed in Yahoo with a quick look in our Yahoo Indexed Pages Checker tool.