Search engines can’t show people your website if they don’t know that it exists.
If you want your website to show up on Google and other search engines, you need to get it indexed first. The best way to do that is by submitting your site to them directly. Telling Google and Bing where to find your website just takes a few steps, and it allows you to compete for valuable organic traffic.
Read on to see how you can get Google to index your site and what to do if it still doesn’t show up in search after you submit it.
Why Do I Need to Submit My Website to Search Engines?
Websites don’t just appear on search engines like Google and Bing.
Search engines use special programs, known as bots or web crawlers, to explore the internet. These bots compile lists of all the sites they find and place them in search engines’ indices. They crawl the internet using links from pages that are already in the index to discover and catalog new ones.
But the bots may never find your site if it doesn’t have any links from pages in the index.
Websites and pages that aren’t in Google’s index will not appear in the search results. That’s a massive missed opportunity for bringing potential clients to your website.
The good news is that search engines know that this is a problem. They allow website owners to manually submit their websites and request indexing. All you need to do is tell Google that your site exists, and it will send out the bots to crawl your site.
Besides the fact that a manual submission adds your site to Google’s index, there are a few other reasons to do it.
First, it’s easy.
The process of submitting a site to a search engine is simple. And the benefits of being in the index and appearing in search far outweigh waiting for bots to find your site by chance.
Second, it speeds up the process of getting your site ranking for important keywords.
Bots might not discover everything on your site on their own. When you manually submit your website, it becomes easier for major platforms to start cataloging the pages on your site and pairing them with keywords. Your submission also helps search engines with details about your site that they may not pick up otherwise, like the importance of each page.
Third, you get more useful data by being proactive.
You have to use free tools like Google Search Console and Bing Webmaster Tools to submit your site. Both of these platforms have useful dashboards that provide you with data on your site’s performance in search results.
These reasons aren’t to say that submitting your site is an SEO silver bullet.
Your website isn’t going to rank for every keyword you want the minute it’s indexed. It takes time for most sites to appear on the first page for primary search terms. And it can take months to reach the first page in intensely competitive niches, like SEO for lawyers.
But indexing your is the first step to reaping the benefits of ranking in search.
If you’ve never requested indexing for a website before, the steps below will walk you through the process.
1. Find Your Website’s Sitemap
The first step in getting search engines to crawl your website is telling them that you exist. You can do this by submitting a special file, called a sitemap, to them. When you submit this file to Google and Bing, they’ll begin their own crawling and indexing processes.
If you already know where your sitemap is, you can copy its URL and move on to the next steps.
If you don’t know if you have a sitemap or you don’t know where it is, the methods below will walk you through what you need to do to find it.
What is an XML Sitemap?
An XML sitemap is a file hosted on your website that lists all of your live page URLs.
You can think of it as a roadmap for search engines to use when they come to your website. They’re written in a format (eXtensible Markup Language) that’s easy for web crawlers to read and understand. Your sitemap gives crawlers a starting point for navigating and indexing your site instead of crawling random links.
There are three different methods you can use to find your sitemap.
Method 1: Check yourdomain.com/sitemap.xml
The simplest method is to look in the most obvious place.
Most websites have their sitemap on a URL that follows the root domain. All that means is if your site is example.com, then your sitemap is most likely located at example.com/sitemap.xml.
Here’s how to check if that’s where your sitemap is located:
If you don’t find your sitemap using this method, it doesn’t mean it doesn’t exist. It could be on an unusual URL.
Method 2: Check Your robots.txt File
Robots.txt is another file on your website that gives instructions to web crawlers. Instead of being a roadmap, though, these files tell crawlers what pages they should and should not visit.
Many robots.txt files will list the sitemap in addition to setting boundaries for crawlers. Here’s how you can check yours:
Robots.txt files don’t always list the location of a sitemap. If you don’t see your sitemap in this file, there’s another method you can use.
Method 3: Use Advanced Search Operators
If the last two methods didn’t turn up your sitemap, it might be because it doesn’t follow the standard naming convention.
Google’s advanced search operators may be able to help you find sitemaps with unconventional names. The walkthrough below demonstrates how to use this method.
This advanced search operator below tells Google to show you all pages on example.com with the word “sitemap” in the URL.
site:example.com inurl:sitemapIf that doesn’t work, “sitemap” may not be in the filename at all. You can use the following advanced search operator to tell Google to show you all of the XML files on your site instead.
site:example.com ext:xmlYour website may not have a sitemap if you cannot find it through any of these three methods. You’ll need to create a new sitemap before you can move on to the next steps.
If your website is on WordPress, there are plenty of plugin options available that can generate sitemaps for you. Some options include SEOPress, Rank Math, or Yoast SEO.
Other content management systems, like Webflow, have built-in sitemap generators.
Finding your sitemap is usually the hardest part of requesting indexing. Once you’ve found your sitemap, submitting it to the major search engines is a breeze.
2. Submit Your Website to Google
You’ll need to use Google Search Console to request a site crawl from Google.
Go to your website’s Google Search Console property and then select the Sitemaps menu from the sidebar. Then enter your sitemap to request indexing. Use the walkthrough below to see how this process works.
If you haven’t set up Google Search Console for your website, you’ll need to create an account first. You’ll need to log into the tool here with a Gmail account. Then you’ll need to verify that you own the website and set up a new property following this guide from Google.
How to Submit Multiple Sitemaps to Google
Some websites will have more than one sitemap. For example, a lawyer may have separate sitemaps for their blog posts and practice area pages.
Google Search Console makes submitting multiple sitemaps easy. All you need to do is follow the process outlined in the last walkthrough for each of your sitemap URLs.
How to Submit a Single Web Page to Google
Google will schedule your site for a crawl as soon as you’ve submitted the sitemap to Search Console. It will also schedule your site for crawling at regular intervals.
But if you publish a new page or post between crawls, you’ll want to make sure it lands in Google’s index too.
You can manually submit new pages through Google Search Console by entering the single URLs into the search bar at the top of the tool. Here’s a walkthrough for requesting a crawl for a single page:
Submitting URLs this way is perfect when you have one or two new pages that you want Google to index. It’s not a solution for getting your whole site indexed, though.
It’s just a matter of time until Google can crawl and index your site after you submit your sitemap through Search Console. In the meantime, you can request a crawl from other search engines.
3. Submit Your Website to Bing
Submitting a website to Bing is a lot like submitting a sitemap to Google. You’ll just need to use Bing Webmaster Tools instead of Google Search Console.
If you already have a Bing Webmaster Tools account and you’ve verified that you own the website, you can:
- Log into Bing Webmaster Tools
- Select the Sitemaps option from the lefthand sidebar
- Click on the Submit Sitemap button
- Enter your sitemap URL into the popup dialog box that appears on your screen
- Click Submit
You’ll need to repeat this process for each of your sitemaps. Bing will then schedule a crawl of your website.
4. Submit Your Website to Yahoo
Submitting to Yahoo is the easiest of them all because there’s nothing for you to do.
Bing is the search engine that powers Yahoo. Anything that Bing indexes will show up in Yahoo search results.
How to Check if Google Has Indexed Your Website
Advanced search operators can show you if Google has indexed a new page on your site.
All you need to do is copy the URL that you want to check and use the site: search operator in Google. Here’s a walkthrough with an example:
It can take anywhere from a few minutes to several days for Google to index a new page. Don’t worry if the advanced search doesn’t turn anything up right away. You can check back with the same method in a few days to see if the page has been indexed yet.
Note: the site: search operator works on Bing, too. You can use the same method to see if your page is in Microsoft’s index.
You can also check in Google Search Console to see all the pages on your site that it discovered while crawling.
What’s useful here is that the tool can show you both indexed pages and all the pages Google couldn’t index. Follow this walkthrough to see which pages are indexed in Google Serch Console:
Search Console’s page indexing report is a helpful tool for finding potential indexing issues with your site in between any technical SEO audits. But if you see a single page or set of pages that Google can’t index, you may need to do a deep dive.
What to Check if Your Website Isn’t Indexed
There are three common issues to check for if Google can’t index your website or a page on your website.
Some Web Pages are Flagged as noindex
Google will not index pages that have the noindex meta tag in their HTML code.
This is a useful tag if you don’t want a page to show up on Google. For example, an appointment confirmation page that shows up after a visitor submits a free consultation form doesn’t need to be in the search results.
But these tags are a real problem if you want a page to appear in search.
The easiest way to check for this issue on the page level is to use DevTools in the Chrome browser. Go to the page that you want to inspect, and then follow the walkthrough below.
You’ll need to remove the noindex tag from the header of each page with this issue. Then you can go to Google Search Console to request indexing for each one.
Your robots.txt File Is Blocking Search Engines From the Site
If you see that your entire site, or large sections of it, can’t be indexed, you may have an issue with your robots.txt file.
Go to your robots.txt file and check that your site isn’t blocking Googlebot. Robots.txt instructions that block Googlebot look like this:
User-agent: Googlebot
Disallow: /The code above tells Google that it should not crawl any of the subfolders on your site.
You’ll need to go to your SEO plugin (Yoast, SEOPress, Rank Math, etc.) to remove this from your robots.txt file if you’re on WordPress. If you’re on Webflow, the options to change this are under the SEO tab.
Then you can resubmit your sitemap to Google Search Console for indexing.
Your .htaccess File Is Blocking Search Engines From the Site
Sites that run on an Apache-based server have a file called .htaccess that gives web admins control over various server settings. An issue with this file can cause indexing issues. You’ll need access to the server your website lives on to check and adjust this file.
If you can get to the .htaccess file, look for the following code:
Header set X-Robots-Tag "noindex, nofollow"Removing the noindex and nofollow parts will allow Google and other search engines to access your site.
An issue with your .htaccess file is the most technical of the three common indexing issues. If you feel that it’s outside of your skillset, it may be time to contact someone with a technical SEO background.
What if Your Website Is Indexed but Isn’t Ranking?
Just because your site is in Google’s index does not mean it will rank.
Getting your site indexed is important. Making sure that Google can index your pages is a prerequisite for appearing in the search results. These are the table stakes to getting organic search rankings.
But ranking well on Google takes more than just indexing.
One of the biggest ranking factors is whether your website content matches the searcher’s intent. Google’s goal is to provide people with results that answer their questions and satisfy their needs.
You stand a better chance of ranking when you create content that matches the intent for each keyword you target. Performing keyword research can help you find high value keywords to create content around. It can also point you in the right direction for each keyword’s intent.
Understanding on page optimization can help you take the content you already have and improve it for people and search engines.
Finally, link building activities can help your site attract links from relevant, authoritative sources. Google sees links as endorsements of your site. Authority from relevant sites passes to your own and can improve your rankings.
Wrapping Up
Getting your website indexed by search engines is vital if you want to grow your organic traffic.
Indexing is the first step to getting your website in front of potential clients. If your website isn’t in Google’s index, people will not find you in the search results.
The best way to get your site indexed is by manually submitting your site to search engines.
But getting your site indexed does not mean you’ll start ranking for keywords right away. Getting your website indexed is your ticket to enter the race. Once in the race, continuous SEO efforts can help you win it.
To learn more about what your law firm can do to win that race, check out our guide to SEO for lawyers.
The post How to Get Google to Index Your Website appeared first on Rankings.
