• About
    • History of Dallas SEO
    • SEO Expert Witness
  • Contact
  • Topics
    • Bing
    • Blogging
    • Branding
    • Domain Names
    • Google
    • Internet Marketing
    • Link Building
    • Local Search
    • Marketing
    • Public Relations
    • Reputation Management
    • Search Engine Marketing
    • Search Engine Optimization
    • Search Engines
    • Social Media
    • Tech
  • Advertise
  • Email Newsletter

Bill Hartzer

Bill Hartzer on Search, Marketing, Tech, and Domains.

SEMrush

Home » Google » Disallow Search Engines in Robots.txt File? No Worries on Google

Disallow Search Engines in Robots.txt File? No Worries on Google

Posted on January 3, 2014 Written by Bill Hartzer

Here’s a first. Well, actually I’m not surprised when it comes to Google, but this is the first time I have seen a website disallow the search engines from indexing their website. But the website still ranks number one in Google for their primary keyword phrase.

Take a look at this screen capture, for a search query “Dallas Dental Clinics”. Granted it doesn’t have a huge search volume every month, but it is a major city (Dallas, Texas) and there are a lot of dental clinics competing in the market.

dallas dental clinics search

Yep, leave it up to Google to give you a number one ranking even if you purposely are stopping the search engines from indexing your website. Google doesn’t seem to care, they will rank your website number one in the search results whether you like it or not.

I have seen all sorts of crazy search results in the past, but this one certainly takes the cake for me. A website has a robots.txt file directive for the search engines to NOT index their website. And Google actually shows that they know about that directive NOT to index the website.

Let’s take a look at the robots.txt file, here is what it says:
User-agent: *
Disallow: /

Even though this says to the search engines: DO NOT index my website, Google doesn’t care, they index the website, and rank it number one in the search results.

Makes sense to me, what about you?!?

Filed Under: Google, Search Engines

SEMrush

About Bill Hartzer

Bill Hartzer is CEO of Hartzer Consulting, LLC, an SEO Consulting firm that includes services such as search engine optimization, technical SEO audits, domain name consulting, and online reputation management.

Recent Posts

  • Someone Stole My Domain Name: Here’s What You Do January 4, 2023
  • Web Hosting Services Market to Grow to $254.86 Billion by 2029 December 13, 2022
  • This SEO Blog Post Was Written by ChatGPT December 8, 2022
  • Facebook Rolling Out Facebook Articles December 7, 2022
  • Doing SEO is Better Than… December 6, 2022
  • Tucows and GoDaddy Report Q3 2022 Results November 6, 2022
  • How to Measure App Events Sourced by Organic Search and SEO September 20, 2022
  • Google Allegedly Eavesdrops and Monitors the Brain 24 hours a Day to Control Humanity September 14, 2022
  • Why You Shouldn’t Hire SEOs Based on An Email September 13, 2022
  • Global SEO Market to Reach $122.11 Billion by 2028 September 9, 2022
  • Bluehost Launches New Commerce Solutions for WordPress September 8, 2022
  • Which CMS? How to Choose the Best CMS for Your Purposes August 29, 2022
  • Accidental SEO Manager: Interview with Ash Nallawalla August 15, 2022
  • Sometimes Google Isn’t Family Friendly August 1, 2022
  • Something’s Seriously Wrong with Facebook Notifications July 12, 2022
  • Facebook Internet Tracking Settlement June 24, 2022
  • RankSense Acquired by SEOClarity June 1, 2022
  • LinkedIn Links, Digital Marketing News, and SEO Questions Answered May 9, 2022
  • GoDaddy Ending Forwarding of Existing Shortened Links May 5, 2022
  • 7 Ways to Protect Your Domain Name March 31, 2022

US Agency Awards Judge

DFWSEM logo

Bill Hartzer is a Brand Ambassador for:



Industry Friends

I Love SEO
WTFSEO
SEO By the Sea
Jeff Lenney
Jeff Gabriel
Phil Drinkwater
Dixon Jones
Brian Hartzer
Navah Hopkins

Connect With Bill Hartzer

Bill Hartzer on Twitter
Bill Hartzer on Instagram
Hartzer Consulting on Facebook
Bill Hartzer on Facebook
Bill Hartzer on YouTube

Categories

  • Advertising (19)
  • Bing Search Engine (6)
  • Blogging (42)
  • Branding (12)
  • Domain Names (209)
  • Google (236)
  • Internet Marketing (25)
  • Internet Usage (85)
  • Link Building (53)
  • Local Search (39)
  • Marketing (180)
  • Marketing Foo (30)
  • Pay Per Click (3)
  • Podcast (18)
  • Public Relations (8)
  • Reputation Management (9)
  • Search Engine Marketing (44)
  • Search Engine Marketing Events (48)
  • Search Engine Marketing Firms (19)
  • Search Engine Marketing Jobs (33)
  • Search Engine Optimization (164)
  • Search Engines (204)
  • Social Media (192)
  • Tech (7)
  • Web Analytics (17)
  • Webinars (1)

Note: All product names, logos, and brands are property of their respective owners. All company, product and service names used in this website are for identification purposes only, and are mentioned only to help my readers. All other trademarks cited herein are the property of their respective owners. Use of these names, logos, and brands does not imply endorsement.




Hartzer Consulting



Website, Content, and Marketing by Hartzer Consulting, LLC.

Copyright © 2023 ·