Welcome to the Q&A Forum

Browse the forum for helpful insights and fresh discussions about all things SEO.

Your browser does not seem to support JavaScript. As a result, your viewing experience will be diminished, and you have been placed in read-only mode.

Please download a browser that supports JavaScript, or enable it if it's disabled (i.e. NoScript).

Unsolved What would the exact text be for robots.txt to stop Moz crawling a subdomain?

Getting Started

2

2

209

Log in to reply

Simon-Plan Subscriber last edited by

I need Moz to stop crawling a subdomain of my site, and am just checking what the exact text should be in the file to do this.

I assume it would be:

User-agent: Moz
Disallow: /

But just checking so I can tell the agency who will apply it, to avoid paying for their time with the incorrect text!

Many thanks.
1 Reply Last reply
Reply Quote 0
Peter206 @Simon-Plan last edited by

@Simon-Plan No, when you put just slash / you will disallow everything.
Instead you need to put /foo/ where foo is your subdomain. Please see here for a reference to some relevant examples: https://searchfacts.com/robots-txt-allow-disallow-all/
1 Reply Last reply
Reply Quote 0

Got a burning SEO question?

Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.

Start my free trial

Browse Questions

View

From

Sorted by

With category

Explore more categories

Related Questions

Unsolved Can I exclude subdomain from a Campaign?

campaign settings subdomain

Also, I would like to track a subdomain without the root domain, is that possible?
Moz Pro | | AndreVa

0
Unsolved Is Performance Metrics only available in a Campaign?

crawl performance

I'm looking to do a 1-off Performance Metrics analysis across dozens of pages on a single website - a prospective client. I thought it would be part of the On-Demand Crawl.
Moz Tools | | amandacash85896

0
How long does it take, So i can see results of Kewords Ranking search on moz?

Hi, I m the new guy at the Neighborhood Can you help please with the question... how long does it take, So i can see results of Kewords Ranking search on moz? This is the popup note Message: Moz collects and updates campaign data weekly. Not collected means we have not yet updated data for this keyword. TNX
Getting Started | | YarivHPAZ

1
Moz for charities and non-profits

Hi, I noticed an earlier thread on this topic that now seems dead so I decided to start a new one: https://moz.com/community/q/non-profit-rates We're a non-profit organisation based outside of the US and we were wondering if MOZ's non-profit program can be extended out? We're fully registered and audited in our home country and can provide all the necessary documentary proof if needed.
Getting Started | | OBW_Noel

1
Why is analytics.moz.com showing up a traffic source in Google Analytics?

Under Traffic Sources, analytics.moz.com is listed as a source for 37 sessions this past month. What might this be? Is it legitimate traffic or should I be filtering this?
Getting Started | | mphdavidson

0
MOZ Guru Needed for Freelance Work

Hello -- I'm new to Moz and wanted to know if any community members were available for freelance work to help me fine-tune the setup of the Moz account for one of my clients. This could lead to additional work for other clients of my advertising agency.
Getting Started | | KRT

0
New to MOZ and working with Web Mentions. Can I use operators?

Our name is HostDime but often put as Host Dime (2 words) by news sources and other sites. How do I set up my brand mention so I only get a notice when both words appear, in order, together. I don't want "That host is a dime" and such. Can I use a +Host +Dime?"Host Dime"? Do these operators work in MOZ?
Getting Started | | hostdime

0
How do get Moz to spider a Development site PRE LAUNCH?

Hi, Does anyone know how we could get Moz to browse a development site before launch? But without Google and other engines indexing it? Thanks
Getting Started | | bjs2010

0