Home

Disallow search engine access your sub-domain pages

Big problem with SEO one my previous article: There are maybe many URL for one page, that is so bad for SEO. To fix that problem, we should disallow search engines to access your sub-domain pages. How to do that?

Create file robots-subdomain.txt contains:

  1. # This example keeps all robots out
  2. User-agent: *
  3. Disallow: /

Open your .htaccess file add these lines, when search engines request robots.txt from sub-domain, we redirect it to robots-subdomain.txt

  1. RewriteEngine on
  2.  
  3. RewriteCond %{REQUEST_URI} /robots.txt
  4. RewriteCond %{HTTP_HOST} !^www.your-domain\.com$ [NC]
  5. RewriteCond %{HTTP_HOST} ^(.+)\.your-domain\.com$ [NC]
  6. RewriteRule ^(.*)$ robots-subdomain.txt [L]

Comments

Post new comment

The content of this field is kept private and will not be shown publicly.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Allowed HTML tags: <em> <strong> <cite> <code> <ul> <ol> <li> <dl> <dt> <dd> <sup> <br> <p> <h3>
  • Lines and paragraphs break automatically.
  • Link to content with [[some text]], where "some text" is the title of existing content or the title of a new piece of content to create. You can also link text to a different title by using [[link to this title|show this text]]. Link to outside URLs with [[http://www.example.com|some text]], or even [[http://www.example.com]].
  • You can enable syntax highlighting of source code with the following tags: <code>, <blockcode>. Beside the tag style "<foo>" it is also possible to use "[foo]". PHP source code can also be enclosed in <?php ... ?> or <% ... %>.

More information about formatting options

CAPTCHA
This question is used to make sure you are a human visitor and to prevent spam submissions.
Image CAPTCHA
Copy the characters (respecting upper/lower case) from the image.