Easily manage X-Robots-Tag http header (noindex, nofollow) in Symfony2.

• lets you define a default value for the X-Robots-Tag response header
• lets you define the value for the X-Robots-Tag response header for requests that require certain user roles
• lets you manually control the value for the X-Robots-Tag response header

Open a command console, enter your project directory and execute the following command to download the latest stable version of this bundle:

$ composer require locaine/lcn-x-robots-tag-bundle "~1"

This command requires you to have Composer installed globally, as explained in the installation chapter of the Composer documentation.

Then, enable the bundle by adding it to the list of registered bundles in the app/AppKernel.php file of your project:

<?php
// app/AppKernel.php

// ...
class AppKernel extends Kernel
{
    public function registerBundles()
    {
        $bundles = array(
            // ...

            new Lcn\XRobotsTagBundle\LcnXRobotsTagBundle(),
        );

        // ...
    }

    // ...
}

Then, configure the bundle in app/config.yml file of your project:

# app/config.yml
parameters:
    lcn_x_robots_tag.rules.default: { noindex: false, nofollow: false }
    
...

lcn_x_robots_tag:
    enabled: true
    rules:
        default: %lcn_x_robots_tag.rules.default%

Imagine you have a product listing page where you want search engine crawlers to follow the links to the products but not to index the listing page itself (e.g. to avoid duplicate content):

class ProductController

    public function indexAction(Request $request)
    {
        $this->get('lcn_x_robots_tag')->setNoindex(true)->setNofollow(false);
        
        ...

Calling setNoindex or setNofollow on the XRobotsTag service overrides all other rules defined in your app/config.yml file.

If you have user roles and access control rules defined in app/security.yml then you can easily tell search engine crawlers not to index those requests. This is useful if your visitors "login" using a token (see Api Key Authenticator) or when Http Basic Auth user credentials are provided in urls. If you are sending 403/401 Status headers or if you are redirecting unauthenticated users to you login page, this might be less useful.

# app/config.yml
lcn_x_robots_tag:
    enabled: true
    rules:
        user_roles: true

The above syntax is shorthand notation for:

# app/config.yml
lcn_x_robots_tag:
    enabled: true
    rules:
        user_roles:
            *: { noindex: true, nofollow: true }

You can also apply the rules only for certain user roles:

# app/config.yml
lcn_x_robots_tag:
    enabled: true
    rules:
        user_roles:
            ROLE_ADMIN: { noindex: true, nofollow: true }
            ROLE_EDITOR: { noindex: true, nofollow: true }

Of course, your dev environment should not be publicly accessible, but if it is, you can at least avoid that it gets indexed:

# app/config_dev.yml
imports:
    - { resource: config.yml }

parameters:
    lcn_x_robots_tag.rules.default: { noindex: true, nofollow: true }