Possible to edit the robots.txt

Administrators
1

Hi,

is it possible to edit the robots.txt ?
I try to create extendet version vi in my CUSTOMFILES/templates/ or CUSTOMFILES/static/ folder but it didn`t get loaded after a restart of the services.

2

Simply modify the alias in your webserver config and point it to a custom robots.txt file instead of the default one. Can be anywhere your webserver can read it.

3

No success on centos 7 and apache (2.2) config of indico

Alias /robots.txt /opt/indico/CUSTOMDIR/robots.txt

<Location "/robots.txt">
SetHandler None
Require all granted
</Location>

still get ignored and load the default robot.txt of indico…

4

Did you reload/restart Apache? Also, did you remove the existing alias from the config?

Alias /robots.txt /opt/indico/web/htdocs/robots.txt
5

Jep restarted the services, i modified the indico.cfg in /etc/httpd/conf.d/

Alias /robots.txt /opt/indico/CUSTOMDESIGN/robots.txt

<Location "/robots.txt">
SetHandler None
Require all granted
</Location>
6

Can you show the full indico.cfg?

7 
<VirtualHost *:443>
    ServerName HOSTNAME
    DocumentRoot "/var/empty/apache"

    SSLEngine               on
    SSLCertificateFile      /etc/ssl/indico/indico.crt
    SSLCertificateChainFile /etc/ssl/indico/indico.crt
    SSLCertificateKeyFile   /etc/ssl/indico/indico.key
    SSLProtocol             all -SSLv2 -SSLv3
    SSLCipherSuite          ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA:ECDHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES256-SHA:ECDHE-ECDSA-DES-CBC3-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:DES-CBC3-SHA:!DSS
    SSLHonorCipherOrder     on

    XSendFile on
    XSendFilePath /opt/indico
    CustomLog /opt/indico/log/apache/access.log combined
    ErrorLog /opt/indico/log/apache/error.log
    LogLevel error
    ServerSignature Off

    AliasMatch "^/static/assets/(core|(?:plugin|theme)-[^/]+)/(.*)$" "/opt/indico/assets/$1/$2"
    AliasMatch "^/(css|images|js|static(?!/plugins|/assets|/themes|/custom))(/.*)$" "/opt/indico/web/htdocs/$1$2"
    Alias /robots.txt /opt/indico/CUSTOMDESIGN/robots.txt

    <Location "/robots.txt">
    SetHandler None
    Require all granted
    </Location>

    SetEnv UWSGI_SCHEME https
    ProxyPass / uwsgi://127.0.0.1:8008/

    <Directory /opt/indico>
        AllowOverride None
        Require all granted
    </Directory>

</VirtualHost>