Uploaded image for project: 'Infrastructure'
  1. Infrastructure
  2. INFRA-884

robots.txt file is not allowed to be read - 403 Forbidden

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Resolved (View Workflow)
    • Priority: Minor
    • Resolution: Fixed
    • Component/s: wiki, www
    • Labels:
      None
    • Similar Issues:

      Description

      Use case:
      We have a website which includes link to various plugins that we want to check that the deep links are valid and the linkchecker respects the robots.txt feature ( http://www.robotstxt.org/robotstxt.html ). Unfortunately the robots.txt is not allowed to be read on your webserver.

      The error:
      {{
      :~/test$ wget https://wiki.jenkins-ci.org/robots.txt
      -2016-06-30 15:29:53- https://wiki.jenkins-ci.org/robots.txt
      Resolving wiki.jenkins-ci.org (wiki.jenkins-ci.org)... 140.211.9.32
      Connecting to wiki.jenkins-ci.org (wiki.jenkins-ci.org)|140.211.9.32|:443... connected.
      HTTP request sent, awaiting response... 403 Forbidden
      2016-06-30 15:29:54 ERROR 403: Forbidden.
      }}

      Is it possible you could make it readable?

        Attachments

          Activity

          Hide
          rtyler R. Tyler Croy added a comment -

          Hrm, I thought I had fixed that a while back, I'll take a looksee

          Show
          rtyler R. Tyler Croy added a comment - Hrm, I thought I had fixed that a while back, I'll take a looksee
          Hide
          markewaite Mark Waite added a comment -

          This doesn't seem like a newbie-friendly bug, since it requires administrator access to the Confluence server, at least per https://community.atlassian.com/t5/Jira-questions/Where-to-put-the-robots-txt-file-on-standalone-confluence-or/qaq-p/141572

          Show
          markewaite Mark Waite added a comment - This doesn't seem like a newbie-friendly bug, since it requires administrator access to the Confluence server, at least per https://community.atlassian.com/t5/Jira-questions/Where-to-put-the-robots-txt-file-on-standalone-confluence-or/qaq-p/141572
          Hide
          olblak Olivier Vernin added a comment -

          Mark Waite you are totally right, initially I though the only thing we had to do was adding that file to the docker image located here then I realized that file robots.txt was already create by puppet and the only thing I had to do was to chmod /srv/wiki which required root access

          Show
          olblak Olivier Vernin added a comment - Mark Waite you are totally right, initially I though the only thing we had to do was adding that file to the docker image located here then I realized that file robots.txt was already create by puppet and the only thing I had to do was to chmod /srv/wiki which required root access

            People

            Assignee:
            olblak Olivier Vernin
            Reporter:
            bicschneider Claus Schneider
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved: