Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-35666

Regular 404s for running builds that get better after build is finished.

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • None

      For some large (50%?) number of builds (across several projects):

      • The build starts.

      As expected:

      After running for several minutes:

      However:

      • the list of workers still shows the build as occupying the worker (and provides a link which goes to the now 404-ing build page)
      • the build runs to completion successfully.
      • Build-related URLs for other builds still work ok at this point.

      After the build has run to completion, all of the build related URLs which started 404ing work again (for example the complete console output can be seen).

      A screenshot of progressive console output breaking part way due to this is attached - to be clear, this happens for all URLs related to the build, not just the progressive console.

      This has happened in all of the weekly releases that I've tried (three or four over the past two months); previous to that I was not using Jenkins on this codebase/machine.

          [JENKINS-35666] Regular 404s for running builds that get better after build is finished.

          Ben Clifford created issue -

          Daniel Beck added a comment -

          Which is it in your issue description: Job/Project, or Run/Build?

          Daniel Beck added a comment - Which is it in your issue description: Job/Project, or Run/Build?

          Ben Clifford added a comment -

          I don't understand the question.

          Ben Clifford added a comment - I don't understand the question.

          Daniel Beck added a comment -

          Project/Job is the thing you configure. Run/Build is the thing that has a start time and duration. I'm pretty sure at least some uses of "job" in the issue description should be "build", but it's not clear whether it's all of them.

          Daniel Beck added a comment - Project/Job is the thing you configure. Run/Build is the thing that has a start time and duration. I'm pretty sure at least some uses of "job" in the issue description should be "build", but it's not clear whether it's all of them.
          Ben Clifford made changes -
          Description Original: For some large (50%?) number of build jobs:
           * The job starts.
           * Job related URLs work ok. (for example, http://loris.beautifuldestinations.com:2000/job/beautilytics/branch/benc-typo/1/console/job/beautilytics/branch/benc-typo/1/console provides a working progressive console.
           * The job appears in the LHS "build history" pane on the project page, as an in-progress job.
           * The job appears on the list of workers, as occupying a worker.
           * The job is correctly linked on github as an in-progress job.

          After running for several minutes, all job related URLs start returning 404, but the list of workers still shows the job as occupying the worker, and the job runs to completion successfully.

          After the job has run to completion, all of the job related URLs which started 404ing work again (for example the complete console output can be seen).

          A screenshot of progressive console output breaking part way due to this is attached - to be clear, this happens for all URLs related to the job, not just the progressive console.

          This has happened in all of the weekly releases that I've tried (three or four over the past two months); previous to that I was not using Jenkins on this project.
          New: For some large (50%?) number of builds (across several projects):
           * The build starts.

          As expected:
           * build related URLs work ok. (for example, http://loris.beautifuldestinations.com:2000/job/beautilytics/branch/benc-typo/1/console provides a working progressive console.
           * The build appears in the LHS "build history" pane on the project page, as an in-progress build.
           * The build appears on the list of workers, as occupying a worker.
           * The build is correctly linked on github as an in-progress build.

          After running for several minutes, all build related URLs for that particular build start returning 404, but the list of workers still shows the build as occupying the worker (and provides a link which goes to the now 404-ing build page), and the build runs to completion successfully. Build-related URLs for other builds still work ok at this point.

          After the build has run to completion, all of the build related URLs which started 404ing work again (for example the complete console output can be seen).

          A screenshot of progressive console output breaking part way due to this is attached - to be clear, this happens for all URLs related to the build, not just the progressive console.

          This has happened in all of the weekly releases that I've tried (three or four over the past two months); previous to that I was not using Jenkins on this codebase/machine.

          Ben Clifford added a comment -

          updated

          Ben Clifford added a comment - updated
          Ben Clifford made changes -
          Summary Original: Regular 404s for running jobs that get better after job is finished. New: Regular 404s for running builds that get better after build is finished.
          Ben Clifford made changes -
          Description Original: For some large (50%?) number of builds (across several projects):
           * The build starts.

          As expected:
           * build related URLs work ok. (for example, http://loris.beautifuldestinations.com:2000/job/beautilytics/branch/benc-typo/1/console provides a working progressive console.
           * The build appears in the LHS "build history" pane on the project page, as an in-progress build.
           * The build appears on the list of workers, as occupying a worker.
           * The build is correctly linked on github as an in-progress build.

          After running for several minutes, all build related URLs for that particular build start returning 404, but the list of workers still shows the build as occupying the worker (and provides a link which goes to the now 404-ing build page), and the build runs to completion successfully. Build-related URLs for other builds still work ok at this point.

          After the build has run to completion, all of the build related URLs which started 404ing work again (for example the complete console output can be seen).

          A screenshot of progressive console output breaking part way due to this is attached - to be clear, this happens for all URLs related to the build, not just the progressive console.

          This has happened in all of the weekly releases that I've tried (three or four over the past two months); previous to that I was not using Jenkins on this codebase/machine.
          New: For some large (50%?) number of builds (across several projects):
           * The build starts.

          As expected:
           * build related URLs work ok. (for example, http://loris.beautifuldestinations.com:2000/job/beautilytics/branch/benc-typo/1/console provides a working progressive console.
           * The build appears in the LHS "build history" pane on the project page, as an in-progress build.
           * The build appears on the list of workers, as occupying a worker.
           * The build is correctly linked on github as an in-progress build.

          After running for several minutes:

           * all build related URLs for that particular build start returning 404 (in the above url example, everything beginning: http://loris.beautifuldestinations.com:2000/job/beautilytics/branch/benc-typo/1)
           * and the project page ceases to show this build as existing. It displays previously finished builds.

          However:

           * the list of workers still shows the build as occupying the worker (and provides a link which goes to the now 404-ing build page)
           * the build runs to completion successfully.
           * Build-related URLs for other builds still work ok at this point.

          After the build has run to completion, all of the build related URLs which started 404ing work again (for example the complete console output can be seen).

          A screenshot of progressive console output breaking part way due to this is attached - to be clear, this happens for all URLs related to the build, not just the progressive console.

          This has happened in all of the weekly releases that I've tried (three or four over the past two months); previous to that I was not using Jenkins on this codebase/machine.

          Daniel Beck added a comment -

          This is a Workflow Multibranch project, right?

          Go to the Jenkins script console, and run this command:

          org.kohsuke.stapler.Dispatcher.TRACE = true

          Now, when requesting a build related URL, check the HTTP response headers sent by Jenkins. There will be headers named X-Stapler-Trace-001 or similar. Please provide all of them.

          CC jglick

          Daniel Beck added a comment - This is a Workflow Multibranch project, right? Go to the Jenkins script console, and run this command: org.kohsuke.stapler.Dispatcher.TRACE = true Now, when requesting a build related URL, check the HTTP response headers sent by Jenkins. There will be headers named X-Stapler-Trace-001 or similar. Please provide all of them. CC jglick

          Ben Clifford added a comment -

          For a working, finished build:

          {{> GET /job/benc-master-db/47/console HTTP/1.1
          > Host: loris.beautifuldestinations.com:2000
          > Accept-Encoding: deflate, gzip
          > User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:44.0) Gecko/20100101 Firefox/44.0 Iceweasel/44.0.2
          > Accept: text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8
          > Accept-Language: en-US,en;q=0.5
          > DNT: 1
          > Referer: http://loris.beautifuldestinations.com:2000/job/benc-master-db/47/
          > Cookie: *ELIDED BY benclifford*
          > Connection: keep-alive
          > Cache-Control: max-age=0
          >
          < HTTP/1.1 200 OK
          < Date: Mon, 13 Jun 2016 21:20:15 GMT
          < X-Content-Type-Options: nosniff
          < Stapler-Trace-001: -> evaluate(<hudson.model.Hudson@1253fcf2> :hudson.model.Hudson,"/job/benc-master-db/47/console")
          < Stapler-Trace-002: -> evaluate(((StaplerProxy)<hudson.model.Hudson@1253fcf2>).getTarget(),"/job/benc-master-db/47/console")
          < Stapler-Trace-003: -> evaluate(<hudson.model.Hudson@1253fcf2>.getJob("benc-master-db"),"/47/console")
          < Stapler-Trace-004: -> evaluate(<hudson.model.FreeStyleProject@71273b20[benc-master-db]> :hudson.model.FreeStyleProject,"/47/console")
          < Stapler-Trace-005: -> evaluate(<hudson.model.FreeStyleProject@71273b20[benc-master-db]>.getDynamic("47",...),"/console")
          < Stapler-Trace-006: -> evaluate(<benc-master-db #47> :hudson.model.FreeStyleBuild,"/console")
          < Stapler-Trace-007: -> hudson/model/Run/console.jelly on <benc-master-db #47>
          < Content-Encoding: gzip
          < Expires: 0
          < Cache-Control: no-cache,no-store,must-revalidate
          < X-Hudson-Theme: default
          < Content-Type: text/html;charset=UTF-8
          < X-Hudson: 1.395
          < X-Jenkins: 2.8
          < X-Jenkins-Session: 7ae07cb4
          < X-Frame-Options: sameorigin
          < X-Instance-Identity: *ELIDED BY benclifford*
          < X-SSH-Endpoint: loris.beautifuldestinations.com:43014
          < X-ConsoleAnnotator: *ELIDED BY benclifford*
          < Content-Length: 9165
          < Server: Jetty(9.2.z-SNAPSHOT)
          }}

          I'll get some from a failing case when I can poke it into happening.

          Ben Clifford added a comment - For a working, finished build: {{> GET /job/benc-master-db/47/console HTTP/1.1 > Host: loris.beautifuldestinations.com:2000 > Accept-Encoding: deflate, gzip > User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:44.0) Gecko/20100101 Firefox/44.0 Iceweasel/44.0.2 > Accept: text/html,application/xhtml+xml,application/xml;q=0.9, / ;q=0.8 > Accept-Language: en-US,en;q=0.5 > DNT: 1 > Referer: http://loris.beautifuldestinations.com:2000/job/benc-master-db/47/ > Cookie: * ELIDED BY benclifford * > Connection: keep-alive > Cache-Control: max-age=0 > < HTTP/1.1 200 OK < Date: Mon, 13 Jun 2016 21:20:15 GMT < X-Content-Type-Options: nosniff < Stapler-Trace-001: -> evaluate(<hudson.model.Hudson@1253fcf2> :hudson.model.Hudson,"/job/benc-master-db/47/console") < Stapler-Trace-002: -> evaluate(((StaplerProxy)<hudson.model.Hudson@1253fcf2>).getTarget(),"/job/benc-master-db/47/console") < Stapler-Trace-003: -> evaluate(<hudson.model.Hudson@1253fcf2>.getJob("benc-master-db"),"/47/console") < Stapler-Trace-004: -> evaluate(<hudson.model.FreeStyleProject@71273b20 [benc-master-db] > :hudson.model.FreeStyleProject,"/47/console") < Stapler-Trace-005: -> evaluate(<hudson.model.FreeStyleProject@71273b20 [benc-master-db] >.getDynamic("47",...),"/console") < Stapler-Trace-006: -> evaluate(<benc-master-db #47> :hudson.model.FreeStyleBuild,"/console") < Stapler-Trace-007: -> hudson/model/Run/console.jelly on <benc-master-db #47> < Content-Encoding: gzip < Expires: 0 < Cache-Control: no-cache,no-store,must-revalidate < X-Hudson-Theme: default < Content-Type: text/html;charset=UTF-8 < X-Hudson: 1.395 < X-Jenkins: 2.8 < X-Jenkins-Session: 7ae07cb4 < X-Frame-Options: sameorigin < X-Instance-Identity: * ELIDED BY benclifford * < X-SSH-Endpoint: loris.beautifuldestinations.com:43014 < X-ConsoleAnnotator: * ELIDED BY benclifford * < Content-Length: 9165 < Server: Jetty(9.2.z-SNAPSHOT) }} I'll get some from a failing case when I can poke it into happening.

            mjdetullio Matthew DeTullio
            benclifford Ben Clifford
            Votes:
            1 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated: