Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-54304

Jobs in parallel dont display the variant they are running.

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Major Major
    • workflow-job-plugin
    • None
    • Jenkins 2.138.2
    • workflow-job 2.31

      This issue manifested itself since October 15th.

      When a build is triggered and it runs on multiple slaves parallely, you normally get an output for each line corresponding to the variant its running. Since a Jenkins update on October 15th, this has not been the case. This is making it harder for us to debug the issues on the variants/slaves, as we don't know on which variant the issue is being manifested.

      NOTE:

      Although unrelated, this issue was manifested together with another bug which doesn't allow an anonymous user to view jobs, even if logged in as a non-admin. Only users with admin privileges are allowed to view the logs. That issue has been found on your backlog. https://issues.jenkins-ci.org/browse/JENKINS-54031

       

        1. earlier.JPG
          earlier.JPG
          35 kB
        2. now.JPG
          now.JPG
          18 kB
        3. now.JPG
          now.JPG
          18 kB
        4. now.JPG
          now.JPG
          18 kB
        5. earlier.JPG
          earlier.JPG
          35 kB
        6. JENKINS-54304.png
          JENKINS-54304.png
          234 kB
        7. image-2018-11-20-17-20-19-040.png
          image-2018-11-20-17-20-19-040.png
          169 kB
        8. image-2018-11-20-16-58-04-034.png
          image-2018-11-20-16-58-04-034.png
          17 kB
        9. NewNodeConsoleNote-script.js
          4 kB
        10. jenkins-browser-log
          13 kB
        11. log-index
          76 kB
        12. log
          4.93 MB
        13. two-variants.png
          two-variants.png
          271 kB
        14. image-2018-11-08-15-41-10-158.png
          image-2018-11-08-15-41-10-158.png
          14 kB
        15. image-2018-11-08-15-39-26-889.png
          image-2018-11-08-15-39-26-889.png
          19 kB
        16. image-2018-11-08-15-38-49-986.png
          image-2018-11-08-15-38-49-986.png
          131 kB
        17. image-2018-11-06-14-39-16-393.png
          image-2018-11-06-14-39-16-393.png
          15 kB
        18. image-2018-11-06-14-35-41-221.png
          image-2018-11-06-14-35-41-221.png
          16 kB
        19. Selection_047.png
          Selection_047.png
          157 kB
        20. Selection_048.png
          Selection_048.png
          196 kB

          [JENKINS-54304] Jobs in parallel dont display the variant they are running.

          Jesse Glick added a comment -

          An experimental build if you care to try it out. (And if you are handy with CSS, please, offer improvements.)

          Jesse Glick added a comment - An experimental build if you care to try it out. (And if you are handy with CSS, please, offer improvements.)

          That did not fix the issue we are having. Everything seems to be running the same way.

          fisnik hajredini added a comment - That did not fix the issue we are having. Everything seems to be running the same way.

          Jesse Glick added a comment -

          Working for me… make sure you are actually running the stated build (check Plugin Manager » Installed) and attach a screenshot.

          Jesse Glick added a comment - Working for me… make sure you are actually running the stated build (check Plugin Manager » Installed ) and attach a screenshot.

          Here you go: 

          I cant find the specific version ID, but here is another one which asks me to downgrade it to the previous one:

          fisnik hajredini added a comment - Here you go:  I cant find the specific version ID, but here is another one which asks me to downgrade it to the previous one:

          Just so that I make sure i am trying it on the correct way: Is there any way i can enable/disable this feature? Maybe with the new update it got disabled by default. I didnt understand the message where u said "This is as designed..........". This was working for us for over a year now, and now it just stopped.

          fisnik hajredini added a comment - Just so that I make sure i am trying it on the correct way: Is there any way i can enable/disable this feature? Maybe with the new update it got disabled by default. I didnt understand the message where u said "This is as designed..........". This was working for us for over a year now, and now it just stopped.

          Jesse Glick added a comment -

          Looks like you were running an experimental build, then upgraded to an official release marked newer than it. Try installing this newer build.

          Jesse Glick added a comment - Looks like you were running an experimental build, then upgraded to an official release marked newer than it. Try installing this newer build .

          So initially there was an update to 2.28 which i tried and it didnt have this issue fixed. Afterwards i tried the newer build you offered and it also didnt work

          Pictures:

          The variant should've been in front each line.

           

          Version:

           

          fisnik hajredini added a comment - So initially there was an update to 2.28 which i tried and it didnt have this issue fixed. Afterwards i tried the newer build you offered and it also didnt work Pictures: The variant should've been in front each line.   Version:  

          Jesse Glick added a comment -

          Please attach the log and log-index files from the build directory ($JENKINS_HOME/jobs/*/builds/*/) and show a full-size screenshot including a point at which output switches from one branch to another, and note any browser console warnings or errors.

          Jesse Glick added a comment - Please attach the log and log-index files from the build directory ( $JENKINS_HOME/jobs/*/builds/*/ ) and show a full-size screenshot including a point at which output switches from one branch to another, and note any browser console warnings or errors.

          I have added a picture called two-variants (I assume that by output of switches from one branch to the other you meant by two different variants). The colors represent different variants which before used to have a notation in front of them in this form:

          [Variant Name]: NOTE: Running task ..... 

          The only warning from the browser is:

          ‘src’ attribute of <script> element is empty.
          

           I've also attached log-index and log as requested.

          fisnik hajredini added a comment - I have added a picture called two-variants (I assume that by output of switches from one branch to the other you meant by two different variants). The colors represent different variants which before used to have a notation in front of them in this form: [Variant Name]: NOTE: Running task ..... The only warning from the browser is: ‘src’ attribute of <script> element is empty.  I've also attached log-index and log as requested.

          Jesse Glick added a comment -

          fhajredini I suspect the browser error is the clue—for some reason the script which displays branch annotations is not loaded. It is hard to be sure from your screenshot since it is only from a region of the screen, not the whole browser window. See if your console page contains something like <script src='/descriptor/org.jenkinsci.plugins.workflow.job.console.NewNodeConsoleNote/script.js' /> and whether the loaded script file contains a line 37

          var branch = label.substring(8)
          

          Also you may need to force a page reload.

          Judging by the frequency of branch transitions, you likely want to run with the startup option -Dorg.jenkinsci.plugins.workflow.steps.durable_task.DurableTaskStep.USE_WATCHING=true to get better performance. This mode is currently disabled pending some unreleased fixes related to coloration and some field reports of failures, but it is intended to be the default mode going forward.

          Jesse Glick added a comment - fhajredini I suspect the browser error is the clue—for some reason the script which displays branch annotations is not loaded. It is hard to be sure from your screenshot since it is only from a region of the screen, not the whole browser window. See if your console page contains something like <script src='/descriptor/org.jenkinsci.plugins.workflow.job.console.NewNodeConsoleNote/script.js' /> and whether the loaded script file contains a line 37 var branch = label.substring(8) Also you may need to force a page reload. Judging by the frequency of branch transitions, you likely want to run with the startup option -Dorg.jenkinsci.plugins.workflow.steps.durable_task.DurableTaskStep.USE_WATCHING=true to get better performance. This mode is currently disabled pending some unreleased fixes related to coloration and some field reports of failures, but it is intended to be the default mode going forward.

          fisnik hajredini added a comment - - edited

          I am attaching my browsers source code when im viewing the console. I named it jenkins-browser-log

          Besides the browser source code I checked the script. The script actually exists but the line 37 is not included. I have also attached the given script. Its called `NewNodeConsoleNote-script.js`

          Just FYI, we have tested this in chromium and in firefox, the issue manifests on both of them.

          fisnik hajredini added a comment - - edited I am attaching my browsers source code when im viewing the console. I named it jenkins-browser-log Besides the browser source code I checked the script. The script actually exists but the line 37 is not included. I have also attached the given script. Its called `NewNodeConsoleNote-script.js` Just FYI, we have tested this in chromium and in firefox, the issue manifests on both of them.

          Jesse Glick added a comment -

          Indeed your browser is not running the code from my pull request. Either you are not actually running the patched version of the plugin, or you need to force-reload the page to convince the browser to ignore its previously cached version of the script.

          (It is likely a Jenkins core bug that a force reload is needed, of course. I see Last-Modified and Expires headers. Normally these are not supposed to matter because restarting Jenkins would serve static resources from a new randomized URL. It seems Functions.generateConsoleAnnotationScriptAndStylesheet is incorrect.)

          Jesse Glick added a comment - Indeed your browser is not running the code from my pull request. Either you are not actually running the patched version of the plugin, or you need to force-reload the page to convince the browser to ignore its previously cached version of the script. (It is likely a Jenkins core bug that a force reload is needed, of course. I see Last-Modified and Expires headers. Normally these are not supposed to matter because restarting Jenkins would serve static resources from a new randomized URL. It seems Functions.generateConsoleAnnotationScriptAndStylesheet is incorrect.)

          Jesse Glick added a comment -

          Regarding the apparent Jenkins core bug, I filed PR 3757.

          Jesse Glick added a comment - Regarding the apparent Jenkins core bug, I filed PR 3757 .

          I assume that by force reload the page, you mean reload without cache. That i have done.

          The plugin version that i have seems to be the correct one, i also sent you the proof for it. What other way can i ensure that im running that version?

          Is there any step i can take here to assist of any help? Or just wait for PR 3757? 

          fisnik hajredini added a comment - I assume that by force reload the page, you mean reload without cache. That i have done. The plugin version that i have seems to be the correct one, i also sent you the proof for it. What other way can i ensure that im running that version? Is there any step i can take here to assist of any help? Or just wait for PR 3757? 

          Jesse Glick added a comment -

          Force reload on Chromium is Ctrl-Shift-R. I do not know about other browsers. Core PR 3757 would merely avoid the need to do this manually.

          Somehow either your browser is continuing to cache the old file, or you are not really running the correct version, or the old version is lying around somewhere and mixing things up. It is hard to guess without having hands-on access to your server. You can check something like

          curl -i http://YOURSERVER/jenkins/descriptor/org.jenkinsci.plugins.workflow.job.console.NewNodeConsoleNote/script.js
          

          and compare the result to this revision which should also be present as org/jenkinsci/plugins/workflow/job/console/NewNodeConsoleNote/script.js in WEB-INF/lib/workflow-job.jar in your $JENKINS_HOME/plugins/workflow-job.jpi. (You installed the experimental build via the upload form in Plugin Manager » Advanced I hope? Check the plugins/ directory for stray files.)

          Jesse Glick added a comment - Force reload on Chromium is Ctrl-Shift-R. I do not know about other browsers. Core PR 3757 would merely avoid the need to do this manually. Somehow either your browser is continuing to cache the old file, or you are not really running the correct version, or the old version is lying around somewhere and mixing things up. It is hard to guess without having hands-on access to your server. You can check something like curl -i http://YOURSERVER/jenkins/descriptor/org.jenkinsci.plugins.workflow.job.console.NewNodeConsoleNote/script.js and compare the result to this revision which should also be present as org/jenkinsci/plugins/workflow/job/console/NewNodeConsoleNote/script.js in WEB-INF/lib/workflow-job.jar in your $JENKINS_HOME/plugins/workflow-job.jpi . (You installed the experimental build via the upload form in Plugin Manager » Advanced I hope? Check the plugins/ directory for stray files.)

          Jesse Glick added a comment -

          By the way I found that the core bug was already filed: JENKINS-38719

          Jesse Glick added a comment - By the way I found that the core bug was already filed: JENKINS-38719

          So that means that the page is already being refreshed without cache. This brings us to the conclusion that the actual patch that you sent me, is not correctly uploaded or not correctly working.

          What steps can i take to upload the plugin besides the manage plugins > advanced?

          Afterwards how can i ensure 100% that the correct plugin is actually uploaded?

          fisnik hajredini added a comment - So that means that the page is already being refreshed without cache. This brings us to the conclusion that the actual patch that you sent me, is not correctly uploaded or not correctly working. What steps can i take to upload the plugin besides the manage plugins > advanced? Afterwards how can i ensure 100% that the correct plugin is actually uploaded?

          Jesse Glick added a comment -

          Well I downloaded https://repo.jenkins-ci.org/incrementals/org/jenkins-ci/plugins/workflow/workflow-job/2.29-rc915.1276983ff71c/workflow-job-2.29-rc915.1276983ff71c.hpi and it has the script.js of length 4877, unlike the older one of length in 3816 in https://repo.jenkins-ci.org/public/org/jenkins-ci/plugins/workflow/workflow-job/2.28/workflow-job-2.28.hpi so I believe the problem is on your end. Please check the diagnostics I suggested in my last message.

          And yes Plugin Manager » Advanced is the correct way to install downloaded plugins. I was making sure you did not manually dump it into the plugins/ directory under an unexpected name, for example.

          Jesse Glick added a comment - Well I downloaded https://repo.jenkins-ci.org/incrementals/org/jenkins-ci/plugins/workflow/workflow-job/2.29-rc915.1276983ff71c/workflow-job-2.29-rc915.1276983ff71c.hpi and it has the script.js of length 4877, unlike the older one of length in 3816 in https://repo.jenkins-ci.org/public/org/jenkins-ci/plugins/workflow/workflow-job/2.28/workflow-job-2.28.hpi so I believe the problem is on your end. Please check the diagnostics I suggested in my last message. And yes Plugin Manager » Advanced is the correct way to install downloaded plugins. I was making sure you did not manually dump it into the plugins/ directory under an unexpected name, for example.

          Hi, sorry for the late reply, we were in a middle of a release.

          From now and on, this issue is prioritized from us. I was trying to investigate on it and below are my results:

          Curling our jobs page is not possible due to another failure with gitOauth plugin, which doesnt give permission to anyone to view the jobs unless you are an admin. However, when this step is manually done, the script seems to be the wrong one.
          I've tried hard refreshing the page manually, but still the script remains unchanged. We even restarted our jenkins, and still the script is unchanged and doesnt match to [the revision|https://github.com/jenkinsci/workflow-job-plugin/blob/1276983ff71ccaf81e22392ff0b5072bcf664c26/src/main/resources/org/jenkinsci/plugins/workflow/job/console/NewNodeConsoleNote/script.js. |https://github.com/jenkinsci/workflow-job-plugin/blob/1276983ff71ccaf81e22392ff0b5072bcf664c26/src/main/resources/org/jenkinsci/plugins/workflow/job/console/NewNodeConsoleNote/script.js].I]

          I have also checked the script on ` org/jenkinsci/plugins/workflow/job/console/NewNodeConsoleNote/script.js` of our `workflow-job.jar` and its still the outdated one.

          I went on the docker container to look further on the plugin. On `plugins/`I have a directory called 'workflow-job' and two files `workflow-job.jpi` and `workflow-job.bak`. The jpi file and bak file i didnt manage to open, however on workflow job there was WEB-INF and META-INF. I looked through these two directories and on `/var/jenkins_home/plugins/workflow-job/META-INF/maven/org.jenkins-ci.plugins.workflow/workflow-job` i saw the a file `pom.properties` which when catted gave me this:

          #Generated by Maven
          #Fri Nov 09 14:36:27 EST 2018
          version=2.29
          groupId=org.jenkins-ci.plugins.workflow
          artifactId=workflow-job
          

          Which ensures the correct version of the plugin.

          I also have pom.xml there. If you think it can be interesting i can send that to you as well.

          At this point I am confused. There is no other version of this plugin installed and yet although the version checks out, the changes dont. How can i further debug this issue?

          fisnik hajredini added a comment - Hi, sorry for the late reply, we were in a middle of a release. From now and on, this issue is prioritized from us. I was trying to investigate on it and below are my results: Curling our jobs page is not possible due to another failure with gitOauth plugin, which doesnt give permission to anyone to view the jobs unless you are an admin. However, when this step is manually done, the script seems to be the wrong one. I've tried hard refreshing the page manually, but still the script remains unchanged. We even restarted our jenkins, and still the script is unchanged and doesnt match to [the revision| https://github.com/jenkinsci/workflow-job-plugin/blob/1276983ff71ccaf81e22392ff0b5072bcf664c26/src/main/resources/org/jenkinsci/plugins/workflow/job/console/NewNodeConsoleNote/script.js . |https://github.com/jenkinsci/workflow-job-plugin/blob/1276983ff71ccaf81e22392ff0b5072bcf664c26/src/main/resources/org/jenkinsci/plugins/workflow/job/console/NewNodeConsoleNote/script.js].I] I have also checked the script on ` org/jenkinsci/plugins/workflow/job/console/NewNodeConsoleNote/script.js ` of our `workflow-job.jar` and its still the outdated one. I went on the docker container to look further on the plugin. On `plugins/`I have a directory called 'workflow-job' and two files `workflow-job.jpi` and `workflow-job.bak`. The jpi file and bak file i didnt manage to open, however on workflow job there was WEB-INF and META-INF. I looked through these two directories and on `/var/jenkins_home/plugins/workflow-job/META-INF/maven/org.jenkins-ci.plugins.workflow/workflow-job` i saw the a file `pom.properties` which when catted gave me this: #Generated by Maven #Fri Nov 09 14:36:27 EST 2018 version=2.29 groupId=org.jenkins-ci.plugins.workflow artifactId=workflow-job Which ensures the correct version of the plugin. I also have pom.xml there. If you think it can be interesting i can send that to you as well. At this point I am confused. There is no other version of this plugin installed and yet although the version checks out, the changes dont. How can i further debug this issue?

          Jesse Glick added a comment -

          2.29 is a released version of the plugin. The last version with the fix that I supplied you is 2.29-rc915.1276983ff71c. So you are not running the correct version. Possibly the issue is that another release was cut of the plugin after I last brought the PR up to date, and you may have applied that update from the update center without noticing that it would clobber the manual update. Here is a new build, 2.30-rc928.54893f9a5f5c.

          Regarding *.jpi files, these are simply ZIP (JAR) files.

          Jesse Glick added a comment - 2.29 is a released version of the plugin. The last version with the fix that I supplied you is 2.29-rc915.1276983ff71c . So you are not running the correct version. Possibly the issue is that another release was cut of the plugin after I last brought the PR up to date, and you may have applied that update from the update center without noticing that it would clobber the manual update. Here is a new build, 2.30-rc928.54893f9a5f5c . Regarding *.jpi files, these are simply ZIP (JAR) files.

          fisnik hajredini added a comment - - edited

          Have now moved to a local build on my machine so that i can test new stuff.

          I had the 2.29 by default on my machine and i got the 2.30 (the one u just suggested)

          I still get the same issue. The JS script is not running on my browser. Its the outdated one.

          Edit: Is it possible that this plugin is somehow depended on other plugins?

          fisnik hajredini added a comment - - edited Have now moved to a local build on my machine so that i can test new stuff. I had the 2.29 by default on my machine and i got the 2.30 (the one u just suggested) I still get the same issue. The JS script is not running on my browser. Its the outdated one. Edit: Is it possible that this plugin is somehow depended on other plugins?

          Jesse Glick added a comment -

          Please go through the steps I outlined before to verify what is really present in your $JENKINS_HOME/plugins/workflow-job.jpi.

          Jesse Glick added a comment - Please go through the steps I outlined before to verify what is really present in your $JENKINS_HOME/plugins/workflow-job.jpi .

          -Unzipped workflow-job.jpi
          --unzipped WEB-INF/lib/workflow-job.jar
          — Checked WEB-INF/lib/org/jenkinsci/plugins/workflow/job/console/NewNodeConsoleNote/script.js. The File checks out with the latest github code.

          First time the variants werent visible, so i force refreshed my browser. I got them (although slightly disoriented)

           

          Thank you for the help.

          fisnik hajredini added a comment - -Unzipped workflow-job.jpi --unzipped WEB-INF/lib/workflow-job.jar — Checked WEB-INF/lib/org/jenkinsci/plugins/workflow/job/console/NewNodeConsoleNote/script.js. The File checks out with the latest github code. First time the variants werent visible, so i force refreshed my browser. I got them (although slightly disoriented)   Thank you for the help.

          Jesse Glick added a comment -

          There we go at last. The visual appearance can probably be improved by someone more familiar with CSS.

          Jesse Glick added a comment - There we go at last. The visual appearance can probably be improved by someone more familiar with CSS.

          Also noticed another minor issue. The variants are visible only on /consoleFull logs. If its only console then the variants wont be visible.

          fisnik hajredini added a comment - Also noticed another minor issue. The variants are visible only on /consoleFull logs. If its only console then the variants wont be visible.

          The issue doesnt seem that minor anymore. The  [variant] is not searchable through CTRL+F. Also if u view the logs as plain text, you get no information of the variants. Having ~10MB of logs with >5 different variants makes it difficult to track the variant.

          fisnik hajredini added a comment - The issue doesnt seem that minor anymore. The  [variant] is not searchable through CTRL+F. Also if u view the logs as plain text, you get no information of the variants. Having ~10MB of logs with >5 different variants makes it difficult to track the variant.

          Jesse Glick added a comment -

          Not sure about the console vs. consoleFull issue. Should be visible in both. Would have to see if this is a reproducible issue.

          Regarding Ctrl-F, well, this is up to the browser. In this case the text is CSS-injected so it may decide not to include it in search.

          Regarding the raw console log, yes there is no branch information since this is added as a UI nicety on top of the HTML view. If you are interested in raw output from a particular step, you can download it directly.

          Jesse Glick added a comment - Not sure about the console vs. consoleFull issue. Should be visible in both. Would have to see if this is a reproducible issue. Regarding Ctrl-F, well, this is up to the browser. In this case the text is CSS-injected so it may decide not to include it in search. Regarding the raw console log, yes there is no branch information since this is added as a UI nicety on top of the HTML view. If you are interested in raw output from a particular step, you can download it directly.

          Devin Nusbaum added a comment -

          I just released Pipeline Job Plugin 2.31, which shows the branch names for parallel branches whenever log output transitions from one branch to another in the classic UI. See the attached screenshot for an idea of what it looks like:

          Devin Nusbaum added a comment - I just released Pipeline Job Plugin 2.31, which shows the branch names for parallel branches whenever log output transitions from one branch to another in the classic UI. See the attached screenshot for an idea of what it looks like:

          dnusbaum Just as a side note, its quite important to have the logs viewed on the raw console log as well as being searched via CTRL+F. Might want to open a bug regarding that one and also the logs not being seen on the console(only visible on consoleFull).

          fisnik hajredini added a comment - dnusbaum Just as a side note, its quite important to have the logs viewed on the raw console log as well as being searched via CTRL+F. Might want to open a bug regarding that one and also the logs not being seen on the console(only visible on consoleFull).

          Devin Nusbaum added a comment - - edited

          fhajredini FWIW I tried to reproduce your console vs. consoleFull issue before I released the change but could not reproduce it. Both URLs showed the branch names for me correctly.

          Regarding branch names in the raw logs, I think it would be possible to add them back, although I do not recall if jglick removed them intentionally (perhaps to cut down on bloat in raw logs?), so it would be good to make sure we understand whether there would be any drawbacks before we add them back in.

          Devin Nusbaum added a comment - - edited fhajredini FWIW I tried to reproduce your console vs. consoleFull issue before I released the change but could not reproduce it. Both URLs showed the branch names for me correctly. Regarding branch names in the raw logs, I think it would be possible to add them back, although I do not recall if jglick removed them intentionally (perhaps to cut down on bloat in raw logs?), so it would be good to make sure we understand whether there would be any drawbacks before we add them back in.

          Regarding the console vs consoleFull : I think its possible that if the Console only, shows all the log, then it will still print out the variants. Otherwise it wont. At least in my case i just double checked and it still doesnt show the logs on Console only (even with Pipeline Job 2.31)

           

          fisnik hajredini added a comment - Regarding the console vs consoleFull : I think its possible that if the Console only, shows all the log, then it will still print out the variants. Otherwise it wont. At least in my case i just double checked and it still doesnt show the logs on Console only (even with Pipeline Job 2.31)  

          Devin Nusbaum added a comment - - edited

          fhajredini You mean that it doesn't work if the logs are truncated and only the most recent lines are shown? I will try to reproduce the issue that way.

          Devin Nusbaum added a comment - - edited fhajredini You mean that it doesn't work if the logs are truncated and only the most recent lines are shown? I will try to reproduce the issue that way.

          Exactly. If the logs are long enough that the 'ConsoleLog' view is required to see all the logs, then on 'Console' only the variants wont be shown. Just a wild guess, cos i saw your test output was short, and our log outputs are always >10.000 line. 

          fisnik hajredini added a comment - Exactly. If the logs are long enough that the 'ConsoleLog' view is required to see all the logs, then on 'Console' only the variants wont be shown. Just a wild guess, cos i saw your test output was short, and our log outputs are always >10.000 line. 

          Devin Nusbaum added a comment -

          fhajredini Ok, I was able to reproduce the issue for log files that are truncated. I'm not sure what is going on, but I'll take a look.

          Devin Nusbaum added a comment - fhajredini Ok, I was able to reproduce the issue for log files that are truncated. I'm not sure what is going on, but I'll take a look.

          Let me know if you need any help.

          fisnik hajredini added a comment - Let me know if you need any help.

          Devin Nusbaum added a comment -

          The problem is that when the log is truncated, the span elements with the pipeline-new-node class that would have triggered this behavior.js rule are not present (because they only occur at the beginning of the parallel branch), so the rule is not triggered. If you have a parallel block that shows up in full at the end of the log then its branches are displayed even if the branches for prior parallel blocks are not. Maybe we could separate the rule that shows branch names from the rule that adds show/hide buttons and have it look for  `pipeline-node-*` blocks instead so that the start of the branch doesn't need to be present for it to work.

          Devin Nusbaum added a comment - The problem is that when the log is truncated, the span elements with the pipeline-new-node class that would have triggered this behavior.js rule are not present (because they only occur at the beginning of the parallel branch), so the rule is not triggered. If you have a parallel block that shows up in full at the end of the log then its branches are displayed even if the branches for prior parallel blocks are not. Maybe we could separate the rule that shows branch names from the rule that adds show/hide buttons and have it look for  `pipeline-node-*` blocks instead so that the start of the branch doesn't need to be present for it to work.

          Jesse Glick added a comment -

          Yes the lack of pipeline-new-node spans from truncated lines is a known issue affecting also show / hide links. I am not sure whether the proposed separation would work well; the current code uses the Behaviour system which is more efficient than just searching for all elements. Possibly the server side could emit the opening spans for nodes started before the truncation point but completed after it, though this cannot be done trivially by inspecting the start parameter since that would break incremental display.

          Jesse Glick added a comment - Yes the lack of pipeline-new-node spans from truncated lines is a known issue affecting also show / hide links. I am not sure whether the proposed separation would work well; the current code uses the Behaviour system which is more efficient than just searching for all elements. Possibly the server side could emit the opening spans for nodes started before the truncation point but completed after it, though this cannot be done trivially by inspecting the start parameter since that would break incremental display.

          Jesse Glick added a comment -

          Regarding branch names in the raw logs, I think it would be possible to add them back

          They were removed from the raw log intentionally; rendering of branch names is now done on the client side rather than being (redundantly) kept in the log file. Since the consoleText streams the raw log without any metadata, we do not see it. There could I suppose be some intermediate variant that processes the log in certain ways like this one but emits text/plain.

          Jesse Glick added a comment - Regarding branch names in the raw logs, I think it would be possible to add them back They were removed from the raw log intentionally; rendering of branch names is now done on the client side rather than being (redundantly) kept in the log file. Since the consoleText streams the raw log without any metadata, we do not see it. There could I suppose be some intermediate variant that processes the log in certain ways like this one but emits text/plain .

          Joerg Schwaerzler added a comment - - edited

          Since this ticket is closed I have to ask: Is there already somewhere another ticket out there about the branch names not being searchable anymore? Doesn't work with Chrome or Firefox.
          And you clearly do not want to use IE or Edge with Jenkins - so I didn't even give them a shot.

          In addition to that Chrome used to be the only browser being able to highlight all occurrences of a particular branch in the scroll bar - that way one could easily find an issue related to one branch only.

          Especially for bigger pipelines with lots of branches we still need this.

          The show/hide feature is quite nice. However what we would need is to be able to completely hide all branches (completely, all occurrences) except the chosen one. Only that would make the search obsolete. Is that / will that be possible?

          Joerg Schwaerzler added a comment - - edited Since this ticket is closed I have to ask: Is there already somewhere another ticket out there about the branch names not being searchable anymore? Doesn't work with Chrome or Firefox. And you clearly do not want to use IE or Edge with Jenkins - so I didn't even give them a shot. In addition to that Chrome used to be the only browser being able to highlight all occurrences of a particular branch in the scroll bar - that way one could easily find an issue related to one branch only. Especially for bigger pipelines with lots of branches we still need this. The show/hide feature is quite nice. However what we would need is to be able to completely hide all branches (completely, all occurrences) except the chosen one. Only that would make the search obsolete. Is that / will that be possible?

          Jesse Glick added a comment -

          Is there already somewhere another ticket out there

          If it is not Link ed here, then I would presume not.

          what we would need is to be able to completely hide all branches (completely, all occurrences) except the chosen one. Only that would make the search obsolete. Is that / will that be possible?

          It could be, using some CSS/JS manipulations, as suggested here. That would of course deserve its own (linked) issue.

          Jesse Glick added a comment - Is there already somewhere another ticket out there If it is not Link ed here, then I would presume not. what we would need is to be able to completely hide all branches (completely, all occurrences) except the chosen one. Only that would make the search obsolete. Is that / will that be possible? It could be, using some CSS/JS manipulations, as suggested here . That would of course deserve its own (linked) issue.

          Joerg Schwaerzler added a comment - - edited

          jglick: Thanks for the quick feedback. I created two new tickets linking them to this one (and to each other):
          JENKINS-56910
          JENKINS-56913

          Joerg Schwaerzler added a comment - - edited jglick : Thanks for the quick feedback. I created two new tickets linking them to this one (and to each other): JENKINS-56910 JENKINS-56913

          sakshi sood added a comment - - edited

          Not sure why this ticket is closed. What was the fix ?
          After upgrading Jenkins the logformat for Pipeline jobs changed. Due to this change it's no longer clear which line belongs to which step. This is a problem when looking at parallel steps.
           
          Please check my earlier and now screenshot. 

           

          Using plugin version 2.31

          and Jenkins version 2.150.3

          sakshi sood added a comment - - edited Not sure why this ticket is closed. What was the fix ? After upgrading Jenkins the logformat for Pipeline jobs changed. Due to this change it's no longer clear which line belongs to which step. This is a problem when looking at parallel steps.   Please check my earlier and now screenshot.    Using plugin version 2.31 and Jenkins version 2.150.3

          Jesse Glick added a comment -

          sakshisood yes the format has changed. In your second screenshot, everything at or below [junit] is coming from that branch, until the next such mark.

          Jesse Glick added a comment - sakshisood yes the format has changed. In your second screenshot, everything at or below [junit] is coming from that branch, until the next such mark.

          sakshi sood added a comment -

          jglick This change has made troubleshooting of the issues difficult. it's no longer clear which line belongs to which step. I have to scroll too many times to understand if i get some error to identify the error belongs to which step. Can this be reverted? If not, Is there any configuration change I can make to get the old behavior ?

          sakshi sood added a comment - jglick This change has made troubleshooting of the issues difficult. it's no longer clear which line belongs to which step. I have to scroll too many times to understand if i get some error to identify the error belongs to which step. Can this be reverted? If not, Is there any configuration change I can make to get the old behavior ?

          Jesse Glick added a comment -

          sakshisood no and no, but see linked issues.

          Jesse Glick added a comment - sakshisood no and no, but see linked issues.

          https://issues.jenkins-ci.org/browse/JENKINS-54304?focusedCommentId=357033&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-357033

           "Regarding branch names in the raw logs, I think it would be possible to add them back"

          We need the Branch Names in the Raw Logs. We have various tools/log-formatters that parse through raw logs. Please put this back.

          Shah Chowdhury added a comment - https://issues.jenkins-ci.org/browse/JENKINS-54304?focusedCommentId=357033&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-357033  "Regarding branch names in the raw logs, I think it would be possible to add them back" We need the Branch Names in the Raw Logs. We have various tools/log-formatters that parse through raw logs. Please put this back.

          Timm Drevensek added a comment - - edited

          Hey jglick,

          the change done here has killed our complete option to forward the log data to an external instance! This is really a major change that should be undone! Who cares about the log to be shown within the UI if you operate Jenkins within a corporation where you have some hundert jenkins jobs - you simply don't scroll trough the jobs anymore - you have to have some external processing.

          Before this change we where able to forward all logs to an external FileBeat, split the logs via Logstash and process it within ElasticSearch. With this change, we are limited to the Jenkins UI.

          With this change, there is currently NO efficient way to forward the logs anymore! The only way is the logstash plugin - but here you have to adapt all jobs and don't have any global "forward everything" option anymore (if you rely on 100% pipeline jobs).

          Please re-add the [pipeline step name] and also remove the binary encoded stuff (8mha:////) from the logs.

          Timm Drevensek added a comment - - edited Hey jglick , the change done here has killed our complete option to forward the log data to an external instance! This is really a major change that should be undone! Who cares about the log to be shown within the UI if you operate Jenkins within a corporation where you have some hundert jenkins jobs - you simply don't scroll trough the jobs anymore - you have to have some external processing. Before this change we where able to forward all logs to an external FileBeat, split the logs via Logstash and process it within ElasticSearch. With this change, we are limited to the Jenkins UI. With this change, there is currently NO efficient way to forward the logs anymore! The only way is the logstash plugin - but here you have to adapt all jobs and don't have any global "forward everything" option anymore (if you rely on 100% pipeline jobs). Please re-add the [pipeline step name] and also remove the binary encoded stuff (8mha:////) from the logs.

          Jesse Glick added a comment -

          abubadabu please see linked issues and do not reopen.

          Jesse Glick added a comment - abubadabu please see linked issues and do not reopen.

          We ended up reverting back to the old (2.25) level.

          Branch Names in the Raw Logs is critical for us. We have various tools/log-formatters that parse through raw logs. Please put this back. 

          Shah Chowdhury added a comment - We ended up reverting back to the old (2.25) level. Branch Names in the Raw Logs is critical for us. We have various tools/log-formatters that parse through raw logs. Please put this back. 

          Jesse Glick added a comment -

          tuttul as above.

          Jesse Glick added a comment - tuttul as above.

          this issue might be fixed in the GUI ...
          but it still exists for 2 uses cases in which variant information is not available anymore and makes some parts of the logs unusable:

          • when logs are accessed programmatically (using currentBuild.rawBuild.log for example)
          • when logs are accessed through the /consoleText link (to import the logs in a log gatherer for example)

          for those 2 uses cases this workaround might help: https://github.com/gdemengin/pipeline-logparser
          (implementation of https://stackoverflow.com/a/57351397/8380249)

          Guillaume DeMengin added a comment - this issue might be fixed in the GUI ... but it still exists for 2 uses cases in which variant information is not available anymore and makes some parts of the logs unusable: when logs are accessed programmatically (using currentBuild.rawBuild.log for example) when logs are accessed through the /consoleText link (to import the logs in a log gatherer for example) for those 2 uses cases this workaround might help: https://github.com/gdemengin/pipeline-logparser (implementation of https://stackoverflow.com/a/57351397/8380249 )

            jglick Jesse Glick
            fhajredini fisnik hajredini
            Votes:
            1 Vote for this issue
            Watchers:
            17 Start watching this issue

              Created:
              Updated:
              Resolved: