• Icon: Bug Bug
    • Resolution: Incomplete
    • Icon: Critical Critical
    • core
    • Jenkins 1.506

      Revisiting https://issues.jenkins-ci.org/browse/JENKINS-15156 once again.

      The issue still persists. Builds are disappearing from history, even though they are present on the disk. They reappear after Jenkins restart.

        1. check_invalid_builds.sh
          3 kB
        2. check_missing_builds.sh
          1 kB
        3. check_invalid_builds.sh
          3 kB
        4. check_missing_builds.sh
          1 kB
        5. plugin1.bmp
          plugin1.bmp
          3.65 MB
        6. plugin2.bmp
          plugin2.bmp
          1.92 MB
        7. plugin3.bmp
          plugin3.bmp
          3.65 MB

          [JENKINS-17265] Builds disappearing from history

          kflorian added a comment -

          Confirming this on Jenkins 1.505, running on Linux with Linux slaves.
          Build results seem to disappear from some jobs, while they show up as usual on others.
          Seems to happen even more frequently in 1.505 than in the version(s) before!
          Also confirming that "Reload Configuration from Disk" brings back the history.

          kflorian added a comment - Confirming this on Jenkins 1.505, running on Linux with Linux slaves. Build results seem to disappear from some jobs, while they show up as usual on others. Seems to happen even more frequently in 1.505 than in the version(s) before! Also confirming that "Reload Configuration from Disk" brings back the history.

          Jose Sa added a comment -

          Please notice that "Reload from disk" may be unsafe if you have actively running jobs as indicated in JENKINS-3265

          Jose Sa added a comment - Please notice that "Reload from disk" may be unsafe if you have actively running jobs as indicated in JENKINS-3265

          wbauer added a comment -

          Same issue for me, had it with the 1.4xxx builds and now still with 1.505 - really annoying!

          wbauer added a comment - Same issue for me, had it with the 1.4xxx builds and now still with 1.505 - really annoying!

          We need to try and identify what may/may not be causing the problems. Quite a few fixes went into 1.505 and earlier but there are obviously still a few gremlins in there.

          Here's a few questions that those of you seeing the problem can try to answer:

          • are there any relevant messages in any of the Jenkins logs. Any exceptions, notices about failure to load are useful.
          • what job types do problem occur on? maven, freestyle, other.
          • are you running Jenkins direct from the war file or within tomcat/similar environment.
          • what OS are your masters/slaves running?

          Answers to those will help to point towards the right person to fix/investigate further.

          Richard Mortimer added a comment - We need to try and identify what may/may not be causing the problems. Quite a few fixes went into 1.505 and earlier but there are obviously still a few gremlins in there. Here's a few questions that those of you seeing the problem can try to answer: are there any relevant messages in any of the Jenkins logs. Any exceptions, notices about failure to load are useful. what job types do problem occur on? maven, freestyle, other. are you running Jenkins direct from the war file or within tomcat/similar environment. what OS are your masters/slaves running? Answers to those will help to point towards the right person to fix/investigate further.

          Harpreet Nain added a comment -

          Richard,

          Here are the answers as per our environment:

          1. There is nothing that we can see coming through in the logs
          2. We are using freestyle jobs
          3. The problem occurs on both i.e running direct from war file and from tomcat.
          4. We dont have a master / slave setup. We have 2 instances one on linux and and one on windows. It happens on both.

          Harpreet Nain added a comment - Richard, Here are the answers as per our environment: 1. There is nothing that we can see coming through in the logs 2. We are using freestyle jobs 3. The problem occurs on both i.e running direct from war file and from tomcat. 4. We dont have a master / slave setup. We have 2 instances one on linux and and one on windows. It happens on both.

          1. Havent spotted exceptions. The only thing reporting error is Copy artifact plugin that cant find a build
          2. Freestyle jobs. Before 1.505 in matrix jobs builds disappeared veeery often . Matrix jobs dont seem to disappear now or eat least not so often
          3. Running using winstone embedded in jenkins
          4. HP-UX 11.21 Itanium Java 7 master. HP-UX 11.31 Itanium Java 7 and RHEL 5.4/5.6/5.8 Java 7 slaves.

          Nikolay Martynov added a comment - 1. Havent spotted exceptions. The only thing reporting error is Copy artifact plugin that cant find a build 2. Freestyle jobs. Before 1.505 in matrix jobs builds disappeared veeery often . Matrix jobs dont seem to disappear now or eat least not so often 3. Running using winstone embedded in jenkins 4. HP-UX 11.21 Itanium Java 7 master. HP-UX 11.31 Itanium Java 7 and RHEL 5.4/5.6/5.8 Java 7 slaves.

          wbauer added a comment -

          1. Nothing visible in the logs.
          2. We use freestyle jobs across the board.
          3. Running from war file.
          4. master jenkins 1.505 on Mac OS X 10.8.2 / Java 1.7 latest, slaves Win XP/7/w2k8s + RedHat 6.2 with Java 1.6 + Mac OS X 10.6/7/8 with Java 1.6

          wbauer added a comment - 1. Nothing visible in the logs. 2. We use freestyle jobs across the board. 3. Running from war file. 4. master jenkins 1.505 on Mac OS X 10.8.2 / Java 1.7 latest, slaves Win XP/7/w2k8s + RedHat 6.2 with Java 1.6 + Mac OS X 10.6/7/8 with Java 1.6

          FYI - This issue hasn't occurred again for us since "Reload from Disk" and updating all plugins about a week ago. That's progress, this use to happen all the time.

          1. Can't find anything in the logs.
          2. All freestyle projects.
          3. Running from war file I assume.
          4. Amazon Linux (amzn-ami-pv-2012.03.3.x86_64-ebs (ami-aecd60c7)), installed Jenkins through yum.

          Matt Williamson added a comment - FYI - This issue hasn't occurred again for us since "Reload from Disk" and updating all plugins about a week ago. That's progress, this use to happen all the time. 1. Can't find anything in the logs. 2. All freestyle projects. 3. Running from war file I assume. 4. Amazon Linux (amzn-ami-pv-2012.03.3.x86_64-ebs (ami-aecd60c7)), installed Jenkins through yum.

          Alex Vesely added a comment -

          I'm not sure, but at one time I was under the impression that this issue is seen more often with newly-created jobs - i.e., jobs created during the current Jenkins session.
          I'm using the Jenkins CLI to create chains of jobs (A runs B, B runs C, etc). After several runs of the chain, some or all builds disappear.

          1. Haven't seen anything. Where should I look?
          2. "Freestyle -> matrix -> matrix -> freestyle" chain
          3. From the .war
          4. SLES 10 for the master, SLES 10, Win2003, Win2008 slaves.

          Alex Vesely added a comment - I'm not sure, but at one time I was under the impression that this issue is seen more often with newly-created jobs - i.e., jobs created during the current Jenkins session. I'm using the Jenkins CLI to create chains of jobs (A runs B, B runs C, etc). After several runs of the chain, some or all builds disappear. 1. Haven't seen anything. Where should I look? 2. "Freestyle -> matrix -> matrix -> freestyle" chain 3. From the .war 4. SLES 10 for the master, SLES 10, Win2003, Win2008 slaves.

          @Alex Vesely you can generally find the logs at
          http://<myjenkins.url>/log/all
          (replace <myjenkins.uk> with the base URL of your instance).

          Richard Mortimer added a comment - @Alex Vesely you can generally find the logs at http://<myjenkins.url>/log/all (replace <myjenkins.uk> with the base URL of your instance).

          All, thanks for you quick feedback. It certainly looks like freestyle jobs are affected and maybe other types of jobs too. Certainly there no obvious common denominator in what has been reported.

          Ok. So I guess its time to look for a few more possibilities.

          • does it happen on any specific revision control system? SVN, GIT etc.
          • what plugins are loaded in your jenkins and are there any common build steps added to failing jobs.
          • are there any jobs on your system that do not seem to go missing?

          Richard Mortimer added a comment - All, thanks for you quick feedback. It certainly looks like freestyle jobs are affected and maybe other types of jobs too. Certainly there no obvious common denominator in what has been reported. Ok. So I guess its time to look for a few more possibilities. does it happen on any specific revision control system? SVN, GIT etc. what plugins are loaded in your jenkins and are there any common build steps added to failing jobs. are there any jobs on your system that do not seem to go missing?

          Jose Sa added a comment -

          I was trying to create some script to crosscheck locally available builds with the ones reported on web page using REST api appending to job "/api/json?tree=builds[number]" and noticed something curious, not all builds are reported in that list when comparing with the ones actually visible in build history, only the most recent ones on variable ammount.

          Not sure if there is any limitation/restriction to the api, but maybe this is a symptom of this problem because if it already thinks it has less builds that it actually has, on some cleanup procedure it might only keep the ones reported and discard the rest.

          Jose Sa added a comment - I was trying to create some script to crosscheck locally available builds with the ones reported on web page using REST api appending to job "/api/json?tree=builds [number] " and noticed something curious, not all builds are reported in that list when comparing with the ones actually visible in build history, only the most recent ones on variable ammount. Not sure if there is any limitation/restriction to the api, but maybe this is a symptom of this problem because if it already thinks it has less builds that it actually has, on some cleanup procedure it might only keep the ones reported and discard the rest.

          Harpreet Nain added a comment -

          Richard,

          We are using Git. Builds disappear randomly. I have not seen any pattern. On one occasion a build that had been run about 200 times without issues, lost all the build history which obviously reappeared when we did a reload.

          I am attaching screenshots of the plugins installed and their versions

          Harpreet Nain added a comment - Richard, We are using Git. Builds disappear randomly. I have not seen any pattern. On one occasion a build that had been run about 200 times without issues, lost all the build history which obviously reappeared when we did a reload. I am attaching screenshots of the plugins installed and their versions

          Harpreet Nain added a comment -

          HN

          Harpreet Nain added a comment - HN

          Using SVN, standard plugin. Using only matrix and freestyle jobs. Previously both were disappearing like crazy. Recently matrix jobs seems to become more stable. Freestyle jobs still have problems. Perhaps makes to note that jobs are not standalone: both matrix and freestyle jobs trigger each other (build other builds) heavily.

          Installed plugins (env injector is disabled):
          Jenkins Mailer Plugin 1.4
          External Monitor Job Type Plugin 1.1
          LDAP Plugin 1.2
          pam-auth 1.0
          Javadoc Plugin 1.1
          Token Macro Plugin
          This plug-in adds reusable macro expansion capability for other plug-ins to use. 1.6
          Maven 2 Project Plugin 1.505
          Jenkins Subversion Plug-in 1.45
          Ant Plugin 1.2
          Jenkins CVS Plug-in
          Integrates Jenkins with CVS version control system using a modified version of the Netbeans cvsclient. 2.8
          Dump Info BuildWrapper Plugin
          The Dump Info BuildWrapper Plugin allows jobs to automatically dump some important Hudson-specific information into the job log. 1.1
          Hudson platformlabeler plugin
          Assigns labels on nodes automatically based on the computer operating system. 1.1
          Tracking SVN Plugin 1.1
          Priority Sorter
          This plugin allows for the build queue to be sorted based on pre-assigned priorities for each job. 1.3
          Jenkins Translation Assistance plugin 1.10
          Static Analysis Utilities
          This plug-in provides utilities for the static code analysis plug-ins.
          If you like this open source plug-in please consider supporting my work by buying my Android game Inca Trails. 1.49
          Task Scanner Plugin
          This plug-in scans for open tasks in a specified set of files in the project modules and visualizes the results.
          If you like this open source plug-in please consider supporting my work by buying my Android game Inca Trails. 4.35
          Disk Usage Plugin
          This plugin counts disk usage. 0.18
          SSH Slaves plugin 0.22
          FindBugs Plug-in
          This plug-in collects the FindBugs analysis results of the project modules and visualizes the found warnings.
          If you like this open source plug-in please consider supporting my work by buying my Android game Inca Trails. 4.48
          Checkstyle Plug-in
          This plug-in collects the Checkstyle analysis results of the project modules and visualizes the found warnings.
          If you like this open source plug-in please consider supporting my work by buying my Android game Inca Trails. 3.35
          Git Client Plugin 1.0.3
          Git Plugin
          This plugin integrates GIT with Jenkins. 1.2.0
          View Job Filters
          Create smart views with exactly the jobs you want. Your smart views can automatically include or exclude jobs by using things like the SCM path or type, the job type, build statuses or trends or triggers, relevance to the logged-in user, email recipients, Maven configuration, job parameterization, and user permissions. Mix and match filters to narrow down to exactly what you want. 1.22
          Warnings Plug-in
          This plug-in collects the compiler warnings of the project modules and visualizes the results.
          If you like this open source plug-in please consider supporting my work by buying my Android game Inca Trails. 4.23
          PMD Plug-in
          This plug-in collects the PMD analysis results of the project modules and visualizes the found warnings.
          If you like this open source plug-in please consider supporting my work by buying my Android game Inca Trails. 3.34
          Copy Artifact Plugin
          Adds a build step to copy artifacts from another project. 1.25
          Downstream build view
          This plug-in allows you to view the full status all the downstream builds so that we can graphically see that everything for this build has been completed successfully. 1.8
          Hudson Blame Subversion Plug-in 1.200
          Groovy Postbuild
          Groovy postbuild task. 1.8
          groovyaxis
          This plugin is a sample to explain how to write a Jenkins plugin. 0.3
          Duplicate Code Scanner Plug-in
          This plug-in collects the duplicate code analysis results of the project modules and visualizes the found warnings. Supported duplicate code analysis tools:
          • PMD's Copy Paste Detector (CPD)
          • Simian
          If you like this open source plug-in please consider supporting my work by buying my Android game Inca Trails. 2.34
          Static Analysis Collector Plug-in
          This plug-in is an add-on for the plug-ins Checkstyle, Dry, FindBugs, PMD, Tasks, and Warnings: the plug-in collects the different analysis results and shows the results in a combined trend graph. Additionally, the plug-in provides health reporting and build stability based on these combined results.
          If you like this open source plug-in please consider supporting my work by buying my Android game Inca Trails. 1.35
          Slave Prerequisites plugin
          Run a script on slave node before starting job on it to check some prerequisites. 1.0
          Matrix Tie Parent plugin 1.1
          Jenkins Maven Invoker plugin 1.2
          M2 Repository Cleanup Plugin
          This plugin allows you to configure a maven2 job to clean some or all of the artifacts from the repository before it runs. 1.0
          Jenkins Multijob plugin
          This plugin is a MultiJob plugin. 1.8
          Jenkins Maven Release Plug-in Plug-in
          A plug-in that enables you to perform releases using the maven-release-plugin from Hudson. 0.9.1
          Config File Provider Plugin
          Ability to provide configuration files (e.g. settings.xml for maven, XML, groovy, custom files,...) loaded trough the UI which will be copy to the job workspace. 2.4
          Run Condition Plugin 0.10
          conditional-buildstep
          A buildstep wrapping any number of other buildsteps, controlling there execution based on a defined condition (e.g. BuildParameter). 1.2.1
          Copy data to workspace plugin
          This plugin copies data to workspace directory before each build. After build data will be deleted. 1.0
          Tool Environment plugin
          Lets you use "tools" in unusual ways, such as from shell scripts. 1.0
          Copy To Slave Plugin
          This plugin allows copying files located somewhere on the master node into the jobs' workspaces, whether their builds take place on the master node or on slave nodes. 1.4
          Async Http Client
          This plugin provides a shared dependency on the async-http-client library so that other plugins can co-operate when using this library. 1.7.8
          Maven Deployment Linker 1.5
          Build Pipeline Plugin
          This plugin renders upstream and downstream connected jobs that typically form a build pipeline. In addition, it offers the ability to define manual triggers for jobs that require intervention prior to execution, e.g. an approval process outside of Jenkins. 1.3.3
          Downstream-Ext
          This plugin provides extended options for triggering downstream projects. 1.8
          Matrix Reloaded Plugin
          The Matrix Reloaded plugin. Used for rebuilding a subset of a matrix' configurations, reusing the rest. 1.1.0
          Environment Injector Plugin
          This plugin makes it possible to set an environment for the builds. 1.83
          Jenkins Maven Repository Server Plugin 0.11
          Hudson Groovy builder
          This plugin executes Groovy code. 1.13
          All changes plugin
          This plugin shows all changes (also from dependent projects, sub-projects, ...) for a project 1.3
          Pending Changes plugin
          This plugin adds a view of all changes in the scm after the last build 0.3.0
          Changes since last successfull build Plugin
          Generate changelog for a range of jenkins builds. 0.5

          Nikolay Martynov added a comment - Using SVN, standard plugin. Using only matrix and freestyle jobs. Previously both were disappearing like crazy. Recently matrix jobs seems to become more stable. Freestyle jobs still have problems. Perhaps makes to note that jobs are not standalone: both matrix and freestyle jobs trigger each other (build other builds) heavily. Installed plugins (env injector is disabled): Jenkins Mailer Plugin 1.4 External Monitor Job Type Plugin 1.1 LDAP Plugin 1.2 pam-auth 1.0 Javadoc Plugin 1.1 Token Macro Plugin This plug-in adds reusable macro expansion capability for other plug-ins to use. 1.6 Maven 2 Project Plugin 1.505 Jenkins Subversion Plug-in 1.45 Ant Plugin 1.2 Jenkins CVS Plug-in Integrates Jenkins with CVS version control system using a modified version of the Netbeans cvsclient. 2.8 Dump Info BuildWrapper Plugin The Dump Info BuildWrapper Plugin allows jobs to automatically dump some important Hudson-specific information into the job log. 1.1 Hudson platformlabeler plugin Assigns labels on nodes automatically based on the computer operating system. 1.1 Tracking SVN Plugin 1.1 Priority Sorter This plugin allows for the build queue to be sorted based on pre-assigned priorities for each job. 1.3 Jenkins Translation Assistance plugin 1.10 Static Analysis Utilities This plug-in provides utilities for the static code analysis plug-ins. If you like this open source plug-in please consider supporting my work by buying my Android game Inca Trails. 1.49 Task Scanner Plugin This plug-in scans for open tasks in a specified set of files in the project modules and visualizes the results. If you like this open source plug-in please consider supporting my work by buying my Android game Inca Trails. 4.35 Disk Usage Plugin This plugin counts disk usage. 0.18 SSH Slaves plugin 0.22 FindBugs Plug-in This plug-in collects the FindBugs analysis results of the project modules and visualizes the found warnings. If you like this open source plug-in please consider supporting my work by buying my Android game Inca Trails. 4.48 Checkstyle Plug-in This plug-in collects the Checkstyle analysis results of the project modules and visualizes the found warnings. If you like this open source plug-in please consider supporting my work by buying my Android game Inca Trails. 3.35 Git Client Plugin 1.0.3 Git Plugin This plugin integrates GIT with Jenkins. 1.2.0 View Job Filters Create smart views with exactly the jobs you want. Your smart views can automatically include or exclude jobs by using things like the SCM path or type, the job type, build statuses or trends or triggers, relevance to the logged-in user, email recipients, Maven configuration, job parameterization, and user permissions. Mix and match filters to narrow down to exactly what you want. 1.22 Warnings Plug-in This plug-in collects the compiler warnings of the project modules and visualizes the results. If you like this open source plug-in please consider supporting my work by buying my Android game Inca Trails. 4.23 PMD Plug-in This plug-in collects the PMD analysis results of the project modules and visualizes the found warnings. If you like this open source plug-in please consider supporting my work by buying my Android game Inca Trails. 3.34 Copy Artifact Plugin Adds a build step to copy artifacts from another project. 1.25 Downstream build view This plug-in allows you to view the full status all the downstream builds so that we can graphically see that everything for this build has been completed successfully. 1.8 Hudson Blame Subversion Plug-in 1.200 Groovy Postbuild Groovy postbuild task. 1.8 groovyaxis This plugin is a sample to explain how to write a Jenkins plugin. 0.3 Duplicate Code Scanner Plug-in This plug-in collects the duplicate code analysis results of the project modules and visualizes the found warnings. Supported duplicate code analysis tools: • PMD's Copy Paste Detector (CPD) • Simian If you like this open source plug-in please consider supporting my work by buying my Android game Inca Trails. 2.34 Static Analysis Collector Plug-in This plug-in is an add-on for the plug-ins Checkstyle, Dry, FindBugs, PMD, Tasks, and Warnings: the plug-in collects the different analysis results and shows the results in a combined trend graph. Additionally, the plug-in provides health reporting and build stability based on these combined results. If you like this open source plug-in please consider supporting my work by buying my Android game Inca Trails. 1.35 Slave Prerequisites plugin Run a script on slave node before starting job on it to check some prerequisites. 1.0 Matrix Tie Parent plugin 1.1 Jenkins Maven Invoker plugin 1.2 M2 Repository Cleanup Plugin This plugin allows you to configure a maven2 job to clean some or all of the artifacts from the repository before it runs. 1.0 Jenkins Multijob plugin This plugin is a MultiJob plugin. 1.8 Jenkins Maven Release Plug-in Plug-in A plug-in that enables you to perform releases using the maven-release-plugin from Hudson. 0.9.1 Config File Provider Plugin Ability to provide configuration files (e.g. settings.xml for maven, XML, groovy, custom files,...) loaded trough the UI which will be copy to the job workspace. 2.4 Run Condition Plugin 0.10 conditional-buildstep A buildstep wrapping any number of other buildsteps, controlling there execution based on a defined condition (e.g. BuildParameter). 1.2.1 Copy data to workspace plugin This plugin copies data to workspace directory before each build. After build data will be deleted. 1.0 Tool Environment plugin Lets you use "tools" in unusual ways, such as from shell scripts. 1.0 Copy To Slave Plugin This plugin allows copying files located somewhere on the master node into the jobs' workspaces, whether their builds take place on the master node or on slave nodes. 1.4 Async Http Client This plugin provides a shared dependency on the async-http-client library so that other plugins can co-operate when using this library. 1.7.8 Maven Deployment Linker 1.5 Build Pipeline Plugin This plugin renders upstream and downstream connected jobs that typically form a build pipeline. In addition, it offers the ability to define manual triggers for jobs that require intervention prior to execution, e.g. an approval process outside of Jenkins. 1.3.3 Downstream-Ext This plugin provides extended options for triggering downstream projects. 1.8 Matrix Reloaded Plugin The Matrix Reloaded plugin. Used for rebuilding a subset of a matrix' configurations, reusing the rest. 1.1.0 Environment Injector Plugin This plugin makes it possible to set an environment for the builds. 1.83 Jenkins Maven Repository Server Plugin 0.11 Hudson Groovy builder This plugin executes Groovy code. 1.13 All changes plugin This plugin shows all changes (also from dependent projects, sub-projects, ...) for a project 1.3 Pending Changes plugin This plugin adds a view of all changes in the scm after the last build 0.3.0 Changes since last successfull build Plugin Generate changelog for a range of jenkins builds. 0.5

          Jose has made interesting note. I recall that sometimes we can see build listed in UI but Copy Artifacts Plugin seems to fail to find the build.

          Nikolay Martynov added a comment - Jose has made interesting note. I recall that sometimes we can see build listed in UI but Copy Artifacts Plugin seems to fail to find the build.

          Ivan Veselkov added a comment - - edited

          We have a matrix job with two configurations (os = hpux or linux ). There is a keep-forever build which appears in parent history but it is absent in histories of configurations. From parent build one can still access both configurations.
          The copy plugin almost always cannot copy artifacts from this build: "Unable to find a build for artifact copy from" but at least once it succeeded for os=hpux.

          Ivan Veselkov added a comment - - edited We have a matrix job with two configurations (os = hpux or linux ). There is a keep-forever build which appears in parent history but it is absent in histories of configurations. From parent build one can still access both configurations. The copy plugin almost always cannot copy artifacts from this build: "Unable to find a build for artifact copy from" but at least once it succeeded for os=hpux.

          Frank Merrow added a comment -

          Just chiming in with "me too" on 1.502 . . . it is new installation and no matter what I do it only seems to show jobs from "today". (Have not been around at midnight to look . . . but in morning onward I only see jobs that ran today).

          FWIW: If I look in the jobs directory, the correct number of history directories are there . . . but web view just doesn't show them. (???)

          Windows7, no slaves, nothing odd about the installation that isn't in 10 others I have that are working on Jenkins from 1.488 onward.

          Also, I have a second 1.502 installation that was brought on-line around the same time . . . everything shows there. I cannot say the plugin mix on the two systems is identical, but it is very similar.

          Frank Merrow added a comment - Just chiming in with "me too" on 1.502 . . . it is new installation and no matter what I do it only seems to show jobs from "today". (Have not been around at midnight to look . . . but in morning onward I only see jobs that ran today). FWIW: If I look in the jobs directory, the correct number of history directories are there . . . but web view just doesn't show them. (???) Windows7, no slaves, nothing odd about the installation that isn't in 10 others I have that are working on Jenkins from 1.488 onward. Also, I have a second 1.502 installation that was brought on-line around the same time . . . everything shows there. I cannot say the plugin mix on the two systems is identical, but it is very similar.

          Jose Sa added a comment - - edited

          Both "/api/json?tree=builds[number]" and "/rssAll" job urls seem to be returning only latest builds (not all as expected), like I wrote above this could be a symptom for this case.

          Luckily "/buildHistory/all" does still provide all visible builds, so I was able to create created a bash script to help automatically to check for this kind of problems and provide a text report.

          # Assuming some host specific variables
          jenkins_url=https://localhost:8080
          jobs_dir=/work/jenkins/jobs
          
          # This should work for any unix environment using bash
          for job_dir in ${jobs_dir}/*; do
              job=$(basename ${job_dir})
              echo "Checking $job..." >&2
              job_url=${jenkins_url}/job/${job}
              wget -q --no-check-certificate -O all ${job_url}/buildHistory/all
              gawk -F'#' '/ #/{print $2}' all > online_builds
              find ${jobs_dir}/${job}/builds/* -type l ! -name "last*" -prune | sed "s,${jobs_dir}/${job}/builds/,,g" > local_builds
              missing=$(gawk '
              BEGIN {first_file = ""}
              {
                  if (first_file == "") first_file = FILENAME
                  if (first_file == FILENAME) {build_list[$1]=1} else {delete build_list[$1]}
              }
              END {for (build in build_list) print build}
              ' local_builds online_builds)
              [ ! -z "${missing}" ] && echo Job $job missing builds: $missing
          done | tee missing_builds.txt
          

          I found out that after 3 days online, 85 (out of 1406) jobs are showing missing builds varying from a few to all.

          I have FreeStyle and MultiJob types and they both show this problem.
          We are using mixed ClearCase and SVN (both showing same problems)
          Server is Solaris 10 Sparc, having slaves on HP-UX, Windows (XP, 2k3, 7), RHEL (5, 6), Solaris.
          Running Jenkins from war file, currently with 1.505 (but noticing this problem since 1.485)

          Jose Sa added a comment - - edited Both "/api/json?tree=builds [number] " and "/rssAll" job urls seem to be returning only latest builds (not all as expected), like I wrote above this could be a symptom for this case. Luckily "/buildHistory/all" does still provide all visible builds, so I was able to create created a bash script to help automatically to check for this kind of problems and provide a text report. # Assuming some host specific variables jenkins_url=https: //localhost:8080 jobs_dir=/work/jenkins/jobs # This should work for any unix environment using bash for job_dir in ${jobs_dir}/*; do job=$(basename ${job_dir}) echo "Checking $job..." >&2 job_url=${jenkins_url}/job/${job} wget -q --no-check-certificate -O all ${job_url}/buildHistory/all gawk -F '#' '/ #/{print $2}' all > online_builds find ${jobs_dir}/${job}/builds/* -type l ! -name "last*" -prune | sed "s,${jobs_dir}/${job}/builds/,,g" > local_builds missing=$(gawk ' BEGIN {first_file = ""} { if (first_file == "") first_file = FILENAME if (first_file == FILENAME) {build_list[$1]=1} else {delete build_list[$1]} } END { for (build in build_list) print build} ' local_builds online_builds) [ ! -z "${missing}" ] && echo Job $job missing builds: $missing done | tee missing_builds.txt I found out that after 3 days online, 85 (out of 1406) jobs are showing missing builds varying from a few to all. I have FreeStyle and MultiJob types and they both show this problem. We are using mixed ClearCase and SVN (both showing same problems) Server is Solaris 10 Sparc, having slaves on HP-UX, Windows (XP, 2k3, 7), RHEL (5, 6), Solaris. Running Jenkins from war file, currently with 1.505 (but noticing this problem since 1.485)

          I confirm that script is perfect reproducer:

          Checking Infrastructure_update_sys...
          Checking Infrastructure_update_valgrind...
          Checking NOM_6_build...
          Job NOM_6_build missing builds: 887 618
          Checking NOM_6_kit...
          Job NOM_6_kit missing builds: 204
          Checking NOM_6_kit_smoke...
          Job NOM_6_kit_smoke missing builds: 139
          Checking NOM_6_test_f...
          Checking NOM_6_test_f_m...
          Job NOM_6_test_f_m missing builds: 88
          Checking NOM_6_test_l...
          Checking NOM_6_test_l_m...
          Job NOM_6_test_l_m missing builds: 204
          Checking NOM_6_test_sdk...
          Job NOM_6_test_sdk missing builds: 85
          Checking NOM_6_test_sdk_m...
          Checking NOM_V600_build...
          Checking NOM_V600_kit...
          Checking NOM_V600_kit_smoke...

          Nikolay Martynov added a comment - I confirm that script is perfect reproducer: Checking Infrastructure_update_sys... Checking Infrastructure_update_valgrind... Checking NOM_6_build... Job NOM_6_build missing builds: 887 618 Checking NOM_6_kit... Job NOM_6_kit missing builds: 204 Checking NOM_6_kit_smoke... Job NOM_6_kit_smoke missing builds: 139 Checking NOM_6_test_f... Checking NOM_6_test_f_m... Job NOM_6_test_f_m missing builds: 88 Checking NOM_6_test_l... Checking NOM_6_test_l_m... Job NOM_6_test_l_m missing builds: 204 Checking NOM_6_test_sdk... Job NOM_6_test_sdk missing builds: 85 Checking NOM_6_test_sdk_m... Checking NOM_V600_build... Checking NOM_V600_kit... Checking NOM_V600_kit_smoke...

          Jose Sa added a comment -

          Got this exception on some jobs when doing the wget for <job>/buildHistory/all a few minutes after jenkins was restarted.

          Mar 29, 2013 9:26:04 PM hudson.ExpressionFactory2$JexlExpression evaluate
          WARNING: Caught exception evaluating: it.renderList in /hudson/job/PM_OSS5_EBSS_LV2_FUNCTIONAL_GA12/buildHistory/all. Reason: java.lan
          g.reflect.InvocationTargetException
          java.lang.reflect.InvocationTargetException
                  at sun.reflect.GeneratedMethodAccessor239.invoke(Unknown Source)
                  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
                  at java.lang.reflect.Method.invoke(Method.java:601)
                  at org.apache.commons.jexl.util.PropertyExecutor.execute(PropertyExecutor.java:125)
                  at org.apache.commons.jexl.util.introspection.UberspectImpl$VelGetterImpl.invoke(UberspectImpl.java:314)
                  at org.apache.commons.jexl.parser.ASTArrayAccess.evaluateExpr(ASTArrayAccess.java:185)
                  at org.apache.commons.jexl.parser.ASTIdentifier.execute(ASTIdentifier.java:75)
                  at org.apache.commons.jexl.parser.ASTReference.execute(ASTReference.java:83)
                  at org.apache.commons.jexl.parser.ASTReference.value(ASTReference.java:57)
                  at org.apache.commons.jexl.parser.ASTReferenceExpression.value(ASTReferenceExpression.java:51)
                  at org.apache.commons.jexl.ExpressionImpl.evaluate(ExpressionImpl.java:80)
                  at hudson.ExpressionFactory2$JexlExpression.evaluate(ExpressionFactory2.java:74)
                  at org.apache.commons.jelly.expression.ExpressionSupport.evaluateRecurse(ExpressionSupport.java:61)
                  at org.apache.commons.jelly.expression.ExpressionSupport.evaluateAsIterator(ExpressionSupport.java:94)
                  at org.apache.commons.jelly.tags.core.ForEachTag.doTag(ForEachTag.java:89)
                  at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
                  at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
                  at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105)
                  at org.kohsuke.stapler.jelly.JellyViewScript.run(JellyViewScript.java:81)
                  at org.kohsuke.stapler.jelly.IncludeTag.doTag(IncludeTag.java:146)
                  at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
                  at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
                  at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105)
                  at org.kohsuke.stapler.jelly.JellyViewScript.run(JellyViewScript.java:81)
                  at org.kohsuke.stapler.jelly.IncludeTag.doTag(IncludeTag.java:146)
                  at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
                  at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
                  at org.kohsuke.stapler.jelly.CallTagLibScript$1.run(CallTagLibScript.java:98)
                  at org.apache.commons.jelly.tags.define.InvokeBodyTag.doTag(InvokeBodyTag.java:91)
                  at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
                  at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
                  at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
                  at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
                  at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105)
                  at org.kohsuke.stapler.jelly.CallTagLibScript.run(CallTagLibScript.java:119)
                  at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
                  at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105)
                  at org.kohsuke.stapler.jelly.JellyViewScript.run(JellyViewScript.java:81)
                  at org.kohsuke.stapler.jelly.IncludeTag.doTag(IncludeTag.java:146)
                  at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
                  at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
                  at org.kohsuke.stapler.jelly.CallTagLibScript$1.run(CallTagLibScript.java:98)
                  at org.apache.commons.jelly.tags.define.InvokeBodyTag.doTag(InvokeBodyTag.java:91)
                  at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
                  at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
                  at org.apache.commons.jelly.TagSupport.invokeBody(TagSupport.java:161)
                  at org.apache.commons.jelly.tags.core.OtherwiseTag.doTag(OtherwiseTag.java:41)
                  at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
                  at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
                  at org.apache.commons.jelly.TagSupport.invokeBody(TagSupport.java:161)
                  at org.apache.commons.jelly.tags.core.ChooseTag.doTag(ChooseTag.java:38)
                  at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
                  at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105)
                  at org.kohsuke.stapler.jelly.CallTagLibScript.run(CallTagLibScript.java:119)
                  at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105)
                  at org.kohsuke.stapler.jelly.JellyViewScript.run(JellyViewScript.java:81)
                  at org.kohsuke.stapler.jelly.DefaultScriptInvoker.invokeScript(DefaultScriptInvoker.java:63)
                  at org.kohsuke.stapler.jelly.DefaultScriptInvoker.invokeScript(DefaultScriptInvoker.java:53)
                  at org.kohsuke.stapler.jelly.JellyFacet$1.dispatch(JellyFacet.java:95)
                  at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:573)
                  at org.kohsuke.stapler.Stapler.invoke(Stapler.java:658)
                  at org.kohsuke.stapler.MetaClass$12.dispatch(MetaClass.java:384)
                  at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:573)
                  at org.kohsuke.stapler.Stapler.invoke(Stapler.java:658)
                  at org.kohsuke.stapler.MetaClass$6.doDispatch(MetaClass.java:241)
                  at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:53)
                  at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:573)
                  at org.kohsuke.stapler.Stapler.invoke(Stapler.java:658)
                  at org.kohsuke.stapler.Stapler.invoke(Stapler.java:487)
                  at org.kohsuke.stapler.Stapler.service(Stapler.java:164)
                  at javax.servlet.http.HttpServlet.service(HttpServlet.java:45)
                  at winstone.ServletConfiguration.execute(ServletConfiguration.java:248)
                  at winstone.RequestDispatcher.forward(RequestDispatcher.java:333)
                  at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:376)
                  at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:95)
                  at hudson.plugins.greenballs.GreenBallFilter.doFilter(GreenBallFilter.java:58)
                  at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:98)
                  at net.bull.javamelody.MonitoringFilter.doFilter(MonitoringFilter.java:206)
                  at net.bull.javamelody.MonitoringFilter.doFilter(MonitoringFilter.java:179)
                  at net.bull.javamelody.PluginMonitoringFilter.doFilter(PluginMonitoringFilter.java:86)
                  at org.jvnet.hudson.plugins.monitoring.HudsonMonitoringFilter.doFilter(HudsonMonitoringFilter.java:84)
                  at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:98)
                  at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:87)
                  at winstone.FilterConfiguration.execute(FilterConfiguration.java:194)
                  at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:366)
                  at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:47)
                  at winstone.FilterConfiguration.execute(FilterConfiguration.java:194)
                  at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:366)
                  at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84)
                  at hudson.security.UnwrapSecurityExceptionFilter.doFilter(UnwrapSecurityExceptionFilter.java:51)
                  at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
                  at org.acegisecurity.ui.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:124)
                  at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
                  at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125)
                  at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
                  at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:135)
                  at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
                  at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271)
                  at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
                  at org.acegisecurity.ui.basicauth.BasicProcessingFilter.doFilter(BasicProcessingFilter.java:174)
                  at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
                  at jenkins.security.ApiTokenFilter.doFilter(ApiTokenFilter.java:64)
                  at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
                  at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249)
                  at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:66)
                  at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
                  at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:76)
                  at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:164)
                  at winstone.FilterConfiguration.execute(FilterConfiguration.java:194)
                  at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:366)
                  at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:50)
                  at winstone.FilterConfiguration.execute(FilterConfiguration.java:194)
                  at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:366)
                  at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:81)
                  at winstone.FilterConfiguration.execute(FilterConfiguration.java:194)
                  at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:366)
                  at winstone.RequestDispatcher.forward(RequestDispatcher.java:331)
                  at winstone.RequestHandlerThread.processRequest(RequestHandlerThread.java:227)
                  at winstone.RequestHandlerThread.run(RequestHandlerThread.java:150)
                  at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
                  at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
                  at java.util.concurrent.FutureTask.run(FutureTask.java:166)
                  at winstone.BoundedExecutorService$1.run(BoundedExecutorService.java:77)
                  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
                  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
                  at java.lang.Thread.run(Thread.java:722)
          Caused by: java.lang.NullPointerException
                  at hudson.model.AbstractBuild.getPreviousBuild(AbstractBuild.java:214)
                  at hudson.tasks.Fingerprinter$FingerprintAction.onLoad(Fingerprinter.java:349)
                  at hudson.model.Run.onLoad(Run.java:319)
                  at hudson.model.RunMap.retrieve(RunMap.java:226)
                  at hudson.model.RunMap.retrieve(RunMap.java:59)
                  at jenkins.model.lazy.AbstractLazyLoadRunMap.load(AbstractLazyLoadRunMap.java:657)
                  at jenkins.model.lazy.AbstractLazyLoadRunMap.load(AbstractLazyLoadRunMap.java:619)
                  at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:358)
                  at hudson.model.AbstractBuild.getPreviousBuild(AbstractBuild.java:214)
                  at hudson.tasks.Fingerprinter$FingerprintAction.onLoad(Fingerprinter.java:349)
                  at hudson.model.Run.onLoad(Run.java:319)
                  at hudson.model.RunMap.retrieve(RunMap.java:226)
                  at hudson.model.RunMap.retrieve(RunMap.java:59)
                  at jenkins.model.lazy.AbstractLazyLoadRunMap.load(AbstractLazyLoadRunMap.java:657)
                  at jenkins.model.lazy.AbstractLazyLoadRunMap.load(AbstractLazyLoadRunMap.java:619)
                  at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:358)
                  at hudson.model.AbstractBuild.getPreviousBuild(AbstractBuild.java:214)
                  at hudson.tasks.Fingerprinter$FingerprintAction.onLoad(Fingerprinter.java:349)
                  at hudson.model.Run.onLoad(Run.java:319)
                  at hudson.model.RunMap.retrieve(RunMap.java:226)
                  at hudson.model.RunMap.retrieve(RunMap.java:59)
                  at jenkins.model.lazy.AbstractLazyLoadRunMap.load(AbstractLazyLoadRunMap.java:657)
                  at jenkins.model.lazy.AbstractLazyLoadRunMap.load(AbstractLazyLoadRunMap.java:619)
                  at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:358)
                  at hudson.model.AbstractBuild.getPreviousBuild(AbstractBuild.java:214)
                  at hudson.tasks.Fingerprinter$FingerprintAction.onLoad(Fingerprinter.java:349)
                  at hudson.model.Run.onLoad(Run.java:319)
                  at hudson.model.RunMap.retrieve(RunMap.java:226)
                  at hudson.model.RunMap.retrieve(RunMap.java:59)
                  at jenkins.model.lazy.AbstractLazyLoadRunMap.load(AbstractLazyLoadRunMap.java:657)
                  at jenkins.model.lazy.AbstractLazyLoadRunMap.load(AbstractLazyLoadRunMap.java:619)
                  at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:358)
                  at hudson.model.AbstractBuild.getPreviousBuild(AbstractBuild.java:214)
                  at hudson.model.AbstractBuild.getPreviousBuild(AbstractBuild.java:101)
                  at hudson.model.RunMap$1.next(RunMap.java:107)
                  at hudson.model.RunMap$1.next(RunMap.java:96)
                  at hudson.util.Iterators$2$1.next(Iterators.java:204)
                  at hudson.widgets.HistoryWidget.updateFirstTransientBuildKey(HistoryWidget.java:110)
                  at hudson.widgets.HistoryWidget.getRenderList(HistoryWidget.java:139)
                  ... 126 more
          

          Which seems related to another issue JENKINS-16845 regarding the specific NPE on AbstractBuild.getPreviousBuild.

          Jose Sa added a comment - Got this exception on some jobs when doing the wget for <job>/buildHistory/all a few minutes after jenkins was restarted. Mar 29, 2013 9:26:04 PM hudson.ExpressionFactory2$JexlExpression evaluate WARNING: Caught exception evaluating: it.renderList in /hudson/job/PM_OSS5_EBSS_LV2_FUNCTIONAL_GA12/buildHistory/all. Reason: java.lan g.reflect.InvocationTargetException java.lang.reflect.InvocationTargetException at sun.reflect.GeneratedMethodAccessor239.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at org.apache.commons.jexl.util.PropertyExecutor.execute(PropertyExecutor.java:125) at org.apache.commons.jexl.util.introspection.UberspectImpl$VelGetterImpl.invoke(UberspectImpl.java:314) at org.apache.commons.jexl.parser.ASTArrayAccess.evaluateExpr(ASTArrayAccess.java:185) at org.apache.commons.jexl.parser.ASTIdentifier.execute(ASTIdentifier.java:75) at org.apache.commons.jexl.parser.ASTReference.execute(ASTReference.java:83) at org.apache.commons.jexl.parser.ASTReference.value(ASTReference.java:57) at org.apache.commons.jexl.parser.ASTReferenceExpression.value(ASTReferenceExpression.java:51) at org.apache.commons.jexl.ExpressionImpl.evaluate(ExpressionImpl.java:80) at hudson.ExpressionFactory2$JexlExpression.evaluate(ExpressionFactory2.java:74) at org.apache.commons.jelly.expression.ExpressionSupport.evaluateRecurse(ExpressionSupport.java:61) at org.apache.commons.jelly.expression.ExpressionSupport.evaluateAsIterator(ExpressionSupport.java:94) at org.apache.commons.jelly.tags.core.ForEachTag.doTag(ForEachTag.java:89) at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105) at org.kohsuke.stapler.jelly.JellyViewScript.run(JellyViewScript.java:81) at org.kohsuke.stapler.jelly.IncludeTag.doTag(IncludeTag.java:146) at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105) at org.kohsuke.stapler.jelly.JellyViewScript.run(JellyViewScript.java:81) at org.kohsuke.stapler.jelly.IncludeTag.doTag(IncludeTag.java:146) at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.kohsuke.stapler.jelly.CallTagLibScript$1.run(CallTagLibScript.java:98) at org.apache.commons.jelly.tags.define.InvokeBodyTag.doTag(InvokeBodyTag.java:91) at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105) at org.kohsuke.stapler.jelly.CallTagLibScript.run(CallTagLibScript.java:119) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105) at org.kohsuke.stapler.jelly.JellyViewScript.run(JellyViewScript.java:81) at org.kohsuke.stapler.jelly.IncludeTag.doTag(IncludeTag.java:146) at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.kohsuke.stapler.jelly.CallTagLibScript$1.run(CallTagLibScript.java:98) at org.apache.commons.jelly.tags.define.InvokeBodyTag.doTag(InvokeBodyTag.java:91) at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.apache.commons.jelly.TagSupport.invokeBody(TagSupport.java:161) at org.apache.commons.jelly.tags.core.OtherwiseTag.doTag(OtherwiseTag.java:41) at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269) at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95) at org.apache.commons.jelly.TagSupport.invokeBody(TagSupport.java:161) at org.apache.commons.jelly.tags.core.ChooseTag.doTag(ChooseTag.java:38) at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269) at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105) at org.kohsuke.stapler.jelly.CallTagLibScript.run(CallTagLibScript.java:119) at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105) at org.kohsuke.stapler.jelly.JellyViewScript.run(JellyViewScript.java:81) at org.kohsuke.stapler.jelly.DefaultScriptInvoker.invokeScript(DefaultScriptInvoker.java:63) at org.kohsuke.stapler.jelly.DefaultScriptInvoker.invokeScript(DefaultScriptInvoker.java:53) at org.kohsuke.stapler.jelly.JellyFacet$1.dispatch(JellyFacet.java:95) at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:573) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:658) at org.kohsuke.stapler.MetaClass$12.dispatch(MetaClass.java:384) at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:573) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:658) at org.kohsuke.stapler.MetaClass$6.doDispatch(MetaClass.java:241) at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:53) at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:573) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:658) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:487) at org.kohsuke.stapler.Stapler.service(Stapler.java:164) at javax.servlet.http.HttpServlet.service(HttpServlet.java:45) at winstone.ServletConfiguration.execute(ServletConfiguration.java:248) at winstone.RequestDispatcher.forward(RequestDispatcher.java:333) at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:376) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:95) at hudson.plugins.greenballs.GreenBallFilter.doFilter(GreenBallFilter.java:58) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:98) at net.bull.javamelody.MonitoringFilter.doFilter(MonitoringFilter.java:206) at net.bull.javamelody.MonitoringFilter.doFilter(MonitoringFilter.java:179) at net.bull.javamelody.PluginMonitoringFilter.doFilter(PluginMonitoringFilter.java:86) at org.jvnet.hudson.plugins.monitoring.HudsonMonitoringFilter.doFilter(HudsonMonitoringFilter.java:84) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:98) at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:87) at winstone.FilterConfiguration.execute(FilterConfiguration.java:194) at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:366) at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:47) at winstone.FilterConfiguration.execute(FilterConfiguration.java:194) at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:366) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84) at hudson.security.UnwrapSecurityExceptionFilter.doFilter(UnwrapSecurityExceptionFilter.java:51) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.ui.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:124) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:135) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.ui.basicauth.BasicProcessingFilter.doFilter(BasicProcessingFilter.java:174) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at jenkins.security.ApiTokenFilter.doFilter(ApiTokenFilter.java:64) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249) at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:66) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:76) at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:164) at winstone.FilterConfiguration.execute(FilterConfiguration.java:194) at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:366) at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:50) at winstone.FilterConfiguration.execute(FilterConfiguration.java:194) at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:366) at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:81) at winstone.FilterConfiguration.execute(FilterConfiguration.java:194) at winstone.RequestDispatcher.doFilter(RequestDispatcher.java:366) at winstone.RequestDispatcher.forward(RequestDispatcher.java:331) at winstone.RequestHandlerThread.processRequest(RequestHandlerThread.java:227) at winstone.RequestHandlerThread.run(RequestHandlerThread.java:150) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) at java.util.concurrent.FutureTask.run(FutureTask.java:166) at winstone.BoundedExecutorService$1.run(BoundedExecutorService.java:77) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang. Thread .run( Thread .java:722) Caused by: java.lang.NullPointerException at hudson.model.AbstractBuild.getPreviousBuild(AbstractBuild.java:214) at hudson.tasks.Fingerprinter$FingerprintAction.onLoad(Fingerprinter.java:349) at hudson.model.Run.onLoad(Run.java:319) at hudson.model.RunMap.retrieve(RunMap.java:226) at hudson.model.RunMap.retrieve(RunMap.java:59) at jenkins.model.lazy.AbstractLazyLoadRunMap.load(AbstractLazyLoadRunMap.java:657) at jenkins.model.lazy.AbstractLazyLoadRunMap.load(AbstractLazyLoadRunMap.java:619) at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:358) at hudson.model.AbstractBuild.getPreviousBuild(AbstractBuild.java:214) at hudson.tasks.Fingerprinter$FingerprintAction.onLoad(Fingerprinter.java:349) at hudson.model.Run.onLoad(Run.java:319) at hudson.model.RunMap.retrieve(RunMap.java:226) at hudson.model.RunMap.retrieve(RunMap.java:59) at jenkins.model.lazy.AbstractLazyLoadRunMap.load(AbstractLazyLoadRunMap.java:657) at jenkins.model.lazy.AbstractLazyLoadRunMap.load(AbstractLazyLoadRunMap.java:619) at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:358) at hudson.model.AbstractBuild.getPreviousBuild(AbstractBuild.java:214) at hudson.tasks.Fingerprinter$FingerprintAction.onLoad(Fingerprinter.java:349) at hudson.model.Run.onLoad(Run.java:319) at hudson.model.RunMap.retrieve(RunMap.java:226) at hudson.model.RunMap.retrieve(RunMap.java:59) at jenkins.model.lazy.AbstractLazyLoadRunMap.load(AbstractLazyLoadRunMap.java:657) at jenkins.model.lazy.AbstractLazyLoadRunMap.load(AbstractLazyLoadRunMap.java:619) at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:358) at hudson.model.AbstractBuild.getPreviousBuild(AbstractBuild.java:214) at hudson.tasks.Fingerprinter$FingerprintAction.onLoad(Fingerprinter.java:349) at hudson.model.Run.onLoad(Run.java:319) at hudson.model.RunMap.retrieve(RunMap.java:226) at hudson.model.RunMap.retrieve(RunMap.java:59) at jenkins.model.lazy.AbstractLazyLoadRunMap.load(AbstractLazyLoadRunMap.java:657) at jenkins.model.lazy.AbstractLazyLoadRunMap.load(AbstractLazyLoadRunMap.java:619) at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:358) at hudson.model.AbstractBuild.getPreviousBuild(AbstractBuild.java:214) at hudson.model.AbstractBuild.getPreviousBuild(AbstractBuild.java:101) at hudson.model.RunMap$1.next(RunMap.java:107) at hudson.model.RunMap$1.next(RunMap.java:96) at hudson.util.Iterators$2$1.next(Iterators.java:204) at hudson.widgets.HistoryWidget.updateFirstTransientBuildKey(HistoryWidget.java:110) at hudson.widgets.HistoryWidget.getRenderList(HistoryWidget.java:139) ... 126 more Which seems related to another issue JENKINS-16845 regarding the specific NPE on AbstractBuild.getPreviousBuild.

          Good detective work all!

          Caused by: java.lang.NullPointerException
                  at hudson.model.AbstractBuild.getPreviousBuild(AbstractBuild.java:214)
                  at hudson.tasks.Fingerprinter$FingerprintAction.onLoad(Fingerprinter.java:349)
                  at hudson.model.Run.onLoad(Run.java:319)
                  at hudson.model.RunMap.retrieve(RunMap.java:226)
                  at hudson.model.RunMap.retrieve(RunMap.java:59)
                  at jenkins.model.lazy.AbstractLazyLoadRunMap.load(AbstractLazyLoadRunMap.java:657)
          

          I think this relates to the following in AbstractBuild

                          R pb = getParent().builds.search(number-1, Direction.DESC);
          

          This could be builds not being initialised as per the suggested JENKINS-16845 or it could be getParent() returning null.

          getParent() returns the transient project value from Run.java

          protected transient final JobT project;
          

          @Jose Sa can you confirm what version you saw this error on.

          Richard Mortimer added a comment - Good detective work all! Caused by: java.lang.NullPointerException at hudson.model.AbstractBuild.getPreviousBuild(AbstractBuild.java:214) at hudson.tasks.Fingerprinter$FingerprintAction.onLoad(Fingerprinter.java:349) at hudson.model.Run.onLoad(Run.java:319) at hudson.model.RunMap.retrieve(RunMap.java:226) at hudson.model.RunMap.retrieve(RunMap.java:59) at jenkins.model.lazy.AbstractLazyLoadRunMap.load(AbstractLazyLoadRunMap.java:657) I think this relates to the following in AbstractBuild R pb = getParent().builds.search(number-1, Direction.DESC); This could be builds not being initialised as per the suggested JENKINS-16845 or it could be getParent() returning null. getParent() returns the transient project value from Run.java protected transient final JobT project; @Jose Sa can you confirm what version you saw this error on.

          Jose Sa added a comment - - edited

          I'm with core 1.505 now. Tried more recent versions like 1.506 and 1.508, but had to revert back due worse problems caused by NPE not being able to load jobs like the previous issue, but now extended to Promote plugin JENKINS-17381 along with weird changed in SSH Slaves plugin credentials definitions that got all messed up.

          Jose Sa added a comment - - edited I'm with core 1.505 now. Tried more recent versions like 1.506 and 1.508, but had to revert back due worse problems caused by NPE not being able to load jobs like the previous issue, but now extended to Promote plugin JENKINS-17381 along with weird changed in SSH Slaves plugin credentials definitions that got all messed up.

          Jose Sa added a comment - - edited

          After some analysis of my previous script I realized it contained a lot of false positives possibly due to years of accumulated history of broken builds, java crashes, wrong rsync of jobs between servers, etc, that lead to the following most common problems:

          • numbered links pointing to removed build directories (solution: remove links)
          • build directories without build.xml file (solution: remove directories)
          • build directories without numbered links (solution: create links)
          • build directories not in date format (solution: delete directories)

          Since these happen often I created some scripts to:

          • Detect those common problems with optional cleanup (passing argument '-c') - check_invalid_builds.sh
          • Checking the missing builds as given as excerpt above - check_missing_builds.sh

          NOTE: to effectively apply the cleanups first run the script without '-c' to run in preview mode and review the suggestions, when ready to apply them shutdown Jenkins first so that there are no files pending in memory to be written to file system.

          Tested in RHEL6 and Solaris 10 (with gnu utils)

          Jose Sa added a comment - - edited After some analysis of my previous script I realized it contained a lot of false positives possibly due to years of accumulated history of broken builds, java crashes, wrong rsync of jobs between servers, etc, that lead to the following most common problems: numbered links pointing to removed build directories (solution: remove links) build directories without build.xml file (solution: remove directories) build directories without numbered links (solution: create links) build directories not in date format (solution: delete directories) Since these happen often I created some scripts to: Detect those common problems with optional cleanup (passing argument '-c') - check_invalid_builds.sh Checking the missing builds as given as excerpt above - check_missing_builds.sh NOTE: to effectively apply the cleanups first run the script without '-c' to run in preview mode and review the suggestions, when ready to apply them shutdown Jenkins first so that there are no files pending in memory to be written to file system. Tested in RHEL6 and Solaris 10 (with gnu utils)

          Jesse Glick added a comment -

          Seems to be the same as JENKINS-16845.

          Jesse Glick added a comment - Seems to be the same as JENKINS-16845 .

          Adam Westhusing added a comment - - edited

          The shell scripts don't work when the jobs have spaces in the names. What modifications need to be made to the scripts to adapt it to be able to work when there are spaces in the job names?

          Additionally, running this script in Cygwin on Windows is generating output that indicates EVERY build has a missing build link.

          Adam Westhusing added a comment - - edited The shell scripts don't work when the jobs have spaces in the names. What modifications need to be made to the scripts to adapt it to be able to work when there are spaces in the job names? Additionally, running this script in Cygwin on Windows is generating output that indicates EVERY build has a missing build link.

          Jose Sa added a comment -

          Allowing jobs with spaces in names does complicate things, making it harder (but not impossible) to handle via script. I prefer not to allow any spaces using "Create Job Advanced" plugin to enforce some common rules.

          To allow spaces, basically any path that contains the job name should be enclosed with double quotes either on variable assignment as well in usage. Since command "find" doesn't like to have wildcard enclosed in double quotes, that can be split into first a change directory and then the find command with wildcard. I uploaded new versions of scripts but it didn't seem to have replaced the old ones when uploaded, so refer to the most recent ones.

          Jenkins on Windows also seems to have a different behavior since it doesn't seem to use any symbolic links (not visible on cygwin anyway with 1.484), so the the link related checks probably are not applicable on Windows.

          Jose Sa added a comment - Allowing jobs with spaces in names does complicate things, making it harder (but not impossible) to handle via script. I prefer not to allow any spaces using "Create Job Advanced" plugin to enforce some common rules. To allow spaces, basically any path that contains the job name should be enclosed with double quotes either on variable assignment as well in usage. Since command "find" doesn't like to have wildcard enclosed in double quotes, that can be split into first a change directory and then the find command with wildcard. I uploaded new versions of scripts but it didn't seem to have replaced the old ones when uploaded, so refer to the most recent ones. Jenkins on Windows also seems to have a different behavior since it doesn't seem to use any symbolic links (not visible on cygwin anyway with 1.484), so the the link related checks probably are not applicable on Windows.

          Thanks for updating the script, but I suppose the behavior is different on Windows. Jenkins has never created symbolic links on Windows until upgrading from 1.506 to 1.509. In the logs, Jenkins is complaining all over the place that it can't find directories but the build directories are actually there and nothing appears to be wrong when bringing up the specific build being complained about in the Jenkins logs.

          Adam Westhusing added a comment - Thanks for updating the script, but I suppose the behavior is different on Windows. Jenkins has never created symbolic links on Windows until upgrading from 1.506 to 1.509. In the logs, Jenkins is complaining all over the place that it can't find directories but the build directories are actually there and nothing appears to be wrong when bringing up the specific build being complained about in the Jenkins logs.

          Jose Sa added a comment -

          You can remove from the script the link sections in Windows environment. But probably in some past upgrade it may have been possible that this same behaviour was accidentally induced in Unix environments which could explain why so many situations similar to that.

          Still after running the cleanup and running for a few days I don't the problem happening so often now. Actually i saw it happening right in the the first day, but somehow it "recovered" without doing a reload from disk or restart.

          Jose Sa added a comment - You can remove from the script the link sections in Windows environment. But probably in some past upgrade it may have been possible that this same behaviour was accidentally induced in Unix environments which could explain why so many situations similar to that. Still after running the cleanup and running for a few days I don't the problem happening so often now. Actually i saw it happening right in the the first day, but somehow it "recovered" without doing a reload from disk or restart.

          I just saw the same issue on version 1.511. It occurred after renaming a job and running that job with a scheduled build. Oddly, it did not occur with the first few manual builds.

          Reece Johnston added a comment - I just saw the same issue on version 1.511. It occurred after renaming a job and running that job with a scheduled build. Oddly, it did not occur with the first few manual builds.

          Seeing this problem coming back on LTS 1.509.2. I have no idea how to debug...

          Martin d'Anjou added a comment - Seeing this problem coming back on LTS 1.509.2. I have no idea how to debug...

          Stefan Prietl added a comment -

          Same problem with 1.509.1 LTS.

          We are using two Jenkins instances with the same version right now, and this issue seems to affect only the one where the builds are executed on slaves only. Additionally only builds which where renamed and are triggered automatically by croned upstream projects are having this issue.

          The only thing I noticed in the logs was the following:

          Aug 01, 2013 12:44:28 AM hudson.model.Run execute
          INFO: some-build #15 main build action completed: SUCCESS
          Aug 01, 2013 12:21:20 AM hudson.model.listeners.RunListener report
          WARNING: RunListener failed
          java.lang.NullPointerException
          

          This happened right before/during every build which "lost" builds. However, this does not occur on the Jenkins instance which builds on master only.
          Unfortunately there was no trace attached to the NullPointerException in the log, but I'll try to get one with a higher log output setting. Maybe it's connected to issue JENKINS-18823.

          Stefan Prietl added a comment - Same problem with 1.509.1 LTS. We are using two Jenkins instances with the same version right now, and this issue seems to affect only the one where the builds are executed on slaves only . Additionally only builds which where renamed and are triggered automatically by croned upstream projects are having this issue. The only thing I noticed in the logs was the following: Aug 01, 2013 12:44:28 AM hudson.model.Run execute INFO: some-build #15 main build action completed: SUCCESS Aug 01, 2013 12:21:20 AM hudson.model.listeners.RunListener report WARNING: RunListener failed java.lang.NullPointerException This happened right before/during every build which "lost" builds. However, this does not occur on the Jenkins instance which builds on master only. Unfortunately there was no trace attached to the NullPointerException in the log, but I'll try to get one with a higher log output setting. Maybe it's connected to issue JENKINS-18823 .

          binary added a comment -

          Status: Resolved
          Resolution: Duplicate


          If it is resolved, why is it still appearing?

          We have it for multiple jobs, including jobs that are triggered manually. Core 1.526, master on Ubuntu 10.04, slaves on different platforms (linux, windows, os x; all are affected).

          binary added a comment - Status: Resolved Resolution: Duplicate If it is resolved, why is it still appearing? We have it for multiple jobs, including jobs that are triggered manually. Core 1.526, master on Ubuntu 10.04, slaves on different platforms (linux, windows, os x; all are affected).

          Jesse Glick added a comment -

          @binary: this was marked as a duplicate of JENKINS-16845, a confirmed and fixed bug, based on a detailed match of symptoms with that bug. If you have a general issue with builds disappearing from history, it is probably a different cause and should be opened as a separate bug report with any steps to reproduce; or, failing that, exhaustive diagnostics such as warnings in the log, analysis of the contents of the builds directory, anomalies in build.xml files, etc.

          Jesse Glick added a comment - @binary: this was marked as a duplicate of JENKINS-16845 , a confirmed and fixed bug, based on a detailed match of symptoms with that bug. If you have a general issue with builds disappearing from history, it is probably a different cause and should be opened as a separate bug report with any steps to reproduce; or, failing that, exhaustive diagnostics such as warnings in the log, analysis of the contents of the builds directory, anomalies in build.xml files, etc.

          This is more than funny! I suggest you to open a new section for bugs: "Builds disappearing from history".
          Let's hope the 'reload configuration workarround' will work for next years.

          Voiculescu Bogdan Alexandru added a comment - This is more than funny! I suggest you to open a new section for bugs: "Builds disappearing from history". Let's hope the 'reload configuration workarround' will work for next years.

          eguess74 added a comment -

          @Jesse Glick
          I tend to agree with others that it seems unreasonable to mark this issue as a resolved and duplicate, because obviously the fix of the JENKINS-16845 did not fix this issue. So, by asking users to resubmit another one we are adding unnecessary burden on users + "loosing all the information" that we have already gathered.
          Clearly the issue persist and by some reports even to the most current release (529) - so this is hardly a dup.

          I for one see this issue on the 513 - two releases after 16845 was pushed.
          But we have two instances and one is currently on LTS 509.1 and it doesn't seem to have such problem at all. The major difference between those - promoted builds plugin. We don't have it on LTS instance and we do have it on the 513.

          It would be nice if other reporters could confirm if they use promoted builds - if so may be this is the culprit?
          Just throwing an idea...

          eguess74 added a comment - @Jesse Glick I tend to agree with others that it seems unreasonable to mark this issue as a resolved and duplicate, because obviously the fix of the JENKINS-16845 did not fix this issue. So, by asking users to resubmit another one we are adding unnecessary burden on users + "loosing all the information" that we have already gathered. Clearly the issue persist and by some reports even to the most current release (529) - so this is hardly a dup. I for one see this issue on the 513 - two releases after 16845 was pushed. But we have two instances and one is currently on LTS 509.1 and it doesn't seem to have such problem at all. The major difference between those - promoted builds plugin. We don't have it on LTS instance and we do have it on the 513. It would be nice if other reporters could confirm if they use promoted builds - if so may be this is the culprit? Just throwing an idea...

          I do not use promoted builds and I do see the issue.

          Martin d'Anjou added a comment - I do not use promoted builds and I do see the issue.

          wbauer added a comment -

          I see the issue on 1.525 still and don't use promoted builds.

          wbauer added a comment - I see the issue on 1.525 still and don't use promoted builds.

          eguess74 added a comment -

          ok, seems like the promoted builds is not the reason... something else then...

          eguess74 added a comment - ok, seems like the promoted builds is not the reason... something else then...

          Jesse Glick added a comment -

          There may be (in fact probably are) multiple issues with similar-seeming symptoms; without a way to reproduce it may not be possible to tell. JENKINS-19418 is the one issue in this area with a diagnosis known to me.

          Jesse Glick added a comment - There may be (in fact probably are) multiple issues with similar-seeming symptoms; without a way to reproduce it may not be possible to tell. JENKINS-19418 is the one issue in this area with a diagnosis known to me.

          eguess74 added a comment -

          The simplest workaround to get the full list of the builds that i found is to use the "trend" link on the top right corner of the build history. All the builds are listed there correctly even if they are not shown in the main history table. May be that fact can help pinpoint faulty code?

          eguess74 added a comment - The simplest workaround to get the full list of the builds that i found is to use the "trend" link on the top right corner of the build history. All the builds are listed there correctly even if they are not shown in the main history table. May be that fact can help pinpoint faulty code?

          Jesse Glick added a comment -

          @eguess74: interesting. Means RunList.iterator() is working, though that is what the main history list (HistoryWidget) should be showing too. Might not actually matter which widget is being used to search through the list—could simply be that the first iteration triggers loading of something which does not get returned until the second time.

          Really we need better diagnostics in this area. JENKINS-19400 might help.

          Jesse Glick added a comment - @eguess74: interesting. Means RunList.iterator() is working, though that is what the main history list ( HistoryWidget ) should be showing too. Might not actually matter which widget is being used to search through the list—could simply be that the first iteration triggers loading of something which does not get returned until the second time. Really we need better diagnostics in this area. JENKINS-19400 might help.

          @Jesse
          It's not about first time load as reloading history page several times after displaying the trend page with full list still produces partial result for history widget. See it here: http://s13.postimg.org/kaccjxc07/Clipboard_1.gif
          The interesting thing, however, is that history widget shows exactly "Max # of builds to keep" builds. There are more than this numbers of builds due to keep forever and trend finds them. Setting "Max # of builds to keep" to bigger value, however, doesn't make the rest of builds appear in history widget. Might be something in xml that is corrupted because of "Max # of builds to keep" coupled with lazy loading. Just a guess...

          Nikolay Martynov added a comment - @Jesse It's not about first time load as reloading history page several times after displaying the trend page with full list still produces partial result for history widget. See it here: http://s13.postimg.org/kaccjxc07/Clipboard_1.gif The interesting thing, however, is that history widget shows exactly "Max # of builds to keep" builds. There are more than this numbers of builds due to keep forever and trend finds them. Setting "Max # of builds to keep" to bigger value, however, doesn't make the rest of builds appear in history widget. Might be something in xml that is corrupted because of "Max # of builds to keep" coupled with lazy loading. Just a guess...

          iikka eklund added a comment -

          We are having exactly the same problem with Jenkins version 1.547
          This error is closed but in which Jenkins version this is fixed?

          iikka eklund added a comment - We are having exactly the same problem with Jenkins version 1.547 This error is closed but in which Jenkins version this is fixed?

          Alex Ouzounis added a comment -

          Hi all,

          I am seeing the same problem occasionally and either restarting or reloading from disk fixes the problem.

          I have a feeling that its related with:

          "WARNING: Multiple builds have the same number: "

          because every time I delete conflicting builds it goes away.

          Alex Ouzounis added a comment - Hi all, I am seeing the same problem occasionally and either restarting or reloading from disk fixes the problem. I have a feeling that its related with: "WARNING: Multiple builds have the same number: " because every time I delete conflicting builds it goes away.

          Alex Ouzounis added a comment -

          Problem no longer present in the latest versions ie 1.570

          Alex Ouzounis added a comment - Problem no longer present in the latest versions ie 1.570

          eguess74 added a comment -

          I'm seeing it happening right now on Jenkins 1.578
          I had a build that i triggered using gerrit trigger (query and trigger gerrit patches)
          It appeared in the GUI, I saw it running in the the build queue, it sent an email about build being fixed, registered the result with gerrit, but no matter how i'm trying to access it the link to the build number leads to 404. Build is present on the file system as well.

          eguess74 added a comment - I'm seeing it happening right now on Jenkins 1.578 I had a build that i triggered using gerrit trigger (query and trigger gerrit patches) It appeared in the GUI, I saw it running in the the build queue, it sent an email about build being fixed, registered the result with gerrit, but no matter how i'm trying to access it the link to the build number leads to 404. Build is present on the file system as well.

          Jesse Glick added a comment -

          eguess74 if you are using the Gerrit Trigger plugin, you are probably seeing JENKINS-23152.

          Jesse Glick added a comment - eguess74 if you are using the Gerrit Trigger plugin, you are probably seeing JENKINS-23152 .

          eguess74 added a comment -

          Jesse,

          Thanks for pointing me to the right direction!

          eguess74 added a comment - Jesse, Thanks for pointing me to the right direction!

          Frank Canova added a comment -

          I'm seeing this same issue when changing between v1.590 and v1.610. Is there a work-around or way to recreate the build.xml files? The history directories are intact but history has disappeared from the web view.

          Frank Canova added a comment - I'm seeing this same issue when changing between v1.590 and v1.610. Is there a work-around or way to recreate the build.xml files? The history directories are intact but history has disappeared from the web view.

          Daniel Beck added a comment -

          I'm seeing this same issue when changing between v1.590 and v1.610. Is there a work-around or way to recreate the build.xml files? The history directories are intact but history has disappeared from the web view.

          Unrelated issue, almost sure this is the storage layout change from http://jenkins-ci.org/changelog#v1.597 – Jenkins provides information how to downgrade the storage format in its logs, and on the UI of 1.597+. As there are 50+ watchers here, and your issue has nothing to do with this one, please move any follow-up discussion to the users mailing list or IRC.

          Daniel Beck added a comment - I'm seeing this same issue when changing between v1.590 and v1.610. Is there a work-around or way to recreate the build.xml files? The history directories are intact but history has disappeared from the web view. Unrelated issue, almost sure this is the storage layout change from http://jenkins-ci.org/changelog#v1.597 – Jenkins provides information how to downgrade the storage format in its logs, and on the UI of 1.597+. As there are 50+ watchers here, and your issue has nothing to do with this one, please move any follow-up discussion to the users mailing list or IRC.

            Unassigned Unassigned
            alex01ves Alex Vesely
            Votes:
            22 Vote for this issue
            Watchers:
            57 Start watching this issue

              Created:
              Updated:
              Resolved: