Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-73380

discoverReferenceJob doesn't update if other job completes in the meantime

    • Icon: Improvement Improvement
    • Resolution: Fixed
    • Icon: Major Major
    • forensics-api-plugin
    • None
    • Jenkins: 2.452.1
      OS: Linux - 4.18.0-553.5.1.el8_10.x86_64
      Java: 17.0.11 - Red Hat, Inc. (OpenJDK 64-Bit Server VM)
      ---
      forensics-api:2.4.0
      warnings-ng:11.3.0

      {{discoverReferenceBuild }}will not update to the latest completed build after being called for the first time.

      Scenario:

      1. Build A is completed
      2. Build B starts
      3. Build C starts and calls discoverReferenceJob -> points to A
      4. Build B publishes some report with the same ID as used in the other jobs
      5. Build C calls discoverReferenceJob again -> Reference Build switches to B but WarningsNG still uses A.

      Behavior before the switch to Forensics API was that Job C's delta would have pointed to Job B as soon as it published the report. Now, even after calling discoverReferenceJob, it doesn't update.

      The reference build is correctly set to the latest completed one, but warningsNG sticks with the one found when discoverReferenceBuild was called the first time.

      -> Updating WarningsNG breaks some pipeline logic from us.

          [JENKINS-73380] discoverReferenceJob doesn't update if other job completes in the meantime

          Ulli Hafner added a comment -

          I am assuming you mean build and not job in your example?

          Are you using discoverReferenceBuild multiple times in one build? The action is supposed to be called only once per build.

          Can you please add the logging output of the warnings plugin?

          Ulli Hafner added a comment - I am assuming you mean build and not job in your example? Are you using discoverReferenceBuild multiple times in one build? The action is supposed to be called only once per build. Can you please add the logging output of the warnings plugin?

          Jan added a comment - - edited

          Thanks for your response!

          Yes, sorry - build.

          discoverReferenceBuild is indeed being called multiple times per build - I would have expected that calling it again would allow updating the reference (just as it behaved before without Forensics API, where the reference updated automatically)

          I attached the warningsNG output, including at the very end the reference output. Interestingly, the log shows that it takes 1643 (just once) and the UI shows that it updated:

          (rev1.1643 = # 1257, rev1.1644 = # 1258, screenshots from rev1.1645 = # 1259)

           

          This is a significant problem, as in the other issue (JENKINS-73328): if the in-progress report finds that issues were fixed, the already running next build would not detect this and show a wrong delta and thus wrong blame. This is a big no-go for audit-sensitive projects. One run may show that its changes fixed something even though the previous run was responsible.

           

          Jan added a comment - - edited Thanks for your response! Yes, sorry - build. discoverReferenceBuild is indeed being called multiple times per build - I would have expected that calling it again would allow updating the reference (just as it behaved before without Forensics API, where the reference updated automatically) I attached the warningsNG output, including at the very end the reference output. Interestingly, the log shows that it takes 1643 (just once) and the UI shows that it updated: (rev1.1643 = # 1257, rev1.1644 = # 1258, screenshots from rev1.1645 = # 1259)   This is a significant problem, as in the other issue ( JENKINS-73328 ): if the in-progress report finds that issues were fixed, the already running next build would not detect this and show a wrong delta and thus wrong blame. This is a big no-go for audit-sensitive projects. One run may show that its changes fixed something even though the previous run was responsible.  

          Ulli Hafner added a comment -

          The reference build action (and thus the corresponding values) are cached by Jenkins and therefore the new reference build will not be available to the current process.

          Somehow your pipeline contradicts the idea behind the reference build concept: create a unique and persistent build that will be used as reference (or target) for all post build steps. In your case you have a moving target.

          Ulli Hafner added a comment - The reference build action (and thus the corresponding values) are cached by Jenkins and therefore the new reference build will not be available to the current process. Somehow your pipeline contradicts the idea behind the reference build concept: create a unique and persistent build that will be used as reference (or target) for all post build steps. In your case you have a moving target.

          Jan added a comment -

          Hi Ulli,
          Thanks for your response!

          In our case, the pipeline is a continuous testing pipeline running several testsuites contained in a particular "revision/baseline" of a repository. The developers update the repository frequently, pushing manually a new revision for each set of changes. The requirements are to test every revision. Each run takes around 3-4h which is too long to wait for completion (-> sequentially running builds).

          Hence concurrent builds were activated, however with a synchronization mechanism ensuring the calling sequence of recordIssues for a particular id between different builds is respected (sequential as per repository revisions tested). This was taking advantage of the previous reference build logic.

          There are probably other ways to design pipeline(s) now to get equivalent behavior with the new plugin logic but this means a major rework of 50+ pipelines and most importantly a break in the reporting consistency in ongoing projects.

           

          I would propose adding a "ignoreCache" parameter for the discoverReferenceBuild to support the previous behaviour. I am not sure if that would work since I was not yet able to fully understand the reference logic in the source code so I can't make more detailed proposals so far.

           

          Thanks,
          Jan

          Jan added a comment - Hi Ulli, Thanks for your response! In our case, the pipeline is a continuous testing pipeline running several testsuites contained in a particular "revision/baseline" of a repository. The developers update the repository frequently, pushing manually a new revision for each set of changes. The requirements are to test every revision. Each run takes around 3-4h which is too long to wait for completion (-> sequentially running builds). Hence concurrent builds were activated, however with a synchronization mechanism ensuring the calling sequence of recordIssues for a particular id between different builds is respected (sequential as per repository revisions tested). This was taking advantage of the previous reference build logic. There are probably other ways to design pipeline(s) now to get equivalent behavior with the new plugin logic but this means a major rework of 50+ pipelines and most importantly a break in the reporting consistency in ongoing projects.   I would propose adding a "ignoreCache" parameter for the discoverReferenceBuild to support the previous behaviour. I am not sure if that would work since I was not yet able to fully understand the reference logic in the source code so I can't make more detailed proposals so far.   Thanks, Jan

          Jan added a comment -

          Thank you, drulli - I appreciate it!!

          Jan added a comment - Thank you, drulli - I appreciate it!!

          Ulli Hafner added a comment -

          Sorry, seems that I closed the wrong issue, the fix is actually for JENKINS-73328.

          Ulli Hafner added a comment - Sorry, seems that I closed the wrong issue, the fix is actually for JENKINS-73328 .

            drulli Ulli Hafner
            meiswjn Jan
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: