Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-25975

More reliable test infrastructure

    XMLWordPrintable

Details

    Description

      There are too many @RandomlyFails tests. SemaphoreStep should be used more consistently in place of WatchYourStep and waiting for the execution to suspend. (Already prototyped in WorkflowTest.env.)

      Also any build logs should be streamed immediately to stderr, rather than forcing the test to include the current log in every assertion message observed to fail. (In combination with SemaphoreStep, this should also more reliably flush recent output: WorkflowRun flushes logs when new steps are run.)

      Attachments

        Issue Links

          is related to

          Bug - A problem which impairs or prevents the functions of the product. JENKINS-26398 Fix createSlave for RestartableJenkinsRule

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. JENKINS-26399 Utility method in JenkinsRule to wait for build to complete

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          links to

          Web Link PR 59

          Web Link PR 92

          Activity

            Ascending order - Click to sort in descending order
            jglick Jesse Glick added a comment -

            Note to self: if deleting WatchYourStep, can also close PR 10.

            jglick Jesse Glick added a comment - Note to self: if deleting WatchYourStep , can also close PR 10 .
            jglick Jesse Glick added a comment -

            Implementing some of these things in PR 59 just to get things to pass on Windows, which is easier using SemaphoreStep than WatchYourStep because we do not need to deal with local file paths embedded in the script.

            One class of random failure I think I have diagnosed: CpsThreadGroup.notifyNewHead notifies listeners asynchronously, so WorkflowRun.copyLogs does not run instantly after a step completes, and so the build log may not immediately show its output. CpsFlowExecution.waitForSuspension seems to solve that.

            jglick Jesse Glick added a comment - Implementing some of these things in PR 59 just to get things to pass on Windows, which is easier using SemaphoreStep than WatchYourStep because we do not need to deal with local file paths embedded in the script. One class of random failure I think I have diagnosed: CpsThreadGroup.notifyNewHead notifies listeners asynchronously, so WorkflowRun.copyLogs does not run instantly after a step completes, and so the build log may not immediately show its output. CpsFlowExecution.waitForSuspension seems to solve that.
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Jesse Glick
            Path:
            test/src/main/java/org/jvnet/hudson/test/BuildWatcher.java
            test/src/main/java/org/jvnet/hudson/test/JenkinsRule.java
            http://jenkins-ci.org/commit/jenkins/9ac2257d0a3fa86928267a82d69d55c5a4e423c1
            Log:
            [FIXED JENKINS-26399] JENKINS-25975 Merged #1609.

            Compare: https://github.com/jenkinsci/jenkins/compare/b02e71e3dfcb...9ac2257d0a3f

            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: test/src/main/java/org/jvnet/hudson/test/BuildWatcher.java test/src/main/java/org/jvnet/hudson/test/JenkinsRule.java http://jenkins-ci.org/commit/jenkins/9ac2257d0a3fa86928267a82d69d55c5a4e423c1 Log: [FIXED JENKINS-26399] JENKINS-25975 Merged #1609. Compare: https://github.com/jenkinsci/jenkins/compare/b02e71e3dfcb...9ac2257d0a3f
            jglick Jesse Glick added a comment -

            I am occasionally getting test failures whereby a sh step is evidently run but its output is not included in the log. I suspect a race condition between the GraphListener calling copyLogs and the one calling PrintStream.close when DefaultStepContext makes a LogActionImpl.

            jglick Jesse Glick added a comment - I am occasionally getting test failures whereby a sh step is evidently run but its output is not included in the log. I suspect a race condition between the GraphListener calling copyLogs and the one calling PrintStream.close when DefaultStepContext makes a LogActionImpl .
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Jesse Glick
            Path:
            aggregator/src/test/java/org/jenkinsci/plugins/workflow/WorkflowTest.java
            durable-task-step/src/main/java/org/jenkinsci/plugins/workflow/steps/durable_task/DurableTaskStep.java
            http://jenkins-ci.org/commit/workflow-plugin/27e5b23046ae4e913edff0fa05256fdfafff0029
            Log:
            JENKINS-25975 Seem to have fixed a race condition whereby quick `sh` steps would sometimes not print all of their output.

            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: aggregator/src/test/java/org/jenkinsci/plugins/workflow/WorkflowTest.java durable-task-step/src/main/java/org/jenkinsci/plugins/workflow/steps/durable_task/DurableTaskStep.java http://jenkins-ci.org/commit/workflow-plugin/27e5b23046ae4e913edff0fa05256fdfafff0029 Log: JENKINS-25975 Seem to have fixed a race condition whereby quick `sh` steps would sometimes not print all of their output.
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Jesse Glick
            Path:
            aggregator/src/test/java/org/jenkinsci/plugins/workflow/SingleJobTestBase.java
            aggregator/src/test/java/org/jenkinsci/plugins/workflow/WorkflowRunTest.java
            aggregator/src/test/java/org/jenkinsci/plugins/workflow/WorkflowTest.java
            support/src/test/java/org/jenkinsci/plugins/workflow/test/steps/WatchYourStep.java
            http://jenkins-ci.org/commit/workflow-plugin/d3ba64ddd6eaaacad401624d5b17809ebbec9b30
            Log:
            [FIXED JENKINS-25975] Finally able to delete WatchYourStep; using SemaphoreStep instead.

            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: aggregator/src/test/java/org/jenkinsci/plugins/workflow/SingleJobTestBase.java aggregator/src/test/java/org/jenkinsci/plugins/workflow/WorkflowRunTest.java aggregator/src/test/java/org/jenkinsci/plugins/workflow/WorkflowTest.java support/src/test/java/org/jenkinsci/plugins/workflow/test/steps/WatchYourStep.java http://jenkins-ci.org/commit/workflow-plugin/d3ba64ddd6eaaacad401624d5b17809ebbec9b30 Log: [FIXED JENKINS-25975] Finally able to delete WatchYourStep; using SemaphoreStep instead.
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Jesse Glick
            Path:
            test/src/main/java/org/jvnet/hudson/test/BuildWatcher.java
            test/src/main/java/org/jvnet/hudson/test/JenkinsRule.java
            http://jenkins-ci.org/commit/jenkins-test-harness/fc7807ec16c4c1d495f293332bf5890f2a3fa282
            Log:
            [FIXED JENKINS-26399] JENKINS-25975 Merged #1609.

            Originally-Committed-As: 9ac2257d0a3fa86928267a82d69d55c5a4e423c1

            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: test/src/main/java/org/jvnet/hudson/test/BuildWatcher.java test/src/main/java/org/jvnet/hudson/test/JenkinsRule.java http://jenkins-ci.org/commit/jenkins-test-harness/fc7807ec16c4c1d495f293332bf5890f2a3fa282 Log: [FIXED JENKINS-26399] JENKINS-25975 Merged #1609. Originally-Committed-As: 9ac2257d0a3fa86928267a82d69d55c5a4e423c1
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Jesse Glick
            Path:
            aggregator/src/test/java/org/jenkinsci/plugins/workflow/WorkflowTest.java
            http://jenkins-ci.org/commit/workflow-cps-plugin/47ff7954de7713b74ed094a37863046cdedbee28
            Log:
            JENKINS-25975 Seem to have fixed a race condition whereby quick `sh` steps would sometimes not print all of their output.
            Originally-Committed-As: 27e5b23046ae4e913edff0fa05256fdfafff0029

            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: aggregator/src/test/java/org/jenkinsci/plugins/workflow/WorkflowTest.java http://jenkins-ci.org/commit/workflow-cps-plugin/47ff7954de7713b74ed094a37863046cdedbee28 Log: JENKINS-25975 Seem to have fixed a race condition whereby quick `sh` steps would sometimes not print all of their output. Originally-Committed-As: 27e5b23046ae4e913edff0fa05256fdfafff0029
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Jesse Glick
            Path:
            aggregator/src/test/java/org/jenkinsci/plugins/workflow/SingleJobTestBase.java
            aggregator/src/test/java/org/jenkinsci/plugins/workflow/WorkflowTest.java
            http://jenkins-ci.org/commit/workflow-cps-plugin/78b05a55574421d0d9383eab50f926d3954c7dd6
            Log:
            [FIXED JENKINS-25975] Finally able to delete WatchYourStep; using SemaphoreStep instead.
            Originally-Committed-As: d3ba64ddd6eaaacad401624d5b17809ebbec9b30

            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: aggregator/src/test/java/org/jenkinsci/plugins/workflow/SingleJobTestBase.java aggregator/src/test/java/org/jenkinsci/plugins/workflow/WorkflowTest.java http://jenkins-ci.org/commit/workflow-cps-plugin/78b05a55574421d0d9383eab50f926d3954c7dd6 Log: [FIXED JENKINS-25975] Finally able to delete WatchYourStep; using SemaphoreStep instead. Originally-Committed-As: d3ba64ddd6eaaacad401624d5b17809ebbec9b30
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Jesse Glick
            Path:
            aggregator/src/test/java/org/jenkinsci/plugins/workflow/WorkflowRunTest.java
            http://jenkins-ci.org/commit/workflow-job-plugin/360c2d2337098108e8c3ecee329e8cc0fd4512f3
            Log:
            [FIXED JENKINS-25975] Finally able to delete WatchYourStep; using SemaphoreStep instead.
            Originally-Committed-As: d3ba64ddd6eaaacad401624d5b17809ebbec9b30

            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: aggregator/src/test/java/org/jenkinsci/plugins/workflow/WorkflowRunTest.java http://jenkins-ci.org/commit/workflow-job-plugin/360c2d2337098108e8c3ecee329e8cc0fd4512f3 Log: [FIXED JENKINS-25975] Finally able to delete WatchYourStep; using SemaphoreStep instead. Originally-Committed-As: d3ba64ddd6eaaacad401624d5b17809ebbec9b30

            People

              jglick Jesse Glick
              jglick Jesse Glick
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: