• Evergreen - Milestone 2

      Problem statement

      JENKINS-52581 has introduced some basic smoke testing for the AWS flavor, but we can (and do) still miss a bunch of errors that will only ever be seen when spawning an actual instance and trying to trigger a build.

      Expected

      We must run an actual AWS flavor provisioning during each PR build. We must then trigger a build and check the agent does provision and the build succeeds.

          [JENKINS-54293] Automatically run AWS flavor tests in CI

          Baptiste Mathus created issue -
          Baptiste Mathus made changes -
          Epic Link New: JENKINS-49853 [ 188837 ]
          Baptiste Mathus made changes -
          Assignee Original: Baptiste Mathus [ batmat ]
          Baptiste Mathus made changes -
          Description Original: We must have some level of automated testing ASAP.

          My plan is to introduce at least a third branch to test the AWS flavor locally: the Jenkins master should at least start (which can well be not the case with a missing plugin, or wrong CasC config files).

          ---

          This is important for obvious general reasons, but also more specifically because for instance [~jglick] is about to file a PR for the AWS flavor, and it's currently very hard to test if you are not involved deeply in the project. Some smoke tests should at least be available/done through the now usual {{make check}}
          New: h3. Problem statement

          JENKINS-52581 has introduced some basic smoke testing for the AWS flavor, but we can (and do) still miss a bunch of errors that will only ever be seen when spawning an actual instance and trying to trigger a build.

          h3. Expected

          We must run an actual AWS flavor provisioning during each PR build. We must then trigger a build and check the agent does provision and the build succeeds.
          Baptiste Mathus made changes -
          Sprint New: Evergreen - Milestone 2 [ 516 ]
          Baptiste Mathus made changes -
          Labels Original: evergreen New: evergreen technical-debt
          Baptiste Mathus made changes -
          Labels Original: evergreen technical-debt New: evergreen technical-debt testing
          Mark Waite made changes -
          Resolution New: Won't Do [ 10001 ]
          Status Original: Open [ 1 ] New: Closed [ 6 ]

            Unassigned Unassigned
            batmat Baptiste Mathus
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: