Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-37759

Usage of @Initializer(after = InitMilestone.COMPLETED) in plugins breaks the initialization logic

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Resolved (View Workflow)
    • Priority: Critical
    • Resolution: Fixed
    • Component/s: core
    • Labels:
    • Environment:
      jenkins-2.7.3-rc
    • Similar Issues:

      Description

      The problem is in the Initializer code:

      /**
           * Indicates that the specified milestone is necessary before executing this initializer.
           *
           * <p>
           * This has the identical purpose as {@link #requires()}, but it's separated to allow better type-safety
           * when using {@link InitMilestone} as a requirement (since enum member definitions need to be constant.)
           */
          InitMilestone after() default STARTED;
      
          /**
           * Indicates that this initializer is a necessary step before achieving the specified milestone.
           *
           * <p>
           * This has the identical purpose as {@link #attains()}. See {@link #after()} for why there are two things
           * to achieve the same goal.
           */
          InitMilestone before() default COMPLETED;
      

      If an initializer overrides <code>after</code> as COMPLETED, hence we end up in the situation when (<code>before == after == COMPLETED</code>). In such case Jenkins' reactor can never succeed in the canRun() check, and finally we do not invoke "Initialization completed" milestone since the code requires the dependency to be executed. So we get into chicken-and-egg

      Sample script for the issue analysis:

      import hudson.init.InitMilestone; 
      import hudson.util.HttpResponses;
      import javax.servlet.http.HttpServletResponse;
      
      final Jenkins jenkins = Jenkins.getInstance();
      
      if (jenkins.getInitLevel().compareTo(InitMilestone.COMPLETED) < 0) {
        println "Error: Jenkins initialization has not reached the COMPLETED state. Current state is ${jenkins.getInitLevel()}"
        return HttpResponses.status(HttpServletResponse.SC_SERVICE_UNAVAILABLE);
      }
      

      Examples:

      Proposed changes:

      • Add injected test, which verifies the startup InitMilestone of Jenkins
      • Make Jenkins robust against wrong definitions
      • Finally add support of Initializers after COMPLETED by introducing a transient milestone

        Attachments

          Issue Links

            Activity

            oleg_nenashev Oleg Nenashev created issue -
            oleg_nenashev Oleg Nenashev made changes -
            Field Original Value New Value
            Issue Type Task [ 3 ] Bug [ 1 ]
            oleg_nenashev Oleg Nenashev made changes -
            Labels lts-candidate
            recampbell Ryan Campbell made changes -
            Remote Link This issue links to "PR (Web Link)" [ 14776 ]
            oleg_nenashev Oleg Nenashev made changes -
            Link This issue is related to JENKINS-37780 [ JENKINS-37780 ]
            oleg_nenashev Oleg Nenashev made changes -
            Link This issue is related to JENKINS-37772 [ JENKINS-37772 ]
            oleg_nenashev Oleg Nenashev made changes -
            Link This issue is related to JENKINS-37805 [ JENKINS-37805 ]
            oleg_nenashev Oleg Nenashev made changes -
            Priority Blocker [ 1 ] Major [ 3 ]
            oleg_nenashev Oleg Nenashev made changes -
            Summary After startup of the instance with many jobs1, the setup does not end up in the COMPLETED milestone Usage of @Initializer(after = InitMilestone.COMPLETED) in plugins breaks the initialization logic
            oleg_nenashev Oleg Nenashev made changes -
            Description It happens due to https://github.com/jenkinsci/jenkins/pull/2177/files, which makes COMPLETED dependent on the following logic:

            {code}
                        // All plugins are loaded. Now we can figure out who depends on who.
                         requires(PLUGINS_PREPARED).attains(COMPLETED).add("Resolving Dependant Plugins Graph", new Executable() {
                             @Override
                             public void run(Reactor reactor) throws Exception {
                                 resolveDependantPlugins();
                             }
                         });
            {code}

            In this case COMPLETED state logic does not depend on the Job loading state, hence COMPLETED milestone can be achieved before JOB_LOADED in the reactor. It may easily happen when Job loading takes MUCH time. In such case COMPLETED state gets overridden by JOB_LOADED.

            Sample script for the issue analysis:
            {code}
            import hudson.init.InitMilestone;
            import hudson.util.HttpResponses;
            import javax.servlet.http.HttpServletResponse;

            final Jenkins jenkins = Jenkins.getInstance();

            if (jenkins.getInitLevel().compareTo(InitMilestone.COMPLETED) < 0) {
              println "Error: Jenkins initialization has not reached the COMPLETED state. Current state is ${jenkins.getInitLevel()}"
              return HttpResponses.status(HttpServletResponse.SC_SERVICE_UNAVAILABLE);
            }
            {code}


            The problem is in the Initializer code:

            {code}
            /**
                 * Indicates that the specified milestone is necessary before executing this initializer.
                 *
                 * <p>
                 * This has the identical purpose as {@link #requires()}, but it's separated to allow better type-safety
                 * when using {@link InitMilestone} as a requirement (since enum member definitions need to be constant.)
                 */
                InitMilestone after() default STARTED;

                /**
                 * Indicates that this initializer is a necessary step before achieving the specified milestone.
                 *
                 * <p>
                 * This has the identical purpose as {@link #attains()}. See {@link #after()} for why there are two things
                 * to achieve the same goal.
                 */
                InitMilestone before() default COMPLETED;
            {code}

            If an initializer overrides <code>after</code> as COMPLETED, hence we end up in the situation when (<code>before == after == COMPLETED</code>). In such case Jenkins' reactor can never succeed in the canRun() check, and finally we do not invoke "Initialization completed" milestone since the code requires the dependency to be executed. So we get into chicken-and-egg

            Sample script for the issue analysis:
            {code}
            import hudson.init.InitMilestone;
            import hudson.util.HttpResponses;
            import javax.servlet.http.HttpServletResponse;

            final Jenkins jenkins = Jenkins.getInstance();

            if (jenkins.getInitLevel().compareTo(InitMilestone.COMPLETED) < 0) {
              println "Error: Jenkins initialization has not reached the COMPLETED state. Current state is ${jenkins.getInitLevel()}"
              return HttpResponses.status(HttpServletResponse.SC_SERVICE_UNAVAILABLE);
            }
            {code}

            Examples:
            * https://github.com/jenkinsci/extreme-notification-plugin/blob/0249acced3d5483841e06849a4481c47a7e28fe5/src/main/java/org/jenkinsci/plugins/extremenotification/ServerStateListener.java#L51 (JENKINS-37780)
            * https://github.com/jenkinsci/support-core-plugin/blob/3cc53c1af1d2332bf5c6396987fe1516880b81a8/src/main/java/com/cloudbees/jenkins/support/SupportPlugin.java#L409 (JENKINS-37772)

            Proposed changes:
            * Add injected test, which verifies the startup InitMilestone of Jenkins
            * Make Jenkins robust against wrong definitions
            * Finally add support of Initializers after COMPLETED by introducing a transient milestone
            oleg_nenashev Oleg Nenashev made changes -
            Link This issue is related to JENKINS-37807 [ JENKINS-37807 ]
            oleg_nenashev Oleg Nenashev made changes -
            Priority Major [ 3 ] Critical [ 2 ]
            oleg_nenashev Oleg Nenashev made changes -
            Labels lts-candidate
            oleg_nenashev Oleg Nenashev made changes -
            Labels startup
            oleg_nenashev Oleg Nenashev made changes -
            Assignee Oleg Nenashev [ oleg_nenashev ]
            oleg_nenashev Oleg Nenashev made changes -
            Status Open [ 1 ] In Progress [ 3 ]
            Hide
            oleg_nenashev Oleg Nenashev added a comment -

            Created a bunch of follow-up issues

            Show
            oleg_nenashev Oleg Nenashev added a comment - Created a bunch of follow-up issues
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Oleg Nenashev
            Path:
            src/main/java/org/jenkinsci/plugins/extremenotification/MyItemListener.java
            src/main/java/org/jenkinsci/plugins/extremenotification/ServerStateListener.java
            src/test/java/org/jenkinsci/plugins/extremenotification/PluginInitializationTest.java
            http://jenkins-ci.org/commit/extreme-notification-plugin/1814673042bb450de5ec5401bf8096abccc5ff67
            Log:
            [FIXED JENKINS-37780] - The code should not use Initializer(after=InitMilestone.COMPLETED) due to JENKINS-37759

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Oleg Nenashev Path: src/main/java/org/jenkinsci/plugins/extremenotification/MyItemListener.java src/main/java/org/jenkinsci/plugins/extremenotification/ServerStateListener.java src/test/java/org/jenkinsci/plugins/extremenotification/PluginInitializationTest.java http://jenkins-ci.org/commit/extreme-notification-plugin/1814673042bb450de5ec5401bf8096abccc5ff67 Log: [FIXED JENKINS-37780] - The code should not use Initializer(after=InitMilestone.COMPLETED) due to JENKINS-37759
            oleg_nenashev Oleg Nenashev made changes -
            Link This issue is related to JENKINS-37874 [ JENKINS-37874 ]
            Hide
            oleg_nenashev Oleg Nenashev added a comment -

            The fix has not been released by the maintainer yet, but I think my work is done here

            Show
            oleg_nenashev Oleg Nenashev added a comment - The fix has not been released by the maintainer yet, but I think my work is done here
            oleg_nenashev Oleg Nenashev made changes -
            Resolution Fixed [ 1 ]
            Status In Progress [ 3 ] Resolved [ 5 ]
            oleg_nenashev Oleg Nenashev made changes -
            Link This issue is related to JENKINS-44453 [ JENKINS-44453 ]
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Oleg Nenashev
            Path:
            src/main/java/org/jvnet/hudson/test/JenkinsRule.java
            http://jenkins-ci.org/commit/jenkins-test-harness/fc7ef54f6aadfbf457e746847adbad0d4616dc56
            Log:
            [FIXED JENKINS-44453] - JenkinsRule should ensure that Jenkins reaches the COMPLETED milestone.

            It should make issues like https://issues.jenkins-ci.org/browse/JENKINS-37759 much more explicit in funtional tests.
            From what I see there is no valid use-case for proceeding with the test if the initialization fails.

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Oleg Nenashev Path: src/main/java/org/jvnet/hudson/test/JenkinsRule.java http://jenkins-ci.org/commit/jenkins-test-harness/fc7ef54f6aadfbf457e746847adbad0d4616dc56 Log: [FIXED JENKINS-44453] - JenkinsRule should ensure that Jenkins reaches the COMPLETED milestone. It should make issues like https://issues.jenkins-ci.org/browse/JENKINS-37759 much more explicit in funtional tests. From what I see there is no valid use-case for proceeding with the test if the initialization fails.
            cloudbees CloudBees Inc. made changes -
            Remote Link This issue links to "CloudBees Internal OSS-1385 (Web Link)" [ 18721 ]

              People

              Assignee:
              oleg_nenashev Oleg Nenashev
              Reporter:
              oleg_nenashev Oleg Nenashev
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: