Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-15003

Track and verify plugins mentioned in configuration XML

    XMLWordPrintable

Details

    Description

      Many jobs or other model objects require certain plugins. This should be represented in the actual config.xml, e.g.

      <builders>
        <org.jenkinsci.plugins.hello.HelloWorldBuilder plugin="hello 1.5">
          <idiosyncraticConfig/>
        </org.jenkinsci.plugins.hello.HelloWorldBuilder>
      </builders>
      

      (XML namespaces would in some ways be natural for this, but XStream probably is more comfortable with a simple attribute on the outermost element “owned” by a plugin.)

      Need to make XmlFile.write insert these attributes as it serializes configuration, perhaps by making it take some interface implemented by PluginManager mapping Class to String shortName × String version. JRubyXStreamConverter might be used for inspiration.

      onLoad can either ignore these attributes, or use them as hints—for example, silently ignoring deserialization failures from a section owned by a plugin which is not currently loaded.

      Web methods like doCreateItem and doConfigDotXml, and their corresponding Java methods like createProjectFromXML and updateByXml, can be left alone, but a caller needs some way of first checking whether all required plugins are present, and attempting to install them if not. So there needs to be a web method on Jenkins and a corresponding Java method which can be given an arbitrary XML file and will return:

      1. OK, if all plugins mentioned therein are present. (Perhaps also check versions.)
      2. A security error, if caller is missing ADMINISTER permission.
      3. Or a list of jobs given as UpdateCenterJob.id (introduced in a3e57da) indicating ongoing progress of installing (or updating?) plugins, and perhaps even restarting Jenkins (trackable as of 4bd0696) for those plugins which require it.

      May also want some higher-level convenience interface, e.g. in CLI, to validate XML and wait for all update center jobs to complete (including restart as of 9ae0ba0).

      Attachments

        Issue Links

          Activity

            jglick Jesse Glick created issue -
            jglick Jesse Glick made changes -
            Field Original Value New Value
            Status Open [ 1 ] In Progress [ 3 ]
            jglick Jesse Glick added a comment - https://github.com/jglick/jenkins/compare/track-plugins-15003
            jglick Jesse Glick added a comment -

            Something like https://github.com/jenkinsci/cli-channel-demo/blob/master/src/main/java/org/jenkinsci/demo/cli/App.java (CLI.upgrade) could suffice for a convenient remote API, especially if the client were enhanced to automatically reconnect after a server restart.

            jglick Jesse Glick added a comment - Something like https://github.com/jenkinsci/cli-channel-demo/blob/master/src/main/java/org/jenkinsci/demo/cli/App.java ( CLI.upgrade ) could suffice for a convenient remote API, especially if the client were enhanced to automatically reconnect after a server restart.
            jglick Jesse Glick made changes -
            Description Many jobs or other model objects require certain plugins. This should be represented in the actual {{config.xml}}, e.g.

            {code:xml}
            <builders>
              <org.jenkinsci.plugins.hello.HelloWorldBuilder plugin="hello 1.5">
                <idiosyncraticConfig/>
              </org.jenkinsci.plugins.hello.HelloWorldBuilder>
            </builders>
            {code}

            (XML namespaces would in some ways be natural for this, but XStream probably is more comfortable with a simple attribute on the outermost element “owned” by a plugin.)

            Need to make {{XmlFile.write}} insert these attributes as it serializes configuration, perhaps by making it take some interface implemented by {{PluginManager}} mapping {{Class}} to {{String shortName}} × {{String version}}. {{JRubyXStreamConverter}} might be used for inspiration.

            {{onLoad}} can either ignore these attributes, or use them as hints—for example, silently ignoring deserialization failures from a section owned by a plugin which is not currently loaded.

            Web methods like {{doCreateItem}} and {{doConfigDotXml}}, and their corresponding Java methods like {{createProjectFromXML}} and {{updateByXml}}, can be left alone, but a caller needs some way of first checking whether all required plugins are present, and attempting to install them if not. So there needs to be a web method on {{Jenkins}} and a corresponding Java method which can be given an arbitrary XML file and will return:

            # OK, if all plugins mentioned therein are present. (Perhaps also check versions.)
            # A security error, if caller is missing {{ADMINISTER}} permission.
            # Or a list of jobs given as {{UpdateCenterJob.id}} (introduced in {{a3e57da}}) indicating ongoing progress of installing (or updating?) plugins, and perhaps even restarting Jenkins (trackable as of {{a3e57da}}) for those plugins which require it.

            May also want some higher-level convenience interface, e.g. in CLI, to validate XML and wait for all update center jobs to complete.
            Many jobs or other model objects require certain plugins. This should be represented in the actual {{config.xml}}, e.g.

            {code:xml}
            <builders>
              <org.jenkinsci.plugins.hello.HelloWorldBuilder plugin="hello 1.5">
                <idiosyncraticConfig/>
              </org.jenkinsci.plugins.hello.HelloWorldBuilder>
            </builders>
            {code}

            (XML namespaces would in some ways be natural for this, but XStream probably is more comfortable with a simple attribute on the outermost element “owned” by a plugin.)

            Need to make {{XmlFile.write}} insert these attributes as it serializes configuration, perhaps by making it take some interface implemented by {{PluginManager}} mapping {{Class}} to {{String shortName}} × {{String version}}. {{JRubyXStreamConverter}} might be used for inspiration.

            {{onLoad}} can either ignore these attributes, or use them as hints—for example, silently ignoring deserialization failures from a section owned by a plugin which is not currently loaded.

            Web methods like {{doCreateItem}} and {{doConfigDotXml}}, and their corresponding Java methods like {{createProjectFromXML}} and {{updateByXml}}, can be left alone, but a caller needs some way of first checking whether all required plugins are present, and attempting to install them if not. So there needs to be a web method on {{Jenkins}} and a corresponding Java method which can be given an arbitrary XML file and will return:

            # OK, if all plugins mentioned therein are present. (Perhaps also check versions.)
            # A security error, if caller is missing {{ADMINISTER}} permission.
            # Or a list of jobs given as {{UpdateCenterJob.id}} (introduced in {{a3e57da}}) indicating ongoing progress of installing (or updating?) plugins, and perhaps even restarting Jenkins (trackable as of {{4bd0696}}) for those plugins which require it.

            May also want some higher-level convenience interface, e.g. in CLI, to validate XML and wait for all update center jobs to complete (including restart as of {{9ae0ba0}}).
            jglick Jesse Glick added a comment -

            To find the list of update jobs using the remote API:

            curl -g 'http://localhost:8080/updateCenter/api/json?pretty&tree=jobs[id,type,status[type],plugin[name,version]]'
            
            jglick Jesse Glick added a comment - To find the list of update jobs using the remote API: curl -g 'http://localhost:8080/updateCenter/api/json?pretty&tree=jobs[id,type,status[type],plugin[name,version]]'
            jglick Jesse Glick added a comment -

            To prevalidate a job:

            curl -i -d @config.xml http://localhost:8080/pluginManager/prevalidateConfig
            

            and note the X-Jenkins-Session for later. To track its progress:

            (while :; do curl -ig 'http://localhost:8080/updateCenter/api/json?pretty&tree=jobs[id,type,status[type],plugin[name,version]]'; sleep 1; done)
            

            Initially all plugins will show e.g.

            {
              "id" : 1,
              "type" : "InstallationJob",
              "status" : {
                "type" : "Pending"
              },
              "plugin" : {
                "name" : "htmlpublisher",
                "version" : "1.0"
              }
            }
            

            but when done they will show

            {
               "id" : 1,
               "type" : "InstallationJob",
               "status" : {
                 "type" : "Success"
               },
               "plugin" : {
                 "name" : "htmlpublisher",
                 "version" : "1.0"
               }
             }
            

            If you see "SuccessButRequiresRestart" then you must

            curl -d '' http://localhost:8080/updateCenter/safeRestart
            

            and then poll on

            curl -I 'http://localhost:8080/'
            

            or (more efficiently) any …/api/… such as

            curl -i 'http://localhost:8080/updateCenter/api/json?tree=none'
            

            until you get a 200 status with a new X-Jenkins-Session indicating that the server has restarted.

            Now you may actually use the XML:

            curl -d @/tmp/config.xml -H 'Content-Type: text/xml' 'http://localhost:8080/createItem?name=newjob'
            
            jglick Jesse Glick added a comment - To prevalidate a job: curl -i -d @config.xml http://localhost:8080/pluginManager/prevalidateConfig and note the X-Jenkins-Session for later. To track its progress: (while :; do curl -ig 'http://localhost:8080/updateCenter/api/json?pretty&tree=jobs[id,type,status[type],plugin[name,version]]'; sleep 1; done) Initially all plugins will show e.g. { "id" : 1, "type" : "InstallationJob", "status" : { "type" : "Pending" }, "plugin" : { "name" : "htmlpublisher", "version" : "1.0" } } but when done they will show { "id" : 1, "type" : "InstallationJob", "status" : { "type" : "Success" }, "plugin" : { "name" : "htmlpublisher", "version" : "1.0" } } If you see "SuccessButRequiresRestart" then you must curl -d '' http://localhost:8080/updateCenter/safeRestart and then poll on curl -I 'http://localhost:8080/' or (more efficiently) any …/api/… such as curl -i 'http://localhost:8080/updateCenter/api/json?tree=none' until you get a 200 status with a new X-Jenkins-Session indicating that the server has restarted. Now you may actually use the XML: curl -d @/tmp/config.xml -H 'Content-Type: text/xml' 'http://localhost:8080/createItem?name=newjob'
            jglick Jesse Glick added a comment - https://github.com/jenkinsci/jenkins/pull/566
            jglick Jesse Glick added a comment - https://github.com/jenkinsci/jenkins/commit/e1b7ac281b93b7afa33056d59da987e9542a05e4
            jglick Jesse Glick made changes -
            Resolution Fixed [ 1 ]
            Status In Progress [ 3 ] Resolved [ 5 ]
            jglick Jesse Glick added a comment - Convenience client: https://github.com/jenkinsci/install-necessary-plugins
            rtyler R. Tyler Croy made changes -
            Workflow JNJira [ 145722 ] JNJira + In-Review [ 191597 ]
            jglick Jesse Glick made changes -
            Link This issue relates to JENKINS-34002 [ JENKINS-34002 ]

            People

              jglick Jesse Glick
              jglick Jesse Glick
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: