Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-27694

Download update data immediately when installing a new tool installer plugin

    • Icon: Improvement Improvement
    • Resolution: Fixed
    • Icon: Minor Minor
    • core
    • None

      I've seen it noted in various places recently (mailing list, IRC, personal experience) that the situation after installing a tool installer plugin is suboptimal.

      The JSON file containing the installation information for that tool is only downloaded periodically (daily?) along with all other tool installer data.

      Ideally, users should not have to wait up to 24 hours (or use a workaround: Manage Plugins > Advanced > Check Now) before they can use these plugins.

      So, whenever a DownloadService.Downloadable is loaded into Jenkins, it should fetch the corresponding JSON file from updates.jenkins-ci.org, if necessary.

          [JENKINS-27694] Download update data immediately when installing a new tool installer plugin

          Sorry for late answer, steps to reproduce on docker:

          1. mkdir /tmp/jenkins && chmod 777 /tmp/jenkins && docker run --rm -v /tmp/jenkins:/var/jenkins_home -p 10000:8080 jenkins:1.625.1
          2. cd /tmp/jenkins/plugins  && wget http://updates.jenkins-ci.org/latest/golang.hpi
          3. restart Jenkins via /safeRestart
          

          Expected:

          • Jenkins unpacks the .hpi file and runs UpdateCenter to populate GoLang data dropdown

          Actual:

          • Jenkins unpacks the .hpi file but UpdateCenter is not run

          After hitting the "Check Now" on the UI, indeed I can see:

          Nov 24, 2015 7:28:16 PM hudson.model.UpdateSite updateData
          INFO: Obtained the latest update center data file for UpdateSource default
          Nov 24, 2015 7:28:16 PM hudson.model.DownloadService$Downloadable load
          INFO: Obtained the updated data file for hudson.tasks.Maven.MavenInstaller
          Nov 24, 2015 7:28:17 PM hudson.model.DownloadService$Downloadable load
          INFO: Obtained the updated data file for hudson.tasks.Ant.AntInstaller
          Nov 24, 2015 7:28:19 PM hudson.model.DownloadService$Downloadable load
          INFO: Obtained the updated data file for hudson.tools.JDKInstaller
          Nov 24, 2015 7:28:21 PM hudson.model.DownloadService$Downloadable load
          INFO: Obtained the updated data file for org.jenkinsci.plugins.golang.GolangInstaller
          

          However, seems that DownloadService is not invoked automatically during the restart phase - what makes sense if you think of it - but in this case it makes the UI unusable, so maybe the golang plugin itself is not registering something properly?

          Radek Antoniuk added a comment - Sorry for late answer, steps to reproduce on docker: 1. mkdir /tmp/jenkins && chmod 777 /tmp/jenkins && docker run --rm -v /tmp/jenkins:/ var /jenkins_home -p 10000:8080 jenkins:1.625.1 2. cd /tmp/jenkins/plugins && wget http: //updates.jenkins-ci.org/latest/golang.hpi 3. restart Jenkins via /safeRestart Expected: Jenkins unpacks the .hpi file and runs UpdateCenter to populate GoLang data dropdown Actual: Jenkins unpacks the .hpi file but UpdateCenter is not run After hitting the "Check Now" on the UI, indeed I can see: Nov 24, 2015 7:28:16 PM hudson.model.UpdateSite updateData INFO: Obtained the latest update center data file for UpdateSource default Nov 24, 2015 7:28:16 PM hudson.model.DownloadService$Downloadable load INFO: Obtained the updated data file for hudson.tasks.Maven.MavenInstaller Nov 24, 2015 7:28:17 PM hudson.model.DownloadService$Downloadable load INFO: Obtained the updated data file for hudson.tasks.Ant.AntInstaller Nov 24, 2015 7:28:19 PM hudson.model.DownloadService$Downloadable load INFO: Obtained the updated data file for hudson.tools.JDKInstaller Nov 24, 2015 7:28:21 PM hudson.model.DownloadService$Downloadable load INFO: Obtained the updated data file for org.jenkinsci.plugins.golang.GolangInstaller However, seems that DownloadService is not invoked automatically during the restart phase - what makes sense if you think of it - but in this case it makes the UI unusable, so maybe the golang plugin itself is not registering something properly?

          Update:
          This WORKS with jenkins:weekly image that is 1.596 (old for weekly though).
          Seems that it does not work with newer Jenkins 1.625.1...
          or another magic/race condition is happening - but I am consistently reproducing it with the above scenario.

          Radek Antoniuk added a comment - Update: This WORKS with jenkins:weekly image that is 1.596 (old for weekly though). Seems that it does not work with newer Jenkins 1.625.1... or another magic/race condition is happening - but I am consistently reproducing it with the above scenario.

          I can reproduce with that Docker command and with the `golang` plugin (which has that UI bug — JENKINS-27499), but it also happens with others, e.g. the `gradle` plugin.

          Though note that hitting "Check Now" fails, due to the signature error: https://groups.google.com/forum/#!topic/jenkinsci-users/vv3UAIE7-Mg

          So perhaps that's a reason why you see it failing with certain Jenkins versions — the metadata fetch process is failing in the background?

          Christopher Orr added a comment - I can reproduce with that Docker command and with the `golang` plugin (which has that UI bug — JENKINS-27499 ), but it also happens with others, e.g. the `gradle` plugin. Though note that hitting "Check Now" fails, due to the signature error: https://groups.google.com/forum/#!topic/jenkinsci-users/vv3UAIE7-Mg So perhaps that's a reason why you see it failing with certain Jenkins versions — the metadata fetch process is failing in the background?

          I don't think that this is related to the signature bug, because the manual Check Now would not work either and it worked for me (see above logs).

          Radek Antoniuk added a comment - I don't think that this is related to the signature bug, because the manual Check Now would not work either and it worked for me (see above logs).

          The workaround doesn't seem to work with Jenkins 1.625.2.

          Moreover, after clicking "Check now" the page displays
          "Signature verification failed in downloadable 'hudson.tasks.Maven.MavenInstaller' (show details)"
          where "show details" doesn't show any detail.

          Giacomo Boccardo added a comment - The workaround doesn't seem to work with Jenkins 1.625.2. Moreover, after clicking "Check now" the page displays "Signature verification failed in downloadable 'hudson.tasks.Maven.MavenInstaller' (show details)" where "show details" doesn't show any detail.

          The workaround should work fine, but you need to first do a workaround for the other JENKINS-31089.

          Radek Antoniuk added a comment - The workaround should work fine, but you need to first do a workaround for the other JENKINS-31089 .

          warden, that's amazing
          Thanks for you help.

          Giacomo Boccardo added a comment - warden , that's amazing Thanks for you help.

          Daniel Beck added a comment -

          Sorry for late answer, steps to reproduce on docker:

          As you did never really installed the plugin, but just put the file there, Jenkins never got the trigger that it needs to update its metadata. Install using the UI or the CLI and the issue will disappear. Support for scanning manually installed plugins, like you're doing, for missing metadata is a different issue.

          Your attempt with an older release is likely only the result of there not being (recent enough) update center metadata to begin with. If Jenkins downloads everything anew, like in your log snippet, it's a case of exactly that.

          Daniel Beck added a comment - Sorry for late answer, steps to reproduce on docker: As you did never really installed the plugin, but just put the file there, Jenkins never got the trigger that it needs to update its metadata. Install using the UI or the CLI and the issue will disappear. Support for scanning manually installed plugins, like you're doing, for missing metadata is a different issue. Your attempt with an older release is likely only the result of there not being (recent enough) update center metadata to begin with. If Jenkins downloads everything anew, like in your log snippet, it's a case of exactly that.

          A user of the Go plugin emailed me to ask about this problem, and we discovered that they were installing the plugin using the "Download now and install
          after restart" option.

          Indeed, I can reproduce that the tool installer data is downloaded automatically if you install a plugin by clicking "Install without restart", but if you click on "Download now and install after restart", then the tool installer metadata is not downloaded automatically after the download, nor when Jenkins restarts.

          Users should preferably install plugins without restarting, but so long as the other option exists, people are going to click on it. Or they'll install new plugins at the same time as updating older ones. There's also a (probably very rare) chance that a plugin update will add a tool installer. Jenkins should also download the tool installer data in these situations.

          Should this go into a new ticket?

          Christopher Orr added a comment - A user of the Go plugin emailed me to ask about this problem, and we discovered that they were installing the plugin using the "Download now and install after restart" option. Indeed, I can reproduce that the tool installer data is downloaded automatically if you install a plugin by clicking "Install without restart", but if you click on "Download now and install after restart", then the tool installer metadata is not downloaded automatically after the download, nor when Jenkins restarts. Users should preferably install plugins without restarting, but so long as the other option exists, people are going to click on it. Or they'll install new plugins at the same time as updating older ones. There's also a (probably very rare) chance that a plugin update will add a tool installer. Jenkins should also download the tool installer data in these situations. Should this go into a new ticket?

          Daniel Beck added a comment -

          orrc File a new bug, assign to me.

          Daniel Beck added a comment - orrc File a new bug, assign to me.

            danielbeck Daniel Beck
            orrc Christopher Orr
            Votes:
            2 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated:
              Resolved: