Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-59696

support-core and thus advisor require to restart the instance to be activated

    • support-core-2.63

      Right now the plugin requires to have the master restarted otherwise (often?) it doesn't start to send bundles.

      We want to verify if we could fix this issue (it's painful to have to restart an instance to install a plugin)

       

      I suppose the issue could come from the instanciation/activation of the AsyncPeriodicWork task

      https://github.com/jenkinsci/cloudbees-jenkins-advisor-plugin/blob/master/src/main/java/com/cloudbees/jenkins/plugins/advisor/BundleUpload.java

      Is there any good practices to follow ? Any sample of such AsyncPeriodicWork extensions which aren't requiring to restart the master ?

        1. Screenshot 2019-10-25 17.28.38.png
          67 kB
          Arnaud Héritier
        2. extension-loading-with-issue.log
          129 kB
          Allan BURDAJEWICZ
        3. 2.193.loading-issues.log
          64 kB
          Allan BURDAJEWICZ
        4. Screenshot 2019-10-16 09.00.42.png
          21 kB
          Arnaud Héritier
        5. Screenshot 2019-10-16 08.57.20.png
          18 kB
          Arnaud Héritier

          [JENKINS-59696] support-core and thus advisor require to restart the instance to be activated

          Arnaud Héritier created issue -
          Arnaud Héritier made changes -
          Priority Original: Minor [ 4 ] New: Major [ 3 ]

          I took a clean 2.138.4, installed the plugin with `Install without restart` and the plugin isn't in the Manage Jenkins page

          I have the warning `Changes will take effect when you restart Jenkins`

          Arnaud Héritier added a comment - I took a clean 2.138.4, installed the plugin with `Install without restart` and the plugin isn't in the Manage Jenkins page I have the warning `Changes will take effect when you restart Jenkins`
          Arnaud Héritier made changes -
          Status Original: Open [ 1 ] New: In Progress [ 3 ]

          Arnaud Héritier added a comment - - edited

          More tests:

          1/ Jenkins 2.138.4 installed from scratch with suggested plugins. "Install without Restart" adds Variant, Metrics and Support core plugins. Going in Manage Jenkins allows to configure the plugin. Trying to trigger it using the Groovy Console is failing:

          Oct 09, 2019 2:51:33 PM hudson.model.AsyncPeriodicWork$1 run
          INFO: Started Bundle Upload
          Oct 09, 2019 2:51:33 PM com.cloudbees.jenkins.plugins.advisor.BundleUpload logError
          SEVERE: ERROR: Could not save support bundle
          java.lang.IllegalStateException: Expected 1 instance of com.cloudbees.jenkins.support.filter.ContentFilters but got 2
          	at hudson.ExtensionList.lookupSingleton(ExtensionList.java:450)
          	at com.cloudbees.jenkins.support.filter.ContentFilters.get(ContentFilters.java:47)
          	at com.cloudbees.jenkins.support.SupportPlugin.getContentFilter(SupportPlugin.java:402)
          	at com.cloudbees.jenkins.support.SupportPlugin.writeBundle(SupportPlugin.java:291)
          	at com.cloudbees.jenkins.plugins.advisor.BundleUpload.generateBundle(BundleUpload.java:96)
          	at com.cloudbees.jenkins.plugins.advisor.BundleUpload.execute(BundleUpload.java:71)
          	at hudson.model.AsyncPeriodicWork$1.run(AsyncPeriodicWork.java:101)
          	at java.lang.Thread.run(Thread.java:748)
          
          Oct 09, 2019 2:51:33 PM com.cloudbees.jenkins.plugins.advisor.BundleUpload log
          SEVERE: ERROR: Unable to generate support bundle
          Oct 09, 2019 2:51:33 PM hudson.model.AsyncPeriodicWork$1 run
          INFO: Finished Bundle Upload. 8 ms
          Oct 09, 2019 2:51:34 PM hudson.triggers.SafeTimerTask run
          SEVERE: Timer task com.cloudbees.jenkins.support.slowrequest.SlowRequestChecker@63dd7a2 failed
          java.lang.IllegalStateException: Expected 1 instance of com.cloudbees.jenkins.support.filter.ContentFilters but got 2
          	at hudson.ExtensionList.lookupSingleton(ExtensionList.java:450)
          	at com.cloudbees.jenkins.support.filter.ContentFilters.get(ContentFilters.java:47)
          	at com.cloudbees.jenkins.support.SupportPlugin.getContentFilter(SupportPlugin.java:402)
          	at com.cloudbees.jenkins.support.slowrequest.SlowRequestChecker.doRun(SlowRequestChecker.java:80)
          	at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:72)
          	at jenkins.security.ImpersonatingScheduledExecutorService$1.run(ImpersonatingScheduledExecutorService.java:58)
          	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
          	at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
          	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
          	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
          	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
          	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
          	at java.lang.Thread.run(Thread.java:748)
          
          Oct 09, 2019 2:51:37 PM hudson.triggers.SafeTimerTask run
          SEVERE: Timer task com.cloudbees.jenkins.support.slowrequest.SlowRequestChecker@63dd7a2 failed
          java.lang.IllegalStateException: Expected 1 instance of com.cloudbees.jenkins.support.filter.ContentFilters but got 2
          	at hudson.ExtensionList.lookupSingleton(ExtensionList.java:450)
          	at com.cloudbees.jenkins.support.filter.ContentFilters.get(ContentFilters.java:47)
          	at com.cloudbees.jenkins.support.SupportPlugin.getContentFilter(SupportPlugin.java:402)
          	at com.cloudbees.jenkins.support.slowrequest.SlowRequestChecker.doRun(SlowRequestChecker.java:80)
          	at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:72)
          	at jenkins.security.ImpersonatingScheduledExecutorService$1.run(ImpersonatingScheduledExecutorService.java:58)
          	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
          	at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
          	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
          	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
          	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
          	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
          	at java.lang.Thread.run(Thread.java:748)
          
          Oct 09, 2019 2:51:40 PM hudson.triggers.SafeTimerTask run
          SEVERE: Timer task com.cloudbees.jenkins.support.slowrequest.SlowRequestChecker@63dd7a2 failed
          java.lang.IllegalStateException: Expected 1 instance of com.cloudbees.jenkins.support.filter.ContentFilters but got 2
          	at hudson.ExtensionList.lookupSingleton(ExtensionList.java:450)
          	at com.cloudbees.jenkins.support.filter.ContentFilters.get(ContentFilters.java:47)
          	at com.cloudbees.jenkins.support.SupportPlugin.getContentFilter(SupportPlugin.java:402)
          	at com.cloudbees.jenkins.support.slowrequest.SlowRequestChecker.doRun(SlowRequestChecker.java:80)
          	at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:72)
          	at jenkins.security.ImpersonatingScheduledExecutorService$1.run(ImpersonatingScheduledExecutorService.java:58)
          	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
          	at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
          	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
          	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
          	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
          	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
          	at java.lang.Thread.run(Thread.java:748)
          

          The support core plugin doesn't like to be installed with no restart

          After a restart everything goes well
           

          Arnaud Héritier added a comment - - edited More tests: 1/ Jenkins 2.138.4 installed from scratch with suggested plugins. "Install without Restart" adds Variant, Metrics and Support core plugins. Going in Manage Jenkins allows to configure the plugin. Trying to trigger it using the Groovy Console is failing: Oct 09, 2019 2:51:33 PM hudson.model.AsyncPeriodicWork$1 run INFO: Started Bundle Upload Oct 09, 2019 2:51:33 PM com.cloudbees.jenkins.plugins.advisor.BundleUpload logError SEVERE: ERROR: Could not save support bundle java.lang.IllegalStateException: Expected 1 instance of com.cloudbees.jenkins.support.filter.ContentFilters but got 2 at hudson.ExtensionList.lookupSingleton(ExtensionList.java:450) at com.cloudbees.jenkins.support.filter.ContentFilters.get(ContentFilters.java:47) at com.cloudbees.jenkins.support.SupportPlugin.getContentFilter(SupportPlugin.java:402) at com.cloudbees.jenkins.support.SupportPlugin.writeBundle(SupportPlugin.java:291) at com.cloudbees.jenkins.plugins.advisor.BundleUpload.generateBundle(BundleUpload.java:96) at com.cloudbees.jenkins.plugins.advisor.BundleUpload.execute(BundleUpload.java:71) at hudson.model.AsyncPeriodicWork$1.run(AsyncPeriodicWork.java:101) at java.lang.Thread.run(Thread.java:748) Oct 09, 2019 2:51:33 PM com.cloudbees.jenkins.plugins.advisor.BundleUpload log SEVERE: ERROR: Unable to generate support bundle Oct 09, 2019 2:51:33 PM hudson.model.AsyncPeriodicWork$1 run INFO: Finished Bundle Upload. 8 ms Oct 09, 2019 2:51:34 PM hudson.triggers.SafeTimerTask run SEVERE: Timer task com.cloudbees.jenkins.support.slowrequest.SlowRequestChecker@63dd7a2 failed java.lang.IllegalStateException: Expected 1 instance of com.cloudbees.jenkins.support.filter.ContentFilters but got 2 at hudson.ExtensionList.lookupSingleton(ExtensionList.java:450) at com.cloudbees.jenkins.support.filter.ContentFilters.get(ContentFilters.java:47) at com.cloudbees.jenkins.support.SupportPlugin.getContentFilter(SupportPlugin.java:402) at com.cloudbees.jenkins.support.slowrequest.SlowRequestChecker.doRun(SlowRequestChecker.java:80) at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:72) at jenkins.security.ImpersonatingScheduledExecutorService$1.run(ImpersonatingScheduledExecutorService.java:58) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Oct 09, 2019 2:51:37 PM hudson.triggers.SafeTimerTask run SEVERE: Timer task com.cloudbees.jenkins.support.slowrequest.SlowRequestChecker@63dd7a2 failed java.lang.IllegalStateException: Expected 1 instance of com.cloudbees.jenkins.support.filter.ContentFilters but got 2 at hudson.ExtensionList.lookupSingleton(ExtensionList.java:450) at com.cloudbees.jenkins.support.filter.ContentFilters.get(ContentFilters.java:47) at com.cloudbees.jenkins.support.SupportPlugin.getContentFilter(SupportPlugin.java:402) at com.cloudbees.jenkins.support.slowrequest.SlowRequestChecker.doRun(SlowRequestChecker.java:80) at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:72) at jenkins.security.ImpersonatingScheduledExecutorService$1.run(ImpersonatingScheduledExecutorService.java:58) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Oct 09, 2019 2:51:40 PM hudson.triggers.SafeTimerTask run SEVERE: Timer task com.cloudbees.jenkins.support.slowrequest.SlowRequestChecker@63dd7a2 failed java.lang.IllegalStateException: Expected 1 instance of com.cloudbees.jenkins.support.filter.ContentFilters but got 2 at hudson.ExtensionList.lookupSingleton(ExtensionList.java:450) at com.cloudbees.jenkins.support.filter.ContentFilters.get(ContentFilters.java:47) at com.cloudbees.jenkins.support.SupportPlugin.getContentFilter(SupportPlugin.java:402) at com.cloudbees.jenkins.support.slowrequest.SlowRequestChecker.doRun(SlowRequestChecker.java:80) at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:72) at jenkins.security.ImpersonatingScheduledExecutorService$1.run(ImpersonatingScheduledExecutorService.java:58) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) The support core plugin doesn't like to be installed with no restart After a restart everything goes well  

          2/ Jenkins 2.138.4 installed from scratch with suggested plugins. "Download now and install after restart" but adds Variant, Metrics and Support core plugins but all of them are "Downloaded Successfully. Will be activated during the next boot". We cannot use any of them before the next restart

          Arnaud Héritier added a comment - 2/ Jenkins 2.138.4 installed from scratch with suggested plugins. "Download now and install after restart" but adds Variant, Metrics and Support core plugins but all of them are "Downloaded Successfully. Will be activated during the next boot". We cannot use any of them before the next restart

          3/ Jenkins 2.138.4 installed from scratch with suggested plugins. *I upload from the UI a patched version of the plugin* with all extensions and marked as loadable dynamically and it works. I can trigger an upload from the console. No issue.

          https://github.com/jenkinsci/cloudbees-jenkins-advisor-plugin/pull/50

          But is it a test which is relevant compared to installations 1/ and 2/ done from the UC ?

          Arnaud Héritier added a comment - 3/ Jenkins 2.138.4 installed from scratch with suggested plugins. * I upload from the UI a patched version of the plugin * with all extensions and marked as loadable dynamically and it works. I can trigger an upload from the console. No issue. https://github.com/jenkinsci/cloudbees-jenkins-advisor-plugin/pull/50 But is it a test which is relevant compared to installations 1/ and 2/ done from the UC ?

          pierrebtz also found something similar done by stephenconnolly a year ago:
          https://github.com/jenkinsci/metrics-plugin/blame/bb4a572004bb708136a57d2d359a930c934786b1/src/main/java/jenkins/metrics/impl/JenkinsMetricProviderImpl.java#L708-L721
          https://issues.jenkins-ci.org/browse/JENKINS-28983
          https://issues.jenkins-ci.org/browse/JENKINS-52072

          But my tests are with 2.138 thus we should have the required fix in core

          And strangely I find ~ no plugin using @Extension(dynamicLoadable = YesNoMaybe.YES)

          Arnaud Héritier added a comment - pierrebtz also found something similar done by stephenconnolly a year ago: https://github.com/jenkinsci/metrics-plugin/blame/bb4a572004bb708136a57d2d359a930c934786b1/src/main/java/jenkins/metrics/impl/JenkinsMetricProviderImpl.java#L708-L721 https://issues.jenkins-ci.org/browse/JENKINS-28983 https://issues.jenkins-ci.org/browse/JENKINS-52072 But my tests are with 2.138 thus we should have the required fix in core And strangely I find ~ no plugin using @Extension(dynamicLoadable = YesNoMaybe.YES)

          In fact doing 3/ with the current master code and not the patched one works too

          Should I conclude that to install a plugin without restarting the master the best strategy is to upload it manually instead of installing it from the UC?

          Arnaud Héritier added a comment - In fact doing 3/ with the current master code and not the patched one works too Should I conclude that to install a plugin without restarting the master the best strategy is to upload it manually instead of installing it from the UC?

          allan_burdajewicz could we discuss about this ?

          Arnaud Héritier added a comment - allan_burdajewicz could we discuss about this ?

            allan_burdajewicz Allan BURDAJEWICZ
            aheritier Arnaud Héritier
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: