Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-10709

multiple builds are triggered for one change in Gerrit

      With Gerrit Trigger 2.3.0 we observe that sometimes there are multiple builds triggered for one change in Gerrit.

      Lets say in Jenkins build #25 is triggered by the Gerrit Trigger Plugin. Then we see for this build job the following message in Jenkins:
      Triggered by Gerrit: http://gerrit:8080/27060
      Other triggered builds for this event.
      myjob #25

      The strange thing is that the build job #25 says that there is another build job for the same event which actually itself (again build job #25).

      On the change in Gerrit we then get the following comments:
      1. Build Started http://jenkins:8080/jenkins/job/myjob/25/ (1/2)
      2. Build Started http://jenkins:8080/jenkins/job/myjob/25/ (2/2)
      3. Build Failed
      http://jenkins:8080/jenkins/job/myjob/25/ : FAILURE
      http://jenkins:8080/jenkins/job/myjob/25/ : FAILURE

      In Jenkins it looks like the build job is indeed triggered twice. At least we saw the build job #25 two times in the list of current builds. Normally our tests are failing if they are executed two times in parallel since they need exclusive access to certain resources.

      For the failed build job I have then the possibility to retirgger it by clicking on 'Retrigger' or 'Retrigger All'.
      'Retrigger' starts another build #26, but for the voting still the result of build #25 is taken into account:
      1. Build Started http://jenkins:8080/jenkins/job/myjob/26/ (2/2)
      2. Build Failed
      http://jenkins:8080/jenkins/job/myjob/25/ : FAILURE
      http://jenkins:8080/jenkins/job/myjob/26/ : SUCCESS

      Why does the Gerrit Trigger start multiple builds for only one change in Gerrit? Is this some feature? Can we configure the Gerrit Trigger plugin to always only start a single build?

      Any help would be appreciated. Thanks!

          [JENKINS-10709] multiple builds are triggered for one change in Gerrit

          rsandell added a comment -

          This is very similar, but not completely, to bug that I have been trying to solve for the last couple of days.
          And it is definitely a bug.
          In our environment we see some builds triggered more than once on the same patch-set event. The cause seems to be that the project gets duplicated in the list of listeners for the events, so it gets called for every occasion it appears in the list, but I have no idea how that can happen and have been trying to recreate the circumstances to no avail.

          I will attach a groovy script that you can run to see if it is the same in your server.

          rsandell added a comment - This is very similar, but not completely, to bug that I have been trying to solve for the last couple of days. And it is definitely a bug. In our environment we see some builds triggered more than once on the same patch-set event. The cause seems to be that the project gets duplicated in the list of listeners for the events, so it gets called for every occasion it appears in the list, but I have no idea how that can happen and have been trying to recreate the circumstances to no avail. I will attach a groovy script that you can run to see if it is the same in your server.

          rsandell added a comment -

          A groovy script you can execute on the master to see if there are any duplicated listeners for a triggered project.

          rsandell added a comment - A groovy script you can execute on the master to see if there are any duplicated listeners for a triggered project.

          zlosch added a comment - - edited

          The script throws an error on the above Jenkins instance:

          groovy.lang.MissingMethodException: No signature of method: java.util.LinkedList.keySet() is applicable for argument types: () values: []
          	at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.unwrap(ScriptBytecodeAdapter.java:54)
          	at org.codehaus.groovy.runtime.callsite.PojoMetaClassSite.call(PojoMetaClassSite.java:48)
          	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:43)
          	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116)
          	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:120)
          	at Script1.run(Script1.groovy:2)
          	at groovy.lang.GroovyShell.evaluate(GroovyShell.java:561)
          	at groovy.lang.GroovyShell.evaluate(GroovyShell.java:536)
          	at hudson.util.RemotingDiagnostics$Script.call(RemotingDiagnostics.java:141)
          	at hudson.util.RemotingDiagnostics$Script.call(RemotingDiagnostics.java:119)
          	at hudson.remoting.LocalChannel.call(LocalChannel.java:45)
          	at hudson.util.RemotingDiagnostics.executeGroovy(RemotingDiagnostics.java:116)
          	at hudson.model.Hudson.doScript(Hudson.java:3257)
          	at hudson.model.Hudson.doScript(Hudson.java:3240)
          	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
          	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
          	at java.lang.reflect.Method.invoke(Method.java:597)
          	at org.kohsuke.stapler.Function$InstanceFunction.invoke(Function.java:282)
          	at org.kohsuke.stapler.Function.bindAndInvoke(Function.java:149)
          	at org.kohsuke.stapler.Function.bindAndInvokeAndServeResponse(Function.java:88)
          	at org.kohsuke.stapler.MetaClass$1.doDispatch(MetaClass.java:103)
          	at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:53)
          	at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:561)
          	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:646)
          	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:477)
          	at org.kohsuke.stapler.Stapler.service(Stapler.java:159)
          	at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
          	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
          	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
          	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:94)
          	at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:86)
          	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
          	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
          	at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:47)
          	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
          	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
          	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84)
          	at hudson.security.UnwrapSecurityExceptionFilter.doFilter(UnwrapSecurityExceptionFilter.java:51)
          	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
          	at org.acegisecurity.ui.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:166)
          	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
          	at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125)
          	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
          	at hudson.security.BasicAuthenticationFilter.doFilter(BasicAuthenticationFilter.java:111)
          	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
          	at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:76)
          	at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:164)
          	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
          	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
          	at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:81)
          	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
          	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
          	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
          	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
          	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:470)
          	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
          	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
          	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
          	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
          	at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:857)
          	at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
          	at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
          	at java.lang.Thread.run(Thread.java:619)
          

          zlosch added a comment - - edited The script throws an error on the above Jenkins instance: groovy.lang.MissingMethodException: No signature of method: java.util.LinkedList.keySet() is applicable for argument types: () values: [] at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.unwrap(ScriptBytecodeAdapter.java:54) at org.codehaus.groovy.runtime.callsite.PojoMetaClassSite.call(PojoMetaClassSite.java:48) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:43) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:120) at Script1.run(Script1.groovy:2) at groovy.lang.GroovyShell.evaluate(GroovyShell.java:561) at groovy.lang.GroovyShell.evaluate(GroovyShell.java:536) at hudson.util.RemotingDiagnostics$Script.call(RemotingDiagnostics.java:141) at hudson.util.RemotingDiagnostics$Script.call(RemotingDiagnostics.java:119) at hudson.remoting.LocalChannel.call(LocalChannel.java:45) at hudson.util.RemotingDiagnostics.executeGroovy(RemotingDiagnostics.java:116) at hudson.model.Hudson.doScript(Hudson.java:3257) at hudson.model.Hudson.doScript(Hudson.java:3240) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.kohsuke.stapler.Function$InstanceFunction.invoke(Function.java:282) at org.kohsuke.stapler.Function.bindAndInvoke(Function.java:149) at org.kohsuke.stapler.Function.bindAndInvokeAndServeResponse(Function.java:88) at org.kohsuke.stapler.MetaClass$1.doDispatch(MetaClass.java:103) at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:53) at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:561) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:646) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:477) at org.kohsuke.stapler.Stapler.service(Stapler.java:159) at javax.servlet.http.HttpServlet.service(HttpServlet.java:717) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:94) at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:86) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:47) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84) at hudson.security.UnwrapSecurityExceptionFilter.doFilter(UnwrapSecurityExceptionFilter.java:51) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.ui.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:166) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at hudson.security.BasicAuthenticationFilter.doFilter(BasicAuthenticationFilter.java:111) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:76) at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:164) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:81) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:470) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:857) at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588) at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489) at java.lang.Thread.run(Thread.java:619)

          rsandell added a comment -

          Oh, sorry about that. The script is written for the snapshot version that we are running.
          We changed some internal structures to try and fix this problem, but haven't yet.

          But now I know that the bug is not because of those changes, because you are running an earlier version than we do.

          rsandell added a comment - Oh, sorry about that. The script is written for the snapshot version that we are running. We changed some internal structures to try and fix this problem, but haven't yet. But now I know that the bug is not because of those changes, because you are running an earlier version than we do.

          Matthias Sohn added a comment -

          Could you shed some light on what these multiple builds for the same project and gerrit event are good for and how to configure this ?

          We also observed that the data Gerrit Trigger reports as Gerrit review comments for these duplicate builds seems to be mixed up wrt to the build numbers,
          so Hudson says job #23 SUCCEEDED and job #24 FAILED but Gerrit Trigger reports the opposite result in Gerrit review comments (#23 FAILED and #24 SUCCEEDED).
          These results are reported in pairs in the same review comment.

          Matthias Sohn added a comment - Could you shed some light on what these multiple builds for the same project and gerrit event are good for and how to configure this ? We also observed that the data Gerrit Trigger reports as Gerrit review comments for these duplicate builds seems to be mixed up wrt to the build numbers, so Hudson says job #23 SUCCEEDED and job #24 FAILED but Gerrit Trigger reports the opposite result in Gerrit review comments (#23 FAILED and #24 SUCCEEDED). These results are reported in pairs in the same review comment.

          Small script to iterate through the listeners (for the next time it happens).

          Glenn J. Mason added a comment - Small script to iterate through the listeners (for the next time it happens).

          rsandell added a comment -

          A couple of different fixes around this area has been done up to (including) version 2.6.0
          Can you see if those fixes has resolved this one as well?

          rsandell added a comment - A couple of different fixes around this area has been done up to (including) version 2.6.0 Can you see if those fixes has resolved this one as well?

          Diego Mendes added a comment - - edited

          Tested this issue with Jenkins ver. 1.489 and Gerrit Trigger 2.6.0 , but it remains occurring.
          I tried to avoid the problem by reducing the number of Receiving Worker Threads, but it did not work.

          When a new version will be released ??

          Diego Mendes added a comment - - edited Tested this issue with Jenkins ver. 1.489 and Gerrit Trigger 2.6.0 , but it remains occurring. I tried to avoid the problem by reducing the number of Receiving Worker Threads, but it did not work. When a new version will be released ??

          Paul Jackson added a comment -

          If it helps to track this down, I have just started seeing this behavior on our Jenkins. When we only had one job triggered by a gerrit trigger then everything was ok. As soon as I added another job with a gerrit trigger, jobs starting executing twice. We are using Jenkins 1.491, Gerrit trigger 2.5.3, and gerrit 2.2.0.

          Paul Jackson added a comment - If it helps to track this down, I have just started seeing this behavior on our Jenkins. When we only had one job triggered by a gerrit trigger then everything was ok. As soon as I added another job with a gerrit trigger, jobs starting executing twice. We are using Jenkins 1.491, Gerrit trigger 2.5.3, and gerrit 2.2.0.

          same problem seen on:
          Jenkins 1.480.2
          Gerrit Trigger 2.7.0

          but now we even have triple builds, one with build number x and two with build number x+1.

          Is there anything we can help with to locate the problem?

          Henrik Nilsson added a comment - same problem seen on: Jenkins 1.480.2 Gerrit Trigger 2.7.0 but now we even have triple builds, one with build number x and two with build number x+1. Is there anything we can help with to locate the problem?

          It is possible to reproduce this fault by reloading the configuration from disc, each time the configuration is reloaded the multiple of build jobs are increased by 1.

          Henrik Nilsson added a comment - It is possible to reproduce this fault by reloading the configuration from disc, each time the configuration is reloaded the multiple of build jobs are increased by 1.

          rsandell added a comment -

          Reloading from disk has been a problem in the past. Other plugins have the same problem, but maybe not as visual as GT.
          It is not an easy thing to fix, not to keep being fixed.

          rsandell added a comment - Reloading from disk has been a problem in the past. Other plugins have the same problem, but maybe not as visual as GT. It is not an easy thing to fix, not to keep being fixed.

          Paul Jackson added a comment -

          Some more information on our situation. I have written a script that manages the configuration of our Jenkins job by posting xml. Every time the configuration of our job containing the gerrit trigger is updated (by posting the config.xml file) we end up with duplicate jobs running. Could be similar to the reloading config from disk?

          If this has been a problem in the past, can you give some pointers to where the problem is? I might be able to spend some time on this.

          Paul Jackson added a comment - Some more information on our situation. I have written a script that manages the configuration of our Jenkins job by posting xml. Every time the configuration of our job containing the gerrit trigger is updated (by posting the config.xml file) we end up with duplicate jobs running. Could be similar to the reloading config from disk? If this has been a problem in the past, can you give some pointers to where the problem is? I might be able to spend some time on this.

          rsandell added a comment -

          I think the "posting configuration xml" feature is using the same code in core that reload config from disk is using, so yes it is a very likely it is a similar issue.

          When we had the bug in the past it was due to the way the GerritHandler (a class in the trigger plugin) was storing each listener i.e. each GerritTrigger instance to notify when an event arrives. And when reload from disk is performed Jenkins actually creates new instances of each job and for some reason the old job instance wasn't removed from the listeners list.
          We solved it by implementing the list as a HashMap with the project name as the key so when a project added itself twice the instance from the last invocation was the one saved in the map. But that caused a bug when a project was renamed, so that was fixed by using a Set instead I think, maybe that caused the original bug to reappear.

          Anyways; some hints on how the flow goes during startup might be of help to anyone wanting to dig into this bug.

          When Jenkins starts: PluginImpl.start() is called before any jobs are loaded. PluginImpl will instanciate a GerritHandler that handles the communication and stream-events coordination towards Gerrit.

          Then when each job has been loaded GerritTrigger.start() is called, and that will add the GerritTrigger instance as a listener in GerritHandler (via a call to PluginImpl.addListener).

          When a job is removed or unloaded GerritTrigger.stop is called and that will remove the GerritTrigger instance as a listener.

          The same happens when you reconfigure a job in Jenkins. When save is clicked first stop will be called on the old instance of GerritTrigger, then a new instance is created by Jenkins and start is called on that.

          So in theory everything should be fine, but something is handled differently when the config is changed from disk or other than clicking save in the UI.
          Maybe stop is never called or called in a different order so the old instance isn't removed.
          Or maybe it's a new bug completely (but probably unlikely).

          rsandell added a comment - I think the "posting configuration xml" feature is using the same code in core that reload config from disk is using, so yes it is a very likely it is a similar issue. When we had the bug in the past it was due to the way the GerritHandler (a class in the trigger plugin) was storing each listener i.e. each GerritTrigger instance to notify when an event arrives. And when reload from disk is performed Jenkins actually creates new instances of each job and for some reason the old job instance wasn't removed from the listeners list. We solved it by implementing the list as a HashMap with the project name as the key so when a project added itself twice the instance from the last invocation was the one saved in the map. But that caused a bug when a project was renamed, so that was fixed by using a Set instead I think, maybe that caused the original bug to reappear. Anyways; some hints on how the flow goes during startup might be of help to anyone wanting to dig into this bug. When Jenkins starts: PluginImpl.start() is called before any jobs are loaded. PluginImpl will instanciate a GerritHandler that handles the communication and stream-events coordination towards Gerrit. Then when each job has been loaded GerritTrigger.start() is called, and that will add the GerritTrigger instance as a listener in GerritHandler (via a call to PluginImpl.addListener). When a job is removed or unloaded GerritTrigger.stop is called and that will remove the GerritTrigger instance as a listener. The same happens when you reconfigure a job in Jenkins. When save is clicked first stop will be called on the old instance of GerritTrigger, then a new instance is created by Jenkins and start is called on that. So in theory everything should be fine, but something is handled differently when the config is changed from disk or other than clicking save in the UI. Maybe stop is never called or called in a different order so the old instance isn't removed. Or maybe it's a new bug completely (but probably unlikely).

          Paul Jackson added a comment -

          Did some testing of this yesterday. When I trigger a reload from disk, the only log message displayed is:

          13-May-2013 16:25:10 com.sonyericsson.hudson.plugins.gerrit.trigger.hudsontrigger.GerritTrigger start
          FINE: Start project: hudson.model.FreeStyleProject@12143d8[test gerrit trigger 1]

          So I'm not seeing stop being called. This was on 1.457 which I had a version of lying around. I'll try an updated version later.

          Paul Jackson added a comment - Did some testing of this yesterday. When I trigger a reload from disk, the only log message displayed is: 13-May-2013 16:25:10 com.sonyericsson.hudson.plugins.gerrit.trigger.hudsontrigger.GerritTrigger start FINE: Start project: hudson.model.FreeStyleProject@12143d8 [test gerrit trigger 1] So I'm not seeing stop being called. This was on 1.457 which I had a version of lying around. I'll try an updated version later.

          Sam Xiao added a comment -

          We are experiencing the same issue on the following versions:

          Jenkins ver. 1.514
          Gerrit Trigger Plugin: 2.90

          Sam Xiao added a comment - We are experiencing the same issue on the following versions: Jenkins ver. 1.514 Gerrit Trigger Plugin: 2.90

          Paul Jackson added a comment -

          I've been looking at this, and made a change that appears to fix the problem.

          I added a new method getGerritEventListenerName to GerritEventListener. I then implemented this in GerritTrigger to return the name of the job. Then in GerritHandler#addListener I can check to see if the same job is already in the gerritEventListeners set. This appears to solve the problem but is it a reasonable approach?

          I am also unsure if I should be removing the old listener and adding the new one, or simply not adding the new listener.

          Paul Jackson added a comment - I've been looking at this, and made a change that appears to fix the problem. I added a new method getGerritEventListenerName to GerritEventListener. I then implemented this in GerritTrigger to return the name of the job. Then in GerritHandler#addListener I can check to see if the same job is already in the gerritEventListeners set. This appears to solve the problem but is it a reasonable approach? I am also unsure if I should be removing the old listener and adding the new one, or simply not adding the new listener.

          Seeing this on Jenkins 1.520 with Gerrit Trigger 2.10.0. We never had the problem before, but recently started generating configs from templates and reloading from disk.

          Paul, could you post a diff? Robert, any opinions thoughts on Paul's approach?

          Thanks,
          Andy

          Andrew Erickson added a comment - Seeing this on Jenkins 1.520 with Gerrit Trigger 2.10.0. We never had the problem before, but recently started generating configs from templates and reloading from disk. Paul, could you post a diff? Robert, any opinions thoughts on Paul's approach? Thanks, Andy

          Adam Pryce added a comment - - edited

          Is there any workaround to remove duplicate triggers once they are there? They've made Jenkins & Gerrit almost unusable.

          Jenkins: 1.523
          Gerrit Trigger Plugin: 2.10.1

          Update: Restarting the Jenkins server seems to clear the duplicate triggers. I suggest that before bouncing the server that admins prepare Jenkins for shutdown using the link in managing Jenkins and let all running jobs finish.

          Adam Pryce added a comment - - edited Is there any workaround to remove duplicate triggers once they are there? They've made Jenkins & Gerrit almost unusable. Jenkins: 1.523 Gerrit Trigger Plugin: 2.10.1 Update : Restarting the Jenkins server seems to clear the duplicate triggers. I suggest that before bouncing the server that admins prepare Jenkins for shutdown using the link in managing Jenkins and let all running jobs finish.

          Brint Kriebel added a comment -

          Another solution that worked for me if you aren't able to restart your Jenkins server:

          1. Disable all Gerrit Triggers on projects.
          2. Run the List_Gerrit_Trigger_Listeners.groovy script to verify that there are still zombie triggers
          3. Modify the groovy script to add "listener.stop();" at the bottom of the "for (listener in listeners)" loop.
          4. Re-enable desired Gerrit Triggers

          The stop command will close out all of the zombie listeners without needing to restart the whole system.

          Brint Kriebel added a comment - Another solution that worked for me if you aren't able to restart your Jenkins server: 1. Disable all Gerrit Triggers on projects. 2. Run the List_Gerrit_Trigger_Listeners.groovy script to verify that there are still zombie triggers 3. Modify the groovy script to add "listener.stop();" at the bottom of the "for (listener in listeners)" loop. 4. Re-enable desired Gerrit Triggers The stop command will close out all of the zombie listeners without needing to restart the whole system.

          Hugo Ares added a comment -

          The problem seems to be in the equals/hashcode methods of gerritTrigger class. It is implemented which should prevent adding twice the same gerritTrigger(for the same job) in the gerrit event listener set but the problem is that it's relying on the AbstractProject (myProject field) equals/hashcode methods but they are not implemented, it is using the default equals/hashcode of object class. After reloading the configuration from disk, 2 abstractProject instances representing the same job will not return true when compared with equals.

          I think the proper way of fixing this would be to implement the equals/hashcode in AbstractItem based on the fullname (I posted on the subject in JenkinsDev: https://groups.google.com/forum/#!topic/jenkinsci-dev/CwbsCJAzcOY).
          Another way would be to patch gerritTrigger equals/hashcode methods to compare the fullname of the project instead of the project itself.

          I just finished implementing the second option(quick fix for now, while waiting for Jenkins community feedback on option 1) and I no longer have multiple builds. I still have more testing before saying it's done but it looks good so far.

          Hugo Ares added a comment - The problem seems to be in the equals/hashcode methods of gerritTrigger class. It is implemented which should prevent adding twice the same gerritTrigger(for the same job) in the gerrit event listener set but the problem is that it's relying on the AbstractProject (myProject field) equals/hashcode methods but they are not implemented, it is using the default equals/hashcode of object class. After reloading the configuration from disk, 2 abstractProject instances representing the same job will not return true when compared with equals. I think the proper way of fixing this would be to implement the equals/hashcode in AbstractItem based on the fullname (I posted on the subject in JenkinsDev: https://groups.google.com/forum/#!topic/jenkinsci-dev/CwbsCJAzcOY ). Another way would be to patch gerritTrigger equals/hashcode methods to compare the fullname of the project instead of the project itself. I just finished implementing the second option(quick fix for now, while waiting for Jenkins community feedback on option 1) and I no longer have multiple builds. I still have more testing before saying it's done but it looks good so far.

          Hugo Ares added a comment -

          More info

          As rsandell said in is comment:

          When a job is removed or unloaded GerritTrigger.stop is called and that will remove the GerritTrigger instance as a listener.

          This is correct except when reloading the configurations, the stop never gets called, this is why we end up with duplicates GerritTrigger for the same job.

          Fixing the GerritTrigger.equals as I proposed and submitted by docwhat on github (https://github.com/jenkinsci/gerrit-trigger-plugin/pull/77) fix the issue when reloading the configurations without changing the job names but if you rename the job on disk, then reload the configurations, you end up with two gerritTrigger, one good and the other one for the the old job name.

          To fix the problem properly, we would need the GerritTrigger.stop to get called when reloading the configurations or an event when reloading the configurations so we could remove the all the GerritTriggers.

          For now, I think this fix is good since it fixes the problem when reloading the configurations (which is the more common cause of it) and we could open a more specific JIRA issue for the duplicates when renaming on disk then reloading the configurations, this second use case is not really good even if it's supported because it won't update all the jobs that calls the renamed job. For this reason, we only rename jobs through Jenkins.

          Hugo Ares added a comment - More info As rsandell said in is comment: When a job is removed or unloaded GerritTrigger.stop is called and that will remove the GerritTrigger instance as a listener. This is correct except when reloading the configurations, the stop never gets called, this is why we end up with duplicates GerritTrigger for the same job. Fixing the GerritTrigger.equals as I proposed and submitted by docwhat on github ( https://github.com/jenkinsci/gerrit-trigger-plugin/pull/77 ) fix the issue when reloading the configurations without changing the job names but if you rename the job on disk, then reload the configurations, you end up with two gerritTrigger, one good and the other one for the the old job name. To fix the problem properly, we would need the GerritTrigger.stop to get called when reloading the configurations or an event when reloading the configurations so we could remove the all the GerritTriggers. For now, I think this fix is good since it fixes the problem when reloading the configurations (which is the more common cause of it) and we could open a more specific JIRA issue for the duplicates when renaming on disk then reloading the configurations, this second use case is not really good even if it's supported because it won't update all the jobs that calls the renamed job. For this reason, we only rename jobs through Jenkins.

          I'm just curious. How come pressing "stop" on the Gerrit management web page doesn't call "stop" on all the GerritTriggers? Are some lost somehow?

          Ciao!

          Christian Höltje added a comment - I'm just curious. How come pressing "stop" on the Gerrit management web page doesn't call "stop" on all the GerritTriggers? Are some lost somehow? Ciao!

          Hugo Ares added a comment - - edited

          In this case, it will stop the connection to the gerrit server.

          Hugo Ares added a comment - - edited In this case, it will stop the connection to the gerrit server.

          Ah. I expected it to remove all the listeners... and start to re-add all the listeners.

          Can that be done?

          Christian Höltje added a comment - Ah. I expected it to remove all the listeners... and start to re-add all the listeners. Can that be done?

          I filed bug JENKINS-19678 to ask for the stop/start/restart buttons to also stop/start/restart the GerritTrigger objects.

          Christian Höltje added a comment - I filed bug JENKINS-19678 to ask for the stop/start/restart buttons to also stop/start/restart the GerritTrigger objects.

          Code changed in jenkins
          User: Christian Höltje
          Path:
          gerrithudsontrigger/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/hudsontrigger/GerritTrigger.java
          http://jenkins-ci.org/commit/gerrit-trigger-plugin/69d03a2a6a2b0c10a64c5a5dff5bb78a36f96b53
          Log:
          GerritTrigger.equals/hashCode – don't use AbstractProject

          AbstractProjects are very temporary and shouldn't be used for
          comparison. Instead, AbstractProject.getFullName() should be
          used.

          JENKINS-10709: multiple builds are triggered for one change in Gerrit

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Christian Höltje Path: gerrithudsontrigger/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/hudsontrigger/GerritTrigger.java http://jenkins-ci.org/commit/gerrit-trigger-plugin/69d03a2a6a2b0c10a64c5a5dff5bb78a36f96b53 Log: GerritTrigger.equals/hashCode – don't use AbstractProject AbstractProjects are very temporary and shouldn't be used for comparison. Instead, AbstractProject.getFullName() should be used. JENKINS-10709 : multiple builds are triggered for one change in Gerrit

          Jenkins: 1.533
          gerrit plugin: 2.1.10

          Gerrit listeners duplicates when you reload config from disk. As a workaround, restart Jenkins.

          miguel aranha baldi horlle added a comment - Jenkins: 1.533 gerrit plugin: 2.1.10 Gerrit listeners duplicates when you reload config from disk. As a workaround, restart Jenkins.

          Hugo Ares added a comment -

          pull request https://github.com/jenkinsci/gerrit-trigger-plugin/pull/77 that was merge resolve the issue.
          Next release will resolve the issue.

          Hugo Ares added a comment - pull request https://github.com/jenkinsci/gerrit-trigger-plugin/pull/77 that was merge resolve the issue. Next release will resolve the issue.

          Jesse Glick added a comment -

          So, fixed then.

          Jesse Glick added a comment - So, fixed then.

          Paul Jackson added a comment -

          I've fetched the latest source and tested locally. I can confirm it is fixed for us.

          Paul Jackson added a comment - I've fetched the latest source and tested locally. I can confirm it is fixed for us.

          Vlad Aginsky added a comment -

          Hi Hugo and all, I still observe this on version 1.535, ang gerrit trigger 2.10.1.
          What release was supposed to contain the solution?

          Vlad Aginsky added a comment - Hi Hugo and all, I still observe this on version 1.535, ang gerrit trigger 2.10.1. What release was supposed to contain the solution?

          Hugo Ares added a comment - - edited

          This is fixed in 2.11.0 which is not released yet. Robert just created version 2.11.0-beta-1[1] so I guess we can expect a release soon.

          [1]https://groups.google.com/forum/#!topic/jenkinsci-dev/K41zja0b5mo

          Hugo Ares added a comment - - edited This is fixed in 2.11.0 which is not released yet. Robert just created version 2.11.0-beta-1 [1] so I guess we can expect a release soon. [1] https://groups.google.com/forum/#!topic/jenkinsci-dev/K41zja0b5mo

          rsandell added a comment -

          Yepp, I just released 2.11.0-beta-1, it should be available in the experimental update center soon. And I am looking for brave beta testers to try it out

          rsandell added a comment - Yepp, I just released 2.11.0-beta-1, it should be available in the experimental update center soon. And I am looking for brave beta testers to try it out

          Curtis Kline added a comment -

          I would be willing to try it out on our test instance. Right now I am removing the Gerrit Trigger from all of the jobs we recently implemented it on due to this bug. Ah well, it was fun while it lasted! (Jenkins 1.509.4 and GT 2.10.1)

          Curtis Kline added a comment - I would be willing to try it out on our test instance. Right now I am removing the Gerrit Trigger from all of the jobs we recently implemented it on due to this bug. Ah well, it was fun while it lasted! (Jenkins 1.509.4 and GT 2.10.1)

          Vlad Aginsky added a comment -

          We still suffer from this. Is there any progress? I would like to be a beta tester if this can help.

          Vlad Aginsky added a comment - We still suffer from this. Is there any progress? I would like to be a beta tester if this can help.

          Hugo Ares added a comment -

          Vlad, have you tried the latest 2.11 beta(2.11.0-beta-4)?

          Hugo Ares added a comment - Vlad, have you tried the latest 2.11 beta(2.11.0-beta-4)?

          Curtis Kline added a comment -

          I am concerned about this bug being marked "fixed". I am running 2.11.1 of the trigger plugin, and I believe it still exhibits this behavior.

          Curtis Kline added a comment - I am concerned about this bug being marked "fixed". I am running 2.11.1 of the trigger plugin, and I believe it still exhibits this behavior.

          sakshi sood added a comment -

           I am using Jenkins 2.19.3 and gerrit trigger version 2.23.0  and I believe it still exhibits this behaviour

          sakshi sood added a comment -  I am using Jenkins 2.19.3 and gerrit trigger version 2.23.0  and I believe it still exhibits this behaviour

            rsandell rsandell
            edwin_kempin Edwin Kempin
            Votes:
            23 Vote for this issue
            Watchers:
            37 Start watching this issue

              Created:
              Updated:
              Resolved: