Hello everybody,

      at first: Thanks for the great plugin. It provides a lot of helpful information about git statistics. Sadly, i observe random crashes during build of main branch. The call stack is:

      an exception which occurred:
      in field io.jenkins.plugins.forensics.miner.FileStatistics.commits
      in object io.jenkins.plugins.forensics.miner.FileStatistics@c36f291c
      in field io.jenkins.plugins.forensics.miner.RepositoryStatistics.fileStatistics
      in object io.jenkins.plugins.forensics.miner.RepositoryStatistics@1ee870fe
      in field io.jenkins.plugins.analysis.core.steps.AnnotatedReport.aggregatedRepositoryStatistics
      in object io.jenkins.plugins.analysis.core.steps.AnnotatedReport@6e530bdf
      in field com.cloudbees.groovy.cps.impl.BlockScopeEnv.locals
      in object com.cloudbees.groovy.cps.impl.BlockScopeEnv@16f316cc
      in field com.cloudbees.groovy.cps.impl.ProxyEnv.parent
      in object com.cloudbees.groovy.cps.impl.BlockScopeEnv@1515a663
      in field com.cloudbees.groovy.cps.impl.CallEnv.caller
      in object com.cloudbees.groovy.cps.impl.FunctionCallEnv@4f5e024b
      in field com.cloudbees.groovy.cps.Continuable.e
      in object org.jenkinsci.plugins.workflow.cps.SandboxContinuable@1ec29fbe
      in field org.jenkinsci.plugins.workflow.cps.CpsThread.program
      in object org.jenkinsci.plugins.workflow.cps.CpsThread@7ea7c27b
      in field org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.threads
      in object org.jenkinsci.plugins.workflow.cps.CpsThreadGroup@5dc66153
      in object org.jenkinsci.plugins.workflow.cps.CpsThreadGroup@5dc66153
      Caused: java.util.ConcurrentModificationException
      at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:911)
      at java.util.ArrayList$Itr.next(ArrayList.java:861)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:581)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1082)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1040)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:920)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:568)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1082)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1040)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:920)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1082)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1040)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:920)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:568)
      at org.jboss.marshalling.river.BlockMarshaller.doWriteObject(BlockMarshaller.java:65)
      at org.jboss.marshalling.river.BlockMarshaller.writeObject(BlockMarshaller.java:56)
      at org.jboss.marshalling.MarshallerObjectOutputStream.writeObjectOverride(MarshallerObjectOutputStream.java:50)
      at org.jboss.marshalling.river.RiverObjectOutputStream.writeObjectOverride(RiverObjectOutputStream.java:179)
      at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:344)
      at java.util.HashMap.internalWriteEntries(HashMap.java:1793)
      at java.util.HashMap.writeObject(HashMap.java:1363)
      at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      at java.lang.reflect.Method.invoke(Method.java:498)
      at org.jboss.marshalling.reflect.JDKSpecific$SerMethods.callWriteObject(JDKSpecific.java:156)
      at org.jboss.marshalling.reflect.SerializableClass.callWriteObject(SerializableClass.java:191)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1028)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:920)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1082)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1040)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:920)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1082)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1040)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1019)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:920)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1082)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1040)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1019)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:920)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1082)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1040)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1019)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:920)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1082)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1040)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:920)
      at org.jboss.marshalling.river.BlockMarshaller.doWriteObject(BlockMarshaller.java:65)
      at org.jboss.marshalling.river.BlockMarshaller.writeObject(BlockMarshaller.java:56)
      at org.jboss.marshalling.MarshallerObjectOutputStream.writeObjectOverride(MarshallerObjectOutputStream.java:50)
      at org.jboss.marshalling.river.RiverObjectOutputStream.writeObjectOverride(RiverObjectOutputStream.java:179)
      at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:344)
      at java.util.concurrent.ConcurrentSkipListMap.writeObject(ConcurrentSkipListMap.java:1437)
      at sun.reflect.GeneratedMethodAccessor475.invoke(Unknown Source)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      at java.lang.reflect.Method.invoke(Method.java:498)
      at org.jboss.marshalling.reflect.JDKSpecific$SerMethods.callWriteObject(JDKSpecific.java:156)
      at org.jboss.marshalling.reflect.SerializableClass.callWriteObject(SerializableClass.java:191)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1028)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:920)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1082)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1040)
      at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:920)
      at org.jboss.marshalling.AbstractObjectOutput.writeObject(AbstractObjectOutput.java:58)
      at org.jboss.marshalling.AbstractMarshaller.writeObject(AbstractMarshaller.java:111)
      at org.jenkinsci.plugins.workflow.support.pickles.serialization.RiverWriter.lambda$writeObject$0(RiverWriter.java:144)
      at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.GroovySandbox.runInSandbox(GroovySandbox.java:237)
      at org.jenkinsci.plugins.workflow.support.pickles.serialization.RiverWriter.writeObject(RiverWriter.java:143)
      at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.saveProgram(CpsThreadGroup.java:559)
      at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.saveProgram(CpsThreadGroup.java:536)
      at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.saveProgramIfPossible(CpsThreadGroup.java:519)
      at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:443)
      at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$400(CpsThreadGroup.java:96)
      at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:314)
      at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:278)
      at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:67)
      at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:139)
      at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
      at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68)
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
      at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      at java.lang.Thread.run(Thread.java:748)
      
      

       i would be happy to assist with testing and, please, ping me, if more information is need.

          [JENKINS-67145] Random ConcurrentModificationException

          Ulli Hafner added a comment -

          Which step are you using in the warnings plugin?

          Ulli Hafner added a comment - Which step are you using in the warnings plugin?

          Thank you for your reply. The pipeline i'm using as pseudo code:

           

          Jenkinsfile
          def CompilerWarnings = []
          builders[builder] = {
              node(builder) {
                  CompilerWarnings += [scanForIssues(filters: filter, tool: tool, sourceDirectory: "${WORKSPACE}")]
              }
          }
          builders.failFast = true
          parallel builders
          if(!CompilerWarnings.isEmpty()) {
              publishIssues failOnError: false, ignoreFailedBuilds: true, issues: CompilerWarnings, trendChartType: 'AGGREGATION_TOOLS'
          }
          

          Would be great to have a possibility to append scans from parallel region, but it's another story

          Alexander Falkenstern added a comment - Thank you for your reply. The pipeline i'm using as pseudo code:   Jenkinsfile def CompilerWarnings = [] builders[builder] = { node(builder) { CompilerWarnings += [scanForIssues(filters: filter, tool: tool, sourceDirectory: "${WORKSPACE}" )] } } builders.failFast = true parallel builders if (!CompilerWarnings.isEmpty()) { publishIssues failOnError: false , ignoreFailedBuilds: true , issues: CompilerWarnings, trendChartType: 'AGGREGATION_TOOLS' } Would be great to have a possibility to append scans from parallel region, but it's another story

          Ulli Hafner added a comment -

          Can you see from the log which of the steps caused the exception (scanForIssues or publishIssues)?

          Ulli Hafner added a comment - Can you see from the log which of the steps caused the exception ( scanForIssues or publishIssues )?

          Ulli Hafner added a comment -

          And another question: which durability settings do you use for your pipeline? See https://www.jenkins.io/doc/book/pipeline/scaling-pipeline/

          Ulli Hafner added a comment - And another question: which durability settings do you use for your pipeline? See https://www.jenkins.io/doc/book/pipeline/scaling-pipeline/

          Alexander Falkenstern added a comment - - edited

          > Can you see from log which of the steps...
          Seems to be a publishIssues:

          > And another question...
          It's MAX_SURVIVABILITY:

          Alexander Falkenstern added a comment - - edited > Can you see from log which of the steps... Seems to be a publishIssues: > And another question... It's MAX_SURVIVABILITY:

          Ulli Hafner added a comment -

          Ok, thanks. I am not sure if I am doing something wrong in my plugin or if this is a general problem with the intermediate serialization of steps while they are running.

          A workaround would be to switch to another durability setting (performance).

          Ulli Hafner added a comment - Ok, thanks. I am not sure if I am doing something wrong in my plugin or if this is a general problem with the intermediate serialization of steps while they are running. A workaround would be to switch to another durability setting (performance).

          Ulli Hafner added a comment -

          Devin looked at it and noticed that my code needs to be improved to make it behave correctly in such situations.

          The CPS VM thread is responsible for saving the Pipeline's execution state, so if you are using a non-blocking step execution (and it looks like you are), it is possible that your step is executing on a background thread while the Pipeline program is being saved on the CPS VM thread. You should account for this in any mutable data reachable from non-blocking step executions that is part of the execution's serialized state, for example by using CopyOnWriteArrayList, replacing fields rather than mutating them, using writeReplace, etc.

          The exception in the Jira ticket suggests that one of the objects inside of one of the AnnotatedReport instances returned by a scanIssues step, which is stored a local variable in the user's Pipeline, is being modified. Perhaps the issue is that there is a filename match in the FileStatistics between the issues collected by the user's different parallel branches, so when this code runs for the first time inside of AnnotatedReport.addAll in PublishIssuesStep.Execution.Run it uses the exact FileStatistics instance from an existing AnnotatedReport instance, but when it runs after that it modifies the previous FileStatistics instance here, and then since the FileStatistics instance is also reachable via one of the AnnotatedReport local variables in the Pipeline you have a potential ConcurrentModificationError depending on the serialization timing.

          Ulli Hafner added a comment - Devin looked at it and noticed that my code needs to be improved to make it behave correctly in such situations. The CPS VM thread is responsible for saving the Pipeline's execution state, so if you are using a non-blocking step execution (and it looks like you are), it is possible that your step is executing on a background thread while the Pipeline program is being saved on the CPS VM thread. You should account for this in any mutable data reachable from non-blocking step executions that is part of the execution's serialized state, for example by using CopyOnWriteArrayList, replacing fields rather than mutating them, using writeReplace, etc. The exception in the Jira ticket suggests that one of the objects inside of one of the AnnotatedReport instances returned by a scanIssues step, which is stored a local variable in the user's Pipeline, is being modified. Perhaps the issue is that there is a filename match in the FileStatistics between the issues collected by the user's different parallel branches, so when this code runs for the first time inside of AnnotatedReport.addAll in PublishIssuesStep.Execution.Run it uses the exact FileStatistics instance from an existing AnnotatedReport instance, but when it runs after that it modifies the previous FileStatistics instance here, and then since the FileStatistics instance is also reachable via one of the AnnotatedReport local variables in the Pipeline you have a potential ConcurrentModificationError depending on the serialization timing.

          Thank you and of course Devin for quick problem analysis!

          Alexander Falkenstern added a comment - Thank you and of course Devin for quick problem analysis!

            drulli Ulli Hafner
            falkena Alexander Falkenstern
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated: