Hi,

      I originally posted this on the mailing list but the lack of replies forced me to create this issue. See the post here: https://groups.google.com/forum/?fromgroups#!topic/jenkinsci-users/s1HyARUbfbs

      Is there any way the Perforce plugin can be changed so that it doesn't do the redundant sync? It can be a real performance killer if the view you're syncing to is particularly large.

      Regards,
      Tom.

          [JENKINS-12983] Redundant syncs when using matrix jobs

          Thomas Fields added a comment -

          How many votes are required for this to be fixed? It'd be nice if this was fixed soon.

          Thomas Fields added a comment - How many votes are required for this to be fixed? It'd be nice if this was fixed soon.

          Changing this to "improvement" as the code is functioning as designed. It's just that Jenkins can do it smarter.

          Kohsuke Kawaguchi added a comment - Changing this to "improvement" as the code is functioning as designed. It's just that Jenkins can do it smarter.

          Chris Withers added a comment -

          Out of curiosity: why was it designed this way? I can't see any benefit to executing the pre-build steps in the parent job; they're not using by the child jobs at all, as best I can tell...

          Chris Withers added a comment - Out of curiosity: why was it designed this way? I can't see any benefit to executing the pre-build steps in the parent job; they're not using by the child jobs at all, as best I can tell...

          Ryan Smith added a comment -

          This feature causes my builds to fail sporadically. Most of my jobs are set up to use the same custom workspace. So when this second sync occurs on top of a running job, it starts modifying the workspace while building.

          Ryan Smith added a comment - This feature causes my builds to fail sporadically. Most of my jobs are set up to use the same custom workspace. So when this second sync occurs on top of a running job, it starts modifying the workspace while building.

          Chris Withers added a comment -

          Ryan: yep, that was my experience too. The workaround is to use the Tie Parent plugin, and tie all matrix parent jobs to a "sacrificial node", that node should do nothing else other than run matrix parent jobs.

          Chris Withers added a comment - Ryan: yep, that was my experience too. The workaround is to use the Tie Parent plugin, and tie all matrix parent jobs to a "sacrificial node", that node should do nothing else other than run matrix parent jobs.

          Thomas Fields added a comment -

          So any ideas when this improvement might be implemented? Seems like its something jenkins users want to see fixed.

          Tom

          Thomas Fields added a comment - So any ideas when this improvement might be implemented? Seems like its something jenkins users want to see fixed. Tom

          Matrix job by design require both head build and all axes to run SCM operations :

          • head to determine the revision to build, and trigger all axes builds accordingly
          • axes to prepare workspace in sync with revision to build.

          see for sample Git-plugin support for matrix jobs : https://github.com/jenkinsci/git-plugin/blob/master/src/main/java/hudson/plugins/git/GitSCM.java#L845

          probably some SCM can do some light weight operation when they already know the revision to checkout, vs a full fetch from matrix head, but they will anyway have to do some stuff BOTH on axes and head.

          Nicolas De Loof added a comment - Matrix job by design require both head build and all axes to run SCM operations : head to determine the revision to build, and trigger all axes builds accordingly axes to prepare workspace in sync with revision to build. see for sample Git-plugin support for matrix jobs : https://github.com/jenkinsci/git-plugin/blob/master/src/main/java/hudson/plugins/git/GitSCM.java#L845 probably some SCM can do some light weight operation when they already know the revision to checkout, vs a full fetch from matrix head, but they will anyway have to do some stuff BOTH on axes and head.

          Specifically for Perforce, I've managed to get some improvement by running a local Perforce proxy. This caches commonly accessed files locally. The first query pulls all of the files into the cache, and then the synch for each matrix configuration just reads the files out of the cache.

          Charles

          Charles Blessing added a comment - Specifically for Perforce, I've managed to get some improvement by running a local Perforce proxy. This caches commonly accessed files locally. The first query pulls all of the files into the cache, and then the synch for each matrix configuration just reads the files out of the cache. Charles

          Andy Bigos added a comment -


          As the matrix functionality has now moved from the core maybe this issue should be moved to the Matrix Project plugin, or maybe to the Perforce plugin as it seems the 'fix' is most likely to be optimizations to reduce syncing in the P4 plugins itself (if possible).

          Andy Bigos added a comment - As the matrix functionality has now moved from the core maybe this issue should be moved to the Matrix Project plugin, or maybe to the Perforce plugin as it seems the 'fix' is most likely to be optimizations to reduce syncing in the P4 plugins itself (if possible).

          Rob Petti added a comment -

          The issue is that the matrix plugin syncs during the main job, which shouldn't be mandatory as it doesn't normally do anything. Perforce is as optimized as it can possibly get.

          Rob Petti added a comment - The issue is that the matrix plugin syncs during the main job, which shouldn't be mandatory as it doesn't normally do anything. Perforce is as optimized as it can possibly get.

            kohsuke Kohsuke Kawaguchi
            tfields Thomas Fields
            Votes:
            20 Vote for this issue
            Watchers:
            18 Start watching this issue

              Created:
              Updated: