-
Improvement
-
Resolution: Unresolved
-
Major
-
None
I have a Jenkins instance which builds about 40 projects from a remote mercurial repository. I have caching and repo sharing enabled. All updates are sent as notifications to Jenkins.
When a new update is pushed to the main repo, a notification is sent to Jenkins which duly updates it's local repo cache. Then 40 jobs want to check for updates to their module (mercurial polling log). In order to do this they acquire the master cache lock. Poll the remote repo for any new changes and decide if they need to be build.
As the remote poll takes 4 seconds it can take the last project about 4*40 = 160 seconds to even be able to start after the commit is pushed which sort of defaults the purpose of having a cache (you hit your hg server 40x harder though).
It would be better if the individual jobs didn't re-query the remote server when they are triggered by a commit.
Could an option be added which stops the poller from updating from the remote site.
The original purpose of caching was to avoid transferring lots of changesets over and over, but it is true that the tree comparison can take a bit of round-trip time as well.
Turning off cache updates from all polling would not work, since then no jobs would be triggered and nothing would happen at all; but it would make sense to avoid the update if it can be determined that the polling was scheduled before the last update was performed, so that only the first job’s poll updates the cache. Unfortunately I do not how to implement that; SCM.compareRemoteRevisionWith does not offer this information.
Another option would be to skip the cache update if called from polling and the last update was within the last 60 seconds. One minute is the finest granularity at which you can schedule polling, so this makes a certain amount of sense.