-
Improvement
-
Resolution: Fixed
-
Major
-
None
Would it be possible to separate the loadrules directories from the directories passed to lshistory? I need multiple VOBs to do my build, but I only want to see changes from a couple of small directories. This would make my builds much faster because lshistory would return much quicker and it would allow me to poll for changes to a limited set of files.
If this change is not possible, would it be possible to not call lshistory at all? Most of my builds are on a timer, so I do not need polling, and I am ok not seeing all of the changes since the last build.
[JENKINS-5394] Limit scope of lshistory
On the first point - no, I don't want to separate the load rules from the lshistory paths. I very deliberately amalgamated the two as of 1.0 - there was too much inconsistency on how they were handled, and it was a giant pain. I know that a one-size-fits-all approach means that it's not optimal in all cases, but it's much, much easier to support this way. You can still have only certain changes be processed by using the excluded regions option, though that won't make a difference in the lshistory time, since the excluded regions logic is applied to the output of lshistory.
How long are you going between builds, for the lshistory to take a significant time? Once we switched to lshistory -all (from the old model of lshistory -recurse), I saw huge improvements in lshistory speed - a few orders of magnitude. If you're not going months between builds, I can't imagine it taking that long.
And on the second point - I'll consider that as an option.
tla612
added a comment - We are building every night and the lshistory is taking about 20 minutes. Thanks.
tla612
added a comment - We are building every night and the lshistory is taking about 20 minutes. Thanks. We have the same problem and our vob is very large. We have lshistory times from 20 mins until more then two hours. This makes continuous integration with hudson quite impossible.
What would help in my opinion is:
1. When clearcase polling considers the 'Excluded Regions' before lshistory in the whay, that it executes the lshistory command only in the path('s) which are left after applying the RegEx. That means it changes in the directory on the vob and executes the lshistory. If more lshistory commands are needed, the results have to be merged, but this shouldn't be a big problem. I tried it on our configuration on command line and this solution is faster using -recursive, otherwise the path is ignored.
2. What also could be done is introducing a new optional configuration field where the poll locations of lshistory can be configured.
3. Since we use snapshot views with update, another solution for us would be to use snapshot update for polling. If there is something updated, the build can be started. This would save the time of lshistory polling.
It seems to me that clearcase lshistory doesn't considers paths when using -all switch. I would prefer to make it configurable if lshistory uses -all and ignores paths or use -r(ecursive) and don't ignore paths. Then the user can decide which solution is faster on his machine.
For our case we would prefer solution three, because we would save some polling time on snapshot views. In my thought's remember the starting time of the last poll and use it for the current would help reducing polling time.
Thanx in advance,
Alfred
schmid03
added a comment - - edited We have the same problem and our vob is very large. We have lshistory times from 20 mins until more then two hours. This makes continuous integration with hudson quite impossible.
What would help in my opinion is:
1. When clearcase polling considers the 'Excluded Regions' before lshistory in the whay, that it executes the lshistory command only in the path('s) which are left after applying the RegEx. That means it changes in the directory on the vob and executes the lshistory. If more lshistory commands are needed, the results have to be merged, but this shouldn't be a big problem. I tried it on our configuration on command line and this solution is faster using -recursive, otherwise the path is ignored.
2. What also could be done is introducing a new optional configuration field where the poll locations of lshistory can be configured.
3. Since we use snapshot views with update, another solution for us would be to use snapshot update for polling. If there is something updated, the build can be started. This would save the time of lshistory polling.
It seems to me that clearcase lshistory doesn't considers paths when using -all switch. I would prefer to make it configurable if lshistory uses -all and ignores paths or use -r(ecursive) and don't ignore paths. Then the user can decide which solution is faster on his machine.
For our case we would prefer solution three, because we would save some polling time on snapshot views. In my thought's remember the starting time of the last poll and use it for the current would help reducing polling time.
Thanx in advance,
Alfred schmid03
added a comment - For us it's at least a major problem, since it makes hudson clearcase polling for continuous integration unusable.
schmid03
added a comment - For us it's at least a major problem, since it makes hudson clearcase polling for continuous integration unusable. 
anb0s
added a comment - Just added some comments to similar issues:
http://issues.jenkins-ci.org/browse/JENKINS-7481
http://issues.jenkins-ci.org/browse/JENKINS-8949
We have already patched it and works for our use case. I can provide patch if usable for other users...
anb0s
added a comment - Just added some comments to similar issues:
http://issues.jenkins-ci.org/browse/JENKINS-7481
http://issues.jenkins-ci.org/browse/JENKINS-8949
We have already patched it and works for our use case. I can provide patch if usable for other users... anb0s
added a comment - Integrated patch at https://github.com/anb0s/clearcase-plugin/tree/integrate_1.3.8
anb0s
added a comment - Integrated patch at https://github.com/anb0s/clearcase-plugin/tree/integrate_1.3.8 
Updating the summary.