Status: Open (View Workflow)
To reduce the scm "noise" it may be good to be able to exclude some files from syncing.
In our environment, the file hudson.plugins.disk_usage.DiskUsageProjectActionFactory.xml was synced every day, the changes reflect the current disc usage only this might be not necessary.
- is duplicated by
JENKINS-28391 Allow excluding files from "Manual synchronization includes"
- is related to
JENKINS-19659 Allow to deactivate bundled scm sync config strategies
JENKINS-19660 Allow to commit SYSTEM-modified files only periodically
JENKINS-22540 Jenkins config files are not tracked
JENKINS-18189 disk-usage pluging: separate configuration and history serialization into different files
In our case, it was more an issue of the disk usage plugin which doesn't separate data from configuration.
But as there was (as I know) no naming convention/location for files storing data, every plugin may provide some .xml files not necessarily have to be synced in scm.
My personal opinion was to give the users the freedom to configure whatever was possible. This make the things flexible and reduces maintenance.
It would be the case if the BasicPluginsConfigScmSyncStrategy would only define a white list (more restricted as today) of sync'ed xml files.
Then, users could play with manual includes feature to add their own files they want to sync.
You have to build up a list of items for syncing anyway, so an exclude list was possible. In case of manual includes especially for directories a exclude list may be handy too.
The scm-sync-configration plugin was on the way to be a save and recovery tool for Jenkins configuration, so it should be possible to fine tune the content synced to scm.
A fixed white list probably requires maintenance for every change/addition of plugins and core.
An exclude list gives the responsibility to the administrator and no white list have to be maintained anymore.
This will make the plugin more flexible.
Exclude lists give an administrator the freedom to save only what he need for his purposes, e. g. save only several job configurations and plugin settings.
This may be needed if there are temporary or not common jobs and plugins are used on a Jenkins server.
An other way to realize this may be a maintainable white list (as xml or plain text) predefined by the plugin. So an administrator could edit this list if necessary.
But this will break the full control over the WEB interface.
This would, for sure, be a gain in flexibility. I completely agree with you.
... but what I don't want is to give administrators the power to shoot themselves in the foot.
To my POV, syncing an entire directory is not recommended.
Since Jenkins is storing everyting in XML files, you'll often have lots of noise brought by lots of unneeded xml files in file hierarchies.
It was the idea behind manual includes : target only important specific files that would be backed up.
I know this is my personal POV which is not obviously shared by everyone
I'll consider adding the exclude list someday, but it's not on the top of my todolist right now.
Its just a proposal for an improvement.
BTW syncing a directory is possible, e. g. I have the need of including one to save a necessary tool needed by some jobs.
There also may be directories containing supporting scripts needed for processing.
We have to be able to restore the complete environment for building a released SW. So we should be able to restore the used Jenkins environment/version too.
So we tag the synced Jenkins environment to be able to restore it.
I agree with Wolfgang that exclusion lists is really important enhancement. I had to disable scm-sync-configuration because I use ci-game which modifies user's xml after each commit (changing score for user who has committed change). In this particular situation, users' config files are modified too often making too many scm-sync-configuration commits (each user's commit means additional scm-sync-configuration commit). I cannot afford such amount of commits cause it makes big mess in our project. Exclusions will be great advantage.
BTW. Do you think it is somehow possible to exclude whole users directory by svn ignore set on this directory?
@Lukasz I see what you're describing as another improvement possibility : allowing to define SYSTEM-commited files only periodically.
I'd imagine some additionnal config file allowing to say "only sync SYSTEM-modified files every XXX hours"
The more difficult would be to identify how to translate this on multiple files commited.
I like your proactive approach. It is indeed interesting improvement. However there is already existing solution to achieve periodical commits of any files (http://jenkins-ci.org/content/keeping-your-configuration-and-data-subversion) while there is no way to exclude default paths from scm-sync-configuration. This is the reason why exclusions (or possibility to define all includes from scratch without any default path) is more important from my point of view.
In fact, in correlation with this feature, I was thinking about making default bundled synchronization strategies (namely JENKINS_HOME/hudson*.xml, JENKINS_HOME/config.xml, JENKINS_HOME/jobs/*/config.xml, JENKINS_HOME/users/*/config.xml) activables or not (with simple checkbox)
By default, every strategies would be activated, but everybody could deactivate specific strategies due to specific cases.
I would greatly prefer this approach than the exclude one.
Just created JENKINS-19659 & JENKINS-19660
@Wolfgang with this in mind, can I close current issue ?
It would be useful to be able to exclude xml elements in included config files from sync, like its done for the job-config-history plugin also.
cforce, please take time to read the reasons why I don't want to exclude files, instead of cross commenting without providing any additionnal arguments.
How else shall i make it possible to sync user's configs importnat settings changes, but keep noise down like issues for every build being failed or fixed. The jenkins game updates the user's pürofile with + or -+1 points for in user profile.xml to often. I only thing to rid of this by being able to explude changes in this setions triggering updates. I don't wanna exlude the content if syncec, but the cause's triggering this syncs.
Ok my bad, didn't understood you were talking about specific file "chunks" avoiding a new commit.
To be honest, what you ask seems really complicated to implement (not triggering a commit based on the chunk modification, in a generic way).
But JENKINS-19660 could bring a part of the answer : this +/-1 are SYSTEM modifications, so we shouldn't take them into account right now.
If I understand you correctly you think that having an exclusion list is not a good option because including a directory is a bad choice, because it leads to too much noise. I disagree. If you list individual files you miss configuration files of new plugins you install if they don't follow a standard pattern. And if you exclude all files which would lead to too much noise then you also address the noise issue. Of course if you install a new plugin which creates a new file which leads to too much noise you have to add that to the exclusion filter (the same was as you now have to include others to the inclusion filter). But I would prefer to have to manually update the filter in that case because if I forget, I have the less big problem (too much noise instead of missing potentially important configurations).
Not really in favor to add excludes because, to my POV, it will complexify things
I would rather be in favor to be more restrictive in the BasicPluginsConfigScmSyncStrategy default implementation (see here)