Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-31308

Creating folder removes existing views within that

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Minor Minor
    • job-dsl-plugin
    • None
    • jenkins 1.625.1
      job-dsl-plugin 1.39
      cloudbees-folder 5.0

      I have a 2 level scheme for using Job DSL. The top level seed job creates folders and seed jobs within them. The per-folder seed job creates jobs and views within that folder.

      If I run the top level seed after the per-folder seed then jobs remain but the views are removed.

      This can be seen by running the following in a "Process Job DSLs" step (with action for removed jobs and views set to "Ignore"). Run this top level seed, then the afolder/seed and the job and view are created. Run the top level seed again and afolder/aview has disappeared.

      folder('afolder')
      job('afolder/seed') {
        description('Create a view and a job')
        steps {
          dsl {
            text("""listView('aview') {}
      job('ajob') {}""")
            lookupStrategy('SEED_JOB')
          }
        }
      }
      

          [JENKINS-31308] Creating folder removes existing views within that

          Thanks for the info. Is there a way of checking if the folder exists from Job DSL code? I can't see one.

          Of course avoiding creating the folder means that changes to displayName or description will not be picked up.

          Russell Gallop added a comment - Thanks for the info. Is there a way of checking if the folder exists from Job DSL code? I can't see one. Of course avoiding creating the folder means that changes to displayName or description will not be picked up.

          Sam Gleske added a comment - - edited

          There's two ways to check for an existing folder: what I call the "file method" and the "runtime method".

          The "file method"

          String project_folder = 'myfolder'
          if(! new File("${JENKINS_HOME}/jobs/${project_folder}/config.xml").exists()) {
              //folder doesn't exist because the config.xml file doesn't exist
              //Therefore, create the folder.
              folder(project_folder) {}
          }
          

          Why does the "file method" work? Because Job DSL executes in the master runtime the Job DSL script can search/manipulate files on the master. This is why you want to be careful about giving out the ability to create Job DSL scripts (for security).

          The "runtime method"

          Query the Jenkins runtime for top level (or sub level) items. If the item exists and is of type folder class then don't create the folder.

          String project_folder = 'myfolder'
          if(!Jenkins.instance.getItem(project_folder)) {
              //folder doesn't exist because item doesn't exist in runtime
              //Therefore, create the folder.
              folder(project_folder) {}
          }
          

          Why does the "runtime method" work? Once again, because Job DSL scripts run in the Jenkins master runtime we can do things like this such as detecting objects in memory (e.g. folder items).

          You can also detect if the item returned is actually a folder by getting the class name. It may make sense to perform this additional safety check to be sure that what you assume is a folder isn't actually another item type (e.g. a job).

          if(Jenkins.instance.getItem(project_folder).getClass().getName() != 'com.cloudbees.hudson.plugins.folder.Folder') {
              //warning: item is not a folder so do something like throw an exception
          }
          

          Update folder values

          You can update changes to the display name and other properties by manipulating the item in the Jenkins runtime with setter and getter functions. See the source code for the folders plugin and core source code for functions.

          • Folder.java, which extends from AbstractFolder.java, which extends from AbstractItem.java, which implements methods renameTo() and setDescription() among other methods. Execute item.save() method to save the folder configuration to disk.

          You can learn more about what methods are available to a folder by executing the following in the Jenkins Script Console.

          //.metaClass.methods*.name.sort().unique() will list methods available on a class
          println Jenkins.instance.getItem(project_folder).metaClass.methods*.name.sort().unique()
          

          If you want to learn how I came up with those solutions, I recently started a blog. I'm going to eventually cover how to reverse engineer Jenkins and write solutions like I described in this comment.

          Sam Gleske added a comment - - edited There's two ways to check for an existing folder: what I call the "file method" and the "runtime method". The "file method" String project_folder = 'myfolder' if (! new File( "${JENKINS_HOME}/jobs/${project_folder}/config.xml" ).exists()) { //folder doesn 't exist because the config.xml file doesn' t exist //Therefore, create the folder. folder(project_folder) {} } Why does the "file method" work? Because Job DSL executes in the master runtime the Job DSL script can search/manipulate files on the master. This is why you want to be careful about giving out the ability to create Job DSL scripts (for security). The "runtime method" Query the Jenkins runtime for top level (or sub level) items. If the item exists and is of type folder class then don't create the folder. String project_folder = 'myfolder' if (!Jenkins.instance.getItem(project_folder)) { //folder doesn 't exist because item doesn' t exist in runtime //Therefore, create the folder. folder(project_folder) {} } Why does the "runtime method" work? Once again, because Job DSL scripts run in the Jenkins master runtime we can do things like this such as detecting objects in memory (e.g. folder items). You can also detect if the item returned is actually a folder by getting the class name. It may make sense to perform this additional safety check to be sure that what you assume is a folder isn't actually another item type (e.g. a job). if (Jenkins.instance.getItem(project_folder).getClass().getName() != 'com.cloudbees.hudson.plugins.folder.Folder' ) { //warning: item is not a folder so do something like throw an exception } Update folder values You can update changes to the display name and other properties by manipulating the item in the Jenkins runtime with setter and getter functions. See the source code for the folders plugin and core source code for functions. Folder.java , which extends from AbstractFolder.java , which extends from AbstractItem.java , which implements methods renameTo() and setDescription() among other methods. Execute item.save() method to save the folder configuration to disk. You can learn more about what methods are available to a folder by executing the following in the Jenkins Script Console. //.metaClass.methods*.name.sort().unique() will list methods available on a class println Jenkins.instance.getItem(project_folder).metaClass.methods*.name.sort().unique() If you want to learn how I came up with those solutions, I recently started a blog . I'm going to eventually cover how to reverse engineer Jenkins and write solutions like I described in this comment.

          Sam Gleske added a comment -

          My recommendation would be to abstract your folder creation to a function and detect if the folder exists. If it does then update the values. If it doesn't then create the folder.

          I also think the Job DSL library would be more friendly if it did this for you as it is not obvious nor clear. Perhaps something that should be implemented upstream? Your thoughts daspilker?

          Sam Gleske added a comment - My recommendation would be to abstract your folder creation to a function and detect if the folder exists. If it does then update the values. If it doesn't then create the folder. I also think the Job DSL library would be more friendly if it did this for you as it is not obvious nor clear. Perhaps something that should be implemented upstream? Your thoughts daspilker ?

          When I built the folder support I did not take into account that in contrast to jobs, the views are saved in the folder config XML. So there are two top-level DSL items which contribute to the folder config. The DSL is designed to generate the config XML on the whole and views within folders violate this rule.

          To fix this, views within folders should only be defined within folders:

          folder('example') {
            listView('my-view') {
              ...
            }
          }
          

          But unfortunately that would not fix Russell's problem.

          Daniel Spilker added a comment - When I built the folder support I did not take into account that in contrast to jobs, the views are saved in the folder config XML. So there are two top-level DSL items which contribute to the folder config. The DSL is designed to generate the config XML on the whole and views within folders violate this rule. To fix this, views within folders should only be defined within folders: folder( 'example' ) { listView( 'my-view' ) { ... } } But unfortunately that would not fix Russell's problem.

          Dirk Kuypers added a comment -

          Maybe it would be a good idea to have a tick like

          Action for existing folders: () ignore changes

          like we have for jobs and views.

          Dirk Kuypers added a comment - Maybe it would be a good idea to have a tick like Action for existing folders: () ignore changes like we have for jobs and views.

          But that would only be a workaround that can only be used when no folder options must be changed. If you want to change folder option you would have to disable the tick and then loose all views.

          Another workaround would be to run all pre-folder seed jobs after running the top-level seed job. Then all views will be regenerated. But there will be a (hopefully short) time where the folders will have no views. There is a DSL method to queue builds: https://jenkinsci.github.io/job-dsl-plugin/#path/queue

          Daniel Spilker added a comment - But that would only be a workaround that can only be used when no folder options must be changed. If you want to change folder option you would have to disable the tick and then loose all views. Another workaround would be to run all pre-folder seed jobs after running the top-level seed job. Then all views will be regenerated. But there will be a (hopefully short) time where the folders will have no views. There is a DSL method to queue builds: https://jenkinsci.github.io/job-dsl-plugin/#path/queue

          Dirk Kuypers added a comment -

          My motivation for something like this is the missing dashboard view in job-dsl which I configure by hand (sometimes). :-D

          Once I really messed up our server at restart with monkey patching the config.xml so I am reluctant since then to use that. And unfortunately I am in a .NET/Windows environment here (with no Java/Groovy at all) which sets the bar quite high for me to start contributing the dashboard view stuff.

          Dirk Kuypers added a comment - My motivation for something like this is the missing dashboard view in job-dsl which I configure by hand (sometimes). :-D Once I really messed up our server at restart with monkey patching the config.xml so I am reluctant since then to use that. And unfortunately I am in a .NET/Windows environment here (with no Java/Groovy at all) which sets the bar quite high for me to start contributing the dashboard view stuff.

          bruce: There is a ticket for adding the dashboard view: JENKINS-29146. Please add a comment over there which mentions the options that you need for a start. The dashboard view has a ton of options and it would be easier to start with a very limited subset.

          Daniel Spilker added a comment - bruce : There is a ticket for adding the dashboard view: JENKINS-29146 . Please add a comment over there which mentions the options that you need for a start. The dashboard view has a ton of options and it would be easier to start with a very limited subset.

          Dirk Kuypers added a comment - - edited

          Thanks for the hint, Daniel!

          Dirk Kuypers added a comment - - edited Thanks for the hint, Daniel!

          Thanks for all of the suggestions.

          My idea is that the top level seed shouldn't have to know about all of the jobs and the per-folder seeds create the jobs and add them to views explicitly. So one option would be to have all of the seeds at the top level (seed, seed_folder1, seed_folder2). seed would create seed_folder1/2 etc. then they would be responsible for creating the folders, jobs and views. The downside of this is that you can't take advantage of per folder lookup strategy, all jobs would have to have full names.

          Russell Gallop added a comment - Thanks for all of the suggestions. My idea is that the top level seed shouldn't have to know about all of the jobs and the per-folder seeds create the jobs and add them to views explicitly. So one option would be to have all of the seeds at the top level (seed, seed_folder1, seed_folder2). seed would create seed_folder1/2 etc. then they would be responsible for creating the folders, jobs and views. The downside of this is that you can't take advantage of per folder lookup strategy, all jobs would have to have full names.

            jamietanna Jamie Tanna
            rg Russell Gallop
            Votes:
            5 Vote for this issue
            Watchers:
            12 Start watching this issue

              Created:
              Updated: