Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-38111

github organization should allow 'clean before checkout' behavior

      we are using the Github Organization plugin, and using a Jenkinsfile checked into each git repo in the organization.

      i want to be able to use the 'clean before checkout' option which is available when we configure pipeline builds outside of the github organization. it seems like this should be an option exposed in the organization configuration.

          [JENKINS-38111] github organization should allow 'clean before checkout' behavior

          Alex Dean created issue -

          In addition, I would prefer if those Additional Behaviors (scm.extensions) were available to the Jenkinsfile since some are not useful for every repository in a GitHub Org.

          Christian Höltje added a comment - In addition, I would prefer if those Additional Behaviors ( scm.extensions ) were available to the Jenkinsfile since some are not useful for every repository in a GitHub Org.
          Christian Höltje made changes -
          Link New: This issue is related to JENKINS-37817 [ JENKINS-37817 ]

          Alex Dean added a comment - - edited

          re: making behaviors available via Jenkinsfile... it's unclear to me if projects which are being build via the GH Organization plugin need a 'checkout scm' step or not.

          I thought I would need this step to update to the latest code before building, but my build logs now contain redundant-seeming git operations. (Operations like "Fetching upstream changes from ..." are logged twice.) I suspect this means I might not need (or want) a 'checkout scm' step in my Jenkinsfile at all, but I can't find documentation one way or the other on this point.

          And tbh it makes sense that the plugin would handle the git operations for me. Otherwise there's a chicken/egg problem - the organization plugin has to check out the code in order to get the Jenkinsfile to start the build in the first place, at which point there doesn't seem to be much point to a second 'checkout scm' step.

          Alex Dean added a comment - - edited re: making behaviors available via Jenkinsfile... it's unclear to me if projects which are being build via the GH Organization plugin need a 'checkout scm' step or not. I thought I would need this step to update to the latest code before building, but my build logs now contain redundant-seeming git operations. (Operations like "Fetching upstream changes from ..." are logged twice.) I suspect this means I might not need (or want) a 'checkout scm' step in my Jenkinsfile at all, but I can't find documentation one way or the other on this point. And tbh it makes sense that the plugin would handle the git operations for me. Otherwise there's a chicken/egg problem - the organization plugin has to check out the code in order to get the Jenkinsfile to start the build in the first place, at which point there doesn't seem to be much point to a second 'checkout scm' step.

          The way GitHub Org Folder works is the first git checkout you see is the Jenkins Server (in a "sub job" or "mini job" or whatever it is called) checking out the source on the Jenkins Server (In a directory like JENKINS_HOME/workspace/<github-org>/<github-repo>/<branch>@script}}) to get the Jenkinsfile.

          It then uses that Jenkinsfile to actually do the build. The Jenkinsfile then allocates Workspaces via node() {} commands which may-or-may-not be empty.

          Christian Höltje added a comment - The way GitHub Org Folder works is the first git checkout you see is the Jenkins Server (in a "sub job" or "mini job" or whatever it is called) checking out the source on the Jenkins Server (In a directory like JENKINS_HOME /workspace/<github-org>/<github-repo>/<branch>@script}}) to get the Jenkinsfile . It then uses that Jenkinsfile to actually do the build. The Jenkinsfile then allocates Workspaces via node() { } commands which may-or-may-not be empty.

          Mark Waite added a comment -

          As far as I know, those behaviors are available from jobs generated by the GitHub Organizations Folder plugin, so long as the behavior is added in the Jenkinsfile. Refer to [the JENKINS-22547 branch}https://github.com/MarkEWaite/jenkins-bugs/blob/JENKINS-22547/Jenkinsfile] of my jenkins-bugs repository for an example of a Jenkinsfile which includes several additional behaviors added to the checkout() step, including CleanBeforeCheckout.

          Could you try that technique and see if it works as expected in your GitHub Organization Folders?

          Mark Waite added a comment - As far as I know, those behaviors are available from jobs generated by the GitHub Organizations Folder plugin, so long as the behavior is added in the Jenkinsfile. Refer to [the JENKINS-22547 branch}https://github.com/MarkEWaite/jenkins-bugs/blob/JENKINS-22547/Jenkinsfile] of my jenkins-bugs repository for an example of a Jenkinsfile which includes several additional behaviors added to the checkout() step, including CleanBeforeCheckout. Could you try that technique and see if it works as expected in your GitHub Organization Folders?

          Alex Dean added a comment -

          Mark: Thank you for the idea. While I think that would work for now, I'm concerned about needing to statically specify so much of what is already provided by the built-in `scm` global variable. (As Christian also raises in JENKINS-37817.)

          Christian: Thank you for that explanation. Makes sense. Based on your explanation, it sounds like being able to specify 'clean before checkout' in the org plugin wouldn't accomplish what I want anyway. (Since I think i'd be cleaning the `<branch>@script` working copy rather than the actual workspace where the build it going to happen? If I've got that right, then I think this issue is probably invalid.

          Alex Dean added a comment - Mark: Thank you for the idea. While I think that would work for now, I'm concerned about needing to statically specify so much of what is already provided by the built-in `scm` global variable. (As Christian also raises in JENKINS-37817 .) Christian: Thank you for that explanation. Makes sense. Based on your explanation, it sounds like being able to specify 'clean before checkout' in the org plugin wouldn't accomplish what I want anyway. (Since I think i'd be cleaning the `<branch>@script` working copy rather than the actual workspace where the build it going to happen? If I've got that right, then I think this issue is probably invalid.

          Right. There are two issues, really...

          1. The <branch>@script checkout on the master should probably always use shallow to prevent huge checkout times. If there was another way to just get the correct Jenkinsfile (e.g. https://raw.<github-url>/<org>/<repo>/<branch-or-ref>/Jenkinsfile?token=<token>) that'd be even better.
          2. The checkout scm needs to be configurable so changes specific to a repository can be setup, such as setting "clean before checkout" or "shallow clone". Otherwise this cannot be setup due to the way the GitHub Org Folder works.

          The second item is what I'm concerned about here. I should file another issue on the first item.

          Christian Höltje added a comment - Right. There are two issues, really... The <branch>@script checkout on the master should probably always use shallow to prevent huge checkout times. If there was another way to just get the correct Jenkinsfile (e.g. https://raw .<github-url>/<org>/<repo>/<branch-or-ref>/Jenkinsfile?token=<token> ) that'd be even better. The checkout scm needs to be configurable so changes specific to a repository can be setup, such as setting "clean before checkout" or "shallow clone". Otherwise this cannot be setup due to the way the GitHub Org Folder works. The second item is what I'm concerned about here. I should file another issue on the first item.

          I created JENKINS-38212 to address the "getting the Jenkinsfile" story.

          Christian Höltje added a comment - I created JENKINS-38212 to address the "getting the Jenkinsfile" story.
          Christian Höltje made changes -
          Link New: This issue is related to JENKINS-38212 [ JENKINS-38212 ]

            kohsuke Kohsuke Kawaguchi
            alexdean Alex Dean
            Votes:
            2 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated:
              Resolved: