Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-22510

Clean After Checkout Results in Failed to Checkout Revision

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Major Major
    • git-plugin
    • None

      We're executing a job, and trying to perform a git clean. One of our submodules is dirty, and unfortunately since the Jenkins GIT plugin cleans after checkout, we get the following error:

      FATAL: Command "git submodule update --init --recursive" returned status code 1:
      stdout:
      stderr: error: Your local changes to the following files would be overwritten by checkout:
      minified/sidecar.js
      minified/sidecar.lite.js
      minified/sidecar.lite.min.js
      minified/sidecar.min.js
      Please, commit your changes or stash them before you can switch branches.
      Aborting
      Unable to checkout 'd58c3304230e4bf26ffb3fa6986d4b6ba90d4c66' in submodule path 'sugarcrm/sidecar'

      I tested using the Pre-SCM Buildstep plugin and performing a git clean before the checkout fixes the problem. Specifically having it perform:
      git submodule foreach --recursive git clean -fdx
      git submodule foreach --recursive git reset --hard

      before we do the checkout fixes it.

          [JENKINS-22510] Clean After Checkout Results in Failed to Checkout Revision

          Ray Sennewald created issue -

          Ray Sennewald added a comment -

          David Wang has implemented functionality to Clean Before Checkout, which actually helps fix this scenario. His pull request is here: https://github.com/jenkinsci/git-plugin/pull/220.

          Ray Sennewald added a comment - David Wang has implemented functionality to Clean Before Checkout, which actually helps fix this scenario. His pull request is here: https://github.com/jenkinsci/git-plugin/pull/220 .
          Ray Sennewald made changes -
          Description Original: We're executing a job, and trying to perform a git clean. One of our submodules is dirty, and unfortunately since the Jenkins GIT plugin cleans after checkout, we get the following error:

          FATAL: Command "git submodule update --init --recursive" returned status code 1:
          stdout:
          stderr: error: Your local changes to the following files would be overwritten by checkout:
          minified/sidecar.js
          minified/sidecar.lite.js
          minified/sidecar.lite.min.js
          minified/sidecar.min.js
          Please, commit your changes or stash them before you can switch branches.
          Aborting
          Unable to checkout 'd58c3304230e4bf26ffb3fa6986d4b6ba90d4c66' in submodule path 'sugarcrm/sidecar'

          I tested using the Pre-SCM Buildstep plugin and performing a git clean before the checkout fixes the problem.
          New: We're executing a job, and trying to perform a git clean. One of our submodules is dirty, and unfortunately since the Jenkins GIT plugin cleans after checkout, we get the following error:

          FATAL: Command "git submodule update --init --recursive" returned status code 1:
          stdout:
          stderr: error: Your local changes to the following files would be overwritten by checkout:
          minified/sidecar.js
          minified/sidecar.lite.js
          minified/sidecar.lite.min.js
          minified/sidecar.min.js
          Please, commit your changes or stash them before you can switch branches.
          Aborting
          Unable to checkout 'd58c3304230e4bf26ffb3fa6986d4b6ba90d4c66' in submodule path 'sugarcrm/sidecar'

          I tested using the Pre-SCM Buildstep plugin and performing a git clean before the checkout fixes the problem. Specifically having it perform:
          git submodule foreach --recursive git clean -fdx
          git submodule foreach --recursive git reset --hard

          before we do the checkout fixes it.
          Mark Waite made changes -
          Link New: This issue is related to JENKINS-7376 [ JENKINS-7376 ]
          Mark Waite made changes -
          Link New: This issue is related to JENKINS-13910 [ JENKINS-13910 ]
          Mark Waite made changes -
          Link New: This issue is related to JENKINS-8315 [ JENKINS-8315 ]
          Mark Waite made changes -
          Link New: This issue is related to JENKINS-7445 [ JENKINS-7445 ]
          Mark Waite made changes -
          Link New: This issue is related to JENKINS-8503 [ JENKINS-8503 ]

          Mark Waite added a comment - - edited

          Considering the number of cases related to this topic over the course of the evolution of the git plugin, I'd very much prefer that we have a very deep set of tests created which will assure that submodules behave as desired. Some of the related bugs (JENKINS-8315, JENKINS-7445) are listed as fixed, yet those fixes either did not survive, or do not meet your need. I'd really like more tests that express the failure cases, since the git-client-plugin has very few submodule tests.

          Ultimately, we may need a "clean before checkout" operation, but it seems like we should first invest effort to create unit tests which will assure that submodules are behaving as expected in the git client plugin (and possibly in the git plugin).

          Relying on the user to know when they should select "clean before checkout" and when they should "clean after checkout" seems like we're expecting too much from the users. I would very much prefer to find a sequence which avoids requiring the addition of yet another form of clean. We have "wipe workspace" which removes everything, we have "clean after checkout", and this will add "clean before checkout".

          Since you and David are active users of submodules, would you be willing to provide one or more test cases which illustrate this failure mode, then use that test case to evaluate alternatives to fix the test?

          As an example of an alternative, would the "git clean -ffxd" command (double f arguments clean untracked directories managed by another repository) resolve the issue without requiring the addition of a new command?

          Mark Waite added a comment - - edited Considering the number of cases related to this topic over the course of the evolution of the git plugin, I'd very much prefer that we have a very deep set of tests created which will assure that submodules behave as desired. Some of the related bugs ( JENKINS-8315 , JENKINS-7445 ) are listed as fixed, yet those fixes either did not survive, or do not meet your need. I'd really like more tests that express the failure cases, since the git-client-plugin has very few submodule tests. Ultimately, we may need a "clean before checkout" operation, but it seems like we should first invest effort to create unit tests which will assure that submodules are behaving as expected in the git client plugin (and possibly in the git plugin). Relying on the user to know when they should select "clean before checkout" and when they should "clean after checkout" seems like we're expecting too much from the users. I would very much prefer to find a sequence which avoids requiring the addition of yet another form of clean. We have "wipe workspace" which removes everything, we have "clean after checkout", and this will add "clean before checkout". Since you and David are active users of submodules, would you be willing to provide one or more test cases which illustrate this failure mode, then use that test case to evaluate alternatives to fix the test? As an example of an alternative, would the " git clean -ffxd " command (double f arguments clean untracked directories managed by another repository) resolve the issue without requiring the addition of a new command?

          Mark Waite added a comment - - edited

          I've been trying to understand the sequence of git operations which arrive at that state described in the bug report, and haven't yet understood how to do that. My attempts have persuaded me that I don't understand submodules. I used my fork of the git-client-plugin for my experiments.

          The failing sequence I see currently is:

          • checkout master - ok
          • checkout tests/notSubmodules - ok
          • checkout tests/getSubmodules - ok
          • checkout tests/notSubmodules - fails

          I defined a bash shell function "checkout" for my testing convenience. It is

          checkout ()
          {
              branch=$1;
              git checkout -f $branch;
              git clean -xffd;
              git submodule update --init --recursive;
              git status
          }
          

          Clone the repository - works

          git clone git://github.com/MarkEWaite/git-client-plugin
          cd git-client-plugin
          ls modules/ntp # reports an error, no content in modules/ntp
          

          Checkout tests/notSubmodules - works

          checkout tests/notSubmodules
          

          Checkout tests/getSubmodules - works

          checkout tests/getSubmodules
          

          Checkout tests/notSubmodules - works

          checkout tests/notSubmodules
          

          Checkout tests/getSubmodules - fails

          checkout tests/getSubmodules
          

          The "mostly" working sequence in my case seems to be "force checkout, git clean -xffd, submodule update". That seems to work for all the cases I've tested except the master to notSubmodules to getSubmodules to notSubmodules to getSubmodules case. However, since my case does not duplicate your message, I'm not sure if you'll see the same behavior as I see.

          In that repository, the branch to branch transitions seem to behave like this:

          Current Branch Branch to Checkout Result
          master tests/getSubmodules ok
          master tests/notSubmodules ok
          tests/getSubmodules master ok
          tests/notSubmodules master ok
          tests/getSubmodules tests/notSubmodules fails in some cases
          tests/notSubmodules tests/getSubmodules fails in some cases

          However, the failure message I'm seeing is not the same as the failure message you're seeing. I'm confident that means I don't understand enough about submodules. Can you provide a similar series of steps (preferably using a fork of the git-client-plugin repository) to show your case?

          If you're interested in my first attempt to express that failure mode as a unit test, refer to the proto-submodule-tests branch in my fork of the git-client-plugin source. The test is "not yet ready for prime time".

          Mark Waite added a comment - - edited I've been trying to understand the sequence of git operations which arrive at that state described in the bug report, and haven't yet understood how to do that. My attempts have persuaded me that I don't understand submodules. I used my fork of the git-client-plugin for my experiments. The failing sequence I see currently is: checkout master - ok checkout tests/notSubmodules - ok checkout tests/getSubmodules - ok checkout tests/notSubmodules - fails I defined a bash shell function "checkout" for my testing convenience. It is checkout () { branch=$1; git checkout -f $branch; git clean -xffd; git submodule update --init --recursive; git status } Clone the repository - works git clone git://github.com/MarkEWaite/git-client-plugin cd git-client-plugin ls modules/ntp # reports an error, no content in modules/ntp Checkout tests/notSubmodules - works checkout tests/notSubmodules Checkout tests/getSubmodules - works checkout tests/getSubmodules Checkout tests/notSubmodules - works checkout tests/notSubmodules Checkout tests/getSubmodules - fails checkout tests/getSubmodules The "mostly" working sequence in my case seems to be "force checkout, git clean -xffd, submodule update". That seems to work for all the cases I've tested except the master to notSubmodules to getSubmodules to notSubmodules to getSubmodules case. However, since my case does not duplicate your message, I'm not sure if you'll see the same behavior as I see. In that repository, the branch to branch transitions seem to behave like this: Current Branch Branch to Checkout Result master tests/getSubmodules ok master tests/notSubmodules ok tests/getSubmodules master ok tests/notSubmodules master ok tests/getSubmodules tests/notSubmodules fails in some cases tests/notSubmodules tests/getSubmodules fails in some cases However, the failure message I'm seeing is not the same as the failure message you're seeing. I'm confident that means I don't understand enough about submodules. Can you provide a similar series of steps (preferably using a fork of the git-client-plugin repository) to show your case? If you're interested in my first attempt to express that failure mode as a unit test, refer to the proto-submodule-tests branch in my fork of the git-client-plugin source. The test is "not yet ready for prime time".

            ndeloof Nicolas De Loof
            rsennewald Ray Sennewald
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: