Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-44598

git workspace should be reset before build PR merged with base branch

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • None
    • OS: CentOS 7.3.1611
      Jenkins: 2.46.2
      GitHub Branch Source Plugin: 2.0.6
      git version: 1.8.3.1

      The plugin attempts to merge the PR and the base branch before build. However, if git-merge introduces a conflict, the unmerged files will be left in the PR's git workspace, which causes git-merge in the following builds always fail, because the conflict is not solved.

      Here's the log snippets:

      For build #3, which introduces the merge conflict

      > git fetch --tags --progress git@github.mycompany.com:SERVER_REPO.git +refs/pull//head:refs/remotes/origin/pr/
       Merging dev_6_2 commit fc999a1dae8861c1e5d3c21c8deb81f1e7bdf3a8 into PR head commit 5042e295186f8fe4bbb61ac97d97d69d1932912c
       > git config core.sparsecheckout # timeout=10
       > git checkout -f 5042e295186f8fe4bbb61ac97d97d69d1932912c
       > git merge fc999a1dae8861c1e5d3c21c8deb81f1e7bdf3a8 # timeout=10
       > git config core.sparsecheckout # timeout=10
       > git checkout -f 5042e295186f8fe4bbb61ac97d97d69d1932912c
      GitHub has been notified of this commit’s build result
      hudson.plugins.git.GitException: Command "git merge fc999a1dae8861c1e5d3c21c8deb81f1e7bdf3a8" returned status code 1:
       stdout: Auto-merging docker-environment/ansible/group_vars/all.yml
       CONFLICT (add/add): Merge conflict in docker-environment/ansible/group_vars/all.yml
       Automatic merge failed; fix conflicts and then commit the result.
      stderr: 
       at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1877)
       at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1845)
       at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1841)
       at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommand(CliGitAPIImpl.java:1486)
       at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$3.execute(CliGitAPIImpl.java:621)
       at org.jenkinsci.plugins.github_branch_source.GitHubSCMSource$MergeWith.decorateRevisionToBuild(GitHubSCMSource.java:1003)
       at hudson.plugins.git.GitSCM.determineRevisionToBuild(GitSCM.java:1015)
       at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1108)
       at org.jenkinsci.plugins.workflow.steps.scm.SCMStep.checkout(SCMStep.java:109)
       at org.jenkinsci.plugins.workflow.cps.CpsScmFlowDefinition.create(CpsScmFlowDefinition.java:130)
       at org.jenkinsci.plugins.workflow.multibranch.SCMBinder.create(SCMBinder.java:107)
       at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:234)
       at hudson.model.ResourceController.execute(ResourceController.java:98)
       at hudson.model.Executor.run(Executor.java:405)
       Finished: FAILURE
      

      For the following build #4, #5 and so on, the merge always fails

      > git fetch --tags --progress git@github.mycompany.com:SERVER_REPO.git +refs/pull//head:refs/remotes/origin/pr/
       Merging dev_6_2 commit fc999a1dae8861c1e5d3c21c8deb81f1e7bdf3a8 into PR head commit 32fe21329c3f0349347d029917ba29a39fc0364a
       > git config core.sparsecheckout # timeout=10
       > git checkout -f 32fe21329c3f0349347d029917ba29a39fc0364a
       > git merge fc999a1dae8861c1e5d3c21c8deb81f1e7bdf3a8 # timeout=10
       > git config core.sparsecheckout # timeout=10
       > git checkout -f 32fe21329c3f0349347d029917ba29a39fc0364a
      GitHub has been notified of this commit’s build result
      hudson.plugins.git.GitException: Command "git merge fc999a1dae8861c1e5d3c21c8deb81f1e7bdf3a8" returned status code 128:
       stdout: 
       stderr: error: 'merge' is not possible because you have unmerged files.
       hint: Fix them up in the work tree,
       hint: and then use 'git add/rm <file>' as
       hint: appropriate to mark resolution and make a commit,
       hint: or use 'git commit -a'.
       fatal: Exiting because of an unresolved conflict.
      at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1877)
       at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1845)
       at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1841)
       at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommand(CliGitAPIImpl.java:1486)
       at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$3.execute(CliGitAPIImpl.java:621)
       at org.jenkinsci.plugins.github_branch_source.GitHubSCMSource$MergeWith.decorateRevisionToBuild(GitHubSCMSource.java:1003)
       at hudson.plugins.git.GitSCM.determineRevisionToBuild(GitSCM.java:1015)
       at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1108)
       at org.jenkinsci.plugins.workflow.steps.scm.SCMStep.checkout(SCMStep.java:109)
       at org.jenkinsci.plugins.workflow.cps.CpsScmFlowDefinition.create(CpsScmFlowDefinition.java:130)
       at org.jenkinsci.plugins.workflow.multibranch.SCMBinder.create(SCMBinder.java:107)
       at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:234)
       at hudson.model.ResourceController.execute(ResourceController.java:98)
       at hudson.model.Executor.run(Executor.java:405)
       Finished: FAILURE
      

       

      Then this build will be stuck in this situation unless we manually issue 'git reset HEAD --hard' in that git workspace.

          [JENKINS-44598] git workspace should be reset before build PR merged with base branch

          Nick Jones added a comment -

          I just observed the same thing today, running GitHub Branch Source 2.0.7, git 2.13.2, and Jenkins 2.60.1 (LTS). The only solution was to track down the appropriate multibranch pipeline directory on the master node (which doesn't otherwise do builds, but where the merge conflict and the unmerged changes occurred), and run 'git reset HEAD --hard' there. 

          Even if it didn't make sense to do resets automatically in all cases prior to attempting to merge, it would be nice to have the ability to add behaviors to the GitHub SCM for Multibranch Pipeline jobs as is possible with Freestyle jobs.

          Nick Jones added a comment - I just observed the same thing today, running GitHub Branch Source 2.0.7, git 2.13.2, and Jenkins 2.60.1 (LTS). The only solution was to track down the appropriate multibranch pipeline directory on the master node (which doesn't otherwise do builds, but where the merge conflict and the unmerged changes occurred), and run 'git reset HEAD --hard' there.  Even if it didn't make sense to do resets automatically in all cases prior to attempting to merge, it would be nice to have the ability to add behaviors to the GitHub SCM for Multibranch Pipeline jobs as is possible with Freestyle jobs.

          medianick with JENKINS-43507 you can now add the Clean Before Checkout behaviour, so that should at least provide a workaround.

          More generally, we probably should apply this behaviour fix after JENKINS-45771 has been merged in order to ensure the fix is picked up by all git-based scm source implementations

          Stephen Connolly added a comment - medianick with JENKINS-43507 you can now add the Clean Before Checkout behaviour, so that should at least provide a workaround. More generally, we probably should apply this behaviour fix after JENKINS-45771 has been merged in order to ensure the fix is picked up by all git-based scm source implementations

          Joshua Noble added a comment -

          I'm also now seeing this exact issue too. Any ideas around a fix? If someone can point me in the right direction I can attempt one.

          Joshua Noble added a comment - I'm also now seeing this exact issue too. Any ideas around a fix? If someone can point me in the right direction I can attempt one.

          add the Clean Before Checkout behaviour

          Stephen Connolly added a comment - add the Clean Before Checkout behaviour

          trejkaz added a comment -

          Clean Before Checkout didn't help for us... maybe cleaning isn't cleaning it enough?

          Cleaning workspace
           > git rev-parse --verify HEAD # timeout=10
          Resetting working tree
           > git reset --hard # timeout=10
           > git clean -fdx # timeout=10
          Pruning obsolete local branches
          Fetching upstream changes from git@github.eng.acme.com:acme/core.git
           > git --version # timeout=10
          using GIT_SSH to set credentials GitHub BuildSlaves Account (Pipeline-Github)
           > git fetch --tags --progress git@github.eng.acme.com:acme/core.git +refs/pull/145/head:refs/remotes/origin/PR-145 +refs/heads/master:refs/remotes/origin/master --prune # timeout=40
          Merging remotes/origin/master commit fb359ab126eef7794d4118f45c9057695d422162 into PR head commit dc24fa110e813e0c43a2230db6d319f5ea7c11b9
           > git config core.sparsecheckout # timeout=10
           > git checkout -f dc24fa110e813e0c43a2230db6d319f5ea7c11b9 # timeout=40
           > git merge fb359ab126eef7794d4118f45c9057695d422162 # timeout=10
           > git config core.sparsecheckout # timeout=10
           > git checkout -f dc24fa110e813e0c43a2230db6d319f5ea7c11b9 # timeout=40
          Command "git merge fb359ab126eef7794d4118f45c9057695d422162" returned status code 1:
          stdout: Auto-merging processing/src/test/java/com/acme/export/print/TestAsposeVisioPdfExporter.java
          Auto-merging investigator/src/resources/com/acme/investigator/charts/acme_metadatarequest.js
          CONFLICT (add/add): Merge conflict in investigator/src/resources/com/acme/investigator/charts/acme_metadatarequest.js
          Auto-merging investigator/src/resources/com/acme/investigator/charts/main.html
          Auto-merging investigator/src/resources/com/acme/investigator/charts/datamanager.js
          Auto-merging investigator/src/java/com/acme/investigator/charts/ChartsController.java
          CONFLICT (content): Merge conflict in investigator/src/java/com/acme/investigator/charts/ChartsController.java
          Removing gui-util/src/resources/com/acme/browser/resources/test.html
          Auto-merging buildSrc/hooks/pre-commit
          CONFLICT (add/add): Merge conflict in buildSrc/hooks/pre-commit
          Auto-merging build.gradle
          CONFLICT (content): Merge conflict in build.gradle
          Auto-merging Jenkinsfile
          CONFLICT (content): Merge conflict in Jenkinsfile
          Automatic merge failed; fix conflicts and then commit the result.
          

          At this rate it feels like the only workaround is to blow away the checkout directory before starting the build, like what we had to do with svn due to similar sorts of issues...

          trejkaz added a comment - Clean Before Checkout didn't help for us... maybe cleaning isn't cleaning it enough? Cleaning workspace > git rev-parse --verify HEAD # timeout=10 Resetting working tree > git reset --hard # timeout=10 > git clean -fdx # timeout=10 Pruning obsolete local branches Fetching upstream changes from git@github.eng.acme.com:acme/core.git > git --version # timeout=10 using GIT_SSH to set credentials GitHub BuildSlaves Account (Pipeline-Github) > git fetch --tags --progress git@github.eng.acme.com:acme/core.git +refs/pull/145/head:refs/remotes/origin/PR-145 +refs/heads/master:refs/remotes/origin/master --prune # timeout=40 Merging remotes/origin/master commit fb359ab126eef7794d4118f45c9057695d422162 into PR head commit dc24fa110e813e0c43a2230db6d319f5ea7c11b9 > git config core.sparsecheckout # timeout=10 > git checkout -f dc24fa110e813e0c43a2230db6d319f5ea7c11b9 # timeout=40 > git merge fb359ab126eef7794d4118f45c9057695d422162 # timeout=10 > git config core.sparsecheckout # timeout=10 > git checkout -f dc24fa110e813e0c43a2230db6d319f5ea7c11b9 # timeout=40 Command "git merge fb359ab126eef7794d4118f45c9057695d422162" returned status code 1: stdout: Auto-merging processing/src/test/java/com/acme/export/print/TestAsposeVisioPdfExporter.java Auto-merging investigator/src/resources/com/acme/investigator/charts/acme_metadatarequest.js CONFLICT (add/add): Merge conflict in investigator/src/resources/com/acme/investigator/charts/acme_metadatarequest.js Auto-merging investigator/src/resources/com/acme/investigator/charts/main.html Auto-merging investigator/src/resources/com/acme/investigator/charts/datamanager.js Auto-merging investigator/src/java/com/acme/investigator/charts/ChartsController.java CONFLICT (content): Merge conflict in investigator/src/java/com/acme/investigator/charts/ChartsController.java Removing gui-util/src/resources/com/acme/browser/resources/test.html Auto-merging buildSrc/hooks/pre-commit CONFLICT (add/add): Merge conflict in buildSrc/hooks/pre-commit Auto-merging build.gradle CONFLICT (content): Merge conflict in build.gradle Auto-merging Jenkinsfile CONFLICT (content): Merge conflict in Jenkinsfile Automatic merge failed; fix conflicts and then commit the result. At this rate it feels like the only workaround is to blow away the checkout directory before starting the build, like what we had to do with svn due to similar sorts of issues...

          trejkaz is the PR mergable without conflicts? If it is not mergable then the merge should fail. What does GitHub say about the merge status?

          Stephen Connolly added a comment - trejkaz is the PR mergable without conflicts? If it is not mergable then the merge should fail. What does GitHub say about the merge status?

          trejkaz added a comment - - edited

          GitHub's merge status isn't really useful - it just says "Required statuses must pass before merging" because Jenkins failed the build.

          I imagine that if Jenkins passed, it would just let me merge, because the PR only changes two files, neither of which are the ones the command is complaining about.

          Additionally, the output seems a bit suspicious:

           > git checkout -f dc24fa110e813e0c43a2230db6d319f5ea7c11b9 # timeout=40
          Command "git merge fb359ab126eef7794d4118f45c9057695d422162" returned status code 1:
          

          Shouldn't git merge be the last command output before that failure?

          (Also, why is it running git checkout twice?)

          In an attempt to reproduce it locally, I checked the same sequence of commands locally, and they worked fine on my box, but I'm running macOS and the build machine is running Debian, so it isn't a realistic test.

          trejkaz added a comment - - edited GitHub's merge status isn't really useful - it just says "Required statuses must pass before merging" because Jenkins failed the build. I imagine that if Jenkins passed, it would just let me merge, because the PR only changes two files, neither of which are the ones the command is complaining about. Additionally, the output seems a bit suspicious: > git checkout -f dc24fa110e813e0c43a2230db6d319f5ea7c11b9 # timeout=40 Command "git merge fb359ab126eef7794d4118f45c9057695d422162" returned status code 1: Shouldn't git merge be the last command output before that failure? (Also, why is it running git checkout twice?) In an attempt to reproduce it locally, I checked the same sequence of commands locally, and they worked fine on my box, but I'm running macOS and the build machine is running Debian, so it isn't a realistic test.

          trejkaz Aha!

          git config core.sparsecheckout # timeout=10

          You cannot merge if you are doing a sparse checkout, because git doesn't have the history with which to do a real merge. 

          Can you try turning off "Shallow clone" in "Advanced clone behaviours". If that fixes the issue then likely we need to configure the GitHub Branch Source to override shallow clone configuration for PR-merge SCMHeads

          Stephen Connolly added a comment - trejkaz Aha! git config core.sparsecheckout # timeout=10 You cannot merge if you are doing a sparse checkout, because git doesn't have the history with which to do a real merge.  Can you try turning off "Shallow clone" in "Advanced clone behaviours". If that fixes the issue then likely we need to configure the GitHub Branch Source to override shallow clone configuration for PR-merge SCMHeads

          trejkaz added a comment - - edited

          Looks like "Shallow clone" is already off. "Shallow clone depth" is 10... but I'm hoping it's ignored if the setting isn't even turned on. (Maybe that is a false hope, though?)

          I tried setting it to blank to see if it changes anything. It ends up being set to 0... I can't get it back to an empty box like shown in your screenshot. :/

          trejkaz added a comment - - edited Looks like "Shallow clone" is already off. "Shallow clone depth" is 10... but I'm hoping it's ignored if the setting isn't even turned on. (Maybe that is a false hope, though?) I tried setting it to blank to see if it changes anything. It ends up being set to 0... I can't get it back to an empty box like shown in your screenshot. :/

          trejkaz added a comment -

          OK, it's still running `git config core.sparsecheckout` even with both fields reset, so maybe `git config core.sparsecheckout` just means to turn the setting off? :/

          trejkaz added a comment - OK, it's still running `git config core.sparsecheckout` even with both fields reset, so maybe `git config core.sparsecheckout` just means to turn the setting off? :/

          andrew morton added a comment -

          The clean before build worked for me. The sparse checkout option didn't seem to have any effect on this issue.

          andrew morton added a comment - The clean before build worked for me. The sparse checkout option didn't seem to have any effect on this issue.

          I'm still facing the problem. The Jira issue is in OPEN state. Can this be resolved somehow?

          Thirunavukkarasu Ravichandran added a comment - I'm still facing the problem. The Jira issue is in OPEN state. Can this be resolved somehow?

          Brian J Murrell added a comment - - edited

          I too am hitting this and would appreciate a status update on whether anything is being done to resolve it.

          Adding the Clean before checkout (from Jenkins Git plugin) isn't really an acceptable solution because I really only want the cleaning before the initial checkout on the master (just to resolve the initial checkout merge failure before the Pipeline is even run) and not before every checkout for every Pipeline build step.  This is because cleaning in some of the subsequent Pipeline steps leads to other errors such as:

          10:21:35  Fetching changes from the remote Git repository
          10:21:35  Cleaning workspace
          10:21:35  ERROR: Error fetching remote repo 'origin'
          10:21:35  hudson.plugins.git.GitException: Failed to fetch from https://github.com/daos-stack/daos.git
          10:21:35  	at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:904)
          10:21:35  	at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1144)
          10:21:35  	at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1175)
          10:21:35  	at org.jenkinsci.plugins.workflow.steps.scm.SCMStep.checkout(SCMStep.java:120)
          10:21:35  	at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:90)
          10:21:35  	at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:77)
          10:21:35  	at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)
          10:21:35  	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
          10:21:35  	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
          10:21:35  	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
          10:21:35  	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
          10:21:35  	at java.lang.Thread.run(Thread.java:748)
          10:21:35  Caused by: hudson.plugins.git.GitException: Command "git clean -fdx" returned status code 128:
          10:21:35  stdout: 
          10:21:35  stderr: fatal: Not a git repository: install/include/boost/../../.git/modules/src/boost
          10:21:35  
          10:21:35  	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2318)
          10:21:35  	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2248)
          10:21:35  	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2244)
          10:21:35  	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommand(CliGitAPIImpl.java:1777)
          10:21:35  	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommand(CliGitAPIImpl.java:1789)
          10:21:35  	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.clean(CliGitAPIImpl.java:901)
          10:21:35  	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.clean(CliGitAPIImpl.java:912)
          10:21:35  	at hudson.plugins.git.GitAPI.clean(GitAPI.java:311)
          10:21:35  	at sun.reflect.GeneratedMethodAccessor94.invoke(Unknown Source)
          10:21:35  	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
          10:21:35  	at java.lang.reflect.Method.invoke(Method.java:498)
          10:21:35  	at hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:929)
          10:21:35  	at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:903)
          10:21:35  	at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:855)
          10:21:35  	at hudson.remoting.UserRequest.perform(UserRequest.java:212)
          10:21:35  	at hudson.remoting.UserRequest.perform(UserRequest.java:54)
          10:21:35  	at hudson.remoting.Request$2.run(Request.java:369)
          10:21:35  	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
          10:21:35  	... 4 more
          10:21:35  	Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to wolf-33_docker_1
          10:21:35  		at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
          10:21:35  		at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
          10:21:35  		at hudson.remoting.Channel.call(Channel.java:955)
          10:21:35  		at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:283)
          10:21:35  		at com.sun.proxy.$Proxy102.clean(Unknown Source)
          10:21:35  		at org.jenkinsci.plugins.gitclient.RemoteGitImpl.clean(RemoteGitImpl.java:463)
          10:21:35  		at hudson.plugins.git.extensions.impl.CleanBeforeCheckout.decorateFetchCommand(CleanBeforeCheckout.java:30)
          10:21:35  		at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:900)
          10:21:35  		at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1144)
          10:21:35  		at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1175)
          10:21:35  		at org.jenkinsci.plugins.workflow.steps.scm.SCMStep.checkout(SCMStep.java:120)
          10:21:35  		at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:90)
          10:21:35  		at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:77)
          10:21:35  		at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)
          10:21:35  		at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
          10:21:35  		... 4 more
          

          Yes, to be sure, the above is only a problem because of the content of the tree in the workspace, but it's only a problem because of this needed explicit Clean before checkout option that really should be the implied-and-always-executed behaviour when doing the initial merge checkout.  IOW, adding the explicit behaviour should not be necessary.

          Brian J Murrell added a comment - - edited I too am hitting this and would appreciate a status update on whether anything is being done to resolve it. Adding the Clean before checkout  (from Jenkins Git plugin) isn't really an acceptable solution because I really only want the cleaning before the initial checkout on the master (just to resolve the initial checkout merge failure before the Pipeline is even run) and not before every checkout for every Pipeline build step.  This is because cleaning in some of the subsequent Pipeline steps leads to other errors such as: 10:21:35 Fetching changes from the remote Git repository 10:21:35 Cleaning workspace 10:21:35 ERROR: Error fetching remote repo 'origin' 10:21:35 hudson.plugins.git.GitException: Failed to fetch from https://github.com/daos-stack/daos.git 10:21:35 at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:904) 10:21:35 at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1144) 10:21:35 at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1175) 10:21:35 at org.jenkinsci.plugins.workflow.steps.scm.SCMStep.checkout(SCMStep.java:120) 10:21:35 at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:90) 10:21:35 at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:77) 10:21:35 at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47) 10:21:35 at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 10:21:35 at java.util.concurrent.FutureTask.run(FutureTask.java:266) 10:21:35 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) 10:21:35 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 10:21:35 at java.lang.Thread.run(Thread.java:748) 10:21:35 Caused by: hudson.plugins.git.GitException: Command "git clean -fdx" returned status code 128: 10:21:35 stdout: 10:21:35 stderr: fatal: Not a git repository: install/include/boost/../../.git/modules/src/boost 10:21:35 10:21:35 at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2318) 10:21:35 at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2248) 10:21:35 at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2244) 10:21:35 at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommand(CliGitAPIImpl.java:1777) 10:21:35 at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommand(CliGitAPIImpl.java:1789) 10:21:35 at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.clean(CliGitAPIImpl.java:901) 10:21:35 at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.clean(CliGitAPIImpl.java:912) 10:21:35 at hudson.plugins.git.GitAPI.clean(GitAPI.java:311) 10:21:35 at sun.reflect.GeneratedMethodAccessor94.invoke(Unknown Source) 10:21:35 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 10:21:35 at java.lang.reflect.Method.invoke(Method.java:498) 10:21:35 at hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:929) 10:21:35 at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:903) 10:21:35 at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:855) 10:21:35 at hudson.remoting.UserRequest.perform(UserRequest.java:212) 10:21:35 at hudson.remoting.UserRequest.perform(UserRequest.java:54) 10:21:35 at hudson.remoting.Request$2.run(Request.java:369) 10:21:35 at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72) 10:21:35 ... 4 more 10:21:35 Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to wolf-33_docker_1 10:21:35 at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741) 10:21:35 at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357) 10:21:35 at hudson.remoting.Channel.call(Channel.java:955) 10:21:35 at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:283) 10:21:35 at com.sun.proxy.$Proxy102.clean(Unknown Source) 10:21:35 at org.jenkinsci.plugins.gitclient.RemoteGitImpl.clean(RemoteGitImpl.java:463) 10:21:35 at hudson.plugins.git.extensions.impl.CleanBeforeCheckout.decorateFetchCommand(CleanBeforeCheckout.java:30) 10:21:35 at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:900) 10:21:35 at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1144) 10:21:35 at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1175) 10:21:35 at org.jenkinsci.plugins.workflow.steps.scm.SCMStep.checkout(SCMStep.java:120) 10:21:35 at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:90) 10:21:35 at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:77) 10:21:35 at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47) 10:21:35 at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 10:21:35 ... 4 more Yes, to be sure, the above is only a problem because of the content of the tree in the workspace, but it's only a problem because of this needed explicit Clean before checkout option that really should be the implied-and-always-executed behaviour when doing the initial merge checkout.  IOW, adding the explicit behaviour should not be necessary.

          Bharath R added a comment - - edited

          brianjmurrell rthirucs trejkaz asouchang drewish  Did somebody got this working?
          I get the below error:
          stderr: error: Your local changes to the following files would be overwritten by merge: Zipfile.zip

          This started after I enabled Git LFS pull after checkout option in Multibranch Bitbucket pipeline,
          So i see this option doing a spare checkout as below
          Enabling Git LFS pull
          > git config core.sparsecheckout # timeout=10
          And looks like as per the discussion above, with spare checkout, merge won't work?

          Also, this happens on master node, even before my script execution starts.
          Clean before/after checkout, Wipeout repository & force clone, none of them help.

          The only workaround I have at the moment is to rebase the PR to master. This somehow solves the problem

          Bharath R added a comment - - edited brianjmurrell rthirucs trejkaz asouchang drewish   Did somebody got this working? I get the below error: stderr: error: Your local changes to the following files would be overwritten by merge: Zipfile.zip This started after I enabled Git LFS pull after checkout option in Multibranch Bitbucket pipeline, So i see this option doing a spare checkout as below Enabling Git LFS pull > git config core.sparsecheckout # timeout=10 And looks like as per the discussion above, with spare checkout, merge won't work? Also, this happens on master node, even before my script execution starts. Clean before/after checkout, Wipeout repository & force clone, none of them help. The only workaround I have at the moment is to rebase the PR to master. This somehow solves the problem

          Seeing "git config core.sparsecheckout" in the log does not mean that sparse checkout is enabled. git-client-plugin (source) runs that command just to ask Git whether sparse checkout is already enabled. Depending on how Git answers and whether you have configured "Sparse Checkout paths" in Jenkins, git-client-plugin can then run "git config core.sparsecheckout true" or "git config core.sparsecheckout false" to change the setting.

          Also, shallow clone and sparse checkout are separate features. Shallow clone controls how much history Git clones from the remote repository to the local repository, and sparse checkout controls which files Git checks out from the local repository to the working tree.

          Kalle Niemitalo added a comment - Seeing "git config core.sparsecheckout" in the log does not mean that sparse checkout is enabled. git-client-plugin ( source ) runs that command just to ask Git whether sparse checkout is already enabled. Depending on how Git answers and whether you have configured "Sparse Checkout paths" in Jenkins, git-client-plugin can then run "git config core.sparsecheckout true" or "git config core.sparsecheckout false" to change the setting. Also, shallow clone and sparse checkout are separate features. Shallow clone controls how much history Git clones from the remote repository to the local repository, and sparse checkout controls which files Git checks out from the local repository to the working tree.

          Any idea how to continue on this failure in Jenkinsfile (post always does not work in this case), it seems that this plugin failure exits the build always, I wanted to notify the user automatically on this kind of failure (conflicts) in slack.

          Limor Segal Shevah added a comment - Any idea how to continue on this failure in Jenkinsfile (post always does not work in this case), it seems that this plugin failure exits the build always, I wanted to notify the user automatically on this kind of failure (conflicts) in slack.

          Krzysztof added a comment - - edited

           Hi guys was it solved? I got similar issues:


          Steps to reproduce:

          1. Make a PR that will surely get conflicts fast
          2. Wait for some commits to come to master, that will create CONFLICTS
          3. Solve conflicts, git commit --amend --no-edit, and git push --force to PR
          4. Bitbucket will say that conflicts are solved
          5. Jenkins CI build will fail, claiming that conflicts are still there

          Any idea how can I fix it. Right now rebase with master doesn't work for Jenkins. It still thinks that there is a conflict :/

          Krzysztof added a comment - - edited  Hi guys was it solved? I got similar issues: Steps to reproduce: Make a PR that will surely get conflicts fast Wait for some commits to come to master, that will create CONFLICTS Solve conflicts, git commit --amend --no-edit, and git push --force to PR Bitbucket will say that conflicts are solved Jenkins CI build will fail, claiming that conflicts are still there Any idea how can I fix it. Right now rebase with master doesn't work for Jenkins. It still thinks that there is a conflict :/

            Unassigned Unassigned
            asouchang Asou Chang
            Votes:
            9 Vote for this issue
            Watchers:
            16 Start watching this issue

              Created:
              Updated: