Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-44598

git workspace should be reset before build PR merged with base branch

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Open (View Workflow)
    • Priority: Major
    • Resolution: Unresolved
    • Labels:
      None
    • Environment:
      OS: CentOS 7.3.1611
      Jenkins: 2.46.2
      GitHub Branch Source Plugin: 2.0.6
      git version: 1.8.3.1
    • Similar Issues:

      Description

      The plugin attempts to merge the PR and the base branch before build. However, if git-merge introduces a conflict, the unmerged files will be left in the PR's git workspace, which causes git-merge in the following builds always fail, because the conflict is not solved.

      Here's the log snippets:

      For build #3, which introduces the merge conflict

      > git fetch --tags --progress git@github.mycompany.com:SERVER_REPO.git +refs/pull//head:refs/remotes/origin/pr/
       Merging dev_6_2 commit fc999a1dae8861c1e5d3c21c8deb81f1e7bdf3a8 into PR head commit 5042e295186f8fe4bbb61ac97d97d69d1932912c
       > git config core.sparsecheckout # timeout=10
       > git checkout -f 5042e295186f8fe4bbb61ac97d97d69d1932912c
       > git merge fc999a1dae8861c1e5d3c21c8deb81f1e7bdf3a8 # timeout=10
       > git config core.sparsecheckout # timeout=10
       > git checkout -f 5042e295186f8fe4bbb61ac97d97d69d1932912c
      GitHub has been notified of this commit’s build result
      hudson.plugins.git.GitException: Command "git merge fc999a1dae8861c1e5d3c21c8deb81f1e7bdf3a8" returned status code 1:
       stdout: Auto-merging docker-environment/ansible/group_vars/all.yml
       CONFLICT (add/add): Merge conflict in docker-environment/ansible/group_vars/all.yml
       Automatic merge failed; fix conflicts and then commit the result.
      stderr: 
       at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1877)
       at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1845)
       at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1841)
       at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommand(CliGitAPIImpl.java:1486)
       at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$3.execute(CliGitAPIImpl.java:621)
       at org.jenkinsci.plugins.github_branch_source.GitHubSCMSource$MergeWith.decorateRevisionToBuild(GitHubSCMSource.java:1003)
       at hudson.plugins.git.GitSCM.determineRevisionToBuild(GitSCM.java:1015)
       at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1108)
       at org.jenkinsci.plugins.workflow.steps.scm.SCMStep.checkout(SCMStep.java:109)
       at org.jenkinsci.plugins.workflow.cps.CpsScmFlowDefinition.create(CpsScmFlowDefinition.java:130)
       at org.jenkinsci.plugins.workflow.multibranch.SCMBinder.create(SCMBinder.java:107)
       at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:234)
       at hudson.model.ResourceController.execute(ResourceController.java:98)
       at hudson.model.Executor.run(Executor.java:405)
       Finished: FAILURE
      

      For the following build #4, #5 and so on, the merge always fails

      > git fetch --tags --progress git@github.mycompany.com:SERVER_REPO.git +refs/pull//head:refs/remotes/origin/pr/
       Merging dev_6_2 commit fc999a1dae8861c1e5d3c21c8deb81f1e7bdf3a8 into PR head commit 32fe21329c3f0349347d029917ba29a39fc0364a
       > git config core.sparsecheckout # timeout=10
       > git checkout -f 32fe21329c3f0349347d029917ba29a39fc0364a
       > git merge fc999a1dae8861c1e5d3c21c8deb81f1e7bdf3a8 # timeout=10
       > git config core.sparsecheckout # timeout=10
       > git checkout -f 32fe21329c3f0349347d029917ba29a39fc0364a
      GitHub has been notified of this commit’s build result
      hudson.plugins.git.GitException: Command "git merge fc999a1dae8861c1e5d3c21c8deb81f1e7bdf3a8" returned status code 128:
       stdout: 
       stderr: error: 'merge' is not possible because you have unmerged files.
       hint: Fix them up in the work tree,
       hint: and then use 'git add/rm <file>' as
       hint: appropriate to mark resolution and make a commit,
       hint: or use 'git commit -a'.
       fatal: Exiting because of an unresolved conflict.
      at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1877)
       at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1845)
       at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1841)
       at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommand(CliGitAPIImpl.java:1486)
       at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$3.execute(CliGitAPIImpl.java:621)
       at org.jenkinsci.plugins.github_branch_source.GitHubSCMSource$MergeWith.decorateRevisionToBuild(GitHubSCMSource.java:1003)
       at hudson.plugins.git.GitSCM.determineRevisionToBuild(GitSCM.java:1015)
       at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1108)
       at org.jenkinsci.plugins.workflow.steps.scm.SCMStep.checkout(SCMStep.java:109)
       at org.jenkinsci.plugins.workflow.cps.CpsScmFlowDefinition.create(CpsScmFlowDefinition.java:130)
       at org.jenkinsci.plugins.workflow.multibranch.SCMBinder.create(SCMBinder.java:107)
       at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:234)
       at hudson.model.ResourceController.execute(ResourceController.java:98)
       at hudson.model.Executor.run(Executor.java:405)
       Finished: FAILURE
      

       

      Then this build will be stuck in this situation unless we manually issue 'git reset HEAD --hard' in that git workspace.

        Attachments

          Issue Links

            Activity

            Hide
            rthirucs Thirunavukkarasu Ravichandran added a comment -

            I'm still facing the problem. The Jira issue is in OPEN state. Can this be resolved somehow?

            Show
            rthirucs Thirunavukkarasu Ravichandran added a comment - I'm still facing the problem. The Jira issue is in OPEN state. Can this be resolved somehow?
            Hide
            brianjmurrell Brian J Murrell added a comment - - edited

            I too am hitting this and would appreciate a status update on whether anything is being done to resolve it.

            Adding the Clean before checkout (from Jenkins Git plugin) isn't really an acceptable solution because I really only want the cleaning before the initial checkout on the master (just to resolve the initial checkout merge failure before the Pipeline is even run) and not before every checkout for every Pipeline build step.  This is because cleaning in some of the subsequent Pipeline steps leads to other errors such as:

            10:21:35  Fetching changes from the remote Git repository
            10:21:35  Cleaning workspace
            10:21:35  ERROR: Error fetching remote repo 'origin'
            10:21:35  hudson.plugins.git.GitException: Failed to fetch from https://github.com/daos-stack/daos.git
            10:21:35  	at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:904)
            10:21:35  	at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1144)
            10:21:35  	at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1175)
            10:21:35  	at org.jenkinsci.plugins.workflow.steps.scm.SCMStep.checkout(SCMStep.java:120)
            10:21:35  	at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:90)
            10:21:35  	at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:77)
            10:21:35  	at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)
            10:21:35  	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
            10:21:35  	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
            10:21:35  	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
            10:21:35  	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
            10:21:35  	at java.lang.Thread.run(Thread.java:748)
            10:21:35  Caused by: hudson.plugins.git.GitException: Command "git clean -fdx" returned status code 128:
            10:21:35  stdout: 
            10:21:35  stderr: fatal: Not a git repository: install/include/boost/../../.git/modules/src/boost
            10:21:35  
            10:21:35  	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2318)
            10:21:35  	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2248)
            10:21:35  	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2244)
            10:21:35  	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommand(CliGitAPIImpl.java:1777)
            10:21:35  	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommand(CliGitAPIImpl.java:1789)
            10:21:35  	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.clean(CliGitAPIImpl.java:901)
            10:21:35  	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.clean(CliGitAPIImpl.java:912)
            10:21:35  	at hudson.plugins.git.GitAPI.clean(GitAPI.java:311)
            10:21:35  	at sun.reflect.GeneratedMethodAccessor94.invoke(Unknown Source)
            10:21:35  	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
            10:21:35  	at java.lang.reflect.Method.invoke(Method.java:498)
            10:21:35  	at hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:929)
            10:21:35  	at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:903)
            10:21:35  	at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:855)
            10:21:35  	at hudson.remoting.UserRequest.perform(UserRequest.java:212)
            10:21:35  	at hudson.remoting.UserRequest.perform(UserRequest.java:54)
            10:21:35  	at hudson.remoting.Request$2.run(Request.java:369)
            10:21:35  	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
            10:21:35  	... 4 more
            10:21:35  	Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to wolf-33_docker_1
            10:21:35  		at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
            10:21:35  		at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
            10:21:35  		at hudson.remoting.Channel.call(Channel.java:955)
            10:21:35  		at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:283)
            10:21:35  		at com.sun.proxy.$Proxy102.clean(Unknown Source)
            10:21:35  		at org.jenkinsci.plugins.gitclient.RemoteGitImpl.clean(RemoteGitImpl.java:463)
            10:21:35  		at hudson.plugins.git.extensions.impl.CleanBeforeCheckout.decorateFetchCommand(CleanBeforeCheckout.java:30)
            10:21:35  		at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:900)
            10:21:35  		at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1144)
            10:21:35  		at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1175)
            10:21:35  		at org.jenkinsci.plugins.workflow.steps.scm.SCMStep.checkout(SCMStep.java:120)
            10:21:35  		at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:90)
            10:21:35  		at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:77)
            10:21:35  		at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)
            10:21:35  		at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
            10:21:35  		... 4 more
            

            Yes, to be sure, the above is only a problem because of the content of the tree in the workspace, but it's only a problem because of this needed explicit Clean before checkout option that really should be the implied-and-always-executed behaviour when doing the initial merge checkout.  IOW, adding the explicit behaviour should not be necessary.

            Show
            brianjmurrell Brian J Murrell added a comment - - edited I too am hitting this and would appreciate a status update on whether anything is being done to resolve it. Adding the Clean before checkout  (from Jenkins Git plugin) isn't really an acceptable solution because I really only want the cleaning before the initial checkout on the master (just to resolve the initial checkout merge failure before the Pipeline is even run) and not before every checkout for every Pipeline build step.  This is because cleaning in some of the subsequent Pipeline steps leads to other errors such as: 10:21:35 Fetching changes from the remote Git repository 10:21:35 Cleaning workspace 10:21:35 ERROR: Error fetching remote repo 'origin' 10:21:35 hudson.plugins.git.GitException: Failed to fetch from https://github.com/daos-stack/daos.git 10:21:35 at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:904) 10:21:35 at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1144) 10:21:35 at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1175) 10:21:35 at org.jenkinsci.plugins.workflow.steps.scm.SCMStep.checkout(SCMStep.java:120) 10:21:35 at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:90) 10:21:35 at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:77) 10:21:35 at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47) 10:21:35 at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 10:21:35 at java.util.concurrent.FutureTask.run(FutureTask.java:266) 10:21:35 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) 10:21:35 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 10:21:35 at java.lang.Thread.run(Thread.java:748) 10:21:35 Caused by: hudson.plugins.git.GitException: Command "git clean -fdx" returned status code 128: 10:21:35 stdout: 10:21:35 stderr: fatal: Not a git repository: install/include/boost/../../.git/modules/src/boost 10:21:35 10:21:35 at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2318) 10:21:35 at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2248) 10:21:35 at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2244) 10:21:35 at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommand(CliGitAPIImpl.java:1777) 10:21:35 at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommand(CliGitAPIImpl.java:1789) 10:21:35 at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.clean(CliGitAPIImpl.java:901) 10:21:35 at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.clean(CliGitAPIImpl.java:912) 10:21:35 at hudson.plugins.git.GitAPI.clean(GitAPI.java:311) 10:21:35 at sun.reflect.GeneratedMethodAccessor94.invoke(Unknown Source) 10:21:35 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 10:21:35 at java.lang.reflect.Method.invoke(Method.java:498) 10:21:35 at hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:929) 10:21:35 at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:903) 10:21:35 at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:855) 10:21:35 at hudson.remoting.UserRequest.perform(UserRequest.java:212) 10:21:35 at hudson.remoting.UserRequest.perform(UserRequest.java:54) 10:21:35 at hudson.remoting.Request$2.run(Request.java:369) 10:21:35 at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72) 10:21:35 ... 4 more 10:21:35 Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to wolf-33_docker_1 10:21:35 at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741) 10:21:35 at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357) 10:21:35 at hudson.remoting.Channel.call(Channel.java:955) 10:21:35 at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:283) 10:21:35 at com.sun.proxy.$Proxy102.clean(Unknown Source) 10:21:35 at org.jenkinsci.plugins.gitclient.RemoteGitImpl.clean(RemoteGitImpl.java:463) 10:21:35 at hudson.plugins.git.extensions.impl.CleanBeforeCheckout.decorateFetchCommand(CleanBeforeCheckout.java:30) 10:21:35 at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:900) 10:21:35 at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1144) 10:21:35 at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1175) 10:21:35 at org.jenkinsci.plugins.workflow.steps.scm.SCMStep.checkout(SCMStep.java:120) 10:21:35 at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:90) 10:21:35 at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:77) 10:21:35 at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47) 10:21:35 at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 10:21:35 ... 4 more Yes, to be sure, the above is only a problem because of the content of the tree in the workspace, but it's only a problem because of this needed explicit Clean before checkout option that really should be the implied-and-always-executed behaviour when doing the initial merge checkout.  IOW, adding the explicit behaviour should not be necessary.
            Hide
            bharathr0209 Bharath R added a comment - - edited

            Brian J Murrell Thirunavukkarasu Ravichandran trejkaz Asou Chang andrew morton  Did somebody got this working?
            I get the below error:
            stderr: error: Your local changes to the following files would be overwritten by merge: Zipfile.zip

            This started after I enabled Git LFS pull after checkout option in Multibranch Bitbucket pipeline,
            So i see this option doing a spare checkout as below
            Enabling Git LFS pull
            > git config core.sparsecheckout # timeout=10
            And looks like as per the discussion above, with spare checkout, merge won't work?

            Also, this happens on master node, even before my script execution starts.
            Clean before/after checkout, Wipeout repository & force clone, none of them help.

            The only workaround I have at the moment is to rebase the PR to master. This somehow solves the problem

            Show
            bharathr0209 Bharath R added a comment - - edited Brian J Murrell Thirunavukkarasu Ravichandran trejkaz Asou Chang andrew morton   Did somebody got this working? I get the below error: stderr: error: Your local changes to the following files would be overwritten by merge: Zipfile.zip This started after I enabled Git LFS pull after checkout option in Multibranch Bitbucket pipeline, So i see this option doing a spare checkout as below Enabling Git LFS pull > git config core.sparsecheckout # timeout=10 And looks like as per the discussion above, with spare checkout, merge won't work? Also, this happens on master node, even before my script execution starts. Clean before/after checkout, Wipeout repository & force clone, none of them help. The only workaround I have at the moment is to rebase the PR to master. This somehow solves the problem
            Hide
            kon Kalle Niemitalo added a comment -

            Seeing "git config core.sparsecheckout" in the log does not mean that sparse checkout is enabled. git-client-plugin (source) runs that command just to ask Git whether sparse checkout is already enabled. Depending on how Git answers and whether you have configured "Sparse Checkout paths" in Jenkins, git-client-plugin can then run "git config core.sparsecheckout true" or "git config core.sparsecheckout false" to change the setting.

            Also, shallow clone and sparse checkout are separate features. Shallow clone controls how much history Git clones from the remote repository to the local repository, and sparse checkout controls which files Git checks out from the local repository to the working tree.

            Show
            kon Kalle Niemitalo added a comment - Seeing "git config core.sparsecheckout" in the log does not mean that sparse checkout is enabled. git-client-plugin ( source ) runs that command just to ask Git whether sparse checkout is already enabled. Depending on how Git answers and whether you have configured "Sparse Checkout paths" in Jenkins, git-client-plugin can then run "git config core.sparsecheckout true" or "git config core.sparsecheckout false" to change the setting. Also, shallow clone and sparse checkout are separate features. Shallow clone controls how much history Git clones from the remote repository to the local repository, and sparse checkout controls which files Git checks out from the local repository to the working tree.
            Hide
            limors Limor Segal Shevah added a comment -

            Any idea how to continue on this failure in Jenkinsfile (post always does not work in this case), it seems that this plugin failure exits the build always, I wanted to notify the user automatically on this kind of failure (conflicts) in slack.

            Show
            limors Limor Segal Shevah added a comment - Any idea how to continue on this failure in Jenkinsfile (post always does not work in this case), it seems that this plugin failure exits the build always, I wanted to notify the user automatically on this kind of failure (conflicts) in slack.

              People

              Assignee:
              Unassigned Unassigned
              Reporter:
              asouchang Asou Chang
              Votes:
              7 Vote for this issue
              Watchers:
              14 Start watching this issue

                Dates

                Created:
                Updated: