Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-71349

git plugin left a lock file in controller cache folder for pipeline repos, which block all flows

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • git-plugin
    • None
    • Jenkins 2.346.1 with git plugin 4.12.1

      git plugin left a lock file in controller cache folder for pipeline repos, which block all flows.

      It happens after we uplift git plugin to 4.12.1.

      I think it may related to the feature disable hooks or something else.

      we do not enable hooks in controller.

       

      It happens three times in one week.

      I think sometime it could not release lock file success or it happens that multiple job try to lock same file.

      When it happens, all job fails to load pipeline repos until we delete the lock file munally.

      Could you please help to check it? Could we ignore this error when fail to change hooks configuration?

       

      error log:
      ERROR: Checkout failed
      org.eclipse.jgit.errors.LockFailedException: Cannot lock /proj/XXX/workspace@libs/72398dd74922d3156ed234ba0cbb097decd3eeabff521f0f59e1fa740478dc95/.git/config. Ensure that no other process has an open file handle on the lock file /proj/XXX/workspace@libs/72398dd74922d3156ed234ba0cbb097decd3eeabff521f0f59e1fa740478dc95/.git/config.lock, then you may delete the lock file and retry.
      at org.eclipse.jgit.storage.file.FileBasedConfig.save(FileBasedConfig.java:221)
      at jenkins.plugins.git.GitHooksConfiguration.disable(GitHooksConfiguration.java:139)
      at jenkins.plugins.git.GitHooksConfiguration.lambda$configure$2f1f6402$1(GitHooksConfiguration.java:113)
      at org.jenkinsci.plugins.gitclient.AbstractGitAPIImpl.withRepository(AbstractGitAPIImpl.java:29)
      at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.withRepository(CliGitAPIImpl.java:87)
      at jenkins.plugins.git.GitHooksConfiguration.configure(GitHooksConfiguration.java:112)
      at jenkins.plugins.git.GitHooksConfiguration.configure(GitHooksConfiguration.java:99)
      at jenkins.plugins.git.GitHooksConfiguration.configure(GitHooksConfiguration.java:91)
      at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1240)
      at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1312)
      at org.jenkinsci.plugins.workflow.steps.scm.SCMStep.checkout(SCMStep.java:129)
      at org.jenkinsci.plugins.workflow.libs.SCMSourceRetriever.lambda$doRetrieve$1(SCMSourceRetriever.java:201)
      at org.jenkinsci.plugins.workflow.libs.SCMSourceRetriever.retrySCMOperation(SCMSourceRetriever.java:148)
      at org.jenkinsci.plugins.workflow.libs.SCMSourceRetriever.doRetrieve(SCMSourceRetriever.java:200)
      at org.jenkinsci.plugins.workflow.libs.SCMSourceRetriever.retrieve(SCMSourceRetriever.java:137)
      at org.jenkinsci.plugins.workflow.libs.LibraryAdder.retrieve(LibraryAdder.java:260)
      at org.jenkinsci.plugins.workflow.libs.LibraryAdder.add(LibraryAdder.java:150)
      at org.jenkinsci.plugins.workflow.libs.LibraryDecorator$1.call(LibraryDecorator.java:125)
      at org.codehaus.groovy.control.CompilationUnit.applyToPrimaryClassNodes(CompilationUnit.java:1087)
      at org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:624)
      at org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:602)
      at org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:579)
      at groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:323)
      at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:293)
      at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.GroovySandbox$Scope.parse(GroovySandbox.java:163)
      at org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.doParse(CpsGroovyShell.java:142)
      at org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.reparse(CpsGroovyShell.java:127)
      at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.parseScript(CpsFlowExecution.java:561)
      at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.start(CpsFlowExecution.java:513)
      at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:335)
      at hudson.model.ResourceController.execute(ResourceController.java:101)
      at hudson.model.Executor.run(Executor.java:442)

          [JENKINS-71349] git plugin left a lock file in controller cache folder for pipeline repos, which block all flows

          Mark Waite added a comment -

          There are at least two options available to you to reduce the chances of a collision on a Pipeline shared library. They include:

          Mark Waite added a comment - There are at least two options available to you to reduce the chances of a collision on a Pipeline shared library. They include: Enable Pipeline shared library caching as described in JENKINS-38992 and implemented in releases of Pipeline: Groovy Libraries Allow git hooks on the controller as described in the git plugin documentation

          Stone Lee added a comment - - edited

          Thanks markewaite .

          I already allow git hooks on the controller.

          Will try library caching.

           

          And is there any plan to improve git plugin for this error?

           

          Stone Lee added a comment - - edited Thanks markewaite . I already allow git hooks on the controller. Will try library caching.   And is there any plan to improve git plugin for this error?  

          Mark Waite added a comment -

          And is there any plan to improve git plugin for this error?

          I don't plan to implement any changes in that area. If you'd like to improve it, you're welcome to propose a pull request that either adds a conditional to avoid saving the configuration if it is not changed or that adds a try/catch to detect the exception and ignore it. The lines of code that need the implementation are in GitHooksConfiguration.

          Mark Waite added a comment - And is there any plan to improve git plugin for this error? I don't plan to implement any changes in that area. If you'd like to improve it, you're welcome to propose a pull request that either adds a conditional to avoid saving the configuration if it is not changed or that adds a try/catch to detect the exception and ignore it. The lines of code that need the implementation are in GitHooksConfiguration .

            Unassigned Unassigned
            li445861132 Stone Lee
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated: