Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-69597

Exception in DeltaRepositoryCallback for large binary files

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Major Major
    • git-forensics-plugin
    • Forensics API Plugin (forensics-api): 1.15.1
      Git plugin (git): 4.11.3
      Git client plugin (git-client): 3.11.0
      Git Forensics Plugin (git-forensics): 1.9.0
      GIT server Plugin (git-server): 1.11

      Jenkins 2.346.2

      Introducing a large jar file (~63MB) in our software fails the git forensic plugin with an exception for calculating the delta:

      Also:   hudson.remoting.Channel$CallSiteStackTrace: Remote call to jenkins1
      		at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1784)
      		at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:356)
      		at hudson.remoting.Channel.call(Channel.java:1000)
      		at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:285)
      		at com.sun.proxy.$Proxy84.withRepository(Unknown Source)
      		at org.jenkinsci.plugins.gitclient.RemoteGitImpl.withRepository(RemoteGitImpl.java:234)
      		at io.jenkins.plugins.forensics.git.delta.GitDeltaCalculator.calculateDelta(GitDeltaCalculator.java:56)
      		at io.jenkins.plugins.coverage.model.CodeDeltaCalculator.calculateCodeDeltaToReference(CodeDeltaCalculator.java:78)
      		at io.jenkins.plugins.coverage.model.CoverageReporter.run(CoverageReporter.java:86)
      		at io.jenkins.plugins.coverage.CoverageProcessor.performCoverageReport(CoverageProcessor.java:133)
      		at io.jenkins.plugins.coverage.CoveragePublisher.perform(CoveragePublisher.java:114)
      		at jenkins.tasks.SimpleBuildStep.perform(SimpleBuildStep.java:123)
      		at org.jenkinsci.plugins.workflow.steps.CoreStep$Execution.run(CoreStep.java:101)
      		at org.jenkinsci.plugins.workflow.steps.CoreStep$Execution.run(CoreStep.java:71)
      		at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)
      		at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
      org.eclipse.jgit.errors.LargeObjectException: 464eb05a2f22d2b1f209022897a0df93c53b5115 exceeds size limit
      	at org.eclipse.jgit.internal.storage.file.LargePackedWholeObject.getCachedBytes(LargePackedWholeObject.java:69)
      	at io.jenkins.plugins.forensics.git.delta.DeltaRepositoryCallback.getFileContent(DeltaRepositoryCallback.java:212)
      	at io.jenkins.plugins.forensics.git.delta.DeltaRepositoryCallback.createFileChanges(DeltaRepositoryCallback.java:182)
      	at io.jenkins.plugins.forensics.git.delta.DeltaRepositoryCallback.calculateDelta(DeltaRepositoryCallback.java:107)
      	at io.jenkins.plugins.forensics.git.delta.DeltaRepositoryCallback.invoke(DeltaRepositoryCallback.java:70)
      	at io.jenkins.plugins.forensics.git.delta.DeltaRepositoryCallback.invoke(DeltaRepositoryCallback.java:41)
      	at org.jenkinsci.plugins.gitclient.AbstractGitAPIImpl.withRepository(AbstractGitAPIImpl.java:29)
      	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.withRepository(CliGitAPIImpl.java:84)
      	at jdk.internal.reflect.GeneratedMethodAccessor244.invoke(Unknown Source)
      	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
      	at hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:924)
      	at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:902)
      	at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:853)
      	at hudson.remoting.UserRequest.perform(UserRequest.java:211)
      	at hudson.remoting.UserRequest.perform(UserRequest.java:54)
      	at hudson.remoting.Request$2.run(Request.java:376)
      	at hudson.remoting.InterceptingExecutorService.lambda$wrap$0(InterceptingExecutorService.java:78)
      	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
      	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
      	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
      	at java.base/java.lang.Thread.run(Thread.java:829)
      

      The git repository itself handles the file without an issue.

      Solution would be to exclude large files from the delta calculation (and consequently informing the user that the displayed information doesn't include all files) or to handle binary files in some other way (it is probably safe to assume that 99% of these cases are large binary files).

          [JENKINS-69597] Exception in DeltaRepositoryCallback for large binary files

          Uwe Hausbrand added a comment -

          Reduced priority as we can just disable the plugin.

          Uwe Hausbrand added a comment - Reduced priority as we can just disable the plugin.

          Uwe Hausbrand added a comment -

          Would like to contribute and try fixing this issue, but encounter problems with setting up the warnings-dev- environment

          Uwe Hausbrand added a comment - Would like to contribute and try fixing this issue, but encounter problems with setting up the warnings-dev- environment

          Ulli Hafner added a comment -

          Let me know how I can help to set up the development environment. I think that I currently did not include the git-forensics plugin in https://github.com/uhafner/warnings-ng-plugin-devenv.

          You can also use the warnings-ng Gitter channel to ask questions.

          Ulli Hafner added a comment - Let me know how I can help to set up the development environment. I think that I currently did not include the git-forensics plugin in https://github.com/uhafner/warnings-ng-plugin-devenv. You can also use the warnings-ng Gitter channel to ask questions.

          Uwe Hausbrand added a comment -

          drulli The git forensics plugin is in the warnings-ng-plugin-devenv. I also like the effort to provide a full dev environment which is in most cases the biggest issue for contributing. Especially with something where you need such things as an jenkins instance. I had some trouble with a required maven version and after that some tests fail. So I don't have a clean and working base. I will gather information and post it to the gitter channel to solve these issues.

          Uwe Hausbrand added a comment - drulli The git forensics plugin is in the warnings-ng-plugin-devenv. I also like the effort to provide a full dev environment which is in most cases the biggest issue for contributing. Especially with something where you need such things as an jenkins instance. I had some trouble with a required maven version and after that some tests fail. So I don't have a clean and working base. I will gather information and post it to the gitter channel to solve these issues.

          Uwe Hausbrand added a comment -

          This issue is a greater problem than I thought. The code-coverage-api is using the git-forensic code for calculating delta data also, which means even someone not using git-forensics will run in this exception in case the delta calculation runs into an commit with a large enough file.

          Uwe Hausbrand added a comment - This issue is a greater problem than I thought. The code-coverage-api is using the git-forensic code for calculating delta data also, which means even someone not using git-forensics will run in this exception in case the delta calculation runs into an commit with a large enough file.

            drulli Ulli Hafner
            uhausbrand Uwe Hausbrand
            Votes:
            1 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: