Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-69597

Exception in DeltaRepositoryCallback for large binary files

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Major Major
    • git-forensics-plugin
    • Forensics API Plugin (forensics-api): 1.15.1
      Git plugin (git): 4.11.3
      Git client plugin (git-client): 3.11.0
      Git Forensics Plugin (git-forensics): 1.9.0
      GIT server Plugin (git-server): 1.11

      Jenkins 2.346.2

      Introducing a large jar file (~63MB) in our software fails the git forensic plugin with an exception for calculating the delta:

      Also:   hudson.remoting.Channel$CallSiteStackTrace: Remote call to jenkins1
      		at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1784)
      		at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:356)
      		at hudson.remoting.Channel.call(Channel.java:1000)
      		at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:285)
      		at com.sun.proxy.$Proxy84.withRepository(Unknown Source)
      		at org.jenkinsci.plugins.gitclient.RemoteGitImpl.withRepository(RemoteGitImpl.java:234)
      		at io.jenkins.plugins.forensics.git.delta.GitDeltaCalculator.calculateDelta(GitDeltaCalculator.java:56)
      		at io.jenkins.plugins.coverage.model.CodeDeltaCalculator.calculateCodeDeltaToReference(CodeDeltaCalculator.java:78)
      		at io.jenkins.plugins.coverage.model.CoverageReporter.run(CoverageReporter.java:86)
      		at io.jenkins.plugins.coverage.CoverageProcessor.performCoverageReport(CoverageProcessor.java:133)
      		at io.jenkins.plugins.coverage.CoveragePublisher.perform(CoveragePublisher.java:114)
      		at jenkins.tasks.SimpleBuildStep.perform(SimpleBuildStep.java:123)
      		at org.jenkinsci.plugins.workflow.steps.CoreStep$Execution.run(CoreStep.java:101)
      		at org.jenkinsci.plugins.workflow.steps.CoreStep$Execution.run(CoreStep.java:71)
      		at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)
      		at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
      org.eclipse.jgit.errors.LargeObjectException: 464eb05a2f22d2b1f209022897a0df93c53b5115 exceeds size limit
      	at org.eclipse.jgit.internal.storage.file.LargePackedWholeObject.getCachedBytes(LargePackedWholeObject.java:69)
      	at io.jenkins.plugins.forensics.git.delta.DeltaRepositoryCallback.getFileContent(DeltaRepositoryCallback.java:212)
      	at io.jenkins.plugins.forensics.git.delta.DeltaRepositoryCallback.createFileChanges(DeltaRepositoryCallback.java:182)
      	at io.jenkins.plugins.forensics.git.delta.DeltaRepositoryCallback.calculateDelta(DeltaRepositoryCallback.java:107)
      	at io.jenkins.plugins.forensics.git.delta.DeltaRepositoryCallback.invoke(DeltaRepositoryCallback.java:70)
      	at io.jenkins.plugins.forensics.git.delta.DeltaRepositoryCallback.invoke(DeltaRepositoryCallback.java:41)
      	at org.jenkinsci.plugins.gitclient.AbstractGitAPIImpl.withRepository(AbstractGitAPIImpl.java:29)
      	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.withRepository(CliGitAPIImpl.java:84)
      	at jdk.internal.reflect.GeneratedMethodAccessor244.invoke(Unknown Source)
      	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
      	at hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:924)
      	at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:902)
      	at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:853)
      	at hudson.remoting.UserRequest.perform(UserRequest.java:211)
      	at hudson.remoting.UserRequest.perform(UserRequest.java:54)
      	at hudson.remoting.Request$2.run(Request.java:376)
      	at hudson.remoting.InterceptingExecutorService.lambda$wrap$0(InterceptingExecutorService.java:78)
      	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
      	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
      	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
      	at java.base/java.lang.Thread.run(Thread.java:829)
      

      The git repository itself handles the file without an issue.

      Solution would be to exclude large files from the delta calculation (and consequently informing the user that the displayed information doesn't include all files) or to handle binary files in some other way (it is probably safe to assume that 99% of these cases are large binary files).

          [JENKINS-69597] Exception in DeltaRepositoryCallback for large binary files

          Uwe Hausbrand created issue -
          Uwe Hausbrand made changes -
          Priority Original: Blocker [ 1 ] New: Major [ 3 ]

          Uwe Hausbrand added a comment -

          Reduced priority as we can just disable the plugin.

          Uwe Hausbrand added a comment - Reduced priority as we can just disable the plugin.
          Ulli Hafner made changes -
          Assignee Original: Ulli Hafner [ drulli ]
          Labels New: help-wanted newbie-friendly

          Uwe Hausbrand added a comment -

          Would like to contribute and try fixing this issue, but encounter problems with setting up the warnings-dev- environment

          Uwe Hausbrand added a comment - Would like to contribute and try fixing this issue, but encounter problems with setting up the warnings-dev- environment

          Ulli Hafner added a comment -

          Let me know how I can help to set up the development environment. I think that I currently did not include the git-forensics plugin in https://github.com/uhafner/warnings-ng-plugin-devenv.

          You can also use the warnings-ng Gitter channel to ask questions.

          Ulli Hafner added a comment - Let me know how I can help to set up the development environment. I think that I currently did not include the git-forensics plugin in https://github.com/uhafner/warnings-ng-plugin-devenv. You can also use the warnings-ng Gitter channel to ask questions.
          Ulli Hafner made changes -
          Labels Original: help-wanted newbie-friendly New: hacktoberfest help-wanted newbie-friendly

          Uwe Hausbrand added a comment -

          drulli The git forensics plugin is in the warnings-ng-plugin-devenv. I also like the effort to provide a full dev environment which is in most cases the biggest issue for contributing. Especially with something where you need such things as an jenkins instance. I had some trouble with a required maven version and after that some tests fail. So I don't have a clean and working base. I will gather information and post it to the gitter channel to solve these issues.

          Uwe Hausbrand added a comment - drulli The git forensics plugin is in the warnings-ng-plugin-devenv. I also like the effort to provide a full dev environment which is in most cases the biggest issue for contributing. Especially with something where you need such things as an jenkins instance. I had some trouble with a required maven version and after that some tests fail. So I don't have a clean and working base. I will gather information and post it to the gitter channel to solve these issues.

          Uwe Hausbrand added a comment -

          This issue is a greater problem than I thought. The code-coverage-api is using the git-forensic code for calculating delta data also, which means even someone not using git-forensics will run in this exception in case the delta calculation runs into an commit with a large enough file.

          Uwe Hausbrand added a comment - This issue is a greater problem than I thought. The code-coverage-api is using the git-forensic code for calculating delta data also, which means even someone not using git-forensics will run in this exception in case the delta calculation runs into an commit with a large enough file.
          Ulli Hafner made changes -
          Summary Original: Git Forensic Plugin throws exception for large binary files New: Exception in DeltaRepositoryCallback for large binary files

            drulli Ulli Hafner
            uhausbrand Uwe Hausbrand
            Votes:
            1 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: