Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-69597

Git Forensic Plugin throws exception for large binary files

    XMLWordPrintable

Details

    • Bug
    • Status: Open (View Workflow)
    • Major
    • Resolution: Unresolved
    • git-forensics-plugin
    • Forensics API Plugin (forensics-api): 1.15.1
      Git plugin (git): 4.11.3
      Git client plugin (git-client): 3.11.0
      Git Forensics Plugin (git-forensics): 1.9.0
      GIT server Plugin (git-server): 1.11

      Jenkins 2.346.2

    Description

      Introducing a large jar file (~63MB) in our software fails the git forensic plugin with an exception for calculating the delta:

      Also:   hudson.remoting.Channel$CallSiteStackTrace: Remote call to jenkins1
      		at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1784)
      		at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:356)
      		at hudson.remoting.Channel.call(Channel.java:1000)
      		at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:285)
      		at com.sun.proxy.$Proxy84.withRepository(Unknown Source)
      		at org.jenkinsci.plugins.gitclient.RemoteGitImpl.withRepository(RemoteGitImpl.java:234)
      		at io.jenkins.plugins.forensics.git.delta.GitDeltaCalculator.calculateDelta(GitDeltaCalculator.java:56)
      		at io.jenkins.plugins.coverage.model.CodeDeltaCalculator.calculateCodeDeltaToReference(CodeDeltaCalculator.java:78)
      		at io.jenkins.plugins.coverage.model.CoverageReporter.run(CoverageReporter.java:86)
      		at io.jenkins.plugins.coverage.CoverageProcessor.performCoverageReport(CoverageProcessor.java:133)
      		at io.jenkins.plugins.coverage.CoveragePublisher.perform(CoveragePublisher.java:114)
      		at jenkins.tasks.SimpleBuildStep.perform(SimpleBuildStep.java:123)
      		at org.jenkinsci.plugins.workflow.steps.CoreStep$Execution.run(CoreStep.java:101)
      		at org.jenkinsci.plugins.workflow.steps.CoreStep$Execution.run(CoreStep.java:71)
      		at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)
      		at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
      org.eclipse.jgit.errors.LargeObjectException: 464eb05a2f22d2b1f209022897a0df93c53b5115 exceeds size limit
      	at org.eclipse.jgit.internal.storage.file.LargePackedWholeObject.getCachedBytes(LargePackedWholeObject.java:69)
      	at io.jenkins.plugins.forensics.git.delta.DeltaRepositoryCallback.getFileContent(DeltaRepositoryCallback.java:212)
      	at io.jenkins.plugins.forensics.git.delta.DeltaRepositoryCallback.createFileChanges(DeltaRepositoryCallback.java:182)
      	at io.jenkins.plugins.forensics.git.delta.DeltaRepositoryCallback.calculateDelta(DeltaRepositoryCallback.java:107)
      	at io.jenkins.plugins.forensics.git.delta.DeltaRepositoryCallback.invoke(DeltaRepositoryCallback.java:70)
      	at io.jenkins.plugins.forensics.git.delta.DeltaRepositoryCallback.invoke(DeltaRepositoryCallback.java:41)
      	at org.jenkinsci.plugins.gitclient.AbstractGitAPIImpl.withRepository(AbstractGitAPIImpl.java:29)
      	at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.withRepository(CliGitAPIImpl.java:84)
      	at jdk.internal.reflect.GeneratedMethodAccessor244.invoke(Unknown Source)
      	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
      	at hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:924)
      	at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:902)
      	at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:853)
      	at hudson.remoting.UserRequest.perform(UserRequest.java:211)
      	at hudson.remoting.UserRequest.perform(UserRequest.java:54)
      	at hudson.remoting.Request$2.run(Request.java:376)
      	at hudson.remoting.InterceptingExecutorService.lambda$wrap$0(InterceptingExecutorService.java:78)
      	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
      	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
      	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
      	at java.base/java.lang.Thread.run(Thread.java:829)
      

      The git repository itself handles the file without an issue.

      Solution would be to exclude large files from the delta calculation (and consequently informing the user that the displayed information doesn't include all files) or to handle binary files in some other way (it is probably safe to assume that 99% of these cases are large binary files).

      Attachments

        Activity

          uhausbrand Uwe Hausbrand added a comment -

          Reduced priority as we can just disable the plugin.

          uhausbrand Uwe Hausbrand added a comment - Reduced priority as we can just disable the plugin.
          uhausbrand Uwe Hausbrand added a comment -

          Would like to contribute and try fixing this issue, but encounter problems with setting up the warnings-dev- environment

          uhausbrand Uwe Hausbrand added a comment - Would like to contribute and try fixing this issue, but encounter problems with setting up the warnings-dev- environment
          drulli Ulli Hafner added a comment -

          Let me know how I can help to set up the development environment. I think that I currently did not include the git-forensics plugin in https://github.com/uhafner/warnings-ng-plugin-devenv.

          You can also use the warnings-ng Gitter channel to ask questions.

          drulli Ulli Hafner added a comment - Let me know how I can help to set up the development environment. I think that I currently did not include the git-forensics plugin in https://github.com/uhafner/warnings-ng-plugin-devenv. You can also use the warnings-ng Gitter channel to ask questions.
          uhausbrand Uwe Hausbrand added a comment -

          drulli The git forensics plugin is in the warnings-ng-plugin-devenv. I also like the effort to provide a full dev environment which is in most cases the biggest issue for contributing. Especially with something where you need such things as an jenkins instance. I had some trouble with a required maven version and after that some tests fail. So I don't have a clean and working base. I will gather information and post it to the gitter channel to solve these issues.

          uhausbrand Uwe Hausbrand added a comment - drulli The git forensics plugin is in the warnings-ng-plugin-devenv. I also like the effort to provide a full dev environment which is in most cases the biggest issue for contributing. Especially with something where you need such things as an jenkins instance. I had some trouble with a required maven version and after that some tests fail. So I don't have a clean and working base. I will gather information and post it to the gitter channel to solve these issues.
          uhausbrand Uwe Hausbrand added a comment -

          This issue is a greater problem than I thought. The code-coverage-api is using the git-forensic code for calculating delta data also, which means even someone not using git-forensics will run in this exception in case the delta calculation runs into an commit with a large enough file.

          uhausbrand Uwe Hausbrand added a comment - This issue is a greater problem than I thought. The code-coverage-api is using the git-forensic code for calculating delta data also, which means even someone not using git-forensics will run in this exception in case the delta calculation runs into an commit with a large enough file.

          People

            Unassigned Unassigned
            uhausbrand Uwe Hausbrand
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated: