Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-66025

Builds lost with large projects repository

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      Hello,

      When using docker-plugin with hundred of simultaneous builds, sometimes the connection is lost.

      docker-plugin should be able to restore the connection, it doesn't, instead the job is broken and can't be canceled. This is a bug.

       

      We are using this docker-plugin in a heavy load installation of Jenkins.

      Thanks,

      Jean-Sébastien

        Attachments

          Activity

          Hide
          jsbevilacqua Jean-Sébastien added a comment -

          This issue is linked to the 65872 https://issues.jenkins.io/browse/JENKINS-65872

          Show
          jsbevilacqua Jean-Sébastien added a comment - This issue is linked to the 65872 https://issues.jenkins.io/browse/JENKINS-65872
          Hide
          jsbevilacqua Jean-Sébastien added a comment -

          The problem is linked to the utilization of netty as transport layer, Http resolves the problem

          Show
          jsbevilacqua Jean-Sébastien added a comment - The problem is linked to the utilization of netty as transport layer, Http resolves the problem
          Hide
          pjdarton pjdarton added a comment -

          OK, without a mass of additional logging/diagnostics it'll be difficult to independently confirm this.
          e.g. we use this plugin where I work, under heavy load across thousands of Jenkins instances, but we've not seen this to be a problem.
          ... but if you're saying that you've already traced the issue to the netty transport layer and found that replacing it with Apache HttpClient then that pretty much takes it out of the hands of the docker-plugin code - the docker-plugin code doesn't control the netty transport layer.
          FYI the "chain of command" here is that the docker-plugin uses the docker-java-api-plugin to provide it with a java library that gives it access to the docker API. The docker-java-api-plugin's job is to provide a java library (and with its dependencies) to the rest of Jenkins, and it uses the docker-java library. The docker-java library is able to use a few different transports, but back when the docker-plugin was designed the only option was Netty, so that's what it did (and did so successfully).
          i.e. if you're saying that switching the transport layer from netty to http resolves this issue then that's it's not a bug in the docker-plugin as the docker plugin is "working as designed" here (even if it is, as you claim, being let down by underlying layers of code).

          I am aware that switching transport layers to Apache's http library may provide other benefits that would be welcomed (e.g. better support for Microsoft Npipes - Windows had no docker support back when the docker-plugin was designed) so that wasn't even a consideration back then), but that'd be a (non-trivial) enhancement rather than a bugfix.

          i.e. As a maintainer of the docker-plugin, I'd welcome a PR (or, more likely, a set of PRs) that made such an enhancement (because Apache HttpClient 5 seems to be the docker-java library's preferred transport these days, and especially if the PR adds support for Microsoft pipes - now Microsoft have finally started to support docker, it'd be nice to improve the docker-plugin's support for their docker variant too), but beware that it may be a non-trivial bit of work and it would be well outside the scope of a mere "bugfix".

          Show
          pjdarton pjdarton added a comment - OK, without a mass of additional logging/diagnostics it'll be difficult to independently confirm this. e.g. we use this plugin where I work, under heavy load across thousands of Jenkins instances, but we've not seen this to be a problem. ... but if you're saying that you've already traced the issue to the netty transport layer and found that replacing it with Apache HttpClient then that pretty much takes it out of the hands of the docker-plugin code - the docker-plugin code doesn't control the netty transport layer. FYI the "chain of command" here is that the docker-plugin uses the docker-java-api-plugin to provide it with a java library that gives it access to the docker API. The docker-java-api-plugin's job is to provide a java library (and with its dependencies) to the rest of Jenkins, and it uses the docker-java library. The docker-java library is able to use a few different transports, but back when the docker-plugin was designed the only option was Netty, so that's what it did (and did so successfully). i.e. if you're saying that switching the transport layer from netty to http resolves this issue then that's it's not a bug in the docker-plugin as the docker plugin is "working as designed" here (even if it is, as you claim, being let down by underlying layers of code). I am aware that switching transport layers to Apache's http library may provide other benefits that would be welcomed (e.g. better support for Microsoft Npipes - Windows had no docker support back when the docker-plugin was designed) so that wasn't even a consideration back then), but that'd be a (non-trivial) enhancement rather than a bugfix. i.e. As a maintainer of the docker-plugin, I'd welcome a PR (or, more likely, a set of PRs) that made such an enhancement (because Apache HttpClient 5 seems to be the docker-java library's preferred transport these days, and especially if the PR adds support for Microsoft pipes - now Microsoft have finally started to support docker, it'd be nice to improve the docker-plugin's support for their docker variant too), but beware that it may be a non-trivial bit of work and it would be well outside the scope of a mere "bugfix".
          Show
          jsbevilacqua Jean-Sébastien added a comment - Pull request: https://github.com/jenkinsci/docker-plugin/pull/843

            People

            Assignee:
            ndeloof Nicolas De Loof
            Reporter:
            jsbevilacqua Jean-Sébastien
            Votes:
            1 Vote for this issue
            Watchers:
            3 Start watching this issue

              Dates

              Created:
              Updated: