Hello,

      When using docker-plugin with hundred of simultaneous builds, sometimes the connection is lost.

      docker-plugin should be able to restore the connection, it doesn't, instead the job is broken and can't be canceled. This is a bug.

       

      We are using this docker-plugin in a heavy load installation of Jenkins.

      Thanks,

      Jean-Sébastien

          [JENKINS-66025] Builds lost with large projects repository

          This issue is linked to the 65872 https://issues.jenkins.io/browse/JENKINS-65872

          Jean-Sébastien added a comment - This issue is linked to the 65872 https://issues.jenkins.io/browse/JENKINS-65872

          The problem is linked to the utilization of netty as transport layer, Http resolves the problem

          Jean-Sébastien added a comment - The problem is linked to the utilization of netty as transport layer, Http resolves the problem

          pjdarton added a comment -

          OK, without a mass of additional logging/diagnostics it'll be difficult to independently confirm this.
          e.g. we use this plugin where I work, under heavy load across thousands of Jenkins instances, but we've not seen this to be a problem.
          ... but if you're saying that you've already traced the issue to the netty transport layer and found that replacing it with Apache HttpClient then that pretty much takes it out of the hands of the docker-plugin code - the docker-plugin code doesn't control the netty transport layer.
          FYI the "chain of command" here is that the docker-plugin uses the docker-java-api-plugin to provide it with a java library that gives it access to the docker API. The docker-java-api-plugin's job is to provide a java library (and with its dependencies) to the rest of Jenkins, and it uses the docker-java library. The docker-java library is able to use a few different transports, but back when the docker-plugin was designed the only option was Netty, so that's what it did (and did so successfully).
          i.e. if you're saying that switching the transport layer from netty to http resolves this issue then that's it's not a bug in the docker-plugin as the docker plugin is "working as designed" here (even if it is, as you claim, being let down by underlying layers of code).

          I am aware that switching transport layers to Apache's http library may provide other benefits that would be welcomed (e.g. better support for Microsoft Npipes - Windows had no docker support back when the docker-plugin was designed) so that wasn't even a consideration back then), but that'd be a (non-trivial) enhancement rather than a bugfix.

          i.e. As a maintainer of the docker-plugin, I'd welcome a PR (or, more likely, a set of PRs) that made such an enhancement (because Apache HttpClient 5 seems to be the docker-java library's preferred transport these days, and especially if the PR adds support for Microsoft pipes - now Microsoft have finally started to support docker, it'd be nice to improve the docker-plugin's support for their docker variant too), but beware that it may be a non-trivial bit of work and it would be well outside the scope of a mere "bugfix".

          pjdarton added a comment - OK, without a mass of additional logging/diagnostics it'll be difficult to independently confirm this. e.g. we use this plugin where I work, under heavy load across thousands of Jenkins instances, but we've not seen this to be a problem. ... but if you're saying that you've already traced the issue to the netty transport layer and found that replacing it with Apache HttpClient then that pretty much takes it out of the hands of the docker-plugin code - the docker-plugin code doesn't control the netty transport layer. FYI the "chain of command" here is that the docker-plugin uses the docker-java-api-plugin to provide it with a java library that gives it access to the docker API. The docker-java-api-plugin's job is to provide a java library (and with its dependencies) to the rest of Jenkins, and it uses the docker-java library. The docker-java library is able to use a few different transports, but back when the docker-plugin was designed the only option was Netty, so that's what it did (and did so successfully). i.e. if you're saying that switching the transport layer from netty to http resolves this issue then that's it's not a bug in the docker-plugin as the docker plugin is "working as designed" here (even if it is, as you claim, being let down by underlying layers of code). I am aware that switching transport layers to Apache's http library may provide other benefits that would be welcomed (e.g. better support for Microsoft Npipes - Windows had no docker support back when the docker-plugin was designed) so that wasn't even a consideration back then), but that'd be a (non-trivial) enhancement rather than a bugfix. i.e. As a maintainer of the docker-plugin, I'd welcome a PR (or, more likely, a set of PRs) that made such an enhancement (because Apache HttpClient 5 seems to be the docker-java library's preferred transport these days, and especially if the PR adds support for Microsoft pipes - now Microsoft have finally started to support docker, it'd be nice to improve the docker-plugin's support for their docker variant too), but beware that it may be a non-trivial bit of work and it would be well outside the scope of a mere "bugfix".

          Jean-Sébastien added a comment - Pull request: https://github.com/jenkinsci/docker-plugin/pull/843

          pjdarton added a comment -

          Sitrep:

          • Switching transport layers is reported to improve stability.
          • ...but requires matching changes to both the docker-java-api-plugin and the docker-plugin.
          • Changes to the latter are blocked upon the former.

          pjdarton added a comment - Sitrep: Switching transport layers is reported to improve stability. ...but requires matching changes to both the docker-java-api-plugin and the docker-plugin. Changes to the latter are blocked upon the former.

            Unassigned Unassigned
            jsbevilacqua Jean-Sébastien
            Votes:
            1 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated: