Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-30600

git isn't run inside build container

    XMLWordPrintable

Details

    Description

      Using docker-slaves, the checkout build phase is ran with commands inside the remoting container (which by change has git) not inside the configured build container.

      This is due to git-client to create it's own LocalLauncher vs using the Node's one.

      Attachments

        Issue Links

          Activity

            git-client plugin do not rely on Launcher/FilePath abstraction but sends the whole GitClient implementation class to remote, making it just impossible to rely on a Launcher from caller. Would need to fully refactor GitClient implementation to run on master.

            ndeloof Nicolas De Loof added a comment - git-client plugin do not rely on Launcher/FilePath abstraction but sends the whole GitClient implementation class to remote, making it just impossible to rely on a Launcher from caller. Would need to fully refactor GitClient implementation to run on master.
            ndeloof Nicolas De Loof added a comment - see https://github.com/jenkinsci/git-client-plugin/pull/211
            jglick Jesse Glick added a comment -

            Affects docker-workflow as well from what I can tell: if Jenkins is configured with only a CLI Git tool (the default), and an agent lacks a Git executable,

            docker.image('cloudbees/java-build-tools').inside {git '…'}
            

            will fail on the git command with an error indicating that the command was not found—because it is being run in the agent’s namespace, not the container’s.

            jglick Jesse Glick added a comment - Affects docker-workflow as well from what I can tell: if Jenkins is configured with only a CLI Git tool (the default), and an agent lacks a Git executable, docker.image( 'cloudbees/java-build-tools' ).inside {git '…' } will fail on the git command with an error indicating that the command was not found—because it is being run in the agent’s namespace, not the container’s.
            jglick Jesse Glick added a comment -

            At least adding a warning in git #811.

            jglick Jesse Glick added a comment - At least adding a warning in git #811.
            markewaite Mark Waite added a comment -

            Warning message released in git plugin 4.1.0, Jan 18, 2020. Does not resolve the bug, but provides a warning about the bug.

            markewaite Mark Waite added a comment - Warning message released in git plugin 4.1.0, Jan 18, 2020. Does not resolve the bug, but provides a warning about the bug.
            vladichko Vlad Aginsky added a comment -

            I think it is not a minor issue, since it can be a blocket to use git plugin in q8s enviroment. Raising prio.

            vladichko Vlad Aginsky added a comment - I think it is not a minor issue, since it can be a blocket to use git plugin in q8s enviroment. Raising prio.

            Hello,

            are there any updates with this issue? In my company we're trying to setup Jenkins running on k8s, to launch build in dedicated Docker images...we run into this bug, so I cannot run git (neither tool like repo) in container namespace, but only in the agent one...do you have any workaround for this? It seems that PR on git plugin to fix this issue are not being maintained...

            Thank you,

            Alessio

            spinus1 Alessio Moscatello added a comment - Hello, are there any updates with this issue? In my company we're trying to setup Jenkins running on k8s, to launch build in dedicated Docker images...we run into this bug, so I cannot run git (neither tool like repo) in container namespace, but only in the agent one...do you have any workaround for this? It seems that PR on git plugin to fix this issue are not being maintained... Thank you, Alessio
            markewaite Mark Waite added a comment - - edited

            spinus1, I believe that] the withCredentials step and the ssh-agent step can be used to perform the checkout instead of using the checkout scm step. Those give you full control of command line git arguments.

            markewaite Mark Waite added a comment - - edited spinus1 , I believe that] the withCredentials step and the ssh-agent step can be used to perform the checkout instead of using the checkout scm step. Those give you full control of command line git arguments.
            jglick Jesse Glick added a comment -

            …but you then lose behaviors of checkout scm for multibranch projects (pinning an exact commit, merging with base branch); for non-multibranch projects, you lose the changelog & polling integrations. Pick your poison.

            Also see JENKINS-28335.

            jglick Jesse Glick added a comment - …but you then lose behaviors of checkout scm for multibranch projects (pinning an exact commit, merging with base branch); for non-multibranch projects, you lose the changelog & polling integrations. Pick your poison. Also see JENKINS-28335 .

            Hello Mark Waite, it could work, but I'll loose functionalities like repository polling...

            What if I just checkout code from agent name space? I think it's just a matt
            er of adding repo to JNPL container that is used to run Jenkins agent...

            spinus1 Alessio Moscatello added a comment - Hello Mark Waite, it could work, but I'll loose functionalities like repository polling... What if I just checkout code from agent name space? I think it's just a matt er of adding repo to JNPL container that is used to run Jenkins agent...

            This also affects K8N plugin when you use pod+container: 

            podTemplate(cloud: 'kubernetes', containers: [containerTemplate(args: '-c', command: '/bin/bash ', image: 'image with git ca-cert', name: 'base', workingDir: '/home/jenkins')], label: 'base') {
              node('base'){
                container('base')

            {       git credentialsId: 'something', url: '[https://some-ca-cert-git|https://some/].git' }

            }

            cvakiitho Tomas Hartmann added a comment - This also affects K8N plugin when you use pod+container:  podTemplate(cloud: 'kubernetes', containers: [containerTemplate(args: '-c', command: '/bin/bash ', image: 'image with git ca-cert', name: 'base', workingDir: '/home/jenkins')] , label: 'base') {   node('base'){     container('base') {       git credentialsId: 'something', url: '[https://some-ca-cert-git|https://some/].git' } }
            faroguy0 Ian Panzer added a comment -

            Any update on this issue? Encountering it in a k8s environment on AWS.

            faroguy0 Ian Panzer added a comment - Any update on this issue? Encountering it in a k8s environment on AWS.
            markewaite Mark Waite added a comment -

            No change faroguy0

            markewaite Mark Waite added a comment - No change faroguy0

            Are there any plans to actually fix this?

            piotr_niebylski Piotr Niebylski added a comment - Are there any plans to actually fix this?
            markewaite Mark Waite added a comment -

            There have been 3 attempts to fix it. All 3 attempts failed because they were unable to retain compatibility with existing installations and existing code. Additional attempts are certainly welcomed.

            I don't plan to attempt a fix in the next 12 months.

            markewaite Mark Waite added a comment - There have been 3 attempts to fix it. All 3 attempts failed because they were unable to retain compatibility with existing installations and existing code. Additional attempts are certainly welcomed. I don't plan to attempt a fix in the next 12 months.
            dcserrot Dani T added a comment - - edited

            Does anyone know any workaround? In my case I want to take advantage of the credentials used in the host to perform a git push.

            Regards!

            dcserrot Dani T added a comment - - edited Does anyone know any workaround? In my case I want to take advantage of the credentials used in the host to perform a git push. Regards!
            markewaite Mark Waite added a comment -

            Use a Pipeline, wrap your own custom call to `git clone` or `git fetch` with a `withCredentials` wrapper as described in the git plugin documentation of the `withCredentials block

            markewaite Mark Waite added a comment - Use a Pipeline, wrap your own custom call to `git clone` or `git fetch` with a `withCredentials` wrapper as described in the git plugin documentation of the `withCredentials block
            sdarwin Sam Darwin added a comment -

            Hi, I just saw an error in a jenkins job:

            "07:50:43 Warning: JENKINS-30600: special launcher com.cloudbees.jenkins.plugins.docker_build_env.DockerDecoratedLauncher@5ed555df; decorates RemoteLauncher[hudson.remoting.Channel@1c49281:EC2 (__) - jenkinspool1 (i-0b2aadb0c68d3dc94)] will be ignored (a typical symptom is the Git executable not being run inside a designated container)

            Could someone explain what this means, in simple terms?    What bug is happening to the job?    The explanation in the above description isn't clear.

             

            sdarwin Sam Darwin added a comment - Hi, I just saw an error in a jenkins job: "07:50:43 Warning: JENKINS-30600 : special launcher com.cloudbees.jenkins.plugins.docker_build_env.DockerDecoratedLauncher@5ed555df; decorates RemoteLauncher [hudson.remoting.Channel@1c49281:EC2 (__) - jenkinspool1 (i-0b2aadb0c68d3dc94)] will be ignored (a typical symptom is the Git executable not being run inside a designated container) Could someone explain what this means, in simple terms?    What bug is happening to the job?    The explanation in the above description isn't clear.  
            duemir Denys Digtiar added a comment -

            sdarwin I cannot find which plugin that launcher decorator comes from, so I am unsure what it is doing. If it is just modifying the environment, there might be no issue.

            The problem is most acute when the container wrapper from the Kubernetes plugin or Docker plugins' inside container feature is used. Both use Launcher decorator to change where external processes are executed, but the Git plugin ignores it. For example, you think that by wrapping git step in the container wrapper, you would execute git CLI in the specified container. Still, it is not the case, git CLI is always executed in the same container where the Jenkins agent process is running (jnlp container in case of the Kubernetes plugin).

            duemir Denys Digtiar added a comment - sdarwin I cannot find which plugin that launcher decorator comes from, so I am unsure what it is doing. If it is just modifying the environment, there might be no issue. The problem is most acute when the container wrapper from the Kubernetes plugin or Docker plugins' inside container feature is used. Both use Launcher decorator to change where external processes are executed, but the Git plugin ignores it. For example, you think that by wrapping git step in the container wrapper, you would execute git CLI in the specified container. Still, it is not the case, git CLI is always executed in the same container where the Jenkins agent process is running ( jnlp container in case of the Kubernetes plugin).
            ysmaoui Yacine added a comment - - edited

            is this also affecting sshagent? ( so not only checkout scm ) ?

            so the ssh keys in the following snippet are not setup correctly inside the container block in the following example

            podTemplate(...) {
                node("<myLabel>"){
                    container("<myContainer>"){
            
                        sshagent([credentialsId]){
                            sh("do something that requires the ssh auth")
                        }
                    }
                }  
            }
            

            UPDATE:

            No issue with the sshagent step: the ssh-agent command is indeed executed in the correct container

            but the knownhosts file was not updated:
            the checkout scm step, updates automatically the knownhosts file
            so if we run in the same container where the checkout scm was executed, sshagent works without problems.

            but if we switch containers and execute on a different container than 'jnlp' ( reminder of the main topic in this ticket: checkout scm is always executed in the container named 'jnlp' )
            then in the new container the knowshosts file needs to be updated manually...

            ysmaoui Yacine added a comment - - edited is this also affecting sshagent? ( so not only checkout scm ) ? so the ssh keys in the following snippet are not setup correctly inside the container block in the following example podTemplate(...) { node( "<myLabel>" ){ container( "<myContainer>" ){ sshagent([credentialsId]){ sh( " do something that requires the ssh auth" ) } } } } UPDATE: No issue with the sshagent step: the ssh-agent command is indeed executed in the correct container but the knownhosts file was not updated: the checkout scm step, updates automatically the knownhosts file so if we run in the same container where the checkout scm was executed, sshagent works without problems. but if we switch containers and execute on a different container than 'jnlp' ( reminder of the main topic in this ticket: checkout scm is always executed in the container named 'jnlp' ) then in the new container the knowshosts file needs to be updated manually...

            People

              Unassigned Unassigned
              ndeloof Nicolas De Loof
              Votes:
              23 Vote for this issue
              Watchers:
              36 Start watching this issue

              Dates

                Created:
                Updated: