[JENKINS-65451] inconsistent behaviour when performing variable interpolation - Jenkins Jira

XML

Word

Printable

Details

Type: Bug
Status: Closed (View Workflow)
Priority: Major
Resolution: Not A Defect
Component/s: credentials-binding-plugin
Labels:
None
Environment:
2.277.3
credentials-binding-plugin 1.24

Similar Issues:

Show

Description

I have a groovy function, runAnsiblePlaybook which I use to run ansible playbooks in a docker container:

// Runs deployment playbook for prod/non-prod environments def call(docker_reg, jenkins_docker_creds_id, _docker_reg_url, ssh_key_creds, inventory, playbook, env_vars=[]) {     withDockerRegistry([ credentialsId: jenkins_docker_creds_id,                          url: _docker_reg_url]) {
         withCredentials([             sshUserPrivateKey(credentialsId: ssh_key_creds,                               usernameVariable: "ANSIBLE_USER",                               keyFileVariable: "KEY_PATH")
         ]) {
             script {
                 echo "env_vars: ${env_vars}"
                 env_string = ""
                 env_vars.each { entry -> env_string = env_string + "-e $entry.key=$entry.value " }
                 echo "env_string: ${env_string}"
                 sh """
                     # docker run here
                 """
             }
         }
     }
 }

This function is wrapped

// Attempts a PaaS deployment using Ansible
def lgtAnsibleDeployToPaaS(paas_deploy_creds_id, paas_host, paas_project,                            ansible_inventory, ansible_playbook, paas_env) {
     withCredentials([             usernamePassword(credentialsId: paas_deploy_creds_id,             usernameVariable: "PAAS_USER_UNUSED", passwordVariable: "PAAS_PASSWORD")
         ]) {
         lgtAnsiblePlaybook(ansible_inventory,
                            ansible_playbook,
                            [
                                helm_cli_version: "v3.3.4",
                                paas_host: paas_host,
                                paas_token: env.PAAS_PASSWORD,
                                paas_project: paas_project,
                                ts_release_env: paas_env                             ])
     }
 }

This function is run twice (in a good case), to deploy against ref and live environments.

                        lgtAnsibleDeployToPaaS(paas_deploy_creds_id, paas_api_url, paas_docker_repo,                                                "deployment/inventory_non_prod/hosts.yml",                                                "deployment/playbooks/deploy_ts_helm.yml",                                                "ref")

and

                        // Note deliberately using inventory_non_prod as that has helm installed.                         lgtAnsibleDeployToPaaS(paas_deploy_creds_id, paas_api_url, paas_docker_repo,                                                "deployment/inventory_non_prod/hosts.yml",                                                "deployment/playbooks/deploy_ts_helm.yml",                                                "live")

First time through it correctly interpolates the variables:

But then when it deploys to live I get:

Note: env_vars is identical for the two cases with the exception of the ts_release_env variable.

expansion of env_vars into env_string in the 'live' run is missing any params before paas_project and lacking the secret warning this time.

As these key variables are not in the expanded env_string, this subsequently leads to the docker run failure (not shown)

I've tried modifying the env_vars.each clause to use' ' rather than "":

 env_vars.each { entry -> env_string = env_string + '-e $entry.key=$entry.value ' }

but this does not perform the $entry expansion.

What am I missing?

Attachments

- Sort By Name
- Sort By Date
- Ascending
- Descending
- Thumbnails
- List
- Download All

image-2021-04-23-14-01-50-790.png
53 kB
2021-04-23 13:01
image-2021-04-23-14-04-09-468.png
37 kB
2021-04-23 13:04

Activity

Ascending order - Click to sort in descending order

Donald created issue - 2021-04-23 13:16

Donald made changes - 2021-04-23 13:20

Field	Original Value	New Value
Description	I have a groovy function, runAnsiblePlaybook which I use to run ansible playbooks in a docker container: {code:java} // Runs deployment playbook for prod/non-prod environments def call(docker_reg, jenkins_docker_creds_id, _docker_reg_url, ssh_key_creds, inventory, playbook, env_vars=[]) { withDockerRegistry([ credentialsId: jenkins_docker_creds_id, url: _docker_reg_url]) { withCredentials([ sshUserPrivateKey(credentialsId: ssh_key_creds, usernameVariable: "ANSIBLE_USER", keyFileVariable: "KEY_PATH") ]) { script { echo "env_vars: ${env_vars}" env_string = "" env_vars.each { entry -> env_string = env_string + "-e $entry.key=$entry.value " } echo "env_string: ${env_string}" sh """ # docker run here """ } } } } {code} This function is wrapped {code:java} // Attempts a PaaS deployment using Ansible def lgtAnsibleDeployToPaaS(paas_deploy_creds_id, paas_host, paas_project, ansible_inventory, ansible_playbook, paas_env) { withCredentials([ usernamePassword(credentialsId: paas_deploy_creds_id, usernameVariable: "PAAS_USER_UNUSED", passwordVariable: "PAAS_PASSWORD") ]) { lgtAnsiblePlaybook(ansible_inventory, ansible_playbook, [ helm_cli_version: "v3.3.4", paas_host: paas_host, paas_token: env.PAAS_PASSWORD, paas_project: paas_project, ts_release_env: paas_env ]) } }{code} This function is run twice (in a good case), to deploy against ref and live environments. {code:java} lgtAnsibleDeployToPaaS(paas_deploy_creds_id, paas_api_url, paas_docker_repo, "deployment/inventory_non_prod/hosts.yml", "deployment/playbooks/deploy_ts_helm.yml", "ref") {code} and {code:java} // Note deliberately using inventory_non_prod as that has helm installed. lgtAnsibleDeployToPaaS(paas_deploy_creds_id, paas_api_url, paas_docker_repo, "deployment/inventory_non_prod/hosts.yml", "deployment/playbooks/deploy_ts_helm.yml", "live"){code} First time through it correctly interpolates the variables: !image-2021-04-23-14-01-50-790.png! But then when it deploys to live I get: !image-2021-04-23-14-04-09-468.png! Note that the expansion of env_vars into env_string is incomplete and lacking the secret warning this time. As these key variables are not passed, this subsequently leads to the docker run failure I've tried modifying the env_vars.each clause to use:	I have a groovy function, runAnsiblePlaybook which I use to run ansible playbooks in a docker container: {code:java} // Runs deployment playbook for prod/non-prod environments def call(docker_reg, jenkins_docker_creds_id, _docker_reg_url, ssh_key_creds, inventory, playbook, env_vars=[]) { withDockerRegistry([ credentialsId: jenkins_docker_creds_id, url: _docker_reg_url]) { withCredentials([ sshUserPrivateKey(credentialsId: ssh_key_creds, usernameVariable: "ANSIBLE_USER", keyFileVariable: "KEY_PATH") ]) { script { echo "env_vars: ${env_vars}" env_string = "" env_vars.each { entry -> env_string = env_string + "-e $entry.key=$entry.value " } echo "env_string: ${env_string}" sh """ # docker run here """ } } } } {code} This function is wrapped {code:java} // Attempts a PaaS deployment using Ansible def lgtAnsibleDeployToPaaS(paas_deploy_creds_id, paas_host, paas_project, ansible_inventory, ansible_playbook, paas_env) { withCredentials([ usernamePassword(credentialsId: paas_deploy_creds_id, usernameVariable: "PAAS_USER_UNUSED", passwordVariable: "PAAS_PASSWORD") ]) { lgtAnsiblePlaybook(ansible_inventory, ansible_playbook, [ helm_cli_version: "v3.3.4", paas_host: paas_host, paas_token: env.PAAS_PASSWORD, paas_project: paas_project, ts_release_env: paas_env ]) } }{code} This function is run twice (in a good case), to deploy against ref and live environments. {code:java} lgtAnsibleDeployToPaaS(paas_deploy_creds_id, paas_api_url, paas_docker_repo, "deployment/inventory_non_prod/hosts.yml", "deployment/playbooks/deploy_ts_helm.yml", "ref") {code} and {code:java} // Note deliberately using inventory_non_prod as that has helm installed. lgtAnsibleDeployToPaaS(paas_deploy_creds_id, paas_api_url, paas_docker_repo, "deployment/inventory_non_prod/hosts.yml", "deployment/playbooks/deploy_ts_helm.yml", "live"){code} First time through it correctly interpolates the variables: !image-2021-04-23-14-01-50-790.png! But then when it deploys to live I get: !image-2021-04-23-14-04-09-468.png! Note: env_vars is identical for the two cases with the exception of the ts_release_env variable. expansion of env_vars into env_string in the 'live' run is missing any params before paas_project and lacking the secret warning this time. As these key variables are not in the expanded env_string, this subsequently leads to the docker run failure (not shown) I've tried modifying the env_vars.each clause to use' ' rather than "": {code:java} env_vars.each { entry -> env_string = env_string + '-e $entry.key=$entry.value ' }{code} but this does not perform the $entry expansion. What am I missing?

Donald made changes - 2021-04-26 08:20

Description

I have a groovy function, runAnsiblePlaybook which I use to run ansible playbooks in a docker container:

{code:java}
// Runs deployment playbook for prod/non-prod environments def call(docker_reg, jenkins_docker_creds_id, _docker_reg_url, ssh_key_creds, inventory, playbook, env_vars=[]) {     withDockerRegistry([ credentialsId: jenkins_docker_creds_id,                          url: _docker_reg_url]) {
         withCredentials([             sshUserPrivateKey(credentialsId: ssh_key_creds,                               usernameVariable: "ANSIBLE_USER",                               keyFileVariable: "KEY_PATH")
         ]) {
             script {
                 echo "env_vars: ${env_vars}"
                 env_string = ""
                 env_vars.each { entry -> env_string = env_string + "-e $entry.key=$entry.value " }
                 echo "env_string: ${env_string}"
                 sh """
                     # docker run here
                 """
             }
         }
     }
} {code}

This function is wrapped

{code:java}
// Attempts a PaaS deployment using Ansible def lgtAnsibleDeployToPaaS(paas_deploy_creds_id, paas_host, paas_project,                            ansible_inventory, ansible_playbook, paas_env) {     withCredentials([             usernamePassword(credentialsId: paas_deploy_creds_id,             usernameVariable: "PAAS_USER_UNUSED", passwordVariable: "PAAS_PASSWORD")         ]) {         lgtAnsiblePlaybook(ansible_inventory,                            ansible_playbook,                             [                                 helm_cli_version: "v3.3.4",                                 paas_host: paas_host,                                 paas_token: env.PAAS_PASSWORD,                                 paas_project: paas_project,                                 ts_release_env: paas_env                             ])     } }{code}

This function is run twice (in a good case), to deploy against ref and live environments.

{code:java}
                        lgtAnsibleDeployToPaaS(paas_deploy_creds_id, paas_api_url, paas_docker_repo,                                                "deployment/inventory_non_prod/hosts.yml",                                                "deployment/playbooks/deploy_ts_helm.yml",                                                "ref") {code}
and

{code:java}
                        // Note deliberately using inventory_non_prod as that has helm installed.                         lgtAnsibleDeployToPaaS(paas_deploy_creds_id, paas_api_url, paas_docker_repo,                                                "deployment/inventory_non_prod/hosts.yml",                                                "deployment/playbooks/deploy_ts_helm.yml",                                                "live"){code}


First time through it correctly interpolates the variables:
!image-2021-04-23-14-01-50-790.png!

But then when it deploys to live I get:
!image-2021-04-23-14-04-09-468.png!
Note: env_vars is identical for the two cases with the exception of the ts_release_env variable.

expansion of env_vars into env_string in the 'live' run is missing any params before paas_project and lacking the secret warning this time.

As these key variables are not in the expanded env_string, this subsequently leads to the docker run failure (not shown)

I've tried modifying the env_vars.each clause to use' ' rather than "":
{code:java}
env_vars.each { entry -> env_string = env_string + '-e $entry.key=$entry.value ' }{code}
but this does not perform the $entry expansion.

What am I missing?

I have a groovy function, runAnsiblePlaybook which I use to run ansible playbooks in a docker container:

{code:java}
// Runs deployment playbook for prod/non-prod environments def call(docker_reg, jenkins_docker_creds_id, _docker_reg_url, ssh_key_creds, inventory, playbook, env_vars=[]) {     withDockerRegistry([ credentialsId: jenkins_docker_creds_id,                          url: _docker_reg_url]) {
         withCredentials([             sshUserPrivateKey(credentialsId: ssh_key_creds,                               usernameVariable: "ANSIBLE_USER",                               keyFileVariable: "KEY_PATH")
         ]) {
             script {
                 echo "env_vars: ${env_vars}"
                 env_string = ""
                 env_vars.each { entry -> env_string = env_string + "-e $entry.key=$entry.value " }
                 echo "env_string: ${env_string}"
                 sh """
                     # docker run here
                 """
             }
         }
     }
} {code}

This function is wrapped

{code:java}
// Attempts a PaaS deployment using Ansible
def lgtAnsibleDeployToPaaS(paas_deploy_creds_id, paas_host, paas_project,                            ansible_inventory, ansible_playbook, paas_env) {
     withCredentials([             usernamePassword(credentialsId: paas_deploy_creds_id,             usernameVariable: "PAAS_USER_UNUSED", passwordVariable: "PAAS_PASSWORD")         ]) {         lgtAnsiblePlaybook(ansible_inventory,
                            ansible_playbook,
                             [
                                 helm_cli_version: "v3.3.4",
                                 paas_host: paas_host,
                                 paas_token: env.PAAS_PASSWORD,
                                 paas_project: paas_project,
                                 ts_release_env: paas_env                             ])
     }
}{code}

This function is run twice (in a good case), to deploy against ref and live environments.

{code:java}
                        lgtAnsibleDeployToPaaS(paas_deploy_creds_id, paas_api_url, paas_docker_repo,                                                "deployment/inventory_non_prod/hosts.yml",                                                "deployment/playbooks/deploy_ts_helm.yml",                                                "ref") {code}
and

{code:java}
                        // Note deliberately using inventory_non_prod as that has helm installed.                         lgtAnsibleDeployToPaaS(paas_deploy_creds_id, paas_api_url, paas_docker_repo,                                                "deployment/inventory_non_prod/hosts.yml",                                                "deployment/playbooks/deploy_ts_helm.yml",                                                "live"){code}


First time through it correctly interpolates the variables:
!image-2021-04-23-14-01-50-790.png!

But then when it deploys to live I get:
!image-2021-04-23-14-04-09-468.png!
Note: env_vars is identical for the two cases with the exception of the ts_release_env variable.

expansion of env_vars into env_string in the 'live' run is missing any params before paas_project and lacking the secret warning this time.

As these key variables are not in the expanded env_string, this subsequently leads to the docker run failure (not shown)

I've tried modifying the env_vars.each clause to use' ' rather than "":
{code:java}
env_vars.each { entry -> env_string = env_string + '-e $entry.key=$entry.value ' }{code}
but this does not perform the $entry expansion.

What am I missing?

Donald made changes - 2021-04-26 08:20

Description

I have a groovy function, runAnsiblePlaybook which I use to run ansible playbooks in a docker container:

{code:java}
// Runs deployment playbook for prod/non-prod environments def call(docker_reg, jenkins_docker_creds_id, _docker_reg_url, ssh_key_creds, inventory, playbook, env_vars=[]) {     withDockerRegistry([ credentialsId: jenkins_docker_creds_id,                          url: _docker_reg_url]) {
         withCredentials([             sshUserPrivateKey(credentialsId: ssh_key_creds,                               usernameVariable: "ANSIBLE_USER",                               keyFileVariable: "KEY_PATH")
         ]) {
             script {
                 echo "env_vars: ${env_vars}"
                 env_string = ""
                 env_vars.each { entry -> env_string = env_string + "-e $entry.key=$entry.value " }
                 echo "env_string: ${env_string}"
                 sh """
                     # docker run here
                 """
             }
         }
     }
} {code}

This function is wrapped

{code:java}
// Attempts a PaaS deployment using Ansible
def lgtAnsibleDeployToPaaS(paas_deploy_creds_id, paas_host, paas_project,                            ansible_inventory, ansible_playbook, paas_env) {
     withCredentials([             usernamePassword(credentialsId: paas_deploy_creds_id,             usernameVariable: "PAAS_USER_UNUSED", passwordVariable: "PAAS_PASSWORD")         ]) {         lgtAnsiblePlaybook(ansible_inventory,
                            ansible_playbook,
                             [
                                 helm_cli_version: "v3.3.4",
                                 paas_host: paas_host,
                                 paas_token: env.PAAS_PASSWORD,
                                 paas_project: paas_project,
                                 ts_release_env: paas_env                             ])
     }
}{code}

This function is run twice (in a good case), to deploy against ref and live environments.

{code:java}
                        lgtAnsibleDeployToPaaS(paas_deploy_creds_id, paas_api_url, paas_docker_repo,                                                "deployment/inventory_non_prod/hosts.yml",                                                "deployment/playbooks/deploy_ts_helm.yml",                                                "ref") {code}
and

{code:java}
                        // Note deliberately using inventory_non_prod as that has helm installed.                         lgtAnsibleDeployToPaaS(paas_deploy_creds_id, paas_api_url, paas_docker_repo,                                                "deployment/inventory_non_prod/hosts.yml",                                                "deployment/playbooks/deploy_ts_helm.yml",                                                "live"){code}


First time through it correctly interpolates the variables:
!image-2021-04-23-14-01-50-790.png!

But then when it deploys to live I get:
!image-2021-04-23-14-04-09-468.png!
Note: env_vars is identical for the two cases with the exception of the ts_release_env variable.

expansion of env_vars into env_string in the 'live' run is missing any params before paas_project and lacking the secret warning this time.

As these key variables are not in the expanded env_string, this subsequently leads to the docker run failure (not shown)

I've tried modifying the env_vars.each clause to use' ' rather than "":
{code:java}
env_vars.each { entry -> env_string = env_string + '-e $entry.key=$entry.value ' }{code}
but this does not perform the $entry expansion.

What am I missing?

I have a groovy function, runAnsiblePlaybook which I use to run ansible playbooks in a docker container:

{code:java}
// Runs deployment playbook for prod/non-prod environments def call(docker_reg, jenkins_docker_creds_id, _docker_reg_url, ssh_key_creds, inventory, playbook, env_vars=[]) {     withDockerRegistry([ credentialsId: jenkins_docker_creds_id,                          url: _docker_reg_url]) {
         withCredentials([             sshUserPrivateKey(credentialsId: ssh_key_creds,                               usernameVariable: "ANSIBLE_USER",                               keyFileVariable: "KEY_PATH")
         ]) {
             script {
                 echo "env_vars: ${env_vars}"
                 env_string = ""
                 env_vars.each { entry -> env_string = env_string + "-e $entry.key=$entry.value " }
                 echo "env_string: ${env_string}"
                 sh """
                     # docker run here
                 """
             }
         }
     }
} {code}

This function is wrapped

{code:java}
// Attempts a PaaS deployment using Ansible
def lgtAnsibleDeployToPaaS(paas_deploy_creds_id, paas_host, paas_project,                            ansible_inventory, ansible_playbook, paas_env) {
     withCredentials([             usernamePassword(credentialsId: paas_deploy_creds_id,             usernameVariable: "PAAS_USER_UNUSED", passwordVariable: "PAAS_PASSWORD")
         ]) {
         lgtAnsiblePlaybook(ansible_inventory,
                            ansible_playbook,
                            [
                                helm_cli_version: "v3.3.4",
                                paas_host: paas_host,
                                paas_token: env.PAAS_PASSWORD,
                                paas_project: paas_project,
                                ts_release_env: paas_env                             ])
     }
}{code}

This function is run twice (in a good case), to deploy against ref and live environments.

{code:java}
                        lgtAnsibleDeployToPaaS(paas_deploy_creds_id, paas_api_url, paas_docker_repo,                                                "deployment/inventory_non_prod/hosts.yml",                                                "deployment/playbooks/deploy_ts_helm.yml",                                                "ref") {code}
and

{code:java}
                        // Note deliberately using inventory_non_prod as that has helm installed.                         lgtAnsibleDeployToPaaS(paas_deploy_creds_id, paas_api_url, paas_docker_repo,                                                "deployment/inventory_non_prod/hosts.yml",                                                "deployment/playbooks/deploy_ts_helm.yml",                                                "live"){code}


First time through it correctly interpolates the variables:
!image-2021-04-23-14-01-50-790.png!

But then when it deploys to live I get:
!image-2021-04-23-14-04-09-468.png!
Note: env_vars is identical for the two cases with the exception of the ts_release_env variable.

expansion of env_vars into env_string in the 'live' run is missing any params before paas_project and lacking the secret warning this time.

As these key variables are not in the expanded env_string, this subsequently leads to the docker run failure (not shown)

I've tried modifying the env_vars.each clause to use' ' rather than "":
{code:java}
env_vars.each { entry -> env_string = env_string + '-e $entry.key=$entry.value ' }{code}
but this does not perform the $entry expansion.

What am I missing?

Donald added a comment - 2021-04-26 15:12

Problem appears to be with my function.

the each function was incorrect, as it appeared to be incorrect for a map.

Replaced with the formal map.each approach.

Donald added a comment - 2021-04-26 15:12 Problem appears to be with my function. the each function was incorrect, as it appeared to be incorrect for a map. Replaced with the formal map.each approach.

Donald made changes - 2021-04-26 15:12

Resolution		Not A Defect [ 7 ]
Status	Open [ 1 ]	Closed [ 6 ]

People

Assignee:: Unassigned

Reporter:: Donald

Votes:: 0 Vote for this issue

Watchers:: 1 Start watching this issue

Dates

Created:: 2021-04-23 13:16

Updated:: 2021-04-26 15:12

Resolved:: 2021-04-26 15:12