Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-33727

Invoke Ant - Build failed if parameters contain spaces due to changes in quotes since 1.653

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Blocker Blocker
    • ant-plugin, core
    • Jenkins greater than 1.652

      IStarting from Jenkins 1.653 with the same plugins versions we noticed that the ant command is created differently and it does not work with our configuration.

      Add build step Invoke Ant with the following values:

      Ant Version: ANT_1.9.4
      Targets: -lib D:\ANT_1.9.4\ANT_libs Alpha.i
      Build File: buildInstallers.xml
      Properties:
      iaPath=${InstallAnywhereLocation}
      iaOutputDir=$WORKSPACE
      iaMergeModulesDir=

      buildView=$WORKSPACE
      buildLogsPath=$WORKSPACE
      installersPath=$WORKSPACE

      1.652 - Command output
      cmd.exe /C '"D:\ANT_1.9.4\bin\ant.bat -file buildInstallers.xml -DiaOutputDir=D:\JENKINS-WS\Alpha\workspace "-DiaPath=C:\Program Files (x86)\InstallAnywhere 2015" -DbuildLogsPath=D:\JENKINS-WS\Alpha\workspace -DiaMergeModulesDir="" -DbuildView=D:\JENKINS-WS\Alpha\workspace -DinstallersPath=D:\JENKINS-WS\Alpha\workspace -lib D:\ANT_1.9.4\ANT_libs Alpha.i && exit %%ERRORLEVEL%%"'

      1.653 and 1.654 - Command output
      cmd.exe /C "D:\ANT_1.9.4\bin\ant.bat -file buildInstallers.xml -DiaOutputDir=D:\JENKINS-WS\Alpha\workspace '"-DiaPath=C:\Program Files (x86)\InstallAnywhere 2015"' -DbuildLogsPath=D:\JENKINS-WS\Alpha\workspace -DiaMergeModulesDir= -DbuildView=D:\JENKINS-WS\Alpha\workspace -DinstallersPath=D:\JENKINS-WS\Alpha\workspace -lib D:\ANT_1.9.4\ANT_libs Alpha.i && exit %%ERRORLEVEL%%"

      BUILD FAILED
      Target "D:\JENKINS-WS\Alpha\workspace" does not exist in the project "TNMS Installers".

          [JENKINS-33727] Invoke Ant - Build failed if parameters contain spaces due to changes in quotes since 1.653

          Ben Apgar added a comment -

          We are getting this too.

          The problem is related to parameters that contain spaces.

          Previous behavior:

          • Full command enclosed by single quotes and doublequotes.
          • Parameters containing spaces enclosed by double-quotes.

          Example:

          '"ant.bat -DPARAM1=hello "-DPARAM2=hello world" -DPARAM3=world"'
          

          Current behavior:

          • Full command enclosed by double quotes.
          • Parameters containing spaces enclosed by single-quotes and double-quotes.
          "ant.bat -DPARAM1=hello '"-DPARAM2=hello world"' -DPARAM3=world"
          

          Ben Apgar added a comment - We are getting this too. The problem is related to parameters that contain spaces. Previous behavior: Full command enclosed by single quotes and doublequotes. Parameters containing spaces enclosed by double-quotes. Example: ' "ant.bat -DPARAM1=hello " -DPARAM2=hello world " -DPARAM3=world" ' Current behavior: Full command enclosed by double quotes. Parameters containing spaces enclosed by single-quotes and double-quotes. "ant.bat -DPARAM1=hello '" -DPARAM2=hello world "' -DPARAM3=world"

          Ben Apgar added a comment - - edited

          escoem I think this is related to the changes in ArgumentListBuilder for your fix for JENKINS-28790

          Ben Apgar added a comment - - edited escoem I think this is related to the changes in ArgumentListBuilder for your fix for JENKINS-28790

          Emilio Escobar added a comment - - edited

          Thanks for the mention benapgar I will check it asap

          Emilio Escobar added a comment - - edited Thanks for the mention benapgar I will check it asap

          Ant-plugin does a post-process so may be can fix there

          Emilio Escobar added a comment - Ant-plugin does a post-process so may be can fix there

          Daniel Beck added a comment -

          Besides fixing any obvious bugs, it could make sense to add an option to not convert build parameters into system properties. They can still be accessed from the environment, but this would cut down the crazy wrt. command line arguments.

          (I'm pretty sure Text Parameters, i.e. multiline strings, aren't handled well here at all either.)

          Daniel Beck added a comment - Besides fixing any obvious bugs, it could make sense to add an option to not convert build parameters into system properties. They can still be accessed from the environment, but this would cut down the crazy wrt. command line arguments. (I'm pretty sure Text Parameters, i.e. multiline strings, aren't handled well here at all either.)

          Christian Opitz added a comment - - edited

          Same issue here.

          Example (without changing the job configuration):
          This is the old version that worked:

          cmd.exe /C '"ant.bat -file postResultsToRTC.xml -DbuildResultUUID="" "-DrtcBuildDefinitionId=My_Test Jenkins build" -DrtcRepositoryAddress=https://repository-server.com/ccm "-DrtcBuildEngineId=My_Test Jenkins engine" "-DjenkinsJobDisplayName=#136 - SVN revision r2265" -DjenkinsJobBuildURL=http://myaddress.com:8080/job/Test/job/trunk-restructured-CBA/136/ -lib C:\IBM\TeamConcertBuild\buildsystem\buildtoolkit all && exit %%ERRORLEVEL%%"'
          

          This is the new output not working

          cmd.exe /C "ant.bat -file postResultsToRTC.xml -DbuildResultUUID= '"-DrtcBuildDefinitionId=My_Test Jenkins build"' -DrtcRepositoryAddress=https://repository-server.com/ccm '"-DrtcBuildEngineId=My_Test Jenkins engine"' '"-DjenkinsJobDisplayName=#137 - SVN revision r2285"' -DjenkinsJobBuildURL=http://myaddress.com:8080/job/Test/job/trunk-restructured-CBA/137/ -lib C:\IBM\TeamConcertBuild\buildsystem\buildtoolkit all && exit %%ERRORLEVEL%%"
          

          Christian Opitz added a comment - - edited Same issue here. Example (without changing the job configuration): This is the old version that worked: cmd.exe /C '"ant.bat -file postResultsToRTC.xml -DbuildResultUUID="" "-DrtcBuildDefinitionId=My_Test Jenkins build" -DrtcRepositoryAddress=https://repository-server.com/ccm "-DrtcBuildEngineId=My_Test Jenkins engine" "-DjenkinsJobDisplayName=#136 - SVN revision r2265" -DjenkinsJobBuildURL=http://myaddress.com:8080/job/Test/job/trunk-restructured-CBA/136/ -lib C:\IBM\TeamConcertBuild\buildsystem\buildtoolkit all && exit %%ERRORLEVEL%%"' This is the new output not working cmd.exe /C "ant.bat -file postResultsToRTC.xml -DbuildResultUUID= '"-DrtcBuildDefinitionId=My_Test Jenkins build"' -DrtcRepositoryAddress=https://repository-server.com/ccm '"-DrtcBuildEngineId=My_Test Jenkins engine"' '"-DjenkinsJobDisplayName=#137 - SVN revision r2285"' -DjenkinsJobBuildURL=http://myaddress.com:8080/job/Test/job/trunk-restructured-CBA/137/ -lib C:\IBM\TeamConcertBuild\buildsystem\buildtoolkit all && exit %%ERRORLEVEL%%"

          Sergej Kleva added a comment -

          Any workaround or solution?!

          We are stucked (

          Sergej Kleva added a comment - Any workaround or solution?! We are stucked (

          K P added a comment -

          @Sergej : a solution seems to be implemented since a month by now, but no-one seems to bother processing it so it can get released ( https://github.com/jenkinsci/ant-plugin/pull/14 is already waiting for more than two weeks ), even though this is blocking upgrade to jenkins 2.x and even seems to break the LTS releases.

          K P added a comment - @Sergej : a solution seems to be implemented since a month by now, but no-one seems to bother processing it so it can get released ( https://github.com/jenkinsci/ant-plugin/pull/14 is already waiting for more than two weeks ), even though this is blocking upgrade to jenkins 2.x and even seems to break the LTS releases.

          Daniel Beck added a comment -

          armfergom Please consider looking into merging and releasing the PR (assuming it looks good). As you can see above, users are getting restless

          (Everyone, please be kind to Armando, he took over maintainership of Ant plugin just a few days ago!)

          Daniel Beck added a comment - armfergom Please consider looking into merging and releasing the PR (assuming it looks good). As you can see above, users are getting restless (Everyone, please be kind to Armando, he took over maintainership of Ant plugin just a few days ago!)

          Sergej Kleva added a comment -

          Sure ... i am kind ... but how's that this was implemented month ago and haven't been merged into main branch?

          Hope for the best!

          Brgds,
          Sergej

          Sergej Kleva added a comment - Sure ... i am kind ... but how's that this was implemented month ago and haven't been merged into main branch? Hope for the best! Brgds, Sergej

          Hi,

          As Daniel mentioned, I just took over maintainership couple of days ago. I am looking at pending PRs, including this one. Will get a new version ready including a fix for this today.

          Cheers,
          Armando

          Armando Fernandez added a comment - Hi, As Daniel mentioned, I just took over maintainership couple of days ago. I am looking at pending PRs, including this one. Will get a new version ready including a fix for this today. Cheers, Armando

          Sergej Kleva added a comment -

          Thanks Armando and the rest of the team!

          Brgds,
          S.

          Sergej Kleva added a comment - Thanks Armando and the rest of the team! Brgds, S.

          Armando Fernandez added a comment - - edited

          Hi all,

          So, I have built the PR with the fix and am testing it at the moment. The thing is, I don't see any issues related to parameters with spaces (have not been able to reproduce it locally). I do see issues with parameters that are empty though. I have noticed that the examples that have been provided in the ticket use empty parameters as well, so I am wondering whether this is the real cause of the failures.

          The PR I am looking at solves the empty parameters issue.

          Could you please let me know what error message are you getting to double check that the issue is caused by the spaces in the parameter and not by the empty parameters?

          For example, in my case, I am getting this error message (that goes away when including the fix mentioned above):

          [workspace] $ cmd.exe /C "C:\apache-ant-1.9.7\bin\ant.bat '"-DcustomParameter=algo cno espacions"' -DotroParametro= && exit %%ERRORLEVEL%%"
          Missing value for property otroParametro
          Build step 'Invoke Ant' marked build as failure
          Finished: FAILURE

          Cheers,
          Armando

          Armando Fernandez added a comment - - edited Hi all, So, I have built the PR with the fix and am testing it at the moment. The thing is, I don't see any issues related to parameters with spaces (have not been able to reproduce it locally). I do see issues with parameters that are empty though. I have noticed that the examples that have been provided in the ticket use empty parameters as well, so I am wondering whether this is the real cause of the failures. The PR I am looking at solves the empty parameters issue. Could you please let me know what error message are you getting to double check that the issue is caused by the spaces in the parameter and not by the empty parameters? For example, in my case, I am getting this error message (that goes away when including the fix mentioned above): [workspace] $ cmd.exe /C "C:\apache-ant-1.9.7\bin\ant.bat '"-DcustomParameter=algo cno espacions"' -DotroParametro= && exit %%ERRORLEVEL%%" Missing value for property otroParametro Build step 'Invoke Ant' marked build as failure Finished: FAILURE Cheers, Armando

          Sergej Kleva added a comment -

          In jenkins we define global parameters variables which we pass into ant properties plugin.
          Sometimes those global parameters variables are empty ... for different reasons ...

          Sergej Kleva added a comment - In jenkins we define global parameters variables which we pass into ant properties plugin. Sometimes those global parameters variables are empty ... for different reasons ...

          sergejk So, in your case, I assume your problem comes from empty parameters. I experience the same issue and that is alright because there is already a fix for that (not released yet)

          I am wondering about parameters whose value is not empty and contain spaces which is being claimed in this ticket to cause problems. I cannot reproduce it, so would like some more info about the errors that people are getting, if possible.

          Armando Fernandez added a comment - sergejk So, in your case, I assume your problem comes from empty parameters. I experience the same issue and that is alright because there is already a fix for that (not released yet) I am wondering about parameters whose value is not empty and contain spaces which is being claimed in this ticket to cause problems. I cannot reproduce it, so would like some more info about the errors that people are getting, if possible.

          In my case (see above), I have the following scenario:

          1. Empty parameter buildResultUUID in job A (job is configured as parametrized job, value is left empty)
          2. I am using "Trigger parameterized build on other projects" in job A, the triggered job B is executing the ant script. In job A the following predefined parameters are set (for the parameterized build of job B):
          rtcRepositoryAddress=https://repository-server.com/ccm
          rtcBuildDefinitionId=My_Test Jenkins build
          rtcBuildEngineId=My_Test Jenkins engine
          jenkinsJobBuildURL=${BUILD_URL}
          jenkinsJobDisplayName=${BUILD_DISPLAY_NAME} - SVN revision r${SVN_REVISION}
          buildResultUUID=${buildResultUUID}
          

          Christian Opitz added a comment - In my case (see above), I have the following scenario: Empty parameter buildResultUUID in job A (job is configured as parametrized job, value is left empty) I am using "Trigger parameterized build on other projects" in job A, the triggered job B is executing the ant script. In job A the following predefined parameters are set (for the parameterized build of job B): rtcRepositoryAddress=https: //repository-server.com/ccm rtcBuildDefinitionId=My_Test Jenkins build rtcBuildEngineId=My_Test Jenkins engine jenkinsJobBuildURL=${BUILD_URL} jenkinsJobDisplayName=${BUILD_DISPLAY_NAME} - SVN revision r${SVN_REVISION} buildResultUUID=${buildResultUUID}

          chrop Could you paste here the error message you get in the build? Hopefully you will be able to see something like:

          Missing value for property {nameOfParameter}
          

          The thing is, the description and name of the issue suggests that the problem comes from spaces in the values of the parameters, which I suspect is not the case. Just want to be sure that everyone gets what they are expecting

          Armando Fernandez added a comment - chrop Could you paste here the error message you get in the build? Hopefully you will be able to see something like: Missing value for property {nameOfParameter} The thing is, the description and name of the issue suggests that the problem comes from spaces in the values of the parameters, which I suspect is not the case. Just want to be sure that everyone gets what they are expecting

          Ben Apgar added a comment - - edited

          armfergom

          The related issue with empty parameters is "JENKINS-33712 Empty parameterized build properties not quoted when passed to ant".

          This issue is slightly different. It is caused by spaces in the parameter.

          When there are spaces in a parameter value, the parameter name and value are being enclosed in single and double quotes. This breaks the command, because of another new behavior where the full command is no longer enclosed in single and double quotes--it is only enclosed in double quotes. This causes the parameters to be parsed incorrectly because the quotes are mismatched and the command fails.

          If there are no spaces in the parameter, there are no quotes at all in the name value pair which still works fine.

          Ben Apgar added a comment - - edited armfergom The related issue with empty parameters is " JENKINS-33712 Empty parameterized build properties not quoted when passed to ant". This issue is slightly different. It is caused by spaces in the parameter. When there are spaces in a parameter value, the parameter name and value are being enclosed in single and double quotes. This breaks the command, because of another new behavior where the full command is no longer enclosed in single and double quotes--it is only enclosed in double quotes. This causes the parameters to be parsed incorrectly because the quotes are mismatched and the command fails. If there are no spaces in the parameter, there are no quotes at all in the name value pair which still works fine.

          armfergom I don't get any error message at all, the build even passes (but I guess this is due to the setup of the ant file). What I see however, is that the existing arguments are not passed correctly to the ant file, so it is not executed as it was in previous versions.

          For example, the following line is contained in the ant file:

          <echo message="buildResultUUID: .${buildResultUUID}."/>
          

          The output is

          [echo] buildResultUUID: .-DrtcBuildDefinitionId=My_Test Jenkins build.
          

          which is obviously not correct as it takes the definition of the following variables as content of the first one

          Christian Opitz added a comment - armfergom I don't get any error message at all, the build even passes (but I guess this is due to the setup of the ant file). What I see however, is that the existing arguments are not passed correctly to the ant file, so it is not executed as it was in previous versions. For example, the following line is contained in the ant file: <echo message= "buildResultUUID: .${buildResultUUID}." /> The output is [echo] buildResultUUID: .-DrtcBuildDefinitionId=My_Test Jenkins build. which is obviously not correct as it takes the definition of the following variables as content of the first one

          Hi, the 1.3 version has been released. hugo_reis, can you try it, please?

          Emilio Escobar added a comment - Hi, the 1.3 version has been released. hugo_reis , can you try it, please?

          For me the 1.3 is working.

          The quotes are different to the initial version, i.e. all parameters with blanks are now quoted both with ' and ". But it seems to be plausible and is working now.

          cmd.exe /C "ant.bat -file postResultsToRTC.xml -DbuildResultUUID="" '"-DrtcBuildDefinitionId=My_Test Jenkins build"' -DrtcRepositoryAddress=https://repository-server.com/ccm '"-DrtcBuildEngineId=My_Test Jenkins engine"' '"-DjenkinsJobDisplayName=#211 - SVN revision r2437"' -DjenkinsJobBuildURL=http://myaddress.com:8080/job/Test/job/trunk-restructured-CBA/211/ -lib C:\IBM\TeamConcertBuild\buildsystem\buildtoolkit all && exit %%ERRORLEVEL%%"
          

          Thanks for the fix!

          Christian Opitz added a comment - For me the 1.3 is working. The quotes are different to the initial version, i.e. all parameters with blanks are now quoted both with ' and ". But it seems to be plausible and is working now. cmd.exe /C "ant.bat -file postResultsToRTC.xml -DbuildResultUUID=" " '" -DrtcBuildDefinitionId=My_Test Jenkins build "' -DrtcRepositoryAddress=https: //repository-server.com/ccm '" -DrtcBuildEngineId=My_Test Jenkins engine "' '" -DjenkinsJobDisplayName=#211 - SVN revision r2437 "' -DjenkinsJobBuildURL=http://myaddress.com:8080/job/Test/job/trunk-restructured-CBA/211/ -lib C:\IBM\TeamConcertBuild\buildsystem\buildtoolkit all && exit %%ERRORLEVEL%%" Thanks for the fix!

          Hugo Reis added a comment -

          Tested successfully:

          • Jenkins 1.652 with Ant plugin 1.3
          • Jenkins 2.3 with Ant plugin 1.3

          escoem you can close this issue.

          Thanks for the support.

          Hugo Reis added a comment - Tested successfully: Jenkins 1.652 with Ant plugin 1.3 Jenkins 2.3 with Ant plugin 1.3 escoem you can close this issue. Thanks for the support.

          Ugur Ceylan added a comment -

          Hi,

          With Jenkins 2.23 and ant-plugin 1.4 problem still occurs. BUILD_SELECTOR parameter is injected by the copy artifact plugin.

          cmd.exe /C "D:\work\tools\sfw\ant\bin\ant.bat -file build_jenkins.xml '"-DBUILD_SELECTOR=<StatusBuildSelector plugin=""copyartifact@1.38.1""> <stable>true</stable></StatusBuildSelector>"' jenkins.junit.runtests && exit %%ERRORLEVEL%%"
          17:26:09 -Xms256m -Xmx2048m
          17:26:09 Buildfile: build_jenkins.xml
          17:26:11
          17:26:11 BUILD FAILED
          17:26:11 Target `<stable>true</stable></StatusBuildSelector> jenkins.junit.runtests' does not exist in this project.

          Ugur Ceylan added a comment - Hi, With Jenkins 2.23 and ant-plugin 1.4 problem still occurs. BUILD_SELECTOR parameter is injected by the copy artifact plugin. cmd.exe /C "D:\work\tools\sfw\ant\bin\ant.bat -file build_jenkins.xml '"-DBUILD_SELECTOR=<StatusBuildSelector plugin=""copyartifact@1.38.1""> <stable>true</stable></StatusBuildSelector>"' jenkins.junit.runtests && exit %%ERRORLEVEL%%" 17:26:09 -Xms256m -Xmx2048m 17:26:09 Buildfile: build_jenkins.xml 17:26:11 17:26:11 BUILD FAILED 17:26:11 Target `<stable>true</stable></StatusBuildSelector> jenkins.junit.runtests' does not exist in this project.

          yucey I'd say it seems to be related to the two double quotes in ""copyartifact@1.38.1"", but would need to look deeper into it

          Armando Fernandez added a comment - yucey I'd say it seems to be related to the two double quotes in ""copyartifact@1.38.1"" , but would need to look deeper into it

            escoem Emilio Escobar
            hugo_reis Hugo Reis
            Votes:
            4 Vote for this issue
            Watchers:
            11 Start watching this issue

              Created:
              Updated:
              Resolved: