Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-33727

Invoke Ant - Build failed if parameters contain spaces due to changes in quotes since 1.653

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      IStarting from Jenkins 1.653 with the same plugins versions we noticed that the ant command is created differently and it does not work with our configuration.

      Add build step Invoke Ant with the following values:

      Ant Version: ANT_1.9.4
      Targets: -lib D:\ANT_1.9.4\ANT_libs Alpha.i
      Build File: buildInstallers.xml
      Properties:
      iaPath=${InstallAnywhereLocation}
      iaOutputDir=$WORKSPACE
      iaMergeModulesDir=

      buildView=$WORKSPACE
      buildLogsPath=$WORKSPACE
      installersPath=$WORKSPACE

      1.652 - Command output
      cmd.exe /C '"D:\ANT_1.9.4\bin\ant.bat -file buildInstallers.xml -DiaOutputDir=D:\JENKINS-WS\Alpha\workspace "-DiaPath=C:\Program Files (x86)\InstallAnywhere 2015" -DbuildLogsPath=D:\JENKINS-WS\Alpha\workspace -DiaMergeModulesDir="" -DbuildView=D:\JENKINS-WS\Alpha\workspace -DinstallersPath=D:\JENKINS-WS\Alpha\workspace -lib D:\ANT_1.9.4\ANT_libs Alpha.i && exit %%ERRORLEVEL%%"'

      1.653 and 1.654 - Command output
      cmd.exe /C "D:\ANT_1.9.4\bin\ant.bat -file buildInstallers.xml -DiaOutputDir=D:\JENKINS-WS\Alpha\workspace '"-DiaPath=C:\Program Files (x86)\InstallAnywhere 2015"' -DbuildLogsPath=D:\JENKINS-WS\Alpha\workspace -DiaMergeModulesDir= -DbuildView=D:\JENKINS-WS\Alpha\workspace -DinstallersPath=D:\JENKINS-WS\Alpha\workspace -lib D:\ANT_1.9.4\ANT_libs Alpha.i && exit %%ERRORLEVEL%%"

      BUILD FAILED
      Target "D:\JENKINS-WS\Alpha\workspace" does not exist in the project "TNMS Installers".

        Attachments

          Issue Links

            Activity

            Hide
            benapgar Ben Apgar added a comment -

            We are getting this too.

            The problem is related to parameters that contain spaces.

            Previous behavior:

            • Full command enclosed by single quotes and doublequotes.
            • Parameters containing spaces enclosed by double-quotes.

            Example:

            '"ant.bat -DPARAM1=hello "-DPARAM2=hello world" -DPARAM3=world"'
            

            Current behavior:

            • Full command enclosed by double quotes.
            • Parameters containing spaces enclosed by single-quotes and double-quotes.
            "ant.bat -DPARAM1=hello '"-DPARAM2=hello world"' -DPARAM3=world"
            
            Show
            benapgar Ben Apgar added a comment - We are getting this too. The problem is related to parameters that contain spaces. Previous behavior: Full command enclosed by single quotes and doublequotes. Parameters containing spaces enclosed by double-quotes. Example: ' "ant.bat -DPARAM1=hello " -DPARAM2=hello world " -DPARAM3=world" ' Current behavior: Full command enclosed by double quotes. Parameters containing spaces enclosed by single-quotes and double-quotes. "ant.bat -DPARAM1=hello '" -DPARAM2=hello world "' -DPARAM3=world"
            Hide
            benapgar Ben Apgar added a comment - - edited

            Emilio Escobar I think this is related to the changes in ArgumentListBuilder for your fix for JENKINS-28790

            Show
            benapgar Ben Apgar added a comment - - edited Emilio Escobar I think this is related to the changes in ArgumentListBuilder for your fix for JENKINS-28790
            Hide
            escoem Emilio Escobar added a comment - - edited

            Thanks for the mention Ben Apgar I will check it asap

            Show
            escoem Emilio Escobar added a comment - - edited Thanks for the mention Ben Apgar I will check it asap
            Hide
            escoem Emilio Escobar added a comment -

            Ant-plugin does a post-process so may be can fix there

            Show
            escoem Emilio Escobar added a comment - Ant-plugin does a post-process so may be can fix there
            Hide
            danielbeck Daniel Beck added a comment -

            Besides fixing any obvious bugs, it could make sense to add an option to not convert build parameters into system properties. They can still be accessed from the environment, but this would cut down the crazy wrt. command line arguments.

            (I'm pretty sure Text Parameters, i.e. multiline strings, aren't handled well here at all either.)

            Show
            danielbeck Daniel Beck added a comment - Besides fixing any obvious bugs, it could make sense to add an option to not convert build parameters into system properties. They can still be accessed from the environment, but this would cut down the crazy wrt. command line arguments. (I'm pretty sure Text Parameters, i.e. multiline strings, aren't handled well here at all either.)
            Hide
            chrop Christian Opitz added a comment - - edited

            Same issue here.

            Example (without changing the job configuration):
            This is the old version that worked:

            cmd.exe /C '"ant.bat -file postResultsToRTC.xml -DbuildResultUUID="" "-DrtcBuildDefinitionId=My_Test Jenkins build" -DrtcRepositoryAddress=https://repository-server.com/ccm "-DrtcBuildEngineId=My_Test Jenkins engine" "-DjenkinsJobDisplayName=#136 - SVN revision r2265" -DjenkinsJobBuildURL=http://myaddress.com:8080/job/Test/job/trunk-restructured-CBA/136/ -lib C:\IBM\TeamConcertBuild\buildsystem\buildtoolkit all && exit %%ERRORLEVEL%%"'
            

            This is the new output not working

            cmd.exe /C "ant.bat -file postResultsToRTC.xml -DbuildResultUUID= '"-DrtcBuildDefinitionId=My_Test Jenkins build"' -DrtcRepositoryAddress=https://repository-server.com/ccm '"-DrtcBuildEngineId=My_Test Jenkins engine"' '"-DjenkinsJobDisplayName=#137 - SVN revision r2285"' -DjenkinsJobBuildURL=http://myaddress.com:8080/job/Test/job/trunk-restructured-CBA/137/ -lib C:\IBM\TeamConcertBuild\buildsystem\buildtoolkit all && exit %%ERRORLEVEL%%"
            
            Show
            chrop Christian Opitz added a comment - - edited Same issue here. Example (without changing the job configuration): This is the old version that worked: cmd.exe /C '"ant.bat -file postResultsToRTC.xml -DbuildResultUUID="" "-DrtcBuildDefinitionId=My_Test Jenkins build" -DrtcRepositoryAddress=https://repository-server.com/ccm "-DrtcBuildEngineId=My_Test Jenkins engine" "-DjenkinsJobDisplayName=#136 - SVN revision r2265" -DjenkinsJobBuildURL=http://myaddress.com:8080/job/Test/job/trunk-restructured-CBA/136/ -lib C:\IBM\TeamConcertBuild\buildsystem\buildtoolkit all && exit %%ERRORLEVEL%%"' This is the new output not working cmd.exe /C "ant.bat -file postResultsToRTC.xml -DbuildResultUUID= '"-DrtcBuildDefinitionId=My_Test Jenkins build"' -DrtcRepositoryAddress=https://repository-server.com/ccm '"-DrtcBuildEngineId=My_Test Jenkins engine"' '"-DjenkinsJobDisplayName=#137 - SVN revision r2285"' -DjenkinsJobBuildURL=http://myaddress.com:8080/job/Test/job/trunk-restructured-CBA/137/ -lib C:\IBM\TeamConcertBuild\buildsystem\buildtoolkit all && exit %%ERRORLEVEL%%"
            Hide
            sergejk Sergej Kleva added a comment -

            Any workaround or solution?!

            We are stucked (

            Show
            sergejk Sergej Kleva added a comment - Any workaround or solution?! We are stucked (
            Hide
            belpk K P added a comment -

            @Sergej : a solution seems to be implemented since a month by now, but no-one seems to bother processing it so it can get released ( https://github.com/jenkinsci/ant-plugin/pull/14 is already waiting for more than two weeks ), even though this is blocking upgrade to jenkins 2.x and even seems to break the LTS releases.

            Show
            belpk K P added a comment - @Sergej : a solution seems to be implemented since a month by now, but no-one seems to bother processing it so it can get released ( https://github.com/jenkinsci/ant-plugin/pull/14 is already waiting for more than two weeks ), even though this is blocking upgrade to jenkins 2.x and even seems to break the LTS releases.
            Hide
            danielbeck Daniel Beck added a comment -

            Armando Fernandez Please consider looking into merging and releasing the PR (assuming it looks good). As you can see above, users are getting restless

            (Everyone, please be kind to Armando, he took over maintainership of Ant plugin just a few days ago!)

            Show
            danielbeck Daniel Beck added a comment - Armando Fernandez Please consider looking into merging and releasing the PR (assuming it looks good). As you can see above, users are getting restless (Everyone, please be kind to Armando, he took over maintainership of Ant plugin just a few days ago!)
            Hide
            sergejk Sergej Kleva added a comment -

            Sure ... i am kind ... but how's that this was implemented month ago and haven't been merged into main branch?

            Hope for the best!

            Brgds,
            Sergej

            Show
            sergejk Sergej Kleva added a comment - Sure ... i am kind ... but how's that this was implemented month ago and haven't been merged into main branch? Hope for the best! Brgds, Sergej
            Hide
            armfergom Armando Fernandez added a comment -

            Hi,

            As Daniel mentioned, I just took over maintainership couple of days ago. I am looking at pending PRs, including this one. Will get a new version ready including a fix for this today.

            Cheers,
            Armando

            Show
            armfergom Armando Fernandez added a comment - Hi, As Daniel mentioned, I just took over maintainership couple of days ago. I am looking at pending PRs, including this one. Will get a new version ready including a fix for this today. Cheers, Armando
            Hide
            sergejk Sergej Kleva added a comment -

            Thanks Armando and the rest of the team!

            Brgds,
            S.

            Show
            sergejk Sergej Kleva added a comment - Thanks Armando and the rest of the team! Brgds, S.
            Hide
            armfergom Armando Fernandez added a comment - - edited

            Hi all,

            So, I have built the PR with the fix and am testing it at the moment. The thing is, I don't see any issues related to parameters with spaces (have not been able to reproduce it locally). I do see issues with parameters that are empty though. I have noticed that the examples that have been provided in the ticket use empty parameters as well, so I am wondering whether this is the real cause of the failures.

            The PR I am looking at solves the empty parameters issue.

            Could you please let me know what error message are you getting to double check that the issue is caused by the spaces in the parameter and not by the empty parameters?

            For example, in my case, I am getting this error message (that goes away when including the fix mentioned above):

            [workspace] $ cmd.exe /C "C:\apache-ant-1.9.7\bin\ant.bat '"-DcustomParameter=algo cno espacions"' -DotroParametro= && exit %%ERRORLEVEL%%"
            Missing value for property otroParametro
            Build step 'Invoke Ant' marked build as failure
            Finished: FAILURE

            Cheers,
            Armando

            Show
            armfergom Armando Fernandez added a comment - - edited Hi all, So, I have built the PR with the fix and am testing it at the moment. The thing is, I don't see any issues related to parameters with spaces (have not been able to reproduce it locally). I do see issues with parameters that are empty though. I have noticed that the examples that have been provided in the ticket use empty parameters as well, so I am wondering whether this is the real cause of the failures. The PR I am looking at solves the empty parameters issue. Could you please let me know what error message are you getting to double check that the issue is caused by the spaces in the parameter and not by the empty parameters? For example, in my case, I am getting this error message (that goes away when including the fix mentioned above): [workspace] $ cmd.exe /C "C:\apache-ant-1.9.7\bin\ant.bat '"-DcustomParameter=algo cno espacions"' -DotroParametro= && exit %%ERRORLEVEL%%" Missing value for property otroParametro Build step 'Invoke Ant' marked build as failure Finished: FAILURE Cheers, Armando
            Hide
            sergejk Sergej Kleva added a comment -

            In jenkins we define global parameters variables which we pass into ant properties plugin.
            Sometimes those global parameters variables are empty ... for different reasons ...

            Show
            sergejk Sergej Kleva added a comment - In jenkins we define global parameters variables which we pass into ant properties plugin. Sometimes those global parameters variables are empty ... for different reasons ...
            Hide
            armfergom Armando Fernandez added a comment -

            Sergej Kleva So, in your case, I assume your problem comes from empty parameters. I experience the same issue and that is alright because there is already a fix for that (not released yet)

            I am wondering about parameters whose value is not empty and contain spaces which is being claimed in this ticket to cause problems. I cannot reproduce it, so would like some more info about the errors that people are getting, if possible.

            Show
            armfergom Armando Fernandez added a comment - Sergej Kleva So, in your case, I assume your problem comes from empty parameters. I experience the same issue and that is alright because there is already a fix for that (not released yet) I am wondering about parameters whose value is not empty and contain spaces which is being claimed in this ticket to cause problems. I cannot reproduce it, so would like some more info about the errors that people are getting, if possible.
            Hide
            chrop Christian Opitz added a comment -

            In my case (see above), I have the following scenario:

            1. Empty parameter buildResultUUID in job A (job is configured as parametrized job, value is left empty)
            2. I am using "Trigger parameterized build on other projects" in job A, the triggered job B is executing the ant script. In job A the following predefined parameters are set (for the parameterized build of job B):
            rtcRepositoryAddress=https://repository-server.com/ccm
            rtcBuildDefinitionId=My_Test Jenkins build
            rtcBuildEngineId=My_Test Jenkins engine
            jenkinsJobBuildURL=${BUILD_URL}
            jenkinsJobDisplayName=${BUILD_DISPLAY_NAME} - SVN revision r${SVN_REVISION}
            buildResultUUID=${buildResultUUID}
            
            Show
            chrop Christian Opitz added a comment - In my case (see above), I have the following scenario: Empty parameter buildResultUUID in job A (job is configured as parametrized job, value is left empty) I am using "Trigger parameterized build on other projects" in job A, the triggered job B is executing the ant script. In job A the following predefined parameters are set (for the parameterized build of job B): rtcRepositoryAddress=https: //repository-server.com/ccm rtcBuildDefinitionId=My_Test Jenkins build rtcBuildEngineId=My_Test Jenkins engine jenkinsJobBuildURL=${BUILD_URL} jenkinsJobDisplayName=${BUILD_DISPLAY_NAME} - SVN revision r${SVN_REVISION} buildResultUUID=${buildResultUUID}
            Hide
            armfergom Armando Fernandez added a comment -

            Christian Opitz Could you paste here the error message you get in the build? Hopefully you will be able to see something like:

            Missing value for property {nameOfParameter}
            

            The thing is, the description and name of the issue suggests that the problem comes from spaces in the values of the parameters, which I suspect is not the case. Just want to be sure that everyone gets what they are expecting

            Show
            armfergom Armando Fernandez added a comment - Christian Opitz Could you paste here the error message you get in the build? Hopefully you will be able to see something like: Missing value for property {nameOfParameter} The thing is, the description and name of the issue suggests that the problem comes from spaces in the values of the parameters, which I suspect is not the case. Just want to be sure that everyone gets what they are expecting
            Hide
            benapgar Ben Apgar added a comment - - edited

            Armando Fernandez

            The related issue with empty parameters is "JENKINS-33712 Empty parameterized build properties not quoted when passed to ant".

            This issue is slightly different. It is caused by spaces in the parameter.

            When there are spaces in a parameter value, the parameter name and value are being enclosed in single and double quotes. This breaks the command, because of another new behavior where the full command is no longer enclosed in single and double quotes--it is only enclosed in double quotes. This causes the parameters to be parsed incorrectly because the quotes are mismatched and the command fails.

            If there are no spaces in the parameter, there are no quotes at all in the name value pair which still works fine.

            Show
            benapgar Ben Apgar added a comment - - edited Armando Fernandez The related issue with empty parameters is " JENKINS-33712 Empty parameterized build properties not quoted when passed to ant". This issue is slightly different. It is caused by spaces in the parameter. When there are spaces in a parameter value, the parameter name and value are being enclosed in single and double quotes. This breaks the command, because of another new behavior where the full command is no longer enclosed in single and double quotes--it is only enclosed in double quotes. This causes the parameters to be parsed incorrectly because the quotes are mismatched and the command fails. If there are no spaces in the parameter, there are no quotes at all in the name value pair which still works fine.
            Hide
            chrop Christian Opitz added a comment -

            Armando Fernandez I don't get any error message at all, the build even passes (but I guess this is due to the setup of the ant file). What I see however, is that the existing arguments are not passed correctly to the ant file, so it is not executed as it was in previous versions.

            For example, the following line is contained in the ant file:

            <echo message="buildResultUUID: .${buildResultUUID}."/>
            

            The output is

            [echo] buildResultUUID: .-DrtcBuildDefinitionId=My_Test Jenkins build.
            

            which is obviously not correct as it takes the definition of the following variables as content of the first one

            Show
            chrop Christian Opitz added a comment - Armando Fernandez I don't get any error message at all, the build even passes (but I guess this is due to the setup of the ant file). What I see however, is that the existing arguments are not passed correctly to the ant file, so it is not executed as it was in previous versions. For example, the following line is contained in the ant file: <echo message= "buildResultUUID: .${buildResultUUID}." /> The output is [echo] buildResultUUID: .-DrtcBuildDefinitionId=My_Test Jenkins build. which is obviously not correct as it takes the definition of the following variables as content of the first one
            Hide
            escoem Emilio Escobar added a comment -

            Hi, the 1.3 version has been released. Hugo Reis, can you try it, please?

            Show
            escoem Emilio Escobar added a comment - Hi, the 1.3 version has been released. Hugo Reis , can you try it, please?
            Hide
            chrop Christian Opitz added a comment -

            For me the 1.3 is working.

            The quotes are different to the initial version, i.e. all parameters with blanks are now quoted both with ' and ". But it seems to be plausible and is working now.

            cmd.exe /C "ant.bat -file postResultsToRTC.xml -DbuildResultUUID="" '"-DrtcBuildDefinitionId=My_Test Jenkins build"' -DrtcRepositoryAddress=https://repository-server.com/ccm '"-DrtcBuildEngineId=My_Test Jenkins engine"' '"-DjenkinsJobDisplayName=#211 - SVN revision r2437"' -DjenkinsJobBuildURL=http://myaddress.com:8080/job/Test/job/trunk-restructured-CBA/211/ -lib C:\IBM\TeamConcertBuild\buildsystem\buildtoolkit all && exit %%ERRORLEVEL%%"
            

            Thanks for the fix!

            Show
            chrop Christian Opitz added a comment - For me the 1.3 is working. The quotes are different to the initial version, i.e. all parameters with blanks are now quoted both with ' and ". But it seems to be plausible and is working now. cmd.exe /C "ant.bat -file postResultsToRTC.xml -DbuildResultUUID=" " '" -DrtcBuildDefinitionId=My_Test Jenkins build "' -DrtcRepositoryAddress=https: //repository-server.com/ccm '" -DrtcBuildEngineId=My_Test Jenkins engine "' '" -DjenkinsJobDisplayName=#211 - SVN revision r2437 "' -DjenkinsJobBuildURL=http://myaddress.com:8080/job/Test/job/trunk-restructured-CBA/211/ -lib C:\IBM\TeamConcertBuild\buildsystem\buildtoolkit all && exit %%ERRORLEVEL%%" Thanks for the fix!
            Hide
            hugo_reis Hugo Reis added a comment -

            Tested successfully:

            • Jenkins 1.652 with Ant plugin 1.3
            • Jenkins 2.3 with Ant plugin 1.3

            Emilio Escobar you can close this issue.

            Thanks for the support.

            Show
            hugo_reis Hugo Reis added a comment - Tested successfully: Jenkins 1.652 with Ant plugin 1.3 Jenkins 2.3 with Ant plugin 1.3 Emilio Escobar you can close this issue. Thanks for the support.
            Hide
            yucey Ugur Ceylan added a comment -

            Hi,

            With Jenkins 2.23 and ant-plugin 1.4 problem still occurs. BUILD_SELECTOR parameter is injected by the copy artifact plugin.

            cmd.exe /C "D:\work\tools\sfw\ant\bin\ant.bat -file build_jenkins.xml '"-DBUILD_SELECTOR=<StatusBuildSelector plugin=""copyartifact@1.38.1""> <stable>true</stable></StatusBuildSelector>"' jenkins.junit.runtests && exit %%ERRORLEVEL%%"
            17:26:09 -Xms256m -Xmx2048m
            17:26:09 Buildfile: build_jenkins.xml
            17:26:11
            17:26:11 BUILD FAILED
            17:26:11 Target `<stable>true</stable></StatusBuildSelector> jenkins.junit.runtests' does not exist in this project.

            Show
            yucey Ugur Ceylan added a comment - Hi, With Jenkins 2.23 and ant-plugin 1.4 problem still occurs. BUILD_SELECTOR parameter is injected by the copy artifact plugin. cmd.exe /C "D:\work\tools\sfw\ant\bin\ant.bat -file build_jenkins.xml '"-DBUILD_SELECTOR=<StatusBuildSelector plugin=""copyartifact@1.38.1""> <stable>true</stable></StatusBuildSelector>"' jenkins.junit.runtests && exit %%ERRORLEVEL%%" 17:26:09 -Xms256m -Xmx2048m 17:26:09 Buildfile: build_jenkins.xml 17:26:11 17:26:11 BUILD FAILED 17:26:11 Target `<stable>true</stable></StatusBuildSelector> jenkins.junit.runtests' does not exist in this project.
            Hide
            armfergom Armando Fernandez added a comment -

            Ugur Ceylan I'd say it seems to be related to the two double quotes in ""copyartifact@1.38.1"", but would need to look deeper into it

            Show
            armfergom Armando Fernandez added a comment - Ugur Ceylan I'd say it seems to be related to the two double quotes in ""copyartifact@1.38.1"" , but would need to look deeper into it

              People

              Assignee:
              escoem Emilio Escobar
              Reporter:
              hugo_reis Hugo Reis
              Votes:
              4 Vote for this issue
              Watchers:
              11 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: