Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-30114

CMake builder cannot find NMake

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed (View Workflow)
    • Priority: Minor
    • Resolution: Fixed
    • Component/s: cmakebuilder-plugin
    • Labels:
      None
    • Environment:
      Host:
      - Debian Wheezy amd64
      - Jenkins ver. 1.625
      - CMake plugin 2.1

      Guest:
      - Windows 7 64 bits
      - Visual Stutio 2010

    • Similar Issues:

      Description

      I got an issue where i use the CMake Plugins to build on a slave machine on Windows. It's seems the environment is not get when using the builder.

      I can build the project using "NMake Makefiles" generator, but the command "nmake" is not found at compile time, but found before the cmake execution.

      The fact is i'm using dynamic environment, where i inject the environment before the build, it found at the cmake step, but not at the build step.

      Details in the attached file.

      Is there a bug, or i miss something?

      Regards

        Attachments

          Activity

          ebeuque Eric Beuque created issue -
          Hide
          15knots Martin Weber added a comment -

          From your log:
          {{[build] $ nmake
          The system cannot find the file specified}}

          Looks like nmake is not on the system path.

          And the line
          {{Build step 'Use builders from another project' marked build as failure
          }}
          makes me think the Use builders from another project plugin is invoking nmake, not the cmakebuilder-plugin.

          To clarify:
          After build-script generation, cmakebuilder-plugin parses the cmake-cache file and exposes $CMAKE_BUILD_TOOL, if possible.
          Can you post the line with CMAKE_MAKE_PROGRAM from <build>/CMakeCache.txt here?
          If it does not contain an absolute file-system path to nmake, most likely nmake is not in a location where cmake can find it.

          Show
          15knots Martin Weber added a comment - From your log: {{ [build] $ nmake The system cannot find the file specified}} Looks like nmake is not on the system path. And the line {{Build step 'Use builders from another project' marked build as failure }} makes me think the Use builders from another project plugin is invoking nmake, not the cmakebuilder-plugin. To clarify: After build-script generation, cmakebuilder-plugin parses the cmake-cache file and exposes $CMAKE_BUILD_TOOL, if possible. Can you post the line with CMAKE_MAKE_PROGRAM from <build>/CMakeCache.txt here? If it does not contain an absolute file-system path to nmake, most likely nmake is not in a location where cmake can find it.
          Hide
          ebeuque Eric Beuque added a comment -

          Yes the line is :
          CMAKE_MAKE_PROGRAM:String = nmake.

          So it's not an absolute path, but i don't understand why the cmake plugin can find the compiler (when invoking cmake), but not on the build step, since it's in the same directory . All is in the "C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\BIN" directory which is present in the environment. You think it's a problem with the template plugins (providing the "Use builders from another project"). I was thinking that nmake is invoked by the cmake plugin.

          Show
          ebeuque Eric Beuque added a comment - Yes the line is : CMAKE_MAKE_PROGRAM:String = nmake. So it's not an absolute path, but i don't understand why the cmake plugin can find the compiler (when invoking cmake), but not on the build step, since it's in the same directory . All is in the "C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\BIN" directory which is present in the environment. You think it's a problem with the template plugins (providing the "Use builders from another project"). I was thinking that nmake is invoked by the cmake plugin.
          Hide
          ebeuque Eric Beuque added a comment -

          Note that i also tried to define the CMAKE_MAKE_PROGRAM in the toolchain file with an absolute file, it found the nmake program, but not the compiler (cl.exe), which is nevertheless found when invoking cmake.
          I think there something is dropping the environment before calling the make program.

          Show
          ebeuque Eric Beuque added a comment - Note that i also tried to define the CMAKE_MAKE_PROGRAM in the toolchain file with an absolute file, it found the nmake program, but not the compiler (cl.exe), which is nevertheless found when invoking cmake. I think there something is dropping the environment before calling the make program.
          Hide
          ebeuque Eric Beuque added a comment -

          I tested with defining the project without template, but problem is the same.

          Show
          ebeuque Eric Beuque added a comment - I tested with defining the project without template, but problem is the same.
          Hide
          15knots Martin Weber added a comment -

          I was thinking that nmake is invoked by the cmake plugin.
          Only if you add a 'Run build tool' (sub-)step to the cmake plugin.

          Your log shows that cmake finds the compiler. It's unclear, why cmake does not set the absolute path for nmake, at least it does so for the Mingw, MSYS, Makefiles and ninja generators. At least for the ninja generator, cmake will complain and fail at generation time, if it cannot find the ninja tool. (Did not test that with other generators.)

          Your set-build-env.bat script sets SHELL environment variables. AFAIK, these are visible to the invoked shell only, but not by the calling process (jenkins-slave.exe).

          Try to replace the 'Use builders from another project' build step by an Invoke Shell step that prints the SHELL environment variables to diagnose that.

          Show
          15knots Martin Weber added a comment - I was thinking that nmake is invoked by the cmake plugin. Only if you add a 'Run build tool' (sub-)step to the cmake plugin. Your log shows that cmake finds the compiler. It's unclear, why cmake does not set the absolute path for nmake, at least it does so for the Mingw, MSYS, Makefiles and ninja generators. At least for the ninja generator, cmake will complain and fail at generation time, if it cannot find the ninja tool. (Did not test that with other generators.) Your set-build-env.bat script sets SHELL environment variables. AFAIK, these are visible to the invoked shell only, but not by the calling process (jenkins-slave.exe). Try to replace the 'Use builders from another project' build step by an Invoke Shell step that prints the SHELL environment variables to diagnose that.
          Hide
          ebeuque Eric Beuque added a comment -

          New build without using template and checking environment before nmake

          Show
          ebeuque Eric Beuque added a comment - New build without using template and checking environment before nmake
          ebeuque Eric Beuque made changes -
          Field Original Value New Value
          Attachment jenkins-build2.txt [ 30541 ]
          Hide
          ebeuque Eric Beuque added a comment - - edited

          I found that the CMakeNMakeFindMake.cmake just set:

          set (CMAKE_MAKE_PROGRAM "nmake" CACHE STRING "Program used to build from makefiles.")
          

          So it's seems not settings an absolute path.

          I tried to override the make command by setting the CMAKE_MAKE_PROGRAM with a batch file :

          @echo off
          Rem Copy this at the side of nmake.exe
          echo "Initial environnement for nmake"
          set
          echo "Set environnement for Visual Studio"
          call "C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\vcvarsall.bat" x86
          echo "New environnement for nmake"
          set
          echo "Launching nmake command"
          nmake %*
          

          You can see in the attached new log that env is lost before invoking nmake. jenkins-build2.txt

          Show
          ebeuque Eric Beuque added a comment - - edited I found that the CMakeNMakeFindMake.cmake just set: set (CMAKE_MAKE_PROGRAM "nmake" CACHE STRING "Program used to build from makefiles." ) So it's seems not settings an absolute path. I tried to override the make command by setting the CMAKE_MAKE_PROGRAM with a batch file : @echo off Rem Copy this at the side of nmake.exe echo "Initial environnement for nmake" set echo "Set environnement for Visual Studio" call "C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\vcvarsall.bat" x86 echo "New environnement for nmake" set echo "Launching nmake command" nmake %* You can see in the attached new log that env is lost before invoking nmake. jenkins-build2.txt
          Hide
          15knots Martin Weber added a comment - - edited

          Similar problems with nmake and other MS build tools have already been discussed on the jenkins user list.
          To summarize the above job`s build configuration, the following happens:

          1. Runs a batch file
            cmd /c call C:\Windows\TEMP\hudson9148105424479844890.bat
            C:\jenkins\workspace\Trunk_Toolkit4\BuildPlatform\platform-windows-msvc32\BuildProduct\jet1oeil-toolkit\BuildType\Release>call "E:/build/set-build-env.bat" platform-windows-msvc32

            to set the SHELL environment variables for the build.

          2. Lets the plugin generate the makefiles.
          3. Runs nmake, which requires the SHELL environment variables set in step 1 above to succeed.

          This approach won`t work, because the SHELL environment variables get forgotten when the shell (cmd.exe) in step 1 above exits.
          But finally you got it right with the script in https://issues.jenkins-ci.org/browse/JENKINS-30114?focusedCommentId=235348&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-235348:

          1. Let the plugin generate the makefiles.
          2. Invoke a shell which first sets the required environment variables and then invokes nmake.

          For everybody else who may stumble on problems with nmake (esp: MS build tools): Do not let the cmake builder plugin invoke nmake, use an extra build step (Invoke windows command) which sets the required env vars first!

          Show
          15knots Martin Weber added a comment - - edited Similar problems with nmake and other MS build tools have already been discussed on the jenkins user list. To summarize the above job`s build configuration, the following happens: Runs a batch file cmd /c call C:\Windows\TEMP\hudson9148105424479844890.bat C:\jenkins\workspace\Trunk_Toolkit4\BuildPlatform\platform-windows-msvc32\BuildProduct\jet1oeil-toolkit\BuildType\Release>call "E:/build/set-build-env.bat" platform-windows-msvc32 to set the SHELL environment variables for the build. Lets the plugin generate the makefiles. Runs nmake, which requires the SHELL environment variables set in step 1 above to succeed. This approach won`t work, because the SHELL environment variables get forgotten when the shell (cmd.exe) in step 1 above exits. But finally you got it right with the script in https://issues.jenkins-ci.org/browse/JENKINS-30114?focusedCommentId=235348&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-235348: Let the plugin generate the makefiles. Invoke a shell which first sets the required environment variables and then invokes nmake. For everybody else who may stumble on problems with nmake (esp: MS build tools): Do not let the cmake builder plugin invoke nmake, use an extra build step (Invoke windows command) which sets the required env vars first!
          Hide
          15knots Martin Weber added a comment -

          Closing this as it is related to how environment variables are inherited by processes, but not related to the plugin

          Show
          15knots Martin Weber added a comment - Closing this as it is related to how environment variables are inherited by processes, but not related to the plugin
          15knots Martin Weber made changes -
          Resolution Not A Defect [ 7 ]
          Status Open [ 1 ] Closed [ 6 ]
          Hide
          akallabeth armin novak added a comment -

          I've encountered the same issue.

          • Using "Inject environment variables to the build process" for set up
          • The build environment contains the correct path settings as can be seen after the build
          • The cmake call still fails, looks like it uses some different path setting than what was set up before the first build step
          Show
          akallabeth armin novak added a comment - I've encountered the same issue. Using "Inject environment variables to the build process" for set up The build environment contains the correct path settings as can be seen after the build The cmake call still fails, looks like it uses some different path setting than what was set up before the first build step
          Hide
          akallabeth armin novak added a comment -

          As stated in my last comment,
          this also happens when the environment variables are set by EnvInject.
          The paths are properly set up and contain NMake (executing a batch shell can run nmake just fine without additional environment magic)
          Looks like it is indeed a problem with inheritance, namely this plugin using a different one from the jenkins provided environment.

          As for my setup:

          • Jenkins 1.625.1
          • CMake Plugin 2.3.2
          Show
          akallabeth armin novak added a comment - As stated in my last comment, this also happens when the environment variables are set by EnvInject. The paths are properly set up and contain NMake (executing a batch shell can run nmake just fine without additional environment magic) Looks like it is indeed a problem with inheritance, namely this plugin using a different one from the jenkins provided environment. As for my setup: Jenkins 1.625.1 CMake Plugin 2.3.2
          akallabeth armin novak made changes -
          Resolution Not A Defect [ 7 ]
          Status Closed [ 6 ] Reopened [ 4 ]
          Hide
          15knots Martin Weber added a comment -

          Repeating comment-235252 here:
          After build-script generation, cmakebuilder-plugin parses the cmake-cache file in <build>/CMakeCache.txt for the value of CMAKE_MAKE_PROGRAM and exposes it as $CMAKE_BUILD_TOOL, if possible.
          The Run build tool build step takes the value of $CMAKE_BUILD_TOOL to invoke the actual tool.
          Unfortunately, the cmake generator for nmake sets CMAKE_MAKE_PROGRAM=nmake instead of setting the absolute file system path of nmake. To invoke nmake, jenkins needs an absolute file system path or nmake.exe must be in $PATH of the jenkins process (either slave or master).

          Using "Inject environment variables to the build process" for set up, as you tried, will not help, since that will set environment variables for the nmake process but will not help jenkins to find nmake.exe.

          Show
          15knots Martin Weber added a comment - Repeating comment-235252 here: After build-script generation, cmakebuilder-plugin parses the cmake-cache file in <build>/CMakeCache.txt for the value of CMAKE_MAKE_PROGRAM and exposes it as $CMAKE_BUILD_TOOL, if possible. The Run build tool build step takes the value of $CMAKE_BUILD_TOOL to invoke the actual tool. Unfortunately, the cmake generator for nmake sets CMAKE_MAKE_PROGRAM=nmake instead of setting the absolute file system path of nmake. To invoke nmake, jenkins needs an absolute file system path or nmake.exe must be in $PATH of the jenkins process (either slave or master). Using "Inject environment variables to the build process" for set up , as you tried, will not help, since that will set environment variables for the nmake process but will not help jenkins to find nmake.exe .
          Hide
          akallabeth armin novak added a comment -

          The problem is that the build tool step discards the environment set up!
          For the configuration step the environment is fine and my injected variables and modified path is there.

          Had a short look at the code, looks like BuildToolStep.getEnvironmentVars is stripping everything and passing on an empty environment to the build tool steps.

          Show
          akallabeth armin novak added a comment - The problem is that the build tool step discards the environment set up! For the configuration step the environment is fine and my injected variables and modified path is there. Had a short look at the code, looks like BuildToolStep.getEnvironmentVars is stripping everything and passing on an empty environment to the build tool steps.
          Hide
          akallabeth armin novak added a comment -

          A patch for this issue.
          Each build tool step now takes the job environment and overrides it with the supplied custom environment instead of only passing on the custom step environment variables.

          Show
          akallabeth armin novak added a comment - A patch for this issue. Each build tool step now takes the job environment and overrides it with the supplied custom environment instead of only passing on the custom step environment variables.
          akallabeth armin novak made changes -
          Attachment 0001-A-fix-for-JENKINS-30114.patch [ 31298 ]
          15knots Martin Weber made changes -
          Assignee Martin Weber [ 15knots ]
          Hide
          15knots Martin Weber added a comment -
          Show
          15knots Martin Weber added a comment - armin novak Thanks for pointing me to the build tool step discarding th environment setup. Please install and test https://jenkins.ci.cloudbees.com/job/plugins/job/cmakebuilder-plugin/org.jenkins-ci.plugins$cmakebuilder/lastSuccessfulBuild/artifact/org.jenkins-ci.plugins/cmakebuilder/2.4-SNAPSHOT/cmakebuilder-2.4-SNAPSHOT.hpi to verify whether it works.
          Hide
          akallabeth armin novak added a comment -

          Martin Weber Ok, looks like we both were right.
          With the patch the NMake tool works if called by ```cmake --build``` (if the path to NMake is set up with EnvInject)
          If called directly it still fails. Well, better than before I'd say

          Show
          akallabeth armin novak added a comment - Martin Weber Ok, looks like we both were right. With the patch the NMake tool works if called by ```cmake --build``` (if the path to NMake is set up with EnvInject) If called directly it still fails. Well, better than before I'd say
          Hide
          15knots Martin Weber added a comment -

          Version 2.3.4 allows to apply the suggested workaround.

          Show
          15knots Martin Weber added a comment - Version 2.3.4 allows to apply the suggested workaround.
          15knots Martin Weber made changes -
          Resolution Fixed [ 1 ]
          Status Reopened [ 4 ] Resolved [ 5 ]
          15knots Martin Weber made changes -
          Priority Blocker [ 1 ] Minor [ 4 ]
          rtyler R. Tyler Croy made changes -
          Workflow JNJira [ 165198 ] JNJira + In-Review [ 197649 ]
          15knots Martin Weber made changes -
          Status Resolved [ 5 ] Closed [ 6 ]

            People

            Assignee:
            15knots Martin Weber
            Reporter:
            ebeuque Eric Beuque
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved: