Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-16176

ShiningPanda PATH update uses wrong separator on Windows

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • shiningpanda-plugin
    • None
    • Linux master, Windows XP slave with Python 2.7

      When running ShiningPanda on a Windows slave, the plugin (or virtualenv?) updates the PATH to include the virtualenv directory.

      0. Install ShiningPanda plugin (this result is for v1.15)
      1. Create a build with a Virtualenv build step
      2. Use the Shell or XShell command with a minimal script:

      set 
      pip.exe install -r pkg_requirements.txt
      

      The build fails because the pip command can not be found. The output shows the Path environment variable uses the : (colon) separator for the virtualenv Scripts directory. On Windows, the PATH separator should be ; (semicolon).

      [1-SMSDK-apps-venv] $ cmd.exe /c call C:\DOCUME~1\builder\LOCALS~1\Temp\shiningpanda4030889806584404855.bat
      
      c:\jenkins-dev\workspace\1-SMSDK-apps-venv>set
      ... [edited for relevance] ...
      Path=c:\jenkins-dev\workspace\1-SMSDK-apps-venv:c:\jenkins-dev\shiningpanda\deb23754\virtualenvs\d41d8cd9\Scripts:C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\Program Files\FLEXlm;C:\Program Files\CollabNet\Subversion Client;c:\cygwin\bin;c:\cygwin\usr\bin;C:\Program Files\Java\jre6\bin
      PYTHON_EXE=c:\jenkins-dev\shiningpanda\deb23754\virtualenvs\d41d8cd9\Scripts\python.exe
      VIRTUAL_ENV=c:\jenkins-dev\shiningpanda\deb23754\virtualenvs\d41d8cd9
      WORKSPACE=c:\jenkins-dev\workspace\1-SMSDK-apps-venv
      
      c:\jenkins-dev\workspace\1-SMSDK-apps-venv>pip.exe install -r pkg_requirements.pip 
      'pip.exe' is not recognized as an internal or external command,
      operable program or batch file.
      

          [JENKINS-16176] ShiningPanda PATH update uses wrong separator on Windows

          Dave Bacher created issue -

          C Girard added a comment -

          When I search for "PATH+" in the code base, I noticed the following things...

          First, all of the calls are either environment.put("PATH+", ...) or environment.override("PATH+", ...), but the class of "environment" differs in each case. In the first one (the "put" case), the "environment" variable is a regular Java HashMap. Its put() method does not contain special handling for the "+" character, so passing "PATH+" to put() seems wrong.

          Second, for the calls that use override(), the "environment" variable is a Hudson EnvVars map. This method does support the "+" notation, but the underlying implementation of override() assumes that the pathsep character (either : or ; ) is for the local environment. So I suspect these locations either need to handle the "PATH+" manually (not relying on the EnvVars.override() method), or this code needs to be moved to executing on the remote slave machine.

          C Girard added a comment - When I search for "PATH+" in the code base, I noticed the following things... First, all of the calls are either environment.put("PATH+", ...) or environment.override("PATH+", ...), but the class of "environment" differs in each case. In the first one (the "put" case), the "environment" variable is a regular Java HashMap. Its put() method does not contain special handling for the "+" character, so passing "PATH+" to put() seems wrong. Second, for the calls that use override(), the "environment" variable is a Hudson EnvVars map. This method does support the "+" notation, but the underlying implementation of override() assumes that the pathsep character (either : or ; ) is for the local environment. So I suspect these locations either need to handle the "PATH+" manually (not relying on the EnvVars.override() method), or this code needs to be moved to executing on the remote slave machine.

          Shannon Kerr added a comment -

          We can confirm that we are seeing this same bug at our site. We are successfully executing a sed replace of : with ; at the start of the command execution. This allows us to continue unblocked by this bug.

          Shannon Kerr added a comment - We can confirm that we are seeing this same bug at our site. We are successfully executing a sed replace of : with ; at the start of the command execution. This allows us to continue unblocked by this bug.

          Chris Withers added a comment -

          Shannon: how did you install sed on windows? What does your sed look like?

          Chris Withers added a comment - Shannon: how did you install sed on windows? What does your sed look like?

          Shannon Kerr added a comment -

          Good question. I have GNU Sed 4.2.1 installed in Program Files (x86)/GnuWin32/bin/sed.exe

          I also have sed in my Cygwin installation, by my PATH sets the GNU sed in Program Files (x86) as my primary sed.

          Shannon Kerr added a comment - Good question. I have GNU Sed 4.2.1 installed in Program Files (x86)/GnuWin32/bin/sed.exe I also have sed in my Cygwin installation, by my PATH sets the GNU sed in Program Files (x86) as my primary sed.

          Shannon Kerr added a comment -

          Shannon Kerr added a comment - Here is a link to that sed: http://gnuwin32.sourceforge.net/packages/sed.htm

          Shannon Kerr added a comment -

          Sorry for the spam, here is the code we are using in Jenkins to fix the PATH:

          REM ShiningPanda has an existing bug (https://issues.jenkins-ci.org/browse/JENKINS-16176) where
          REM the delimiter used to add the virtual Python location ton the PATH is ':' and not ';',
          REM causing the generated Python paths to be inaccessible. The following corrects the PATH.
          echo Correcting path for ShiningPanda bug 16176
          for /f "delims=" %%a in ('\cygwin\bin\echo "%PATH%" ^| sed "s/:C:/;C:/"') do @set "PATH=%%a"

          Shannon Kerr added a comment - Sorry for the spam, here is the code we are using in Jenkins to fix the PATH: REM ShiningPanda has an existing bug ( https://issues.jenkins-ci.org/browse/JENKINS-16176 ) where REM the delimiter used to add the virtual Python location ton the PATH is ':' and not ';', REM causing the generated Python paths to be inaccessible. The following corrects the PATH. echo Correcting path for ShiningPanda bug 16176 for /f "delims=" %%a in ('\cygwin\bin\echo "%PATH%" ^| sed "s/:C:/;C:/"') do @set "PATH=%%a"

          Chris Withers added a comment -

          C Girard: What suggests to you that this code is running on the master not the slave? How would the code be moved to execute on the slave?

          Chris Withers added a comment - C Girard: What suggests to you that this code is running on the master not the slave? How would the code be moved to execute on the slave?

          Chris Withers added a comment -

          Chris Withers added a comment - Latest diggings on this are here: https://groups.google.com/forum/#!topic/jenkinsci-dev/VgCqabekqPM

          C Girard added a comment -

          I think the analysis from the prior comment is the same reasoning I had for assuming this was running on the master. The path separator is either coming from the "File" class or from the "platform" instance which appears to only come from Platform.current(). Note that "platform" is a private class member, so this rules out subclasses having overridden this or manually setting prior to copy construction.

          In either case, getting a colon back would indicate the code is running on a linux machine. In our setup, we have a linux master and windows slave, so it seems like the code is running on the master.

          C Girard added a comment - I think the analysis from the prior comment is the same reasoning I had for assuming this was running on the master. The path separator is either coming from the "File" class or from the "platform" instance which appears to only come from Platform.current(). Note that "platform" is a private class member, so this rules out subclasses having overridden this or manually setting prior to copy construction. In either case, getting a colon back would indicate the code is running on a linux machine. In our setup, we have a linux master and windows slave, so it seems like the code is running on the master.

            Unassigned Unassigned
            dbacher Dave Bacher
            Votes:
            5 Vote for this issue
            Watchers:
            10 Start watching this issue

              Created:
              Updated: