Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-32715

virtualenv creation blows up on concurrent jobs started at the same-ish time

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      In our environment, we often have a dynamically-provisioned, multi-executor slave come online and immediately run parallel builds of the same job (for example, for different pull requests).

      The ShiningPanda plugin can sometimes cause a build failure in this scenario. It appears to try to create the same virtualenv in each job, and one fails:

      Succeeding job:

      15:47:55 [some_job@2] $ /usr/bin/python2.7 /home/jenkins/shiningpanda/jobs/5cdc8cf6/virtualenv.py /home/jenkins/shiningpanda/jobs/5cdc8cf6/virtualenvs/d41d8cd9
      15:47:56 New python executable in /home/jenkins/shiningpanda/jobs/5cdc8cf6/virtualenvs/d41d8cd9/bin/python2.7
      15:47:56 Also creating executable in /home/jenkins/shiningpanda/jobs/5cdc8cf6/virtualenvs/d41d8cd9/bin/python
      15:47:57 Installing setuptools, pip, wheel...done.
      15:48:02 [some_job@2] $ /bin/sh -xe /tmp/shiningpanda8205115405688864725.sh
      

      Failing job:

      15:47:55 [some_job] $ /usr/bin/python2.7 /home/jenkins/shiningpanda/jobs/5cdc8cf6/virtualenv.py /home/jenkins/shiningpanda/jobs/5cdc8cf6/virtualenvs/d41d8cd9
      15:47:55 Traceback (most recent call last):
      15:47:55   File "/home/jenkins/shiningpanda/jobs/5cdc8cf6/virtualenv.py", line 2363, in <module>
      15:47:55     main()
      15:47:55   File "/home/jenkins/shiningpanda/jobs/5cdc8cf6/virtualenv.py", line 832, in main
      15:47:55     symlink=options.symlink)
      15:47:55   File "/home/jenkins/shiningpanda/jobs/5cdc8cf6/virtualenv.py", line 994, in create_environment
      15:47:55     site_packages=site_packages, clear=clear, symlink=symlink))
      15:47:55   File "/home/jenkins/shiningpanda/jobs/5cdc8cf6/virtualenv.py", line 1170, in install_python
      15:47:55     mkdir(lib_dir)
      15:47:55   File "/home/jenkins/shiningpanda/jobs/5cdc8cf6/virtualenv.py", line 448, in mkdir
      15:47:55     os.makedirs(path)
      15:47:55   File "/usr/lib/python2.7/os.py", line 157, in makedirs
      15:47:55     mkdir(name, mode)
      15:47:55 OSError: [Errno 17] File exists: '/home/jenkins/shiningpanda/jobs/5cdc8cf6/virtualenvs/d41d8cd9/lib/python2.7'
      15:47:56 Build step 'Virtualenv Builder' marked build as failure
      

        Attachments

          Activity

          Hide
          schniedergers Klaus Schniedergers added a comment -

          This bites me very often, as I have one job running multiple builds in parallel on the same node.

          If there was a way to expand environment variables into the "Name" config, then it could be solved by appending the ${EXECUTOR_NUMBER} to the virtualenv name.

          Show
          schniedergers Klaus Schniedergers added a comment - This bites me very often, as I have one job running multiple builds in parallel on the same node. If there was a way to expand environment variables into the "Name" config, then it could be solved by appending the ${EXECUTOR_NUMBER} to the virtualenv name.

            People

            Assignee:
            Unassigned Unassigned
            Reporter:
            oeuftete oeuftete
            Votes:
            3 Vote for this issue
            Watchers:
            3 Start watching this issue

              Dates

              Created:
              Updated: