Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-32715

virtualenv creation blows up on concurrent jobs started at the same-ish time

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Critical Critical
    • shiningpanda-plugin
    • None

      In our environment, we often have a dynamically-provisioned, multi-executor slave come online and immediately run parallel builds of the same job (for example, for different pull requests).

      The ShiningPanda plugin can sometimes cause a build failure in this scenario. It appears to try to create the same virtualenv in each job, and one fails:

      Succeeding job:

      15:47:55 [some_job@2] $ /usr/bin/python2.7 /home/jenkins/shiningpanda/jobs/5cdc8cf6/virtualenv.py /home/jenkins/shiningpanda/jobs/5cdc8cf6/virtualenvs/d41d8cd9
15:47:56 New python executable in /home/jenkins/shiningpanda/jobs/5cdc8cf6/virtualenvs/d41d8cd9/bin/python2.7
15:47:56 Also creating executable in /home/jenkins/shiningpanda/jobs/5cdc8cf6/virtualenvs/d41d8cd9/bin/python
15:47:57 Installing setuptools, pip, wheel...done.
15:48:02 [some_job@2] $ /bin/sh -xe /tmp/shiningpanda8205115405688864725.sh

      Failing job:

      15:47:55 [some_job] $ /usr/bin/python2.7 /home/jenkins/shiningpanda/jobs/5cdc8cf6/virtualenv.py /home/jenkins/shiningpanda/jobs/5cdc8cf6/virtualenvs/d41d8cd9
15:47:55 Traceback (most recent call last):
15:47:55   File "/home/jenkins/shiningpanda/jobs/5cdc8cf6/virtualenv.py", line 2363, in <module>
15:47:55     main()
15:47:55   File "/home/jenkins/shiningpanda/jobs/5cdc8cf6/virtualenv.py", line 832, in main
15:47:55     symlink=options.symlink)
15:47:55   File "/home/jenkins/shiningpanda/jobs/5cdc8cf6/virtualenv.py", line 994, in create_environment
15:47:55     site_packages=site_packages, clear=clear, symlink=symlink))
15:47:55   File "/home/jenkins/shiningpanda/jobs/5cdc8cf6/virtualenv.py", line 1170, in install_python
15:47:55     mkdir(lib_dir)
15:47:55   File "/home/jenkins/shiningpanda/jobs/5cdc8cf6/virtualenv.py", line 448, in mkdir
15:47:55     os.makedirs(path)
15:47:55   File "/usr/lib/python2.7/os.py", line 157, in makedirs
15:47:55     mkdir(name, mode)
15:47:55 OSError: [Errno 17] File exists: '/home/jenkins/shiningpanda/jobs/5cdc8cf6/virtualenvs/d41d8cd9/lib/python2.7'
15:47:56 Build step 'Virtualenv Builder' marked build as failure

            Unassigned Unassigned
            oeuftete oeuftete
            Votes:
            3 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated: