Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-19506

[integration] - android-plugin + custom-tools installation fails due to NPE in ProcStarter::envs()

      I run customtools-plugin together with android-emulator-plugin, and when jenkins launches tasks for android-emulator, they are launched without the default PATH varable.
      This means that the shell scripts that start the emulator can't find common tools as dirname, file and so on.

      Workaround is to add a custom tool that defines default paths too, but thats site-wide and when some nodes have other PATH's than other, i can only add the directories that exists on all nodes.

          [JENKINS-19506] [integration] - android-plugin + custom-tools installation fails due to NPE in ProcStarter::envs()

          Oleg Nenashev added a comment -

          Hello,

          You can use label-specific tool installations to define PATH individually (since customtools-0.3).
          In addition, you can specify relative paths + node-specific home dirs.

          Do you need anything else? If yes, please recategorize the issue and describe your vision.

          BR, Oleg Nenashev

          Oleg Nenashev added a comment - Hello, You can use label-specific tool installations to define PATH individually (since customtools-0.3). In addition, you can specify relative paths + node-specific home dirs. Do you need anything else? If yes, please recategorize the issue and describe your vision. BR, Oleg Nenashev

          Anton Lundin added a comment -

          Ok for how you can make the workaround better, but the main issue is still that it overrides the default PATH, not adds things to it.

          Eg. the android-emulator plugin works just fine on jobs without customtools but as soon as you activate customtools, it overrides the PATH and breaks everything depending on it.

          Anton Lundin added a comment - Ok for how you can make the workaround better, but the main issue is still that it overrides the default PATH, not adds things to it. Eg. the android-emulator plugin works just fine on jobs without customtools but as soon as you activate customtools, it overrides the PATH and breaks everything depending on it.

          Oleg Nenashev added a comment - - edited

          custom-tools appends PATHs by default. BTW, there could be issues with EnvInject plugin. Do you use it for initial PATH setup? Anyway, please mention the custom-tools plugin version.

          Oleg Nenashev added a comment - - edited custom-tools appends PATHs by default. BTW, there could be issues with EnvInject plugin. Do you use it for initial PATH setup? Anyway, please mention the custom-tools plugin version.

          Anton Lundin added a comment -

          I read about problems with EnvInject so i have removed EnvInject. This issues was reproduced with 0.4-SNAPSHOT 468a968cb4197acccbb843612b963cb451af04d7 (git head as of 2013-09-08).

          There were fixes that looked promising in the commit log so i tested the latest git HEAD before reporting it.

          Anton Lundin added a comment - I read about problems with EnvInject so i have removed EnvInject. This issues was reproduced with 0.4-SNAPSHOT 468a968cb4197acccbb843612b963cb451af04d7 (git head as of 2013-09-08). There were fixes that looked promising in the commit log so i tested the latest git HEAD before reporting it.

          Oleg Nenashev added a comment - - edited

          It is quite strange.

          Unfortunately, I have not environment to reproduce the issue. Could you provide the custom tools confiigs, job's configuration and build outputs? I'm also interested in the initial build environment and steps internal build env.

          I have not used Android Plugin before, but probably I will be able to locate the issue. Custom-tools works well for many unix-based platforms, so I expect the integration issue with that plugin only.

          Oleg Nenashev added a comment - - edited It is quite strange. Unfortunately, I have not environment to reproduce the issue. Could you provide the custom tools confiigs, job's configuration and build outputs? I'm also interested in the initial build environment and steps internal build env. I have not used Android Plugin before, but probably I will be able to locate the issue. Custom-tools works well for many unix-based platforms, so I expect the integration issue with that plugin only.

          Anton Lundin added a comment -

          Errors produced with customtools and android-emulator.

          Anton Lundin added a comment - Errors produced with customtools and android-emulator.

          Anton Lundin added a comment -

          How to reproduce:

          download a fresh jenkins from http://jenkins-ci.org/

          install android-emulator and customtools plugins.

          Create a job with customtools and a android-emulator (ex. custom.xml)

          click build, android-emulator installs the android sdk, and tries to install its tools but it can't.

          Install android build-tools and platform-tools by hand to show the later error message.

          When you try to build then, it tries to start a emulator, but the shell-script emulator (jenkins/tools/android-sdk/tools/emulator) can't find dirname and basename due to that custom-tools have removed PATH

          Disable customtools in the job and watch how jenkins can start a emulator.

          Anton Lundin added a comment - How to reproduce: download a fresh jenkins from http://jenkins-ci.org/ install android-emulator and customtools plugins. Create a job with customtools and a android-emulator (ex. custom.xml) click build, android-emulator installs the android sdk, and tries to install its tools but it can't. Install android build-tools and platform-tools by hand to show the later error message. When you try to build then, it tries to start a emulator, but the shell-script emulator (jenkins/tools/android-sdk/tools/emulator) can't find dirname and basename due to that custom-tools have removed PATH Disable customtools in the job and watch how jenkins can start a emulator.

          Oleg Nenashev added a comment - - edited

          Seems that you have found a minor bug in the Jenkins core. envs may be null and there's no check (see code below).
          I'll create PR, but it won't resolve your issue,

          public String[] envs() {
          return envs.clone();
          }

          ------------------

          Custom-tools plugin receives the launcher from Android-Emulator. As I see, this plugin Android-Emulator this launcher from doRun(), but other functions may modify the launcher's contents. I'm almost sure that issue has not been caused by Custom-tools. I'll reassign it to @orrc in order to get his comments.

          [android] Installing the 'platform-tool' SDK component(s)...
          FATAL: null
          java.lang.NullPointerException
          at hudson.Launcher$ProcStarter.envs(Launcher.java:302)
          at com.cloudbees.jenkins.plugins.customtools.CustomToolInstallWrapper$2.launch(CustomToolInstallWrapper.java:186)
          at hudson.Launcher$ProcStarter.start(Launcher.java:353)
          at hudson.plugins.android_emulator.SdkInstaller.installComponent(SdkInstaller.java:213)
          at hudson.plugins.android_emulator.SdkInstaller.doInstall(SdkInstaller.java:94)
          at hudson.plugins.android_emulator.SdkInstaller.install(SdkInstaller.java:66)
          at hudson.plugins.android_emulator.AndroidEmulator.setUp(AndroidEmulator.java:241)
          at hudson.model.Build$BuildExecution.doRun(Build.java:154)
          at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:586)
          at hudson.model.Run.execute(Run.java:1603)
          at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
          at hudson.model.ResourceController.execute(ResourceController.java:88)
          at hudson.model.Executor.run(Executor.java:247)

          Oleg Nenashev added a comment - - edited Seems that you have found a minor bug in the Jenkins core. envs may be null and there's no check (see code below). I'll create PR, but it won't resolve your issue, public String[] envs() { return envs.clone(); } ------------------ Custom-tools plugin receives the launcher from Android-Emulator. As I see, this plugin Android-Emulator this launcher from doRun(), but other functions may modify the launcher's contents. I'm almost sure that issue has not been caused by Custom-tools. I'll reassign it to @orrc in order to get his comments. [android] Installing the 'platform-tool' SDK component(s)... FATAL: null java.lang.NullPointerException at hudson.Launcher$ProcStarter.envs(Launcher.java:302) at com.cloudbees.jenkins.plugins.customtools.CustomToolInstallWrapper$2.launch(CustomToolInstallWrapper.java:186) at hudson.Launcher$ProcStarter.start(Launcher.java:353) at hudson.plugins.android_emulator.SdkInstaller.installComponent(SdkInstaller.java:213) at hudson.plugins.android_emulator.SdkInstaller.doInstall(SdkInstaller.java:94) at hudson.plugins.android_emulator.SdkInstaller.install(SdkInstaller.java:66) at hudson.plugins.android_emulator.AndroidEmulator.setUp(AndroidEmulator.java:241) at hudson.model.Build$BuildExecution.doRun(Build.java:154) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:586) at hudson.model.Run.execute(Run.java:1603) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:247)

          Oleg Nenashev added a comment -

          Reassigned issue

          Oleg Nenashev added a comment - Reassigned issue

          Anton Lundin added a comment -

          customtools togeather with ShiningPanda Plugin also triggers the same codepath:


          $ /usr/bin/python /home/jenkins/jenkins/shiningpanda/jobs/e0427f50/virtualenv.py --distribute --system-site-packages /home/jenkins/jenkins/shiningpanda/jobs/e0427f50/virtualenvs/d41d8cd9
          PYTHONHOME is set. You must activate the virtualenv before using it
          New python executable in /home/jenkins/jenkins/shiningpanda/jobs/e0427f50/virtualenvs/d41d8cd9/bin/python
          Installing distribute...........................................................................................................................................................................................................................done.
          Installing pip...........done.
          FATAL: null
          java.lang.NullPointerException
          at hudson.Launcher$ProcStarter.envs(Launcher.java:302)
          at com.cloudbees.jenkins.plugins.customtools.CustomToolInstallWrapper$2.launch(CustomToolInstallWrapper.java:204)
          at hudson.Launcher$ProcStarter.start(Launcher.java:353)
          at hudson.Launcher$ProcStarter.join(Launcher.java:360)
          at jenkins.plugins.shiningpanda.utils.LauncherUtil.createSymlink(LauncherUtil.java:98)
          at jenkins.plugins.shiningpanda.interpreters.Virtualenv.create(Virtualenv.java:292)
          at jenkins.plugins.shiningpanda.builders.VirtualenvBuilder.perform(VirtualenvBuilder.java:190)
          at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
          at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:782)
          at hudson.model.Build$BuildExecution.build(Build.java:199)
          at hudson.model.Build$BuildExecution.doRun(Build.java:160)
          at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:567)
          at hudson.model.Run.execute(Run.java:1603)
          at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
          at hudson.model.ResourceController.execute(ResourceController.java:88)
          at hudson.model.Executor.run(Executor.java:246)

          Anton Lundin added a comment - customtools togeather with ShiningPanda Plugin also triggers the same codepath: $ /usr/bin/python /home/jenkins/jenkins/shiningpanda/jobs/e0427f50/virtualenv.py --distribute --system-site-packages /home/jenkins/jenkins/shiningpanda/jobs/e0427f50/virtualenvs/d41d8cd9 PYTHONHOME is set. You must activate the virtualenv before using it New python executable in /home/jenkins/jenkins/shiningpanda/jobs/e0427f50/virtualenvs/d41d8cd9/bin/python Installing distribute...........................................................................................................................................................................................................................done. Installing pip...........done. FATAL: null java.lang.NullPointerException at hudson.Launcher$ProcStarter.envs(Launcher.java:302) at com.cloudbees.jenkins.plugins.customtools.CustomToolInstallWrapper$2.launch(CustomToolInstallWrapper.java:204) at hudson.Launcher$ProcStarter.start(Launcher.java:353) at hudson.Launcher$ProcStarter.join(Launcher.java:360) at jenkins.plugins.shiningpanda.utils.LauncherUtil.createSymlink(LauncherUtil.java:98) at jenkins.plugins.shiningpanda.interpreters.Virtualenv.create(Virtualenv.java:292) at jenkins.plugins.shiningpanda.builders.VirtualenvBuilder.perform(VirtualenvBuilder.java:190) at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20) at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:782) at hudson.model.Build$BuildExecution.build(Build.java:199) at hudson.model.Build$BuildExecution.doRun(Build.java:160) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:567) at hudson.model.Run.execute(Run.java:1603) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:246)

          Oleg Nenashev added a comment -

          Agreed. Issue seems to be on my side. I'll try to debug it within the one or two weeks.

          Oleg Nenashev added a comment - Agreed. Issue seems to be on my side. I'll try to debug it within the one or two weeks.

          Oleg Nenashev added a comment -

          @Anton
          I apologize for a long delay. Seems I've located the issue.
          Will you be able to test a local build with a fix?
          If yes, I'll provide it tomorrow.

          Oleg Nenashev added a comment - @Anton I apologize for a long delay. Seems I've located the issue. Will you be able to test a local build with a fix? If yes, I'll provide it tomorrow.

          Anton Lundin added a comment -

          Just give me a url/git-commit-ish and ill take it for a spin.

          Anton Lundin added a comment - Just give me a url/git-commit-ish and ill take it for a spin.

          Oleg Nenashev added a comment -

          Hello Anton,

          You can find the fix in https://github.com/jenkinsci/customtools-plugin/pull/11
          The fix works well in my tests cases, please check it against your plugins.

          In real, the fix is just a workaround for issue inside Jenkins core or its API, which cannot be avoided on custom-tools side. If you invoke commands on nodes from setUp() functions, you should explicitly call Launcher::envs(...) to prevent NPEs in Launcher::envs(). I'll take care about the issue in the core.

          Oleg Nenashev added a comment - Hello Anton, You can find the fix in https://github.com/jenkinsci/customtools-plugin/pull/11 The fix works well in my tests cases, please check it against your plugins. In real, the fix is just a workaround for issue inside Jenkins core or its API, which cannot be avoided on custom-tools side. If you invoke commands on nodes from setUp() functions, you should explicitly call Launcher::envs(...) to prevent NPEs in Launcher::envs(). I'll take care about the issue in the core.

          Anton Lundin added a comment -

          Tested and now it doesn't die on a npe. Thanks.

          Anton Lundin added a comment - Tested and now it doesn't die on a npe. Thanks.

          Oleg Nenashev added a comment -

          Thank you too.
          Does the plugin install android-emulator properly?

          P.S: I'm going to release a new version soon, so all new bugs will be appreciated

          Oleg Nenashev added a comment - Thank you too. Does the plugin install android-emulator properly? P.S: I'm going to release a new version soon, so all new bugs will be appreciated

          Anton Lundin added a comment -

          pull/11 solves all blocking bugs.

          The part with the default path being emptied is still there, so it fails as shown in error-custom-tools-0.5-snap-with-platform-tools.txt, but i currently workaround that by adding a custom-tool, DEFAULT_PATH which exports the default path.

          It would be neat to get that one fixed to.

          Anton Lundin added a comment - pull/11 solves all blocking bugs. The part with the default path being emptied is still there, so it fails as shown in error-custom-tools-0.5-snap-with-platform-tools.txt, but i currently workaround that by adding a custom-tool, DEFAULT_PATH which exports the default path. It would be neat to get that one fixed to.

          Oleg Nenashev added a comment -

          Moved the initial issue to JENKINS-20560

          Oleg Nenashev added a comment - Moved the initial issue to JENKINS-20560

          Code changed in jenkins
          User: Oleg Nenashev
          Path:
          src/main/java/com/cloudbees/jenkins/plugins/customtools/CustomToolInstallWrapper.java
          src/main/java/com/cloudbees/jenkins/plugins/customtools/DecoratedLauncher.java
          src/test/java/com/cloudbees/jenkins/plugins/customtools/CustomToolInstallWrapperTest.java
          src/test/java/com/cloudbees/jenkins/plugins/customtools/CustomToolInstallerTest.java
          src/test/java/com/synopsys/arc/jenkins/plugins/customtools/util/CommandCallerInstaller.java
          src/test/java/com/synopsys/arc/jenkins/plugins/customtools/util/StubWrapper.java
          http://jenkins-ci.org/commit/customtools-plugin/784a0c091b306d27df4aeb13e54b2ced43fd2db1
          Log:
          [FIXED JENKINS-19506] - Prevent errors on nested wrappers usage
          The provides workarounds for NPEs in Launcher::envs() and also properly calls inner launcher wrappers.
          Resolves https://issues.jenkins-ci.org/browse/JENKINS-19506

          Signed-off-by: Oleg Nenashev <nenashev@synopsys.com>

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Oleg Nenashev Path: src/main/java/com/cloudbees/jenkins/plugins/customtools/CustomToolInstallWrapper.java src/main/java/com/cloudbees/jenkins/plugins/customtools/DecoratedLauncher.java src/test/java/com/cloudbees/jenkins/plugins/customtools/CustomToolInstallWrapperTest.java src/test/java/com/cloudbees/jenkins/plugins/customtools/CustomToolInstallerTest.java src/test/java/com/synopsys/arc/jenkins/plugins/customtools/util/CommandCallerInstaller.java src/test/java/com/synopsys/arc/jenkins/plugins/customtools/util/StubWrapper.java http://jenkins-ci.org/commit/customtools-plugin/784a0c091b306d27df4aeb13e54b2ced43fd2db1 Log: [FIXED JENKINS-19506] - Prevent errors on nested wrappers usage The provides workarounds for NPEs in Launcher::envs() and also properly calls inner launcher wrappers. Resolves https://issues.jenkins-ci.org/browse/JENKINS-19506 Signed-off-by: Oleg Nenashev <nenashev@synopsys.com>

          Code changed in jenkins
          User: Oleg Nenashev
          Path:
          src/main/java/com/cloudbees/jenkins/plugins/customtools/CustomToolInstallWrapper.java
          src/main/java/com/cloudbees/jenkins/plugins/customtools/DecoratedLauncher.java
          src/test/java/com/cloudbees/jenkins/plugins/customtools/CustomToolInstallWrapperTest.java
          src/test/java/com/cloudbees/jenkins/plugins/customtools/CustomToolInstallerTest.java
          src/test/java/com/synopsys/arc/jenkins/plugins/customtools/util/CommandCallerInstaller.java
          src/test/java/com/synopsys/arc/jenkins/plugins/customtools/util/StubWrapper.java
          http://jenkins-ci.org/commit/customtools-plugin/5d9a150ffd107c1a247c0b54e57e16ed38671a3f
          Log:
          Merge pull request #11 from synopsys-arc-oss/nested_wrappers_fix

          [FIXED JENKINS-19506] - Prevent errors on nested wrappers usage

          Compare: https://github.com/jenkinsci/customtools-plugin/compare/dcf9ac719bd1...5d9a150ffd10

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Oleg Nenashev Path: src/main/java/com/cloudbees/jenkins/plugins/customtools/CustomToolInstallWrapper.java src/main/java/com/cloudbees/jenkins/plugins/customtools/DecoratedLauncher.java src/test/java/com/cloudbees/jenkins/plugins/customtools/CustomToolInstallWrapperTest.java src/test/java/com/cloudbees/jenkins/plugins/customtools/CustomToolInstallerTest.java src/test/java/com/synopsys/arc/jenkins/plugins/customtools/util/CommandCallerInstaller.java src/test/java/com/synopsys/arc/jenkins/plugins/customtools/util/StubWrapper.java http://jenkins-ci.org/commit/customtools-plugin/5d9a150ffd107c1a247c0b54e57e16ed38671a3f Log: Merge pull request #11 from synopsys-arc-oss/nested_wrappers_fix [FIXED JENKINS-19506] - Prevent errors on nested wrappers usage Compare: https://github.com/jenkinsci/customtools-plugin/compare/dcf9ac719bd1...5d9a150ffd10

          Oleg Nenashev added a comment -

          @Anton
          https://github.com/jenkinsci/customtools-plugin/pull/13 provides a fix for your initial issue

          Oleg Nenashev added a comment - @Anton https://github.com/jenkinsci/customtools-plugin/pull/13 provides a fix for your initial issue

            oleg_nenashev Oleg Nenashev
            glance Anton Lundin
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: