Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-33672

File Setting Failure in Extended Choice Parameter Plugin on Slave

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • None

      I have two Jenkins instances, a QA system with a single machine, and a Production system with slaves.

      I developed a simulation on the QA system that would read a configuration file in the workspace.
      I used the full path to the location in the machine's internal file system.

      The plugin recognizes the file and configures the UI (a 3 level deep multi-level single select) correctly.
      The location is: /var/lib/jenkins/workspace/simulate-chef-push/jenkins-chef/config/extended-choice-config.txt

      I then went to try to set up the same configuration a new job, with additional features such as the Chef identity, on a designated slave on the Production machine.
      The location is set to /home/babylon/jenkins-slave/jenkins-slave/workspace/chef-push-job/config/extended-choice-config.txt.

      I get the error
      /home/babylon/jenkins-slave/jenkins-slave/workspace/chef-push-job/config/extended-choice-config.txt doesnt seem to exist..

      I can, of course verify that the path on the QA Jenkins is correct. (The plugin can find the path, after all!)

      I can ALSO verify that the path was created on the slave machine by logging into it:

      -sh-4.2$ sudo su babylon
      sh-4.2$ ls /home/babylon/jenkins-slave/jenkins-slave/workspace/chef-push-job/config/extended-choice-config.txt
      /home/babylon/jenkins-slave/jenkins-slave/workspace/chef-push-job/config/extended-choice-config.txt
      

      I can also verify that the file is present and readable in another way by actually outputting the contents of the file in a Shell in the job itself.

      + cat /home/babylon/jenkins-slave/jenkins-slave/workspace/chef-push-job/config/extended-choice-config.txt
      Organization	Job	Quorum
      orgA	jobA1	orgA:jobA1:90%
      orgA	jobA1	orgA:jobA1:85%
      orgA	jobA1	orgA:jobA1:100%
      orgA	jobA2	orgA:jobA2:90%
      orgA	jobA3	orgA:jobA3:90%
      orgB	jobB1	orgB:jobB1:90%
      orgB	jobB1	orgB:jobB1:80%
      orgB	jobB2	orgB:jobB2:70%
      orgB	jobB3	orgB:jobB3:85%
      orgB	jobB3	orgB:jobB3:75%
      

          [JENKINS-33672] File Setting Failure in Extended Choice Parameter Plugin on Slave

          Ed Smiley added a comment -

          Is there a difference in the way that the lifecycle of the extended choice parameter differs on a machine with slaves?

          Further information:
          The config file is in a Git repo. I will be running the job periodically without parameters on a schedule, and without the workspace being deleted.
          So there should always be a copy of this file in the workspace. As I understand it, the file will be read from the workspace prior to the latest check out.
          That's the behavior I am seeing on the Jenkins system that is working. I put in a build step prior to checkout temporarily to debug. It sees that the file is there.

          Ed Smiley added a comment - Is there a difference in the way that the lifecycle of the extended choice parameter differs on a machine with slaves? Further information: The config file is in a Git repo. I will be running the job periodically without parameters on a schedule, and without the workspace being deleted. So there should always be a copy of this file in the workspace. As I understand it, the file will be read from the workspace prior to the latest check out. That's the behavior I am seeing on the Jenkins system that is working. I put in a build step prior to checkout temporarily to debug. It sees that the file is there.

          Ed Smiley added a comment -

          I attached a config file for this job. I suppressed the Chef identity, git repo and branch and restrict to label for slave values for security reasons.

          Ed Smiley added a comment - I attached a config file for this job. I suppressed the Chef identity, git repo and branch and restrict to label for slave values for security reasons.

          Ed Smiley added a comment - - edited

          I found out another interesting fact.
          Groovy scripts also cannot "see" the file just like the extended choice parameter!
          This has something to do with using the full path!

          Here I cat out the file in a shell before and after running a Groovy script to read it and dump its contents!

          console log

          + cat /home/babylon/jenkins-slave/jenkins-slave/workspace/chef-push-job/config/extended-choice-config.txt
          Organization	Job	Quorum
          orgA	jobA1	orgA:jobA1:90%
          orgA	jobA1	orgA:jobA1:85%
          orgA	jobA1	orgA:jobA1:100%
          orgA	jobA2	orgA:jobA2:90%
          orgA	jobA3	orgA:jobA3:90%
          orgB	jobB1	orgB:jobB1:90%
          orgB	jobB1	orgB:jobB1:80%
          orgB	jobB2	orgB:jobB2:70%
          orgB	jobB3	orgB:jobB3:85%
          orgB	jobB3	orgB:jobB3:75%
          ...
          

          the Groovy script.

          try{
          String fileContents = new File("/home/babylon/jenkins-slave/jenkins-slave/workspace/chef-push-job/config/extended-choice-config.txt").text
          println "fileContents are: "
          println fileContents
          } catch (Exception e){
           println e
          }
          
          java.io.FileNotFoundException: /home/babylon/jenkins-slave/jenkins-slave/workspace/chef-push-job/config/extended-choice-config.txt (Permission denied)
          

          running cat again.

          + cat /home/babylon/jenkins-slave/jenkins-slave/workspace/chef-push-job/config/extended-choice-config.txt
          Organization	Job	Quorum
          orgA	jobA1	orgA:jobA1:90%
          orgA	jobA1	orgA:jobA1:85%
          orgA	jobA1	orgA:jobA1:100%
          orgA	jobA2	orgA:jobA2:90%
          orgA	jobA3	orgA:jobA3:90%
          orgB	jobB1	orgB:jobB1:90%
          orgB	jobB1	orgB:jobB1:80%
          orgB	jobB2	orgB:jobB2:70%
          orgB	jobB3	orgB:jobB3:85%
          orgB	jobB3	orgB:jobB3:75%
          

          Ed Smiley added a comment - - edited I found out another interesting fact. Groovy scripts also cannot "see" the file just like the extended choice parameter! This has something to do with using the full path! Here I cat out the file in a shell before and after running a Groovy script to read it and dump its contents! console log + cat /home/babylon/jenkins-slave/jenkins-slave/workspace/chef-push-job/config/extended-choice-config.txt Organization Job Quorum orgA jobA1 orgA:jobA1:90% orgA jobA1 orgA:jobA1:85% orgA jobA1 orgA:jobA1:100% orgA jobA2 orgA:jobA2:90% orgA jobA3 orgA:jobA3:90% orgB jobB1 orgB:jobB1:90% orgB jobB1 orgB:jobB1:80% orgB jobB2 orgB:jobB2:70% orgB jobB3 orgB:jobB3:85% orgB jobB3 orgB:jobB3:75% ... the Groovy script. try { String fileContents = new File( "/home/babylon/jenkins-slave/jenkins-slave/workspace/chef-push-job/config/extended-choice-config.txt" ).text println "fileContents are: " println fileContents } catch (Exception e){ println e } java.io.FileNotFoundException: /home/babylon/jenkins-slave/jenkins-slave/workspace/chef-push-job/config/extended-choice-config.txt (Permission denied) running cat again. + cat /home/babylon/jenkins-slave/jenkins-slave/workspace/chef-push-job/config/extended-choice-config.txt Organization Job Quorum orgA jobA1 orgA:jobA1:90% orgA jobA1 orgA:jobA1:85% orgA jobA1 orgA:jobA1:100% orgA jobA2 orgA:jobA2:90% orgA jobA3 orgA:jobA3:90% orgB jobB1 orgB:jobB1:90% orgB jobB1 orgB:jobB1:80% orgB jobB2 orgB:jobB2:70% orgB jobB3 orgB:jobB3:85% orgB jobB3 orgB:jobB3:75%

          Ed Smiley added a comment - - edited

          Possible workaround. Hideous kludge....

          I was able to set up a similar job to run on Jenkins master.
          I think the only workaround I can see at this time is to set it up this way, and then use it to send its parameters on another job to run on a slave where all the real work is done.

          I still consider this a BUG.
          We are trying to restrict master so that it does not run any jobs directly.

          I will let you know if this workaround works (I can't see why not).

          Ed Smiley added a comment - - edited Possible workaround. Hideous kludge.... I was able to set up a similar job to run on Jenkins master . I think the only workaround I can see at this time is to set it up this way, and then use it to send its parameters on another job to run on a slave where all the real work is done. I still consider this a BUG. We are trying to restrict master so that it does not run any jobs directly. I will let you know if this workaround works (I can't see why not).

          Ed Smiley added a comment -

          Yes this workaround (aka hideous kludge) was successful.
          So now I have two jobs. One job is the ONLY job running on master.
          It is then calling the other job, which checks out the Git repo again (this time for a Ruby script) to process it.

          However, as your "How to File a Bug Report" documentation explains DON'T CLOSE A BUG just because you have a workaround.

          regards...
          --Ed

          Ed Smiley added a comment - Yes this workaround (aka hideous kludge) was successful. So now I have two jobs. One job is the ONLY job running on master. It is then calling the other job, which checks out the Git repo again (this time for a Ruby script) to process it. However, as your "How to File a Bug Report" documentation explains DON'T CLOSE A BUG just because you have a workaround. regards... --Ed

          vimil added a comment -

          Ed, the plugin runs on the master to display the parameters. Therefore it looks for all files on the master. and not the slave.

          Here is another work around,

          Create one job to run on the slave. however place the config file on the master. You then don't need two jobs.

          vimil added a comment - Ed, the plugin runs on the master to display the parameters. Therefore it looks for all files on the master. and not the slave. Here is another work around, Create one job to run on the slave. however place the config file on the master. You then don't need two jobs.

          Ed Smiley added a comment -

          Thanks vimil,
          You're absolutely right, unless you are running a job to periodically update the configuration file from a Git (or other VCS) repository.
          Many of us are sophisticated enough to be thinking of dynamic, rather than static configurations, and in terms of workspace relative rather than absolute paths. It's unfortunate that the ECP is somewhat funky about that, as it's so useful!

          Ideally you'd want to be able to access the path to a workspace on the slave to do that.

          I would therefore consider this either
          a) a bug
          b) a documentation bug
          c) a feature request to consider making the ECP more flexible in terms of configurations.

          Thanks
          --Ed

          Ed Smiley added a comment - Thanks vimil, You're absolutely right, unless you are running a job to periodically update the configuration file from a Git (or other VCS) repository. Many of us are sophisticated enough to be thinking of dynamic, rather than static configurations, and in terms of workspace relative rather than absolute paths. It's unfortunate that the ECP is somewhat funky about that, as it's so useful! Ideally you'd want to be able to access the path to a workspace on the slave to do that. I would therefore consider this either a) a bug b) a documentation bug c) a feature request to consider making the ECP more flexible in terms of configurations. Thanks --Ed

          vimil added a comment - - edited

          Ed,

          You could also specify an http url path to the file in source control repository, without having to create a job to download the file into the workspace

          The issue with using a workspace relative path is that ECP plugin is invoked before the job is started. Jenkins decides which slave to use only when the job starts, so ECP cannot predict behore-hand which slave's workspace the configuration should be read from.

          vimil added a comment - - edited Ed, You could also specify an http url path to the file in source control repository, without having to create a job to download the file into the workspace The issue with using a workspace relative path is that ECP plugin is invoked before the job is started. Jenkins decides which slave to use only when the job starts, so ECP cannot predict behore-hand which slave's workspace the configuration should be read from.

            vimil vimil
            edsmiley Ed Smiley
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated: