Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-24810

The "invoke build for each property file" ParameterFactory Generates More than One Set of Builds, Doesn't Stop

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      We are using the ParameterFactory, "for each property file, invoke one build," to create builds for a UI testing project. Lately (behavior started last week, no change to plugins or Jenkins version), the ParameterFactory does not stop generating sets of builds. After reaching the Parameterized Trigger build step, a correct number of builds are generated (about 72 builds of one project with different parameters), but after an undetermined amount of time between 1 and 10 minutes, another set of 72 builds are generated. This continues in a loop, without stopping, with an observed maximum of over 3000 builds of the same project.

      More use case information on the parameterized trigger build step:

      • Block until triggered projects finish=false
      • Predefined parameters used=true
      • Parameters from properties file used=true
      • For every property file, invoke one build=true
        • File pattern: "test_list/current_run/*.properties"
        • Action to perform when no files found: Fail the build step

      Logger output for "hudson.plugins.parameterizedtrigger":

      Sep 22, 2014 8:55:20 AM INFO hudson.plugins.parameterizedtrigger.FileBuildParameterFactory getParameters
      null
      Sep 22, 2014 8:55:20 AM INFO hudson.plugins.parameterizedtrigger.FileBuildParameterFactory getParameters
      null
      Sep 22, 2014 8:55:20 AM INFO hudson.plugins.parameterizedtrigger.FileBuildParameterFactory getParameters
      null
      Sep 22, 2014 8:55:20 AM INFO hudson.plugins.parameterizedtrigger.FileBuildParameterFactory getParameters
      null
      Sep 22, 2014 8:55:20 AM INFO hudson.plugins.parameterizedtrigger.FileBuildParameterFactory getParameters
      null
      Sep 22, 2014 8:55:20 AM INFO hudson.plugins.parameterizedtrigger.FileBuildParameterFactory getParameters
      null

      (This continues for another ten minutes with the same regularity)

        Attachments

          Activity

          Hide
          cbarr Clinton Barr added a comment - - edited

          New information:

          Ignore everything before this.

          Environment:
          1. Parent project runs on master, child project runs on slave (allocated by node label)
          2. Windows 2012 Server master, Windows 2008 R2 slave
          3. Jenkins v1.581, Parameterized Trigger Plugin v2.25
          4. Parent project copies one .properties file into workspace, triggers child project from it using ParameterFactory, finishes
          5. See attachments for configs and properties file used to trigger child project

          I can condense the behavior of this issue down to the following:

          • Each child build triggered by the parent has a chance to be built twice.
          • The duplicate chance occurs each time a child build finishes and the executor of the slave that just built it is freed (to consume a queued child build).
          • The parent project does not need to be running for duplicate child builds to be created.
          • Jenkins Master reboots have no effect (we reboot our master each day)

          EDIT: Couldn't remove obsolete attachments. #5 above is referring to ParentProject_config.xml, ChildProject_config.xml and 0.properties.

          Show
          cbarr Clinton Barr added a comment - - edited New information: Ignore everything before this. Environment: 1. Parent project runs on master, child project runs on slave (allocated by node label) 2. Windows 2012 Server master, Windows 2008 R2 slave 3. Jenkins v1.581, Parameterized Trigger Plugin v2.25 4. Parent project copies one .properties file into workspace, triggers child project from it using ParameterFactory, finishes 5. See attachments for configs and properties file used to trigger child project I can condense the behavior of this issue down to the following: Each child build triggered by the parent has a chance to be built twice . The duplicate chance occurs each time a child build finishes and the executor of the slave that just built it is freed (to consume a queued child build). The parent project does not need to be running for duplicate child builds to be created. Jenkins Master reboots have no effect (we reboot our master each day) EDIT: Couldn't remove obsolete attachments. #5 above is referring to ParentProject_config.xml, ChildProject_config.xml and 0.properties.
          Hide
          danielbeck Daniel Beck added a comment -

          I assume for the condition to occur you need more than one properties file, so there can actually be queued builds?

          Does child need the node parameter?

          Show
          danielbeck Daniel Beck added a comment - I assume for the condition to occur you need more than one properties file, so there can actually be queued builds? Does child need the node parameter?
          Hide
          cbarr Clinton Barr added a comment -

          I just ran the setup again with only one properties file and it built two child project jobs.

          As for your second question, yes - I can't believe I didn't mention this before. The child project accepts a node label-type parameter. Just now, I found that without the node label parameter, this bug does not show up. If I remove the node label parameter from the child (but still pass it in the .properties file), no build duplication occurs. If I restore the node label parameter and manually build the child project, entering an existing node, no build duplication occurs.

          If the parent project triggers a child project build using a .properties file (that contains an existing node label), build duplication DOES occur.

          By the way, thank you both for paying attention to this issue.

          Show
          cbarr Clinton Barr added a comment - I just ran the setup again with only one properties file and it built two child project jobs. As for your second question, yes - I can't believe I didn't mention this before. The child project accepts a node label-type parameter. Just now, I found that without the node label parameter, this bug does not show up . If I remove the node label parameter from the child (but still pass it in the .properties file), no build duplication occurs. If I restore the node label parameter and manually build the child project, entering an existing node, no build duplication occurs. If the parent project triggers a child project build using a .properties file (that contains an existing node label), build duplication DOES occur. By the way, thank you both for paying attention to this issue.
          Hide
          ikedam ikedam added a comment -

          It sounds the correct behavior of nodelabelparameter plugin.
          You should uncheck "Run on all nodes matching the label" of NODE_LABEL.

          Show
          ikedam ikedam added a comment - It sounds the correct behavior of nodelabelparameter plugin. You should uncheck "Run on all nodes matching the label" of NODE_LABEL.
          Hide
          cbarr Clinton Barr added a comment -

          That's strange. After unchecking "Run on all nodes...," my original projects behaved correctly and no duplicate builds were generated. I didn't find this behavior stated in the NodeLabel Parameter plugin documentation. When would duplicate builds ever be intended? This doesn't explain why when I had upwards of 70 property files to use with the ParameterFactory, the triggered builds cascaded beyond just one extra build (the original problem was discovered by 3000+ builds in queue).

          Also, I believe I had this setting checked from the beginning (I've got at least a 2-month, 40-build history through development), but only just last week observed this effect. Well, I can't explain it, but thank you all for the attention and help.

          TL;DR (For others not expecting the behavior in the description) - If you are using a Property file ParameterFactory to trigger a child project with a node label parameter, if you check "Run on all nodes matching the label," you can expect multiple builds of the child project.

          Show
          cbarr Clinton Barr added a comment - That's strange. After unchecking "Run on all nodes...," my original projects behaved correctly and no duplicate builds were generated. I didn't find this behavior stated in the NodeLabel Parameter plugin documentation. When would duplicate builds ever be intended? This doesn't explain why when I had upwards of 70 property files to use with the ParameterFactory, the triggered builds cascaded beyond just one extra build (the original problem was discovered by 3000+ builds in queue). Also, I believe I had this setting checked from the beginning (I've got at least a 2-month, 40-build history through development), but only just last week observed this effect. Well, I can't explain it, but thank you all for the attention and help. TL;DR (For others not expecting the behavior in the description) - If you are using a Property file ParameterFactory to trigger a child project with a node label parameter, if you check "Run on all nodes matching the label," you can expect multiple builds of the child project.

            People

            Assignee:
            Unassigned Unassigned
            Reporter:
            cbarr Clinton Barr
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved: