Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-27948

Script timeout during start of configure a jenkins job

    XMLWordPrintable

Details

    Description

      I updated jenkins from version 1.596 to version 1.609.
      No I get a script timeout if I try to configure some of the larger jobs.
      I can configure small jobs without any problems.
      I increase the firefox script timeout from 10 seconds to 100 seconds.
      Now I can configure the jenkins jobs, but it takes a very long time to start editing.
      With version 1.596 it was possible to start configuring of the same larger jobs in the predefined timout of 10 seconds

      Attachments

        Issue Links

          Activity

            rw250155 Reiner Wirtz created issue -
            danielbeck Daniel Beck added a comment -

            Please provide a list of installed plugins and their versions, as well as a sample config.xml of such a 'large job'.

            danielbeck Daniel Beck added a comment - Please provide a list of installed plugins and their versions, as well as a sample config.xml of such a 'large job'.
            rw250155 Reiner Wirtz added a comment - - edited

            The list of the installed plugins (copied from display of the system properties):

            ant 1.2 true true
            antisamy-markup-formatter 1.3 true true
            any-buildstep 0.1 true false
            artifactdeployer 0.33 true false
            audit-trail 2.1 true false
            backup 1.6.1 true false
            batch-task 1.17 true false
            build-blocker-plugin 1.6 true false
            build-environment 1.4 true false
            build-pipeline-plugin 1.4.7 true false
            build-user-vars-plugin 1.4 true false
            clone-workspace-scm 0.6 true false
            conditional-buildstep 1.3.3 true false
            config-file-provider 2.7.5 true false
            copy-to-slave 1.4.4 true false
            copyartifact 1.35 true false
            credentials 1.22 true true
            custom-tools-plugin 0.4.4 true false
            CustomHistory 1.6 true false
            cvs 2.12 true true
            dashboard-view 2.9.4 true false
            delivery-pipeline-plugin 0.8.11 true false
            dependencyanalyzer 0.7 true false
            deploy 1.10 true false
            disk-usage 0.25 true false
            downstream-buildview 1.9 true false
            downstream-ext 1.8 true false
            dynamic-axis 1.0.3 true false
            dynamic_extended_choice_parameter 1.0.1 true false
            dynamicparameter 0.2.0 true false
            email-ext 2.39.3 true false
            envfile 1.2 true false
            envinject 1.91.2 true false
            extended-choice-parameter 0.35 true false
            extensible-choice-parameter 1.2.2 true false
            external-monitor-job 1.4 true true
            flexible-publish 0.15.1 true false
            git 2.3.5 true false
            git-client 1.16.1 true false
            git-server 1.6 true false
            groovy 1.24 true false
            groovy-label-assignment 1.1.0 true false
            groovyaxis 0.3 true false
            ivy 1.24 true false
            javadoc 1.3 true true
            job-log-logger-plugin 1.0 true false
            job-node-stalker 1.0.3 true false
            jobConfigHistory 2.10 true false
            jquery 1.11.2-0 true false
            jquery-ui 1.0.2 true false
            junit 1.5 true true
            labeled-test-groups-publisher 1.2.7 true false
            ldap 1.11 true true
            m2-repo-reaper 1.0 true false
            mailcommander 1.0.0 true false
            mailer 1.15 true true
            managed-scripts 1.1.1 true false
            mantis 0.26 true false
            mapdb-api 1.0.6.0 true false
            matrix-auth 1.2 true true
            matrix-combinations-parameter 1.0.7 true false
            matrix-project 1.4.1 true false
            Matrix-sorter-plugin 1.1 true false
            maven-plugin 2.9 true true
            metadata 1.1.0b true false
            metrics 3.0.9 true false
            monitoring 1.55.0 true false
            nodelabelparameter 1.5.1 true false
            pam-auth 1.2 true true
            parameterized-trigger 2.26 true false
            periodicbackup 1.3 true false
            postbuild-task 1.8 true false
            postbuildscript 0.17 true false
            PrioritySorter 2.11 true false
            role-strategy 2.2.0 true false
            run-condition 1.0 true false
            scm-api 0.2 true false
            script-security 1.13 true false
            scriptler 2.7 true false
            selection-tasks-plugin 1.0 true false
            shared-objects 0.44 true false
            show-build-parameters 1.0 true false
            skip-certificate-check 1.0 true false
            sonar 2.2 true false
            ssh-credentials 1.11 true true
            ssh-slaves 1.9 true true
            statusmonitor 1.3 true false
            strawboss 1.3 true false
            subversion 2.5 true true
            support-core 2.20 true false
            svn-tag 1.17 true false
            testlink 3.10 true false
            throttle-concurrents 1.8.4 true false
            token-macro 1.10 true false
            translation 1.12 true true
            uno-choice 0.24 true false
            update-sites-manager 1.0.1 true false
            validating-string-parameter 2.3 true false
            vsphere-cloud 2.4 true false
            weblogic-deployer-plugin 3.2 true false
            websphere-deployer 1.3-SNAPSHOT (private-08/05/2014 09:34-nothhard) true false
            windows-slaves 1.0 true false
            ws-cleanup 0.25 true false
            xunit 1.94 true false

            rw250155 Reiner Wirtz added a comment - - edited The list of the installed plugins (copied from display of the system properties): ant 1.2 true true antisamy-markup-formatter 1.3 true true any-buildstep 0.1 true false artifactdeployer 0.33 true false audit-trail 2.1 true false backup 1.6.1 true false batch-task 1.17 true false build-blocker-plugin 1.6 true false build-environment 1.4 true false build-pipeline-plugin 1.4.7 true false build-user-vars-plugin 1.4 true false clone-workspace-scm 0.6 true false conditional-buildstep 1.3.3 true false config-file-provider 2.7.5 true false copy-to-slave 1.4.4 true false copyartifact 1.35 true false credentials 1.22 true true custom-tools-plugin 0.4.4 true false CustomHistory 1.6 true false cvs 2.12 true true dashboard-view 2.9.4 true false delivery-pipeline-plugin 0.8.11 true false dependencyanalyzer 0.7 true false deploy 1.10 true false disk-usage 0.25 true false downstream-buildview 1.9 true false downstream-ext 1.8 true false dynamic-axis 1.0.3 true false dynamic_extended_choice_parameter 1.0.1 true false dynamicparameter 0.2.0 true false email-ext 2.39.3 true false envfile 1.2 true false envinject 1.91.2 true false extended-choice-parameter 0.35 true false extensible-choice-parameter 1.2.2 true false external-monitor-job 1.4 true true flexible-publish 0.15.1 true false git 2.3.5 true false git-client 1.16.1 true false git-server 1.6 true false groovy 1.24 true false groovy-label-assignment 1.1.0 true false groovyaxis 0.3 true false ivy 1.24 true false javadoc 1.3 true true job-log-logger-plugin 1.0 true false job-node-stalker 1.0.3 true false jobConfigHistory 2.10 true false jquery 1.11.2-0 true false jquery-ui 1.0.2 true false junit 1.5 true true labeled-test-groups-publisher 1.2.7 true false ldap 1.11 true true m2-repo-reaper 1.0 true false mailcommander 1.0.0 true false mailer 1.15 true true managed-scripts 1.1.1 true false mantis 0.26 true false mapdb-api 1.0.6.0 true false matrix-auth 1.2 true true matrix-combinations-parameter 1.0.7 true false matrix-project 1.4.1 true false Matrix-sorter-plugin 1.1 true false maven-plugin 2.9 true true metadata 1.1.0b true false metrics 3.0.9 true false monitoring 1.55.0 true false nodelabelparameter 1.5.1 true false pam-auth 1.2 true true parameterized-trigger 2.26 true false periodicbackup 1.3 true false postbuild-task 1.8 true false postbuildscript 0.17 true false PrioritySorter 2.11 true false role-strategy 2.2.0 true false run-condition 1.0 true false scm-api 0.2 true false script-security 1.13 true false scriptler 2.7 true false selection-tasks-plugin 1.0 true false shared-objects 0.44 true false show-build-parameters 1.0 true false skip-certificate-check 1.0 true false sonar 2.2 true false ssh-credentials 1.11 true true ssh-slaves 1.9 true true statusmonitor 1.3 true false strawboss 1.3 true false subversion 2.5 true true support-core 2.20 true false svn-tag 1.17 true false testlink 3.10 true false throttle-concurrents 1.8.4 true false token-macro 1.10 true false translation 1.12 true true uno-choice 0.24 true false update-sites-manager 1.0.1 true false validating-string-parameter 2.3 true false vsphere-cloud 2.4 true false weblogic-deployer-plugin 3.2 true false websphere-deployer 1.3-SNAPSHOT (private-08/05/2014 09:34-nothhard) true false windows-slaves 1.0 true false ws-cleanup 0.25 true false xunit 1.94 true false
            rw250155 Reiner Wirtz added a comment -

            The attached zip file contains the config.xml file from a job called DOXiS4_CSB-Test-All-Comb".
            Opening this job for configuration exceeds the limit of 100 seconds configured in my firefox-browser.
            During this time one core of the quad-core-cpu is busy for the whole time.

            rw250155 Reiner Wirtz added a comment - The attached zip file contains the config.xml file from a job called DOXiS4_CSB-Test-All-Comb". Opening this job for configuration exceeds the limit of 100 seconds configured in my firefox-browser. During this time one core of the quad-core-cpu is busy for the whole time.
            rw250155 Reiner Wirtz made changes -
            Field Original Value New Value
            Attachment config.zip [ 29457 ]
            rw250155 Reiner Wirtz added a comment -

            I increased the timeout value for scripts to run to 200 seconds.
            The start to configure the job I added in the attachments took 180 seconds (yes 3 minutes) on my machine.

            rw250155 Reiner Wirtz added a comment - I increased the timeout value for scripts to run to 200 seconds. The start to configure the job I added in the attachments took 180 seconds (yes 3 minutes) on my machine.
            rw250155 Reiner Wirtz added a comment -

            If I should present the data in a special way or format, please feel free to contact me.

            rw250155 Reiner Wirtz added a comment - If I should present the data in a special way or format, please feel free to contact me.
            danielbeck Daniel Beck added a comment -

            Haven't had time to investigate, but you could do some of that yourself:
            While the form is submitted and the page is loading, repeatedly load the /threadDump URL and look for a thread named Handling POST /job/JOBNAME/configSubmit or something similar. Look where it's stuck processing for most of the time.

            danielbeck Daniel Beck added a comment - Haven't had time to investigate, but you could do some of that yourself: While the form is submitted and the page is loading, repeatedly load the /threadDump URL and look for a thread named Handling POST /job/JOBNAME/configSubmit or something similar. Look where it's stuck processing for most of the time.
            rw250155 Reiner Wirtz added a comment -

            Excuse my late answer.
            I am glad that I can help.
            I started configuring the job I added in this call as example.
            The browser was about 3 minutes in the state loading data.
            At this time a call to the threadDump URL does not show a thread "Handling POST" or "configSubmit".
            The only Thread with Hanling was the threadDump itself.

            It look like jenkins generates Javascript, which runs all the time in the browser.
            It seems it runs a lot longer, if there a more objects or lines to display.

            rw250155 Reiner Wirtz added a comment - Excuse my late answer. I am glad that I can help. I started configuring the job I added in this call as example. The browser was about 3 minutes in the state loading data. At this time a call to the threadDump URL does not show a thread "Handling POST" or "configSubmit". The only Thread with Hanling was the threadDump itself. It look like jenkins generates Javascript, which runs all the time in the browser. It seems it runs a lot longer, if there a more objects or lines to display.
            rw250155 Reiner Wirtz added a comment -

            Skript: http://sst-bld-master.sst.loca…/0da8a904/scripts/prototype.js:2188

            It seems the script is calculating the size of the objects to display.

            rw250155 Reiner Wirtz added a comment - Skript: http://sst-bld-master.sst.loca …/0da8a904/scripts/prototype.js:2188 It seems the script is calculating the size of the objects to display.
            rw250155 Reiner Wirtz added a comment -

            Depending on the time, I configure as timeout value for the time a script may run, there are different javascripts active at the time the browser stops the script.

            rw250155 Reiner Wirtz added a comment - Depending on the time, I configure as timeout value for the time a script may run, there are different javascripts active at the time the browser stops the script.

            I observe the same issue also in a different scenario: being in a job with a large list of completed builds, clicking on 'more...' at the bottom of the build list results in the same annoying popups with timeouts in prototype.js. Firefox developer tools console logs:

            Error: Script terminated by timeout at:
            Element.Methods.removeClassName@https://server:8080/static/63938483/scripts/prototype.js:2371:9
            getElementOverflowParams@https://server:8080/static/63938483/scripts/hudson-behavior.js:1952:5
            checkRowCellOverflows@https://server:8080/static/63938483/scripts/hudson-behavior.js:1693:37
            checkAllRowCellOverflows@https://server:8080/static/63938483/scripts/hudson-behavior.js:1883:13
            updateBuilds/<.onSuccess@https://server:8080/static/63938483/scripts/hudson-behavior.js:1927:21
            Ajax.Request<.respondToReadyState@https://server:8080/static/63938483/scripts/prototype.js:1655:10
            Ajax.Request<.onStateChange@https://server:8080/static/63938483/scripts/prototype.js:1600:7
            bind/<@https://server:8080/static/63938483/scripts/prototype.js:414:14

            raspy Krzysztof Malinowski added a comment - I observe the same issue also in a different scenario: being in a job with a large list of completed builds, clicking on 'more...' at the bottom of the build list results in the same annoying popups with timeouts in prototype.js. Firefox developer tools console logs: Error: Script terminated by timeout at: Element.Methods.removeClassName@ https://server:8080/static/63938483/scripts/prototype.js:2371:9 getElementOverflowParams@ https://server:8080/static/63938483/scripts/hudson-behavior.js:1952:5 checkRowCellOverflows@ https://server:8080/static/63938483/scripts/hudson-behavior.js:1693:37 checkAllRowCellOverflows@ https://server:8080/static/63938483/scripts/hudson-behavior.js:1883:13 updateBuilds/<.onSuccess@ https://server:8080/static/63938483/scripts/hudson-behavior.js:1927:21 Ajax.Request<.respondToReadyState@ https://server:8080/static/63938483/scripts/prototype.js:1655:10 Ajax.Request<.onStateChange@ https://server:8080/static/63938483/scripts/prototype.js:1600:7 bind/<@ https://server:8080/static/63938483/scripts/prototype.js:414:14
            danielbeck Daniel Beck added a comment -

            same issue also in a different scenario

            In other words, a completely different issue: JENKINS-26445

            danielbeck Daniel Beck added a comment - same issue also in a different scenario In other words, a completely different issue: JENKINS-26445

            Thank you, I could not find it However, I actually get both issues, JENKINS-26445 and JENKINS-27948, so I would suggest they might be related.

            raspy Krzysztof Malinowski added a comment - Thank you, I could not find it However, I actually get both issues, JENKINS-26445 and JENKINS-27948 , so I would suggest they might be related.
            brianeray Brian Ray added a comment - - edited

            Our upgrade to 1.609.1 seems to be exhibiting the same JS timeouts on an admittedly too-big job config, regardless of the number of builds. The timeouts usually happen in codemirror.js. New to JIRA and am wondering if adding a copy of the config scrubbed of proprietary information will help. I can gather other information about our install as well: plugins, client and server attributes, etc.

            Luckily the workaround of setting JS timeouts in Firefox is working. (32 bit Windows v38.0.5, all three dom.max_*_script_run_time params set to 120.) The page is coming up somewhere in the neighborhood of 100 seconds.

            If it would help to attach our bits of evidence, let me know.

            brianeray Brian Ray added a comment - - edited Our upgrade to 1.609.1 seems to be exhibiting the same JS timeouts on an admittedly too-big job config, regardless of the number of builds. The timeouts usually happen in codemirror.js. New to JIRA and am wondering if adding a copy of the config scrubbed of proprietary information will help. I can gather other information about our install as well: plugins, client and server attributes, etc. Luckily the workaround of setting JS timeouts in Firefox is working. (32 bit Windows v38.0.5, all three dom.max_*_script_run_time params set to 120.) The page is coming up somewhere in the neighborhood of 100 seconds. If it would help to attach our bits of evidence, let me know.

            I can configure small jobs without any problems.
            I increase the firefox script timeout from 10 seconds to 100 seconds.
            Now I can configure the jenkins jobs, but it takes a very long time to start editing.

            Same issue with LTS 1.609.1 & LTS 1.625.1 too.

            dilipm79 Dilip Mahadevappa added a comment - I can configure small jobs without any problems. I increase the firefox script timeout from 10 seconds to 100 seconds. Now I can configure the jenkins jobs, but it takes a very long time to start editing. Same issue with LTS 1.609.1 & LTS 1.625.1 too.
            scoheb Scott Hebert added a comment -

            We were able to workaround this by doing the following:

            Navigate to:

            • Manage Jenkins
            • Configure Global Security
            • In the Markup Formatter section, ensure:
              • Safe HTML is selected
              • Click on Disable syntax highlighting

            With this change, we went from a config page load that would timeout in Firefox to one that loads in under 20s.

            scoheb Scott Hebert added a comment - We were able to workaround this by doing the following: Navigate to: Manage Jenkins Configure Global Security In the Markup Formatter section, ensure: Safe HTML is selected Click on Disable syntax highlighting With this change, we went from a config page load that would timeout in Firefox to one that loads in under 20s.
            scoheb Scott Hebert made changes -
            Component/s antisamy-markup-formatter-plugin [ 18424 ]
            Component/s core [ 15593 ]

            Suggestion above by @scotthebert works.

            yeongsheng_tan Yeong Sheng Tan added a comment - Suggestion above by @scotthebert works.
            rw250155 Reiner Wirtz added a comment -

            I have a jenkins job, which has about 20 steps.
            Every step has a condition depending on a expression with jenkins variables.
            About 15 of the steps call simple a unix shell script, two others system groovy scripts.
            The size of the unix shell scripts vary from 5 to 30 lines.

            To open this job for configure takes more than 2 minutes.
            I haven't seen a difference in time if I use the workaround mentioned by Scott Hebert or not.

            rw250155 Reiner Wirtz added a comment - I have a jenkins job, which has about 20 steps. Every step has a condition depending on a expression with jenkins variables. About 15 of the steps call simple a unix shell script, two others system groovy scripts. The size of the unix shell scripts vary from 5 to 30 lines. To open this job for configure takes more than 2 minutes. I haven't seen a difference in time if I use the workaround mentioned by Scott Hebert or not.
            chriscralle Chris Cralle added a comment -

            There is a problem with CodeMirror and textarea windows.
            I just upgraded to LTS 1.642.3 this weekend and getting a little rubbing from my comrades about how long the configure load times are.
            I have one 2x4 Matrix multi-step job with conditionals that have shell, python, and windows batch executes. The configure page for this job generates 8-10 timeouts in codemirror.js and takes minutes to load.
            Common timeouts are on lines 1247, 1207, 853
            These deal with the gutter and scrollbar as far as I can tell.

            I've spent Saturday trying everything I can to somehow just DISABLE codemirror. But since it's core, not a plugin, and there's no exposed configuration for it, I'm pretty much stuck with it's behavior. Is it possible to turn it off?
            The setting in security for syntax highlighting is not related, the number of builds or history is not related. This is purely codemirror barfing on complex jobs with recursive textareas.

            I'm surprised codemirror is not a tag/label on this ticket.

            I am creating a sanitized config.xml which I hope to upload to this ticket in the hope that it helps with a resolution.

            chriscralle Chris Cralle added a comment - There is a problem with CodeMirror and textarea windows. I just upgraded to LTS 1.642.3 this weekend and getting a little rubbing from my comrades about how long the configure load times are. I have one 2x4 Matrix multi-step job with conditionals that have shell, python, and windows batch executes. The configure page for this job generates 8-10 timeouts in codemirror.js and takes minutes to load. Common timeouts are on lines 1247, 1207, 853 These deal with the gutter and scrollbar as far as I can tell. I've spent Saturday trying everything I can to somehow just DISABLE codemirror. But since it's core, not a plugin, and there's no exposed configuration for it, I'm pretty much stuck with it's behavior. Is it possible to turn it off? The setting in security for syntax highlighting is not related, the number of builds or history is not related. This is purely codemirror barfing on complex jobs with recursive textareas. I'm surprised codemirror is not a tag/label on this ticket. I am creating a sanitized config.xml which I hope to upload to this ticket in the hope that it helps with a resolution.
            chriscralle Chris Cralle made changes -
            Labels configuration jenkins performance codemirror configuration jenkins performance
            danielbeck Daniel Beck made changes -
            Component/s core [ 15593 ]
            Component/s antisamy-markup-formatter-plugin [ 18424 ]
            chriscralle Chris Cralle made changes -
            Link This issue is related to JENKINS-32027 [ JENKINS-32027 ]
            cmoroexpedia Carlos Moro added a comment -

            I followed the instructions from Scott (Global Security -> In the Markup Formatter section, ensure: Safe HTML is selected and Click on Disable syntax highlighting) and now I can configure jobs in Firefox, but Chrome and Safari are still broken. At least we have a workaround.

            cmoroexpedia Carlos Moro added a comment - I followed the instructions from Scott (Global Security -> In the Markup Formatter section, ensure: Safe HTML is selected and Click on Disable syntax highlighting) and now I can configure jobs in Firefox, but Chrome and Safari are still broken. At least we have a workaround.
            hai_lh hai li added a comment -

            Any idea on how to turn off codemirror to bypass this issue?

            hai_lh hai li added a comment - Any idea on how to turn off codemirror to bypass this issue?
            rw250155 Reiner Wirtz added a comment -

            Hello,

            it seems that the bug is fixed.
            I have two jobs, which took about 10 minutes to open for edit with jenkins up to 2.11
            Since I use 2.12 the same jobs takes about 10 to 20 seconds to open for edit.
            And the browser does'n allocated about 10 GB of memory to open the job any more.

            rw250155 Reiner Wirtz added a comment - Hello, it seems that the bug is fixed. I have two jobs, which took about 10 minutes to open for edit with jenkins up to 2.11 Since I use 2.12 the same jobs takes about 10 to 20 seconds to open for edit. And the browser does'n allocated about 10 GB of memory to open the job any more.
            danielbeck Daniel Beck added a comment -

            Resolved as JENKINS-32027 in 2.12.

            danielbeck Daniel Beck added a comment - Resolved as JENKINS-32027 in 2.12.
            danielbeck Daniel Beck made changes -
            Resolution Fixed [ 1 ]
            Status Open [ 1 ] Resolved [ 5 ]
            rtyler R. Tyler Croy made changes -
            Workflow JNJira [ 162547 ] JNJira + In-Review [ 196979 ]

            People

              Unassigned Unassigned
              rw250155 Reiner Wirtz
              Votes:
              6 Vote for this issue
              Watchers:
              14 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: