[JENKINS-60142] ConnectionHelper.java constructors call connectionRetry() - Jenkins Jira

Details

Type: Bug
Status: Closed (View Workflow)
Priority: Major
Resolution: Won't Fix
Component/s: p4-plugin
Labels:
- P4_A
- P4_VERIFY

Similar Issues:

Show

Description

The constructor for the ConnectionHelper class calls the 'connectionRetry()' method that opens up a connection to the perforce server regardless of whether the connection is needed or not.

Example in action:

Monitor traffic to the perforce server using a network packet monitor such as wireshark
Connect to Jenkins from a browser.
Find a job that uses the Jenkins Perforce plugin
Click on ANY previous ran build from the left size of the window (i.e. items in the 'Build History')

Wireshark will report a large amount of traffic to the Perforce server

OK you might think that this isn't important, but if you have a Groovy script that parses old build jobs (checking say the environment variables that were used by the build) then each and every job that the script checks will generate hundreds of perforce network connections which are completely pointless. (slowing down the script in the process)

An example groovy script that can be ran from the 'Script Console' window (Start wireshark and monitor network traffic to the perforce server before running the script)

String jobName = 'A_valid_Jenkins_job_that_uses_the_perforce_plugin';

FreeStyleProject buildJob = Jenkins.instance.getItemByFullName(jobName);
if(buildJob) {
     for(FreeStyleBuild job in buildJob.builds) {
		println job.url;
		job.getEnvironment(TaskListener.NULL).each { k, v ->
			println '  | ' + k + ' = ' + v
		}
		println '';
	}
}

There is absolutely no reason why the above script should generate any network traffic to the perforce server.

(NOTE: Re-running the groovy script a 2nd time might produce less network traffic since I think there might be some caching somewhere)

Attachments

Issue Links

is related to

JENKINS-60141 Excessive number of "login -s" requests

Closed

Activity

Ascending order - Click to sort in descending order

Hide

Permalink

Rob Petti added a comment - 2019-11-12 15:26

Wrong plugin. Perforce-plugin is deprecated and no longer supported. Please make sure you are filing tickets under the correct plugin (p4-plugin).

Show

Rob Petti added a comment - 2019-11-12 15:26 Wrong plugin. Perforce-plugin is deprecated and no longer supported. Please make sure you are filing tickets under the correct plugin (p4-plugin).

Hide

Permalink

Karl Wirth added a comment - 2019-11-29 15:43

Hi James Bateman - Sorry for the delay. This one slipped through our monitoring script that looks for new unowned bugs for 'p4-plugin' every hour.

I think this we made a change where to save disk space on the Jenkins server we no longer keep all the change history. I'll try and repro this here.

Show

Karl Wirth added a comment - 2019-11-29 15:43 Hi James Bateman - Sorry for the delay. This one slipped through our monitoring script that looks for new unowned bugs for 'p4-plugin' every hour. I think this we made a change where to save disk space on the Jenkins server we no longer keep all the change history. I'll try and repro this here.

Hide

Permalink

Karl Wirth added a comment - 2019-12-03 17:44 - edited

Confirmed. One invocation of the above script against a freestyle job that had 8 successful executions caused 24 P4 commands to be run:

	2019/12/03 17:32:31 pid 1868 nouser@_____CLIENT_UNSET_____ 127.0.0.1 [jenkins.p4-plugin/1.10.7/Linux] 'user-info'
	2019/12/03 17:32:31 pid 1869 super@_____CLIENT_UNSET_____ 127.0.0.1 [jenkins.p4-plugin/1.10.7/Linux] 'user-login -s'
	2019/12/03 17:32:31 pid 1870 super@_____CLIENT_UNSET_____ 127.0.0.1 [jenkins.p4-plugin/1.10.7/Linux] 'user-login -s'
	2019/12/03 17:32:31 pid 1871 nouser@_____CLIENT_UNSET_____ 127.0.0.1 [jenkins.p4-plugin/1.10.7/Linux] 'user-info'
	2019/12/03 17:32:31 pid 1872 super@_____CLIENT_UNSET_____ 127.0.0.1 [jenkins.p4-plugin/1.10.7/Linux] 'user-login -s'
	2019/12/03 17:32:31 pid 1873 super@_____CLIENT_UNSET_____ 127.0.0.1 [jenkins.p4-plugin/1.10.7/Linux] 'user-login -s'
	2019/12/03 17:32:31 pid 1874 nouser@_____CLIENT_UNSET_____ 127.0.0.1 [jenkins.p4-plugin/1.10.7/Linux] 'user-info'
	2019/12/03 17:32:31 pid 1875 super@_____CLIENT_UNSET_____ 127.0.0.1 [jenkins.p4-plugin/1.10.7/Linux] 'user-login -s'
	2019/12/03 17:32:31 pid 1876 super@_____CLIENT_UNSET_____ 127.0.0.1 [jenkins.p4-plugin/1.10.7/Linux] 'user-login -s'
	2019/12/03 17:32:31 pid 1877 nouser@_____CLIENT_UNSET_____ 127.0.0.1 [jenkins.p4-plugin/1.10.7/Linux] 'user-info'
	2019/12/03 17:32:31 pid 1878 super@_____CLIENT_UNSET_____ 127.0.0.1 [jenkins.p4-plugin/1.10.7/Linux] 'user-login -s'
	2019/12/03 17:32:31 pid 1879 super@_____CLIENT_UNSET_____ 127.0.0.1 [jenkins.p4-plugin/1.10.7/Linux] 'user-login -s'
	2019/12/03 17:32:31 pid 1880 nouser@_____CLIENT_UNSET_____ 127.0.0.1 [jenkins.p4-plugin/1.10.7/Linux] 'user-info'
	2019/12/03 17:32:31 pid 1881 super@_____CLIENT_UNSET_____ 127.0.0.1 [jenkins.p4-plugin/1.10.7/Linux] 'user-login -s'
	2019/12/03 17:32:31 pid 1882 super@_____CLIENT_UNSET_____ 127.0.0.1 [jenkins.p4-plugin/1.10.7/Linux] 'user-login -s'
	2019/12/03 17:32:31 pid 1883 nouser@_____CLIENT_UNSET_____ 127.0.0.1 [jenkins.p4-plugin/1.10.7/Linux] 'user-info'
	2019/12/03 17:32:31 pid 1884 super@_____CLIENT_UNSET_____ 127.0.0.1 [jenkins.p4-plugin/1.10.7/Linux] 'user-login -s'
	2019/12/03 17:32:31 pid 1885 super@_____CLIENT_UNSET_____ 127.0.0.1 [jenkins.p4-plugin/1.10.7/Linux] 'user-login -s'
	2019/12/03 17:32:32 pid 1886 nouser@_____CLIENT_UNSET_____ 127.0.0.1 [jenkins.p4-plugin/1.10.7/Linux] 'user-info'
	2019/12/03 17:32:32 pid 1887 super@_____CLIENT_UNSET_____ 127.0.0.1 [jenkins.p4-plugin/1.10.7/Linux] 'user-login -s'
	2019/12/03 17:32:32 pid 1888 super@_____CLIENT_UNSET_____ 127.0.0.1 [jenkins.p4-plugin/1.10.7/Linux] 'user-login -s'
	2019/12/03 17:32:32 pid 1889 nouser@_____CLIENT_UNSET_____ 127.0.0.1 [jenkins.p4-plugin/1.10.7/Linux] 'user-info'
	2019/12/03 17:32:32 pid 1890 super@_____CLIENT_UNSET_____ 127.0.0.1 [jenkins.p4-plugin/1.10.7/Linux] 'user-login -s'
	2019/12/03 17:32:32 pid 1891 super@_____CLIENT_UNSET_____ 127.0.0.1 [jenkins.p4-plugin/1.10.7/Linux] 'user-login -s'
$ grep user- tmpa.txt  | wc -l
24

For a job with 4 successful runs I saw 11 commands kicked off:

	2019/12/03 17:40:17 pid 2509 super@_____CLIENT_UNSET_____ 127.0.0.1 [jenkins.p4-plugin/1.10.7/Linux] 'user-login -s'
	2019/12/03 17:40:17 pid 2510 super@_____CLIENT_UNSET_____ 127.0.0.1 [jenkins.p4-plugin/1.10.7/Linux] 'user-login -s'
	2019/12/03 17:40:48 pid 2512 nouser@_____CLIENT_UNSET_____ 127.0.0.1 [jenkins.p4-plugin/1.10.7/Linux] 'user-info'
	2019/12/03 17:40:48 pid 2513 super@_____CLIENT_UNSET_____ 127.0.0.1 [jenkins.p4-plugin/1.10.7/Linux] 'user-login -s'
	2019/12/03 17:40:48 pid 2514 super@_____CLIENT_UNSET_____ 127.0.0.1 [jenkins.p4-plugin/1.10.7/Linux] 'user-login -s'
	2019/12/03 17:40:48 pid 2515 nouser@_____CLIENT_UNSET_____ 127.0.0.1 [jenkins.p4-plugin/1.10.7/Linux] 'user-info'
	2019/12/03 17:40:48 pid 2516 super@_____CLIENT_UNSET_____ 127.0.0.1 [jenkins.p4-plugin/1.10.7/Linux] 'user-login -s'
	2019/12/03 17:40:48 pid 2517 super@_____CLIENT_UNSET_____ 127.0.0.1 [jenkins.p4-plugin/1.10.7/Linux] 'user-login -s'
	2019/12/03 17:40:48 pid 2518 nouser@_____CLIENT_UNSET_____ 127.0.0.1 [jenkins.p4-plugin/1.10.7/Linux] 'user-info'
	2019/12/03 17:40:48 pid 2519 super@_____CLIENT_UNSET_____ 127.0.0.1 [jenkins.p4-plugin/1.10.7/Linux] 'user-login -s'
	2019/12/03 17:40:48 pid 2520 super@_____CLIENT_UNSET_____ 127.0.0.1 [jenkins.p4-plugin/1.10.7/Linux] 'user-login -s'

Have a suspicion that the variable code gets ready to retrieve any extra information that's needed from the Perforce server but then doesn't need to run 'p4 describe' etc. Passing to developers.

Reproduction Steps:

(1) Create an empty Jenkins system.

(2) Add a freestyle job called 'Freestyle1'.

(3) Execute 10 builds of the freestyle job.

(4) Delete the existing P4D log.

(5) Open 'Manage Jenkins' > ' Script Console'.

(6) Enter the following text and click run:

String jobName = 'Freestyle1';

FreeStyleProject buildJob = Jenkins.instance.getItemByFullName(jobName);
if(buildJob) {
     for(FreeStyleBuild job in buildJob.builds) {
		println job.url;
		job.getEnvironment(TaskListener.NULL).each { k, v ->
			println '  | ' + k + ' = ' + v
		}
		println '';
	}
}

(7) Check the P4D log immediately.

Important Note: There is some Jenkins caching going on here so the behavior above only occurs the first time you run the script against a job. Not sure if that cache expires after an hour/day/week/next restart.

Show

Karl Wirth added a comment - 2019-12-03 17:44 - edited Confirmed. One invocation of the above script against a freestyle job that had 8 successful executions caused 24 P4 commands to be run: 2019/12/03 17:32:31 pid 1868 nouser@_____CLIENT_UNSET_____ 127.0.0.1 [jenkins.p4-plugin/1.10.7/Linux] 'user-info' 2019/12/03 17:32:31 pid 1869 super @_____CLIENT_UNSET_____ 127.0.0.1 [jenkins.p4-plugin/1.10.7/Linux] 'user-login -s' 2019/12/03 17:32:31 pid 1870 super @_____CLIENT_UNSET_____ 127.0.0.1 [jenkins.p4-plugin/1.10.7/Linux] 'user-login -s' 2019/12/03 17:32:31 pid 1871 nouser@_____CLIENT_UNSET_____ 127.0.0.1 [jenkins.p4-plugin/1.10.7/Linux] 'user-info' 2019/12/03 17:32:31 pid 1872 super @_____CLIENT_UNSET_____ 127.0.0.1 [jenkins.p4-plugin/1.10.7/Linux] 'user-login -s' 2019/12/03 17:32:31 pid 1873 super @_____CLIENT_UNSET_____ 127.0.0.1 [jenkins.p4-plugin/1.10.7/Linux] 'user-login -s' 2019/12/03 17:32:31 pid 1874 nouser@_____CLIENT_UNSET_____ 127.0.0.1 [jenkins.p4-plugin/1.10.7/Linux] 'user-info' 2019/12/03 17:32:31 pid 1875 super @_____CLIENT_UNSET_____ 127.0.0.1 [jenkins.p4-plugin/1.10.7/Linux] 'user-login -s' 2019/12/03 17:32:31 pid 1876 super @_____CLIENT_UNSET_____ 127.0.0.1 [jenkins.p4-plugin/1.10.7/Linux] 'user-login -s' 2019/12/03 17:32:31 pid 1877 nouser@_____CLIENT_UNSET_____ 127.0.0.1 [jenkins.p4-plugin/1.10.7/Linux] 'user-info' 2019/12/03 17:32:31 pid 1878 super @_____CLIENT_UNSET_____ 127.0.0.1 [jenkins.p4-plugin/1.10.7/Linux] 'user-login -s' 2019/12/03 17:32:31 pid 1879 super @_____CLIENT_UNSET_____ 127.0.0.1 [jenkins.p4-plugin/1.10.7/Linux] 'user-login -s' 2019/12/03 17:32:31 pid 1880 nouser@_____CLIENT_UNSET_____ 127.0.0.1 [jenkins.p4-plugin/1.10.7/Linux] 'user-info' 2019/12/03 17:32:31 pid 1881 super @_____CLIENT_UNSET_____ 127.0.0.1 [jenkins.p4-plugin/1.10.7/Linux] 'user-login -s' 2019/12/03 17:32:31 pid 1882 super @_____CLIENT_UNSET_____ 127.0.0.1 [jenkins.p4-plugin/1.10.7/Linux] 'user-login -s' 2019/12/03 17:32:31 pid 1883 nouser@_____CLIENT_UNSET_____ 127.0.0.1 [jenkins.p4-plugin/1.10.7/Linux] 'user-info' 2019/12/03 17:32:31 pid 1884 super @_____CLIENT_UNSET_____ 127.0.0.1 [jenkins.p4-plugin/1.10.7/Linux] 'user-login -s' 2019/12/03 17:32:31 pid 1885 super @_____CLIENT_UNSET_____ 127.0.0.1 [jenkins.p4-plugin/1.10.7/Linux] 'user-login -s' 2019/12/03 17:32:32 pid 1886 nouser@_____CLIENT_UNSET_____ 127.0.0.1 [jenkins.p4-plugin/1.10.7/Linux] 'user-info' 2019/12/03 17:32:32 pid 1887 super @_____CLIENT_UNSET_____ 127.0.0.1 [jenkins.p4-plugin/1.10.7/Linux] 'user-login -s' 2019/12/03 17:32:32 pid 1888 super @_____CLIENT_UNSET_____ 127.0.0.1 [jenkins.p4-plugin/1.10.7/Linux] 'user-login -s' 2019/12/03 17:32:32 pid 1889 nouser@_____CLIENT_UNSET_____ 127.0.0.1 [jenkins.p4-plugin/1.10.7/Linux] 'user-info' 2019/12/03 17:32:32 pid 1890 super @_____CLIENT_UNSET_____ 127.0.0.1 [jenkins.p4-plugin/1.10.7/Linux] 'user-login -s' 2019/12/03 17:32:32 pid 1891 super @_____CLIENT_UNSET_____ 127.0.0.1 [jenkins.p4-plugin/1.10.7/Linux] 'user-login -s' $ grep user- tmpa.txt | wc -l 24 For a job with 4 successful runs I saw 11 commands kicked off: 2019/12/03 17:40:17 pid 2509 super @_____CLIENT_UNSET_____ 127.0.0.1 [jenkins.p4-plugin/1.10.7/Linux] 'user-login -s' 2019/12/03 17:40:17 pid 2510 super @_____CLIENT_UNSET_____ 127.0.0.1 [jenkins.p4-plugin/1.10.7/Linux] 'user-login -s' 2019/12/03 17:40:48 pid 2512 nouser@_____CLIENT_UNSET_____ 127.0.0.1 [jenkins.p4-plugin/1.10.7/Linux] 'user-info' 2019/12/03 17:40:48 pid 2513 super @_____CLIENT_UNSET_____ 127.0.0.1 [jenkins.p4-plugin/1.10.7/Linux] 'user-login -s' 2019/12/03 17:40:48 pid 2514 super @_____CLIENT_UNSET_____ 127.0.0.1 [jenkins.p4-plugin/1.10.7/Linux] 'user-login -s' 2019/12/03 17:40:48 pid 2515 nouser@_____CLIENT_UNSET_____ 127.0.0.1 [jenkins.p4-plugin/1.10.7/Linux] 'user-info' 2019/12/03 17:40:48 pid 2516 super @_____CLIENT_UNSET_____ 127.0.0.1 [jenkins.p4-plugin/1.10.7/Linux] 'user-login -s' 2019/12/03 17:40:48 pid 2517 super @_____CLIENT_UNSET_____ 127.0.0.1 [jenkins.p4-plugin/1.10.7/Linux] 'user-login -s' 2019/12/03 17:40:48 pid 2518 nouser@_____CLIENT_UNSET_____ 127.0.0.1 [jenkins.p4-plugin/1.10.7/Linux] 'user-info' 2019/12/03 17:40:48 pid 2519 super @_____CLIENT_UNSET_____ 127.0.0.1 [jenkins.p4-plugin/1.10.7/Linux] 'user-login -s' 2019/12/03 17:40:48 pid 2520 super @_____CLIENT_UNSET_____ 127.0.0.1 [jenkins.p4-plugin/1.10.7/Linux] 'user-login -s' Have a suspicion that the variable code gets ready to retrieve any extra information that's needed from the Perforce server but then doesn't need to run 'p4 describe' etc. Passing to developers. Reproduction Steps: (1) Create an empty Jenkins system. (2) Add a freestyle job called 'Freestyle1'. (3) Execute 10 builds of the freestyle job. (4) Delete the existing P4D log. (5) Open 'Manage Jenkins' > ' Script Console'. (6) Enter the following text and click run: String jobName = 'Freestyle1' ; FreeStyleProject buildJob = Jenkins.instance.getItemByFullName(jobName); if (buildJob) { for (FreeStyleBuild job in buildJob.builds) { println job.url; job.getEnvironment(TaskListener.NULL).each { k, v -> println ' | ' + k + ' = ' + v } println ''; } } (7) Check the P4D log immediately. Important Note: There is some Jenkins caching going on here so the behavior above only occurs the first time you run the script against a job. Not sure if that cache expires after an hour/day/week/next restart.

Hide

Permalink

Matthew Smeeth added a comment - 2019-12-10 11:22 - edited

Hi James Bateman,

"getEnvironment" fetches various Perforce variables including p4ticket, unfortunately some of our customers require a Perforce ticket to be added to the environment for their scripts. If you set the "Hide TICKET" option in the "Perforce: Secure P4_TICKET" section on the Jenkins configuration page, this will prevent Jenkins from getting a ticket when you call the "getEnvironment" function.

Show

Matthew Smeeth added a comment - 2019-12-10 11:22 - edited Hi James Bateman , "getEnvironment" fetches various Perforce variables including p4ticket, unfortunately some of our customers require a Perforce ticket to be added to the environment for their scripts. If you set the "Hide TICKET" option in the "Perforce: Secure P4_TICKET" section on the Jenkins configuration page, this will prevent Jenkins from getting a ticket when you call the "getEnvironment" function.

Hide

Permalink

Matthew Smeeth added a comment - 2019-12-11 11:25

Not fixing due to being able to prevent login -s being called by enabling the "hide tickets" option.

Show

Matthew Smeeth added a comment - 2019-12-11 11:25 Not fixing due to being able to prevent login -s being called by enabling the "hide tickets" option.

Jenkins

ConnectionHelper.java constructors call connectionRetry()

Details

Description

Attachments

Issue Links

Activity

People

Dates