We are setting up a lot of builds in a single multibranch pipeline. When we first added them we hit an error where our jenkins master was running out of ports to connect to perforce as it scanned. So we made a simple repro. We generated 153 jenkinsfiles in folders (I've attached an example jenkinsfile we used, and a screenshot showing the folder structure). Then we created a new multibranch pipeline pointing at those files (see branch sources.png). When that pipeline is added it tries to scan the multibranch pipeline then generates the error in scan_multibranch.txt (attached). Running netstat on the master shows that all ports from 49152 to 65535 are open (in TIME_WAIT) to our perforce box!
We can delay the running out of ports error by adding the "Suppress automatic SCM triggering" property, however, if we then try to build about 10 of the builds at once we hit the run out of ports error, so this is not a solution for us.
Looking at the tty when it builds one of the jobs - it appears to grab all the jenkins files, rather than just the jenkinsfile for that particular job - see build_abcdef.txt. Why does it do this? Have I misunderstood how to configure a relatively large number of pipelines, or is this a bug? I assume this is why we have so many TCP connections open to perforce when this occurs.
When building job abcdef it grabs <folder>/abcdef/JenkinsFile from perforce, then does the build
When building job abcdef it grabs <folder>/abcdef/JenkinsFile from perforce, <folder>/abcdfe/JenkinsFile, ... all the 153 jenkins files, before trying the build.
This leads to many many connections to the perforce server, and it then runs out of ports.