> What are the versions of the git-plugin dependencies to use?

I hit the same issue in my experiments. Version in my package (created by Custom WAR Packager) is older than the one required by a test dependency, so the PCT run fails.

As we discussed with vilacides, it's better to take the newest version in this case even if the bundled version is older.

Version in my package (created by Custom WAR Packager) is older than the one required by a test dependency

I do not understand how this situation would even arise. Your WAR package should be specifying sufficiently new versions of everything. If you found it necessary to update a test dependency, certainly the WAR package version should be updated to at least that version, if not newer.

In other words I would recommend treating such a situation as a user error and fail with an explicit message.

My suggestions for the questions laid out in acceptance criteria:

all the critical upstream dependencies

Downstream I think you mean?

The run has to be against a TBD version of Jenkins core: Latest one

Most recent weekly, you mean? No, because this will be nondeterministic.

Latest LTS

Same, but also too old.

The one included in essentials

Great, so long as the particular version is encoded in a file in the git-plugin repo, to be routinely updated (in a PR).

What are the versions of the git-plugin dependencies to use?
the ones included in essentials

Yes, with the same caveat as above.

a war file generated from the new custom war packager tool using essentials.yml as source

This.

I think it would be useful to implement JENKINS-47498 so that every Essentials snapshot deployed to production also deploys a matching Maven BOM to Artifactory, which may be consumed (in a specific version) by plugin POMs to ensure that all their (test and main) dependencies are close to the tip. This will greatly reduce the chances of PCT runs failing due to wide divergence between the versions requested in the POM, and thus usually run by developers, and the versions coming from Essentials.

To eliminate redundancy, the runPCT step could automatically create a WAR package to feed into PCT based on the same snapshot of Essentials as produced the BOM being imported.

Regarding the dependency overrides, I have proposed and opt-in System property for JTH in https://github.com/jenkinsci/jenkins-test-harness/pull/99

So I am going to take this in two steps, one will use the current existing support in runPCT to test local artifacts, but that is not going to be enough as we do not want to test artifacts in isolation but as a set, so the second part will implement the complete flow which would be:

• Use the work on Custom war packager and JEP-305 to generate a war file that contains the SUT and the essentials plugins in the proper versions that means the ones defined in essentials BOM which as jglick stated before should be deployed to Artifactory on every production change.
• That war file is then fed into ATH and PCT

As the "complete" flow is not yet possible I am going to provide a simplified version which stills provides a lot of value and create a follow-up task to deal with the complete flow once all the tooling is there

So I have the flow already implemented but I am having failing tests in PCT, checking now how to fix them as I want to start the flow with a green build

I suspect at least one problem is the lack of the .git folder in the sources run by the PCT as I am stashing with default exclusions and some of the errors refer to the lack of a plugin repo AFAIUI in the sources folder, so testing now without default excludes

Also the code used in runPCT for copying plugin sources to the proper dir was not copying hidden files

PCT#75 and some changes in the Jenkinsfile seems to solve the failing PCT tests

And also pipeline-library#48

Code changed in jenkins
User: Raul Arabaolaza
Path:
src/main/docker/run-pct.sh
http://jenkins-ci.org/commit/plugin-compat-tester/17058d791b8e69b02950481093701059d5229046
Log:
JENKINS-50540 Make sure to copy hidden files (like .git) on custom sources

Code changed in jenkins
User: Raúl Arabaolaza Barquin
Path:
src/main/docker/run-pct.sh
http://jenkins-ci.org/commit/plugin-compat-tester/7ff1ce59b4d9ebb01b70b9fc77443dee45cbd5e3
Log:
Merge pull request #75 from raul-arabaolaza/JENKINS-50540

JENKINS-50540 Make sure to copy hidden files (like .git) on custom sources

Compare: https://github.com/jenkinsci/plugin-compat-tester/compare/f0d0ce72c68f...7ff1ce59b4d9

Code changed in jenkins
User: Raul Arabaolaza
Path:
Jenkinsfile
http://jenkins-ci.org/commit/git-plugin/4e838dac2f56110b410ae749d5ecbdc8ca3ae40c
Log:
JENKINS-50540 Keep .git folder for tests

Code changed in jenkins
User: Raul Arabaolaza
Path:
Jenkinsfile
http://jenkins-ci.org/commit/git-plugin/27030980db49467d0169ed8671aebceb572c0c46
Log:
JENKINS-50540 runPCT already includes the log option

Plus there is no need to delete anything after runPCT

Included in git plugin 3.9.0 released 12 May 2018