This problem seems to manifest only when the repo didn't have a Jenkinsfile before. If we commit a simple "hello world" Jenkinsfile, then all works as expected with PRs after that.

Likely this is just a side-effect of the limits of the collaborators query API, in which case switching to the trust strategy in JENKINS-36240 should resolve the issue.

Please re-open if theory is incorrect

How can I test if this theory is correct? Should I give some user push access directly as a collaborator in a repo (instead of via team) and then try to make a PR that adds a Jenkinsfile and if it works the theory is correct?

create two a multibranch projects, one using the new trust strategy and the other using the old collaborators strategy... both pointing to the same virgin github repo.

Have somebody from a fork create a PR that adds a Jenkinsfile.

Should work on the new trust strategy job and give the error you reported on the old (collaborators) trust strategy job

Then you can delete the repo and the two multibranch projects and party like it's 1999

OK, I'm doing a test the the "Everyone" strategy, since it seems the more appropriate for an organization that only contains private repos. But just FYI, querying the github API using https://developer.github.com/v3/repos/collaborators/, the user that was considered as untrusted by jenkins was indeed a collaborator.

Well to confirm the bug as fixed we need to know that the "Admin or Write permissions" trust strategy looks up the details correctly.

The collaborators API end-point would need to have been checked using the api token that your jenkins was using (as the issue with that API is very often Jenkins does not have permission to check the list of collaborators)

OK, I can do that, but what I don't understand is why this only affects PRs created from the same repo (not from forks) since the trust evaluation seems to be for forks, you can always trusts branches in the same repo...

Now all strategies seem to work. Is hard to test this as I don't have new private repos to use, and I don't know if once I tested something in a repo if it becomes "tainted" and then everything I test will work (there is no Jenkinks file in the repo I'm testing though, only PRs with Jenkinsfiles).

I will report back if I find any issues again in the future. Thanks!