-
Bug
-
Resolution: Unresolved
-
Blocker
-
Jenkins: 2.103
Github plugin: 1.29.0
GitHub API Plugin: 1.90
OS: Ubuntu 14.04.5 LTS
GitHub Pull Request Builder: 1.40.0
-
Powered by SuggestiMate
Under GitHub Pull Request Builder configuration Auto-manage webhooks is enabled.
In the GitHub Pull Request Builder added user credentials. User belongs to owners' User's credential tested and are working.
Webhooks management however does not work. For every private repository there's an error message displayed in Jenkins:
There is no credentials with admin access to manage hooks on GitHubRepositoryName[host=github.com,username=ORG_NAME,repository=REPO_NAME]
From the error message it looks like plugin is trying to use organisation name as a user name, so authentication fails.
[JENKINS-49332] Jenkins unable to manage webhooks of Github organization
Same message after upgrading to Github plugin 1.29.
Also Enterprise Github instance.
Manage web hooks in Github pull request builder: Disabled
Manage hooks under github server: Disabled
Error messages persist after disabling both manage web hook check boxes.
Work around: I downgraded back to the last version I used, 1.28.1, which got rid of the message spam. Jenkins+Github worked even when I got the error messages, except things seemed a bit slower.
Here is a full stack trace from the error I got on 1.29, server, org, and repo names removed:
org.jenkinsci.plugins.github.webhook.WebhookManager$2 applyNullSafe WARNING: Failed to add GitHub webhook for GitHubRepositoryName[host=github.XXX.com,username=orgName,repository=repoName] java.lang.NullPointerException: There is no credentials with admin access to manage hooks on GitHubRepositoryName[host=github.XXX.com,username=orgName,repository=repoName] at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:231) at org.jenkinsci.plugins.github.webhook.WebhookManager$2.applyNullSafe(WebhookManager.java:179) at org.jenkinsci.plugins.github.webhook.WebhookManager$2.applyNullSafe(WebhookManager.java:175) at org.jenkinsci.plugins.github.util.misc.NullSafeFunction.apply(NullSafeFunction.java:18) at com.google.common.collect.Iterators$8.next(Iterators.java:812) at com.google.common.collect.Iterators$7.computeNext(Iterators.java:648) at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143) at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138) at com.google.common.collect.Iterators$7.computeNext(Iterators.java:647) at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143) at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138) at com.google.common.collect.Lists.newArrayList(Lists.java:138) at com.google.common.collect.Lists.newArrayList(Lists.java:119) at org.jenkinsci.plugins.github.util.FluentIterableWrapper.toList(FluentIterableWrapper.java:147) at org.jenkinsci.plugins.github.webhook.WebhookManager$1.run(WebhookManager.java:127) at hudson.util.SequentialExecutionQueue$QueueEntry.run(SequentialExecutionQueue.java:119) at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
The error messages appeared every time my github organizations were scanning, every 2 minutes. Again, I did not have manage hooks enabled for my github server or pull request builder configurations.
I generated a brand new github token for jenkins, which I made sure had the right permissions, but the errors persisted after saving it (and then restarting jenkins for good measure). The funny thing was Github reported the token was 'never used' after the errors were being thrown - it only said the token was used once I went back to Jenkins configuration and hit the 'test connection' to github button. Related possibly considering the 'null' parts of the stacktrace.
We're having the same problem trying to use the GitHub Branch Source Plugin.
It seems to be ignoring the username in the configured username/password credentials and defaulting to the organization (Owner) value.
This is completely blocking us from making use of the plugin since no WebHook registration can occur so our builds can't be triggered by PRs/etc.
Any idea when this might be fixed? Also, is the priority of this really 'minor'?
It is good to know that others are having this issue. I couldn't figure out how I broke this. Can this issue be changed to critical? This bug has really thrown a wrench in our pipeline.
Also encountering this same error.
Is there a workaround that can be used while this issue is being worked on, or a previous version of a plugin that doesn't exhibit this behavior?
As far as I can tell, the plugin doesn't support webhook management for servers under the GitHub Enterprise Servers header. We're still seeing a lot of this in our logs, though.
I think I'm seeing this too - after a while trying all kinds of work arounds, I noticed that my ORG name there which is being used with a "personal" access token.
I am also experiencing this with non-Enterprise. It seems this issue is only impacting private repos, otherwise I couldn't understand how this isn't a much, much bigger issue here.
Edit: The issue I'm seeing here is with a public repo.
I think this might have been in a relatively recent upgrade of the GitHub plugin or something. I started getting a lot of these, here's what I did to resolve it:
- Add a 'Secret Text' global credential. The secret should be your access token from GitHub that probably has at least write access to repos (I didn't investigate which roles are required). ID and description doesn't matter. This must be 'Secret Text' for the next pieces to work right
- Go to Manage Jenkins -> Configure System and down to 'GitHub Servers'
- Hit 'Add GitHub Server'
- Name - doesn't matter
- API URL - since I don't have a private GitHub Enterprise install (just private repos on github.com) I used the default of https://api.github.com
- Credentials - the 'Secret Text' credential from earlier (this dropdown only pulls 'Secret Text' credentials)
- Manage Hooks - checked
- Hit 'Test Connection' to make sure everything is kosher
- Go to your organization and 'Scan Organization Now' on the left
This should clean up all of the warnings about installing webhooks.
phillipuniverse Your resolution steps didn't say how you got the plugin to specify which account is associated with the access token. That seems to be the issue here. It seems the plugin is assuming that the account name the access token is for is the same name as the organisation as dmytro_kovalov included in his description:
There is no credentials with admin access to manage hooks on GitHubRepositoryName[host=github.com,username=ORG_NAME,repository=REPO_NAME]
I could imagine this works for non organisation repos since the username would be the same for the repo owner and the access token. But I don't think there is any way to get an access token associated with a GitHub organisation the same way one can with personal accounts.
brianjmurrell I did not need to specify anything about an account for the access token. I have a "Jenkins User" account that I created on GitHub and is specified as an owner for my organization. I went into that account and created an access token (from https://github.com/settings/tokens) and used that for the value of the secret text in my workaround. As I mentioned, I did not investigate which scopes were required; the access token I use has access to all scopes.
This definitely works for organization repos, that is my only use case. I have been warning-free in my Jenkins install since I posted my previous comment.
Maybe the key is that the account for which the personal access token is generated is an organization owner account?
and is specified as an owner for my organization
So, this makes sense given the nature of this ticket, however the help text for the Projects->GitHub Organization->Owner under your Jenkins GitHub organisation configuration says:
Specify the name of the GitHub Organization or GitHub User Account*.*
which doesn't seem to be what you have put in that field.
I went into that account and created an access token
AFAIU a GitHub organisation isn't an account with which you make access tokens for (among many other account-related things).
So just to be clear, if your GitHub organisation link is https://github.com/some_organization/, some_organisation is not what you put into Projects->GitHub Organization->Owner in your organisation configuration in Jenkins? You put in the name of some other account with which you could make access tokens against?
which doesn't seem to be what you have put in that field.
No, I put my GitHub organization in that field. When you create an organization in Jenkins, you essentially have to fill out 2 fields:
- Credentials - this is the username/password of the owner user I was talking about earlier
- Owner - this is the actual GitHub organization name, e.g. https://github.com/some_organization/, so some_organization is what you put in that field
Additionally, you have to go to Manage Jenkins -> Configure System (I had a typo in my original comment; this used to say 'Configure Security and now it says 'Configure System'), scroll down to the 'GitHub' section and add a new GitHub server. Then follow my instructions about the secret text with the access token, etc.
So creating the organization to scan in Jenkins is not the only place you have to update. You also have to go and add a GitHub Server with the secret text in 'Manage Jenkins'.
I agree with you that it seems wrong to manage this in both places and I don't know why this is the case. This is just the workaround as it worked for me, YMMV.
Credentials - this is the username/password of the owner user I was talking about earlier
Ahhh. So for this credential you have given permissions to create webhooks?
The help text for this field says:
Credentials used to scan branches and pull requests, check out sources and mark commit statuses.
Nothing about commit hooks. So I think we are in general agreement that there is definitely some confusion here as to what goes where and from where credentials are being taken to perform various operations. It's definitely not working as documented/suggested by the help text(s).
Hi guys we are struggling with this issue since a few weeks.
What makes it worse is that we disabled the Auto-manage Webhooks as we don't need it but registered a webhook on organisation level instead. Still randomly some webhooks don't trigger a pipeline (maybe 98/100 PRs trigger our Jenkins pipelines, 2/100 do nothing except writing an error message in the logs). In that case we get the same error message as mentioned in the description.
These are our settings in Jenkins:
We use the Jenkins Github Organisation feature. In our organisation we have ~80 repos that are scanned by the plugin. The issue appears totally randomly throughout different repos. We create very simple PRs without descriptions or any fancy stuff and even after thorough inspection I could not find any difference in the PRs that result in an error and those that trigger a pipeline. The issue is always solved by closing the PR and creating a new one immediately afterwards, which is exact same as the one that failed. That new PR triggers the pipeline as expected.
Maybe a hint is, that > 90% of the failing webhooks belong to PRs of a single user, who is also admin of our GitHub organisation (but not Jenkins admin). Again this user also creates the majority of our PRs so if you see it like this, it might be statistically possible that this is coincidence.
neidhart your issue is that you do not have a 'Secret Text' credentials with a personal access token. Try my instructions above (https://issues.jenkins-ci.org/browse/JENKINS-49332?focusedCommentId=340820&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-340820) with a 'Secret Text' credential and see if that makes a difference.
phillipuniverse thank you I was going to try that workaround - but this does not explain why it works in 98% of the cases but randomly in 2% not so the issue remains.
phillipuniverse I tried your workaround and indeed the error messages in the Jenkins logs went away but the issue remained the same: 2% of our PRs do not trigger the CI.
I disabled the workaround again because it was not helping and I am still sure it's related:
Everytime when a PR does not trigger the CI I see the message
Everytime when I check the logs after a PR that actually triggered the CI, the message is not there
neidhart You have a lot of repos to scan. Maybe periodically you hit your quota for the Github API. Do you only experience on problems on days with lot of pushes to Github?
Any news here?
I have tried very hard to find a place in Jenkins to set the username for the secret key, but have failed and thus Jenkins reports my ORGNAME as the username when connecting to GitHub.
Is there a way to configure this in Jenkins I'm not seeing? In the comments above I see some references to "Projects->GitHub Organization->Owner under your Jenkins GitHub organisation" but I cannot find any such link in my Jenkins install.
Regards,
I don't know if this is of help, and it's a bit embarrasing that the most common usecase ("I have a github org, I want to build all the projects, and have webhooks added so this is efficient/fast") is so terrible UX-wise.
But:
In the project configuration, top level (github organization), you need to set up the organisation name (so it finds the right projects).
You need some credentials to access github. It only lists "username/password" items on the list.
However : for github if you're using a personal access token, the username is irrelevant. It really should offer 'secret' types in this list.
I got ours to work by (re)creating a username/password credentials as username = ORG_NAME, password = personal_access_token.
This might be of help.
This is not only happening on GitHub organizations items, I have a free-style job that started failing with this same error.
A possible solution in the org.jenkinsci.plugins.github.webhook.WebhookManager plugin would be to look up the username given the secret text via the API: https://api.github.com/user? Or, if its an enterprise github, the API for current_user_url from https://<MY ENTERPRISE API URL>/? Use the login from that response as the username in the connection:
host=github.com,username=<MY ORG>,repository=<MY REPO>
I think this will work for all use cases, including private orgs, public, and enterprise github.
In my case I didn't have an entry in jenkins configuration, under GitHub -> GitHub Servers
It was not obvious from the error and the pipeline configuration that you needed to have the github repo configured both in global jenkins configuration and in the pipeline job itself
I am also getting the same error. The webhook is trying to be built using the ORG name. I am running against an GitHub Enterprise instance.