-
Bug
-
Resolution: Fixed
-
Minor
-
None
-
Windows Server 2016 64-bit
-
Powered by SuggestiMate
Currently running the latest version of Jenkins with the latest plugins required for Blue Ocean.
Jenkins is unable to check-out certain repositories that we use for an unknown reason.
The ArrayIndexOutOfBoundsException us occurring here:
java.lang.ArrayIndexOutOfBoundsException: 1 at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.parseBranches(CliGitAPIImpl.java:2043) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.getBranches(CliGitAPIImpl.java:2064) at hudson.plugins.git.GitAPI.getBranches(GitAPI.java:195) at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$9.execute(CliGitAPIImpl.java:2199) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:153) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:146) at hudson.remoting.UserRequest.perform(UserRequest.java:153) at hudson.remoting.UserRequest.perform(UserRequest.java:50) at hudson.remoting.Request$2.run(Request.java:336) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at hudson.remoting.Engine$1$1.run(Engine.java:94) at java.lang.Thread.run(Thread.java:748) at ......remote call to JNLP4-connect connection from JENKINSSLAVE01/10.200.8.6:57602(Native Method) at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1545) at hudson.remoting.UserResponse.retrieve(UserRequest.java:253) at hudson.remoting.Channel.call(Channel.java:830) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.execute(RemoteGitImpl.java:146) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.invoke(RemoteGitImpl.java:132) at com.sun.proxy.$Proxy105.execute(Unknown Source) at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1167) at org.jenkinsci.plugins.workflow.steps.scm.SCMStep.checkout(SCMStep.java:113) at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:85) at org.jenkinsci.plugins.workflow.steps.scm.SCMStep$StepExecutionImpl.run(SCMStep.java:75) at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1$1.call(AbstractSynchronousNonBlockingStepExecution.java:47) at hudson.security.ACL.impersonate(ACL.java:260) at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1.run(AbstractSynchronousNonBlockingStepExecution.java:44) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748)
- is duplicated by
-
JENKINS-47196 java.lang.ArrayIndexOutOfBoundsException: 1- Source checkout issue
-
- Closed
-
[JENKINS-45894] Unable to checkout repository with git-plugin ArrayIndexOutOfBoundsException
Glad to have you work on it.
That exception seems to indicate that the output of "git branch -v" in that repository contains unexpected output. The unexpected output is longer than 44 characters and includes a space character in the string, but it does not appear to have a space character after the first two characters of the string.
That method is parsing the output of git branch -v --no-abbrev, and I'm not aware of any output of git branch -v --no-abbrev which has leading spaces, is at least 44 characters long, but includes no spaces after the initial two characters.
I would be suspicious of some unexpected condition on that repository or on that machine.
Does that repository fail to clone on every Jenkins agent, or only on a specific agent?
What is the output of git branch -v --no-abbrev on that agent, in that working directory?
Hi Mark.
It's only for about 2 repositories that we use (that I've tested thus far)
I've run git branch -v --no-abbrev, on our remote branch and nothing seems to not meet those conditions.
They all look similar to this:
origin-http/xx-xxxx-xxxxxxxxxxx-xxxxxxxxx-xxxx-xxxxxxxxxxx 4419xxxxxxxxxxxxxxxxxxxxxxxxxxx4cfa347e8 xx-xxxx Xxx xxx xxx xxxxx xxx xxx xxxx.
However. I followed your instructions and ran it on the specific agent itself in the working directory and I've found the problem:
- (HEAD detached at 4d5095049) 4d50950494af2182e52755406e72b0e55bced6dd Merge pull request #1826 in xx/xx from bugfix/xx-xxxxx-xxxxx-raised-by-xxxxxxxxxxxxxxxxxxxxxxxxxxxxx to dev
It's the 'HEAD detached' part which is causing the error. I'm unsure what's causing this.
This is the first command I'm calling:
git(url: "ssh://git@xxx:xxxx/xxxx/xxxxx.git", branch: "dev", credentialsId: 'xxxx')
So nothing should be moving it out.
I see that same output when I checkout a detached head (which the git plugin does very frequently). The git plugin is willing to operate on a repository that is checked out at a specific SHA1, without any branch at that point.
You can see that type of condition if you use the command git checkout HEAD^. That will checkout the source code as it existed one commit prior to the current location.
Since I see that condition on multiple git versions and on multiple machines, I'm skeptical that is the root cause of the problem. However, it certainly could be. If it is the root of the problem, then I am perplexed why thousands of users are not reporting the same problem. It is a very common use case that the git plugin performs a checkout with a detached HEAD.
Hmmm I agree that it's a little odd that thousands of users are not reporting the same problem.
However, I did go to the same agent and look at a repository where it worked using the exact same code and noticed that it was on the dev branch and not a detached head.
- dev 8abf5d0fa92810e3ba2739802bb77452cf9eedbd Merge pull request #894
I'm beginning to think something is causing the repository to checkout in a detached state. I've done a comparison between the two repositories on our SCM and can't notice any differences. I'm totally perplexed.
A detached head is used if the project (job) is configured without the additional behaviour to checkout as local branch. However, since that additional behaviour is not the default, I would expect that there are many, many jobs in the world which are not performing a checkout to a specific local branch.
I suspect the job which has the problem does not include that additional behaviour, while the job with the problem does not include that additional behaviour.
I specifically added that behaviour to the BitBucket Pipeline plugin and it still checked out the detached head. Whilst this is probably a problem with another plugins local checkout behaviour I'm still slightly perplexed at the idea that no one else has run into this before as you stated because it's not default behaviour.
I checked the working repository and it had everything set to default. Could it be a git option or something other than the Jenkins plugins?
I've attempted to duplicate the problem (unsuccessfully) in my jenkins-bugs repository. Could you compare that config.xml file with the config.xml file from one of your failing jobs so that I know the difference between your failing job and my job that passes (and still checks out a detached head).
Hey Mark. I'm not going to lie.
I'm not super sure what's going on in your config.xml
I've attached the config.xml that is the resulting job to see if something sticks out in to you.
Thanks.config_for_Mark.xml
Thanks for posting your config.xml. It shows that you're using a pipeline job ("flow-definition") which is pulling from Bitbucket (I assume an enterprise installation, since it isn't bitbucket.org),
It appears to be from a multi-branch pipeline job, and is reading the Jenkinsfile from inside the branch.
Can you confirm that is correct?
If that is correct, is the Jenkinsfile modifying the checkout in any way (for example, adding extensions to the checkout command, or specifying more than the default arguments to the checkout command)?
Thanks Mark. That's all correct.
So in my Jenkinsfile I'm using a global SharedLibrary (so I can re-use the same Jenkinsfile) across multiple repositories.
The part of the code that does the checkout is as such:
dir(path: "${codeRepositoryLowercase}") {
git(url: "ssh://git@stash:7999/${env.REPO_FIRST}/${env.REPO_SECOND}.git", branch: "${env.BRANCH_NAME}", credentialsId: 'xxxx')
}
EDIT:
env.REPO_FIRST and env.REPO_SECOND are two different names defined in the environment section of said Jenkinsfile.
Repo_First might be Tools
and Repo_Second might be Debugging
to then be Tools-Debugging in Enterprise Bitbucket
After some more investigation I think it's a problem with the GtiSCM plugin potentially.
it looks like that where I call the shorthand version in my jenkinsFile in from my sharedLibrary it's checking out the code repository with a detatched head and not the local branch which would potentially explain why it's not happening to a large userbase?
Trying to get just the checkout to work by doing the following:
checkout(
[$class: 'GitSCM',
branches: [[name: "$\{env.BRANCH_NAME}"]],
doGenerateSubmoduleConfigurations: false,
extensions: [[$class: 'LocalBranch',
localBranch: "**"]],
submoduleCfg: [],
userRemoteConfigs: [[credentialsId: 'svc_jenkins',
url: "ssh://git@stash:7999/${env.REPO_FIRST}/${env.REPO_SECOND}.git"]]
]
)
Unfortunately it's still checking out the code repository with a detached head on the Jenkins agent.
After far too much investigation.... I think I've found the exact step that's failing.
This is the log of the command of the failing stage in Jenkins for the failing repository:
Cloning the remote Git repository Cloning repository ssh://git@stash:7999/xx/xx.git > git.exe init E:\Jenkins\workspace\builds\xx-xx\dev\xx-xx # timeout=10 Fetching upstream changes from ssh://git@stash:7999/xx/xx.git > git.exe --version # timeout=10 > git.exe fetch --tags --progress ssh://git@stash:7999/xx/xx.git +refs/heads/*:refs/remotes/origin/* > git.exe config remote.origin.url ssh://git@stash:7999/xx/xx.git # timeout=10 > git.exe config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10 > git.exe config remote.origin.url ssh://git@stash:7999/xx/xx.git # timeout=10 Fetching upstream changes from ssh://git@stash:7999/xx/xx.git > git.exe fetch --tags --progress ssh://git@stash:7999/xx/xx.git +refs/heads/*:refs/remotes/origin/* > git.exe rev-parse "refs/remotes/origin/dev^{commit}" # timeout=10 > git.exe rev-parse "refs/remotes/origin/origin/dev^{commit}" # timeout=10 Checking out Revision 4d50950494af2182e52755406e72b0e55bced6dd (refs/remotes/origin/dev) Commit message: "Merge pull request #1826 in xx/xx from bugfix/VN-11921-error-raised-by-transactionintegrationservice to dev" > git.exe config core.sparsecheckout # timeout=10 > git.exe checkout -f 4d50950494af2182e52755406e72b0e55bced6dd > git.exe branch -a -v --no-abbrev # timeout=10 1
So the next step that's failing to do is:
git checkout -b dev 83f99324eb28647d46c57842666cb2c2e09ee629
Whick looks like it deletes the sha1 it checked out at a remote head and makes the local branch.
I found that manually running this works on said agent(via powershell):
git init C:\test# timeout=10 git --version # timeout=10 git fetch --tags --progress ssh://git@stash:7999/xx/xx.git +refs/heads/*:refs/remotes/origin/* git config remote.origin.url ssh://git@stash:7999/xx/xx.git # timeout=10 git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10 git config remote.origin.url ssh://git@stash:7999/xx/xx.git # timeout=10 git fetch --tags --progress ssh://git@stash:7999/xx/xx.git +refs/heads/*:refs/remotes/origin/* git config core.sparsecheckout # timeout=10 git checkout -f 83f99324eb28647d46c57842666cb2c2e09ee629 git branch -a -v --no-abbrev # timeout=10 git checkout -b dev 83f99324eb28647d46c57842666cb2c2e09ee629
Which is the output of the commands that the successful repository runs in Jenkins so I have to assume that it's unable to create that dev branch for whatever reason. But that baffles me because when I manually run the commands on the agent it appears to work and have a local dev branch!
When you say that it is not creating the dev branch, is that in the shared library workspace, or in the workspace where the repository specified in the Jenkinsfile is placed?
I don't think it is that the code is unable to create the "dev" branch, but rather that the job is not configured to create the "dev" branch. If the shared library workspace is the place where the "dev" branch is not created, then that is a different area for me to investigate.
Hey Mark.
I believe it's not related to the shared library at all.
I still receive the same out of bounds error for particular repositories.
Here's an example pipeline:
pipeline { agent any stages { stage('Successful pull'){ steps { dir(path: "test1") { git(url: "ssh://git@stash:7999/xx/xx.git", branch: "dev", credentialsId: 'xxxx') } } } stage('Failing pull'){ steps { dir(path: "test2") { git(url: "ssh://git@stash:7999/yy/yy.git", branch: "dev", credentialsId: 'xxxx') } } } } }
I can't for the life of me figure out why it's rejecting some code repositories and not others.
Clutching at straws here but there's some weird locking behavior when I manually try to perform the next step that the plugin would normally do?
francommit usually that type of message means that there is already a git process running in that repository. The index.lock file is used to request exclusive access to the git repository data structures.
Hey Mark.
Sorry for taking a while to reply. Been experimenting trying to find the root cause of this issue.
After messing around with quite a bit I've been led to believe there's a problem with the GitScm plugin parsing some of the Git history in two of repositories we use.
For the culprit repository I created a new repository with the same levels of permission and copied all the files to it and ran it through the GitScm plugin via a simple pipeline (like the one described above) and it worked just fine.
I'm unsure what else I can do to try to this repository to attempt to get it to accept its data.
Ok markewaite I deleted a whole heap of branches and it appears to have gone away.
I possibly think it's got to do with the branch name being too long and just windows/the plugin failing to deal with a character limit somewhere.
I'm happy for you to cancel this but the issue will still potentially plague other windows users.
Thanks francommit for your diligent efforts to duplicate the problem. I have repositories with many, many branches (some with thousands), so I suspect that the problem is not in the count of branches, but one of those branch names may have exercised a bug in the code.
I'll double check as time allows to see if I can find another way to see the problem.
markewaite the issue came back. Turned out a fullstop in the name of a branch was making it go bananas.
Deleted that branch with the fullstop and the problem went away.
Hopefully you can replicate this yourself.
Cheers.
I also ran into this problem
I had this git option enabled: "Check out to specific local branch", in pipeline: extensions : [[$class: 'LocalBranch']]
I solved the problem by deleting some branches. I'm not sure exactly what was going wrong but some index checking couldn't hurt.
lukelast I'm open to a pull request with one or more tests which show the problem, or even to a description of the specific branch names which cause the problem. I'm hesitant to insert bounds checking for a case that isn't duplicated by a test in the code.
I'm not sure what francommit means as a "fullstop" in a branch name, though I assume he means ASCII 2E. If that's the case, then I don't see how that would cause a problem as a branch name, since '.' (ASCII 2E) is allowed as a character in a branch name.
I could see that could being broken by a newline (or other end of line character) in the branch name. However, if a branch name includes some form of line break character (newline, carriage return, etc.) then it is not a valid git branch name, at least per the definition of the valid characters in the git man page.
I've placed a test in my regression test kit which checks for ASCII 2E in a branch name. The test passes. Unfortunately, that means I still can't duplicate this problem.
lukelast the solution I also came across to fix the problem was deleting multiple branches until it worked.
Thanks to vadivel, I've finally understood the condition. lukelast was correct that bounds checking is what is needed. Sorry for my slow comprehension!
The bounds checking which Luke recommended (along with an automated test) was provided about 3 weeks ago in a merge to the master branch. It will be included in the next release of the git client plugin.
You can test an early build by downloading the latest artifact from the ci.jenkins.io build.
Hey I'm happy to work on this. Making a fork from the Github but I'm unsure how to hook it back into JIRA here.
Is there an easy way? I'll just name my branch the same name as this ticket.