-
Bug
-
Resolution: Fixed
-
Major
-
None
-
Powered by SuggestiMate
We use the JClouds plugin to provision OpenNebula instances via aws-ec2 api
Since update to 2.28 the connection fails.
From the stack trace i guess this is related to JENKINS-38736
Cannot connect to specified cloud, please check the identity and credentials: Unable to create injector, see the following errors:
1) Overriding @Provides methods is not allowed.
@Provides method: org.jclouds.logging.config.LoggingModule.createLoggerFactory()
overridden by: jenkins.plugins.jclouds.compute.JCloudsCloud$1.createLoggerFactory()
at com.google.inject.internal.ProviderMethodsModule.getProviderMethods(ProviderMethodsModule.java:163)
2) Overriding @Provides methods is not allowed.
@Provides method: org.jclouds.compute.config.BaseComputeServiceContextModule.provideTemplateOptions()
overridden by: org.jclouds.aws.ec2.compute.config.AWSEC2ComputeServiceContextModule.provideTemplateOptions()
at com.google.inject.internal.ProviderMethodsModule.getProviderMethods(ProviderMethodsModule.java:163)
3) Overriding @Provides methods is not allowed.
@Provides method: org.jclouds.compute.config.BaseComputeServiceContextModule.provideImageExtension()
overridden by: org.jclouds.aws.ec2.compute.config.AWSEC2ComputeServiceContextModule.provideImageExtension()
at com.google.inject.internal.ProviderMethodsModule.getProviderMethods(ProviderMethodsModule.java:163)
4) Overriding @Provides methods is not allowed.
@Provides method: org.jclouds.compute.config.BaseComputeServiceContextModule.provideSecurityGroupExtension()
overridden by: org.jclouds.aws.ec2.compute.config.AWSEC2ComputeServiceContextModule.provideSecurityGroupExtension()
at com.google.inject.internal.ProviderMethodsModule.getProviderMethods(ProviderMethodsModule.java:163)
- relates to
-
JENKINS-38736 update guice to release version
-
- Resolved
-
-
JENKINS-50520 Remove jClouds shading
-
- Resolved
-
[JENKINS-39505] JClouds plugin fails on Jenkins v2.28
Code changed in jenkins
User: Fritz Elfert
Path:
jclouds-plugin/pom.xml
pom.xml
http://jenkins-ci.org/commit/jclouds-plugin/2af5ce412ccb79e70e26a6eab82e6e8ca0049eb4
Log:
Attempt to fix JENKINS-39505
I have just made a commit to https://github.com/jenkinsci/jclouds-plugin which should fix your problem.
Are you able to build this yourself or do you need an installable .hpi?
If you need an installable .hpi, please contact me privately via email.
ATTENTION-ATTENTION-ATTENTION
BEFORE updating, be shure to backup your your <jenkins-homedir>/config.xml and <jenkins-homedir>/credentials.xml !!!!!
Reason: The new plugin uses jenkins credentials framework and auto-migrates any old jclouds-credentials into the new credential-system.
If you want to go downgrade for some reason, you must downgrade the plugin, then stop jenkins and copy the backup over the existing xmls
After that, you can start jenkins again and should have the old configuration.
Correction: The latest build artifact is also available here: https://jenkins.ci.cloudbees.com/job/plugins/job/jclouds-plugin/ws/jclouds-plugin/target/jclouds-jenkins.hpi
Indirectly yes. The well known classloader isolation problem. The jclouds libs depend on an older guice and some other conflicting libs.
As a developer, I was always struggling with that. It is also the reason why a shaded jclouds lib is included in the plugin.
However, I was able to fix this by enabling the pluginFirstClassLoader.
-Fritz
Any further feedback from Reporter or Bernd Farka appreciated!
Did the provided build help?
hi
sorry for the late feedback:
i deployed the new version and suddelny our Jenkins started to take more memory and going OOM...
until now i have no furtherer stacktraces for you, but I'll try to repdoruce it and give you new findings...
some (not very helpful) finding
WARNING: Error while serving https://jenkins-one.lab.dynatrace.org/descriptorByName/jenkins.plugins.jclouds.compute.JCloudsSlaveTemplate/fillLocationIdItems java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.kohsuke.stapler.Function$InstanceFunction.invoke(Function.java:335) at org.kohsuke.stapler.Function.bindAndInvoke(Function.java:175) at org.kohsuke.stapler.Function.bindAndInvokeAndServeResponse(Function.java:108) at org.kohsuke.stapler.MetaClass$1.doDispatch(MetaClass.java:124) at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:58) at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:746) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:876) at org.kohsuke.stapler.MetaClass$5.doDispatch(MetaClass.java:236) at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:58) at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:746) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:876) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:649) at org.kohsuke.stapler.Stapler.service(Stapler.java:238) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:812) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1669) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:135) at org.jenkinsci.plugins.ssegateway.Endpoint$SSEListenChannelFilter.doFilter(Endpoint.java:206) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:132) at io.jenkins.blueocean.ResourceCacheControl.doFilter(ResourceCacheControl.java:134) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:132) at jenkins.metrics.impl.MetricsFilter.doFilter(MetricsFilter.java:125) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:132) at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:126) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:80) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84) at hudson.security.UnwrapSecurityExceptionFilter.doFilter(UnwrapSecurityExceptionFilter.java:51) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at jenkins.security.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:117) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:142) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at jenkins.security.BasicHeaderProcessor.doFilter(BasicHeaderProcessor.java:93) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249) at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:67) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:76) at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:171) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:49) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:82) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:553) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) at org.eclipse.jetty.server.Server.handle(Server.java:499) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257) at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544) at winstone.BoundedExecutorService$1.run(BoundedExecutorService.java:77) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.OutOfMemoryError: Java heap space at java.util.Arrays.copyOf(Arrays.java:3332) at java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:137) at java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:121) at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:421) at java.lang.StringBuilder.append(StringBuilder.java:136) at java.lang.StringBuilder.append(StringBuilder.java:76) at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:457) at java.lang.StringBuilder.append(StringBuilder.java:166) at java.lang.StringBuilder.append(StringBuilder.java:76) at java.util.Formatter$FormatSpecifier.print(Formatter.java:2913) at java.util.Formatter$FormatSpecifier.printString(Formatter.java:2886) at java.util.Formatter$FormatSpecifier.print(Formatter.java:2763) at java.util.Formatter.format(Formatter.java:2526) at java.util.Formatter.format(Formatter.java:2455) at java.lang.String.format(String.java:2940) at java.util.logging.SimpleFormatter.format(SimpleFormatter.java:161) at java.util.logging.StreamHandler.publish(StreamHandler.java:211) at java.util.logging.ConsoleHandler.publish(ConsoleHandler.java:116) at java.util.logging.Logger.log(Logger.java:738) at java.util.logging.Logger.doLog(Logger.java:765) at java.util.logging.Logger.log(Logger.java:875) at jenkins.plugins.jclouds.compute.JCloudsSlaveTemplate$DescriptorImpl.doFillLocationIdItems(JCloudsSlaveTemplate.java:785) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.kohsuke.stapler.Function$InstanceFunction.invoke(Function.java:335) at org.kohsuke.stapler.Function.bindAndInvoke(Function.java:175) at org.kohsuke.stapler.Function.bindAndInvokeAndServeResponse(Function.java:108) at org.kohsuke.stapler.MetaClass$1.doDispatch(MetaClass.java:124) at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:58) at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:746)
fyi: we have 6GB xmx and using 2 on normal conditions...
Have you built the plugin yourself or used the provided one? If you downloaded it: when did you fetch it?
Background:
After providing the link, I also upgraded from jclouds-1.9.1 to jclouds-1.9.2 (which might be the memleak cause)
Can you build the plugin yourself? If yes, the checkout 2af5ce412ccb79e70e26a6eab82e6e8ca0049eb4 which has exacly this single fix.
BTW:
The trace ends up in doFillLocationIdItems() where it iterates over the possible locations from your cloud provider and builds a combo-box from them.
This shold happen only, if you actually click on thet specific combobox - oh and it is old code that has not changed compared to 1.8.1.
Indeed probably purely coincidential.
FYI: In the meantime, I have set up a jenkins 2.28 here and was able to reproduce your original issue (using the old plugin), but not the OOM using the latest plugin snapshot. On the other hand I don't have any OpenNebula (using OpenStack juno here) In other words: My lates build runs fine here.
I downloaded it about 6 hours ago....
i can build it, but not quite now, we would have to wait a few hours until i have time...
If you do, make shure you use maven 3.2.x, not maven-3.3! See README.md
Since I did not get any further feedback, in the meantime, I have released jclouds-plugin v2.9 which runs fine here with 2.28.
Maybe you want to give that one a try.
@Bernd Farka resp. Georg Brunmayr:
Did you had any further chance to test this? (It runs fine here)
We have installed the latest version and the original issue is gone.
The aws ec2 driver against OpenNebula has the tendency to grab all the memory it can get but that's another topic and most likely related to the endpoint. The nova plugin connects fine even though it does not work to provision instances but that's caused by OpenNebula not supporting SecurityGroup creation via the API.
Thank you for your response. Closing this issue now.
Regarding the memory problem:
I am going to release v1.11 soon (next week) which contains an important change:
- Upgrade to jclouds-2.0.0 (which was released last week)
This also implies that jclouds now supports guice-4.0 which is identical to the
guice-version used by jenkins itself. This solves many problems and might
even solve the memory hog.
So: I'd suggest to upgrade when it's available and simply open another issue, if
the memory problem persists.
Which version of the jclouds plugin are you using? (From the plugins page or built from git).