Details
-
Bug
-
Status: Resolved (View Workflow)
-
Major
-
Resolution: Fixed
-
None
-
Jenkins 1.651.1
Openstack cloud plugin 2.8
Description
After installing Openstack Cloud Plugin 2.8 from 1.x version noticed a lot of spam messages in jenkins.log:
Jun 09, 2016 12:56:08 PM org.jenkinsci.plugins.cloudstats.CloudStatistics getActivityFor WARNING: No activity tracked for ProvisioningActivity for OpenstackCloud_SAS/vanilla-precise-medium-sas/null (546613729) java.lang.IllegalStateException at org.jenkinsci.plugins.cloudstats.CloudStatistics.getActivityFor(CloudStatistics.java:413) at org.jenkinsci.plugins.cloudstats.CloudStatistics$OperationListener.onLaunchFailure(CloudStatistics.java:311) at hudson.slaves.SlaveComputer$1.call(SlaveComputer.java:272) at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Jun 09, 2016 12:56:10 PM org.jenkinsci.plugins.cloudstats.CloudStatistics getActivityFor WARNING: No activity tracked for ProvisioningActivity for OpenstackCloud_SAS/vanilla-precise-medium-sas/null (546613729) java.lang.IllegalStateException at org.jenkinsci.plugins.cloudstats.CloudStatistics.getActivityFor(CloudStatistics.java:413) at org.jenkinsci.plugins.cloudstats.CloudStatistics$OperationListener.preLaunch(CloudStatistics.java:297) at hudson.slaves.SlaveComputer$1.call(SlaveComputer.java:251) at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745)
OpenstackCloud_SAS/vanilla-precise-medium-sas here:
OpenstackCloud_SAS - cloud name
vanilla-precise-medium-sas - slave name
Attachments
Activity
Hello!
Here is my flow:
- delete all nodes
- install Openstack cloud plugin 2.8 (with cloud-sats)
- restart jenkins
- no warnings
- create new nodes
- get this warnings...
- delete nodes
- get this warnings...
- restart jenkins
- no warnings
- create new nodes
- get this warnings...
LawYard LawYard
added a comment - - edited Hello!
Here is my flow:
delete all nodes
install Openstack cloud plugin 2.8 (with cloud-sats)
restart jenkins
no warnings
create new nodes
get this warnings...
delete nodes
get this warnings...
restart jenkins
no warnings
create new nodes
get this warnings...
LawYard LawYard
added a comment - Addition: I do all of that within new server. I installed jenkins and copyed only my jenkins config from production jenkins (with all configured clouds and slave properties).
LawYard LawYard
added a comment - Addition: I do all of that within new server. I installed jenkins and copyed only my jenkins config from production jenkins (with all configured clouds and slave properties). LawYard LawYard
added a comment - Hello! Is there any news? How can I avoid this messages? As I look at the problem again, it seems I was wrong with the analysis. I am trying to reproduce the problem now. Can you grep your $JENKINS_HOME/nodes/ and $JENKINS_HOME/org.jenkinsci.plugins.cloudstats.CloudStatistics.xml for the fingerprint (the number in ())? In case the node still exists and there is no record in statistics, one possible explanation is there is more than 100 records and that particular record was rotated as there are newer ones. Can that be the case you have provisioned ~100 new slaves after given computer that is causing problems?
LawYard LawYard
added a comment - Hi!
There are no records about existing slaves in CloudStatistics.xml. And yes - I created about 400 slaves. I am really need it)
LawYard LawYard
added a comment - Hi!
There are no records about existing slaves in CloudStatistics.xml. And yes - I created about 400 slaves. I am really need it) It seems this is the cause, then. The cloud statistics needs to use 2 collections in fact: one for all activities that are not completed (so updates will never be sent to rotated activities) and the current log will only be used for completed activities that are kept for statistics only.
Does it sill appear for newly provisioned slaves or only for slaves that existed before restart? You see, the plugin is now able to reconnect openstack machine after Jenkins master restart. But if the slave was created before cloud-stats was integrated, the activity is not tracked and this failure can occur.EDIT: It is a bug one way or another but I still would like to know.