After upgrading influxDB from 2.5 to 3.0.0 we started to have a huge amount of thread leak causing our jenkins instance to fail. We had to revert and redeploy Jenkins, and I was not able to capture a thread dump. I will update this ticket if/when I am able to have more information on the thread leak.
It happened about ~6h after startup, so for us this means after about 15,000 jobs, all with post-deploy influxDb publisher.
We used telegraf to have an influxDB listener v2, and configured the plugin to use the v2 HTTP client.
causes
JENKINS-66017OOM: unable to create new native thread (many RxNewThreadScheduler threads)
If this strated after upgrade and your 15 000 jobs have run perfectly until then, I believe the issue is the changed API. influxdb-java doesn't support InfluxDB 2.0, so I had to change it to influxdb-java-client and at least there is a ticket about threading. Very likely it's caused by that. I have to check if this can be avoided by calling some other functions.
Aleksi Simell
added a comment - - edited Hi!
If this strated after upgrade and your 15 000 jobs have run perfectly until then, I believe the issue is the changed API. influxdb-java doesn't support InfluxDB 2.0, so I had to change it to influxdb-java-client and at least there is a ticket about threading. Very likely it's caused by that. I have to check if this can be avoided by calling some other functions.
Joe Athman
added a comment - I think the bug is that this line: https://github.com/jenkinsci/influxdb-plugin/blob/970319deaf87b848edafa6b5056ab6ba6ce88bc8/src/main/java/jenkinsci/plugins/influxdb/InfluxDbPublicationService.java#L354 calls the `getWriteApi` for each metric. This is not how the API is intended to work, the influx client and the write api are intended to be singletons. The ticket you link to shows that many people have this same problem. It is not clear at all that calling the getWriteApi will cause problems when done from a loop.
The getWriteApi() is not called for each metric. It's called for each separate target. The measurement points are batched into a list, which is written for multiple InfluxDB databases (you're able to have global listeners that write data from every job without specifying it directly in the job). And since you have the option to write to multiple targets, you need a separate InfluxDBClient for each target.
Aleksi Simell
added a comment - - edited Hi jjathman
The getWriteApi() is not called for each metric. It's called for each separate target. The measurement points are batched into a list, which is written for multiple InfluxDB databases (you're able to have global listeners that write data from every job without specifying it directly in the job). And since you have the option to write to multiple targets, you need a separate InfluxDBClient for each target.
The getWriteApi() is supposed to run as a singleton because it is use a background thread to ingesting data into InfluxDB. The better choice for influxdb-pluging is getWriteApiBlocking(), also the InfluxDBClient should be closed to free-up resources.
Jakub
added a comment - Hi All,
The getWriteApi() is supposed to run as a singleton because it is use a background thread to ingesting data into InfluxDB. The better choice for influxdb-pluging is getWriteApiBlocking() , also the InfluxDBClient should be closed to free-up resources.
I've prepare the PR with fixes: https://github.com/jenkinsci/influxdb-plugin/pull/121
Regards
aleksisimell - you are correct, it's not called for each metric, but is called for each invocation of the step/target.
Looks like jakub_bednar has a nice fix though. Hope we can get that PR merged in quickly. Thanks!
Joe Athman
added a comment - aleksisimell - you are correct, it's not called for each metric, but is called for each invocation of the step/target.
Looks like jakub_bednar has a nice fix though. Hope we can get that PR merged in quickly. Thanks!
{"searchers":{"groups":[{"searchers":[{"name":"Project","id":"project","key":"issue.field.project","isShown":true,"lastViewed":1741406767921},{"name":"Summary","id":"summary","key":"issue.field.summary","isShown":true},{"name":"Type","id":"issuetype","key":"issue.field.issuetype","isShown":true,"lastViewed":1741406767922},{"name":"Status","id":"status","key":"issue.field.status","isShown":true,"lastViewed":1741406767926},{"name":"Priority","id":"priority","key":"issue.field.priority","isShown":true},{"name":"Resolution","id":"resolution","key":"issue.field.resolution","isShown":true},{"name":"Creator","id":"creator","key":"issue.field.creator","isShown":true},{"name":"Component","id":"component","key":"issue.field.components","isShown":true},{"name":"% Limits","id":"workratio","key":"issue.field.workratio","isShown":true},{"name":"Link types","id":"issue_link_type","key":"issue.field.issuelinks","isShown":true},{"name":"Environment","id":"environment","key":"issue.field.environment","isShown":true},{"name":"Description","id":"description","key":"issue.field.description","isShown":true},{"name":"Comment","id":"comment","key":"issue.field.comment","isShown":true},{"name":"Label","id":"labels","key":"issue.field.labels","isShown":true,"lastViewed":1741406767937},{"name":"Query","id":"text","key":"text","isShown":true},{"name":"Bonfire Browser","id":"customfield_10229","key":"com.atlassian.bonfire.plugin:bonfire-text","isShown":true},{"name":"Bonfire Operating System","id":"customfield_10231","key":"com.atlassian.bonfire.plugin:bonfire-text","isShown":true},{"name":"Bonfire Screen Resolution","id":"customfield_10244","key":"com.atlassian.bonfire.plugin:bonfire-text","isShown":true},{"name":"Bonfire URL","id":"customfield_10237","key":"com.atlassian.bonfire.plugin:bonfire-text","isShown":true},{"name":"Bonfire User Agent","id":"customfield_10226","key":"com.atlassian.bonfire.plugin:bonfire-text","isShown":true},{"name":"Bonfire jQuery Version","id":"customfield_10252","key":"com.atlassian.bonfire.plugin:bonfire-text","isShown":true},{"name":"Business Value","id":"customfield_10333","key":"com.atlassian.jira.plugin.system.customfieldtypes:float","isShown":false},{"name":"Development","id":"customfield_10720","key":"com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary","isShown":true},{"name":"Epic Color","id":"customfield_10328","key":"com.pyxis.greenhopper.jira:gh-epic-color","isShown":false},{"name":"Epic Link","id":"customfield_10325","key":"com.pyxis.greenhopper.jira:gh-epic-link","isShown":true},{"name":"Epic Name","id":"customfield_10327","key":"com.pyxis.greenhopper.jira:gh-epic-label","isShown":true},{"name":"Epic Status","id":"customfield_10326","key":"com.pyxis.greenhopper.jira:gh-epic-status","isShown":false},{"name":"Epic/Theme","id":"customfield_10331","key":"com.atlassian.jira.plugin.system.customfieldtypes:labels","isShown":true},{"name":"Flagged","id":"customfield_10330","key":"com.atlassian.jira.plugin.system.customfieldtypes:multicheckboxes","isShown":true},{"name":"GitHub Users to Authorize as Committers","id":"customfield_10323","key":"com.atlassian.jira.plugin.system.customfieldtypes:textarea","isShown":true},{"name":"Issue Tracker","id":"customfield_11320","key":"com.atlassian.jira.plugin.system.customfieldtypes:select","isShown":true},{"name":"JIRA Capture Browser","id":"customfield_10228","key":"com.atlassian.bonfire.plugin:bonfire-text","isShown":true},{"name":"JIRA Capture Browser","id":"customfield_10230","key":"com.atlassian.bonfire.plugin:bonfire-text","isShown":true},{"name":"JIRA Capture Document Mode","id":"customfield_10258","key":"com.atlassian.bonfire.plugin:bonfire-text","isShown":true},{"name":"JIRA Capture Operating System","id":"customfield_10232","key":"com.atlassian.bonfire.plugin:bonfire-text","isShown":true},{"name":"JIRA Capture Operating System","id":"customfield_10233","key":"com.atlassian.bonfire.plugin:bonfire-text","isShown":true},{"name":"JIRA Capture Operating System","id":"customfield_10234","key":"com.atlassian.bonfire.plugin:bonfire-text","isShown":true},{"name":"JIRA Capture Operating System","id":"customfield_10236","key":"com.atlassian.bonfire.plugin:bonfire-text","isShown":true},{"name":"JIRA Capture Operating System","id":"customfield_10238","key":"com.atlassian.bonfire.plugin:bonfire-text","isShown":true},{"name":"JIRA Capture Operating System","id":"customfield_10239","key":"com.atlassian.bonfire.plugin:bonfire-text","isShown":true},{"name":"JIRA Capture Screen Resolution","id":"customfield_10245","key":"com.atlassian.bonfire.plugin:bonfire-text","isShown":true},{"name":"JIRA Capture Screen Resolution","id":"customfield_10246","key":"com.atlassian.bonfire.plugin:bonfire-text","isShown":true},{"name":"JIRA Capture Screen Resolution","id":"customfield_10247","key":"com.atlassian.bonfire.plugin:bonfire-text","isShown":true},{"name":"JIRA Capture Screen Resolution","id":"customfield_10248","key":"com.atlassian.bonfire.plugin:bonfire-text","isShown":true},{"name":"JIRA Capture Screen Resolution","id":"customfield_10249","key":"com.atlassian.bonfire.plugin:bonfire-text","isShown":true},{"name":"JIRA Capture Screen Resolution","id":"customfield_10250","key":"com.atlassian.bonfire.plugin:bonfire-text","isShown":true},{"name":"JIRA Capture URL","id":"customfield_10240","key":"com.atlassian.bonfire.plugin:bonfire-text","isShown":true},{"name":"JIRA Capture URL","id":"customfield_10241","key":"com.atlassian.bonfire.plugin:bonfire-text","isShown":true},{"name":"JIRA Capture URL","id":"customfield_10242","key":"com.atlassian.bonfire.plugin:bonfire-text","isShown":true},{"name":"JIRA Capture URL","id":"customfield_10243","key":"com.atlassian.bonfire.plugin:bonfire-text","isShown":true},{"name":"JIRA Capture User Agent","id":"customfield_10225","key":"com.atlassian.bonfire.plugin:bonfire-text","isShown":true},{"name":"JIRA Capture User Agent","id":"customfield_10227","key":"com.atlassian.bonfire.plugin:bonfire-text","isShown":true},{"name":"JIRA Capture jQuery Version","id":"customfield_10251","key":"com.atlassian.bonfire.plugin:bonfire-text","isShown":true},{"name":"JIRA Capture jQuery Version","id":"customfield_10253","key":"com.atlassian.bonfire.plugin:bonfire-text","isShown":true},{"name":"JIRA Capture jQuery Version","id":"customfield_10254","key":"com.atlassian.bonfire.plugin:bonfire-text","isShown":true},{"name":"JIRA Capture jQuery Version","id":"customfield_10255","key":"com.atlassian.bonfire.plugin:bonfire-text","isShown":true},{"name":"JIRA Capture jQuery Version","id":"customfield_10256","key":"com.atlassian.bonfire.plugin:bonfire-text","isShown":true},{"name":"JIRA Capture jQuery Version","id":"customfield_10257","key":"com.atlassian.bonfire.plugin:bonfire-text","isShown":true},{"name":"Meeting minutes URL","id":"customfield_10020","key":"com.atlassian.jira.plugin.system.customfieldtypes:url","isShown":false},{"name":"New Repository Name","id":"customfield_10321","key":"com.atlassian.jira.plugin.system.customfieldtypes:textfield","isShown":true},{"name":"Original story points","id":"customfield_11423","key":"com.atlassian.jpo:jpo-custom-field-original-story-points","isShown":true},{"name":"Parent Link","id":"customfield_11420","key":"com.atlassian.jpo:jpo-custom-field-parent","isShown":false},{"name":"Plugin Description","id":"customfield_10322","key":"com.atlassian.jira.plugin.system.customfieldtypes:textarea","isShown":true},{"name":"Raised During","id":"customfield_10220","key":"com.atlassian.bonfire.plugin:bonfire-session-cft","isShown":false},{"name":"Raised During","id":"customfield_10221","key":"com.atlassian.bonfire.plugin:bonfire-session-cft","isShown":false},{"name":"Rank","id":"customfield_10324","key":"com.pyxis.greenhopper.jira:gh-lexo-rank","isShown":true},{"name":"Released As","id":"customfield_10620","key":"com.atlassian.jira.plugin.system.customfieldtypes:textfield","isShown":true},{"name":"Repository URL","id":"customfield_10320","key":"com.atlassian.jira.plugin.system.customfieldtypes:url","isShown":true},{"name":"Similar Issues","id":"customfield_10520","key":"com.suggestimate:similar-issues-custom-field","isShown":true},{"name":"Sprint","id":"customfield_10329","key":"com.pyxis.greenhopper.jira:gh-sprint","isShown":true},{"name":"Story Points","id":"customfield_10332","key":"com.atlassian.jira.plugin.system.customfieldtypes:float","isShown":false},{"name":"Team","id":"customfield_11424","key":"com.atlassian.teams:rm-teams-custom-field-team","isShown":true},{"name":"Test Sessions","id":"customfield_10222","key":"com.atlassian.bonfire.plugin:bonfire-multi-session-cft","isShown":false},{"name":"Test Sessions","id":"customfield_10223","key":"com.atlassian.bonfire.plugin:bonfire-multi-session-cft","isShown":false},{"name":"Test Sessions","id":"customfield_10224","key":"com.atlassian.bonfire.plugin:bonfire-multi-session-cft","isShown":false},{"name":"Testing Status","id":"customfield_10259","key":"com.atlassian.bonfire.plugin:bonfire-testing-status-cft","isShown":false},{"name":"URL","id":"customfield_10000","key":"com.atlassian.jira.plugin.system.customfieldtypes:url","isShown":true}],"type":"DETAILS","title":"Details"},{"searchers":[{"name":"Created Date","id":"created","key":"issue.field.created","isShown":true},{"name":"Updated Date","id":"updated","key":"issue.field.updated","isShown":true},{"name":"Resolution Date","id":"resolutiondate","key":"issue.field.resolution.date","isShown":true},{"name":"Target end","id":"customfield_11422","key":"com.atlassian.jpo:jpo-custom-field-baseline-end","isShown":true},{"name":"Target start","id":"customfield_11421","key":"com.atlassian.jpo:jpo-custom-field-baseline-start","isShown":true}],"type":"DATES","title":"Dates"},{"searchers":[{"name":"Assignee","id":"assignee","key":"issue.field.assignee","isShown":true,"lastViewed":1741406767933},{"name":"Reporter","id":"reporter","key":"issue.field.reporter","isShown":true}],"type":"PEOPLE","title":"People"}]},"values":{"issuetype":{"name":"Type","editHtml":"\n\n\n\n <div class=\"field-group aui-field-issuetype\" >\n <label for=\"searcher-type\">Type</label> <select class=\"select js-default-checkboxmultiselect\"\n id=\"searcher-type\"\n multiple=\"multiple\"\n name=\"type\"\n data-max-inline-results-displayed=\"100\"\n data-placeholder-text=\"Find Issue Types...\">\n <optgroup>\n \n <option class=\" \"\n id=\"type_-2\"\n title=\"All Standard Issue Types\"\n value=\"-2\">All Standard Issue Types</option>\n </optgroup>\n\n <optgroup label=\"Standard Issue Types\">\n \n <option class=\" imagebacked 10730 \"\n data-icon=\"/secure/viewavatar?size=xsmall&avatarId=14673&avatarType=issuetype\"\n data-fallback-icon=\"/images/icons/issuetypes/blank.png\"\n id=\"type_1\"\n title=\"Bug\"\n value=\"1\">Bug</option>\n \n <option class=\" imagebacked 10730 \"\n data-icon=\"/images/icons/issuetypes/epic.png\"\n data-fallback-icon=\"/images/icons/issuetypes/blank.png\"\n id=\"type_10001\"\n title=\"Epic\"\n value=\"10001\">Epic</option>\n \n <option class=\" imagebacked 10730 \"\n data-icon=\"/secure/viewavatar?size=xsmall&avatarId=14680&avatarType=issuetype\"\n data-fallback-icon=\"/images/icons/issuetypes/blank.png\"\n id=\"type_4\"\n title=\"Improvement\"\n value=\"4\">Improvement</option>\n \n <option class=\" imagebacked 10730 \"\n data-icon=\"/secure/viewavatar?size=xsmall&avatarId=14681&avatarType=issuetype\"\n data-fallback-icon=\"/images/icons/issuetypes/blank.png\"\n id=\"type_2\"\n title=\"New Feature\"\n value=\"2\">New Feature</option>\n \n <option class=\" imagebacked 10730 \"\n data-icon=\"/secure/viewavatar?size=xsmall&avatarId=14670&avatarType=issuetype\"\n data-fallback-icon=\"/images/icons/issuetypes/blank.png\"\n id=\"type_5\"\n title=\"Patch\"\n value=\"5\">Patch</option>\n \n <option class=\" imagebacked 10730 \"\n data-icon=\"/secure/viewavatar?size=xsmall&avatarId=14685&avatarType=issuetype\"\n data-fallback-icon=\"/images/icons/issuetypes/blank.png\"\n id=\"type_10002\"\n title=\"Story\"\n value=\"10002\">Story</option>\n \n <option class=\" imagebacked 10730 \"\n data-icon=\"/secure/viewavatar?size=xsmall&avatarId=14688&avatarType=issuetype\"\n data-fallback-icon=\"/images/icons/issuetypes/blank.png\"\n id=\"type_3\"\n title=\"Task\"\n value=\"3\">Task</option>\n </optgroup>\n\n <optgroup label=\"Sub-Task Issue Types\">\n </optgroup>\n </select>\n </div>\n ","validSearcher":true,"isShown":true},"project":{"name":"Project","editHtml":" \n <div class=\"field-group aui-field-project\" >\n <label for=\"searcher-pid\">Project</label> <select class=\"js-project-checkboxmultiselect\"\n data-placeholder-text=\"Find Projects...\"\n id=\"searcher-pid\"\n multiple=\"multiple\"\n name=\"pid\">\n <optgroup label=\"Recent Projects\">\n </optgroup>\n <optgroup label=\"All Projects\" >\n \n <option data-icon=\"/secure/projectavatar?pid=10172&size=small\"\n title=\"Jenkins\"\n value=\"10172\">\n Jenkins (JENKINS)\n </option>\n <option data-icon=\"/secure/projectavatar?pid=10050&size=small\"\n title=\"test\"\n value=\"10050\">\n test (TEST)\n </option>\n </optgroup>\n </select>\n </div>\n \n\n","validSearcher":true,"isShown":true},"assignee":{"name":"Assignee","editHtml":"\n \n <div class=\"field-group aui-field-userlist\" >\n <label for=\"searcher-assigneeSelect\">Assignee</label> <fieldset rel=\"assignee\" class=\"hidden user-group-searcher-params\">\n </fieldset>\n <select class=\"js-usergroup-checkboxmultiselect\" multiple=\"multiple\" id=\"assignee\" name=\"assignee\" data-placeholder-text=\"Enter username or group\">\n <optgroup>\n <option class=\"headerOption\" data-icon=\"https://issues.jenkins.io/secure/useravatar?size=xsmall&avatarId=10293\" value=\"empty\" title=\"Unassigned\">Unassigned</option>\n </optgroup>\n <optgroup>\n </optgroup>\n </select>\n <input type=\"hidden\" name=\"check_prev_assignee\" value=\"true\">\n </div>\n \n","validSearcher":true,"isShown":true},"status":{"name":"Status","editHtml":"\n <div class=\"field-group aui-field-constants\" >\n <label for=\"searcher-status\">Status</label> <select class=\"select js-default-checkboxmultiselectstatuslozenge\"\n data-placeholder-text=\"Find Statuses...\"\n id=\"searcher-status\"\n multiple=\"multiple\"\n name=\"status\"\n data-max-inline-results-displayed=\"100\"\n data-footer-text=\"-88 more options. Continue typing to refine further.\" data-status-lozenge=\"true\">\n <optgroup >\n <option class=\"imagebacked\" data-icon=\"/images/icons/statuses/needinfo.png\" value=\"10001\" title=\"Untriaged\" data-simple-status=\"{"id":"10001","name":"Untriaged","description":"New issue sits in this state until a developer agrees that this is a security issue","iconUrl":"/images/icons/statuses/needinfo.png","statusCategory":{"id":2,"key":"new","colorName":"default"}}\">Untriaged</option>\n <option class=\"imagebacked\" data-icon=\"/images/icons/statuses/open.png\" value=\"1\" title=\"Open\" data-simple-status=\"{"id":"1","name":"Open","description":"The issue is open and ready for the assignee to start work on it.","iconUrl":"/images/icons/statuses/open.png","statusCategory":{"id":2,"key":"new","colorName":"default"}}\">Open</option>\n <option class=\"imagebacked\" data-icon=\"/images/icons/statuses/inprogress.png\" value=\"3\" title=\"In Progress\" data-simple-status=\"{"id":"3","name":"In Progress","description":"This issue is being actively worked on at the moment by the assignee.","iconUrl":"/images/icons/statuses/inprogress.png","statusCategory":{"id":4,"key":"indeterminate","colorName":"inprogress"}}\">In Progress</option>\n <option class=\"imagebacked\" data-icon=\"/images/icons/statuses/reopened.png\" value=\"4\" title=\"Reopened\" data-simple-status=\"{"id":"4","name":"Reopened","description":"This issue was once resolved, but the resolution was deemed incorrect. From here issues are either marked assigned or resolved.","iconUrl":"/images/icons/statuses/reopened.png","statusCategory":{"id":4,"key":"indeterminate","colorName":"inprogress"}}\">Reopened</option>\n <option class=\"imagebacked\" data-icon=\"/images/icons/statuses/information.png\" value=\"10005\" title=\"In Review\" data-simple-status=\"{"id":"10005","name":"In Review","description":"","iconUrl":"/images/icons/statuses/information.png","statusCategory":{"id":4,"key":"indeterminate","colorName":"inprogress"}}\">In Review</option>\n <option class=\"imagebacked\" data-icon=\"/images/icons/statuses/resolved.png\" value=\"10002\" title=\"Fix Prepared\" data-simple-status=\"{"id":"10002","name":"Fix Prepared","description":"A fix is implemented and is waiting for the next security release","iconUrl":"/images/icons/statuses/resolved.png","statusCategory":{"id":4,"key":"indeterminate","colorName":"inprogress"}}\">Fix Prepared</option>\n <option class=\"imagebacked\" data-icon=\"/images/icons/statuses/generic.png\" value=\"10000\" title=\"Verified\" data-simple-status=\"{"id":"10000","name":"Verified","description":"Verified","iconUrl":"/images/icons/statuses/generic.png","statusCategory":{"id":4,"key":"indeterminate","colorName":"inprogress"}}\">Verified</option>\n <option class=\"imagebacked\" data-icon=\"/images/icons/statuses/generic.png\" value=\"10203\" title=\"Fixed but Unreleased\" data-simple-status=\"{"id":"10203","name":"Fixed but Unreleased","description":"This change has been implemented and merged, but not yet released.","iconUrl":"/images/icons/statuses/generic.png","statusCategory":{"id":3,"key":"done","colorName":"success"}}\">Fixed but Unreleased</option>\n <option class=\"imagebacked\" data-icon=\"/images/icons/statuses/resolved.png\" value=\"5\" title=\"Resolved\" data-simple-status=\"{"id":"5","name":"Resolved","description":"A developer had implemented a fix and is waiting for a feedback from the reporter.","iconUrl":"/images/icons/statuses/resolved.png","statusCategory":{"id":3,"key":"done","colorName":"success"}}\">Resolved</option>\n <option class=\"imagebacked\" data-icon=\"/images/icons/statuses/closed.png\" value=\"6\" title=\"Closed\" data-simple-status=\"{"id":"6","name":"Closed","description":"The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.","iconUrl":"/images/icons/statuses/closed.png","statusCategory":{"id":3,"key":"done","colorName":"success"}}\">Closed</option>\n <option class=\"imagebacked\" data-icon=\"/images/icons/statuses/open.png\" value=\"10003\" title=\"To Do\" data-simple-status=\"{"id":"10003","name":"To Do","description":"","iconUrl":"/images/icons/statuses/open.png","statusCategory":{"id":2,"key":"new","colorName":"default"}}\">To Do</option>\n <option class=\"imagebacked\" data-icon=\"/images/icons/statuses/closed.png\" value=\"10004\" title=\"Done\" data-simple-status=\"{"id":"10004","name":"Done","description":"","iconUrl":"/images/icons/statuses/closed.png","statusCategory":{"id":3,"key":"done","colorName":"success"}}\">Done</option>\n </optgroup>\n</select>\n </div>\n \n","validSearcher":true,"isShown":true},"labels":{"name":"Label","viewHtml":" <div class=\"searcherValue\">\n \n <label class=\"fieldLabel\" for=\"fieldlabels\">Label:</label><span id=\"fieldlabels\" class=\"fieldValue\">\n \n influxdb\n</span></div>\n","editHtml":"\n <div class=\"field-group aui-field-labels\" >\n <label for=\"searcher-labels\">Labels</label> <select class=\"js-label-checkboxmultiselect\" multiple=\"multiple\" id=\"searcher-labels\" name=\"labels\" data-placeholder-text=\"Find Labels...\">\n <option value=\"influxdb\" title=\"influxdb\" selected=\"selected\">influxdb</option>\n </select>\n </div>\n \n","jql":"labels = influxdb","validSearcher":true,"isShown":true}}}
[{"id":-1,"name":"My open issues","jql":"assignee = currentUser() AND resolution = Unresolved order by updated DESC","isSystem":true,"sharePermissions":[],"requiresLogin":true},{"id":-2,"name":"Reported by me","jql":"reporter = currentUser() order by created DESC","isSystem":true,"sharePermissions":[],"requiresLogin":true},{"id":-4,"name":"All issues","jql":"order by created DESC","isSystem":true,"sharePermissions":[],"requiresLogin":false},{"id":-5,"name":"Open issues","jql":"resolution = Unresolved order by priority DESC,updated DESC","isSystem":true,"sharePermissions":[],"requiresLogin":false},{"id":-9,"name":"Done issues","jql":"statusCategory = Done order by updated DESC","isSystem":true,"sharePermissions":[],"requiresLogin":false},{"id":-3,"name":"Viewed recently","jql":"issuekey in issueHistory() order by lastViewed DESC","isSystem":true,"sharePermissions":[],"requiresLogin":false},{"id":-6,"name":"Created recently","jql":"created >= -1w order by created DESC","isSystem":true,"sharePermissions":[],"requiresLogin":false},{"id":-7,"name":"Resolved recently","jql":"resolutiondate >= -1w order by updated DESC","isSystem":true,"sharePermissions":[],"requiresLogin":false},{"id":-8,"name":"Updated recently","jql":"updated >= -1w order by updated DESC","isSystem":true,"sharePermissions":[],"requiresLogin":false}]
Bug
Critical
Joe Athman
added a comment - 
Jakub
added a comment -
Hi!
If this strated after upgrade and your 15 000 jobs have run perfectly until then, I believe the issue is the changed API. influxdb-java doesn't support InfluxDB 2.0, so I had to change it to influxdb-java-client and at least there is a ticket about threading. Very likely it's caused by that. I have to check if this can be avoided by calling some other functions.