Problem

We have a large jenkins instance which hosts nearly up to 1000 Pipeline Jobs, and we mainly use gerrit trigger plugin to interact with gerrit server. But somehow we met some unexpected behavior on gerrit-trigger plugin.  The one listed below is a critical one that may affect our codebase quality.

Issue

Sometimes, part of job can not be triggered by gerrit changes event, we checked the dynamic config url setting and its content, Comparing to the other job, the setting and content is totaly the same. It seems some job just stop working.

Temp Workaround:

Manually re-save the job, "Configure -> "Save", would fix the problem. The "broken" job can finally behavior as normal. 

Logs

But this problem happens from time to time, so we debugged into it and found some useful logging information for your information, as attached:

1. Timer task which updating the same one dynamic config url file,  going bad from some point.

2. In some other case, a exception was thrown saying this is not a SafeTimerTask.

3. Study & Investigate

By looking into source code, we found the timertask use "TimerTask", which generally also bring problem and confusion to other people.

 https://stackoverflow.com/questions/19944784/if-a-task-encounters-an-exception-its-further-executions-are-suppressed-why
https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/ScheduledExecutorService.html

 
 
 
Suggest Solution
 
May be we could use SafeTimerTask instead.