When a plugin extends ConsoleAnnotatorFactory, it can serve a JavaScript file and a CSS file. These resources expire after a day, which means that after upgrading or downgrading the plugin, they remain out of date for a while. This is most likely the cause of JENKINS-31213.

It is possible to work around this within the plugins by renaming the ConsoleAnnotatorFactory subclass every time the JavaScript or CSS files are changed. I wasn't able to find a better workaround - see this comment.

This could be fixed within Jenkins by including the plugin version or the SESSION_HASH in the URL. The plugin version would allow the resources to be cached for longer, but the SESSION_HASH is probably easier to implement.