-
Bug
-
Resolution: Fixed
-
Minor
Builds can be deleted via the /doDelete REST API, and the delete-builds CLI command while they are still running. The “Delete build” link in the sidepanel of builds is hidden unless !it.building and !it.keepLog, but this is a UI-only restriction. Run.delete has the same issue, but in practice all of the callers that I found in plugins already check whether the build is done building (most calls are in LogRotator implementations where LogRotator.shouldKeepRun handles the issue), but there are some subtle race conditions regarding checking Run.isBuilding instead of Run.isLogUpdated that can lead to LogRotator deleting Pipeline builds while they are in the middle of finishing.
Deleting a build while it is running can cause various kinds of anomalous behavior as the build continues to run despite its configuration directory being deleted, and it is likely that the build will return after a Jenkins restart since build.xml may be rewritten as the build continues running.
It is awkward to handle this in Run.delete because of how and where LogRotator is called today. When LogRotator runs for a Freestyle build, this call is guaranteed to observe !Run.isBuilding() and Run.isLogUpdated(). For Pipelines this timer task will always observe both !Run.isBuilding and !Run.isLogUpdated. Further, for Pipelines, using !Run.isBuilding() to detect when a build has completed is inaccurate. You must check Run.isLogUpdated() instead to avoid doing things while Pipeline cleanup is still ongoing.
This means we can't make Run.delete just check Run.isBuilding() || Run.isLogUpdated() or else log rotation may throw spurious exceptions for Freestyle builds, and we ideally don't want to only check Run.isBuilding() to avoid race conditions when attempting to delete Pipelines while they are being cleaned up leading to leftover directories with a single build.xml file due to this call running after LogRotator.
So we either need to slightly change when Freestyle jobs trigger log rotation so that it happens while !Run.isLogUpdated, adjust WorkflowRun.isBuilding to match WorkflowRun.isLogUpdated, or fix the issue only for the API and CLI commands directly and ignore the existing race condition with Pipelines and log rotation.