-
Bug
-
Resolution: Unresolved
-
Minor
-
None
-
Jenkins 2.401.1
Audit Trail : 333.vb_e1b_b_0f1238c
Azure VM Agents : 859.v7213476e4fea_
When logging build start/end on azure vm agents, the node name is not displayed, we have #unknown# instead.
Jul 13, 2023 11:23:28,889 AM - myjob #1 Push event to branch main, Parameters:[] on node #unknown# started at 2023-07-13T09:12:02Z completed in 685713ms completed: SUCCESS
[JENKINS-71637] node names are not logged
I can see the same #unknown# in the audit logs for our pipelines when they run on regular on-prem SSH nodes, which doesn't really surprise me much, since we have pipelines that specify agent none as well as those that are created dynamically on the fly, and also considering that pipelines are not shown in e.g. the Status and Build History views for nodes.
For "old school" Freestyle jobs, the node name appears in the "build start" Audit log lines.
pierrebtz how can I help you to fix this ?
With your link, I don't think the matter is that I use Azure VM Agents but more the usage of organization Folder which produce WorkflowRun pipelines instead of AbstractRun
cpottiers njesper indeed there is also another possible case where the node doesn't exist anymore in Jenkins at the time the plugin code is called.
I created a PR that does two things:
- In case the Run is not an AbtractBuild, log the information about the classname. This will help me figuring out a good solution in this case.
- In case the agent doesn't exist anymore, I used a second method available in AbstractRun that should allow to retrieve the agent name.
pierrebtz
It confirm my doubt :
Nov 08, 2023 4:12:12 PM INFO hudson.plugins.audit_trail.AuditTrailRunListener buildNodeName
Run is not an AbstractBuild but a org.jenkinsci.plugins.workflow.job.WorkflowRun, will log the build node as #unknown#.
cpottiers I see. I started a draft PR that tries to extract all the agents used in a workflow run. I ran some tests with no agents (agent none), an agent and several agents in parallel and got the expected results. Would you mind trying on your setup?
This is WIP, I need to polish it a bit and add some tests but I'd like to check if it answers your needs.
... Nov 10, 2023 11:02:54,119 AM - job/myorg/job/test/job/master/ #3 Started by user POTTIERS Cyril, Parameters:[] Nov 10, 2023 11:05:06,363 AM - myorg » test » master #3 Started by user POTTIERS Cyril, Parameters:[] on node no agent started at 2023-11-10T10:02:53Z completed in 131903ms completed: SUCCESS ...
...
[Pipeline] Start of Pipeline
[Pipeline] node
Still waiting to schedule task
Waiting for next available executor
Running on vm-forge-agentc2f100 in /home/jenkins/workspace/myorg_test_master
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Declarative: Checkout SCM)
...
cpottiers could you please share the relevant part of the pipeline definition (I don't care for the steps you execute, but I'd like to see how you declare your agents/nodes). Are you using declarative or scripted pipelines?
Sure :
pipeline {
agent {
label 'azure-agent'
}
stages {
...
}
}
I'm really not sure what's wrong here. My development works with clouds I tested with the Kubernetes plugin. I just made a quick check against Azure and got the proper agent name. I'm also using the same syntax to declare the agent...
I made a change to the PR (https://github.com/jenkinsci/audit-trail-plugin/pull/128) to log a bunch of additional details about your Pipeline in the hope of finding the root cause for this discrepancy. Could you run again the pipeline again this special version. After running the Pipeline, you should find a block looking like the following in your Jenkins logs:
```
2023-11-15 20:08:46.770+0000 [id=109] INFO h.p.a.AuditTrailRunListener#printNodes: -----------Node enumeration starting-----------
2023-11-15 20:08:46.777+0000 [id=109] INFO h.p.a.AuditTrailRunListener#lambda$printNodes$0: Node: FlowEndNode[id=32, exec=CpsFlowExecutionOwner[pipeline3/5:pipeline3 #5]] | Display name: End of Pipeline | StepArgumentsAsString: null
2023-11-15 20:08:46.780+0000 [id=109] INFO h.p.a.AuditTrailRunListener#lambda$printNodes$0: Node: StepEndNode[id=31, exec=CpsFlowExecutionOwner[pipeline3/5:pipeline3 #5]] | Display name: Stage : End | StepArgumentsAsString: null
// ...
2023-11-15 20:08:46.784+0000 [id=109] INFO h.p.a.AuditTrailRunListener#printNodes: -----------Node enumeration done-----------
```
Could you please attach those logs to the ticket? Make sure to go through them to make sure it doesn't contain any private information you don't want shared (it should log things like agent name, execute steps etc).
Nov 16, 2023 1:42:33 PM INFO hudson.plugins.audit_trail.AuditTrailRunListener printNodes ------------Node enumeration starting------------ Nov 16, 2023 1:42:33 PM INFO hudson.plugins.audit_trail.AuditTrailRunListener lambda$printNodes$0 Node: FlowEndNode[id=21, exec=CpsFlowExecution[Owner[myorg/test/master/6:myorg/test/master #6]]] | Display name: End of Pipeline | StepArgumentsAsString: null Nov 16, 2023 1:42:33 PM INFO hudson.plugins.audit_trail.AuditTrailRunListener lambda$printNodes$0 Node: StepEndNode[id=20, exec=CpsFlowExecution[Owner[myorg/test/master/6:myorg/test/master #6]]] | Display name: Allocate node : End | StepArgumentsAsString: null Nov 16, 2023 1:42:33 PM INFO hudson.plugins.audit_trail.AuditTrailRunListener lambda$printNodes$0 Node: StepEndNode[id=19, exec=CpsFlowExecution[Owner[myorg/test/master/6:myorg/test/master #6]]] | Display name: Allocate node : Body : End | StepArgumentsAsString: null Nov 16, 2023 1:42:33 PM INFO hudson.plugins.audit_trail.AuditTrailRunListener lambda$printNodes$0 Node: StepEndNode[id=18, exec=CpsFlowExecution[Owner[myorg/test/master/6:myorg/test/master #6]]] | Display name: Set environment variables : End | StepArgumentsAsString: null Nov 16, 2023 1:42:33 PM INFO hudson.plugins.audit_trail.AuditTrailRunListener lambda$printNodes$0 Node: StepEndNode[id=17, exec=CpsFlowExecution[Owner[myorg/test/master/6:myorg/test/master #6]]] | Display name: Set environment variables : Body : End | StepArgumentsAsString: null Nov 16, 2023 1:42:33 PM INFO hudson.plugins.audit_trail.AuditTrailRunListener lambda$printNodes$0 Node: StepEndNode[id=16, exec=CpsFlowExecution[Owner[myorg/test/master/6:myorg/test/master #6]]] | Display name: Stage : End | StepArgumentsAsString: null Nov 16, 2023 1:42:33 PM INFO hudson.plugins.audit_trail.AuditTrailRunListener lambda$printNodes$0 Node: StepEndNode[id=15, exec=CpsFlowExecution[Owner[myorg/test/master/6:myorg/test/master #6]]] | Display name: Stage : Body : End | StepArgumentsAsString: null Nov 16, 2023 1:42:33 PM INFO hudson.plugins.audit_trail.AuditTrailRunListener lambda$printNodes$0 Node: StepAtomNode[id=14, exec=CpsFlowExecution[Owner[myorg/test/master/6:myorg/test/master #6]]] | Display name: Print Message | StepArgumentsAsString: Hello World Nov 16, 2023 1:42:33 PM INFO hudson.plugins.audit_trail.AuditTrailRunListener lambda$printNodes$0 Node: StepStartNode[id=13, exec=CpsFlowExecution[Owner[myorg/test/master/6:myorg/test/master #6]]] | Display name: Hello | StepArgumentsAsString: null Nov 16, 2023 1:42:33 PM INFO hudson.plugins.audit_trail.AuditTrailRunListener lambda$printNodes$0 Node: StepStartNode[id=12, exec=CpsFlowExecution[Owner[myorg/test/master/6:myorg/test/master #6]]] | Display name: Stage : Start | StepArgumentsAsString: Hello Nov 16, 2023 1:42:33 PM INFO hudson.plugins.audit_trail.AuditTrailRunListener lambda$printNodes$0 Node: StepStartNode[id=11, exec=CpsFlowExecution[Owner[myorg/test/master/6:myorg/test/master #6]]] | Display name: Set environment variables : Body : Start | StepArgumentsAsString: null Nov 16, 2023 1:42:33 PM INFO hudson.plugins.audit_trail.AuditTrailRunListener lambda$printNodes$0 Node: StepStartNode[id=10, exec=CpsFlowExecution[Owner[myorg/test/master/6:myorg/test/master #6]]] | Display name: Set environment variables : Start | StepArgumentsAsString: GIT_BRANCH, GIT_COMMIT, GIT_PREVIOUS_COMMIT, GIT_PREVIOUS_SUCCESSFUL_COMMIT, GIT_URL Nov 16, 2023 1:42:33 PM INFO hudson.plugins.audit_trail.AuditTrailRunListener lambda$printNodes$0 Node: StepEndNode[id=9, exec=CpsFlowExecution[Owner[myorg/test/master/6:myorg/test/master #6]]] | Display name: Stage : End | StepArgumentsAsString: null Nov 16, 2023 1:42:33 PM INFO hudson.plugins.audit_trail.AuditTrailRunListener lambda$printNodes$0 Node: StepEndNode[id=8, exec=CpsFlowExecution[Owner[myorg/test/master/6:myorg/test/master #6]]] | Display name: Stage : Body : End | StepArgumentsAsString: null Nov 16, 2023 1:42:33 PM INFO hudson.plugins.audit_trail.AuditTrailRunListener lambda$printNodes$0 Node: StepAtomNode[id=7, exec=CpsFlowExecution[Owner[myorg/test/master/6:myorg/test/master #6]]] | Display name: Check out from version control | StepArgumentsAsString: null Nov 16, 2023 1:42:33 PM INFO hudson.plugins.audit_trail.AuditTrailRunListener lambda$printNodes$0 Node: StepStartNode[id=6, exec=CpsFlowExecution[Owner[myorg/test/master/6:myorg/test/master #6]]] | Display name: Declarative: Checkout SCM | StepArgumentsAsString: null Nov 16, 2023 1:42:33 PM INFO hudson.plugins.audit_trail.AuditTrailRunListener lambda$printNodes$0 Node: StepStartNode[id=5, exec=CpsFlowExecution[Owner[myorg/test/master/6:myorg/test/master #6]]] | Display name: Stage : Start | StepArgumentsAsString: Declarative: Checkout SCM Nov 16, 2023 1:42:33 PM INFO hudson.plugins.audit_trail.AuditTrailRunListener lambda$printNodes$0 Node: StepStartNode[id=4, exec=CpsFlowExecution[Owner[myorg/test/master/6:myorg/test/master #6]]] | Display name: Allocate node : Body : Start | StepArgumentsAsString: null Nov 16, 2023 1:42:33 PM INFO hudson.plugins.audit_trail.AuditTrailRunListener lambda$printNodes$0 Node: StepStartNode[id=3, exec=CpsFlowExecution[Owner[myorg/test/master/6:myorg/test/master #6]]] | Display name: Allocate node : Start | StepArgumentsAsString: null Nov 16, 2023 1:42:33 PM INFO hudson.plugins.audit_trail.AuditTrailRunListener lambda$printNodes$0 Node: FlowStartNode[id=2, exec=CpsFlowExecution[Owner[myorg/test/master/6:myorg/test/master #6]]] | Display name: Start of Pipeline | StepArgumentsAsString: null Nov 16, 2023 1:42:33 PM INFO hudson.plugins.audit_trail.AuditTrailRunListener printNodes ------------Node enumeration done------------
The job start the azure agent vm-forge-agent0e8090 but no trace of this agent is present neither in this log file, neither in the audit.log file
Nov 16, 2023 1:40:20,507 PM - job/myorg/job/test/job/master/ #6 Started by user POTTIERS Cyril, Parameters:[] Nov 16, 2023 1:42:33,972 PM - myorg » test » master #6 Started by user POTTIERS Cyril, Parameters:[] on node no agent started at 2023-11-16T12:40:20Z completed in 133138ms completed: SUCCESS
cpottiers thanks for bearing with me! I did additional tests using various agent names, various durability, various agent types and various syntax but had no luck reproducing. I understood reading the source comments of the pipeline plugin that security reasons could prevent the fields I'm reading from getting populated but I have no clue which exact setting would be triggering this behavior.
One thing I'd like to check is whether the Pipeline steps screen is showing the same issue or not. Could you go to the page of a run of the job, then click on the Pipeline step action on the left side of the screen. Then have a look to see whether the node name is displayed or not. As a reference, here is a screenshot I took on my system
Mhhh, the node name is not displayed here too (unsurprisingly since I use more or less the same method as the pipeline plugin to grab the node name from the workflow run). I'm really not sure what's going on here, my understanding is that some security things could prevent the node name from being kept, but I have no idea what drives this exactly. I'll try to poke around, no promises though...
Sadly no, since the Pipeline steps view also does not have the information, I'm not even sure it even exists in your case.
Hi pierrebtz ,
I installed the Pipeline Agent Build History plugin and this plugin print well the node names used in each build.
I think it could help you.
Michael Keppler
added a comment - Not sure if that's the root cause of this issue, but I recently saw this in the log, also leading to unknown node name:
Run is not an AbstractBuild but a org.jenkinsci.plugins.workflow.job.WorkflowRun, will log the build node as #unknown#.
That's from https://github.com/jenkinsci/audit-trail-plugin/blob/d22186ef2e405b0276ca425972520c0d4afb86f3/src/main/java/hudson/plugins/audit_trail/AuditTrailRunListener.java#L103. I don't know what kind of job or build that was, so I can't look for any definition.
Michael Keppler
added a comment - Not sure if that's the root cause of this issue, but I recently saw this in the log, also leading to unknown node name:
Run is not an AbstractBuild but a org.jenkinsci.plugins.workflow.job.WorkflowRun, will log the build node as #unknown#.
That's from https://github.com/jenkinsci/audit-trail-plugin/blob/d22186ef2e405b0276ca425972520c0d4afb86f3/src/main/java/hudson/plugins/audit_trail/AuditTrailRunListener.java#L103 . I don't know what kind of job or build that was, so I can't look for any definition. 
Jan
added a comment - I guess mawinter69 solved this in pipeline-agent-build-history. Have not worked with FlowExecutions / DepthFirstScanner before, but I think it should be straight forward:
https://github.com/jenkinsci/pipeline-agent-build-history-plugin/blob/main/src/main/java/io/jenkins/plugins/agent_build_history/AgentBuildHistory.java#L208-L221
Jan
added a comment - I guess mawinter69 solved this in pipeline-agent-build-history. Have not worked with FlowExecutions / DepthFirstScanner before, but I think it should be straight forward:
https://github.com/jenkinsci/pipeline-agent-build-history-plugin/blob/main/src/main/java/io/jenkins/plugins/agent_build_history/AgentBuildHistory.java#L208-L221
cpottiers I suppose the Azure VM Agent plugin doesn't use an `AbstractBuild` (https://github.com/jenkinsci/audit-trail-plugin/blob/b45ca03c816c870b085ff735312425ff1fff2c72/src/main/java/hudson/plugins/audit_trail/AuditTrailRunListener.java#L92-L100). It should be a matter of looking into the Azure-vm-agents plugin to understand what object they use, and enrich the method I linked (maybe an optional dependency over the Azure VM agent plugin will be needed).
I don't have a setup to reproduce though and I'm not a big fan of blind fixing, are you willing to provide a fix?