-
Story
-
Resolution: Fixed
-
Minor
-
None
-
Pipeline Config 0.1
-
Powered by SuggestiMate
It'd be handy to be able to have notifications specified for individual stages as well as for the whole build.
- causes
-
JENKINS-57801 Execution of post stage block is based on pipeline status, not stage result which docs says it should
-
- Open
-
[JENKINS-37792] Pipeline Config: Notifications per stage
When I think about this, I feel that the requirement can probably be satisfied by changing how we structure things.
If the stage (or notifications) block contains node configuration and then an ordered sequence of conditions (where steps is just sort of an alias for success)
Thus we can have
pipeline { agent label:'nix' stages { stage('unix') { steps { ... } success { ... } steps { ... } always { ... } } stage('windows') { agent label:'win' steps { ... } success { ... } failure { ... } } } notifications { // by default these get no node, but we can specify a config here to get a node for them agent label:'irc-server' always { ... } } }
Beyond the messiness of all of those success/failure/etc blocks in the stage block (I admit it, my Lisp background makes me perceive arbitrarily deep nested maps as the natural state of the world), we'd still need a way to signify "run this block of steps after the main block of steps for this stage has completed, regardless of the status of the build at that point" - i.e., for "always run this cleanup script" kind of stuff. Since the "steps" contents aren't guaranteed to all be executed (i.e., if the first one is error 'foo', we'll never actually get to subsequent steps, nor should we!) we can't rely on steps to take the place of always.
And again, I don't like the UX of putting steps, success, failure et al at the same level. It's cluttered and confusing.
hrmpw, michaelneale, thoughts?
abayer each stage is then basically a processing of the list of conditions... (steps being a sort of special condition that is "run this if all the previous steps in this stage completed successfully" which is subtly different from success. all the conditions in a stage would always be evaluated and their contents executed until one of the content steps fails or all content steps are executed.
Having notifications and postBuild blocks outside of stages but only a postBuild block within the stage is too inconsistent for me. I think the terms should be the same.
I'm not sure why success and steps would be aliased, they seem to have different functions, but I somewhat like stephenconnolly suggestion. I'm not sure about multiple steps blocks. We could even use this format instead of an explicit postBuild block if we put success, failure, always within the stages block outside of the stages:
stages { stage('unix') { steps { ... } success { ... } always { ... } } stage('windows') { agent label:'win' steps { ... } success { ... } failure { ... } } success{ ... } failure{ ... } }
If we are going to combine postBuild and notifications for stages I think we should do the same overall.
hrmpw my proposal is that we kill off postBuild there is no postBuild any more.
I would be against having conditionals in the raw stages... if you still see the need for that it would, to my mind, be a finally or lastly
stages { stage('unix') { steps { ... } success { ... } always { ... } } stage('windows') { agent label:'win' steps { ... } success { ... } failure { ... } } lastly { success{ ... } failure{ ... } } }
the notifications still stays as they are always executed without a node
FTR I view postBuild as a terrible name...
1. it is the only camelCase name in the "keywords" of p-m-d
2. it is singular compared with notifications and stages which are the comparable containers
fwiw, the name is 'cos we needed it to be all-one-word, not-starting-with-a-capital-letter, so I shrugged and went with postBuild. =)
hrmpw so what is the difference between a steps block and a success block?
The steps block will only execute if the build result is currently successful...
The success block will only execute if the build result is currently successful...
One suggestion that abayer suggested to me is that effectively each "step" in a success or failure block would be kind of "wrapped" in a try...catch so that all the steps would be executed in that block...
To me, that makes it hard to construct inter-sequencing of steps that should happen on success and steps that should happen always followed by steps that should happen on success... we currently have not defined the execution sequence of the components of the conditional blocks in the notifications and I argue that actually we should just follow declaration order and allow conditions to be repeated...
So that leads to perhaps the only place where I would expect a difference in behaviour for a "step" in a steps block vs a success block... Failure mode...
A "step" in a steps block that has failed should fail the build... a "step" in a success block that has failed should "error" the build... the build has not failed but it is in an error state
I agree about the camelCase of postBuild but figured it only bothered me so I let it pass.
I'm thinking about it from a user's point of view stephenconnolly not how it is implemented in the backend. I have stages that contain steps. Alternating steps with success, failure, always is confusing and no different than wrapping steps in try...catch blocks by hand.
We should only allow a single steps block inside a stage. These steps make up the function of the stage and treat them as a whole unit. If one of these steps fails then the stage fails. Parsing that down into smaller units gets more complex and adds little value. If you want to break it down to further pieces then add a new stage.
Maybe success is superfluous in implementation but would it make sense from a user point of view? What are the use cases for success?
- cleaning up the workspace?
- stashing files?
- notifications?
If we kill postBuild what happens to this functionality? IMO, the main use cases for postBuild (despite the name) is:
- As a user I don't want to configure anything at the stage level but want to have notifications and/or postBuild globally
> If we kill postBuild what happens to this functionality?
So the notifications would still exist. And the other uses of postBuild are subsumbed by a lastly block at the end of all the stages (or we use a special syntax on stage to indicate that the stage is always run in order to remove the discontinuity
I still don't like the idea of having a special notifications block at the top level but a separate method for notifications inside a stage. This creates a poor user experience IMO.
well, obviously "lastly" is literally the worst name ever in the history of bad names. Well maybe after "vietnam4j".
postBuild terminology is used elsewhere (outside of jenkins) and is fairly self explanatory (but it isn't as nice as the pluralised "notifications"). I view it as an abbrevation of "post build actions" - plural. There is probably a single german word for that The camel case isn't optimal, yeah.
Whatever the keyword names/reserved names are, the ideal is that as much as possible, things that can be done globally could be done in a stage, which are then scoped to it. unfortunately the name "post build" doesn't quite work. Notifications as a name works...
I do agree that having the same events as notifications/postBuild in stage is nice (often there is a need to always cleanup, regardless of outcome).
Given this ticket is is about notifications - I think it is ok to say that notifications happening in the node that the stage happens with is ok (doesn't make a lot of difference to the user).
If we want to talk about general events, naming, globally or stages, we should setup a time to explore it on a hangout, as there is lots we could cover, and there are good ideas here.
Code changed in jenkins
User: Robert Sandell
Path:
src/main/groovy/org/jenkinsci/plugins/pipeline/modeldefinition/ast/ModelASTBuildConditionsContainer.groovy
src/main/groovy/org/jenkinsci/plugins/pipeline/modeldefinition/ast/ModelASTNotifications.groovy
src/main/groovy/org/jenkinsci/plugins/pipeline/modeldefinition/ast/ModelASTPostBuild.groovy
src/main/groovy/org/jenkinsci/plugins/pipeline/modeldefinition/ast/ModelASTPostStage.groovy
src/main/groovy/org/jenkinsci/plugins/pipeline/modeldefinition/ast/ModelASTStage.groovy
src/main/groovy/org/jenkinsci/plugins/pipeline/modeldefinition/model/PostStage.groovy
src/main/groovy/org/jenkinsci/plugins/pipeline/modeldefinition/model/Root.groovy
src/main/groovy/org/jenkinsci/plugins/pipeline/modeldefinition/model/Stage.groovy
src/main/groovy/org/jenkinsci/plugins/pipeline/modeldefinition/parser/JSONParser.groovy
src/main/groovy/org/jenkinsci/plugins/pipeline/modeldefinition/parser/ModelParser.groovy
src/main/groovy/org/jenkinsci/plugins/pipeline/modeldefinition/validator/ModelValidator.groovy
src/main/resources/ast-schema.json
src/main/resources/org/jenkinsci/plugins/pipeline/modeldefinition/ModelInterpreter.groovy
src/test/java/org/jenkinsci/plugins/pipeline/modeldefinition/PostStageTest.java
src/test/resources/postStage/failingNotifications.groovy
src/test/resources/postStage/globalAndLocalAlways.groovy
src/test/resources/postStage/localAll.groovy
src/test/resources/postStage/localAlways.groovy
http://jenkins-ci.org/commit/pipeline-model-definition-plugin/061eebc2c0af2fc92420ccc746c2f8349727a34e
Log:
JENKINS-37792 Per stage post build steps
Code changed in jenkins
User: Robert Sandell
Path:
src/test/java/org/jenkinsci/plugins/pipeline/modeldefinition/PostStageTest.java
src/test/resources/postStage/failingNotifications.groovy
http://jenkins-ci.org/commit/pipeline-model-definition-plugin/b9c1124dda30d9968fec6f99627e3fcf1a44caae
Log:
JENKINS-37792 Test when a notification fails
Code changed in jenkins
User: Andrew Bayer
Path:
src/main/groovy/org/jenkinsci/plugins/pipeline/modeldefinition/ast/ModelASTBuildConditionsContainer.groovy
src/main/groovy/org/jenkinsci/plugins/pipeline/modeldefinition/ast/ModelASTNotifications.groovy
src/main/groovy/org/jenkinsci/plugins/pipeline/modeldefinition/ast/ModelASTPostBuild.groovy
src/main/groovy/org/jenkinsci/plugins/pipeline/modeldefinition/ast/ModelASTPostStage.groovy
src/main/groovy/org/jenkinsci/plugins/pipeline/modeldefinition/ast/ModelASTStage.groovy
src/main/groovy/org/jenkinsci/plugins/pipeline/modeldefinition/model/PostStage.groovy
src/main/groovy/org/jenkinsci/plugins/pipeline/modeldefinition/model/Root.groovy
src/main/groovy/org/jenkinsci/plugins/pipeline/modeldefinition/model/Stage.groovy
src/main/groovy/org/jenkinsci/plugins/pipeline/modeldefinition/parser/JSONParser.groovy
src/main/groovy/org/jenkinsci/plugins/pipeline/modeldefinition/parser/ModelParser.groovy
src/main/groovy/org/jenkinsci/plugins/pipeline/modeldefinition/validator/ModelValidator.groovy
src/main/resources/ast-schema.json
src/main/resources/org/jenkinsci/plugins/pipeline/modeldefinition/ModelInterpreter.groovy
src/test/java/org/jenkinsci/plugins/pipeline/modeldefinition/AbstractModelDefTest.java
src/test/java/org/jenkinsci/plugins/pipeline/modeldefinition/PostStageTest.java
src/test/resources/postStage/failingNotifications.groovy
src/test/resources/postStage/globalAndLocalAlways.groovy
src/test/resources/postStage/localAll.groovy
src/test/resources/postStage/localAlways.groovy
http://jenkins-ci.org/commit/pipeline-model-definition-plugin/9517f02825eaa52f20dde2e73c70eed2a8260a59
Log:
Merge pull request #30 from jenkinsci/JENKINS-37792
JENKINS-37792 Per stage post build steps
Compare: https://github.com/jenkinsci/pipeline-model-definition-plugin/compare/b7a3ec68779e...9517f02825ea
Code changed in jenkins
User: Robert Sandell
Path:
SYNTAX.md
http://jenkins-ci.org/commit/pipeline-model-definition-plugin/71c3d60ee486e48757bbd76315dc69a9e0418775
Log:
Update SYNTAX.md
Noting changes introduced in JENKINS-37792, JENKINS-37781 and JENKINS-37778
Also fixed the script example and marked the code sections as groovy
so that GitHub hopefully can color the code a bit more nicely.
Might actually be better to call this postBuild rather than notifications - the core distinction between postBuild and notifications is that postBuild runs within the node block the build executed in, which would be the case for any post-stage action (though with per-stage agents, we'd have to explicitly put this execution inside that, but that's trivial enough)...