can we break this into smaller stories?

1. per stage agent
2. per stage notifications
3. stage DAG

I'm a little concerned that matrix, parallel, and DAG are all going to get confused into a big mush of competing ways to do the same thing.

We most definitely will be. This is specifically for figuring out how to present the per-stage configuration in the syntax and model, not the actual configs themselves.

So what I'm thinking right now, due to the complexity of what could be configured and the fact that the configuration has to be separate from the actual steps (since we just execute that closure at runtime, which wouldn't work at all if config directives were mixed up in the steps closure), is something like this:

stage('foo') {
    config {
        agent docker:'foo'
        notifications {
              always {
                  echo "something or other"
              }
        }
    }
    steps {
          sh "./do-something.sh"
    }
}

We'd still support the no-per-stage-config syntax (i.e., steps as the only things in the stage block) - that may be a pain to do in practice, so it's possible we'll end up adding the above as a new config section called, like, stageWithConfig or something along those lines.

Code changed in jenkins
User: Andrew Bayer
Path:
src/main/groovy/org/jenkinsci/plugins/pipeline/modeldefinition/ast/ModelASTStage.groovy
src/main/groovy/org/jenkinsci/plugins/pipeline/modeldefinition/ast/ModelASTStageConfig.groovy
src/main/groovy/org/jenkinsci/plugins/pipeline/modeldefinition/model/ClosureContentsChecker.groovy
src/main/groovy/org/jenkinsci/plugins/pipeline/modeldefinition/model/Stage.groovy
src/main/groovy/org/jenkinsci/plugins/pipeline/modeldefinition/model/StageConfig.groovy
src/main/groovy/org/jenkinsci/plugins/pipeline/modeldefinition/parser/JSONParser.groovy
src/main/groovy/org/jenkinsci/plugins/pipeline/modeldefinition/parser/ModelParser.groovy
src/main/groovy/org/jenkinsci/plugins/pipeline/modeldefinition/validator/ModelValidator.groovy
src/main/java/org/jenkinsci/plugins/pipeline/modeldefinition/model/MethodMissingWrapperWhitelist.java
src/main/resources/ast-schema.json
src/main/resources/org/jenkinsci/plugins/pipeline/modeldefinition/ClosureModelTranslator.groovy
src/main/resources/org/jenkinsci/plugins/pipeline/modeldefinition/ModelInterpreter.groovy
src/test/java/org/jenkinsci/plugins/pipeline/modeldefinition/AbstractModelDefTest.java
src/test/java/org/jenkinsci/plugins/pipeline/modeldefinition/AgentTest.java
src/test/resources/json/perStageConfigAgent.json
src/test/resources/perStageConfigAgent.groovy
http://jenkins-ci.org/commit/pipeline-model-definition-plugin/385940d33c4184cb5f5947c00c45d7ed57566c47
Log:
JENKINS-38331 Very preliminary version of per-stage agent config

Honestly, I'm doing this for JENKINS-38284 more, but I needed some
section to actually have be per-stage to test it, so...tada?

This is literally the result of three hours of slamming my head
against the wall over stupid mistakes, so I'm fairly sure there are
still a plethora of such mistakes in here. But I'm pretty sure it
works - I just added the JSON version and conversion for
perStageAgentConfig to this and the tests are actively running as I
type this, but I got impatient and made a commit now. So ha.