Details
-
New Feature
-
Status: Closed (View Workflow)
-
Critical
-
Resolution: Won't Fix
-
None
Description
I have several use cases where I may operate on a list or map on different stages of a pipeline. This variable may not be known ahead of time - it could be defined within a stage. I'd like a syntax that allows me to define variables within a pipeline.
Today, I have to define a shared variable before entering pipeline {}.
#!/bin/groovy // Define outside of pipeline to make sure accessible in all script {} blocks def my_list pipeline { agent { label 'label' } stages { stage('stage1') { steps { script { my_list = [1,2,3] } } } stage('stage2') { steps { script { for(int i=0; i<my_list.size();i++) { echo "Doing something with ${my_list[i]}" } } } } } }
A simple approach would be to allow "def" to work anywhere within pipeline {}. This avoids any overly new syntax for pipeline.
Another option would allow the user to provide a special pipeline {} block for defining non-env/params style variables. This could be helpful in allowing the user to use items available in script {} for defining shared state.
#!/bin/groovy
pipeline {
agent { label 'label' }
define {
def my_map = [:] //empty map
def my_list //undefined shared variable
}
stages {
stage('stage1') {
steps {
script {
//If def can be anywhere in pipeline {}, drop script {}
my_list = [1,2,3]
}
}
}
stage('stage2') {
steps {
script {
for(int i=0; i<my_list.size();i++) {
echo "Doing something with ${my_list[i]}"
}
}
}
}
}
}
Attachments
Issue Links
- is blocked by
-
JENKINS-39450 Add a LibraryStep for loading shared libraries within a Pipeline
-
- Resolved
-
- relates to
-
-
- Open
-
-
-
- Closed
-
-
-
- Closed
-
Activity
Sam Van Oort
added a comment - ftherien Caveat: take anything I say here as a personal opinion only. By all means, people should default to Declarative (or Declarative + Shared Libraries as needs grow) because it's easier and shows an opinionated "lit path" to successful CI/CD. And yes, we devote more work to Declarative specifically – I think the split is roughly 30/60/10 for work that is Declarative-only / General Pipeline for both / Scripted-only. That's because Declarative is newer and intended to be the "easy" mechanism that covers 80% of needs, where Scripted lets you roll-your-own and the majority of the features are just General Pipeline Stuff + Groovy.
But as far as actually deprecating Scripted? You see people propose it from time to time, generally without understanding what it actually means, because Declarative is joined at the hip to Scripted and runs on top of it. For Declarative to replace Scripted, it which would have to absorb all the crazy use cases people can implement in Scripted due to its flexibility. Not to mention the number of organizations who would likely drop Pipeline and Jenkins entirely if they made a major investment in Scripted and Shared Libraries and had to walk away from 'em. I'd encourage anybody who thinks it's a good idea to come talk to me personally; I'll be happy to explain the other reasons why it's both extremely technically/architecturally difficult and a very poor idea (not going to run through it here because it's practically a novella).
Ergo why I push back on trying to insert Scripted-like features such as this into Declarative – we don't want to find ourselves maintaining a new programming language. Better to make Declarative the best config language it can be, and delegate fancy stuff to a proper programming language (Groovy).
Hope that all makes sense!
Sam Van Oort
added a comment - ftherien Caveat: take anything I say here as a personal opinion only. By all means, people should default to Declarative (or Declarative + Shared Libraries as needs grow) because it's easier and shows an opinionated "lit path" to successful CI/CD. And yes, we devote more work to Declarative specifically – I think the split is roughly 30/60/10 for work that is Declarative-only / General Pipeline for both / Scripted-only. That's because Declarative is newer and intended to be the "easy" mechanism that covers 80% of needs, where Scripted lets you roll-your-own and the majority of the features are just General Pipeline Stuff + Groovy.
But as far as actually deprecating Scripted? You see people propose it from time to time, generally without understanding what it actually means, because Declarative is joined at the hip to Scripted and runs on top of it. For Declarative to replace Scripted, it which would have to absorb all the crazy use cases people can implement in Scripted due to its flexibility. Not to mention the number of organizations who would likely drop Pipeline and Jenkins entirely if they made a major investment in Scripted and Shared Libraries and had to walk away from 'em. I'd encourage anybody who thinks it's a good idea to come talk to me personally; I'll be happy to explain the other reasons why it's both extremely technically/architecturally difficult and a very poor idea (not going to run through it here because it's practically a novella).
Ergo why I push back on trying to insert Scripted-like features such as this into Declarative – we don't want to find ourselves maintaining a new programming language. Better to make Declarative the best config language it can be, and delegate fancy stuff to a proper programming language (Groovy).
Hope that all makes sense! I want to be clear, I personally am willing to accept that this is not only out of scope, but also counter to core design. My point is I'd like to make sure that core design and direction are documented and understandable.
Florian Miedniak
added a comment - I very agree with bitwiseman and ftherien: Having some background information about the underlying design would it make a lot easier to build Jenkins pipelines the "right" way: Luckily, they are actively developed, thus it is extremely important for users to know about things you should not use because they are getting obsoleted in near future due to changes / advances in design.
Frankly, today it is sometimes quite challening to justify the maintenance effort for pipelines, if the answer to the question "why did the jenkins update break our pipe?" repeatedly is: "they decided to change it" ...
IMO, in the recent past Jenkins came to a very good way of communicating and handling of security related issues (pre-announcement mails, changelog, ...)
Having a design outline and a roadmap with milestones IMO would be a good starting point to achieve a similar level of transparency for the design of a complex, production-grade CI system that Jenkins is.
Florian Miedniak
added a comment - I very agree with bitwiseman and ftherien : Having some background information about the underlying design would it make a lot easier to build Jenkins pipelines the "right" way: Luckily, they are actively developed, thus it is extremely important for users to know about things you should not use because they are getting obsoleted in near future due to changes / advances in design.
Frankly, today it is sometimes quite challening to justify the maintenance effort for pipelines, if the answer to the question "why did the jenkins update break our pipe?" repeatedly is: "they decided to change it" ...
IMO, in the recent past Jenkins came to a very good way of communicating and handling of security related issues (pre-announcement mails, changelog, ...)
Having a design outline and a roadmap with milestones IMO would be a good starting point to achieve a similar level of transparency for the design of a complex, production-grade CI system that Jenkins is. 
svanoort I thought I had read somehwhere that the roadmap was for scripted to eventually be deprecated, and that declarative would replace it.
In any case, I second Liam's point that it would be interesting to see documents explaining this vision. It would help, at the very least, people to make long term decisions for their CI architecture around Jenkins.