Jenkins Declarative Pipeline, run groovy script on slave agent
Turns out Groovy File commands are considered insecure, and although they will run on the master, they will not run on the slave. If you call them from a script that has the agent set to another node, it will still execute the command just fine, just on the master node, not the agent. Here's an excerpt of an article post https://support.cloudbees.com/hc/en-us/articles/230922508-Pipeline-Files-manipulation
The operation with File class are run on master, so only works if build is run on master, in this example I create a file and check if I can access it on a node with method exists, it does not exist because the new File(file) is executed on master, to check this I search for folder Users that exist on my master but not in the node.
stage 'file move wrong way'
//it only works on master
node('slave') {
def ws = pwd()
def context = ws + "/testArtifact"
def file = ws + '/file'
sh 'touch ' + file
sh 'ls ' + ws
echo 'File on node : ' + new File(file).exists()
echo 'Users : ' + new File('/Users').exists()
sh 'mv ' + file + ' ' + context
sh 'ls ' + ws
}
To execute file manipulation command we recommend to use native commands.
This is a simple example of operations in shell
stage 'Create file'
sh 'touch test.txt'
stage 'download file'
def out='$(pwd)/download/maven.tgz'
sh 'mkdir -p ./download'
sh 'curl -L http://ftp.cixug.es/apache/maven/maven-3/3.3.9/binaries/apache-maven-3.3.9-bin.tar.gz -o ' + out
stage 'move/rename'
def newName = 'mvn.tgz'
sh 'mkdir -p $(pwd)/other'
sh 'mv ' + out + ' ' + newName
sh 'cp ' + newName + ' ' + out
}
I have implemented the code which automatically installs Groovy on slave (for scripted pipeline). Perhaps this solution is a little bit cumbersome, but pipelines don't offer any other way to achieve the same functionality as "Execute Groovy Script" stuff from the old Jenkins, because the plugin https://wiki.jenkins.io/display/JENKINS/Groovy+plugin is not supported yet for pipeline.
import hudson.tools.InstallSourceProperty;
import hudson.tools.ToolProperty;
import hudson.tools.ToolPropertyDescriptor;
import hudson.tools.ToolDescriptor;
import hudson.tools.ToolInstallation;
import hudson.tools.ToolInstaller;
import hudson.util.DescribableList;
import hudson.plugins.groovy.GroovyInstaller;
import hudson.plugins.groovy.GroovyInstallation;
/*
Installs Groovy on the node.
The idea was taken from: https://devops.lv/2016/12/05/jenkins-groovy-auto-installer/
and https://github.com/jenkinsci/jenkins-scripts/blob/master/scriptler/configMavenAutoInstaller.groovy
COMMENT 1: If we use this code directly (not as a separate method) then we get
java.io.NotSerializableException: hudson.plugins.groovy.GroovyInstaller
COMMENT 2: For some reason inst.getExecutable(channel) returns null. I use inst.forNode(node, null).getExecutable(channel) instead.
TODO: Check if https://jenkinsci.github.io/job-dsl-plugin/#method/javaposse.jobdsl.dsl.helpers.step.MultiJobStepContext.groovyCommand
works better.
*/
@NonCPS
def installGroovyOnSlave(String version) {
if ((version == null) || (version == "")) {
version = "2.4.7" // some default should be
}
/* Set up properties for our new Groovy installation */
def node = Jenkins.getInstance().slaves.find({it.name == env.NODE_NAME})
def proplist = new DescribableList<ToolProperty<?>, ToolPropertyDescriptor>()
def installers = new ArrayList<GroovyInstaller>()
def autoInstaller = new GroovyInstaller(version)
installers.add(autoInstaller)
def InstallSourceProperty isp = new InstallSourceProperty(installers)
proplist.add(isp)
def inst = new GroovyInstallation("Groovy", "", proplist)
/* Download and install */
autoInstaller.performInstallation(inst, node, null)
/* Define and add our Groovy installation to Jenkins */
def descriptor = Jenkins.getInstance().getDescriptor("hudson.plugins.groovy.Groovy")
descriptor.setInstallations(inst)
descriptor.save()
/* Output the current Groovy installation's path, to verify that it is ready for use */
def groovyInstPath = getGroovyExecutable(version)
println("Groovy " + version + " is installed in the node " + node.getDisplayName())
}
/* Returns the groovy executable path on the current node
If version is specified tries to find the specified version of groovy,
otherwise returns the first groovy installation that was found.
*/
@NonCPS
def getGroovyExecutable(String version=null) {
def node = Jenkins.getInstance().slaves.find({it.name == env.NODE_NAME})
def channel = node.getComputer().getChannel()
for (ToolInstallation tInstallation : Jenkins.getInstance().getDescriptor("hudson.plugins.groovy.Groovy").getInstallations()) {
if (tInstallation instanceof GroovyInstallation) {
if ((version == null) || (version == "")) {
// any version is appropriate for us
return tInstallation.forNode(node, null).getExecutable(channel)
}
// otherwise check for version
for (ToolProperty prop in tInstallation.getProperties()) {
if (prop instanceof InstallSourceProperty) {
for (ToolInstaller tInstaller: prop.installers) {
if (
(tInstaller instanceof GroovyInstaller) &&
(tInstaller.id.equals(version))
)
return tInstallation.forNode(node, null).getExecutable(channel)
}
}
}
}
}
return null
}
/* Wrapper function. Returns the groovy executable path as getGroovyExecutable()
but additionally tries to install if the groovy installation was not found.
*/
def getGroovy(String version=null) {
def installedGroovy = getGroovyExecutable(version)
if (installedGroovy != null) {
return installedGroovy
} else {
installGroovyOnSlave(version)
}
return getGroovyExecutable(version)
}
Just put these 3 methods to your pipeline script and you will be able to get the Groovy executable path with the help of the method getGroovy(). If it is not installed yet then the installation will be done automatically. You can test this code with the simple pipeline, like this:
// Main
parallel(
'Unix' : {
node ('build-unix') {
sh(getGroovy() + ' --version')
}
},
'Windows' : {
node ('build-win') {
bat(getGroovy() + ' --version')
}
}
)
For me the output was:
[build-unix] Groovy Version: 2.4.7 JVM: 1.8.0_222 Vendor: Private Build OS: Linux
[build-win] Groovy Version: 2.4.7 JVM: 11.0.1 Vendor: Oracle Corporation OS: Windows 10
I run into this same issue recently. I had a python file that runs and writes the results to a JSON file. I was trying to access the JSON file to retrieve the data from there. Here is the code I was using inside a stage block of a declarative pipeline:
script {
def jsonSlurper = new JsonSlurper()
def fileParsed = new File("parameters.json")
def dataJSON = jsonSlurper.parse(fileParsed)
}
As everyone stated already, the above was failing with FileNotFoundException because anything inside script{} will only run on master and not the agent.
To work around the issue, I have used the Pipeline Utility Steps plugin (reference: https://plugins.jenkins.io/pipeline-utility-steps/ -- How to use: https://www.jenkins.io/doc/pipeline/steps/pipeline-utility-steps/#writejson-write-json-to-a-file-in-the-workspace)
The plugin will allow you to do any read/write operation on multiple file formats.
Here is an example of the code I used after installing the plugin:
script {
def props = readJSON file: 'parameters.json'
println("just read it..")
println(props)
}
Note: I was using jenkins 2.249.1