Deploy incremental changes using Github and Jenkins
There are a bunch of third party solutions offering native CI for Salesforce. Full disclosure that I lead one of the companies building one but I'll provide you with all options so you can investigate.
Gearset - 30 day trial and no org install so easy to start with.
Copado
AutoRabit
Flosum
SalesforceDX will also bring standardised tooling to make Salesforce CI easier to plug into ANT. The tooling will still be built ontop of existing standards so any investment in a third party today shouldn't hamper you migrating later. Certainly I can speak for Gearset and all our stuff will be completely interopable
Its very late to answer but here is my setup which works:
Code Merged -> Create a deploy folder -> checkout the target branch in workspace - > Merge the current branch to target branch (no commit)-> Run git diff -> Create changes (This will basically create txt files for the next steps) - > Start creating package.xml using the txt files generated in the previous steps –> add these to deploy folder –> Deploy via ant migration tool.
Please find a high level explanation for each step . I have modified all the bash scripts to work for Windows for now.
• Code Merged -> Developer does the code merge Run build.xml via Jenkins Ant trigger and specify the target as “sf_build”
• Create a deploy folder -> As “sf_build” depends on “sf_prepare_deployment”, the make directory gets fired.
• checkout the target branch in workspace - > A batch script to checkout the target branch. And merge the hotfix or feature branch to it. And fire diff command and output it to diff.txt.
• Create changes (This will basically create txt files for the next steps) - > This is again a bash script which splits the changes based on destructive or metadata file types.
• Start creating package.xml using the txt files generated in the previous steps –> A bash file to generate package.xml by reading the txt files created above and based on the files extension create the package.xml file. for this use: generate dynamic package.xml, my file name is "generate_package.sh" which is also used in build.xml. Start reading final.txt file like this :
if [ -e $1 ]
then
SCRIPTFILE=$3'/'$2'.xml'
echo Scriptfile is: $SCRIPTFILE
echo ===PKGXML===
echo Creating new package.xml
echo '<?xml version="1.0" encoding="UTF-8"?><Package></Package>' > $SCRIPTFILE
ARRAY=()
while read CFILE
do
// run logic from site above
done < $1
if [[ "$2" != *"destructive"* ]]
then
xmlstarlet/xml.exe ed -L -s Package -t elem -n version -v "40.0" $SCRIPTFILE
fi
xmlstarlet/xml.exe ed -L -i Package -t attr -n xmlns -v "http://soap.sforce.com/2006/04/metadata" $SCRIPTFILE
echo ====FINAL PACKAGE.XML=====
cat $SCRIPTFILE
• Deploy via ant migration tool. -> The target named “sf_build” gets finally triggered and “sf:deploy” runs and deploys(checkOnly= false) or validates(checkOnly=true) the package created .
Snippet of build.xml
<target name="sf_prepare_deployment">
<echo message="START: Create diff dir" />
<echo message="Checking against targer: ${target}" />
<mkdir dir="../${diffDir}" />
<mkdir dir="../${diffDir}/src" />
<exec executable="C:\Program Files\Git\bin\bash.exe" osfamily="windows" failonerror="true">
<arg value="${scriptName}.sh" />
<arg value="${target}" />
<arg value="${targetName}" />
</exec>
<exec executable="/bin/bash" osfamily="unix" failonerror="true">
<arg value="${scriptName}.sh" />
<arg value="${target}" />
</exec>
<exec executable="C:\Program Files\Git\bin\bash.exe" osfamily="windows">
<arg value="create_changes.sh" />
<arg value="${diffDir}" />
</exec>
<exec executable="/bin/bash" osfamily="unix">
<arg value="create_changes.sh" />
<arg value="${diffDir}" />
</exec>
<echo message="Generating package.xml" />
<exec executable="C:\Program Files\Git\bin\bash.exe" osfamily="windows">
<arg value="generate_package.sh" />
<arg value="../final.txt" />
<arg value="package" />
<arg value="C:/Jenkins/workspace/deploy/src" />
</exec>
<exec executable="/bin/bash" osfamily="unix">
<arg value="generate_package.sh" />
<arg value="../final.txt" />
<arg value="package" />
<arg value="../${diffDir}/src" />
</exec>
<echo message="Package generated." />
<echo message="Adding properties if applicable" />
<exec executable="C:\Program Files\Git\bin\bash.exe" osfamily="windows">
<arg value="properties_helper.sh" />
<arg value="${propsEnv}" />
<arg value="../${diffDir}/src" />
</exec>
<exec executable="/bin/bash" osfamily="unix">
<arg value="properties_helper.sh" />
<arg value="${propsEnv}" />
<arg value="../${diffDir}/src" />
</exec>
<echo message="Properties added. Finished." />
</target>
<target name="sf_build" depends="sf_prepare_deployment">
<echo message="START: SFDC Deployment" />
<echo message="../${diffDir}/src" />
<sf:deploy username="${sf.deploy.username}" password="${sf.deploy.password}"
serverurl="${sf.deploy.serverurl}" sessionId="${sf.deploy.sessionId}"
deployRoot="../${diffDir}/src" maxPoll="${sf.maxPoll}"
pollWaitMillis="${sf.pollWaitMillis}" checkOnly="${sf.checkOnly}"
runAllTests="${sf.runAllTests}" logType="${sf.logType}"
/>
</target>
Jenkins setup as follows:
Jenkins here has two stages/Jobs:
- Validate – Listens to the PR on a particular branch. This only validates the package (checkOnly = true in Ant Migration tool) and run automated code reviews. This runs when Developer creates a PR.
- Merge – Polls the SCM for changes. Only gets activated when PR is validated successfully in the above step. When Scrum Masters clicks merge, this merges the code in the target branch and deploys the package (checkOnly = false in Ant Migration tool).
Process that we finally implemented in Jenkins
- Source Code Management
Connect to your git repository
Execute shell Script will get the previous commit (last one that is deployed) from a file and use it to generate a diff. Finally the script will initialize a nodejs package that I have added to my git repo. (which is convenient as I can now manage it together with my deployments)
prevcommit=$(<lastcommit.txt)
echo Previous commit is $prevcommit
git diff --name-status $prevcommit..HEAD --diff-filter=ACMR > diff.txt
cd packageXMLGenerator
npm install
cd ..Execute nodejs This part will call on the script I wrote. The tools.js script will go over all files in the diff.txt file and:
- copy the file to the target directory (including *.meta-xml file if needed)
- decide if it should create an entry in package.xml or not. An entry is always * for the metadata type of the file. In the end the target directory will contain ALL files from the diff + a package.xml file containing only mentions of metadata in the package. The timeout part of the script will make sure that the next build step only executes if this one is finished.
the script:
//include script file to generate package xml
console.log('workspace' + process.env['WORKSPACE']);
var tools = require(process.env['WORKSPACE'] + '/packageXMLGenerator/tools.js');
//set some params to initialize
var owner = 'USERNAME', project = 'PRODJECTNAME', branch = 'BRANCHNAME', targetdirectory = 'TARGETDIR/';
//execute script
var flag = false;
tools.startDynamicPackageXMLGeneration(owner,project,branch,targetdirectory).then(function(){
flag = true;
});
function checkFlag() {
if(flag == false) {
setTimeout(checkFlag, 100); /* this checks the flag every 100 milliseconds*/
} else {
console.log('done');
}
}
checkFlag();
- Invoke ant - will deploy the target directory to the target org