How/When does Execute Shell mark a build as failure in Jenkins?

First things first, hover the mouse over the grey area below. Not part of the answer, but absolutely has to be said:

If you have a shell script that does "checkout, build, deploy" all by itself, then why are you using Jenkins? You are foregoing all the features of Jenkins that make it what it is. You might as well have a cron or an SVN post-commit hook call the script directly. Jenkins performing the SVN checkout itself is crucial. It allows the builds to be triggered only when there are changes (or on timer, or manual, if you prefer). It keeps track of changes between builds. It shows those changes, so you can see which build was for which set of changes. It emails committers when their changes caused successful or failed build (again, as configured as you prefer). It will email committers when their fixes fixed the failing build. And more and more. Jenkins archiving the artifacts also makes them available, per build, straight off Jenkins. While not as crucial as the SVN checkout, this is once again an integral part of what makes it Jenkins. Same with deploying. Unless you have a single environment, deployment usually happens to multiple environments. Jenkins can keep track of which environment a specific build (with specific set of SVN changes) is deployed it, through the use of Promotions. You are foregoing all of this. It sounds like you are told "you have to use Jenkins" but you don't really want to, and you are doing it just to get your bosses off your back, just to put a checkmark "yes, I've used Jenkins"

The short answer is: the exit code of last command of the Jenkin's Execute Shell build step is what determines the success/failure of the Build Step. 0 - success, anything else - failure. Note, this is determining the success/failure of the build step, not the whole job run. The success/failure of the whole job run can further be affected by multiple build steps, and post-build actions and plugins.

You've mentioned Build step 'Execute shell' marked build as failure, so we will focus just on a single build step. If your Execute shell build step only has a single line that calls your shell script, then the exit code of your shell script will determine the success/failure of the build step. If you have more lines, after your shell script execution, then carefully review them, as they are the ones that could be causing failure.

Finally, have a read here Jenkins Build Script exits after Google Test execution. It is not directly related to your question, but note that part about Jenkins launching the Execute Shell build step, as a shell script with /bin/sh -xe

The -e means that the shell script will exit with failure, even if just 1 command fails, even if you do error checking for that command (because the script exits before it gets to your error checking). This is contrary to normal execution of shell scripts, which usually print the error message for the failed command (or redirect it to null and handle it by other means), and continue.

To circumvent this, add set +e to the top of your shell script.

Since you say your script does all it is supposed to do, chances are the failing command is somewhere at the end of the script. Maybe a final echo? Or copy of artifacts somewhere? Without seeing the full console output, we are just guessing.

Please post the job run's console output, and preferably the shell script itself too, and then we could tell you exactly which line is failing.

Simple and short answer to your question is

Please add following line into your "Execute shell" Build step.

#!/bin/sh

Now let me explain you the reason why we require this line for "Execute Shell" build job.

By default Jenkins take /bin/sh -xe and this means -x will print each and every command.And the other option -e, which causes shell to stop running a script immediately when any command exits with non-zero (when any command fails) exit code.

So by adding the #!/bin/sh will allow you to execute with no option.