Spark runs on Yarn cluster exitCode=13:
It seems that you have set the master in your code to be local
SparkConf.setMaster("local[*]")
You have to let the master unset in the code, and set it later when you issue spark-submit
spark-submit --master yarn-client ...
I had exactly the same problem but the above answer didn't work.
Alternatively, when I ran this with spark-submit --deploy-mode client
everything worked fine.
If it helps someone
Another possibility of this error is when you put incorrectly the --class param
I got this same error running a SparkSQL job in cluster mode. None of the other solutions worked for me but looking in the job history server logs in Hadoop I found this stack trace.
20/02/05 23:01:24 INFO hive.metastore: Connected to metastore.
20/02/05 23:03:03 ERROR yarn.ApplicationMaster: Uncaught exception:
java.util.concurrent.TimeoutException: Futures timed out after [100000 milliseconds]
at scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:223)
at scala.concurrent.impl.Promise$DefaultPromise.result(Promise.scala:227)
at org.apache.spark.util.ThreadUtils$.awaitResult(ThreadUtils.scala:220)
at org.apache.spark.deploy.yarn.ApplicationMaster.runDriver(ApplicationMaster.scala:468)
at org.apache.spark.deploy.yarn.ApplicationMaster.org$apache$spark$deploy$yarn$ApplicationMaster$$runImpl(ApplicationMaster.scala:305)
at org.apache.spark.deploy.yarn.ApplicationMaster$$anonfun$run$1.apply$mcV$sp(ApplicationMaster.scala:245)
at org.apache.spark.deploy.yarn.ApplicationMaster$$anonfun$run$1.apply(ApplicationMaster.scala:245)
at org.apache.spark.deploy.yarn.ApplicationMaster$$anonfun$run$1.apply(ApplicationMaster.scala:245)
...
and looking at the Spark source code you'll find that basically the AM timed out waiting for the spark.driver.port
property to be set by the Thread executing the user class.
So it could either be a transient issue or you should investigate your code for the reason for a timeout.