Why does spark-shell fail with "was unexpected at this time"?

I'm almost sure that your JAVA_HOME environment variable contains a space that breaks spark-shell. Please re-install Java to a directory with no spaces in the path.

---

You can see the relevant piece of code in bin/spark-class2.cmd that spark-shell executes on Windows under the covers (through bin/spark-submit2.cmd shell script):

if "x%1"=="x" (

So when spark-class2.cmd substitutes %1 to a path with a space (or something similar) it ends up as:

if "x"Files\Java\jdk1.8.0_45""=="x" (

that gives the error due to too many double quotes.

The mystery is how does the JAVA_HOME end up in this place. I can't seem to find the reason, but that's what we see here.

You can see this issue when your Java home set like following in windows

JAVA_HOME=C:\Program Files (x86)\Java\jdk1.8.0_162\bin

Issue here is space in "Program Files (x86)" If you add double quotes will not work on window 10

"C:\Program Files (x86)\Java\jdk1.8.0_162\bin" 

You need to copy Java into outside Program Files (x86) then it should work

JAVA_HOME=C:\java\jdk1.8.0_171\bin

I resolved this error by using a short hand like "Progra~1" which is a short hand for "Program Files (x86)".

C:\Progra~1\Java\jdk1.8.0_161