Write and run pyspark in IntelliJ IDEA

Set the env path for (SPARK_HOME and PYTHONPATH) in your program run/debug configuration.

For instance:

SPARK_HOME=/Users/<username>/javalibs/spark-1.5.0-bin-hadoop2.4/python/
PYTHON_PATH=/Users/<username>/javalibs/spark-1.5.0-bin-hadoop2.4/python/pyspark

See attached snapshot in IntelliJ Idea