Python worker failed to connect back

I got the same error. I solved it installing the previous version of Spark (2.3 instead of 2.4). Now it works perfectly, maybe it is an issue of the lastest version of pyspark.

The heart of the problem is the connection between pyspark and python, solved by redefining the environment variable.

I´ve just changed the environment variable's values PYSPARK_DRIVER_PYTHON from ipython to jupyter and PYSPARK_PYTHON from python3 to python.