Search
 
SCRIPT & CODE EXAMPLE
 
CODE EXAMPLE FOR PYTHON

exception: python in worker has different version 3.7 than that in driver 3.8, pyspark cannot run with different minor versions. please check environment variables pyspark_python and pyspark_driver_python are correctly set.

By the way, if you use PyCharm, you could add PYSPARK_PYTHON and PYSPARK_DRIVER_PYTHON to run/debug configurations per image below
 
PREVIOUS NEXT
Tagged: #python #worker #version #driver #pyspark #run #minor #check #environment #variables #correctly
ADD COMMENT
Topic
Name
5+3 =