Set up PySpark with Keras

Posted on 2017-09-10 In Machine Learning System , Deep Learning , Big Data , TensorFlow , Keras

Install libraries

1
2
3

pip install -U matplotlib numpy pandas scipy jupyter ipython scikit-learn scikit-image openslide-python`
pip install tensorflow
pip install pyspark py4j

Enable Jupyter to run PySpark by adding the following to ~/.bash_profile

1 2	export PYSPARK_DRIVER_PYTHON=jupyter export PYSPARK_DRIVER_PYTHON_OPTS='notebook'