Mounting Kubernetes service account secrets for single user Jupyter notebook pod

Photo from Teddy Guerrier in unsplash: https://unsplash.com/photos/85YnQaGt324
Pods in the cluster. Single user notebook pod (jupyter-jovyan pod) is deployed only after user log in and spawn a notebook server.
import pyspark
conf = pyspark.SparkConf()
# Try to connect to K8S API server
conf.setMaster("k8s://https://kubernetes.default.svc.cluster.local:443")
# Set some config
conf.set("spark.kubernetes.container.image", "gcr.io/spark-operator/spark-py:v2.4.5")
...
# Create a Spark Session
spark = pyspark.sql.SparkSession.builder.config(conf=conf).getOrCreate()
Caused by: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
...
Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
...
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
conf.set("spark.kubernetes.authenticate.caCertFile", "/var/run/secrets/kubernetes.io/serviceaccount/ca.crt")conf.set("spark.kubernetes.authenticate.oauthTokenFile", "/var/run/secrets/kubernetes.io/serviceaccount/token")conf.set("spark.kubernetes.authenticate.driver.serviceAccountName", "spark") # "spark" is a service account to deploy Spark on the cluster
Caused by: java.io.FileNotFoundException: /var/run/secrets/kubernetes.io/serviceaccount/token (No such file or directory)
Description of a JupyterHub pod in K8S cluster having information about mounts
Description of jupyter notebook pod having no information about mounts
config c.KubeSpawner.service_account = Unicode(None)

Reference

  1. KubeSpawner documentation: https://jupyterhub-kubespawner.readthedocs.io/en/latest/spawner.html
  2. Jupyterhub config file from zero-to-jupyterhub-k8s repository by Jupyter: https://github.com/jupyterhub/zero-to-jupyterhub-k8s/blob/master/jupyterhub/files/hub/jupyterhub_config.py
  3. Accessing K8S cluster: https://kubernetes.io/docs/tasks/access-application-cluster/access-cluster/
  4. Running Spark with Jupyter Notebook & HDFS on Kubernetes: https://kublr.com/blog/running-spark-with-jupyter-notebook-hdfs-on-kubernetes/

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store