jupyter_spark_scala_R

Jupyter notebook + Spark + Scala + R

Running container

docker compose up -d

Accessing notebooks

• after running container for first time notebooks should be available in work volumen created in all_spark_notebook
• noetebook will be running on localhost:8888

Using Spark (example)

from pyspark.sql import SparkSession
spark = SparkSession.builder.appName("SparkExample").getOrCreate()
data = [("Alice", 34), ("Bob", 45), ("Cathy", 29)]
columns = ["Name", "Age"]

df = spark.createDataFrame(data, columns)
df.show()