How to create SQLContext in spark using scala?
For newer versions of Spark (2.0+), use SparkSession
:
val spark = SparkSession.builder.getOrCreate()
SparkSession
can do everything SQLContext
can do but if needed the SQLContext
can be accessed as follows,
val sqlContext = spark.sqlContext
You need to new
your SparkContext
and that should solve it
Simply we can create SQLContext in scala
scala> val sqlContext = new org.apache.spark.sql.SQLContext(sc);