Spark Scala : Unable to import sqlContext.implicits._

You use an old version of Spark-SQL. Change it to:

<dependency>
    <groupId>org.apache.spark</groupId>
    <artifactId>spark-sql_2.10</artifactId>
    <version>1.5.2</version>
</dependency>

first create

val sqlContext = new org.apache.spark.sql.SQLContext(sc)

now we have sqlContext w.r.t sc (this will be available automatically when we launch spark-shell) now,

import sqlContext.implicits._ 

With the release of Spark 2.0.0 (July 26, 2016) one should now use the following:

import spark.implicits._  // spark = SparkSession.builder().getOrCreate()

https://databricks.com/blog/2016/08/15/how-to-use-sparksession-in-apache-spark-2-0.html


For someone using sbt to build, update the library versions to

libraryDependencies ++= Seq(
  "org.apache.spark" % "spark-core_2.12" % "2.4.6" % "provided",
  "org.apache.spark" % "spark-sql_2.12" % "2.4.6" % "provided"
)

And then import SqlImplicits as below.

val spark = SparkSession.builder()
      .appName("appName")
      .getOrCreate()

    import spark.sqlContext.implicits._;