“value $ is not a member of StringContext” - Missing Scala plugin?
You can use the col
function instead just import it like this :
import org.apache.spark.sql.functions.col
And then change the $"column"
to col("column")
Hope it helps
- @Apurva's answer initially worked for me in that the error vanished from
IntelliJ
- But then it resulted in
"Could not find implicit value for spark"
duringsbt compile
phase
I found a strange work-around by importing spark.implicits._
from SparkSession
referenced from DataFrame
instead of one obtained by getOrCreate
import df.sparkSession.implicits._
where df
is a DataFrame
This could be because my code was placed inside a case class
that received an implicit val spark: SparkSession
parameter; but I'm not really sure as to why this fix worked for me
Add this.. it will work
val spark = SparkSession.builder().getOrCreate()
import spark.implicits._ // << add this