Spark Dataframe change column value
import org.apache.spark.sql.functions._
df.withColumn("name", lit(translate( translate(col("cpf"), ".", ""),"-","")))
You can do it like this
df.withColumn("name", when($"name" === "'", ""))
You can't mutate DataFrames, you can only transform them into new DataFrames with updated values. In this case - you can use the regex_replace
function to perform the mapping on name
column:
import org.apache.spark.sql.functions._
val updatedDf = Df.withColumn("name", regexp_replace(col("name"), ",", ""))