Convert value depending on a type in SparkSQL via case matching of type
Turns out that DecimalType only pattern matches with zero arguments:
case DecimalType() => ...
If you need the precision and scale, you must define the type of the case and manually extract them:
datatype match {
case dt: DecimalType =>
val precision = dt.precision
val scale = dt.scale
...
The problem is the use of the return
in your code. You said you use this code snippet in a function somewhere. What is the return type of that function? Obviously, you intend that sometimes it is Integer, sometimes String, sometimes BigDecimal; but if you use return
, it will look to the type of the returned object to determine the return type of the function. In general, you should strongly avoid using return
in Scala code. The last evaluated value in the function body is returned. The only case for using a return
is when you want to force returning a value somewhere else in the function body. But still, a better way would be to save the return object in a variable and just evaluate that variable in the last line of your function body. And never use return!
Without return
it works
scala> val datatype = DecimalType(10, 2)
datatype: org.apache.spark.sql.types.DecimalType = DecimalType(10,2)
scala> val value = BigDecimal(10)
value: scala.math.BigDecimal = 10
scala> datatype match {case DecimalType(_,_) => value}
res150: scala.math.BigDecimal = 10
** Problems with return **
scala> def test = {datatype match {case DecimalType(_,_) => return value}}
<console>:138: error: method test has return statement; needs result type
def test = {datatype match {case DecimalType(_,_) => return value}}
scala> def test:BigDecimal = {datatype match {case DecimalType(_,_) => return value}}
test: BigDecimal
scala> def test:DataType = {datatype match {case DecimalType(_,_) => return value}}
<console>:138: error: type mismatch;
found : scala.math.BigDecimal
required: org.apache.spark.sql.types.DataType
def test:DataType = {datatype match {case DecimalType(_,_) => return value}}
scala> def test3 = {datatype match {case DecimalType(_,_) => value}}
test3: scala.math.BigDecimal