How to limit decimal values to 2 digits before applying agg function?
You can use bround:
val df = Seq(2.346, 1.549).toDF("A")
df.select(bround(df("A"), 2)).show
+------------+
|bround(A, 2)|
+------------+
| 2.35|
| 1.55|
+------------+
df.agg(sum(bround(df("A"), 2)).as("appSum")).show
+------------------+
| appSum|
+------------------+
|3.9000000000000004|
+------------------+
^
df.agg(sum(df("A")).as("exactSum")).show
+--------+
|exactSum|
+--------+
| 3.895|
+--------+
The above solution does work for spark 2.0 version however for folks like me who are still using 1.5.*+ versions below is something that will work.(I used round function as suggested by @Psidom):
val df = Seq(2.346, 1.549).toDF("A")
df.select(bround(df("A"), 2)).show
+------------+
|bround(A, 2)|
+------------+
| 2.35|
| 1.55|
+------------+
val total=df.agg(sum(round(df.col(colName),2)).cast("double")).first.getDouble(0)
total: Double = 3.90