How to get the last row from DataFrame?

Try this, it works for me.

df.orderBy($"value".desc).show(1)

I would use simply the query that - orders your table by descending order - takes 1st value from this order

df.createOrReplaceTempView("table_df")
query_latest_rec = """SELECT * FROM table_df ORDER BY value DESC limit 1"""
latest_rec = self.sqlContext.sql(query_latest_rec)
latest_rec.show()

I'd simply reduce:

df.reduce { (x, y) => 
  if (x.getAs[Int]("timestamp") > y.getAs[Int]("timestamp")) x else y 
}