How do I check for equality using Spark Dataframe without SQL Query?

There is another simple sql like option. With Spark 1.6 below also should work.

df.filter("state = 'TX'")

This is a new way of specifying sql like filters. For a full list of supported operators, check out this class.

I had the same issue, and the following syntax worked for me:

df.filter(df("state")==="TX").show()

I'm using Spark 1.6.