groupby and calculate mean of difference of columns + pyspark code example
Example: pyspark group by and average in dataframes
df.groupBy("Profession").agg({'Age':'avg', 'Gender':'count'}).show()
df.groupBy("Profession").agg({'Age':'avg', 'Gender':'count'}).show()