pandas profiling with spark dataframe code example
Example: how to do pandas profiling
//get the latest version of pandas_profiling
import numpy as np
import pandas as pd
import pandas_profiling
df1=pd.read_csv(<File path>)
profile = df1.profile_report(title="<give any name you want>")
profile.to_file(output_file="<givefilename>.html")