pyspark read json code example

Example 1: pyspark from_json example

from pyspark.sql.functions import from_json, col
json_schema = spark.read.json(df.rdd.map(lambda row: row.json)).schema
df.withColumn('json', from_json(col('json'), json_schema))

Example 2: Read JSON files with automatic schema inference

# Read JSON files with automatic schema inference

df = spark.read.json("logs.json")
df.where("age > 21").select("name.first").show()