How to create AWS Glue table where partitions have different columns? ('HIVE_PARTITION_SCHEMA_MISMATCH')

I had the same issue, solved it by configuring crawler to update table metadata for preexisting partitions:

This helped me. Posting the image for others in case the link is lost

It also fixed my issue! If somebody need to provision This Configuration Crawler with Terraform so here is how I did it:

resource "aws_glue_crawler" "crawler-s3-rawdata" {
  database_name = "my_glue_database"
  name          = "my_crawler"
  role          = "my_iam_role.arn"

  configuration = <<EOF
{
   "Version": 1.0,
   "CrawlerOutput": {
      "Partitions": { "AddOrUpdateBehavior": "InheritFromTable" }
   }
}
EOF
  s3_target {
    path = "s3://mybucket"
  }
}