MalformedPolicyDocument error when creating policy via terraform

Alternatively change <<EOF to <<-EOF to allow indentation. It will then remove the number of indentation matching the line with fewest indentations at apply.

You need to make sure that you don't have any indentation at the start of your EOF heredoc because your JSON policy should not start with an indented brace.

So you should be fine with this small change:

resource "aws_iam_role_policy" "rds_policy" {
  name = "my-rds-policy"
  role = "${aws_iam_role.rds_role.id}"
  policy = <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
      {
          "Effect": "Allow",
          "Action": [
              "s3:ListBucket",
              "s3:GetBucketLocation"
          ],
          "Resource": [
              "arn:aws:s3:::my-bucket"
          ]
      },
      {
          "Effect": "Allow",
          "Action": [
              "s3:GetObjectMetaData",
              "s3:GetObject",
              "s3:PutObject",
              "s3:ListMultipartUploadParts",
              "s3:AbortMultipartUpload"
          ],
          "Resource": [
              "arn:aws:s3:::my-bucket/backups/*"
          ]
      }
  ]
}
EOF
}