How do you add a comment to a json IAM policy?

No. In general, comments as you describe them are not allowed in JSON. To effectively create a comment, you would need to allow for a new element that describes comments. Since AWS is the master of this json object, they would be responsible for allowing this.

They currently only allow the following elements:

• Version
• Id
• Statement
• Sid
• Effect
• Principal
• NotPrincipal
• Action
• NotAction
• Resource
• NotResource
• Condition

Hyper Anthony's answer is correct in the strict sense of 'comment' - however, in most situations you can at least use the Sid for pseudo comments to communicate the intent or any constraints etc.:

The Sid (statement ID) is an optional identifier that you provide for the policy statement. You can assign a Sid value to each statement in a statement array. In services that let you specify an ID element, such as SQS and SNS, the Sid value is just a sub-ID of the policy document's ID. In IAM, the Sid value must be unique within a policy. [emphasis mine]

This is e.g. exemplified by the use of TheseActionsSupportResourceLevelPermissions within the (very helpful) AWS blog post Demystifying EC2 Resource-Level Permissions:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "TheseActionsSupportResourceLevelPermissions",
            "Effect": "Allow",
            "Action": [
                "ec2:RunInstances",
                "ec2:TerminateInstances",
                "ec2:StopInstances",
                "ec2:StartInstances"
            ],
            "Resource": "arn:aws:ec2:us-east-1:accountid:instance/*"
        }
    ]
}

• As mentioned in Sid some services might require this element and have uniqueness requirements for it, but I haven't experienced resulting naming constraints yet.