Are US PhD admission offers binding?

As far as I know (at least here at Georgia Tech) all the conditions (if there are any) are included in your offer. I have never heard of us rescinding an offer after it has been accepted. If there is a deadline, though, you must accept by then or the offer may no longer be valid.

That said, I think nobody would be offended if your asked, if it puts your mind at ease. I would not mention the "getting good grades in your MS" or similar things.

Just send a note to whomever you have been in contact with and say something like "I'm very happy to have been accepted into the program, and am looking forward to coming. I am pretty sure I've completed everything I need to do to accept admission, but I just wanted to check one last time. I have a number of other acceptances I'm about to decline and I just wanted to check that everything is in order with my admission before I decline the other offers. Thanks for your help and understanding!"

Graduate admissions offers in the U.S. can sometimes be rescinded, but I've never heard of it happening except in extraordinary circumstances. I've known of cases where it was because the applicant failed to graduate, was dishonest in their application, or committed some sort of academic misconduct, but I can't think of any other reason. However, as Johanna noted in the comments, the offer is not conditional on getting sufficiently good grades (unless this is stated explicitly).

I still feel like it is rather informal.

This sounds like it's about as formal as I've seen. I don't think you have anything to worry about. The only worrisome scenario would be if you received an oral offer and gave an oral acceptance, with no documentation, but that shouldn't happen if the department is handling things properly (and indeed it doesn't apply in your case).

The department told me more information would arrive in April.

It's common to send a detailed information packet for incoming students in April, after all the offers have been accepted/rejected. That way the department can deal with it all at once, rather than handling each student individually. This packet typically includes things like registration forms, insurance information, housing forms, etc. (Anything they feel could be useful to students making a decision would be sent earlier.)