Switching fields within biomedical sciences between PhD and postdoc?

My background for context: BSc Mathematics, MSc Applied Mathematics, PhD Mathematical Modelling in Electrophysiology, Postdoc Biomedical statistician in gastroenterology but I avoided statistics like the plague right up to the end of my PhD. I found switching fields overall a hugely beneficial experience, although not without its drawbacks.

Pros:

• You learn a LOT. I've found it really interesting to see how other fields work, and I feel like I'm more competent in a wider range of scientific techniques than I was when I finished my PhD. I was nervous about starting in a second field. Now, I would not be nervous about starting in a third field.
• That said, I prefer this field to my old field. I want to stay in this field beyond the duration of my postdoc. And if you decide you want to go back to your old field, you can.
• You can contribute in ways that no one else in your group can because you can suggest methods that no-one else knew existed. Things that are common in your field might be unheard of in your new field, which is a very fertile ground for novel science (this applies in reverse if afterwards you go back to your old field).
• Your naivety about the field just means that you approach things with fresh eyes. You ask "stupid questions" and it turns out no one had ever asked them and they are actually really interesting.

Cons:

• The imposter syndrome is real. You start off and the 2nd year PhD students know more than you. It feels like you're a student, but you're a postdoc. People mistake you for a student because of your lack of knowledge of some fundamental principles. But from a PhD, the thing you will have learned the most is how to learn, which is a transferable skill, so you'll catch up fast
• Everyone I worked with has been very patient and understanding that this isn't my field, but I can imagine situations in which this isn't the case and people see you as incompetent

If you're willing to spend a while feeling like a first-year PhD student again for the benefits on the other side of that hurdle once you settle a bit, I would recommend it

The time between PhD and Postdoc is the perfect time to change fields. Many, many people in biomedical sciences will change fields between PhD and Postdoc, particularly if your old field is as narrow as a single rare disease - I don't believe anyone would expect you to get a postdoc in that field.