How to prevent editText from auto filling text when fragment is restored from saved instance?

Found the problem with help from a friend!

SaveEnabled is default true on editTexts in Android.

The solution is simply to set:
setSaveEnabled(false);
on your editText.

Turns out Android isn't very smart in how it restores state for views with identical IDs in the same hierarchy (even in parent views with different IDs). I just turned this off because I save my state in a ViewModel object and restore it from my fragment's onCreateView().

This is not the answer, but it fixes the problem:

As commented by @Slugge:
Override the onSaveInstanceState method in your custom editText and clear the text / do what you want from there.