Moved folder into submodule, now getting "Untracked files would be overwritten" message

Just use git checkout -f to swap branches, then remove them like you normally would and merge in master to get your submodule introduction.