Vim changes into replace mode on startup
It was some internal problem, what solved the problem was adding this line:
nnoremap <esc>^[ <esc>^[
Here you can find some details about this solution.
This happened to me connecting to a cray from wsl. My .vimrc is pretty much empty; the distro has a default looking vimrc (no special nohl keymapping).
" FIX: ssh from wsl starting with REPLACE mode
" https://stackoverflow.com/a/11940894
if $TERM =~ 'xterm-256color'
set noek
endif
Add this to vimrc and it no longer starts in REPLACE mode.
See https://stackoverflow.com/a/11940894 (same thread Jonas mentioned).