?4 being added to file name with this Bash script, why?

That ? is just a placeholder character output by ls to represent the newline character that you have embedded in your file name.

echo foo3bar.mp4 | grep -Eo '[0-9]+'

would output:

3
4

As there are two sequences of 1 or more decimal digits in that file name.

Instead you could do:

for file in *[0-9]*.mp4; do
  n=${file%.*} # remove the extension
  n=${n%"${n##*[0-9]}"} # remove anything after the last digit
  n=${n##*[!0-9]} # remove everything up to the last non-digit.

  ffmpeg...
done

Which would extract the right-most sequence of decimal digits on the filename stripped of its extension and avoid running several commands for each file.

Note that that code only uses standard POSIX sh syntax, so you don't need bash to run it. With bash or other ksh93-like shells, if you know the root name of the file only contains one sequence of digits, you can also use:

n=${file%.*} # remove the extension
n=${n//[^0-9]} # remove all non-digit characters

With bash-specific features:

[[ $file =~ ([0-9]+).*\. ]] && n=${BASH_REMATCH[1]}

Would extract the left-most sequence of digits in the root name.

With zsh:

n=${(SM)${file:r}##<->}