How to re-encode a DVD into a single file?

The only way to do everything you ask is to rip a disk image of the DVD and then play the image. Any other process which doesn't preserve the exact format of the disk will likely remove one or more of the features you're after because DVD is a very specific variant of the MPEG-2 standard. DVD player programs — which you still have to use to interpret the menu information — often depend on their input following the standard closely.

There are lots of ways to rip a DVD image, enough that it's worth a separate question. A quick and dirty way to do it is from the command line:

$ dd if=/dev/scd0 of=my-movie.iso bs=2048

Obviously you must substitute the correct DVD drive dev node for scd0 if that is not correct on your system.

Some media players can play ISO files directly, such as VLC and MPlayer.

If you want to use a player that can't open ISO files itself, you can mount an ISO disk image as a virtual DVD. This is indistinguishable from a real DVD, from the player's perspective. I assume you are using Linux, where you use the loop device for this:

$ mkdir -p $HOME/virtualdvd
$ mount -t iso9660 -o loop my-movie.iso $HOME/virtualdvd

Depending on the permissions on /dev/loop?, you might have to be root (or use sudo) to do the second step. Some Linuxes have their permissions set up in a way that lets you add a user to the group that can create loopback devices without being root.

Having done this, you point your DVD player program at that virtualdvd directory.

Other *ixes usually have an equivalent mechanism to Linux's loop device, which may work a little differently.

If you can give up the interactive menu requirement, I recommend Handbrake, which gives you a file containing:

• The movie, encoded as:

  • H.264, MPEG-4 or MPEG-2 video

  • HE-AAC, AAC, AC3, or MP3 audio

  ...in either an Matroska (MKV) or an MPEG-4 part 14 (MP4) container

• Subtitles, if you want them

• As many of the audio tracks as you want. You can elect to strip the ones you don't want, such as the stereo downmix and the foreign language tracks.

• Chapter marks. You won't have the menus to guide you to the chapters, but your player program will at least let you jump between scenes.

You can also ask Handbrake to rip any "extras" on the disc into separate files. If it were really important to have them in the same file as the movie, you could later concatenate them with the movie file.

Handbrake has device profiles to automatically limit bit rate, frame size, encoding complexity and such to ensure that the resulting file will play on your device of choice.

Checkout the ripper.sh script supplied with x264 transcode. From the README:

To create an iso from a disk you own, edit DEFAULT_ISO_STORAGE and DEFAULT_DVD_RIPPING_DEVICE in the config and run:

 ripper.sh

The script takes care of 'unlocking' the drive, such that the resulting .iso does not include any CSS crap (just running dd is not enough).

With this method you get one file, retain the menu, all tracks and all subtitles.