How can I fit a video to a certain size, but don't upscale it with FFmpeg?

A more readable version can look like follows:

-filter_complex "scale=iw*min(1\,min(640/iw\,360/ih)):-1"

640/iw is the horizontal scaling factor and 360/ih is the vertical scaling factor

You want to fit the scaled image inside the output box and keep the (storage) aspect ratio. You do this by selecting the smallest scaling factor with the minimum function: min(640/iw, 360/ih)

You want to prevent any upscaling (i.e. a scaling factor > 1.0) so you add another minimum function: min(1, min(640/iw, 360/ih))

Next step is to calculate the output resolution by multiplying the scaling-factor with input-width and input-height:
output-width = iw * min(1, min(640/iw, 360/ih))
output-height = ih * min(1, min(640/iw, 360/ih))

Last step is to construct the filter command. There is no need to specify the output-height, you can specify -1 and ffmpeg will keep the aspect ratio by applying the same scaling factor as for the width.

With newer ffmpeg versions, you can use the scale filter's force_original_aspect_ratio option. For example, to fit a video into 1280×720, without upscaling (see this post for more info):

ffmpeg -i input.mp4 -filter:v "scale='min(1280,iw)':min'(720,ih)':force_original_aspect_ratio=decrease,pad=1280:720:(ow-iw)/2:(oh-ih)/2" output.mp4

Here, the scale filter scales to 1280×720 if the input video is larger than that. If it is smaller, it will not be upscaled. The pad filter is necessary to bring the output video to 1280×720, in case its aspect ratio or size differs from the target size.

---

With older ffmpeg versions, there is a somewhat hacky workaround. First, define the width, height and aspect ratio of your output. This will save us some typing.

width=640; height=360
aspect=$( bc <<< "scale=3; $width / $height") # <= floating point division

Now, let's apply the super complex filter command that Jim Worrall wrote:

ffmpeg -i input.mp4 -vf "scale = min(1\,gt(iw\,$width)+gt(ih\,$height)) * (gte(a\,$aspect)*$width + \
lt(a\,$aspect)*(($height*iw)/ih)) + not(min(1\,gt(iw\,$width)+gt(ih\,$height)))*iw : \
min(1\,gt(iw\,$width)+gt(ih\,$height)) * (lte(a\,$aspect)*$height + \
gt(a\,$aspect)*(($width*ih)/iw)) + not(min(1\,gt(iw\,$width)+gt(ih\,$height)))*ih" \
output.mp4

I won't really go into explaining what this all does, but basically you can feed it any video, and it will only downscale, not upscale. If you're up for it you can dissect the filter into its individual expressions. It might be possible to shorten this, but it works like that as well.

I had same problem too, but solved by fitting the video in a square 640x640 (because of vertical videos made with smartphones).

So using immerzi logic and some research I end up with this:

-vf "scale=iw*min(1\,if(gt(iw\,ih)\,640/iw\,(640*sar)/ih)):(floor((ow/dar)/2))*2"

the last part is for having a height divisible by 2 that is need by many encoders.