When would you place a module in the app/code folder versus another location?

As you already pointed out, composer is a package management tool. Any external modules/packages which constitutes your Magento application will be managed by the composer and will be available under vendor directory, provided you have adopted the composer way of installation/ updating of your Magento instance. Magento core modules are also treated as external packages which constitute your Magento application.

app/code expects those Magento modules which are "unique" to your Magento application. Those jobs or customization which we cannot achieve by a third party/external modules has to be here. In most of the time, these modules are not installed or updated using composer. In most of the projects, your version system generally holds and manage your app/code folder only (There are other folder files we need to keep inside a versioning system, but none of them as relevant as modules inside app/code.

But it is a good practice to keep a composer.json file for those modules too. Reasons are many:

• Magento recommends it as a good practice.
• You can specify your module dependencies there. ex: php version, core module dependencies etc.
• If in case, if you want to make your module a general third-party extension, then it would be so easy.

If you need more details of composer.json files which are using in Magento, you can follow this official documentation about the composer.json file.

Modules in vendor folder would update using composer

And all the modules in app/code would not be updated through composer

That's why when you need to override any module you add it in app/code