When and where should a connected app be created during DX managed package development?

I found the following in the Special Behavior in Metadata API Deployments:

Connected App

• You cannot set the consumerKey in Metadata API. It is included in a retrieve operation for informational purposes. If you try to move the connected app to another org, you must remove the consumerKey from the .zip file before the deployment to an org. A new key will be generated in the destination org.

This makes sense. While you can retrieve the consumerKey if you need it via the Metadata API, you shouldn't be deploying it to each org. This includes scratch orgs.

So, you can have the connectedApp included in the scratch org metadata. Just make sure you exclude the oauthConfig.consumerKey.

Note: Even after excluding the consumerKey from the source I'm still running into challenges with subsequent pushes - Pushing a Connected App with the SFDX CLI results in “invalid cross reference id”

Connected apps can't be created as an unmanaged package.

Technically you need the connected app only in your Dev Hub and all other orgs were you at deploying metadata.

Scratch orgs can be authenticated from the same machine or command line that created them.

I would recommend creating managed packages for the connected app as it eases deployment and installing in orgs where you want to connect like production org.

You will also need to assign the connected app to the profile that's used in deployment for the rest of the orgs.

Connected Apps Deployment in Scratch orgs

you can have the connected app included in the scratch org metadata. Just make sure you exclude the oauthConfig.consumerKey.

For deployment via Metadata API consider the guidelines outlined here.