MongoDB queries are taking 2-3 seconds from Node.js app on Heroku

Change Streams indeed affect the performance of your server. As noted in this SO question.

As mentioned in the accepted answer there,

The default connection pool size in the Node.js client for MongoDB is 5. Since each change stream cursor opens a new connection, the connection pool needs to be at least as large as the number of cursors.

const mongoConnection = await MongoClient.connect(URL, {poolSize: 100});

(Thanks to MongoDB Inc. for investigating this issue.)

You need to increase your pool size to get back your normal performance.