Is reverse proxy still required between AWS ALB and application server?

A reverse proxy isn't required, but the web servers that ship with Rails have minimal functionality, so for any application that is public facing and can potentially grow, its a good idea to integrate a reverse proxy (eg Nginx) early on.

For example, this would give you advanced rate-limiting capability, caching and the ability to do end to end SSL termination with a lot less fuss that other web server implementations. Logging also becomes a lot easier, and stuff like TCP retries can be a life saver if your application is under load.

You don't need a sidecar. You can install Nginx on the same box as your application. Its super efficient and will not impact your application performance.

We use multiple Nginx proxies in front of Rails, and we serve billions of http requests per day.

Massive reverse proxying implementations such as Cloudflare are also built on Nginx.