SSL certificate and Azure classic load balancer

Azure's Load Balancer is a Layer 4 balancer and can balance TCP and UDP traffic.Therefor, it doesn't support SSL offloading.

The Application Gateway can balance at Layer 7, so it can do SSL offloading. This means you only need to upload the certificate to the App Gateway.

If you want to stick with the LB, all your VMs will need the certificate. You should be able to balance on port 443 with no issue. You'll need a balancing rule and a health probe, and you will need to allow traffic to 443 from the Internet in your Network Security Groups.