Why would a hosting provider not allow SSL over port 443?
Solution 1:
Historically, HTTPS required a dedicated IP per site/certificate, since the browser needs to verify the certificate before sending the Host header. It's possible that your hosting provider uses dedicated ports instead, in order to conserve IPs.
Nowadays, however, pretty much all modern browsers support Server Name Indication, which allows virtual hosting multiple HTTPS sites on the same IP and port, so even that isn't a particularly good reason anymore. If this is a shared hosting service, it's unlikely that there are any config changes you can make to make your site be available on the default port.
Solution 2:
tl;dr: Switch the provider, this one seems a bit weird.
Is there a good reason why they are not letting that happen?
Depends on what one thinks is "a good reason", but in my opinion: No.
Or is there any configuration that I can change on the server that will make it accept https requests on port 443?
If the provider is blocking / filtering access, sadly, you can't do anything about this on your side.
Solution 3:
Possible explanation:
There are at least some providers who sell you virtual machines which have their distinct IPV6 addresses, but a shared IPV4 address. One example is this (this link should in no way be seen as an endorsement or advertisement). You typically get a few port forwards, but thery're TCP based (no SNI, HTTP host header or anything), so the port number is the only way to select "your" host.
With IPV4 addresses becoming rarer and rarer, this is one way of saving on them.
If the website is for a charity or similar organization which has few funds, and you absolutely have to stick to the lowest price possible, you can probably set up a free cloudflare plan that forwards your-domain.com:443 to your 41696 port.
If not, get a different hoster.