Access my server through SSH with the domain (with Cloudflare)

When you connect by IP address the SSH connection goes directly to your server but if you use the domain name it goes through Cloudflare defenses. My suggestion would be to either use direct.pollnote.com (I think CloudFlare creates it automaticaly but people often remove it) or add your own alias like ssh.pollnote.com and disable CloudFlare protection on it.