block all but a few ips with firewalld

The rich rules aren't necessary at all.

If you want to restrict a zone to a specific set of IPs, simply define those IPs as sources for the zone itself (and remove any interface definition that may be present, as they override source IPs).

You probably don't want to do this to the "public" zone, though, since that's semantically meant for public facing services to be open to the world.

Instead, try using a different zone such as "internal" for mostly trusted IP addresses to access potentially sensitive services such as sshd. (You can also create your own zones.)

Warning: don't mistake the special "trusted" zone with the normal "internal" zone. Any sources added to the "trusted" zone will be allowed through on all ports; adding services to "trusted" zone is allowed but it doesn't make any sense to do so.

firewall-cmd --zone=internal --add-service=ssh
firewall-cmd --zone=internal --add-source=192.168.56.105/32
firewall-cmd --zone=internal --add-source=192.168.56.120/32
firewall-cmd --zone=public --remove-service=ssh

The result of this will be a "internal" zone which permits access to ssh, but only from the two given IP addresses. To make it persistent, re-run each command with --permanent appended, or better, by using firewall-cmd --runtime-to-permanent.