How to detect if employees are using Tor?
You can use a list of Tor (uplink) nodes, add this to the outgoing firewall, setup a task to update this once a day and you'll be good. But Tor can also be used over a HTTP(S) proxy, so you will have to detect proxies as well.
I am not sure if this is going to help you secure anything. As long as there is a connection to the internet, it would be possible to bypass these kind of security measures. You could end up spending endless time and energy to prohibit all kinds of proxies, VPN's, SSL tunnels and such. The advice is to just make sure they cannot do any harm by protecting whats important to your business, and leave users be. For example separate the network in compartments, use subnets, VLANs, DMZs and require authentication and authorization on private networks. Keep the important stuff in one zone, while allowing networking without restrictions on another. And so on...
I believe your major concern should be that:
using anonymous proxy in the network
Is a bad assumption. I would straight away ask:
In which network?
Yorick already touched this point but I'll be more blatant.
HIPAA is mostly about privacy of the data in your production system and therefore in the network that is used to connect to the production system. You should have the control of what all machines connecting to this network can do. In other words, these should be company machines managed by the company, and provide the software that is needed to deal with the production system. No other machines should connect to this network, this includes VPN to the production network (which should be best avoided if possible).
A network for your employees that do not connect to the production system, e.g. a development network or office wifi shall be separate from the production network. You only need to explicitly show that the networks are separate (preferably with separate hardware, 802.1Q VLANs are subjects to a couple of attacks if badly configured). The machines in these networks are of no concern for the production system as long as they never connect to it (they should not!). Moving anything into production shall anyway have a QA/QC procedure in which the security of the code/configuration/another change is evaluated.
It is worth noting that the development network containing the development machines shall never see any production data. If your production data is to be safeguarded (e.g. for patient privacy as in HIPAA) you must anonymise all data in development/test setups. Having a secured production environment and then dumping production data into an unsecured network would be just silly.
The only way you can be relatively sure Tor is not in use on the network is to inspect every device on the network and ensure that Tor is not installed or running on any of them. This could require so many man-hours that it may as well be impossible. And you might miss it anyway.
You can attempt to detect the use of Tor by watching for traffic to any of the hardcoded directory authorities, to which all Tor clients will always connect (with an exception, see below). There are usually less than a dozen of these.
You could also attempt to detect traffic to any of the thousands of guard nodes, but that may put a lot of strain on your IDS. Detecting directory authority traffic requires watching for only a few IPs and there will be traffic to these authorities even for otherwise idle clients.
The big exception is when Tor has been configured to use a bridge. These are explicitly designed so that Tor does not communicate directly with any of the normal Tor nodes or directory authorities. Instead they communicate with an unpublished bridge address. Nation states have difficulty detecting these connections; if one is in use on your network, you have pretty much no chance.
What you should be doing instead is more carefully controlling access to and from the network in general. Only traffic which is actually needed should be allowed to and from any device where PHI may be accessed or stored. This means you need to default-deny in both directions, incoming and outgoing, also segregating networks where PHI is present from other networks. It sounds like your current firewall egress policy is default-allow, and blocking things in a default-allow configuration is like punching holes in the sky.