System wide SOCKS5 Proxy

SOCKS5 is a protocol (i.e. in the application layer of OSI), so plain network-routing (e.g. via iptables) alone won't do. (It's probably necessary, but not sufficient.)

What you need is a proxifier. Without having tried it, tun2socks, allowing you to "socksify TCP at the network layer", looks promising (as does proxychains, without iptables but prefixing programs explicitly).

(Could socat be used here? This mentions socat for socksifying, but I'm unsure at the moment.)