iptables forward traffic to vpn tunnel if open

You will need both sets of rules within iptables. The two rulesets ensure that traffic leaving by the specified interfaces is appropriately masqueraded. Here is my suggestion, which is a little simpler than yours:

# Masquerade outgoing traffic
iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
iptables -t nat -A POSTROUTING -o tun0 -j MASQUERADE

# Allow return traffic
iptables -A INPUT -i eth0 -m state --state RELATED,ESTABLISHED -j ACCEPT
iptables -A INPUT -i tun0 -m state --state RELATED,ESTABLISHED -j ACCEPT

# Forward everything
iptables -A FORWARD -j ACCEPT

The part of the puzzle that's missing is the routing. If the tunnel is up you want "all" outgoing traffic to use it. Otherwise use the normal route.

This is handled within OpenVPN using the redirect-gateway def1 parameter in your client configuration.

The handy tool is to list existing rules with line-numbers:

iptables --line-numbers -t filter -L FORWARD

You could delete the rules with -D option:

iptables -t filter -D FORWARD 1

You could insert a new rule at specified location with -I option:

iptables -t filter -I FORWARD 0 blah-blah-blah

this would insert a new rule at the very beginning of a table, so it will be consulted in a first turn.

Edit:

Generally, you need only one rule in the FORWARD table that match -m state --state RELATED,ESTABLISHED:

-I FORWARD 1 -m state --state RELATED,ESTABLISHED -j ACCEPT

as connection tracking would allow all already known connections to be routed.

And yes, you need to set up policy routing to forward your wlan traffic not to default gateway that is most likely reachable through your ethernet interface but trough vpn interface.