Setting up IPSEC on LAN between two hosts (OpenBSD)

Answering my own question like a nerd

Problem 1. OpenIKED (IKEv2) does not support transport mode, so you can only use it for VPNs, and not on a LAN. Use isakmpd (IKEv1)

Problem 2. The documentation for ipsec.conf says that the auth and enc values have defaults, but you seem to need to set them anyways

What else do I need to configure?

You need to set the correct rc.d flags on isakmpd (see below)

Are there logs for IPSEC and iked, and if so, where can I find them?

The logs are at /var/log/daemon

How to tell if IPSEC is working once configured, without looking at packets between the machines?

on B, run tcpdump host A, and on A run ping B . You want to see esp and spi in the tcpdump output

Setup:

Host A (10.0.2.10)

# cat << EOF > /etc/ipsec.conf
ike active esp transport from 10.0.2.10 to 10.0.2.11 \
  main auth hmac-sha1 enc aes \
  quick auth hmac-sha2-256 enc aes 
EOF
# chmod 640 /etc/ipsec.conf

# cd /etc/isakmpd/pubkeys/ipv4
# scp [email protected]:/etc/isakmpd/local.pub 10.0.2.11 `# copy remote's public key`

# rcctl enable ipsec
# rcctl enable isakmpd
# rcctl set isakmpd flags "-KTv" `#K = use ipsec.conf for configuration, T = disable NAT traversal, v = verbose logging`

# ipsecctl -vf /etc/ipsec.conf  `# start ipsec, or reboot`
# rcctl start isakmpd

Host B (10.0.2.11)

# cat << EOF > /etc/ipsec.conf
ike active esp transport from 10.0.2.11 to 10.0.2.10 \
  main auth hmac-sha1 enc aes \
  quick auth hmac-sha2-256 enc aes 
EOF
# chmod 640 /etc/ipsec.conf

# cd /etc/isakmpd/pubkeys/ipv4
# scp [email protected]:/etc/isakmpd/local.pub 10.0.2.10 `# copy remote's public key`

# rcctl enable ipsec
# rcctl enable isakmpd
# rcctl set isakmpd flags "-KTv" `#K = use ipsec.conf for configuration, T = disable NAT traversal, v = verbose logging`

# ipsecctl -vf /etc/ipsec.conf  `# start ipsec, or reboot`
# rcctl start isakmpd