How to block everything (all incoming and outgoing internet access) except those applications are in firewall white-list?

You do not need any 3rd party firewall software.

By default windows firewall functions as follows:

• Inbound - Block all unless in the list (Whitelist)
• Outbound - Allow all unless in the list (Blacklist)

You want to Block all Inbound and all Outbound connections by default.

You can do this using Windows built-in Firewall. The way to do this (though somewhat hidden away way) is to change the settings as follows in these 3 easy steps:

1. Go to: Control Panel\System and Security\Windows Firewall

2. There, right-click as shown in screen shot to get the properties:

   

3. Change Outbound Connections to Block for each profile Now you can add only the programs you want to the list.

You can import/export rules by right clicking the same as in screenshot above and selecting Export Policy. It imports/exports the whole thing. So you can experiment, disabling rules and making your machine more secure. For example my settings are as follows (excluding my programs):

Inbound - there's not a single rule here!

Outbound - only "Core Networking - DNS (UDP-Out)" is enabled

If you're using OpenVPN you will also need to add two more Outbound rules:

Core Networking - Dynamic Host Configuration Protocol (DHCP-Out)
and a rule to allow openvpn.exe

In an elevated shell window, do

Set all profiles to block inbound/outbound traffic:

netsh advfirewall set allprofiles firewallpolicy blockinbound,blockoutbound

Remove all rules:

netsh advfirewall firewall delete rule all

Allow basic outbound rules for ports 80,443,53,67,68

netsh advfirewall firewall add rule name="Core Networking (HTTP-Out)" dir=out action=allow protocol=TCP remoteport=80
netsh advfirewall firewall add rule name="Core Networking (HTTPS-Out)" dir=out action=allow protocol=TCP remoteport=443
netsh advfirewall firewall add rule name="Core Networking (DNS-Out)" dir=out action=allow protocol=UDP remoteport=53 program="%%systemroot%%\system32\svchost.exe" service="dnscache"
netsh advfirewall firewall add rule name="Core Networking (DHCP-Out)" dir=out action=allow protocol=UDP localport=68 remoteport=67 program="%%systemroot%%\system32\svchost.exe" service="dhcp"

And to reset firewall to default values

NETSH advfirewall reset

** All changes take effect immediately

Another very useful, powerful and of course free here:

TinyWall
TinyWall takes a different approach from traditional Firewalls. It does not display popups that "urge users to allow". In fact, it will not notify you of any blocked action at all.
Instead of showing popups, TinyWall makes it easy to whitelist or unblock applications by different means.
For example, you can just initiate whitelisting by a hotkey, then click on a window that you want to allow. Or, you can select an application from the list of running processes.

Of course, the traditional way of selecting an executable also works. This approach avoids popups, but still keeps the firewall very easy to use.

Most importantly, with the no-popup approach, the user will only notice that a program has been denied internet access when he can't use it anymore.
Consequently, users will only unblock applications that they actually need and none more, which is optimal from a security standpoint.

Feature overview

• Multiple and easy ways to whitelist programs
• Automatic learning mode
• Firewall tampering protection
• Password lockdown of settings
• Quick modes, like Normal protection, Allow outgoing, Block all, Allow all and Learning mode
• Support for temporary/timed firewall rules
• Port and domain blocklists
• Hosts file protection
• Option to always allow communication within LAN
• Option to restrict an application to the LAN
• Recognition of safe software and impostors
• Full IPv6 support
• List established and blocked connections
• View open ports on your machine
• 100% free and clean software. No fees, no ads, no paid upgrades.

You can see the Thiny wall white-list in below image: