"On Startup -> Open the New Tab page" not working; it is doing the "Continue where you left off" behavior

Go to Settings > On Startup and ensure that "Continue where you left off" is NOT selected.

You then have to go into your taskbar, right click on "Google Chrome" and exit. Only then will Chrome pick up the new settings. It should work when you reopen Chrome.