Tilde/Backtick button not working on Windows 10

Update (2018-07-05): Recent versions of Windows 10 may have changed the location of the settings menu. See the answer of @Wingu for an alternative.

---

I finally figured it out. The problem is that the Windows language switcher uses the tilde to switch between layouts. To fix this, go to Advanced Language Settings in the Control Panel:

Click on "Change Language Bar Hotkeys", which opens up the following dialog:

Here, click on "Change Key Sequence", and on the following dialog change the assignment away from the tilde/grave accent key. Or simply unassign it.

I have Windows 10 yet I cannot find Advanced Language Settings in the Control Panel so to the others who have the same problem, heres how I found it and follow the instructions by exhuma!

Open Windows Settings and search for Region and language settings

In related settings, you will find Advanced Keyboard Settings

In Switching input methods, you will find Language bar options

Then continue to proceed with exhuma's instructions after the first image included.

I experienced a similar issue with the backtick key not working directly -- on the second keypress, it would type two backticks. This was interfering with my use of Ctrl-` as a shortcut in my IDE.

The solution I found was to change Language Setting to English (United States), to use the 'US Keyboard' settings.

As a side note, I would tend to observe that Windows 10's Settings UI -- being a mixture of new pages, links to older dialogs, and Control Panel -- is a sodden mush of dog's breakfast which makes most tasks harder & more confusing rather than clearer.