How to properly introduce a light/dark mode in Bootstrap?

Bootstrap 5 (update 2021)

Although there is still no definitive support for light/dark mode in Bootstrap 5, SASS can be used to create a dark theme

Bootstrap 4

Here are some answers to your question on Bootstrap light or dark mode:

"Am I supposed to replace all occurrences of bg-dark with bg-light once the "switch" is turned on?"

Yes, but you'd probably also want to switch all -light and -dark classes such as text-dark, navbar-dark, btn-dark, etc..

If I want to slightly modify the colors of bg-light and bg-dark.. I can't find any examples to override these variables (via SASS), except of manually overwrite them in my CSS like .bg-dark...

These are derived from $light and $dark SASS variables so you can change them like this...

$light: #dddddd;
$dark: #011100;

@import "bootstrap";

Demo Bootstrap Light Dark Mode Switch

Also see:
Customizing Bootstrap CSS template
How to extend/modify (customize) Bootstrap 4 with SASS

Bootstrap 4 and 5 do not support Dark Mode. Maybe they will in Bootstrap 6.

Dark Mode is a confusing name in this context because Bootstrap does support a color, $dark: $gray-900, and, it does support several dark-tinted classes, like .bg-dark. But none of these are actual Dark Mode, which is OS-driven, passed-to-browser, website rendering switching based on:

@media (prefers-color-scheme: dark) {

Worse, things like bg-dark imply you would switch to Dark Mode by going around swapping class names out, probably via Javascript. That's a lot of CPU usage to perform a browser built-in, all because the Bootstrap authors ignored an important feature of the Web. Don't do this!

All of this is detailed in a much longer way here.

That will leave you with a lot of options, but in short for Bootstrap 4 and 5 the best, simplest, least-effort way to handle proper prefers-color-scheme: dark is to introduce an indirection into Bootstrap colors by reassigning the Bootstrap color SASS variables you're using to color CSS variables, in your _variables.scss. CSS Variables have been supported for years, are perfect for Dark Mode, and largely overlooked. For example:

:root {
    --body-bg: #fff;
    --body-color: #000;
}

$white: var(--body-bg);
$black: var(--body-color);
$body-color: var(--body-color);

So far all you've done is create indirection - $white still renders as #fff just like in the default Bootstrap. But now you can flip it when the OS flips:

@media (prefers-color-scheme: dark) {
    --body-bg: #000;
    --body-color: #fff;
}

Now when the OS gets set to Dark Mode, the browser sees it and flips that media query, and your CSS variables flip. Since you've spread those variables into the compiled Bootstrap SASS via your _variables overrides, you'll see the result in rendered Bootstrap.

I did say simplest and least-effort, but you've probably noticed this is still going to be a lot of work. Bootstrap 4 and 5 just completely missed the boat on this, so, the legwork is on you. You'll need to override every color in Bootstrap you use and introduce CSS variables for them, and you'll then need to test to find colors that aren't keyed off of any variables, and go override those with a follow-up global stylesheet.