How can I free up space taken up by an incomplete Microsoft Store download?

Does it store its download files in a specific location that's accessible to me? Will any disk cleanup utilities take care of it for me?

Yes; Applications installed from the Windows Store are placed in the WindowsApps directory that exists within Program Files.

Changing the permissions on the WindowsApps directory will break any UWP application you have installed. Changing the permissions will cause more problems than it solves.

How can I free up this space that's reserved by a failed download from the Microsoft Store?

The following actions should be sufficient to solve your problem.

1. Press Windows logo key + X and click Run.

2. Type WSReset.exe in the empty space and hit ENTER

Source: Here

Found the solution. Steps I used to fix this:

1. Open the C:\Program Files\WindowsApps folder, which is where the Windows Store puts its downloads. The WindowsApps folder isn't permissioned to normal users, so this has to be done using some utility that's being run as administrator (either a command prompt window, or some other file manager like 7zip; h/t ManSamVampire for the 7zip suggestion). DO NOT assign ownership of the WindowsApps folder to your own user account to gain access to it, as doing so will cause major problems and can't be easily undone (h/t Ramhound's answer for this warning).
2. Sort the subfolders inside WindowsApps by the date they were last modified; this will let you easily find the folders that have recently had new stuff put into them.
3. In my case, the offending subfolder was called MSIXVC, which is apparently a type of installer used by the Windows Store. Inside, I had an anomalous 37GB file created last night with no file extension, called "8BBF2763-A1C6-4F1B-96CA-B79DD5FDA4EE". Given that this file was enormous, created at the right time, and in a subfolder suggesting it was an installer, I concluded that this was the droid I was looking for.
4. From within your run-as-administrator utility, delete the offending file. I deleted the ginormous, bafflingly-named file from the MSIXVC folder, and lo and behold I had all my disk space back!

Now redownloading and installing Outer Worlds. This time I'll know better than to turn off my computer partway through.