Is it possible for a Chrome extension to access an IndexedDB database created by a specific domain?

No, you can not do it. The data storage is sandboxed http://www.html5rocks.com/en/tutorials/offline/storage/

Yes, IndexedDB can be accessed through chrome extension with the help of content scripts. Content Scripts can access the indexedDB storage of the domain you have specified in manifest.json file.

For more info check here: https://stackoverflow.com/a/60002682/9131807