Which devices store DNS cache and how to flush?
[A]m I correct that routers also often store DNS cache?
Some do but it depends entirely on the router, firmware involved and the current settings if applicable.
I'm assuming the ipconfig command doesn't flush the router[?]
No it does not.
[I'm assuming most] routers would flush their cache on a power cycle?
Again, it depends entirely on the router and firmware involved. That said, it would likely not be unexpected behavior.
So does this mean if there's issues with an invalid DNS entry each intermediary router may [need] to be power cycled?
It could be a possibility, yes, at least if you want them to immediately forget any cache entries (assuming a power cycle would do this).
If they were doing caching, it might also be possible to log into them manually and clear the cache.
Which devices store DNS cache and how to flush?
Almost all DNS servers store DNS cache. Depending on your network requests will usually pass through one or more recursive DNS servers. Normally, they will all cache the data for the time specified by the upstream DNS server.
Most recursive DNS servers will flush their cache when they are restarted. Other than that they usually do not have a method to flush cache. Cached values just time out and get replaced by new values as needed. If the cache size is limited, inactive cached values may be removed from cache before they time out.
Your home router may have a DNS server which may be able to cache DNS values. They are usually configured to use your ISP's DNS servers, which in turn may connect to other recursive servers. Your ISP's DNS servers and other upstream DNS servers are unlikely to be flushed unless they are upgraded or the server is rebooted.
People who maintain DNS tables generally set short timeout values when making changes. This ensures that properly functioning DNS caches clear incorrect values quickly if a mistake is made or a value is being changed. Once the change is verified, the timeout may be increased.
Authoritative DNS servers are almost always configured so that there are redundant servers on different networks to ensure that at least one server is always available.