How to allow users to connect from multiple devices with refresh tokens?

In my opinion the question you're asking now is not really regarding security anymore, instead it's more focused on the implementation of a concept. You have already established that refresh tokens makes it harder for attackers and it appears like you have a working implementation. From what you are describing, you are making use of a custom solution and not something like an OAuth2 library implementation that will provide this functionality to you.

What you now need to decide is if you want multiple device authentication from the same user account. If you do not want users to sign in from multiple devices on one account, congratulations, your solution is already working. They will be logged out from the old device soon since it will not contain the correct refresh token after a timeout.

If you do want to provide users the ability to sign in from more than one device on the same account, you unfortunately require a few changes. A possible solution is to add a device identification field in your database and issue a refresh token per device.

I agree with Joe's answer

A possible solution is to add a device identification field in your database and issue a refresh token per device.

but I'd like to add some implementation details.

IMO the most robust and functional solution is: for each user to have many records in database, each with two columns: refresh_token and device_id.

Lets examine main cases:

• User login. He makes it with the help of the password, of course. He passes device_id (any unique number, actualy). If password is correct, server generates new access and refresh tokens, sends access_token to clien and inserts refresh_token and device_id into database.
• User consumes the service. He sends access_token (witch internelly have to encode user information, obviously (for example his login), and expiration date). Server checks token (with the help of its secret signing key) to make shure that this token isreal and not a fake one. Also server checks access_token expiration date. If expired, user must refresh it with the help of appropriate refresh_token and (important!) device_id. If refresh_token in database doesn't equal user's token, than we get hacked. But don't panic - we just delete refresh token and redirect user to login page! That's it, hacker only will have access as long as access_token is alive (short time period, as usual)! If you need instant access blocking, you can store access_token along with refresh_token in db and also has a in-memory access_tokens blacklist. Then you can add stolen access_token into blacklist, and instantly block access.
• User logout. He erases his access_token from cookies/localstorage, sends refresh_token and device_id to server, and server deletes them from list. (we don't want to collect obsolete unused tokens)
• User change password. The server deletes all previous refresh_tokens. If you use token blacklist - add all current users' access_tokens to it. With such solution it's also very easy to implement "logout from all devices" function.
• Hacker steal access_token and user don't know about it. Hacker only will be able to use it until it expire (short time frame).
• Hacker also steal refresh_token and made refresh. User will not be able to use his old refresh_token, in that case he will prompt to login again, new refresh and access token will be generated, hacker lost access. (see case #2)

The one thing you should clearly understand : refresh token only can save you in case, when hacker steals access/refresh token, but not the real password! Because if he gets the pass then you're in trouble, and you must implement the protocol for restoring password (with the help of email or SMS, for example)

P.S It's a good idea to have refresh_token that can expire too. To prevent case, when jacker stole access and refresh tokens from old device, that is never in use now.

P.P.S It would be perfect to have periodic database task that will clean rows with expired refresh_token (pehaps you'll have to add column refresh_tocken_expiration_date for that)

P.P.P.S. I'm not security expert, so if you think there is a better solution I would be glad to have a discussion.

Thank you for reading, hope this helps)