How do I increase the storage for my stimpacks and radaways?

Their storage is directly tied to the number of rooms that produce them you have. Each room will increase the storage.

They are Not affected by the 'storage' room or upgrading the rooms that produce them.

This information can be read in the in-game guide on page 16 (Storage)

According to the in-game guide (page 17):

• Building MedBays and Science Labs will increase the number of Stimpaks and RadAways you can store.

Every MedBay built increases your Stimpak storage by 10. You begin the game with a capacity of 5.

Upgrading MedBays only increases its performance (page 16); it has nothing to do with storage capacity.

Storage rooms only increase storage for dweller's weapons and outfits.