How can I make stamina regen fast in battle?

The obvious reason for stamina to regenerate slower in combat is balance - it's nice to be able to sprint more while out of combat, but limiting stamina regen in battle adds depth to combat.

You can use armor with fortify stamina regen enchantments to give yourself a passive boost to stamina regen both in and out of combat, as well as a weapon enchanted with absorb stamina to help you replenish stamina in combat. Having a weapon with absorb stamina will allow you to power attack non-stop (it doesn't matter whether you have 1 or 200 stamina, as long as it's not at 0 you can do a power attack), so tread carefully if you enjoy balanced combat.

From the UESP wiki, "Stamina" article:

• Stamina regenerates whenever you are not doing an action that uses stamina. You regenerate 5.00% of your maximum Stamina per second outside of combat and 1.75% inside combat.
  • As a result, sleeping or waiting fully restores your stamina.

This was most likely done to balance the combat in the game and not make it overpowered, encouraging you to manage or optimize your use of Stamina. The same thing is done with Magicka, which regeneration rate also decreases while in combat.

Increasing Stamina Capacity and Regeneration Rate

Use Fortify Stamina potions or enchantments to increase your Stamina capacity and Fortify Stamina Regen potions or enchantments to increase your Stamina regeneration rate. Also, see the list of Fortify Stamina and Fortify Stamina Regen items in the game.

Try to look for or enchant weapons with the Absorb Stamina enchantment, which will allow you to absorb stamina every time you hit an opponent. Some strong unique weapons that have the Absorb Stamina enchantment are Volendrung (Enchantment: Absorb 50 points of Stamina) and the Nightingale Blade (Absorb Stamina enchantment: Absorb Stamina, 25 pts for 1 sec).

Redguards are a preferable player race choice if you want to have the Adrenaline Rush greater power (Stamina regenerates 10x faster for 60 seconds, once per day), which is useful in situations where you need to use a lot of Stamina in combat.

Also, a Stamina tip (from a Reddit /r/Skyrim post):

When you leave just a little bit of stamina it starts recharging instantly when you stop sprinting but when it runs out completely, it takes 2-3 seconds to starts recharging.

I am thinking that this is also the same case in combat, but I haven't been able to test this yet.

PC Mods

If you want to use a mod, you could try Magicka and Stamina Combat Regen, it can increase combat Stamina Regen rate from 1.5x to 15x the normal rate.

If you combine the Redguard Adrenaline Rush greater (racial) power with a mod like No Racial Power Cooldown, then you will effectively make that greater power permanent and will have no need to worry about Stamina regeneration anymore. A mod that will allow you to use your racial power more frequently, but less "cheaty" is the Auto-cast Racial Powers Plugin (Redguard: Adrenaline Rush will now have a 25% chance to activate when hit by an aggressor when Stamina reaches 25% or below and Health is at 80% or below).