Is it worth it to upgrade training rooms?

Just to put some numbers here:

Strength Level | Base    | 1st Upgrade | 2nd Upgrade
----------------------------------------------------
1 >>> 2        | 26m     | 25m         | 24m 
2 >>> 3        | 1h 20m  | 1h 16m      | 1h 13m
3 >>> 4        | 2h 40m  | 2h 32m      | 2h 26m
4 >>> 5        | 4h 27m  | 4h 14m      | 4h 3m
5 >>> 6        | 6h 41m  | 6h 22m      | 6h 5m
6 >>> 7        | 9h 21m  | 8h 55m      | 8h 31m
7 >>> 8        | 12h 28m | 11h 53m     | 11h 21m
8 >>> 9        | 16h 2m  | 15h 17m     | 14h 36m
9 >>> 10       | 20h 3m  | 19h 6m      | 18h 15m

Source: http://www.ign.com/wikis/fallout-shelter/Weight_Room. Other rooms have very similar (though not identical) numbers.

So you should be really only updating those if you have some ungodly amount of CAPS and it's burning a hole in your vault.

Update

I have some 700k caps now, and I really don't know what to spend them on. Perhaps, now might be the time to upgrade some training rooms. However, more than half of my dwellers have full SPECIAL now and training rooms are half-filled, at best. Again, upgrading training rooms wouldn't have any effect, other than making me 50-100k caps lighter.

Regardless of caps costs, you shouldn't upgrade training rooms. The reason is that higher level rooms get more severe incidents. The dwellers in your training rooms are usually low level, so if you don't upgrade your training rooms your rookies will be able to handle incidents unattended. If your training rooms are upgraded, you will have to heal your dwellers during the incident to keep them from dying. So even once you get more caps than you need, you should avoid upgrading your training rooms.

I won't upgrade those rooms. You have later more than enough people to get others for hours in the training room. Buy 2-3 rooms and put the people in them instead one fast room. It's much cheaper and efficient.

By the way. You can further improve this. Just make a woman pregnant and put her inside the training room (not for the real world :D). This way you get more people while you train the existing ones. ;-)