Yeah, the panic buying stinks. I wish we didn't have to buy any more bullets or primers than you need short term, but that is the sad reality if you want to be able to shoot your guns whenever you want.
The local stores in my area are rationing ammo. Two boxes per customer. They are also rationing 30rd AR mags, 2 per customer, $50 ea. One store actually had a Bushmaster AR on the rack (dressed like an M4). Price? $2000! I think that same model could have been had new for around $850 last November.
Might as well get used to it. It took about 18 months for things to rebound back to reasonable supply levels in my area after BHO was elected in 2008. It's gonna be a while. At least as a hand loader, you have a few more options than the casual shooter has.
A handgun is only good for fighting your way to a rifle.