Jason, there is anecdotal evidence to support both theories. We've all heard about the guy who found Grampa's loaded 1911 magazine from WWII, and the old mag and old ammo worked perfectly when fired half a century later. Police departments have magazines that have been fully loaded on duty for years and never failed.
However, I also know police armorers who have had to replace magazine springs routinely, and competitive shooters who make a point of leaving their match magazines empty between shooting events because they've found the ones kept fully loaded for long periods develop weakened springs.
Metallurgy school says the springs should hold their "set" indefinitely. The magazine springs themselves haven't been to metallurgy school, might not have been perfectly manufactured, and occasionally do lose their "set."
Personally, I take a "belt and suspenders" approach and change out my loaded magazines when I change my clocks and smoke alarm batteries and so on with the Daylight Savings time changes. If nothing else, it gives more peace of mind.
YMMV...Your Magazines May Vary.