Any change in the physics of the extraction will affect the trajectory of the expended case.
Certainly polishing or changing the contact areas of the extractor can be one answer. But it may just be a simple problem with the type of ammo you generally shoot with-- and not with higher potency SD ammo.
Other options: change to a different rate recoil spring, or change ejector (40 ejector will work for both 9mm and 40).
Wear a hat at the range, keep your shirt buttoned, and carry +p ammo.