I'd be more inclined to build up the back of the grip safety a little more, myself.
Personally, when I holster i do so with the thumb on the hammer. This allows the thumb to hold the hammer back if the thumb safety has been missed and something interdicts the trigger. It also pulls the web of the hand away from the grip safety, activating it and making it pretty much impossible for the gun to discharge in the holster even if pressure is applied to the trigger.
I haven't personally done a case in which the defendant's grip safety had been deactivated. I have, however, been involved in one case where the prosecutor hammered on the theme, "the defendant was so reckless he deactivated the safety device on a lethal weapon!" That's a powerful argument with a lay jury, and one I'd really rather not have come up if I was the defendant.
You might also try shooting with your firing thumb a little lower. It tends to pull the web of the hand tighter into the grip safety and operate it more effectively.