OP, I assure you that a 4" smith with 180gr JSP's or 180gr HC or 200gr HC bullets all work well, and kill deer in 357mag. One of mine is a 4" Smith 686.
Go have a good time and shoot straight. Dang, they even work great out of the 3" Ruger SP101 with 180 HC bullets. (my hand loads).
Good hunting!
Edited because of lead weight typo's...
CM