Your POI change is do to the difference in velocity (Barrel Harmonics) Even with the same grain bullet! They leave the barrel at different point in the vibration deviation!
I wouldn't think you'd get that much deviation - is it s thin barrel or heavy barrel? Could your barrel be loose? Or your optic=sights? Is this shooting from a rest or off hand?
I usually don't see that much deviation even when shooting heavy bullets -normally maybe an inch or so up down or side to side at 100 yards!