Service rifle guys drill a 1/8" hole through the rear of the sight down into the receiver (while straight) but not into the elev knob. Then open the hole in the sight base to 5/32". Drive a long 1/8" dia roll pin into the base and let the sight slide up and down on the roll pin.
There are other ways, tapping and installing a stud but the RP is the easiest.
I noticed a "ball" under tension on the left side of the actual sight. It appears that this is what cants the sight. If so, then that would make it part of the design. Is that correct? It's a Colt assembly.
Linear Mode
