Navy corpsmen who are assigned to Marine units do not carry a rifle. They just carry a side arm, their job is not to engage the enemy. The next question about uniforms is covered under Marine Corps Order P1020.34F. Certain personel assigned to marine units including but not limited to Religious persons, corpsmens, and medical techs and some officers may decied to go Marine Corps clothing Regulations. http://www.tecom.usmc.mil/mcub/library/MCUR/URCH8.htm
but Navy personel do not wear the Marine Insignia they have their own and the classic Eagle, Globe and Anchor are for exsclusive marine use only nor do they wear anything that says USMC that also is reserved for Marines. The catch for many when going Marine Regs is that they are now under all grooming and uniform standards of the USMC such as hair cuts, weight, uniform apperanse and some navy personal this is Ok with some it isn't.