The answer like many in LR shooting is "it depends." Accuracy comes from consistency shot to shot. Fouling is not bad provided it provides a consistent interface shot to shot, and it does not harm the bullet. I typically shoot my LR rifles thousands of rounds between detailed cleanings because the bores foul in such a way that does not affect accuracy.
Some barrels will collect copper fouling in such a way that eventually accuracy will be harmed.
It is also common that after cleaning "down to bare metal", it may take 5-20 rounds fired before the barrel "settles in" to its original accuracy again.