Maybe the loose nut on the grip helps?

I was shooting some groups this morning with the .41 Magnum sixgun, testing loads with Bullseye. It was doing fair. Then .. using my old standard load that I've used for 25 years, I shot these two groups in row. I was using a rest ... I did not shoot these offhand.

The Black spot was my POA.

After I shot the last group I noticed the cylinder pin retaining screw was loose. Checking the gun I found all the gripframe/trigger guard screws were loose. Now I'm wondering if I tighten 'em up, am I gonna screw up the accuracy? Or will it maybe move that flyer to the right just a big?