First off, how does your shoulder feel after that many shots?
It might be you? I was shooting a 4 inch group at 100 yards today with a smooth bore. I have shot my gun a whole lot, and am very familiar with the gun, as it is the gun I prefer and have used my whole hunting career. Shotguns kick a ton, more than most, if not all rifles I am aware of (especially 12ga. and up). I've had people who shoot .300's and haven't even spent time trying to adjust their sights on their shotgun, because it hurts too much.
So to answer your question, no, I have not had any experience with anything like that at all. I would be guessing that you might be pulling off, especially when shooting that many rounds through a slug gun. Do you go to a range where the people there have tools to adjust you sights? Also, are you using a scope, or open sights? Firstly I would say if you are using a scope, to be sure that everything on there is screwed down tight. The kick alone could simply knock the scope around a little bit. Another thing too that I would recommend, is if you are using an open sight, how to aim it. I'm not trying to be condescending or anything like that, but a quarter of an inch, or less than that even can put the gun off quite a bit. I use open sights, and put the front bead at the base of the back "v" where I can see the whole front sight, and the bottom of the front bead is at base of the "v". Hard to explain, but I hope I did a good enough job. Also, like bow hunting, I like to have my neck/cheek area in the same place on the gun every shot, for consistency. Make sure you are doing some of the things that are often overlooked and give it another shot. In addition, I know you said it was a 20 gauge, but make sure that you don't over shoot. It'll make you jumpy, pull off the target and fear the kick.
Let me know some of the things you have on your gun besides the bullet and the barrel. Such as the sights, where you put your "anchor" point, and how you aim, if you have an open sight shotgun. Then I'll be able to help you out a little bit. I don't think it is the slugs, I could be completely wrong, but I did a google search and nothing came up directly linked to those bullets. I'm sure it's not the 870.