There are several reasons:
In my experience slug gun ammo seems to be particularly susceptible to variations in manufacture. I'm not sure why that is, but I have consistently heard complaints over the years. As a reloader of centerfire rifle, I can tell you that there are differences from one lot of powder to the next, not as much as I was warned by the old timers, but it's there. Coming from somewhat of a manufacturing background, I can tell you that small changes in setup from one batch of product to the next can have dramatic effects. The slug load is also a fairly complex product, more so than centerfire rifle ammo. Sabots make it even more complex. So I can understand the variability.
The shotgun itself changes. Most people have a setup where the barrels are removeable and interchangeable, and this will change the point of aim. If my Model 70 rifle was built like my Model 870 shotgun, I'd be fighting to stay on a pie-plate at 50 yards. Some die-hard slug hunters spot weld their barrels to the receiver to take care of this. Most guys put up with it and don't expect all that much out of their shotgun. You have to remember that this is a system that is built to primarily spray shot at a fleeing or flying bird, not deliver a precision shot at a great distance. The guys who go for maximum accuracy steal a cue from the bench rest shooters and use a dedicated bolt action gun to minimize these problems.
There's also the problem of sights. I started with an 1100 with a smooth bore and rifle sights. Year to year, that barrel has stayed true in part because the barrel and the sights are all hooked to each other. My Mossberg has a scope mounted on the receiver. Anything that changes the alignment of the scope to the barrel (like cleaning) is going to change the point of aim. I look at all the convoluted ways people try to stick scopes on shotguns, and I'm surprised any of them work.
Lastly the whole thing is predicated on somewhat of a flimsy technology. I reload. I spend a lot of time trying to make brass and bullet just so. Then I pick up a slug load and I have to kind of chuckle. Shotgun and rifle used to be the same technology. You used to be able to cram pebbles or whatever down the barrel, and if you wanted accuracy, you used a piece of pillow ticking to jam it in good, so your ball wouldn't rattle down the barrel. Modern centerfire rifles show every bit of the last 200 years of innovation. Slug loads coming out of an average shotgun show how far we've come.
If it were me, I would have all the states like Ohio that mandate shotguns take second looks at states like Indiana and Kentucky. In the former, the state has now allowed some centerfire pistol cartridges out of rifle-length barrels. In the latter, everything centerfire goes. I've hunted all three. I seriously doubt Ohio would suffer greater casualties if it adopted Indiana's model, and a really doubt that Kentucky's model would create any greater carnage either. The problem in Ohio remains the same as Kentucky in that the vast majority of casualties do not come from stray rounds traveling great distances. It comes from hunters not identifying their target properly and shooting another human and extremely close range. That's still something that 12 GA slug does quite well.