Having hunted a lot of crowded public places in the past, I can tell you that it helps tremendously if you can see a hunter on stand or see the blind. If there is an orange hat across the way, you'll be able to plan your shots to avoid hitting hitting your fellow hunter. I happen to be the survivor of one close encounter with a slug when a herd of deer came up a shallow creek bottom and ran between my stand and that of the neighbors'. I would hope that they would have held their shots if they had seen the orange.
It also helps in spacing out hunters. If I can see a hunter off in the distance, I'm not going to walk up and plunk myself down right by his stand and try to hunt. By wearing orange, you're allowing other hunters to make a choice as to whether they are going to encroach on you or not. Otherwise, they have no way of knowing you are there; if you get disturbed, it is your own fault.
I will stand by what I said: it is sheer perversity not to be displaying hunter orange during a firearms season.

