Here are the factors I'd be looking at in making the determination:
1) Time of the beginning or end of legal hunting.
2) Method (bow, gun, etc)
3) Type of stand or blind
In the mornings, I've killed deer up to 11 AM. However, the vast majority have come in the second hour after the start of legal hunting. In the evening, The largest number have come in the last half-hour before the end of legal hunting.
Bow? Look I haven't been more than a few feet up a tree with a climber and had a nice buck come in. If you hunt off a climber, I would say that you need to add a half hour to whatever calculation you make. On the other hand, if you have a ladder stand or ground blind that you can slip into without a lot of sound. Gun? no penalty there.
If you're hunting with a gun and the deer are feeding 200 yards away, slipping in an hour before end of legal hunting is quite reasonable. It is not the best, but it will work. In the mornings, waiting until a half-hour after sunrise is not the worst thing you can do to your hunt.
Bow or primitive smokepole? You'll want extra time. You'll need an extra half-hour at least.
What's optimum? Again, it depends. I prefer to be up in my stand just at first light in my ladder stands. I can wait up to a half-hour extra for the ground blinds. In the afternoon, I go out between 2 and 3 hours before the end of legal hunting.
I recently finished compiling my deer log-- thirty years' worth. (there's a link on my weblog). One thing I found is that I've never bagged a deer between 11 and 3. That's true. It's due to a lot of things, but mostly it comes down to being a father with kids, and needing to deal with being a family man. A noontime hunt during the height of season is quite reasonable. Deer move all the time at midday. In fact, there are a couple of spots where you can almost be assured of a deer sighting at that time.