.This is one area where I have done a whole lot of research. I've tried a lot of options. Do I hide my face? Sometimes.
The whole point of the whole camo thing is to disrupt the head-body-arms-legs gestalt that the deer forms when it recognizes us. When I am in normal street clothes I've seen deer recognize me at 600 yards. When I'm up a tree bow hunting or out turkey hunting in the spring, I've had deer oblivious to me at under 5 yards. They could not figure out I was human.
One of the big clues that was missing was my head. I started out hunting with just a hat. Over the next thirty years, I have tried masks, hoods, facepaint, -- the works. What has worked the best is one of the following:
1) Where I cannot hide the rest of my body, I use a hood or a hat that hides everything. For deer, my favorite is a thing I made out of an old camo t-shirt. I took the shirt, and cut off both arms. I sewed one hole up completely and peek out of the other. I put this over a warm hat. It's floppy, it breaks up my form. It isn't hot. The hat I wear has a brim, so it all sticks out a bit. This has been my best answer to hiding my head from the deer.
2) When I'm up in one of my buddy stands, behind a camo skirt where nobody on the ground can see past mid-torso, I relax a bit. When it's cold and I'm hunting with a rifle, I usually start out with a green wool military balaclava under my orange hat. It keeps me warm. Yes, my face does stick out a bit but the rest of me is concealed.
3) When I'm turkey hunting, I wear camo netting attached to a boonie hat like a veil.
In most instances, what I'm wearing on my head hides not only the brightness and color of my face, but it also distorts the shape of my head. With the balaclava, it is a dark contrast to the bright orange coat and hat. All that's showing is my eyes and nose.
Just the other weekend, I had a doe come in to me She was on to me from the beginning. She'd seen me move from a long way off. I was just wearing a hat. She came in to investigate. I was up in my stand, with a camo skirt attached to the shooting rail. All she could see was my head and barrel of my gun. After a long time circling me and stomping, she wandered off. What threw her off was that just as she was getting ready to bolt, I started chattering at her like a squirrel. That has been a sure-fire trick I've been using for my whole hunting career.
What she was keying on the whole time was that I'd taken my balaclava off, because it was getting too warm. I just had my orange hat and my orange parka on. I looked kind of like this:

The point here is that whether it's the camo skirt on the shooting rail, the camo hood made from the T-shirt, or whatever, the goal is to keep deer from figuring out what they're looking at is a human form.