I suppose there are benefits to both leaving the quiver on and taking it off. When I first started bowhunting I always left it on. The primary benefit I saw to doing so was that it gave me more ready access to another arrow, but I was never is a position that it would have helped me. Also, I would be concerned that the deer would be alerted by the first shot, and, even though standing at 20-25 yards, might jump the string on a follow-up. Also, there have been times that I have held the bow in front of me and watched through the arrows on the quiver as the deer approached. I felt that it concealed my a bit more than a bare bow, especially movements of my eyes. But, this was likely more superstitious on my part than any real benefit.
Now I take it off and hang it on a hook in the tree. My reasons are weight and sound. Most of the time I am in the tree I hold my bow as there have been many times that deer appear out of nowhere and making the movement to remove the bow from a hook alerted them too much. So, the quiver is off to reduce weight during long sits. Second, With the quiver holding 3-4 additional arrows on the bow, I believe there is substantially more sound than with it removed.