The best exercise for dragging a dear is, obviously, dragging a deer out of the woods, but that does not really solve the problem of preparing you for the effort. (Nice deer, by the way msbadger. ) As you mentioned all around outdoor activity on uneven ground will not only do the trick, but it can be a lot more fun than going to a gym. If you are inactive most of the year, then expect to get into deer dragging shape in a few weeks, you might be better off buying an ATV.
I might mention that I did buy an ATV. Except for the harder months of winter I spend a lot of time "fussing and doing things" at our farm that keeps me busy the rest of the year. I had a total knee replacement in 05 and at that time I bought a workout machine and keep it in my mancave with the rest of my toys. I will admit that for a while it lie dormant, but for a long time now I've made good use of it. After all, it is less than 20 steps from my couch.
I think that is the key, along with self motivation. If I had to get myself presentable to go out in public and then drive to a gym, I would find it hard to do with any consistency.
Keep life simple...if you hear hoofbeats, think horses, not zebras.