In The Evolution of Sex Difference in Language, Sexuality, and Visual-Spatial Skills (PDF), R. Joseph, Ph.D. explores the differences between males and females in a variety of primates, with emphasis on humans. Some selected quotes from the paper as it appeared in Archives of Sexual Behavior, Vol. 29, No. 1, 2000 are provided that indicate that females have better language skills than males and males tend to have better visual-spatial skills than females. This difference is also seen in nonhuman primates as well as other animals but appears to be more pronounced in humans.
For example, female primates produce more social and emotional vocalizations and engage in more tool use and gathering activities, whereas males tend to hunt and kill. Similar labor divisions are evident over the course of human evolution. “Woman’s work” such as child rearing, gathering, and domestic tool construction and manipulation contributed to the functional evolution of Broca’s speech area and the angular gyrus—which injects temporal sequences and complex concepts into the stream of language and thought. These activities gave rise, therefore, to a female superiority in grammatical (temporal sequential) vocabulary-rich language. Hunting as a way of life does not require speech but requires excellent visual–spatial skills and, thus, contributed to a male visual–spatial superiority and sex difference in the brain.
...
...Conversely, it is well established that human males excel over females across a variety of visual–spatial problem-solving and perceptual tasks. Visual–spatial superiorities, however, are also demonstrated by other species including male rats.
It is apparent that these same cross-species sex differences have become more pronounced in humans. However, rather than purely a product of societal, political, or parental pressures, the amplification of these sex differences, like other cognitive capabilities such as math, are also neurologically based and a product of our evolutionary heritage.
...
Hence, given 500,000 years of multigenerational male experience in hunting, which usually required days or even weeks of wandering hundreds of miles from the home base, present-day males therefore demonstrate superior visual–spatial skills including superior maze learning, tracking, aiming, and related nonverbal abilities, compared to females. This includes a male superiority in the recall of geometric shapes, detecting figures that are hidden and embedded within a complex array, constructing three-dimensional figures from two-dimensional patterns, visually rotating and detecting the number of objects in a three-dimensional array, and playing and winning at chess (which requires superior spatial abilities).
Not quoted, but presented in the paper, is a discussion on the physical differences in male and female brains. Language ability in male brains appears to be restricted mostly to the left hemisphere but is more developed in the right hemisphere in female brains. The author seems to imply that this allows males to use more of the right hemisphere for the visual-spatial skills that are required for hunting.
Bottom-line: Yes, men tend to have better visual-spatial skills than women.