This article: http://www.livestrong.com/article/240105-does-soy-milk-lower-testosterone/ seems to suggest that it is indecisive. Is there concrete evidence for either side of the argument?
Neither soy protein nor soy isoflavones had any effect on testosterone levels in adult men who consumed soy foods or products