According to The terrible truth about Gaddafi’s harem:
Yet Soraya’s story is typical. She was just one of thousands of young Libyan girls and women who were kidnapped from their schools, homes or places of work and forced to be Gaddafi’s sex slaves.
Nor did the Libyan leader restrict his attentions to women. He also took delight in sexually abusing young male guards in front of his “harem”.
Fuelled by cocaine, alcohol, cigarettes and Viagra, Gaddafi used sex not only as a physical weapon, but as a political tool through which he could exert his power.
Rape subjugates women - and at the same time subjugates the men who are close to them, such as their husbands and fathers.
Gaddafi was all too aware of this. The wives and daughters of senior figures were blackmailed, bribed, cajoled and forced into having sex.
Most of the stories appear to say that he had a harem and it was quite big. But I am yet to find any evidence of it being as big as having 1000s of women and forcing the wives of senior figures to having sex (rape). Another report from clarionproject says:
"He would visit his victims. It was as though they were some sort of macabre souvenirs,” said Baha Kikhia, whose husband, a foreign minister was killed by Gaddafi and whose body was one of those found a freezer after Gaddafi’s demise.
This doesn't mention any enormous harem.
Is there any evidence that Gaddafi forced the wives and daughters of senior figures into having sex and did he have a harem of thousands of women?