29

To quote from a comment on my recent answer (emphasis mine):

Your quoting of Kinsey's bogus studies on homosexuality -- criticized heavily by famed statistician John Tukey -- makes me severely doubt the rest of your answer.

Leaving aside the point that validity of Kinsey's studies has very little logical effect on validity of the answer being commented on, it was the first time I heard something negative about Kinsey's studies (mostly since I never cared enough about them to bother checking into them).

Thus, a skeptical me asks:

Is there any subsequent study/report which would warrant labeling as "bogus"/"debunked" either specific major parts (especially, due to the impetus of the question, the parts related to sexual orientation), or the entire set of Kinsey's studies/reports (as far as methodology or any other criticism).


P.S. For those who don't understand the context, Dr Alfred Kinsey was a mid-20th-century american researcher who studied human sexuality based on extensive questioning of people, and who published what was at some point considered the most important books on the subject ever written (sometimes referred to "Kinsey Reports") - "Sexual Behavior in the Human Male" and "Sexual Behavior in the Human Female".

user5341
  • 31,075
  • 8
  • 130
  • 178
  • 5
    If you want to throw in an illustration, [may I suggest this?](http://homepage.mac.com/fidelitarean/iblog/C889114474/E20060607093600/Media/kinseyfrontsmall.jpg) – Scott Hamilton Apr 15 '11 at 18:07
  • 1
    To decry the Kinsey reports as “bogus” – even in the light of criticisms levelled against it – certainly doesn’t speak in favour of the comment. – Konrad Rudolph Apr 16 '11 at 14:45
  • That you didn't have any other citations in your answer probably didn't help. – Andrew Grimm Apr 22 '11 at 15:41
  • 3
    The people who were interviewed by Kinsey were self selected at a time when there was social pressure against talking about sex. To what extent that makes his results biased, and in which way is unknown. – sal Aug 03 '11 at 02:11

2 Answers2

23

In 1979, Paul Gebhard and Alan Johnson of the Kinsey Institute at Indiana University spent years "cleaning" the data of suspected contaminants (some of which are mentioned in dan04's answer). Their results were published in The Kinsey Data: Marginal Tabulations of the 1938-1963 Interviews Conducted by the Institute for Sex Research (1979), portions of which are available online via Google Books.

This excerpt lists some of the individuals excluded from the re-evaluation of Kinsey's data (the groups mentioned are the sample divided by college education and race):

All of these groups were cleaned in the sense of removing individuals who had derived from sources with known sexual bias. By known sexual bias we mean a group which we know to be substantially biased in some sexual way before we began interviewing its members. Examples include the Mattachine Society (a homosexual orginization), the occupants of homes for unwed mothers, prostitutes employed by a famous madam, personal friends of individuals known to be sexually deviant, and patients in mental hospitals. Also, all individuals who had been convicted of any offense other than a traffic violation were excluded since we now know that such individuals (as a group) differ in terms of sexual attitides and behavior from persons who have never been convicted.

However, the changes only led to a handful of differences. The authors attributed many to the improved statistical methods they were able to employ in the reevaluation, but some to the "cleaning" of Kinsey's data:

In another example, the effects of the cleaning of the sample were evident. The incidence figures of homosexual activity for single and married college-educated males, Table 90 in the Male volume, are quite similar to those of this volume, but our current noncollege sample has much lower incidences than Kinsey's grade-school and high-school-educated samples. A major cause of this discrepancy is clear; Kinsey's noncollege samples included persons who had been incarcerated in jails and prisons (where homosexual activity is relatively common) whereas our present sample excluded them.

Despite these differences, the authors maintained that no significant conclusions produced by the earlier Kinsey studies were debunked:

Despite the flaws of our earlier pioneering publications and the difficulties of comparing them with this volume, it is clear that the major findings of the earlier works regarding age, gender, marital status, and socioeconomic class remain intact. Adding to and cleaning our samples has markedly increased their value, but has not as yet caused us to recant any important assertion. In using our new Ns in analyses, we anticipate we will discover relationships previously unknown to us and we will undoubtedly have to modify some prior statements, but we feel the important contributions of Dr. Kinsey will stand.

As an example, the original Kinsey study indicated that 37% of males has had at least one homosexual experience to orgasm. The re-evaluation placed that figure at that 36.4%. The original study only found that 4% of the white male population to be completely homosexual (a 5 or 6 on the infamous Kinsey scale), while the re-evaluation placed that figure at 9.9% for the college-educated group and 12.7% for those with lesser education, astounding considering opponents expected that latter figure to be much smaller after omitting convicts.

Most resources criticizing Kinsey's work fail to take this re-evaluation into account at all, so it's hard to ascertain whether they missed anything or not. All of the flaws in Kinsey's original work seem to be accounted for. Furthermore, no study of this magnitude has been conducted since. Kinsey's study is by no means perfect, but neither has it been conclusively debunked.

Patches
  • 8,162
  • 8
  • 44
  • 58
  • @TRiG: Thanks a lot! I fixed a few typos and removed the original images. – Patches Aug 03 '11 at 05:11
  • 1
    + I just finished reading a book on the Bayesian vs. Frequentist controversy in statistics, and it says the Kinsey report fell into that. There were very smart people on both sides. Long and short is that frequentist is more defensible but needs too much data for practical problems. Bayesian is more nimble and useful in situations with minimal data and high uncertainty. – Mike Dunlavey Jul 11 '13 at 13:47
  • 2
    And note that since the problem with Kinsey's work is sampling bias all that it calls into account is the percentages he reported, not anything else in his work. – Loren Pechtel Jul 12 '15 at 06:09
1

From http://www.boxturtlebulletin.com/Articles/000,024.htm

But not all criticisms of Kinsey’s research were unwarranted. Writing for the Journal of the American Statistical Association in 1949, W. Allan Wallis noted several serious methodological deficiencies in Kinsey’s methods, which led to his findings not being statistically representative of the U.S. population.14 While it was largest survey of sexual behavior ever undertaken, it was hobbled by the fact that Kinsey didn’t trust probability sampling methods which were in their infancy in those days. (Remember, 1948 was also the year of the famous “Dewey Defeats Truman” headline.) According to Drs. Paul Gebhard and Allen Johnson of Kinsey’s Institute of Sex Research:

To Kinsey, the solution to sampling problems was to increase the size and diversity of the sample, and he felt that all biases and other problems would average out and nullify one another as the sample grew and proliferated. 15

...

One look at the demographics of Kinsey’s sample clearly showed its biases. Kinsey’s sample favored professionals and those under the age of 35. Geographically, Kinsey’s sample was heavily concentrated on the Northeast and Midwest, especially in the state of Indiana where the bulk of Kinsey’s surveys took place.17

Kinsey’s homosexual sample provides another illustration of the problems in Kinsey’s dataset. Because of the difficulty in finding homosexuals in the repressive atmosphere the time, Dr. Kinsey relied on interviews with members of homophile groups such as the Mattachine Society and homosexual communities in a few large cities. He also interviewed prisoners and institutional populations...

dan04
  • 726
  • 5
  • 8
  • 3
    The criticism seems to amount to: Kinsey used the best approach to methodology available to him, and we'll knock him because that wasn't perfect: "didn't trust .. methods that were in their infancy" which naturally led to biases that we would not now tolerate, "problems in Kinsey's dataset [due to] the difficulty in finding homosexuals in the repressive atmosphere the time". – Charles Stewart Apr 19 '11 at 13:44
  • 3
    @Charles: Even if he wasn't deliberately engaging in scientific misconduct, it'd be grounds to ask for more rigorously collected data. – Andrew Grimm Apr 21 '11 at 13:12
  • 2
    @Andrew: I think it's grounds to say that the standards of rigour have risen. But "ask for more rigorously collected data" - from who? I don't know of surveys of comparable ambition. It appears that you can't get public funding in the US for such work anymore. – Charles Stewart Apr 21 '11 at 14:22
  • @Charles: Purveyors of pseudoscience also claim that they can't get public funding for their "research". – Andrew Grimm Apr 21 '11 at 14:31
  • @Andrew: Large-scale surveys of sexual behaviour are pseudoscientific? – Charles Stewart Apr 22 '11 at 13:39
  • @Charles: Human sexuality isn't a pseudoscience, but it's possible for people to research is pseudoscientifically. – Andrew Grimm Apr 22 '11 at 13:55
  • @Andrew: I don't understand what you meant by your response if you weren't suggesting that the unwillingness to fund Kinsey-scale research in the US was because the proposed investigations were pseudoscientific. Are you saying, in particular, that Kinsey conducted pseduoscience? – Charles Stewart Apr 22 '11 at 14:27
  • @Charles: I'm saying that apologists for pseudoscience often say "the reason we don't have rigorous research supporting our claims is because we can't get public funding". It's usually nothing more than a bogus excuse. – Andrew Grimm Apr 22 '11 at 15:18
  • @Andrew: This observation doesn't seem very cogent to the comment I made. If you were going somewhere with your observation, perhaps you would like to expand on your remark? – Charles Stewart Apr 22 '11 at 15:24
  • @Charles: Let me spell it out further: I don't know if you're an advocate for pseudoscience, but you're acting like one. I don't know for sure whether or not Kinsey's research is totally bogus pseudoscience or just imperfectly done legitimate research, or somewhere in between, but if most of his supporters are like you, then I'd be more likely to suspect that it's totally bogus pseudoscience. – Andrew Grimm Apr 22 '11 at 15:41
  • 3
    @Andrew: I think Kinsey's research had good methodology for the day, but would be very problematic today, precisely because the relevant statistical methodology has advanced so far. My impression is that we would be in a much better position to say if the methodological problems had led to serious flaws in the conclusions if the US NIH would fund large-scale studies into sexual behaviour; without that, the empirical basis is too patchy. I do think that Kinsey did a good job given his constraints. With respect to the insult, would you care to visit chat? – Charles Stewart Apr 22 '11 at 16:12
  • @Andrew: http://chat.stackexchange.com/transcript/message/871228#871228 – Charles Stewart Apr 22 '11 at 16:25
  • @Charles: I'm in the room. Are you? – Andrew Grimm Apr 26 '11 at 06:32
  • 1
    @dan: your answer appears to consist entirely of a small quote from a rather large article. I'm not at all clear if you even *expect* this to answer the question, much less whether it actually does. Rather than relying entirely on someone else's words, it would help if you could take the time to relate what you've linked to / quoted to the question that was asked here. @Andrew, @Charles, I've cleared both of your flags, and if this answer is improved may end up deleting the rest of the comments - please continue this discussion in chat. – Shog9 Apr 26 '11 at 22:34
  • -1 until you add reliable sources and write some contexte. – Zenon May 18 '11 at 16:29
  • This answer needs to reference the other answer - do the same problems apply to the "cleaned" dataset, e.g. geographical imbalances? Are there any other potential confounding factors in the cleaned data set? – John Lyon Aug 03 '11 at 01:19
  • +1 For bringing up a point regarding Kinsey's research, but reluctantly. This answer needs to be expanded beyond a straight quote, and ideally give some context per Charles' points. But I'm glad that I'm aware that Kinsey had trouble getting what we would consider now to be good survey data. – Dave Jun 18 '12 at 20:54