33

Various workshops given by Nielsen-Norman Group in the User Experience field, mention a specific experiment that involved Air Force cadets.

The experiment is meant to illustrate the point that there's no such thing as an "average user", and it goes like this:

680 Air Force cadets were lined up in a field. All were roughly the same age and within narrow height and weight limits. Then an officer called out the average size of uniform items - first underwear, then pants, shirts, etc. Everyone not wearing the average size for that particular item were dismissed and left. By the fifth item, only two cadets remained. By item six, one was left. By item seven, all were gone.

This means that out of the 680 cadets there wasn't a single one who was the average size in all uniform items.

I attended a workshop where this was claimed, and there's also a presentation online from a different workshop of theirs, where this claim can be found (slide 57 here).

However I couldn't find any other mentions of this experiment online. I wonder whether it can be just an urban legend.

Sklivvz
  • 78,578
  • 29
  • 321
  • 428
Vitaly Mijiritsky
  • 1,073
  • 10
  • 14
  • 10
    I'd just like to add that this story shows absolutely nothing about User Experience. It's a ridiculous diversion to make you more susceptible to bullshit stories they sell you in workshops. – Davor Jun 03 '15 at 12:29
  • 2
    Yeah, I don't see what point this story would have in any context. Cadets are already a skewed sample because you have to be within a certain size range to be one. Further, once you pile on 7 or 8 different things, the number of combinations is *massive*. It's not really that significant that in only 680 people no one fit any given set completely. –  Jun 03 '15 at 16:52
  • 3
    @Davor It's a nice anecdote that does drive home the claim that when designing for the average user you're optimizing for noone, so that instead of having at least some users who get an optimized experience, nobody gets it. – Vitaly Mijiritsky Jun 03 '15 at 16:56
  • 2
    @VitalyMijiritsky - which is, of course, completely false in the case of software, because user experience is not made in discrete sizes like shoes. – Davor Jun 03 '15 at 19:14
  • 3
    @Davor Not as false as you may think. Users' needs, use cases, workflows and tasks can be pretty discrete and sometimes mutually exclusive, especially in enterprise settings. But I don't think that this is the place for discussing UX, you're welcome over at UX.SE if it interests you. – Vitaly Mijiritsky Jun 03 '15 at 19:21
  • @fredsbend not entirely true: a "cadet" is simply a member of a military academy (i.e. future officer) and is held to military standards for weight and fitness, but not height. It _would_ be true to state that _pilots_ and certain other career fields do require minimum and maximum sizes in order to do the job safely. Source: I am a USAF veteran. –  Jun 04 '15 at 17:25
  • @Snowman During WWII, 680 aviation cadets training at Kelly Field, Texas were measured in numerous body dimensions. These cadets were subject to age, height and weight limits: 18-27 years, 120-200 lbs. and 163-193 cm. http://storage.lib.uchicago.edu/pres/2006/pres2006-1545.pdf – DavePhD Jun 06 '15 at 12:53

1 Answers1

30

This story is sourced to Tyler Blake, professor CSU Northridge.

The story is printed in Advances in Human-computer Interaction (1995) at page 94, and the reference for the story is given as

Blake, T. (1985). Introduction to Principles and Techniques for Interface Design Tutorial Notes for CHI'85 tutorial.

This corresponds to a tutorial by Tyler Blake from 9AM - 12:30 PM Monday 15 April 1985 as part of the Conference on Human Factors in Computing Systems, Hyatt Regency at Embarcadero Center, San Francisco. Source.

see Introduction to Principles and Techniques for Interface Design

The story is also in TOG on Interface (1992)

The older version of the story is somewhat different from the one in OP.

The original version does not include "All were roughly the same age and within narrow height and weight limits" nor "By item seven all were gone". Also, the earlier version says "shoes, pants, and shirts" instead of the "underwear" mentioned in the OP.

It makes clear that anyone not within one standard deviation of average was excluded each round.

Specifically, the original (or at least 1992) version of the story is:

Several years ago, the Air Force carried out a little test to find out how many cadets could fit into what were statistically the average-sized clothes. They assembled 680 cadets in a courtyard and slowly called off the average sizes - plus or minus one standard deviation - of various items such as shoes, pants and shirts. Any cadet that was not in the average range for a given item was asked to leave the courtyard. By the time they finished with the fifth item, there were only 2 cadets left; by the sixth, all but one had been eliminated.


Comparing the story to statistical expectations for normal distributions of independent variables:

68.2 % are within one standard deviation.

0.682^5 = 0.148 (101 / 680)

0.682^6 = 0.101 (69 / 680)

0.682^7 = 0.069 (47 / 680)

So even after 7 round, one would expect 47 cadets to be remaining, and even more given sizes of various clothing items are correlated. The story doesn't seem credible.

On the other hand, there is a study involving 680 cadets discussed in the 1955 article "Physique and success in military flying" American Journal of Physical Anthropology, vol. 13, pages 217-52. The story could have originated from actual measured data, but morphed over time.

Particularly, it seems to be a dramatization of The "Average Man"? (1952) by Gilbert S. Daniels.

The fallacy of the "average man" concept is further illustrated by a study based on body measurements made on over 4,000 Air Force flying personnel. From a total of 131 available measurements a smaller group, all useful in clothing design was selected.

  1. of the original 4063 men, 1055 were of approximately average stature

  2. of the original 1055 men, 302 were of approximately average chest circumference

  3. of the original 302 men, 143 were of approximately average sleeve length

  4. of the original 143 men, 73 were of approximately average crotch height

  5. of the original 73 men, 28 were of approximately average torso circumference

  6. of the original 28 men, 12 were of approximately average hip circumference

  7. of the original 12 men, 6 were of approximately average neck circumference

  8. of the original 6 men, 3 were of approximately average waist circumference

  9. of the original 3 men, 2 were of approximately average thigh circumference

  10. of the original 2 men, 0 were of approximately average crotch length

The article details what was considered average, basically the middle 25-30%.

DavePhD
  • 103,432
  • 24
  • 436
  • 464
  • 6
    That statistical analysis doesn't seem particularly meaningful to me. Aside from correlation, assuming 68.2% of men (even of similar age and build) wear the same size shoe doesn't seem like it comes anywhere near reality. – femtoRgon Jun 02 '15 at 18:23
  • 11
    @femtoRgon maybe my answer isn't clear enough that the original version of the story explicitly specifies that average size "plus or minus one standard deviation" was the selection criterion for each round. Only persons outside this plus or minus one standard deviation range were excluded each round. – DavePhD Jun 02 '15 at 18:31
  • 2
    You're right, I did miss that. Thanks you for clarifying. – femtoRgon Jun 02 '15 at 18:49
  • "average sizes - plus or minus one standard deviation" doesn't make much sense, as it's too wide of a range. For a shoe size it probably would be like 9-13. Unless they're talking about standard deviation within population of the cadets, rather then general population of US. – vartec Jun 02 '15 at 22:09
  • 2
    Your calculations assume independence of the various attributes (pants size, shirt size, etc), yet these tend to be correlated (they're strongly related to height and BMI for example). So you can't just take a product of probabilities and expect to get very close to reasonable approximation of their joint probability. – Glen_b Jun 03 '15 at 02:33
  • 5
    @Glen_b yes, my answer already says "even more given sizes of various clothing items are correlated". The calculation is meant only to show many more than two people would be left after round 6, and even round 7, statistically even without considering correlation. – DavePhD Jun 03 '15 at 02:36
  • 1
    Okay. The real answer may be a lot nearer 68% than one tenth of that. – Glen_b Jun 03 '15 at 02:42
  • 1
    This doesn't seem to answer the original question: did the experiment in fact occur (regardless of the plausibility of the results of the experiment)? – Mario Carneiro Jun 03 '15 at 06:43
  • If the sizes aren't actually normally distributed, then your statistical expectations don't hold. – kutschkem Jun 03 '15 at 07:24
  • The statistics indeed don't need to be so simple. Perhaps the airforce searches for cadets with a certain length, in which case leg length and upper body length may show significant negative correlation within the sample. – Dennis Jaheruddin Jun 03 '15 at 08:48
  • @DennisJaheruddin The US Air Force has no force-wide height requirements. To be a pilot, you must be 64 to 77 inches. – DavePhD Jun 03 '15 at 11:28
  • @MarioCarneiro You are correct. In the question Vitaly said he couldn't find any other references to the story. I traced the story to printed sources and a particular person. I did NOT completely answer the question. – DavePhD Jun 03 '15 at 11:34
  • @DavePhD Wonder if someone feels like emailing Dr. Blake to find out where he heard this story. – Mario Carneiro Jun 04 '15 at 05:40
  • @MarioCarneiro That's what my answer original said for the person asking the question to do if they wanted a complete answer, but I deleted that part because I was worried too many people would email. – DavePhD Jun 04 '15 at 10:29
  • Revisions #11 and #12 have confused the answer. The 1995 and 1992 versions no longer seem relevant. Even the 1955 version seems irrelevant. Maybe the 1952 version should be at the top (rather than below the fold), and some of the others should be culled. – Oddthinking Jun 06 '15 at 01:02
  • @Oddthinking those edits complicate the answer, but add relevant information. The 1992 version is very close to the OP. The 1995 document merely quotes the 1992 document but provides better full text access. The 1952 report seems to be the source of the concept, but it doesn't have the number "680" or the word "cadets". 1955 has "680 airforce cadets" and says "7 tailor's measurements", but I don't have full text access. The story could have come from applying the 1952 paper's idea to the 1955 paper's data set, plus dramatizing. I'll try to find a way to improve. – DavePhD Jun 06 '15 at 01:52
  • @Oddthinking This is the full text of the thesis with the 680 cadet data set. http://storage.lib.uchicago.edu/pres/2006/pres2006-1545.pdf Same title and author as the 1955 paper. Numerous body measurements were made on 680 AAF (US Army Air Force) cadets in 1942. – DavePhD Jun 06 '15 at 12:28
  • @DavePHD you did an amazing job with this, thanks so much! – Vitaly Mijiritsky Jun 06 '15 at 12:57
  • @DavePhD: I'm not really arguing against any of the evidence. I think the *structure* of the answer now needs a clean up. It starts by saying it is all based on one line of evidence. It diverges into a bit of original research (which I figured was fine, in that it added a bit of flavour to the substance of the argument, but it was at the bottom, after the fold, after the real answer that does not at all depend on it). But now it veers back after that to argue the claim is based on a different line of evidence. – Oddthinking Jun 06 '15 at 13:39