12

I remember I read somewhere that the US Nielsen rating was calculated by something like 25,000 households. That sounds extremely small compared to the number of people that actually have TVs.

What's more - these ratings are then broken into demographics.

Also, aside from the sample size, there is statistical bias. I would think that someone who actually wants to participate in one of these things would be different to just your average person. There's probably a good deal of measurement error, as you are reliant on people keeping log books etc.

Yet, these ratings are what TV channels use to auction off their advertising time and decide on which shows they choose to keep or cancel.

Are there any evidence supporting why TV ratings are a accurate mesasure of how popular or watched a show is?

Sklivvz
  • 78,578
  • 29
  • 321
  • 428
Samuelson
  • 3,781
  • 2
  • 29
  • 44
  • On one hand I don't want my cable company collecting my data and using it. On the other hand, With digital set top boxes, it's probably possible for them to determine exactly what every person is watching at every single moment of the day. You'd think they'd be able to build something into the digital cable and satellite boxes so that people could opt into having their viewing habits tracked. Might stop them from cancelling shows that a lot of people like. – Kibbee Dec 26 '12 at 18:26
  • @Kibbee - I may be wrong but I thought set top boxes just converted the tv signal into digital data that a digital tv could intrepret. Aside from that it's works the same way as normal tv in that the tv stations just send the signal out and don't know whose tuning in. I think with internet tv though, where everything is streamed to the user, viewing habits will be much easier to collect and analyze. – Samuelson Dec 31 '12 at 00:42

2 Answers2

8

According to Nielsen:

Electronic metering technology is at the heart of the Nielsen ratings process. Our tools capture not only what channel is being watched, but also who is watching and when, including “time-shifted” viewing.

Nielsen’s TV families represent a cross-section of representative homes throughout the U.S. Their viewing is measured by our TV meters and Local People Meters which capture information on what’s being viewed and when and, in the major U.S. markets, specifically who and how many are watching. Additionally, we collect more than two million paper diaries from across the country each year during “sweeps.”

The method that the asker calls into question is Nielsen's traditional diary based ratings calculations. (Although note that even the automated data collection is only permitted in households that have been invited by Nielsen). Put simply, the diary ratings work like this: A call center places calls to households in target market areas and ask for permission to send a "ratings diary". If the household agrees, they will be sent one diary for each working television in the household (up to five), and each diary will contain a cash (the amount varies) as an incentive to complete the diary. The notion that "only certain types of people" would take time to complete the diaries is strictly opinion, but it isn't a far stretch to think that perhaps more retired or unemployed people would find this arrangement ideal.

The diaries usually contain fourteen pages (although if the household has digital video recording hardware two additional pages are included, allowing for the previously mentioned 'time-shifted' viewing to be recorded). Each page can hold twelve hours worth of viewing.

I found this picture of an actual diary page by searching Google images: http://i88.photobucket.com/albums/k170/Bittybell/Scans/NielsenDiary-2.jpg

As you can see, the household is expected to provide the following information for every fifteen minutes during the survey period (of seven days):

I. Whether or not the tv is on

II. The call letters of the station, or the name

III. The channel number

IV. The name of the program

V. Who is viewing the program

The household can indicate an above response by drawing a vertical line under the information that they wish to "carry down" to lower grids.

While theoretically perfect in it's design, the diaries are rarely filled out with 100% accuracy. So how are the books interpreted? An office in Venice, FL collects all of the diaries via US mail, and specially trained temporary employees enter the data into computer terminals. If data on the diaries conflicts (basic example: what if the channel number does not match the supplied call letters?) a stringent set of guidelines is used to come up with the appropriate data, and sometimes the data cannot be used. The information entered into the terminals is double checked by separate employees who's only job is to verify. The Nielsen company claims that this results in over 99% accuracy.

One personal criticism of the system as it exists now is this: sometimes households misinterpret how to fill out the diaries. As simple as they seem, some people struggle with them. If the employee receives a filled-out diary with information that seems "sketchy" at best (for example: a user mistakes the "tv on" column for the "tv off" column and indicates that the tv is always on), the procedure is to enter the data regardless (in some occasions). The method is scientific (they can't prove the user was confused, what if they really DID mean to do this?), but I'm unconvinced they've really taken into account the margin of human error on the household's end.

The truth is that nobody can provide a definite answer in regards to whether or not the ratings are 100% accurate, the numbers are just too high. In my opinion they're as accurate as they're going to get until Nielsen adopts a strictly electronic form of television viewership monitoring.

Tyler
  • 81
  • 3
6

I work for one of Nielsen's competitors, doing TV ratings.

25,000 households. That sounds extremely small compared to the number of people that actually have TVs

At heart, this is a question about sampling. How can a small sample accurately reflect the entire population? And how accurate is "accurate"? Nobody is really interested in the precise number of viewers; they want a measurement that is accurate enough for practical purposes. So the sample can be much smaller than the total population, if the sample is large enough, and is representative of the whole population.

We (and I assume Nielsen too) do not arbitrarily or entirely randomly select people to be panellists. We collect demographic data about the panellists and have the corresponding data for the whole population, found using an "establishment survey". We recruit panellists so the demographics of the panellists reflect the demographics of the whole population. This can make recruitment difficult for some demographic groups. This is what BARB (the UK TV ratings organization) says:

Panel homes are selected via a multi-stage, stratified and un-clustered sample design so that the panel is representative of all television households across the whole of the UK. A range of individual and household characteristics are deployed as panel controls to ensure that the panel remains representative.

Once we have some measurements for the panel, we do not simply scale the numbers up to the whole population. First, some "weighting" is done. This compensates for (usually short term) deviations of the demographics of the panel from the demographics of the population. So, for example, if it transpired one day that, due to technical problems, the raw measurements came from too few HD-TV households, those panellists that were from HD-cable housholds would be given more weight.

BARB uses "rim-weighting":

Rim-weighting is applied to the panel data to incorporate numerous weighting and grossing variables

Oddthinking
  • 140,378
  • 46
  • 548
  • 638
Raedwald
  • 573
  • 4
  • 8