4

I'm reading the news and came across a story regarding Donald Trump's approval ratings and I'm unclear on who the survey respondents were. I have no basis to see which groups of people were represented by the approval ratings polls, and it appears to me that the approval ratings polls across a number of distinct attempts vary wildly.

This leads me to believe that some of these polls have some sort of institutional or confirmation bias that skews their numbers. I'm seeing on RealClearPolitics anywhere from a -15 point spread to a +4, numbers which should be well outside the margin of error for these (which seem to be between 1-5 points). Unless the ratings are such a roller coaster (which unless there's a scandal or other notable event I shouldn't see right?), am I wrong in assuming they shouldn't fluctuate as much as they are, and leads me to the conclusion above.

Is there any bias in the approval ratings reports that would cause such wild variations between polls or can these changes be explained statistically?

Edit for notability: Can You Trust Trump’s Approval Rating Polls?

In the article:

In the meantime, with polls in the news again, I’d urge my journalistic colleagues to do a better job of reporting on uncertainty when they report on polling data. Not only do polls have a margin of sampling error — for instance, the margin of sampling error on CNN’s poll of 1,000 adults is plus or minus 3 percentage points — but they also have other types of errors, such as nonresponse bias. The people who respond to polls — often under 10 percent of the population contacted — may not be representative of the population as a whole, and that creates a lot of challenges.

Sklivvz
  • 78,578
  • 29
  • 321
  • 428
Anoplexian
  • 557
  • 5
  • 15
  • Can you add a source that claims that the approval ratings may be biased? I'm voting to close for now, as this seems to be your personal opinion, but will retract if you add a source. – tim Feb 08 '17 at 19:14
  • Approval rating polls are just about as consistent as voting polls before the elections. They are notoriously imprecise. I generally ignore them. Case in point: polls showed that Clinton was supposed to win by four or five points over Trump. –  Feb 08 '17 at 19:15
  • 15
    Clinton did win by 3 points, poll margin of errors are typically around three points, so the election polls were not incorrect. Election was decided by some very close states that polled close, or didn't have very much state-specific polling. – antlersoft Feb 08 '17 at 19:46
  • @tim Added, please see the edit made to show a 528 article, written by founder Nate Silver. – Anoplexian Feb 08 '17 at 20:08
  • 1
    @Anoplexian Your quote seems rather selective. The claim of the article is that there is a margin of error of about 4 %points (8 %points for net approval rating / spread). I'll retract my vote, but I don't think that this is a good question for this site. – tim Feb 08 '17 at 20:20
  • 2
    @tim It is, but it shows that there is a possible bias, and Skeptics.SE is amazing at validating statistics. I'm not worried about the specifics, just whether or not a bias exists for these polls one way or another, which would not be a great question for Politics.SE. – Anoplexian Feb 08 '17 at 20:24
  • 1
    It's pretty well-known that all polls have issues of this kind. Are you asking if these issues apply to Trump Approval Rating polls, or are you asking if there is some extra factor at work here? – DJClayworth Feb 09 '17 at 00:43
  • 1
    The problem with poll biases is that some of them are hard to impossible to detect other than by testing what they predict (e.g. actual election). There could be bias - intended or even accidental - in the wording. There could be difficult to predict biases in response rates (at one point, cell phone owners without landlines were being severely under-counted). There are biases when polls are on a sensitive subject ([social desirability bias](http://politics.stackexchange.com/questions/12521/phenomenon-where-politically-incorrect-candidates-do-better-in-actual-elections?noredirect=1&lq=1)) – user5341 Feb 09 '17 at 02:29
  • ... there were studies that people were literally primed by poll wording - responses differed based on question order. – user5341 Feb 09 '17 at 02:29
  • 6
    "unless there's a scandal or other notable event": Um, there's been one of those at least once a day since the inauguration. – Nate Eldredge Feb 09 '17 at 03:22
  • @DJClayworth Specifically for Donald Trump, hence the tag and title. I'd like to know if they're accurate, there's a partisan bias, or if there's a bias specifically for the current president. – Anoplexian Feb 09 '17 at 15:43
  • 1
    @Anoplexian _Some_ reputable organizations certainly do produce partisan and otherwise biased polls intended to elicit a specific response. I attempted to address this part of your question in my (since deleted) answer below. That the ACLU, in their _50-State Survey_, has done so is a certainty and can in no way be considered "theoretical", but I do not know if any of the specific polls you reference suffer from the same disingenuousness. – Michael J. Feb 14 '17 at 14:20
  • @MichaelJ. Yeah it's a tough question for sure, given the context. I would have loved to see your answer, but if you feel it wasn't quality, I definitely understand. – Anoplexian Feb 14 '17 at 15:10
  • Trump seems to do worse in [live interview polls](https://fivethirtyeight.com/features/live-polls-and-online-polls-tell-different-stories-about-the-election/) – Keith McClary Feb 16 '17 at 03:24

0 Answers0