43

Well, that's the conclusion of a 2018 US study, as reported in the press:

"White officers do not kill black suspects at a higher rate compared with nonwhite officers," concludes a research team led by Charles Menifield, dean of the School of Public Affairs and Administration at Rutgers University–Newark. "The killing of black suspects is a police problem, not a white police problem."

The actual paper's abstract isn't too different in its conclusion:

although minority suspects are disproportionately killed by police, white officers appear to be no more likely to use lethal force against minorities than nonwhite officers. [...]

The disproportionate killing of African Americans by police officers does not appear to be driven by micro‐level racism. Rather, it is likely driven by a combination of macro‐level public policies that target minority populations and meso‐level policies and practices of police forces.

So, is this paper "bullet proof"? I mean: Has it been criticized, e.g. for its methodology? Is its conclusion consistent with other research on this topic?

Community
  • 1
Fizz
  • 57,051
  • 18
  • 175
  • 291
  • 67
    Do note that there may be some bias introduced based on how the officers are assigned. If black police officers are preferentially assigned to black districts then they will tend to kill more black people than white. This will distort the measurements. Somehow the relative exposure of the officers to black vs white people must be taken into account to arrive at meaningful numbers. – Daniel R Hicks Apr 08 '19 at 01:35
  • @DanielRHicks - your comment makes no sense. Officers elect to work in an area for the most part. In larger cities they may be transferred to a certain area so that may be what you are refferring to. However better qualified officers will elect to work in safer areas on average - black or white. On average areas with a higher black population will be higher in crime in the US. Therefore these areas - on average - are getting the lesser qualified officers whether black or white. Saying an officer picks area based on race - no way. – blankip Apr 09 '19 at 03:41

1 Answers1

38

TL;DR - we don't know if the study is corroborated.

I will continue to look into this topic to edit and improve the answer. It may be too early to conclude if the results of the 2018 paper are corroborated by other sources.

RE: Methodology, the paper covers it well (emphasis added).

We constructed an original database of all confirmed uses of deadly force by police officers in the United States in 2014 and 2015.

We began by drawing on data gathered by Killed By Police, a nongovernmental entity that tracks police killings reported in the news and updates its data set each day. We chose this source as a base because the site links each killing with a news story that we could locate online. In order to ensure that the accuracy of the data, we cross-checked it with two other websites that collect data on police killings (lethaldb.silk.co and FatalEncounters.org). All three data sets have been used by other scholars studying police killings (Lott and Moody 2016; Nicholson-Crotty, Nicholson-Crotty, and Fernandez 2017).

The Killed By Police data contained the victim’s name, race, age, date of birth, gender, date and time of killing, city, state, and a news account of the killing. We supplemented these data with other variables available in news accounts and other police killing data sets, including local population demographics, cause of death, geographic location of the killing, type of offense, presence of a nonpolice witness, and whether there was a warrant for the suspect.

...We also coded for a range of variables about the officers whenever possible, such as officer race and gender, years of police service, and type of officer. Because of missing data, we had to thoroughly analyze every news story that we could locate on each killing.

First, let's look at the credibility of the websites.

Looking at Killed By Police, each entry in the database indeed includes a news article (and usually, 1+ news articles). Thus, it appears credible. Something interesting to note is that it has been fact-checked by FiveThirtyEight. From FiveThirtyEight:

We randomly sampled 146 incidents (10 percent) from the news links posted to Killed By Police. All the posts linked to established outlets, although in some cases a new url for the article had to be found because the news site had restructured its links.

Looking at lethaldb.silk.co, we find the message "It’s time to say goodbye" and a notice that it has been shut down. The Internet Archive didn't have screenshots of older versions of the webpage, so I cannot verify its authenticity.

FatalEncounters.org also appears credible, as it includes a news story with each entry. Note that for all sites I checked some entries, but not all.

Next, let's see if other scholars have actually used the same data set.

The pdf for Lott and Moody 2016 can be accessed here and is published in the Social Science Research Network. I couldn't find the impact factor for this journal. From what I can tell, it is similar to ArXiv, so I would take documents here with a grain of salt. I could see from the appendix that data was collected from killedbypolice.net and fatalencounters.org, but not from the other source.

Nicholson-Crotty, Nicholson-Crotty, and Fernandez 2017 can be accessed here. It is published in the Public Administration Review, the same journal as is the paper in question. In 2017, the impact factor was 4.591. Reading the paper confirms it also draws data from KilledByPolice and FatalEncounters (but, again, not from the third source).

Thus, some scholars did actually use data from 2 of the sources as the paper in question.

It may be too early to conclude if the conclusion is consistent with other research on this topic. The paper was published in 2018, and as the authors say

However, to our knowledge, no study has directly assessed the racial composition of officer killings of suspects.

To address your questions:

Is this paper "bullet proof"?

No paper is truly bullet proof.

Has it been criticized (e.g. for methodology)?

I have not yet found criticisms of this paper. However, it may be too early to tell. By checking the three sources the paper used for its data set, two (KilledByPolice and FatalEncounters) appear to be credible. The third (lethaldb.silk.co) could not be fact-checked due to lack of an online presence (both current and historical). The paper mentions the three websites as having been used by other scholars. This is a true statement for KilledByPolice and FatalEncounters. This is an unverified (possibly false) statement for lethaldb.silk.co.

Is the conclusion of this paper consistent with other research on this topic?

It may be too early to answer this question as the paper was published in June 2018 and has been cited 2 times. A 2004 paper found undergraduate students were more likely "to shoot Black targets but not Whites."

Barry Harrison
  • 14,093
  • 4
  • 68
  • 101
  • 8
    You establish that none of the data in KBP is fake. You do not establish that the data in KBP is complete: To fake bias in either way, I could simply use true data only, but omit some killings for one of the races. – FooBar Apr 08 '19 at 08:36
  • 12
    Essentially, 538 would need to also randomly sample x% of incidents from some established outlet, and see whether they all appear in KBP. – FooBar Apr 08 '19 at 08:36
  • @FooBar - While I might not trust police to report killings accurately, do you think a non-governmental watchdog is likely to be faking the data? – Obie 2.0 Apr 08 '19 at 09:45
  • 1
    @Obie2.0 If you trust KBP, you don't need to even perform the check that 538 provides. All I'm saying is that the check done is - as is - incomplete and hence meaningless. Furthermore, data can be biased without bad intent. – FooBar Apr 08 '19 at 11:24
  • You might want to look into the studies [PragerU cited in 2016](https://youtu.be/UQCQFH5wOJo). Many of the stats they mention seem uncited, but there are some specific studies mentioned at the beginning. – jpmc26 Apr 09 '19 at 00:48
  • @jpmc26 Thanks for sharing, I will look into it. Do you have a summary of sorts? Or specific article names? – Barry Harrison Apr 09 '19 at 00:49
  • @BarryHarrison Not off-hand. [Here's](https://www.washingtonpost.com/news/true-crime/wp/2016/04/27/this-study-found-race-matters-in-police-shootings-but-the-results-may-surprise-you/?noredirect=on) an article about one of them. I think [this](https://www.nber.org/papers/w22399) is the second one they mention. I just knew of the video as a potential source for info. – jpmc26 Apr 09 '19 at 00:56
  • @jpmc26 I have read the [2016 paper](https://onlinelibrary.wiley.com/doi/abs/10.1111/1745-9133.12187) "The Reverse Racism Effect." It is a very interesting read and surprising (to me at least). Basically, "despite clear evidence of implicit bias against Black suspects, officers were slower to shoot armed Black suspects than armed White suspects, and they were less likely to shoot unarmed Black suspects than unarmed White suspects." Would you want me to edit this into the answer? I feel the community could talk forever (figuratively) about police shootings and race. – Barry Harrison Apr 09 '19 at 01:08
  • @jpmc26 In other words, do you think the study is relevant to the question and will add to the answer? – Barry Harrison Apr 09 '19 at 01:08
  • "credibility of the websites" : none of them even end in, *dot.gov*. Credibility: zero. – Mazura Apr 09 '19 at 01:20
  • @Mazura Interesting point. Do you think that government databases are more complete? – Barry Harrison Apr 09 '19 at 01:29
  • @BarryHarrison - I'd want every police report ever filled out and the race of the responding officers and all parties involved. I doubt that's public record. These are random websites trolling other websites for data that's been cheery picked by some journalist. – Mazura Apr 09 '19 at 01:38
  • Re. reason to doubt the methodology : what you've highlighted is all dubious: "that we could locate online" - "cross-checked it with" KilledByPolice.com? (No bias there, for sure...), and then combed over some news stories. - It's (what had to have been) a low sample size, of an already biased data set. – Mazura Apr 09 '19 at 01:53
  • @Mazura Any suggestions for improvement of the answer? Thanks for your input! – Barry Harrison Apr 09 '19 at 03:03
  • Not unless the FBI has a table for feloniously killed civilians by law enforcement personnel, like in [the answer here](https://skeptics.stackexchange.com/questions/39880/is-a-police-officer-18x-more-likely-to-be-killed-by-a-black-male-than-an-unarmed) showing Law Enforcement Officers Feloniously Killed. – Mazura Apr 09 '19 at 03:56
  • @Mazura OK. I won't be making edits based on your comments then. I couldn't find a similar table. – Barry Harrison Apr 09 '19 at 21:11
  • 2
    I wonder if the need to depend on news articles adds a bias in itself. For example maybe news is more likely to report about police shootings if the officer is white because it leads to more outrage (and thus more views)? alternatively maybe they ignore shootings where the shooting is clearly justified by the situation (such as the individual was firing at the officer at the time) and thus were getting samples emphasis only morally ambiguous shootings which could taint the data set. – dsollen Apr 11 '19 at 15:37
  • Any update on this, one year later? – Nacht Jun 01 '20 at 00:42
  • @BarryHarrison: any further research available now on this topic? – Jayson Virissimo Jun 01 '20 at 23:13
  • 1
    @Nacht It's great to see continued interest in this topic, even over a year later. Today, the paper has 17 citations, which I don't think is a lot, though I am not familiar with the specific field. It's interesting to see that the Georgia Association of Chiefs of Police has taken note of this study (https://gachiefs.com/9335-2/). – Barry Harrison Mar 22 '21 at 06:23
  • 1
    @JaysonVirissimo I will hopefully take a deeper look in the coming days, when I have more time. I just read this NPR article (on a different study) that says "A new peer-reviewed study of fatal police shootings says that white officers are not more likely to shoot and kill minority suspects. Critics contend it doesn't address racial disparities by police." However, this different study in question has been retracted (https://www.pnas.org/content/117/30/18130). – Barry Harrison Mar 22 '21 at 06:27
  • 1
    "Despite this correction, our work has continued to be cited as providing support for the idea that there are no racial biases in fatal shootings, or policing in general. To be clear, our work does not speak to these issues and should not be used to support such statements. We take full responsibility for not being careful enough with the inferences made in our original report, as this directly led to the misunderstanding of our research." – Barry Harrison Mar 22 '21 at 06:27
  • "While our data and statistical approach were appropriate for investigating whether officer characteristics are related to the race of civilians fatally shot by police, they are inadequate to address racial disparities in the probability of being shot." – Barry Harrison Mar 22 '21 at 06:27
  • And surprisingly for me, this Q&A is my second Google result for "lethaldb silk"! – Barry Harrison Mar 22 '21 at 06:32