6

We've all written innumerable essays from middle school onwards. All teachers appear convinced that this will improve students' essay writing skills. However, when trying to find out what this belief is based on, I found only a single study where some students were assigned practice essays and others were not, and their results on an essay-writing exam were compared. The result of the study, in the authors' words, was as follows.

As discussed in the previous Part, we found that, on average, students who received the writing interventions had higher average raw scores on each of the essay questions. Although this point spread did not rise to the level of statistical significance, we found between a two and three point raw score difference (out of a possible thirty raw score points on the first question and thirty-three raw score points on the second question). [My bold]

Effectively the authors seem to have found no effect. Does anyone have any more data on this?

Edit: I can amuse you further with the following quote from the article. It seems I am not the only one having trouble finding support for the claim that essay writing practice improves performance.

For example, we have found no work attempting to measure empirically the impact of practice writing exercises, combined with feedback, on student essay exam performance.

Professor Flitwick
  • 85
  • 2
  • 11
    Are you having difficulty with the idea that practicing a skill, and receiving feedback and instruction on how to do it better, might result in improvement in that skill? – DJClayworth Jul 11 '20 at 19:33
  • 6
    Careful. It isn't that they *found no effect*, it's that they *did not find an effect*. A lack of statistical significance does not rule out the possibility that there is an effect. Absence of evidence is not evidence of absence. – Nate Eldredge Jul 12 '20 at 01:41
  • "did not rise to the level of statistical significance" could mean there was no effect; or it could mean the sample was too small. – GEdgar Jul 12 '20 at 11:45
  • @DJClayworth With regard to this particular skill, yes. I am aware that there are skills which can be improved by simply practicing (as opposed to learning other things relating to the task). – Professor Flitwick Jul 12 '20 at 16:19
  • 5
    @ProfessorFlitwick You have misunderstood my point. When students write essays, they are also getting feedback on whether and why their essays are good or bad. And teachers also provide instruction on how to write better essays. That's pretty much how most teaching related to improving a skill works. – DJClayworth Jul 12 '20 at 16:58
  • 4
    It is difficult to imagine what an answer here might look like. A brief intervention, as described, is likely to have a small impact compared to the innumerable essays in middle school that you describe. A study could easily show that essay from students get better as they get promoted through the grades, but how could we extract the effect that *writing practice* had from maturity, from reading experience, from direct instruction, etc? Comparing with students who did not receive tuition is likely to have other complicating factors. What would you expect an answer to look like? – Oddthinking Jul 12 '20 at 21:07
  • @DJClayworth So if I understand you correctly, you believe that practicing writing essays by itself does not improve your essays, but getting feedback on your essays does? My question was meant to be agnostic on the point of feedback, so feel free to prove or disprove that statement as an answer. – Professor Flitwick Jul 13 '20 at 15:40
  • 1
    I have no information on whether writing essays by itself with no feedback improve your essay writing. But this is not what happens in the real world. You seem to be questioning the idea that practicing a skill and receiving feedback and instruction on how to do it better is effective. This approach is used for all sorts of skills, and is used in training doctors, musicians, sportspeople, soldiers, authors, etc. To single out one skill and say "is it effective for this one skill" seems bizarre. But the real issue is that you don't seem to have a notable claim that the approach isn't effective. – DJClayworth Jul 13 '20 at 16:25
  • 1
    The notable claim is that practicing essay writing improves essay-writing skill. This claim has been made by numerous teachers, parents, etc. OP is skeptical of this claim and wants proof. A strong answer to this question would reference one or more studies which seek to test the impact of practice on essay-writing skill. It is unlikely one could eliminate confounding variables from an investigative study. On the other hand, creating a dedicated study requires one to convince a school board that such a study is worth the risk of possible disruption to student education. – Brian Jul 13 '20 at 19:02
  • For the record, there are numerous studies which test the impact of practice/experience on learning different skills, with a consistent, "it helps" result. OP is asking specifically for studies on essay writing, which is a much narrower request. Mind you, I don't think there's enough evidence against the uselessness of essay-writing practice to justify being skeptical that, in the specific case of essay-writing, such specific evidence is necessary. – Brian Jul 13 '20 at 19:04
  • Do we have anyone who claims that practice of essay-writing (with appropriate feedback) **doesn't** improve essay writing? – DJClayworth Jul 13 '20 at 20:04
  • The thing is that I am not at all sure that essay-writing is a skill in the same way as, say writing Chinese characters. It seems to me that essay quality is largely a result of relevant factual knowledge (especially of the topic and language involved). Compare the situation where you are supposed to take a multiple-choice quiz on subject X: unless you've never seen such a quiz before, studying subject X is going to have a much larger effect than practicing taking quizzes in other subjects. Is essay-writing more like writing Chinese characters or like taking quizzes? – Professor Flitwick Jul 14 '20 at 11:11
  • @Oddthinking I suppose you would have to take two otherwise equivalent groups of students and have one of them do essay-writing while the other group uses that time for something else, making all other instruction identical, and then see if there are any significant differences. And as pointed out in the comments to one of the answers, you would have to choose an appropriately large time frame. – Professor Flitwick Jul 14 '20 at 11:18
  • 2
    @ProfessorFlitwick: I accept that this is an on-topic question for the site, but I suspect your priors are so different to that of an educational ethics committee, that such an experiment would never be permitted. (i.e. if the null hypothesis is correct, and essay writing is improved via practice, then a multi-year experiment which deprived high school students of practice time would jeopardise their opportunity for tertiary study), so I doubt it will receive an answer. I am ready to be surprised though. – Oddthinking Jul 14 '20 at 11:56
  • @Oddthinking I'm a bit doubtful too, but thought it was worth a try at least. But I do think you could still run this experiment by giving some students the usual amount of practice and giving others more (say summer school or something). – Professor Flitwick Jul 18 '20 at 13:54

1 Answers1

5

No, that was not the point of the study: it was the vehicle for it.

The very last sentence of the report's Conclusion states:

Of course, especially initially, the empirical results are not beyond scientific reproach, nor even conclusive. However, as is the case with all social science research, the first step is developing a model that can be replicated and improved upon. That is what we hope to have accomplished.
[My bolding]

So although the report focuses on essay writing, its overall objective was not to determine whether practice improves that skill, but to report on establishing a valid empirical methodology, with the aim of evaluating teaching methods more generally.

Weather Vane
  • 7,653
  • 1
  • 32
  • 42
  • I know it's not a very good study. I put it in because it was the only one I found at all. Really I was hoping somebody would know of any better ones. – Professor Flitwick Jul 11 '20 at 17:32
  • 1
    The remit of Skeptics SE is to challenge **notable claims** and this attempts to answer what seems to be your claim that: a study finds that writing practice makes no difference. If your aim is otherwise you might find it helpful to take [the Tour](https://skeptics.stackexchange.com/tour) and also read [What topics can I ask about here?](https://skeptics.stackexchange.com/help/on-topic). – Weather Vane Jul 11 '20 at 17:37
  • @WeatherVane The claim was supposed to be "Writing practice improves writing skills", as per the title. This is a pretty notable claim since pretty much all school teachers support it. The issue is that I have found zero studies supporting it, and a total of one not supporting it. – Professor Flitwick Jul 12 '20 at 16:22