It seems that Fabre's original claim has been greatly exaggerated by subsequent writers, and that the wasps are not nearly as stereotyped as those subsequent authors have claimed; Fabre's own description noted that not all wasps of the species showed this rigid behavior, and later studies have not supported it.
The Sphex story is an anecdote about a female digger wasp that at first sight seems to act quite intelligently, but subsequently is shown to be a mere automaton that can be made to repeat herself endlessly. ... The repetition was first observed by Henri Fabre in 1879, and the last empirical study I found was published in 1985. In contrast to the story's clear message, the actual results have always been equivocal: the endless repetition is not standard.
--The Sphex story: How the cognitive sciences kept repeating an old and questionable anecdote. By Fred Keijzer. Philosophical Psychology, Volume 26, 2013 - Issue 4
The article is paywalled but apparently can also be found here (I assume that these are the same). Quoting from this version:
Since then this particular behavior has been studied repeatedly by other students of insect behavior. They performed the cricket test on various species of digger wasp, such as Sphex species, but also Ammophila species that prey mostly on caterpillars. Looking at this history, there are several striking features. First and foremost, digger wasps very often do not repeat themselves endlessly when the cricket test is done. After a few trials many wasps take the cricket into their burrow without the visit.
Keijzer cites a number of studies that contradict the claim of absolute rigidity, as well as noting that Fabre's original observations were not as clear-cut as the claims make it. Keijzer quotes Fabre (The Hunting Wasps, 1915):
At the other holes, her neighbours likewise, one sooner, another later, discovered my treachery and entered the dwelling with the game, instead of persisting in abandoning it on the threshold to seize it afterwards. [...] Next day, in a different locality, I repeated my experiment with another Cricket; and every time the Sphex was hoodwinked.
(Edit: Fabre's The Hunting Wasps is available in Google Books and the section in question can be found there on pages 77-79. The description there makes it clear that even in the original version, not all individuals in the species were so stereotypical; but that he does claim that some individuals were completely stereotypical and never broke out of the loop.)
Keijzer lists a number of studies that I won't mention. The 1985 study he cites as "the last empirical study I found" is
Brockmann, H.J. (1985). Provisioning behavior of the great golden digger wasp, Sphex ichneumoneus (L.) (Specidae). Journal of the Kansas Entomological Society, 58(4), 631- 655.
I believe it's available as free full text and is an interesting read; in particular see Figure 4, a flow chart of wasp behavior during the provisioning process, which shows many steps with optional, flexible behavior choices. Her conclusion is
The adaptable provisioning behavior of Sphex ichneumoneus would be surprising to anyone who viewed insect behavior as stereotyped and fixed. The versatility of individuals extended to all phases of their behavior, from the habitats in which they hunted, to the types of prey captured, to the behavior used in getting the prey into the brood cell. Where responses show stereotypy, such as in repeated prey retrievals, there is an obvious, adaptive explanation. I suspect that long-term studies of known individuals in other species of insects would similarly reveal the same kind of adaptive behavioral versatility.
