16

What is the most efficient way to remove all empty List[] objects from all of the Lists that appear in an expression at different levels? The empty List[] should be removed only if it is an element of another List itself.

Sjoerd C. de Vries
  • 16,122
  • 3
  • 42
  • 94
Alexey Popkov
  • 9,355
  • 4
  • 42
  • 93

1 Answers1

21

Andrew and Alexey point out that using expr //. x_List :> DeleteCases[x, {}, Infinity] as I had in my previous answer will also remove the {} in blah[{f[{}]}], whereas it should leave it untouched as its head is f, not a List. The solution, thanks to Leonid, is to not use ReplaceRepeated, but Replace instead with replacements being made at all levels from 0 through Infinity:

Replace[expr, x_List :> DeleteCases[x, {}], {0, Infinity}]

The reason why Replace works and ReplaceRepeated doesn't can be seen from this little example. Consider expr = {a, {}, {b, {}}, c[d, {}]}; in its TreeForm

enter image description here

Replace works by starting with the innermost expression(s) first, i.e., List[b,{}] and c[d,{}], and works upwards to the top node. At each level, checking the head is as simple as looking up to the node right above and see if it matches List. If it does, apply the rule and move up a level, else do nothing and move up a level. This results in a final tree:

enter image description here

ReplaceRepeated (//.), on the other hand, works by starting with the top most node and traversing down the tree. The previous solution starts by checking if the first node is a List and if it is, then DeleteCases is applied and it moves down the tree, ruthlessly replacing every {} it can find. Note that it does not check if the heads of the inner expressions also match List, because this traversal is done by DeleteCases, not ReplaceRepeated. When //. moves to subsequent lower nodes, there is nothing more to replace and it exits quickly. This is the tree that one gets with the previous solution:

enter image description here

Note that the {} inside c[d, {}] has also been removed. This is solely due to the fact that DeleteCases (with level specification {0,Infinity} moves down the tree. Indeed, if the first head had been something other than List, it would've skipped it and moved to the next level, of which only the {} in {b, {}} is a match. To demostrate with expr2 = f[a, {}, {b, {}}, c[d, {}]], we get

enter image description here

Note that in the current solution with Replace, we use DeleteCases with the default level specification, which is the first level only. It does not, therefore, check for and delete empty lists deeper than on the first level, which is exactly what we need here.

Although we used the first node to explain why it fails, the reasoning holds true for every node. Leonid explains these concepts in greater detail in his book

Leonid Shifrin
  • 22,449
  • 4
  • 68
  • 100
abcd
  • 41,765
  • 7
  • 81
  • 98
  • This will also remove empty lists inside heads other than lists, and thus contradict the specs. – Leonid Shifrin Jul 03 '11 at 16:32
  • @Leonid: oops. Didn't read the question carefully... I think my edit should fix it. – abcd Jul 03 '11 at 16:46
  • 3
    Your new solution is great, very efficient. Much faster than either of mine. +1. – Leonid Shifrin Jul 03 '11 at 16:57
  • Since the question was about efficiency, I will delete my answer, as yours is much superior. Interestingly, a similar idea crossed my mind, but I discarded it :) – Leonid Shifrin Jul 03 '11 at 17:00
  • 2
    Doesn't work correct on `blah[a, b, {{{}}}, d, {e, f, {}, g}] //. x_List :> DeleteCases[x, {}]` ==> `blah[a, b, {{{}}}, d, {e, f, g}]`. Add `Infinity` to `DeleteCases`. – Sjoerd C. de Vries Jul 03 '11 at 23:26
  • The current answer doesn't work on e.g. blah[{f[{}]}] //. x_List :> DeleteCases[x, {}, Infinity] ==> blah[{f[]}]. – Andrew Moylan Jul 04 '11 at 04:09
  • @Andrew `blah[{f[{}]}]` should be leaved unchanged since `{}` is not an element of a `List`. – Alexey Popkov Jul 04 '11 at 07:29
  • @yoda The current answer does not work correctly with `{dd[e, f, {}, g]} //. x_List :> DeleteCases[x, {}, Infinity]` => `{dd[e, f, g]}`. It removes `{}` inside `dd` head although must remove `{}` only when it is an element of a `List`. – Alexey Popkov Jul 04 '11 at 07:32
  • 3
    @yoda, @Alexey, @Sjoerd It seems that all that is needed to cure @yoda's solution is to use `Replace`: `Replace[expr, x_List :> DeleteCases[x, {}], {0, Infinity}]`. Since `Replace` acts from bottom to top (depth-first), unlike `ReplaceRepeated`, the problem is then naturally solved. It is much slower than the `DeleteCases[expr,{}]` version however. As a side product, you don't need a repeated rule application. I discussed the differences between `Replace` and `ReplaceRepeated` here: http://www.mathprogramming-intro.org/book/node218.html. This appears to be a good problem to illustrate it. – Leonid Shifrin Jul 04 '11 at 08:16
  • @Leonid Good point! We can speed-up your version even more by wrapping `expr` with `Unevaluated`: `Replace[Unevaluated@expr, x_List :> DeleteCases[Unevaluated@x, {}], {0, Infinity}]`. It gives dramatic speed up for large expressions! – Alexey Popkov Jul 04 '11 at 08:53
  • 1
    @Alexey Did you test your suggestion? It does not seem to work. Try any of the test cases, say `{{{}}}` or `blah[a, b, {{{}}}, d, {e, f, {}, g}]`. The problem is that, if no evaluation happens (which is the case for these test cases), sub-evaluations induced by `Replace` restore the `Unevaluated` wrappers. Therefore, for inert expressions, `Unevaluated` does not buy you anything, just spoils things. If you want to perform the list deletion operation on a piece of code that may evaluate, this is a separate problem. – Leonid Shifrin Jul 04 '11 at 09:13
  • @yoda Can you add a line or two of explanation, for future users? I mean, why `Replace` works and `ReplaceRepeated` does not - something along the lines of what I mentioned in my comment (many people won't read the comments) – Leonid Shifrin Jul 04 '11 at 15:00
  • @Leonid: I've added a small explanation, which might not be entirely thorough. Please feel free to correct mistakes or make it clearer. – abcd Jul 04 '11 at 16:55
  • @yoda Thanks, that's a great explanation! I only added a small paragraph at the end, clarifying the role of level specification in `DeleteCases` for this problem. – Leonid Shifrin Jul 04 '11 at 17:37
  • @Leonid: Thanks! I forgot to mention that and your edit makes it very clear. – abcd Jul 04 '11 at 17:45