15

I'm using MSVC 2010.

I'm trying to remove duplicate (without keeping any of them) from a list

Why is this code 100 times slower in debug mode?

Is there any other way to remove all objects that are equivalent and make it faster in debug mode?

It is to the point I can't use debug at the moment. It take minutes to process while few seconds in release.

void SomeFunction()
{
    std::list<Something> list;
    std::list<Something>::iterator it1;
    std::list<Something>::iterator it2;

    for (it1 = list.begin(); it1 != list.end(); it1++)
    {
        for (it2 = list.begin(); it2!=list.end(); it2++)
        {
            if (it1->SomeValue() == it2->SomeValue())
            {
                if (it1 != it2)
                {
                    list.erase(it1);
                    list.erase(it2);

                    it2 = list.begin();
                    it1 = it2++;
                }
            }
        }
    }
}
Pat
  • 305
  • 2
  • 8
  • 1
    Why not sort the list and use `std::unique`? – Kerrek SB Sep 27 '12 at 23:37
  • 5
    One general way of doing this that would be better is to iterate through the list and store each item in a set (if it isn't already there) as you go, then convert the set back to a list. That's O(n) and it doesn't modify the list as you're traversing it. Kerrek's suggestion is good, and there are probably other ways that are better. – i_am_jorf Sep 27 '12 at 23:39
  • 1
    Likely because of [debug iterators](http://msdn.microsoft.com/en-us/library/aa985982(VS.100).aspx) and [checked iterators](http://msdn.microsoft.com/en-us/library/aa985965(VS.100).aspx) being enabled by default in debug builds (plus, you know, the whole optimizations being disabled thing). Go to the links to see how to disable them, and see if that helps. – ildjarn Sep 27 '12 at 23:39
  • I'd do it as jeffamaphone suggested. Also, IMO, changing the index of a for loop inside the loop is not good programming practice. If you want to do that make it a while loop. (Yeah I know, it works, but it causes confusion). – John3136 Sep 27 '12 at 23:41
  • 1
    Why the upvotes? This is such a basic question, and is highly like to be a duplicate of any question asking "why does debugging make things slow?" – Drise Sep 27 '12 at 23:46
  • I can't use std::unique because I can't sort the list. How would you use a set if it's not possible to compare the object? It is only possible to know if they are equivalent. – Pat Sep 27 '12 at 23:54
  • If you intend to use that (or similar code), it appears buggy, depending on what it's supposed to do. But then again, I supposed that's why you want to know why it's so slow when debugging... If it is supposed to eliminate all but one of any set of duplicates, it'll remove all of them if there's an even number. If it's supposed to remove all elements that have duplicates (such that there are none), then it will leave elements that you don't want if the original has an odd number. – Michael Burr Sep 28 '12 at 00:07
  • @Pat : "*How would you use a set if it's not possible to compare the object? It is only possible to know if they are equivalent.*" Sounds like a good candidate for `std::unordered_set<>` then. – ildjarn Sep 28 '12 at 00:08
  • @jeffamaphone: sorting is never O(N) in terms of number of comparisons (as a `set` uses comparisons). Were you thinking of a hash set ? – Matthieu M. Sep 28 '12 at 06:40

1 Answers1

23

In general, STL is very slow while debugging in Visual Studio due to the iterator debugging support. You can speed this up dramatically by setting _HAS_ITERATOR_DEBUGGING to 0.

Reed Copsey
  • 554,122
  • 78
  • 1,158
  • 1,373
  • I tried to make this work for quite some time but without success. Other libraries I use forbid changing _HAS_ITERATOR_DEBUGGING to 0 – Pat Sep 27 '12 at 23:46
  • 3
    @Pat: You can always fall back to tried and true `printf()` debugging in release mode. If it's good enough for Linux kernel devs, it's should be good enough for anyone! Another possible solution is to change the Release configuration for the project to compile with optimizations off and symbols on. Then debug the Release build. It'll link to non-debug libraries, but you should still be able to step through your code just fine. Set optimization back on when you're done. – Michael Burr Sep 28 '12 at 00:12
  • if you have the source to your libraries you use, you could recompile all those with _HAS_ITERATOR_DEBUGGING 0 that's what I ended up doing for boost and some others... – Kyle C Sep 28 '12 at 00:27
  • Another thing - https://stackoverflow.com/a/9417542/3225391 can help you too. helped me a lot, in connection to the topic of the Q: why is debug so slow.. – ephraim Dec 06 '17 at 09:46
  • 1
    In MSVC, you can individually undefine _HAS_ITERATOR_DEBUGGING on a per project or per file basis by right clicking the troublemaker then "Properties" -> "C/C++" -> "Preprocessor" -> "Undefine Preprocessor Definitions" – Paul Childs Oct 17 '18 at 03:59
  • 1
    You can also isolate it to a certain bit of code with #pragma push_macro/pop_macro and #defining it as 0 in between. – Paul Childs Oct 17 '18 at 04:00
  • 2
    For msvc 2015 they have moved to _ITERATOR_DEBUG_LEVEL; combining both _HAS_ITERATOR_DEBUGGING and the less problematic _SECURE_SCL – Paul Childs Oct 17 '18 at 04:05