Performance tuning

Question

I have the following code segment . For a vector of size 176000 the loop takes upto 8 minutes to execute . I am not sure what is taking so much time

    XEPComBSTR bstrSetWithIdsAsString; //Wrapper class for BSTR
    std::vector<__int64>::const_iterator it;
    for(it = vecIds.begin();
        it != vecIds.end();
        it++)
    {

                   __int64 i64Id = (*it);
                  __int64 i64OID = XPtFunctions::GetOID(i64Id);  

                  // set ',' between two set members
                  if (it != vecIds.begin())
                        bstrSetWithIdsAsString.Append(XEPComBSTR(L","));
                      wchar_t buf[20];
        _i64tow_s(i64OID, buf, 20, 10);
        bstrSetWithIdsAsString.Append(buf);
    }


__int64 GetOID( const __int64 &i64Id)
{
    __int64 numId = i64Id;
    numId <<= 16;
    numId >>= 16;
    return numId;
}

i'll usually profile a program to determine why it does not perform as i think it should... — justin, Feb 23 '11 at 12:16
8 minutes under which circumstances? Are you compiling with optimizations? Which compiler are you using, and in the case of MSVC, have you disable the Secure STL features? — jalf, Feb 23 '11 at 12:19
why on *Earth* would you ask about performance without optimizations? Sane people enable optimizations *before* they measure performance. — jalf, Feb 23 '11 at 12:28
What is XEPComBSTR? The only google hit on it was IRFAN's comment from about a year ago http://stackoverflow.com/questions/2355226/help-deadlock-analysis — CashCow, Feb 23 '11 at 12:36
Your own wrapper? If it's from a well known API or Microsoft I would expect it to get more hits on google. — CashCow, Feb 23 '11 at 15:42

score 2 · Answer 1 · answered Feb 23 '11 at 12:28

I think your bottleneck is the Append function. You see, the string has some allocated memory inside, and when you try to append something that won't fit, it reallocates more memory, which takes a lot of time. Try allocating necessary memory once in the beginning. HTH

jalf · Answer 2 · 2011-02-23T17:42:25.473

The only way to find out what takes up all this time is to profile the application. Some editions of Visual Studio come with a fully functional profiler.

Alternatively, simply run the program in the debugger, and break it at random intervals, and note where in the code you are.

But I can see a few potential trouble spots:

you perform a lot of string appends. Do they allocate new memory each time? Does your string type allow you to reserve memory in advance, like std::string can do? In general, is the string class efficient?
you loop over iterators, and given the horrible Hungarian Notation you appear to use, I am assuming you are working on Windows, probably using MSVC. Some versions of MSVC enable a lot of runtime checking of STL iterators even in release builds, unless you explicitly disable it. VS2005 and 2008 specifically are guilty of this. 2010 only enables this checking in debug mode.
and, of course, you are building with optimizations enabled, right?

But I'm just pointing out what seems like it could potentially slow down your code. I have no clue what's actually happening. To be sure, I'd have to profile your code. You can do that. I can't. So do it.

score 1 · Answer 3 · answered Feb 23 '11 at 12:32

I am not sure what this is doing: bstrSetWithIdsAsString.Append(buf);

but I guess this is where the slowness is, particularly if it has to work out where the end of the buffer is every time by looking for the first zero byte, and possibly needs to do a lot of reallocating.

Why not use wostringstream?

Performance tuning

3 Answers3