Memory corruption during string tokenizing

Question

I am facing memory corruption and crash when doing a string tokenize. The crash is not occuring in the first time of processing . It takes more than an hour to crash. The problemmatic

    wchar_t *query = new wchar_t[inQry.length() + 1];

    memset(query, NULL, inQry.length() + 1);
    memcpy(query, inQry.c_str(), inQry.size());


    wchar_t *Tok = wcstok(query, L" ");

    headStr = L"";
    tableName = L"QUERY";

    while(Tok != NULL)
    {
        vectorSQLEntry.push_back(Tok);
        Tok = wcstok(NULL, L" ");
    }

    int tokCount = vectorSQLEntry.size();
    if(query != NULL)
    {
    delete query;
    }

When diagonized with debugdiag and analyzing the dump created, it pointed out some other line in the code.(sometimes the line will be delete query). So i removed the query as pointer and declared as wstring. And i removed the initialization of headStr and tableName inbetween wcstok and while loop(code as below).

wstring  tmpQuery = inQry;
wchar_t *Tok = wcstok((wchar_t*)tmpQuery.c_str(), L" ");
while(Tok != NULL)
{
    vectorSQLEntry.push_back(Tok);
    Tok = wcstok(NULL, L" ");
}

With this code the crash is not occurring.! So what is the problem with the initial code? This crash dragged me more than two days.

When you have problems with memory corruption, you should also demonstrate that Valgrind couldn't identify problems - otherwise it looks as if you just haven't tried. — Toby Speight, Jun 22 '17 at 09:35

score 2 · Answer 1 · answered Jun 22 '17 at 07:09

2

Your delete statement should be delete []query; I would expect you to be running out of memory after a while and that probably causing new to fail and a crash.

answered Jun 22 '17 at 07:09

Code Gorilla

962
9
23

score 0 · Answer 2 · answered Jun 22 '17 at 07:12

0

if(query != NULL)

Query can not be null because new will throw an exception if fails.

 wcstok((wchar_t*)tmpQuery.c_str(), L" ");

You are trying to write into read-only memory corrupting string.

answered Jun 22 '17 at 07:12

user7860670

35,849
4
58
84

"You are trying to write into read-only memory corrupting string." - But with this code i dint get any crash..! Do u mean , if i access the tmpQuery after this line i ll get an exception? – Sel_va Jun 22 '17 at 07:50
@Sel_va Technically nothing prevents `wcstok` from crashing right away. Neither c-style and const-cast should appear in your code. And this piece definitely does not require any casts at all. Also you should update your question to include [mvce](https://stackoverflow.com/help/mcve). Depending on `vectorSQLEntry` type and whatever happens before / next you may have different problems. – user7860670 Jun 22 '17 at 07:58
vector's type is wstring. I just fill the vector with tokenized strings and search for a word and return that word(wstring) from this function. – Sel_va Jun 22 '17 at 09:20
@VTT new does not always throw an exception, it depends on the implementation, and if new has been overridden. If its creation of a third party type, I would always check for ptr==null after a new. – Code Gorilla Jun 27 '17 at 07:33
1

@CodeGorilla Both default and custom allocation functions that are not marked as non-throwing are required to throw an exception that matches `std::bad_alloc` if allocation fails. Using custom allocation function that does not follow this requirement and / or checking return value of operator new that is not marked as non-throwing is undefined behavior. – user7860670 Jun 27 '17 at 09:18

score 0 · Answer 3 · edited Oct 19 '17 at 04:01

0

I have got the same problem and I found the right way to solve it. You didn't initialize the variable query correct. Your code is :

memset(query, NULL, inQry.length() + 1);

The third parameter is not right. You should modify it just like follow code:

memset(query, NULL, (inQry.length() + 1) * sizeof(wchar_t));
...
delete[] query;

Then it will work normally.

edited Oct 19 '17 at 04:01

wscourge

10,657
14
59
80

answered Oct 19 '17 at 03:56

beibeitu

164
7

Memory corruption during string tokenizing

3 Answers3