Invalid free() / delete / delete[] / realloc() with std::shared_ptr

Question

I'm new to using C++ smart pointers. I wanted to use them in one large project I'm working on, but I'm getting the following error:

==21819== Invalid free() / delete / delete[] / realloc()
==21819==    at 0x4838E7B: operator delete(void*) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==21819==    by 0x188692: std::_Sp_counted_ptr<std::vector<double, std::allocator<double> >*, (__gnu_cxx::_Lock_policy)2>::_M_dispose() (shared_ptr_base.h:377)
==21819==    by 0x12F1C9: std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() (shared_ptr_base.h:155)
==21819==    by 0x12E724: std::__shared_count<(__gnu_cxx::_Lock_policy)2>::~__shared_count() (shared_ptr_base.h:706)
==21819==    by 0x1406D9: std::__shared_ptr<std::vector<double, std::allocator<double> >, (__gnu_cxx::_Lock_policy)2>::~__shared_ptr() (shared_ptr_base.h:1145)
==21819==    by 0x14071B: std::shared_ptr<std::vector<double, std::allocator<double> > >::~shared_ptr() (shared_ptr.h:103)
==21819==    by 0x1CAB5A: myProg::LearningSequence::optimize(myProg::ErrorFunction&, std::basic_ofstream<char, std::char_traits<char> >*) (LearningSequence.cpp:60)
==21819==    by 0x12DD6A: main (myScript_1_2.cpp:107)
==21819==  Address 0xb2b90d8 is 24 bytes inside a block of size 48 alloc'd
==21819==    at 0x4837DBF: operator new(unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==21819==    by 0x1331C8: __gnu_cxx::new_allocator<std::_Sp_counted_ptr_inplace<myProg::RandomSolution, std::allocator<myProg::RandomSolution>, (__gnu_cxx::_Lock_policy)2> >::allocate(unsigned long, void const*) (new_allocator.h:111)
==21819==    by 0x132BA0: std::allocator_traits<std::allocator<std::_Sp_counted_ptr_inplace<myProg::RandomSolution, std::allocator<myProg::RandomSolution>, (__gnu_cxx::_Lock_policy)2> > >::allocate(std::allocator<std::_Sp_counted_ptr_inplace<myProg::RandomSolution, std::allocator<myProg::RandomSolution>, (__gnu_cxx::_Lock_policy)2> >&, unsigned long) (alloc_traits.h:436)
==21819==    by 0x1320E9: std::__allocated_ptr<std::allocator<std::_Sp_counted_ptr_inplace<myProg::RandomSolution, std::allocator<myProg::RandomSolution>, (__gnu_cxx::_Lock_policy)2> > > std::__allocate_guarded<std::allocator<std::_Sp_counted_ptr_inplace<myProg::RandomSolution, std::allocator<myProg::RandomSolution>, (__gnu_cxx::_Lock_policy)2> > >(std::allocator<std::_Sp_counted_ptr_inplace<myProg::RandomSolution, std::allocator<myProg::RandomSolution>, (__gnu_cxx::_Lock_policy)2> >&) (allocated_ptr.h:97)
==21819==    by 0x131988: std::__shared_count<(__gnu_cxx::_Lock_policy)2>::__shared_count<myProg::RandomSolution, std::allocator<myProg::RandomSolution>>(std::_Sp_make_shared_tag, myProg::RandomSolution*, std::allocator<myProg::RandomSolution> const&) (shared_ptr_base.h:654)
==21819==    by 0x131206: std::__shared_ptr<myProg::RandomSolution, (__gnu_cxx::_Lock_policy)2>::__shared_ptr<std::allocator<myProg::RandomSolution>>(std::_Sp_make_shared_tag, std::allocator<myProg::RandomSolution> const&) (shared_ptr_base.h:1322)
==21819==    by 0x130784: std::shared_ptr<myProg::RandomSolution>::shared_ptr<std::allocator<myProg::RandomSolution>>(std::_Sp_make_shared_tag, std::allocator<myProg::RandomSolution> const&) (shared_ptr.h:360)
==21819==    by 0x12FA99: std::shared_ptr<myProg::RandomSolution> std::allocate_shared<myProg::RandomSolution, std::allocator<myProg::RandomSolution>>(std::allocator<myProg::RandomSolution> const&) (shared_ptr.h:707)
==21819==    by 0x12EE24: std::shared_ptr<myProg::RandomSolution> std::make_shared<myProg::RandomSolution>() (shared_ptr.h:723)
==21819==    by 0x12DB48: main (myScript_1_2.cpp:86)

I'm really confused by this, because there is a std::shared_ptr created on line 86.

86    std::shared_ptr<myProg::LearningMethod> new_learning_method = std::make_shared<myProg::RandomSolution>();
    learning_sequence.add_learning_method( new_learning_method );

    std::shared_ptr<myProg::LearningMethod> new_learning_method2 = std::make_shared<myProg::ParticleSwarm>(myProg::ParticleSwarm(&domain_bounds,
                                                                                                     1.711897,
                                                                                                     1.711897,
                                                                                                     0.711897,
                                                                                                     0.5,
                                                                                                     0.3,
                                                                                                     0.7,
                                                                                                     n_particles_swarm,
                                                                                                     max_n_iters_swarm) );
//        learning_sequence.add_learning_method( new_learning_method2 );

    std::shared_ptr<myProg::LearningMethod> new_learning_method3 = std::make_shared<myProg::LevenbergMarquardt>(l4n::LevenbergMarquardt(max_n_iters_gradient_lm, batch_size, prec_lm ) );
    learning_sequence.add_learning_method( new_learning_method3 );


    /* Complex Optimization */
107    learning_sequence.optimize(mse1);  // Network training

The function optimize() look like this:

   void LearningSequence::optimize(myProg::ErrorFunction &ef, std::ofstream *ofs) {

    double error = ef.eval();

    double the_best_error = error;
    int mcycles = this->max_number_of_cycles, cycle_idx = 0;

    std::shared_ptr<std::vector<double>> best_params = std::make_shared<std::vector<double>>(this->best_parameters);
    while( error > this->tol && mcycles != 0){
        mcycles--;
        cycle_idx++;

        for( auto m: this->learning_sequence ){
            m->optimize( ef, ofs );
            error = ef.eval(m->get_parameters());

            ef.get_network_instance()->copy_parameter_space(m->get_parameters());



            if( error < the_best_error ){
                the_best_error = error;
                this->best_parameters = *ef.get_parameters();
            }

            if( error <= this->tol ){
                ef.get_network_instance()->copy_parameter_space( best_params );
                return;
            }
        }
        COUT_DEBUG("Cycle: " << cycle_idx << ", the lowest error: " << the_best_error << std::endl );
    }
    ef.get_network_instance()->copy_parameter_space( best_params );
}

The error further happens in the function eval():

double MSE::eval(std::shared_ptr<std::vector<double>> weights,
                 bool denormalize_data,
                 bool verbose) {
    return this->eval_on_data_set(this->ds,
                                  nullptr,
                                  weights,
                                  denormalize_data,
                                  verbose);
    puts("END OF EVAL()");
}

which calls eval_on_data_set():

   double MSE::eval_on_data_set(myProg::DataSet* data_set,
                                 std::ofstream* results_file_path,
                                 std::shared_ptr<std::vector<double>> weights,
                                 bool denormalize_data,
                                 bool verbose) {
        size_t dim_in = data_set->get_input_dim();
        size_t dim_out = data_set->get_output_dim();
        double error = 0.0, val, output_norm = 0;

        std::vector<std::pair<std::vector<double>, std::vector<double>>>* data = data_set->get_data();
        size_t n_elements = data->size();

        std::vector<std::vector<double>> outputs(data->size());
        std::vector<double> output(dim_out);

        if (verbose) {
            COUT_DEBUG("Evaluation of the error function MSE on the given data-set" << std::endl);
            COUT_DEBUG(R_ALIGN << "[Element index]" << " "
                               << R_ALIGN << "[Input]" << " "
                               << R_ALIGN << "[Real output]" << " "
                               << R_ALIGN << "[Predicted output]" << " "
                               << R_ALIGN << "[Absolute error]" << " "
                               << R_ALIGN << "[Relative error %]"
                               << std::endl);
        }

        if (results_file_path) {
            *results_file_path << R_ALIGN << "[Element index]" << " "
                               << R_ALIGN << "[Input]" << " "
                               << R_ALIGN << "[Real output]" << " "
                               << R_ALIGN << "[Predicted output]" << " "
                               << R_ALIGN << "[Abs. error]" << " "
                               << R_ALIGN << "[Rel. error %]"
                               << std::endl;
        }

        for (auto i = 0; i < data->size(); i++) {  // Iterate through every element in the test set
            /* Compute the net output and store it into 'output' variable */
            this->net->eval_single(data->at(i).first,
                                   output,
                                   weights);

            outputs.at(i) = output;
        }

        double denormalized_output;
        double denormalized_real_input;
        double denormalized_real_output;

        for (auto i = 0; i < data->size(); i++) {

            /* Compute difference for every element of the output vector */
#ifdef DEBUG
            std::stringstream ss_input;
            std::string separator = "";
            for (auto j = 0; j < dim_in; j++) {
                if(denormalize_data) {
                    denormalized_real_input = data_set->get_normalization_strategy()->de_normalize(data->at(i).first.at(j));
                } else {
                    denormalized_real_input = data->at(i).first.at(j);
                }
                ss_input << separator << denormalized_real_input;
                separator = ",";
            }
            if(denormalize_data) {
                denormalized_real_input = data_set->get_normalization_strategy()->de_normalize(data->at(i).first.back());
            } else {
                denormalized_real_input = data->at(i).first.back();
            }

            std::stringstream ss_real_output;
            std::stringstream ss_predicted_output;
#endif

            double loc_error = 0;
            output_norm = 0;
            separator = "";
            for (size_t j = 0; j < dim_out; ++j) {
                if (denormalize_data) {
                    denormalized_real_output = data_set->get_normalization_strategy()->de_normalize(data->at(i).second.at(j));
                    denormalized_output = data_set->get_normalization_strategy()->de_normalize(outputs.at(i).at(j));
                } else {
                    denormalized_real_output = data->at(i).second.at(j);
                    denormalized_output = outputs.at(i).at(j);
                }

#ifdef DEBUG
                ss_real_output << separator << denormalized_real_output;
                ss_predicted_output << separator << denormalized_output;
                separator = ",";
#endif

                val = denormalized_output - denormalized_real_output;
                loc_error += val * val;
                error += loc_error;

                output_norm += denormalized_output * denormalized_output;
            }

#ifdef DEBUG
            std::stringstream ss_ind;
            ss_ind << "[" << i << "]";

            if (verbose) {
                COUT_DEBUG(R_ALIGN << ss_ind.str() << " "
                                   << R_ALIGN << ss_input.str() << " "
                                   << R_ALIGN << ss_real_output.str() << " "
                                   << R_ALIGN << ss_predicted_output.str() << " "
                                   << R_ALIGN << std::sqrt(loc_error) << " "
                                   << R_ALIGN
                                   << 200.0 * std::sqrt(loc_error) / (std::sqrt(loc_error) + std::sqrt(output_norm))
                                   << std::endl);
            }

            if (results_file_path) {
                *results_file_path << R_ALIGN << ss_ind.str() << " "
                                   << R_ALIGN << ss_input.str() << " "
                                   << R_ALIGN << ss_real_output.str() << " "
                                   << R_ALIGN << ss_predicted_output.str() << " "
                                   << R_ALIGN << std::sqrt(loc_error) << " "
                                   << R_ALIGN
                                   << 200.0 * std::sqrt(loc_error) / (std::sqrt(loc_error) + std::sqrt(output_norm))
                                   << std::endl;
            }
#endif
        }

        double result = std::sqrt(error) / n_elements;

        if (verbose) {
            COUT_DEBUG("MSE = " << result << std::endl);
        }

        if (results_file_path) {
            *results_file_path << "MSE = " << result << std::endl;
        }

        puts("END OF EVAL_ON_DATA_SET");
        return result;
    }

The strangest thing is, that before the program crashes, END OF EVAL_ON_DATA_SET is printed out, but END OF EVAL() is not, i.e. the program crashes with the end of the eval_on_data_set() function.

Is there any way to find out, what's happening?

Thank you very much for any advice.

EDIT

I've tried to use Memory Sanitizer according to the xaxxon's advice and here's the output:

=================================================================
==6515==ERROR: AddressSanitizer: alloc-dealloc-mismatch (INVALID vs operator delete) on 0x604000001ee8
    #0 0x7f2ad3a55118 in operator delete(void*, unsigned long) (/usr/lib/x86_64-linux-gnu/libasan.so.5+0xf1118)                                                                              
    #1 0x5589562f445b in std::_Sp_counted_ptr<std::vector<double, std::allocator<double> >*, (__gnu_cxx::_Lock_policy)2>::_M_dispose() /usr/include/c++/8/bits/shared_ptr_base.h:377
    #2 0x558956239de2 in std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() /usr/include/c++/8/bits/shared_ptr_base.h:155
    #3 0x5589562386dd in std::__shared_count<(__gnu_cxx::_Lock_policy)2>::~__shared_count() /usr/include/c++/8/bits/shared_ptr_base.h:706
    #4 0x5589562383f1 in std::__shared_ptr<std::vector<double, std::allocator<double> >, (__gnu_cxx::_Lock_policy)2>::~__shared_ptr() /usr/include/c++/8/bits/shared_ptr_base.h:1145
    #5 0x55895623840d in std::shared_ptr<std::vector<double, std::allocator<double> > >::~shared_ptr() /usr/include/c++/8/bits/shared_ptr.h:103
    #6 0x558956393484 in myProg::LearningSequence::optimize(myProg::ErrorFunction&, std::basic_ofstream<char, std::char_traits<char> >*) /home/martin/myProg/src/LearningMethods/LearningSequence.cpp:60
    #7 0x55895623685e in main /home/martin/myProg/src/examples/simulator_1_2.cpp:108
    #8 0x7f2ad31cf09a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2409a)
    #9 0x558956235669 in _start (/home/martin/myProg/build/bin/examples/simulator_1_2+0x4c669)

0x604000001ee8 is located 24 bytes inside of 48-byte region [0x604000001ed0,0x604000001f00)
allocated by thread T0 here:                                                                                                                                                                 
    #0 0x7f2ad3a53b60 in operator new(unsigned long) (/usr/lib/x86_64-linux-gnu/libasan.so.5+0xefb60)
    #1 0x558956241b31 in __gnu_cxx::new_allocator<std::_Sp_counted_ptr_inplace<myProg::RandomSolution, std::allocator<myProg::RandomSolution>, (__gnu_cxx::_Lock_policy)2> >::allocate(unsigned long, void const*) (/home/martin/myProg/build/bin/examples/simulator_1_2+0x58b31)
    #2 0x55895624128e in std::allocator_traits<std::allocator<std::_Sp_counted_ptr_inplace<myProg::RandomSolution, std::allocator<myProg::RandomSolution>, (__gnu_cxx::_Lock_policy)2> > >::allocate(std::allocator<std::_Sp_counted_ptr_inplace<myProg::RandomSolution, std::allocator<myProg::RandomSolution>, (__gnu_cxx::_Lock_policy)2> >&, unsigned long) (/home/martin/myProg/build/bin/examples/simulator_1_2+0x5828e)
    #3 0x558956240065 in std::__allocated_ptr<std::allocator<std::_Sp_counted_ptr_inplace<myProg::RandomSolution, std::allocator<myProg::RandomSolution>, (__gnu_cxx::_Lock_policy)2> > > std::__allocate_guarded<std::allocator<std::_Sp_counted_ptr_inplace<myProg::RandomSolution, std::allocator<myProg::RandomSolution>, (__gnu_cxx::_Lock_policy)2> > >(std::allocator<std::_Sp_counted_ptr_inplace<myProg::RandomSolution, std::allocator<myProg::RandomSolution>, (__gnu_cxx::_Lock_policy)2> >&) (/home/martin/myProg/build/bin/examples/simulator_1_2+0x57065)
    #4 0x55895623f2ac in std::__shared_count<(__gnu_cxx::_Lock_policy)2>::__shared_count<myProg::RandomSolution, std::allocator<myProg::RandomSolution>>(std::_Sp_make_shared_tag, myProg::RandomSolution*, std::allocator<myProg::RandomSolution> const&) (/home/martin/myProg/build/bin/examples/simulator_1_2+0x562ac)
    #5 0x55895623e4d8 in std::__shared_ptr<myProg::RandomSolution, (__gnu_cxx::_Lock_policy)2>::__shared_ptr<std::allocator<myProg::RandomSolution>>(std::_Sp_make_shared_tag, std::allocator<myProg::RandomSolution> const&) (/home/martin/myProg/build/bin/examples/simulator_1_2+0x554d8)
    #6 0x55895623cb90 in std::shared_ptr<myProg::RandomSolution>::shared_ptr<std::allocator<myProg::RandomSolution>>(std::_Sp_make_shared_tag, std::allocator<myProg::RandomSolution> const&) (/home/martin/myProg/build/bin/examples/simulator_1_2+0x53b90)
    #7 0x55895623ade5 in std::shared_ptr<myProg::RandomSolution> std::allocate_shared<myProg::RandomSolution, std::allocator<myProg::RandomSolution>>(std::allocator<myProg::RandomSolution> const&) (/home/martin/myProg/build/bin/examples/simulator_1_2+0x51de5)
    #8 0x5589562394a5 in std::shared_ptr<myProg::RandomSolution> std::make_shared<myProg::RandomSolution>() (/home/martin/myProg/build/bin/examples/simulator_1_2+0x504a5)
    #9 0x558956236538 in main /home/martin/myProg/src/examples/simulator_1_2.cpp:86
    #10 0x7f2ad31cf09a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2409a)

SUMMARY: AddressSanitizer: alloc-dealloc-mismatch (/usr/lib/x86_64-linux-gnu/libasan.so.5+0xf1118) in operator delete(void*, unsigned long)
==6515==HINT: if you don't care about these errors you may set ASAN_OPTIONS=alloc_dealloc_mismatch=0
==6515==ABORTING

The line 86 with the shared pointer declaration can be seen above and the optimize function too... So, I'm not sure, if I can get some more info this way.

You really need to show all the lines between where you create the shared_ptr and where your program crashes. And indicate exactly what happens within any functions that you call. Chances are that in your conversion process, you may have called `delete` on the shared_ptr's raw pointer. Or you may have undefined behavior / heap corruption due to a buffer overrun somewhere. It's very difficult to tell based on the scant information you have provided thus far. — paddy, Mar 21 '19 at 02:52
@paddy Well, I've tried to provide more code - if you need more information, I'll fill it in. — Eenoku, Mar 21 '19 at 03:11
The error messages smell like a memory corruption error, which is notoriously hard to track down. You need to search for out-of-bounds accesses in a part of the code unrelated to the part that's giving you the errors. — Mark Ransom, Mar 21 '19 at 03:18
@MarkRansom Do you think, it's possible to find it with some Valgrind flag? — Eenoku, Mar 21 '19 at 04:06
I highly recommend adding the compiler flag -fsanitize=address and running it again — xaxxon, Mar 21 '19 at 06:00
@xaxxon Thank you! I've tried it and appended the output into the question. — Eenoku, Mar 21 '19 at 09:33
@MarkRansom Could the newly added log from AddressSanitizer help you? — Eenoku, Mar 21 '19 at 11:41
It seems something is overwriting memory that belongs to `best_params`. My next step would be to run under gdb. Put a breakpoint right after `best_params` is initialized. When you get there, set hardware watchpoints (`watch` command) on `best_params` itself and also on the memory at which it points. `continue`, then each time a hardware watchpoint trigger, look at whether it makes sense for the calling code to be modifying `best_params` or its shared block. — aschepler, Mar 21 '19 at 12:26

Invalid free() / delete / delete[] / realloc() with std::shared_ptr

EDIT

0 Answers0