[serialization] Memory leaks even with shared_ptrs
Hi All, Sorry for posting twice in two days, but I'm having lots of trouble getting this right. Robert points out in the bus route example that the memory leaks can be solved using shared_ptrs (bottom of this page: http://www.boost.org/doc/libs/1_47_0/libs/serialization/doc/tutorial.html#ex... ) I have a basic example (below), quite like the bus route, that has memory leaks when using raw pointers. I've converted it to shared_ptrs but the leaks are exactly the same. Where have I gone wrong? I'm using VS2010 (hence the memory leak headers & function call to _CrtDumpMemoryLeaks();). Any help much appreciated! -------------------------------------------------------- Example that replicates problem: #define _CRTDBG_MAP_ALLOC #include <stdlib.h> #include <crtdbg.h> #include <boost/archive/xml_iarchive.hpp> #include <boost/archive/xml_oarchive.hpp> #include <boost/serialization/string.hpp> #include <boost/serialization/vector.hpp> #include <boost/serialization/export.hpp> #include <boost/serialization/shared_ptr.hpp> #include <boost/shared_ptr.hpp> #include <boost/make_shared.hpp> #include <iostream> #include <fstream> #include <string> class Distribution { public: Distribution(void) {}; ~Distribution(void) {}; virtual std::string getType() = 0; protected: std::string m_Type; private: friend class boost::serialization::access; template<class Archive> void serialize(Archive& ar, const unsigned int version) {}; }; BOOST_SERIALIZATION_ASSUME_ABSTRACT(Distribution) class ContinuousDist : public Distribution { public: ContinuousDist() { m_Type = "Continuous"; m_Location = 80.0; m_Scale = 5.0; m_Shape = 0.0; }; ~ContinuousDist(void) {}; virtual std::string getType() {return m_Type;}; private: double m_Location; double m_Scale; double m_Shape; friend class boost::serialization::access; template<class Archive> void serialize(Archive& ar, const unsigned int version) { ar & BOOST_SERIALIZATION_BASE_OBJECT_NVP(Distribution) & BOOST_SERIALIZATION_NVP(m_Type) & BOOST_SERIALIZATION_NVP(m_Location) & BOOST_SERIALIZATION_NVP(m_Scale) & BOOST_SERIALIZATION_NVP(m_Shape); }; }; BOOST_CLASS_EXPORT(ContinuousDist); class DiscreteDist : public Distribution { public: DiscreteDist(void) { m_Type = "Discrete"; m_Data.push_back(1.2); m_Data.push_back(2.5); m_PMF.push_back(0.8); m_PMF.push_back(0.2); }; ~DiscreteDist(void){}; virtual std::string getType() {return m_Type;}; private: std::vector<double> m_Data; std::vector<double> m_PMF; friend class boost::serialization::access; template<class Archive> void serialize(Archive& ar, const unsigned int version) { ar & BOOST_SERIALIZATION_BASE_OBJECT_NVP(Distribution) & BOOST_SERIALIZATION_NVP(m_Type) & BOOST_SERIALIZATION_NVP(m_Data) & BOOST_SERIALIZATION_NVP(m_PMF); }; }; BOOST_CLASS_EXPORT(DiscreteDist); typedef boost::shared_ptr<Distribution> DistributionPtr; class Parameter { public: Parameter() { m_ID = 0; }; Parameter(int id, int type) { m_ID = id; if(type == 1) m_pDist = boost::make_shared<DiscreteDist>(); else m_pDist = boost::make_shared<ContinuousDist>(); }; ~Parameter(void){}; DistributionPtr getDist() {return m_pDist;}; int getID() {return m_ID;}; private: DistributionPtr m_pDist; int m_ID; friend class boost::serialization::access; template<class Archive> void serialize(Archive& ar, const unsigned int version) { ar & BOOST_SERIALIZATION_NVP(m_ID) & BOOST_SERIALIZATION_NVP(m_pDist); }; }; typedef boost::shared_ptr<Parameter> ParameterPtr; void load(std::vector<ParameterPtr>& vP, std::string file) { std::ifstream ifs(file.c_str()); boost::archive::xml_iarchive ia(ifs); ia >> BOOST_SERIALIZATION_NVP(vP); } void save(std::vector<ParameterPtr>& vP, std::string file) { std::ofstream ofs(file.c_str()); boost::archive::xml_oarchive oa(ofs); oa << BOOST_SERIALIZATION_NVP(vP); } int main() { // tbe file to read/save from/to std::string file = "archive.xml"; // make some Parmeters std::vector<ParameterPtr> vParam; vParam.push_back( boost::make_shared<Parameter>(1,1) ); vParam.push_back( boost::make_shared<Parameter>(2,2) ); // display for checking for(unsigned int i = 0; i != vParam.size(); ++i) std::cout << "P: " << vParam.at(i)->getDist()->getType() << std::endl; // save the file std::cout << "Writing..." << std::endl; save(vParam,file); // read the file std::cout << "Reading..." << std::endl; load(vParam,file); // display for checking for(unsigned int i = 0; i != vParam.size(); ++i) std::cout << "P: " << vParam.at(i)->getDist()->getType() << std::endl; std::cin.get(); // hold console window open _CrtDumpMemoryLeaks(); // report memory leaks }
On 07/21/2011 04:02 PM, cc caprani wrote:
int main() { [...] std::cin.get(); // hold console window open _CrtDumpMemoryLeaks(); // report memory leaks }
Hi, I don't know how _CrtDumpMemoryLeaks() works, but at the call point nothing has been deleted yet as nothing went out of scope. Try putting what is in your main() currently into another scope, for example: int main() { { // ... all your code ... } // Now that we exited the previous scope, everything should've // been deleted... std::cin.get(); _CrtDumpMemoryLeaks(); } -- Maxime
On Thu, Jul 21, 2011 at 3:15 PM, Maxime van Noppen <maxime@altribe.org>wrote:
On 07/21/2011 04:02 PM, cc caprani wrote:
int main() {
[...]
std::cin.get(); // hold console window open
_CrtDumpMemoryLeaks(); // report memory leaks }
<smacks forehead> Oh that could have nearly been embarrassing! Luckily I had actually done that in a previous test - the problem still remains when scoped correctly as suggested. Thanks for the input Maxine.
On Thu, Jul 21, 2011 at 4:26 PM, cc caprani <cc.caprani@gmail.com> wrote:
<smacks forehead> Oh that could have nearly been embarrassing!
Luckily I had actually done that in a previous test - the problem still remains when scoped correctly as suggested.
Thanks for the input Maxine. <- apologies, Maxime.
After some more work on it, I realize that some of the leaks were incorrectly reported by _CrtDumpMemoryLeaks(); as the BOOST_CLASS_EXPORT is only tidied up as the program exits. The very last comment on this page is key: http://msdn.microsoft.com/en-us/library/e5ewb1h3%28v=vs.80%29.aspx. This very simple program still leaks one copy of everything loaded from the archive. The new main() for the previously posted code with improved leak detection is: int main() { _CrtSetDbgFlag ( _CRTDBG_ALLOC_MEM_DF | _CRTDBG_LEAK_CHECK_DF ); // the file to read/save from/to std::string file = "archive.xml"; // make some Parmeters std::vector<ParameterPtr> vParam; vParam.push_back( boost::make_shared<Parameter>(1,1) ); vParam.push_back( boost::make_shared<Parameter>(2,2) ); // display for checking for(unsigned int i = 0; i != vParam.size(); ++i) std::cout << "P: " << vParam.at(i)->getDist()->getType() << std::endl; // save the file std::cout << "Writing..." << std::endl; save(vParam,file); // clear it vParam.clear(); // read the file std::cout << "Reading..." << std::endl; load(vParam,file); // display for checking for(unsigned int i = 0; i != vParam.size(); ++i) std::cout << "P: " << vParam.at(i)->getDist()->getType() << std::endl; std::cin.get(); // hold console window open return 0; }
On Thu, Jul 21, 2011 at 8:32 PM, cc caprani <cc.caprani@gmail.com> wrote:
This very simple program still leaks one copy of everything loaded from the archive.
I've done some more on this and the leak only occurs if a virtual member function is declared in the base class. It doesn't matter if the function is pure virtual or concrete, and whether or not BOOST_SERIALIZATION_ASSUME_ABSTRACT is used. Turning off both virtual base class member functions removes the leak. At this stage I think it must be a bug - should I form a tracker for it? Or am I not understanding something? Here is a minimal program to replicate the leak (using VS2010, boost 1.47). #include <boost/archive/xml_iarchive.hpp> #include <boost/archive/xml_oarchive.hpp> #include <boost/serialization/vector.hpp> #include <boost/serialization/export.hpp> #include <boost/serialization/shared_ptr.hpp> #include <boost/shared_ptr.hpp> #include <boost/make_shared.hpp> #include <iostream> #include <fstream> #include <crtdbg.h> class Distribution { public: Distribution(void) {}; ~Distribution(void) {}; // turn on either to get memory leak virtual int getType() = 0; //virtual int getType() {return 1;}; private: friend class boost::serialization::access; template<class Archive> void serialize(Archive& ar, const unsigned int version) {}; }; typedef boost::shared_ptr<Distribution> DistributionPtr; BOOST_SERIALIZATION_ASSUME_ABSTRACT(Distribution) class ContinuousDist : public Distribution { public: ContinuousDist(){}; ~ContinuousDist(void) {}; virtual int getType() {return 1;}; private: friend class boost::serialization::access; template<class Archive> void serialize(Archive& ar, const unsigned int version) { ar & BOOST_SERIALIZATION_BASE_OBJECT_NVP(Distribution); }; }; BOOST_CLASS_EXPORT(ContinuousDist); class DiscreteDist : public Distribution { public: DiscreteDist(void) { m_Data.push_back(4.0); m_Data.push_back(8.0); }; ~DiscreteDist(void){}; virtual int getType() {return 2;}; private: std::vector<double> m_Data; friend class boost::serialization::access; template<class Archive> void serialize(Archive& ar, const unsigned int version) { ar & BOOST_SERIALIZATION_BASE_OBJECT_NVP(Distribution) & BOOST_SERIALIZATION_NVP(m_Data); }; }; BOOST_CLASS_EXPORT(DiscreteDist); class Parameter { public: Parameter(){}; Parameter(int type) { if(type == 1) m_pDist = boost::make_shared<DiscreteDist>(); else m_pDist = boost::make_shared<ContinuousDist>(); }; ~Parameter(void){}; private: DistributionPtr m_pDist; friend class boost::serialization::access; template<class Archive> void serialize(Archive& ar, const unsigned int version) { ar & BOOST_SERIALIZATION_NVP(m_pDist); }; }; typedef boost::shared_ptr<Parameter> ParameterPtr; void load(std::vector<ParameterPtr>& vP, std::string file) { std::ifstream ifs(file.c_str()); boost::archive::xml_iarchive ia(ifs); ia >> BOOST_SERIALIZATION_NVP(vP); } void save(std::vector<ParameterPtr>& vP, std::string file) { std::ofstream ofs(file.c_str()); boost::archive::xml_oarchive oa(ofs); oa << BOOST_SERIALIZATION_NVP(vP); } int main() { _CrtSetDbgFlag ( _CRTDBG_ALLOC_MEM_DF | _CRTDBG_LEAK_CHECK_DF ); // the file to read/save from/to std::string file = "archive.xml"; // make some Parmeters std::vector<ParameterPtr> vParam; vParam.push_back( boost::make_shared<Parameter>(1) ); vParam.push_back( boost::make_shared<Parameter>(2) ); // save the file std::cout << "Writing..." << std::endl; save(vParam,file); // clear it vParam.clear(); // read the file std::cout << "Reading..." << std::endl; load(vParam,file); return 0; }
when you create a Base class, make sure that the destructor is to be declared as virtual. On Fri, Jul 22, 2011 at 04:55, cc caprani <cc.caprani@gmail.com> wrote:
On Thu, Jul 21, 2011 at 8:32 PM, cc caprani <cc.caprani@gmail.com> wrote:
This very simple program still leaks one copy of everything loaded from the archive.
I've done some more on this and the leak only occurs if a virtual member function is declared in the base class. It doesn't matter if the function is pure virtual or concrete, and whether or not BOOST_SERIALIZATION_ASSUME_ABSTRACT is used. Turning off both virtual base class member functions removes the leak.
At this stage I think it must be a bug - should I form a tracker for it? Or am I not understanding something?
Here is a minimal program to replicate the leak (using VS2010, boost 1.47).
#include <boost/archive/xml_iarchive.hpp> #include <boost/archive/xml_oarchive.hpp> #include <boost/serialization/vector.hpp> #include <boost/serialization/export.hpp> #include <boost/serialization/shared_ptr.hpp> #include <boost/shared_ptr.hpp> #include <boost/make_shared.hpp> #include <iostream> #include <fstream> #include <crtdbg.h>
class Distribution { public: Distribution(void) {}; ~Distribution(void) {}; // turn on either to get memory leak virtual int getType() = 0; //virtual int getType() {return 1;};
private: friend class boost::serialization::access; template<class Archive> void serialize(Archive& ar, const unsigned int version) {}; }; typedef boost::shared_ptr<Distribution> DistributionPtr; BOOST_SERIALIZATION_ASSUME_ABSTRACT(Distribution)
class ContinuousDist : public Distribution { public: ContinuousDist(){}; ~ContinuousDist(void) {}; virtual int getType() {return 1;};
private: friend class boost::serialization::access; template<class Archive> void serialize(Archive& ar, const unsigned int version) { ar & BOOST_SERIALIZATION_BASE_OBJECT_NVP(Distribution); };
}; BOOST_CLASS_EXPORT(ContinuousDist);
class DiscreteDist : public Distribution { public: DiscreteDist(void) { m_Data.push_back(4.0); m_Data.push_back(8.0); }; ~DiscreteDist(void){}; virtual int getType() {return 2;};
private: std::vector<double> m_Data;
friend class boost::serialization::access; template<class Archive> void serialize(Archive& ar, const unsigned int version) { ar & BOOST_SERIALIZATION_BASE_OBJECT_NVP(Distribution) & BOOST_SERIALIZATION_NVP(m_Data); }; }; BOOST_CLASS_EXPORT(DiscreteDist);
class Parameter { public: Parameter(){}; Parameter(int type)
{ if(type == 1) m_pDist = boost::make_shared<DiscreteDist>(); else m_pDist = boost::make_shared<ContinuousDist>(); }; ~Parameter(void){};
private: DistributionPtr m_pDist;
friend class boost::serialization::access; template<class Archive> void serialize(Archive& ar, const unsigned int version) { ar & BOOST_SERIALIZATION_NVP(m_pDist); };
}; typedef boost::shared_ptr<Parameter> ParameterPtr;
void load(std::vector<ParameterPtr>& vP, std::string file) { std::ifstream ifs(file.c_str()); boost::archive::xml_iarchive ia(ifs); ia >> BOOST_SERIALIZATION_NVP(vP); }
void save(std::vector<ParameterPtr>& vP, std::string file) { std::ofstream ofs(file.c_str()); boost::archive::xml_oarchive oa(ofs); oa << BOOST_SERIALIZATION_NVP(vP); }
int main() { _CrtSetDbgFlag ( _CRTDBG_ALLOC_MEM_DF | _CRTDBG_LEAK_CHECK_DF );
// the file to read/save from/to std::string file = "archive.xml";
// make some Parmeters std::vector<ParameterPtr> vParam; vParam.push_back( boost::make_shared<Parameter>(1) ); vParam.push_back( boost::make_shared<Parameter>(2) );
// save the file std::cout << "Writing..." << std::endl; save(vParam,file);
// clear it vParam.clear();
// read the file std::cout << "Reading..." << std::endl; load(vParam,file);
return 0; }
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
On Fri, Jul 22, 2011 at 9:02 AM, Hazrat Pradipta Ranjali <mrdipta@gmail.com>wrote:
when you create a Base class, make sure that the destructor is to be declared as virtual.
Thank you! It worked a treat. Scott V (supremegalacticoverlord) also emailed me this solution. I clearly have some reading to do :)
participants (3)
-
cc caprani
-
Hazrat Pradipta Ranjali
-
Maxime van Noppen