boost.python problem in release branch (GSoc work)
I just found out that two boost.python tests (svn trunk and release) are not valgrind clean. I fear this could be a serious problem, although I've not seen actual failures apart from the valgrind diagnostics. What I know at the moment is: libs/python/test/properties.py libs/python/test/data_members.py produce valgrind errors with trunk revision 56305 but not 56304 (fedora 8 64-bit). The critical revision 56305 is: Merged 2009 GSoC work from sandbox-branches/bhy/py3k branch back into trunk. It is merged into the release branch already. Unfortunately, without revision 56305 boost.python does not work with Python 2.6.3 and 2.6.4, i.e. backing out rev. 56305 and some followup revisions would only replace one problem with another. How far along is the 1.41 release? Is there a chance it could be delayed for a few days? Ralf P.S.: I'll add this issue to the tracker when I get a chance (hopefully tomorrow).
On Sat, Nov 7, 2009 at 2:44 PM, Ralf W. Grosse-Kunstleve <rwgk@yahoo.com> wrote:
Hi, I checked with valgrind on trunk, but didn't see any issue reported by valgrind. Maybe because I'm running on 32-bit Linux and Python 2.6 and 3.1? I have run valgrind in this way: (With PyObject_Free and PyObject_Realloc additionally suppressed) $ valgrind --suppressions=valgrind-python.supp python data_members.py ==15636== Memcheck, a memory error detector ==15636== Copyright (C) 2002-2009, and GNU GPL'd, by Julian Seward et al. ==15636== Using Valgrind-3.5.0 and LibVEX; rerun with -h for copyright info ==15636== Command: python data_members.py ==15636== running... Done. ==15636== ==15636== HEAP SUMMARY: ==15636== in use at exit: 1,723,204 bytes in 823 blocks ==15636== total heap usage: 11,524 allocs, 10,701 frees, 10,587,228 bytes allocated ==15636== ==15636== LEAK SUMMARY: ==15636== definitely lost: 0 bytes in 0 blocks ==15636== indirectly lost: 0 bytes in 0 blocks ==15636== possibly lost: 16,391 bytes in 35 blocks ==15636== still reachable: 1,706,813 bytes in 788 blocks ==15636== suppressed: 0 bytes in 0 blocks ==15636== Rerun with --leak-check=full to see details of leaked memory ==15636== ==15636== For counts of detected and suppressed errors, rerun with: -v ==15636== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 1513 from 89) Which Python version you are running? Anyway, since the problem occur in properties.py and data_members.py, I think it maybe related to this part of the changeset: https://svn.boost.org/trac/boost/changeset/56305/trunk/libs/python#file10 But I don't see any suspicious at here. Any idea? Thanks! -- Haoyu BAI School of Computing, National University of Singapore.
Hi Haoyu, Sorry for the incomplete report last night. It was very late here. Below is a valgrind output. This is with Python 2.6.2 compiled from sources under Fedora 8 x86_64. I've also seen the error with Python 2.6.4, I'm pretty sure, but I'll double check. I'll also try compiling everything with debug symbols to get line numbers. I'll let you know what I find. Ralf P.S.: My file with the valgrind suppressions is here: http://cctbx.svn.sourceforge.net/viewvc/cctbx/trunk/libtbx/valgrind-python24.supp?revision=4665&view=markup chevy:/net/chevy/raid1/rwgk/bintbx_hot % libtbx.valgrind python /net/chevy/raid1/rwgk/hot/boost/libs/python/test/properties.py valgrind-3.2.3 ### LIBTBX_VALGRIND not set: using default. ### To override, define LIBTBX_VALGRIND ### before calling libtbx.valgrind. LIBTBX_VALGRIND=valgrind --tool=memcheck --suppressions=/net/chevy/raid1/rwgk/dist/cctbx_project/libtbx/valgrind-python24.supp ==5967== Memcheck, a memory error detector. ==5967== Copyright (C) 2002-2007, and GNU GPL'd, by Julian Seward et al. ==5967== Using LibVEX rev 1732, a library for dynamic binary translation. ==5967== Copyright (C) 2004-2007, and GNU GPL'd, by OpenWorks LLP. ==5967== Using valgrind-3.2.3, a dynamic binary instrumentation framework. ==5967== Copyright (C) 2000-2007, and GNU GPL'd, by Julian Seward et al. ==5967== For more details, rerun with: -v ==5967== running... ==5967== Conditional jump or move depends on uninitialised value(s) ==5967== at 0x36472DB8B6: (within /usr/lib64/libpython2.5.so.1.0) ==5967== by 0x3647268F96: (within /usr/lib64/libpython2.5.so.1.0) ==5967== by 0x36472DBE37: (within /usr/lib64/libpython2.5.so.1.0) ==5967== by 0x36472DC7ED: _PyObject_GC_Malloc (in /usr/lib64/libpython2.5.so.1.0) ==5967== by 0x36472DC82F: _PyObject_GC_NewVar (in /usr/lib64/libpython2.5.so.1.0) ==5967== by 0x364727E976: PyTuple_New (in /usr/lib64/libpython2.5.so.1.0) ==5967== by 0x36472CAEAA: (within /usr/lib64/libpython2.5.so.1.0) ==5967== by 0x36472CB4CC: (within /usr/lib64/libpython2.5.so.1.0) ==5967== by 0x36472CAEF4: (within /usr/lib64/libpython2.5.so.1.0) ==5967== by 0x36472CB4B8: (within /usr/lib64/libpython2.5.so.1.0) ==5967== by 0x36472CBBAD: PyMarshal_ReadObjectFromString (in /usr/lib64/libpython2.5.so.1.0) ==5967== by 0x36472CD60A: PyMarshal_ReadLastObjectFromFile (in /usr/lib64/libpython2.5.so.1.0) ==5967== ==5967== Conditional jump or move depends on uninitialised value(s) ==5967== at 0x36472DB811: (within /usr/lib64/libpython2.5.so.1.0) ==5967== by 0x3647268F96: (within /usr/lib64/libpython2.5.so.1.0) ==5967== by 0x36472DBE8E: (within /usr/lib64/libpython2.5.so.1.0) ==5967== by 0x36472DC7ED: _PyObject_GC_Malloc (in /usr/lib64/libpython2.5.so.1.0) ==5967== by 0x36472DC82F: _PyObject_GC_NewVar (in /usr/lib64/libpython2.5.so.1.0) ==5967== by 0x364727E976: PyTuple_New (in /usr/lib64/libpython2.5.so.1.0) ==5967== by 0x36472CAEAA: (within /usr/lib64/libpython2.5.so.1.0) ==5967== by 0x36472CB4CC: (within /usr/lib64/libpython2.5.so.1.0) ==5967== by 0x36472CAEF4: (within /usr/lib64/libpython2.5.so.1.0) ==5967== by 0x36472CB4B8: (within /usr/lib64/libpython2.5.so.1.0) ==5967== by 0x36472CBBAD: PyMarshal_ReadObjectFromString (in /usr/lib64/libpython2.5.so.1.0) ==5967== by 0x36472CD60A: PyMarshal_ReadLastObjectFromFile (in /usr/lib64/libpython2.5.so.1.0) Done. ==5967== ==5967== Conditional jump or move depends on uninitialised value(s) ==5967== at 0x36472DB8B6: (within /usr/lib64/libpython2.5.so.1.0) ==5967== by 0x3647268F96: (within /usr/lib64/libpython2.5.so.1.0) ==5967== by 0x36472DBE37: (within /usr/lib64/libpython2.5.so.1.0) ==5967== by 0x36472DC713: PyGC_Collect (in /usr/lib64/libpython2.5.so.1.0) ==5967== by 0x36472D15F6: Py_Finalize (in /usr/lib64/libpython2.5.so.1.0) ==5967== by 0x36472D1717: Py_Exit (in /usr/lib64/libpython2.5.so.1.0) ==5967== by 0x36472D1824: (within /usr/lib64/libpython2.5.so.1.0) ==5967== by 0x36472D1A57: PyErr_PrintEx (in /usr/lib64/libpython2.5.so.1.0) ==5967== by 0x36472D1E1C: PyRun_SimpleFileExFlags (in /usr/lib64/libpython2.5.so.1.0) ==5967== by 0x36472DB26D: Py_Main (in /usr/lib64/libpython2.5.so.1.0) ==5967== by 0x363241E073: (below main) (in /lib64/libc-2.7.so) ==5967== ==5967== Conditional jump or move depends on uninitialised value(s) ==5967== at 0x36472DB811: (within /usr/lib64/libpython2.5.so.1.0) ==5967== by 0x3647268F96: (within /usr/lib64/libpython2.5.so.1.0) ==5967== by 0x36472DBE8E: (within /usr/lib64/libpython2.5.so.1.0) ==5967== by 0x36472DC713: PyGC_Collect (in /usr/lib64/libpython2.5.so.1.0) ==5967== by 0x36472D15F6: Py_Finalize (in /usr/lib64/libpython2.5.so.1.0) ==5967== by 0x36472D1717: Py_Exit (in /usr/lib64/libpython2.5.so.1.0) ==5967== by 0x36472D1824: (within /usr/lib64/libpython2.5.so.1.0) ==5967== by 0x36472D1A57: PyErr_PrintEx (in /usr/lib64/libpython2.5.so.1.0) ==5967== by 0x36472D1E1C: PyRun_SimpleFileExFlags (in /usr/lib64/libpython2.5.so.1.0) ==5967== by 0x36472DB26D: Py_Main (in /usr/lib64/libpython2.5.so.1.0) ==5967== by 0x363241E073: (below main) (in /lib64/libc-2.7.so) ==5967== ==5967== ERROR SUMMARY: 12 errors from 4 contexts (suppressed: 1433 from 7) ==5967== malloc/free: in use at exit: 2,130,538 bytes in 784 blocks. ==5967== malloc/free: 8,392 allocs, 7,608 frees, 8,389,344 bytes allocated. ==5967== For counts of detected errors, rerun with: -v ==5967== searching for pointers to 784 not-freed blocks. ==5967== checked 2,450,896 bytes. ==5967== ==5967== LEAK SUMMARY: ==5967== definitely lost: 0 bytes in 0 blocks. ==5967== possibly lost: 7,768 bytes in 24 blocks. ==5967== still reachable: 2,122,770 bytes in 760 blocks. ==5967== suppressed: 0 bytes in 0 blocks. ==5967== Rerun with --leak-check=full to see details of leaked memory.
Sorry, I took the output from the wrong build. In this build I was using /usr/bin/python instead of a Python compiled from sources. But there was no mix of Python versions. In the meantime I've sent Haoyu more info based on a Python 2.6.4 debug build. Ralf ----- Original Message ---- From: Steven Watanabe <watanabesj@gmail.com> To: boost@lists.boost.org Sent: Sat, November 7, 2009 12:37:19 PM Subject: Re: [boost] boost.python problem in release branch (GSoc work) AMDG Ralf W. Grosse-Kunstleve wrote:
Are the errors because of mixed python versions?
On Sat, Nov 7, 2009 at 1:44 AM, Ralf W. Grosse-Kunstleve <rwgk@yahoo.com> wrote:
How far along is the 1.41 release? Is there a chance it could be delayed for a few days?
We are planning a second beta in a week. See "[1.41.0] Beta 2 plans" Hopefully that will give you enough time to clear the problem. --Beman
For Beman (and the records): This problem is now resolved, thanks to the thorough work of Troy Straszheim. Troy found out that the problem was due to a very long-standing bug exposed by recent changes in Python. It was just an unfortunate coincidence that the GSoC changes came at the same time. Ralf ----- Original Message ---- From: Ralf W. Grosse-Kunstleve <rwgk@yahoo.com> To: boost@lists.boost.org Sent: Fri, November 6, 2009 10:44:42 PM Subject: [boost] boost.python problem in release branch (GSoc work) I just found out that two boost.python tests (svn trunk and release) are not valgrind clean. I fear this could be a serious problem, although I've not seen actual failures apart from the valgrind diagnostics. What I know at the moment is: libs/python/test/properties.py libs/python/test/data_members.py produce valgrind errors with trunk revision 56305 but not 56304 (fedora 8 64-bit). The critical revision 56305 is: Merged 2009 GSoC work from sandbox-branches/bhy/py3k branch back into trunk. It is merged into the release branch already. Unfortunately, without revision 56305 boost.python does not work with Python 2.6.3 and 2.6.4, i.e. backing out rev. 56305 and some followup revisions would only replace one problem with another. How far along is the 1.41 release? Is there a chance it could be delayed for a few days? Ralf
On Thu, Nov 12, 2009 at 7:49 PM, Ralf W. Grosse-Kunstleve <rwgk@yahoo.com> wrote:
For Beman (and the records):
This problem is now resolved, thanks to the thorough work of Troy Straszheim. Troy found out that the problem was due to a very long-standing bug exposed by recent changes in Python. It was just an unfortunate coincidence that the GSoC changes came at the same time.
Thanks for the follow up report! --Beman
participants (4)
-
Beman Dawes -
Haoyu Bai -
Ralf W. Grosse-Kunstleve -
Steven Watanabe