[Libtorrent-devel] Unaddressed memory issue

In Incognito in.incognito at gmail.com
Tue Jan 8 22:12:24 UTC 2008


I ran valgrind with -v for about 45 mins. Instead of the brief duration i
usually ran it at.

==6456== malloc/free: 2,556,094 allocs, 2,371,126 frees, 89,541,842 bytes
allocated.

According to valgrind docs, allocs should equal frees.

==6456== LEAK SUMMARY:
==6456==    definitely lost: 296,352 bytes in 9,261 blocks.
==6456==    indirectly lost: 5,602,116 bytes in 175,273 blocks.
==6456==      possibly lost: 1,277 bytes in 34 blocks.
==6456==    still reachable: 582,196 bytes in 400 blocks.
==6456==         suppressed: 0 bytes in 0 blocks.

it lost 6mb in less than an hour!

Below's cut/paste of certain sections of the log. Alot of stuff about xmlrpc
is in there,

==6456== 32 bytes in 2 blocks are possibly lost in loss record 7 of 39
==6456==    at 0x40053C0: malloc (vg_replace_malloc.c:149)
==6456==    by 0x425C5B5: xmlrpc_mem_block_init (memblock.c:74)
==6456==    by 0x4251056: stringNew (xmlrpc_string.c:666)
==6456==    by 0x80F7FF5: rpc::object_to_xmlrpc(_xmlrpc_env*,
torrent::Object const&) (xmlrpc.cc:354)
==6456==    by 0x80F7FF5: rpc::object_to_xmlrpc(_xmlrpc_env*,
torrent::Object const&) (xmlrpc.cc:354)
==6456==    by 0x80F9909: rpc::xmlrpc_call_command(_xmlrpc_env*,
_xmlrpc_value*, void*) (xmlrpc.cc:391)
==6456==    by 0x42481EF: xmlrpc_dispatchCall (registry.c:239)
==6456==    by 0x424835D: xmlrpc_registry_process_call2 (registry.c:354)
==6456==    by 0x4248490: xmlrpc_registry_process_call (registry.c:394)
==6456==    by 0x80F7A87: rpc::XmlRpc::process(char const*, unsigned,
rak::function2<bool, char const*, unsigned>) (xmlrpc.cc:426)
==6456==    by 0x8071951: rak::mem_fn3_t<rpc::XmlRpc, bool, char const*,
unsigned, rak::function2<bool, char const*, unsigned> >::operator()(char
const*, unsigned, rak::function2<bool, char const*, unsigned>)
(functional_fun.h:239)
==6456==    by 0x80F60B4: rpc::SCgi::receive_call(rpc::SCgiTask*, char
const*, unsigned) (functional_fun.h:153)

==6456== 128 bytes in 4 blocks are definitely lost in loss record 9 of 39
==6456==    at 0x40057F5: operator new[](unsigned) (vg_replace_malloc.c:195)
==6456==    by 0x40CE9BC:
torrent::ProtocolExtension::generate_toggle_message(torrent::ProtocolExtension::MessageType,
bool) (extensions.cc:136)
==6456==    by 0x40D8FD4: torrent::PeerConnectionBase::send_pex_message()
(peer_connection_base.cc:833)
==6456==    by 0x40DF7A4: torrent::PeerConnectionSeed::event_write()
(peer_connection_seed.cc:307)
==6456==    by 0x4077309: torrent::PollEPoll::perform() (poll_epoll.cc:138)
==6456==    by 0x80C291D: core::PollManagerEPoll::poll(rak::timer)
(poll_manager_epoll.cc:112)
==6456==    by 0x807BC7D: main (main.cc:277)

==6456== 768 bytes in 24 blocks are possibly lost in loss record 22 of 39
==6456==    at 0x40053C0: malloc (vg_replace_malloc.c:149)
==6456==    by 0x424FDE2: xmlrpc_createXmlrpcValue (xmlrpc_data.c:319)
==6456==    by 0x4250044: xmlrpc_i8_new (xmlrpc_data.c:354)
==6456==    by 0x80F7F83: rpc::object_to_xmlrpc(_xmlrpc_env*,
torrent::Object const&) (xmlrpc.cc:341)
==6456==    by 0x80F7FF5: rpc::object_to_xmlrpc(_xmlrpc_env*,
torrent::Object const&) (xmlrpc.cc:354)
==6456==    by 0x80F7FF5: rpc::object_to_xmlrpc(_xmlrpc_env*,
torrent::Object const&) (xmlrpc.cc:354)
==6456==    by 0x80F9909: rpc::xmlrpc_call_command(_xmlrpc_env*,
_xmlrpc_value*, void*) (xmlrpc.cc:391)
==6456==    by 0x42481EF: xmlrpc_dispatchCall (registry.c:239)
==6456==    by 0x424835D: xmlrpc_registry_process_call2 (registry.c:354)
==6456==    by 0x4248490: xmlrpc_registry_process_call (registry.c:394)
==6456==    by 0x80F7A87: rpc::XmlRpc::process(char const*, unsigned,
rak::function2<bool, char const*, unsigned>) (xmlrpc.cc:426)
==6456==    by 0x8071951: rak::mem_fn3_t<rpc::XmlRpc, bool, char const*,
unsigned, rak::function2<bool, char const*, unsigned> >::operator()(char
const*, unsigned, rak::function2<bool, char const*, unsigned>)
(functional_fun.h:239)
==6456==

==6456== 468,420 bytes in 11,669 blocks are indirectly lost in loss record
35 of 39
==6456==    at 0x40053C0: malloc (vg_replace_malloc.c:149)
==6456==    by 0x425C5D1: xmlrpc_mem_block_init (memblock.c:74)
==6456==    by 0x4251056: stringNew (xmlrpc_string.c:666)
==6456==    by 0x80F7FF5: rpc::object_to_xmlrpc(_xmlrpc_env*,
torrent::Object const&) (xmlrpc.cc:354)
==6456==    by 0x80F7FF5: rpc::object_to_xmlrpc(_xmlrpc_env*,
torrent::Object const&) (xmlrpc.cc:354)
==6456==    by 0x80F9909: rpc::xmlrpc_call_command(_xmlrpc_env*,
_xmlrpc_value*, void*) (xmlrpc.cc:391)
==6456==    by 0x42481EF: xmlrpc_dispatchCall (registry.c:239)
==6456==    by 0x424835D: xmlrpc_registry_process_call2 (registry.c:354)
==6456==    by 0x4248490: xmlrpc_registry_process_call (registry.c:394)
==6456==    by 0x80F7A87: rpc::XmlRpc::process(char const*, unsigned,
rak::function2<bool, char const*, unsigned>) (xmlrpc.cc:426)
==6456==    by 0x8071951: rak::mem_fn3_t<rpc::XmlRpc, bool, char const*,
unsigned, rak::function2<bool, char const*, unsigned> >::operator()(char
const*, unsigned, rak::function2<bool, char const*, unsigned>)
(functional_fun.h:239)
==6456==    by 0x80F60B4: rpc::SCgi::receive_call(rpc::SCgiTask*, char
const*, unsigned) (functional_fun.h:153)
==6456==
==6456==
==6456== 4,113,472 bytes in 128,546 blocks are indirectly lost in loss
record 36 of 39
==6456==    at 0x40053C0: malloc (vg_replace_malloc.c:149)
==6456==    by 0x424FDE2: xmlrpc_createXmlrpcValue (xmlrpc_data.c:319)
==6456==    by 0x4250044: xmlrpc_i8_new (xmlrpc_data.c:354)
==6456==    by 0x80F7F83: rpc::object_to_xmlrpc(_xmlrpc_env*,
torrent::Object const&) (xmlrpc.cc:341)
==6456==    by 0x80F7FF5: rpc::object_to_xmlrpc(_xmlrpc_env*,
torrent::Object const&) (xmlrpc.cc:354)
==6456==    by 0x80F7FF5: rpc::object_to_xmlrpc(_xmlrpc_env*,
torrent::Object const&) (xmlrpc.cc:354)
==6456==    by 0x80F9909: rpc::xmlrpc_call_command(_xmlrpc_env*,
_xmlrpc_value*, void*) (xmlrpc.cc:391)
==6456==    by 0x42481EF: xmlrpc_dispatchCall (registry.c:239)
==6456==    by 0x424835D: xmlrpc_registry_process_call2 (registry.c:354)
==6456==    by 0x4248490: xmlrpc_registry_process_call (registry.c:394)
==6456==    by 0x80F7A87: rpc::XmlRpc::process(char const*, unsigned,
rak::function2<bool, char const*, unsigned>) (xmlrpc.cc:426)
==6456==    by 0x8071951: rak::mem_fn3_t<rpc::XmlRpc, bool, char const*,
unsigned, rak::function2<bool, char const*, unsigned> >::operator()(char
const*, unsigned, rak::function2<bool, char const*, unsigned>)
(functional_fun.h:239)

==6456== 5,898,340 (296,224 direct, 5,602,116 indirect) bytes in 9,257
blocks are definitely lost in loss record 37 of 39
==6456==    at 0x40053C0: malloc (vg_replace_malloc.c:149)
==6456==    by 0x424FDE2: xmlrpc_createXmlrpcValue (xmlrpc_data.c:319)
==6456==    by 0x4252318: xmlrpc_array_new (xmlrpc_array.c:191)
==6456==    by 0x80F7F18: rpc::object_to_xmlrpc(_xmlrpc_env*,
torrent::Object const&) (xmlrpc.cc:351)
==6456==    by 0x80F7FF5: rpc::object_to_xmlrpc(_xmlrpc_env*,
torrent::Object const&) (xmlrpc.cc:354)
==6456==    by 0x80F9909: rpc::xmlrpc_call_command(_xmlrpc_env*,
_xmlrpc_value*, void*) (xmlrpc.cc:391)
==6456==    by 0x42481EF: xmlrpc_dispatchCall (registry.c:239)
==6456==    by 0x424835D: xmlrpc_registry_process_call2 (registry.c:354)
==6456==    by 0x4248490: xmlrpc_registry_process_call (registry.c:394)
==6456==    by 0x80F7A87: rpc::XmlRpc::process(char const*, unsigned,
rak::function2<bool, char const*, unsigned>) (xmlrpc.cc:426)
==6456==    by 0x8071951: rak::mem_fn3_t<rpc::XmlRpc, bool, char const*,
unsigned, rak::function2<bool, char const*, unsigned> >::operator()(char
const*, unsigned, rak::function2<bool, char const*, unsigned>)
(functional_fun.h:239)
==6456==    by 0x80F60B4: rpc::SCgi::receive_call(rpc::SCgiTask*, char
const*, unsigned) (functional_fun.h:153)


On Jan 8, 2008 3:57 PM, In Incognito <in.incognito at gmail.com> wrote:

> 160 bytes isn't much but over a period of say 30 hours, it could add up.
>
> My memory problem could be just bad use of memory by rtorrent. I never
> said it was some mem leak.
>
> Other folks here said it was a bug with the version of gcc and libstdc++ i
> was using. So i recompiled everything with gcc 4.2.2. Previous version was
> 4.1.2. The memory problems are still here with the new gcc.
>
>
>
> On Jan 8, 2008 3:32 PM, Josef Drexler <josef at ttdpatch.net > wrote:
>
> > On Jan 8, 2008, at 9:04 PM, In Incognito wrote:
> > > I ran valgrind on the newly compiled rtorrent/libtorrent.
> > >
> > > ==5961== ERROR SUMMARY: 3061122 errors from 8 contexts (suppressed:
> > > 52 from 1)
> > > ==5961== malloc/free: in use at exit: 576,825 bytes in 229 blocks.
> > > ==5961== malloc/free: 146,820 allocs, 146,591 frees, 13,357,268
> > > bytes allocated.
> > > ==5961== For counts of detected errors, rerun with: -v
> > > ==5961== searching for pointers to 229 not-freed blocks.
> > > ==5961== checked 850,956 bytes.
> > >
> > > allocs is greater than frees, according to the summary.
> >
> > That is not generally a problem, so long as the memory is still in
> > use when the program exits and it isn't a leak (which refers to now-
> > unused memory that was forgotten to be de-allocated). However,
> >
> > > ==5961== 160 bytes in 5 blocks are definitely lost in loss record 8
> > > of 26
> > > ==5961==    at 0x40057F5: operator new[](unsigned)
> > > (vg_replace_malloc.c:195)
> > > ==5961==    by 0x40CE9BC:
> > > torrent::ProtocolExtension::generate_toggle_message
> > > (torrent::ProtocolExtension::MessageType, bool) (extensions.cc:136)
> >
> > This is an actual memory leak, all 160 bytes of it. Thanks for
> > reporting this, I know how to fix it.
> >
> > Note that valgrind won't notice memory used for transferring files,
> > since that memory isn't allocated, but is simply using mmap to map
> > files into the process's address space, even though that memory is by
> > far the largest amount used by rtorrent, which may be surprising.
> >
> > --
> > Josef Drexler
> > josef at ttdpatch.net
> >
> >
> >
> > _______________________________________________
> > Libtorrent-devel mailing list
> > Libtorrent-devel at rakshasa.no
> > http://rakshasa.no/mailman/listinfo/libtorrent-devel
> >
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://rakshasa.no/pipermail/libtorrent-devel/attachments/20080108/75a7bf6f/attachment.html


More information about the Libtorrent-devel mailing list