[tpop3d-discuss] Memory leak?
prune
prune at lecentre.net
Sat, 11 May 2002 11:58:02 +0200
--------------070207090601030502060109
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit
Hi,
I finaly read the whole thread you had with Chris.
Could you please add the LDAP log when bind or searches fail ?
The error code will maybe help us to understand the problem. There's no
reason for the bind to fail...
When I first wrote the ldap auth module, I was thinking of a very busy
server. So I decided to bind at the start of tpop3d and never again
(except if the connection stops). Some ldap modules as the proftpd one
binds (and connect) for every authentication. It's less efficient. but
maybe it's better ? I don't know ldap libraries that deep to be able to
answer this question right.
Now, the first patch Chris submits you shoud help with the memory leak.
Ldap library should normaly free itself the memory when the unbind is
called. maybe it fail to do it.... so specifying it by hand could improve.
I just don't understand why you get bind or search errors. the only
thing that could help now is ldap logs (or debug... depending what the
level is :)
I'm waiting for you....
Cheers,
Prune
Marc Lewis wrote:
>Tried both the auth_pam and the auth_ldap. Both exhibit the leak.
>
>One other thing that I've noticed is that authenticating from PAM leaks
>memory much, much faster than with LDAP.
>
>Authenticating from LDAP directly is slightly quicker, and doesn't leak as
>quickly, but we get a lot of bad authentications -- a few hundred over a
>four hour period. The authentications are actually good, when they try
>again it works. Looks like there may be a timeout that is too short in the
>ldap_simple_bind_s call when binding as the user.
>
>Looking at this code, though, I think I may have stumbled on something
>else related to the leak.
>
>If an authentication fails, or anything falls outside of the loop, I see
>several "goto fail;" lines, but looking at the code, some of the memory is
>freed before the fail label, and only a small handful of things are freed
>after failing. I would need to study this a bit more before jumping to any
>conclusions, but perhaps this is the source of the leak?
>
>Has anyone ever run the code through purify?
>
>I haven't looked through the PAM code yet...
>
> - Marc
>
>On Thu, May 09, 2002 at 01:50:20PM +0200, prune wrote:
>
>>Hi,
>>
>>did you try to use the auth_pam module built-in tpop3D ?
>>this should be faster than PAM, indeed.
>>It can be configured as you configured PAM.
>>
>>maybe this can help you..
>>
>>Cheers,
>>
>>Prune
>>
>>Marc Lewis wrote:
>>
>>>On Wed, May 08, 2002 at 11:20:07PM +0100, Chris Lightfoot wrote:
>>>
>>>>On Wed, May 08, 2002 at 03:04:42PM -0700, Marc Lewis wrote:
>>>>
>>>[snip]
>>>
>>>>>Well, its better, but its still leaking a bit:
>>>>>
>>>>>root 10829 0.0 0.2 5196 2312 pts/1 S 12:18 0:05 /usr/src/tpop3d-1.4.1/tpop3d -f /etc/tpop3d-test.conf -d -v
>>>>>
>>>>>I have an expect script pounding on it generating about 20-30
>>>>>authentications per minute for the last 30 minutes or so.
>>>>>
>>>>OK. I'll need to look into this one.
>>>>
>>>I'm happy to run any patches or anything that you like on it.
>>>
>>>>>>>I know this is hackish sounding, but what is the motivation for doing all
>>>>>>>of the auth and such before forking? When I've written simple daemons and
>>>>>>>
>>>> [...]
>>>>
>>>>>That makes perfect sense to me, but since the main server doesn't seem to
>>>>>maintain a persistant connection to the authentication source (at least in
>>>>>LDAP), the overhead of binding to the server per authentication still seems
>>>>>to be there. Not criticizing, just trying to understand.
>>>>>
>>>>Yeah-- it's not a problem with LDAP. I can't see myself
>>>>rewriting it to be an inetd-style server, though.
>>>>
>>>>I may add an option to have it restart itself every so
>>>>often.
>>>>
>>>After looking through the code a bit more, I can see the hesitation in
>>>making this an option. Would require major reworking.
>>>
>>>
>>>>>In thinking about this, it is possible that it is a problem in the OpenLDAP
>>>>>libraries. In my first build of tpop3d, I only configured in Maildir and
>>>>>PAM support. Since we are using PAM to access LDAP, its possible the leak
>>>>>was caused by PAM's accesses to the LDAP server, since the program never
>>>>>exited, things kept growing, but because of access to libpam. Building
>>>>>against LDAP directly lessons the leak, but it does definatly appear to
>>>>>still be there. Others using different non-LDAP authentication methods do
>>>>>not appear to have these leaks. Hmmm.
>>>>>
>>>>Which version of the OpenLDAP libraries are you using?
>>>>
>>>v2.0.21-1 from RedHat 7.2 installation, then updates.
>>>
>
--------------070207090601030502060109
Content-Type: text/html; charset=us-ascii
Content-Transfer-Encoding: 7bit
<html>
<head>
</head>
<body>
Hi,<br>
<br>
I finaly read the whole thread you had with Chris.<br>
Could you please add the LDAP log when bind or searches fail ?<br>
The error code will maybe help us to understand the problem. There's no reason
for the bind to fail... <br>
When I first wrote the ldap auth module, I was thinking of a very busy server.
So I decided to bind at the start of tpop3d and never again (except if the
connection stops). Some ldap modules as the proftpd one binds (and connect)
for every authentication. It's less efficient. but maybe it's better ? I
don't know ldap libraries that deep to be able to answer this question right.<br>
<br>
Now, the first patch Chris submits you shoud help with the memory leak. Ldap
library should normaly free itself the memory when the unbind is called.
maybe it fail to do it.... so specifying it by hand could improve.<br>
<br>
I just don't understand why you get bind or search errors. the only thing
that could help now is ldap logs (or debug... depending what the level is
:)<br>
<br>
I'm waiting for you....<br>
<br>
Cheers,<br>
<br>
Prune<br>
<br>
<br>
Marc Lewis wrote:<br>
<blockquote type="cite" cite="mid:20020509122609.D16116@gonk.blarg.net">
<pre wrap="">Tried both the auth_pam and the auth_ldap. Both exhibit the leak.<br><br>One other thing that I've noticed is that authenticating from PAM leaks<br>memory much, much faster than with LDAP.<br><br>Authenticating from LDAP directly is slightly quicker, and doesn't leak as<br>quickly, but we get a lot of bad authentications -- a few hundred over a<br>four hour period. The authentications are actually good, when they try<br>again it works. Looks like there may be a timeout that is too short in the<br>ldap_simple_bind_s call when binding as the user.<br><br>Looking at this code, though, I think I may have stumbled on something<br>else related to the leak.<br><br>If an authentication fails, or anything falls outside of the loop, I see<br>several "goto fail;" lines, but looking at the code, some of the memory is<br>freed before the fail label, and only a small handful of things are freed<br>after failing. I would need to study this a bit more before jumping to an
y<br>conclusions, but perhaps this is the source of the leak? <br><br>Has anyone ever run the code through purify?<br><br>I haven't looked through the PAM code yet...<br><br> - Marc<br><br>On Thu, May 09, 2002 at 01:50:20PM +0200, prune wrote:<br></pre>
<blockquote type="cite">
<pre wrap="">Hi,<br><br>did you try to use the auth_pam module built-in tpop3D ?<br>this should be faster than PAM, indeed.<br>It can be configured as you configured PAM.<br><br>maybe this can help you..<br><br>Cheers,<br><br>Prune<br><br>Marc Lewis wrote:<br><br></pre>
<blockquote type="cite">
<pre wrap="">On Wed, May 08, 2002 at 11:20:07PM +0100, Chris Lightfoot wrote:<br><br></pre>
<blockquote type="cite">
<pre wrap="">On Wed, May 08, 2002 at 03:04:42PM -0700, Marc Lewis wrote:<br><br></pre>
</blockquote>
<pre wrap="">[snip]<br><br></pre>
<blockquote type="cite">
<blockquote type="cite">
<pre wrap="">Well, its better, but its still leaking a bit:<br><br>root 10829 0.0 0.2 5196 2312 pts/1 S 12:18 0:05 /usr/src/tpop3d-1.4.1/tpop3d -f /etc/tpop3d-test.conf -d -v<br><br>I have an expect script pounding on it generating about 20-30<br>authentications per minute for the last 30 minutes or so.<br><br></pre>
</blockquote>
<pre wrap="">OK. I'll need to look into this one.<br><br></pre>
</blockquote>
<pre wrap="">I'm happy to run any patches or anything that you like on it.<br><br></pre>
<blockquote type="cite">
<blockquote type="cite">
<blockquote type="cite">
<blockquote type="cite">
<pre wrap="">I know this is hackish sounding, but what is the motivation for doing all<br>of the auth and such before forking? When I've written simple daemons and<br><br></pre>
</blockquote>
</blockquote>
</blockquote>
<pre wrap=""> [...]<br><br></pre>
<blockquote type="cite">
<pre wrap="">That makes perfect sense to me, but since the main server doesn't seem to<br>maintain a persistant connection to the authentication source (at least in<br>LDAP), the overhead of binding to the server per authentication still seems<br>to be there. Not criticizing, just trying to understand.<br><br></pre>
</blockquote>
<pre wrap="">Yeah-- it's not a problem with LDAP. I can't see myself<br>rewriting it to be an inetd-style server, though.<br><br>I may add an option to have it restart itself every so<br>often.<br><br></pre>
</blockquote>
<pre wrap="">After looking through the code a bit more, I can see the hesitation in<br>making this an option. Would require major reworking.<br><br><br></pre>
<blockquote type="cite">
<blockquote type="cite">
<pre wrap="">In thinking about this, it is possible that it is a problem in the OpenLDAP<br>libraries. In my first build of tpop3d, I only configured in Maildir and<br>PAM support. Since we are using PAM to access LDAP, its possible the leak<br>was caused by PAM's accesses to the LDAP server, since the program never<br>exited, things kept growing, but because of access to libpam. Building<br>against LDAP directly lessons the leak, but it does definatly appear to<br>still be there. Others using different non-LDAP authentication methods do<br>not appear to have these leaks. Hmmm.<br><br></pre>
</blockquote>
<pre wrap="">Which version of the OpenLDAP libraries are you using?<br><br></pre>
</blockquote>
<pre wrap="">v2.0.21-1 from RedHat 7.2 installation, then updates.<br><br></pre>
</blockquote>
</blockquote>
<pre wrap=""><!----><br></pre>
</blockquote>
<br>
</body>
</html>
--------------070207090601030502060109--