apache getting stuck

[Wunk]

We have a DA machine (Redhat 9, 1 GB mem, Dell 650) which regularly has apache crashing out..


The error log says:
[Mon Jan 26 20:37:02 2004] [warn] child process 11927 still did not exit, sending a SIGTERM
[Mon Jan 26 20:37:02 2004] [warn] child process 11972 still did not exit, sending a SIGTERM
[Mon Jan 26 20:37:02 2004] [warn] child process 11995 still did not exit, sending a SIGTERM
[Mon Jan 26 20:37:07 2004] [error] Cannot remove module mod_frontpage.c: not found in module list
[Mon Jan 26 20:37:09 2004] [warn] module perl_module is already loaded, skipping
[Mon Jan 26 20:37:10 2004] [crit] (98)Address already in use: make_sock: could not bind to port 8090


I can't find out what causes this, but it happens quite a lot.., and customers are complaining.., all that helps is killing -9 all running httpd processes and restarting the service..

Any suggestions on what could cause this or even better: a solution ?

[DirectAdmin Support]

Hello,

I think an update / recompile might be in order:


cd /usr/local/directadmin/customapache
rm -f configure.*
./build clean
./build update
./build allJohn

[Wunk]

It downloaded an updated PHP and Apache, so let's hope this will fix it..

Thanks :-)

[Wunk]

It didn't fix it.., last night around 1:00am it crashed again :(

Any other suggestions ?

[DirectAdmin Support]

Hello,

Are there any apache modules that have been added via rpm? Because this can't really be done.. try this:

rm -rf /usr/lib/apache/*

and then do the "./build all" again, as mentioned above.

John

[Wunk]

Nope, it's a plain RedHat 9 server without modules or anything installed..

I've emptied the modules directory, and rebuilt DA, let's hope it'll stay stable now :)

[Icheb]

I donno what exactly attracted my attention to this thread, but i have the exact same problems. With a RH 9.0 server, it can't be the hardware since it's a rather new server.

Recompiling Apache didn't work.
The configure.php from the custombuilder only has one extra line. Support for Freetype...

After i needed to recompile the last time (2 feb or something), the problems started. Every night at 1 am or something (daily cron runs at 0:01 am).

I still have an old ./build on another server, i am thinking of using this one to rebuild this server since i really am getting irritated with this problem...

What exactly has been done to the custombuilder or it's way to compile from January 1, 2004 ?

[DirectAdmin Support]

Hello,

Is you gd library compiled with freetype?? If you include freetype support in php, it will try and call functions in the gd library that may not exists. You can check it by running:

ldd /usr/local/lib/libgd.so
and
ldd /usr/lib/apache/libphp4.so

There were a few changes made the the configure.apache_ssl file.. mainly just the environment headers at the top. You can try removing them as they were only added to support freebsd, but they didn't seem to affect RedHat when I was testing, could be wrong.

John

[Icheb]

The ldd output :


# ldd /usr/local/lib/libgd.so
libfreetype.so.6 => /usr/local/lib/libfreetype.so.6 (0x4004f000)
libpng.so.3 => /usr/local/lib/libpng.so.3 (0x400ac000)
libz.so.1 => /usr/local/lib/libz.so.1 (0x400d4000)
libm.so.6 => /lib/tls/libm.so.6 (0x400e8000)
libc.so.6 => /lib/tls/libc.so.6 (0x42000000)
/lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x80000000)
# ldd /usr/lib/apache/libphp4.so
libcrypt.so.1 => /lib/libcrypt.so.1 (0x401d5000)
libmcrypt.so.4 => /usr/local/lib/libmcrypt.so.4 (0x40202000)
libltdl.so.3 => /usr/lib/libltdl.so.3 (0x40234000)
libfreetype.so.6 => /usr/local/lib/libfreetype.so.6 (0x4023b000)
libpng.so.3 => /usr/local/lib/libpng.so.3 (0x40298000)
libz.so.1 => /usr/local/lib/libz.so.1 (0x402c0000)
libresolv.so.2 => /lib/libresolv.so.2 (0x402ce000)
libm.so.6 => /lib/tls/libm.so.6 (0x402e0000)
libdl.so.2 => /lib/libdl.so.2 (0x40302000)
libnsl.so.1 => /lib/libnsl.so.1 (0x40306000)
libcurl.so.2 => /usr/local/lib/libcurl.so.2 (0x4031b000)
libssl.so.4 => /lib/libssl.so.4 (0x4033f000)
libcrypto.so.4 => /lib/libcrypto.so.4 (0x40375000)
libgssapi_krb5.so.2 => /usr/kerberos/lib/libgssapi_krb5.so.2 (0x40466000)
libkrb5.so.3 => /usr/kerberos/lib/libkrb5.so.3 (0x40479000)
libcom_err.so.3 => /usr/kerberos/lib/libcom_err.so.3 (0x404d7000)
libk5crypto.so.3 => /usr/kerberos/lib/libk5crypto.so.3 (0x404d9000)
libc.so.6 => /lib/tls/libc.so.6 (0x42000000)
/lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x80000000)


So I think it should be working :D

The older build script, from Nov 2003, didn't have that FreeBSD support, did it ?
On another server with exactly the same configs a compile with that build file works perfectly... (Don't know if this is significant)

[Icheb]

I created a temporary cronjob on this server that restarts the httpd every hour. So far i can see it finally stayed stable all night.

I know it's just an temporary solution, but i was hoping to minimize further downtime for users...

If Wunk doesn't have any problems anymore, I will also empty that directory, that currently contains:



total 9.7M
drwxr-xr-x 2 root root 4.0K Feb 10 22:28 .
drwxr-xr-x 47 root root 16K Jan 29 22:40 ..
lrwxrwxrwx 1 root root 20 Feb 10 22:28 apache -> ../../usr/lib/apache
-rw-r--r-- 1 root root 8.2K Feb 10 22:28 httpd.exp
-rwxr-xr-x 1 root root 1.2M Feb 10 22:36 libperl.so
-rwxr-xr-x 1 root root 7.4M Feb 10 22:35 libphp4.so
-rwxr-xr-x 1 root root 99K Feb 10 22:28 libproxy.so
-rwxr-xr-x 1 root root 219K Feb 10 22:28 libssl.so
-rwxr-xr-x 1 root root 12K Feb 10 22:28 mod_access.so
-rwxr-xr-x 1 root root 9.7K Feb 10 22:28 mod_actions.so
-rwxr-xr-x 1 root root 13K Feb 10 22:28 mod_alias.so
-rwxr-xr-x 1 root root 8.0K Feb 10 22:28 mod_asis.so
-rwxr-xr-x 1 root root 9.6K Feb 10 22:28 mod_auth_anon.so
-rwxr-xr-x 1 root root 13K Feb 10 22:28 mod_auth.so
-rwxr-xr-x 1 root root 31K Feb 10 22:28 mod_autoindex.so
-rwxr-xr-x 1 root root 11K Feb 10 22:28 mod_cern_meta.so
-rwxr-xr-x 1 root root 17K Feb 10 22:28 mod_cgi.so
-rwxr-xr-x 1 root root 11K Feb 10 22:28 mod_define.so
-rwxr-xr-x 1 root root 12K Feb 10 22:28 mod_digest.so
-rwxr-xr-x 1 root root 9.4K Feb 10 22:28 mod_dir.so
-rwxr-xr-x 1 root root 9.4K Feb 10 22:28 mod_env.so
-rwxr-xr-x 1 root root 15K Feb 10 22:28 mod_example.so
-rwxr-xr-x 1 root root 11K Feb 10 22:28 mod_expires.so
-rwxr-xr-x 1 root root 19K Feb 10 22:28 mod_frontpage.so
-rwxr-xr-x 1 root root 9.4K Feb 10 22:28 mod_headers.so
-rwxr-xr-x 1 root root 18K Feb 10 22:28 mod_imap.so
-rwxr-xr-x 1 root root 38K Feb 10 22:28 mod_include.so
-rwxr-xr-x 1 root root 21K Feb 10 22:28 mod_info.so
-rwxr-xr-x 1 root root 8.6K Feb 10 22:28 mod_log_agent.so
-rwxr-xr-x 1 root root 20K Feb 10 22:28 mod_log_config.so
-rwxr-xr-x 1 root root 9.7K Feb 10 22:28 mod_log_referer.so
-rwxr-xr-x 1 root root 26K Feb 10 22:28 mod_mime_magic.so
-rwxr-xr-x 1 root root 17K Feb 10 22:28 mod_mime.so
-rwxr-xr-x 1 root root 12K Feb 10 22:28 mod_mmap_static.so
-rwxr-xr-x 1 root root 30K Feb 10 22:28 mod_negotiation.so
-rwxr-xr-x 1 root root 61K Feb 10 22:28 mod_rewrite.so
-rwxr-xr-x 1 root root 12K Feb 10 22:28 mod_setenvif.so
-rwxr-xr-x 1 root root 13K Feb 10 22:28 mod_speling.so
-rwxr-xr-x 1 root root 21K Feb 10 22:28 mod_status.so
-rwxr-xr-x 1 root root 10K Feb 10 22:28 mod_unique_id.so
-rwxr-xr-x 1 root root 10K Feb 10 22:28 mod_userdir.so
-rwxr-xr-x 1 root root 14K Feb 10 22:28 mod_usertrack.so
-rwxr-xr-x 1 root root 11K Feb 10 22:28 mod_vhost_alias.so

I need to note that the apache symlink in it isn't working, it's marked red...

[Wunk]

My output ends up with this, the only thing we added was --enable-dbx support, but problems existed before that was added


[root@shared-dedi-1 root]# ldd /usr/local/lib/libgd.so
libpng.so.3 => /usr/local/lib/libpng.so.3 (0x40057000)
libm.so.6 => /lib/tls/libm.so.6 (0x40086000)
libc.so.6 => /lib/tls/libc.so.6 (0x42000000)
/lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x80000000)


[root@shared-dedi-1 root]# ldd /usr/lib/apache/libphp4.so
libcrypt.so.1 => /lib/libcrypt.so.1 (0x401e6000)
libmcrypt.so.4 => /usr/local/lib/libmcrypt.so.4 (0x40213000)
libltdl.so.3 => /usr/lib/libltdl.so.3 (0x40245000)
libpng.so.3 => /usr/local/lib/libpng.so.3 (0x4024c000)
libresolv.so.2 => /lib/libresolv.so.2 (0x40274000)
libm.so.6 => /lib/tls/libm.so.6 (0x40287000)
libdl.so.2 => /lib/libdl.so.2 (0x402a9000)
libnsl.so.1 => /lib/libnsl.so.1 (0x402ac000)
libcurl.so.2 => /usr/local/lib/libcurl.so.2 (0x402c1000)
libssl.so.4 => /lib/libssl.so.4 (0x402e5000)
libcrypto.so.4 => /lib/libcrypto.so.4 (0x4031a000)
libgssapi_krb5.so.2 => /usr/kerberos/lib/libgssapi_krb5.so.2 (0x40412000)
libkrb5.so.3 => /usr/kerberos/lib/libkrb5.so.3 (0x40425000)
libcom_err.so.3 => /usr/kerberos/lib/libcom_err.so.3 (0x40483000)
libk5crypto.so.3 => /usr/kerberos/lib/libk5crypto.so.3 (0x40485000)
libc.so.6 => /lib/tls/libc.so.6 (0x42000000)
/lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x80000000)
libz.so.1 => /usr/lib/libz.so.1 (0x40495000)

[DirectAdmin Support]

Hello,

How about removing the environment lines from the top of the configure.apache_ssl file and do a clean compile of apache. Not sure what else to check.

John

[Icheb]

Originally posted by DirectAdmin Support
Hello,

How about removing the environment lines from the top of the configure.apache_ssl file and do a clean compile of apache. Not sure what else to check.

John

Hi,

It seems to be an idea :)
I'll do it tonight (dutch time) together with a mysqld update to the latest version (a friend of mine straced the daemon and found a few disturbing things), or at least, if the client in issue will allow me to...

[Wunk]

Well, it crashed again last night, each time it seems to crash around 0:15am or somewhere around that time.. (DA cron issue somewhere ?)

I'll try removing the SSL part..

[DirectAdmin Support]

Hello,

Is the "LoadModule modules/libssl.so" bit still there? Send me your login info again if you want me to have a look. Also, let me know if you commented it when you're are fixing it so we can deteremine if that's what taking it down.

If you want to test if the nightly tally is doing it, you can just run:


echo "action=tally&value=all" >> /usr/local/directadmin/data/task.queueand wait a few minutes to see what happens.

John

[Wunk]

Well, I stayed up for it today, and it just went down again, same log entries:

[Fri Feb 13 23:12:47 2004] [error] [client 213.93.42.151] (13)Permission denied: cannot read directory for multi: /home/robende/domains/sharedip/
[Sat Feb 14 00:12:01 2004] [warn] child process 27179 still did not exit, sending a SIGTERM
[Sat Feb 14 00:12:01 2004] [warn] child process 27236 still did not exit, sending a SIGTERM
[Sat Feb 14 00:12:06 2004] [error] Cannot remove module mod_frontpage.c: not found in module list
[Sat Feb 14 00:12:06 2004] [warn] module perl_module is already loaded, skipping
[Sat Feb 14 00:12:06 2004] [crit] (98)Address already in use: make_sock: could not bind to port 8090


So basically it looks like the cron job that causes it is:
10 0 * * * root echo 'action=tally&value=all' >> /usr/local/directadmin/data/task.queue

And since we don't have other cronjobs running around that time, AND it's happening at this hour each time.., I'm pretty sure that causes it.. (which tasks is it doing around that time that could involve apache ?)


Point is.., it can go fine for a few days and not happen, or be dead 2 days in a row (like now)

Would you like to take a look ? (let me know your mailaddy and I'll provide a login)

For the moment I'm adding the following cronjob as a workaround for the next crash:

0 20 * * * /etc/init.d/httpd stop ; sleep 10 ; killall -9 httpd ; sleep 30 ; /etc/init.d/httpd restart > /dev/null 2> /dev/null


So far this is the only server that's acting up like this, we're hosting well over 30 directadmin panels, all without trouble, server hardware configurations are all equal though (at least most of them), they're all Dell poweredge 650's.., we tried upping the RAM to 1 GB, to no avail..

[Wunk]

btw, the libssl is still loaded, entry in /etc/httpd/conf/httpd.conf:

<IfDefine HAVE_SSL>
LoadModule ssl_module modules/libssl.so
</IfDefine>

[subhosting]

hi,

we have exactly the same problem here.
" only at one server " all the others have no problems.

server suddenly gets load of 60 and apache crashes.

http://www.robinsmit.nl//mailscanner-mrtg/loadavg/loadavg-day.png

[DirectAdmin Support]

Hello,

That's the nightly tally. The dataskq program should spike the load up while it recounts everything and runs webalizer on all the the logs. At the end of the recount, apache is restarted. The tally can take anywhere from 5 minutes to 1.5 hours, depending on how many sites you have, and how busy they are... so the only thing that I can think of that would cause apache to die would be one of

1) not enough memory during the tally. Not sure this would be the case, but perhaps keep an eye on the dataskq program during the tally and see if anything is spking out of control.

2) when the logs are rotated, they are deleted, but apache isn't reloaded until the end of the tally. Perhaps apache isn't handling the the removal of the error logs too well.. Only the ErrorLog keep the filedescriptor open.. so when DA removes the file, perhaps apache panics and doesn't know what to do. To check this, you could edit the virtual_host*.conf templates, and comment out the "ErrorLog" lines. This will put all errors into the /var/log/httpd/error_log file, which is not deleted by DA. Then we can see if that was causing it. (run echo "action=rewrite&value=httpd" >> /usr/local/directadmin/data/task.queue after the mods to the templates to rewite the httpd.conf files.

Let me know if that does anything.

John

[Icheb]

Originally posted by DirectAdmin Support
Hello,

That's the nightly tally.

1) not enough memory during the tally. Not sure this would be the case, but perhaps keep an eye on the dataskq program during the tally and see if anything is spking out of control.

2) when the logs are rotated, they are deleted, but apache isn't reloaded until the end of the tally. Perhaps apache isn't handling the the removal of the error logs too well..
John

I currently am working on the server of Subhosting.
The problems were :
* Load went up too high (upto 20 for just working normally)
* Apache sometimes crashed after the nightly tally

My first idea was to try to get the load down. This took me about 5 to 12 days to do, finally figured out what was causing it (i think).
John: For the next custombuilder release, please, build in a check for FreeBSD or RH. I removed the added code as you recommended (without further editing other files). And the load now goes upto 1.90 as max and at this moment it's stable at 0.80.
Furthermore I updated the MySQLD since it was using enourmous amounts of CPU time without a reason (strace pointed out it had a few problems).

Next i'm going to simulate the nightly tally to see what the max load will be and to see if it would kill Apache.

I don't think it's the reload of Apache that's the problem, since i use a script to parse the http://<ip>/~<user> traffic on my own server that messes around with logs, and Apache doesn't have any problems with it.

[DirectAdmin Support]

John: For the next custombuilder release, please, build in a check for FreeBSD or RH. I removed the added code as you recommended (without further editing other files). And the load now goes upto 1.90 as max and at this moment it's stable at 0.80. To which check are you referring? and you're referring to removing the ErrorLog line?

When the load is really high, whats eating the resoureces? (use 'top')

John

[Icheb]

Originally posted by DirectAdmin Support
To which check are you referring? and you're referring to removing the ErrorLog line?

When the load is really high, whats eating the resoureces? (use 'top')

John

I was wondering, I just finished the nightly tally 'sim', load went up to 8.84 but while I'm typing this it's back to 2.37. (MRTG is messing it up at the moment, it was 0.32 first)
No, I am referring to the /var/log/httpd/access log file.

Can you add a check if the OS is FreeBSD or Linux to see what configure.apache_ssl is needed ?
When i removed the first lines it worked way better, load is lower now.

While the load was high, httpd and mysqld were using 99% CPU time (both) in top. Restarting the service fixed it for a few minutes, but that isn't enough for a production server... ;)
This was caused by a few problems:
* MySQLD was wrong, strace pointed to a bug or something not working so I updated from 4.0.14 to 4.0.17
* httpd was using 99% CPU, but the /server-status indicated that it was using less.
* Perhaps some memory issues, the swap partition isn't working (was wondering if you have time to update the install instructions of DA for RH to reflect a 2x memory SWAP partition instead of a /dev/shm partition, since some people really MAKE that partition (ext3) and fixing that really isn't very much fun...)
But the system has 1 GB ram, so i'd figure everything would run smoothly...

[DirectAdmin Support]

Ok, I'll add the check.

John

[thoroughfare]

Hi,

The tally is killing Apache nightly for me too with a signal 11 - do DA have a solution for this as yet?

The system RAM is not faulty, we had it tested two weeks ago with memtest.

Using FreeBSD 5.3 with Apache 1.3.33.

Many thanks,
Matt

[DirectAdmin Support]

signal 11, as in segfaults?
Try this:
http://help.directadmin.com/item.php?id=95

John

[smtalk]

http://www.directadmin.com/forum/showthread.php?s=&threadid=12232

[vtx]

Same problem here, da running on ubuntu dapper, once in a few weeks apache doesnt restart like it should. It sais port 80 in use, and when I check via ps -auwwx I indeed see 1 /usr/sbin/httpd process running. After killing that (-9) I can start apache again.

Anyone have a solution for this problem?

[vtx]

** Kick **

Anyone?

[vtx]

And it happened again last night:

[Thu Feb 22 00:11:03 2007] [warn] Init: You should not use name-based virtual hosts in conjunction with SSL!!
[Thu Feb 22 00:11:03 2007] [notice] Apache/2.2.4 (Unix) mod_ssl/2.2.4 OpenSSL/0.9.8a PHP/4.4.4 configured -- resuming normal operations
[Thu Feb 22 00:13:02 2007] [notice] caught SIGTERM, shutting down

[Oxygenshell]

I have been having the same issue on 2 servers now for the past 2 days since ive upgraded to apache2 and php5.

I have tried editing the virtual_host*.conf templates, and commenting out the "ErrorLog" lines.

This had not worked.

Currently have a crontab set for a temporary work around like Wunk.

I have been told that the Tally can be shut down. If this is so, how would I go about doing this?

[Oxygenshell]

I have tried everything I have come across on here to get this issue fixed.

Some people say it has something to do with apache getting stuck during the log rotation, some people say it has to do with exim. I have tried commenting out the "ErrorLog" lines in virtual_host*.conf files to rebuilding apache and php right up to shutting down exim just for giggles.

All with the same issue.

Shortly after you do (echo "action=tally&value=all" >> /usr/local/directadmin/data/task.queue) httpd goes bonkers and hoofs it up to 99% cpu.

The only thing that has not been done is to shut down the tallying.

I am at a loss here. Has anyone come up with some solution/fix to this issue besides setting a crontab to kill and reload apache? :confused:

[Oxygenshell]

Ok, I have found what I believe at the moment is a work around and not a fix yet. If you are still having issues with httpd spiking after the nightly tally, try the following:

Try changing the apache boot script to do graceful restarts.
Edit /usr/local/etc/rc.d/httpd

Find this chunk of code:

restart)
stop
waitforexit "httpd" 20
start
;;

change it to look like:

restart)
kill -USR1 `cat $PIDFILE`
;;


However, this isn't a full restart. So test out creating a new user and see how apache behaves with this version of the script.
It may need the previous method for new users, new domains, ssl certificate changes, account suspension.. but graceful for other cases.

Like all changes you make to your system, always make a back up file so PLEASE PLEASE do - cp httpd httpd.bak and then edit the httpd boot script.

Hope this helps.

[smtalk]

I need 1 user who is having problems with apache2, to test a "patch" for this. If there are any - please contact me.

[nobaloney]

What you'e done isn't a restart at all; it's a reload.

Jeff

[Root]

I have been having this problem as well. I don't think it is the tally's, because I will have Apache 2 crash at ~4pm, sometimes at ~7am, and even ~3am. There doesn't seem to be a set minute or any crons that run around the same times. And sometimes it will go days or weeks without a problem, but sometimes it is everyday. I had made crons to restart Apache every hour at 38 minutes after the hour, however, that didn't really help the situation.

DA Service monitor will say that Apache's process has been stopped, however, there is usually a process open still for Apache. Sometimes it will be taking 99% of the CPU cycles, sometimes it will be taking 0%. I think when I catch it right after crash it will be taking the 99%. I didn't see anything in the logs to suggest much of anything. I'll try to catch it again.

Recompiles have not worked.

[pucky]

RH 9 was EOL'ed a long time ago. Surely, you are aware of this? Anyone running RH 9 and version lower than this can expect to get rooted, no matter what you do as far as security is concerned. I guarantee it. Its only a matter of time. It doesnt take much to upgrade the box to at least CentOS. There really is no excuse and the best part of it is, these types of problems will go away.

[Root]

RH 9 was EOL'ed a long time ago. Surely, you are aware of this? Anyone running RH 9 and version lower than this can expect to get rooted, no matter what you do as far as security is concerned. I guarantee it. Its only a matter of time. It doesnt take much to upgrade the box to at least CentOS. There really is no excuse and the best part of it is, these types of problems will go away.

I am using Fedora Core 6 on this box. I'm not sure how having RH 9 will up your chances of being rooted as long as you keep some of the obvious stuff with up to date compiles and use propper security policies...but that is another topic.

[pucky]

We have a DA machine (Redhat 9, 1 GB mem, Dell 650) which regularly has apache crashing out..


The error log says:
[Mon Jan 26 20:37:02 2004] [warn] child process 11927 still did not exit, sending a SIGTERM
[Mon Jan 26 20:37:02 2004] [warn] child process 11972 still did not exit, sending a SIGTERM
[Mon Jan 26 20:37:02 2004] [warn] child process 11995 still did not exit, sending a SIGTERM
[Mon Jan 26 20:37:07 2004] [error] Cannot remove module mod_frontpage.c: not found in module list
[Mon Jan 26 20:37:09 2004] [warn] module perl_module is already loaded, skipping
[Mon Jan 26 20:37:10 2004] [crit] (98)Address already in use: make_sock: could not bind to port 8090


I can't find out what causes this, but it happens quite a lot.., and customers are complaining.., all that helps is killing -9 all running httpd processes and restarting the service..

Any suggestions on what could cause this or even better: a solution ?

Ye, id like to know whats going on with this. We have had this happen at least 3 times this evening exactly the same error. Since our site is mysql driven we get a Mysql error back as well.

[Sat Apr 28 00:32:02 2007] [warn] child process 31824 still did not exit, sending a SIGTERM
[Sat Apr 28 00:32:02 2007] [warn] child process 31825 still did not exit, sending a SIGTERM
[Sat Apr 28 00:32:02 2007] [warn] child process 31826 still did not exit, sending a SIGTERM
[Sat Apr 28 00:32:02 2007] [warn] child process 31827 still did not exit, sending a SIGTERM
[Sat Apr 28 00:32:02 2007] [warn] child process 31828 still did not exit, sending a SIGTERM
[Sat Apr 28 00:32:02 2007] [warn] child process 31829 still did not exit, sending a SIGTERM
[Sat Apr 28 00:32:02 2007] [warn] child process 31830 still did not exit, sending a SIGTERM
[Sat Apr 28 00:32:02 2007] [warn] child process 31831 still did not exit, sending a SIGTERM
[Sat Apr 28 00:32:02 2007] [warn] child process 31832 still did not exit, sending a SIGTERM
[Sat Apr 28 00:32:02 2007] [warn] child process 31833 still did not exit, sending a SIGTERM
[Sat Apr 28 00:32:02 2007] [warn] child process 31836 still did not exit, sending a SIGTERM
[Sat Apr 28 00:32:02 2007] [warn] child process 31837 still did not exit, sending a SIGTERM
[Sat Apr 28 00:32:02 2007] [warn] child process 31838 still did not exit, sending a SIGTERM
[Sat Apr 28 00:32:02 2007] [warn] child process 31840 still did not exit, sending a SIGTERM
[Sat Apr 28 00:32:02 2007] [warn] child process 31841 still did not exit, sending a SIGTERM
[Sat Apr 28 00:32:44 2007] [crit] (48)Address already in use: make_sock: could not bind to port 443

What exactly is going on here? This has to be tally, and its been happening since we upgraded to the latest version of DA. Notice its 12:30am and thats when tally is running.

[felosi]

man, this is becoming very annoying. I dont know what the deal is but I cant keep doing this, everytime the server is half way busy and i try to restart apache i get
98)Address already in use: make_sock: could not bind to address 0.0.0.0:443
no listening sockets available, shutting down
Unable to open logs

Then it wont restart for like 10 minutes. When I do netstat it still shows things connected to port 80, sometimes I can go in and null route the ips and it will work. for some reason it is still holding on to the connection.
Ive tried every fix in here, stopping exim, restarting anything I can think of. Nothing works. restarting apache has became an ordeal now.
You says its not a directadmin issue, then what is it? I have apache 2 running on my other non-directadmin boxes with similar configurations just fine

[smtalk]

felosi, it's exim issue, because it's taking apache's port (for some reason).

[felosi]

not for me, still does it when exim is shut down. Looking in netstat it still shows connections on port 80

[smtalk]

And which process do you see on netstat?

[vtx]

Just replace the restart part of /etc/init.d/httpd with:



restart)
stop
waitforexit "httpd" 20
kill -9 `netstat -lnp | grep ':80 ' | awk '{print $7}' | awk -F "/" '{print $1}'`
kill -9 `netstat -lnp | grep ':443 ' | awk '{print $7}' | awk -F "/" '{print $1}'`
start
;;


Works for me :)

[smtalk]

vtx, yes, but it's only a workaround. :)

[vtx]

vtx, yes, but it's only a workaround. :)

I know, but better a workaround than an apache getting stuck at random nights ;)

[felosi]

Just replace the restart part of /etc/init.d/httpd with:



restart)
stop
waitforexit "httpd" 20
kill -9 `netstat -lnp | grep ':80 ' | awk '{print $7}' | awk -F "/" '{print $1}'`
kill -9 `netstat -lnp | grep ':443 ' | awk '{print $7}' | awk -F "/" '{print $1}'`
start
;;


Works for me :)

this works, thank you

[smtalk]

Someone, who doesn't need exim very much - please try to comment out exim.pl from exim.conf, I guess it's causing problems.

[nobaloney]

At least several posters have had the problem and it wasn't exim, so I'm not sure what this proves.

Martynas, have you ever seen exim.pl in the netstat output? exim.pl of course isn't a daemon; it should be called for each email but of course shouldn't hang around.

John, what do you think? Is there anything in exim.pl that you think could be causing the problem?

Jeff

[felosi]

well i didnt think it was exim but after checking netstat lately it was so I think its exim for sure

[smtalk]

Jeff, yes, I'm sure exim is causing the problem. Exim.pl isn't a daemon, but it's used by exim daemon. Here you are lsof ;) As you see - it's using apache's files.



exim 15135 root 7w REG 8,2 8240
29016088 /var/log/httpd/error_log
exim 15135 root 8w REG 8,2 1660
511063 /var/log/httpd/domains/dddd.info.error.log
exim 15135 root 9w REG 8,2 481770
511064 /var/log/httpd/domains/ccccc.com.error.log
exim 15135 root 10w REG 8,2 561481
511065 /var/log/httpd/domains/bbbbb.com.share.error.log
exim 15135 root 11w REG 8,2 6013
511066 /var/log/httpd/domains/bbbbb.com.error.log
exim 15135 root 12w REG 8,2 461928
511067 /var/log/httpd/domains/dddd.in.error.log
exim 15135 root 13w REG 8,2 1386
511068 /var/log/httpd/domains/aaaa.com.error.log
exim 15135 root 14w REG 8,2 12651
29016526 /var/log/httpd/access_log
exim 15135 root 15w REG 8,2 12
511061 /var/log/httpd/domains/dddd.info.bytes
exim 15135 root 16w REG 8,2 739
511069 /var/log/httpd/domains/dddd.info.log
exim 15135 root 17w REG 8,2 12
511061 /var/log/httpd/domains/dddd.info.bytes
exim 15135 root 18w REG 8,2 739
511069 /var/log/httpd/domains/dddd.info.log
exim 15135 root 19w REG 8,2 752054
511057 /var/log/httpd/domains/ccccc.com.bytes
exim 15135 root 20w REG 8,2 109257281
511070 /var/log/httpd/domains/ccccc.com.log
exim 15135 root 21w REG 8,2 752054
511057 /var/log/httpd/domains/ccccc.com.bytes
exim 15135 root 22w REG 8,2 109257281
511070 /var/log/httpd/domains/ccccc.com.log
exim 15135 root 23w REG 8,2 76224
511040 /var/log/httpd/domains/bbbbb.com.share.bytes
exim 15135 root 24w REG 8,2 3734211
511071 /var/log/httpd/domains/bbbbb.com.share.log
exim 15135 root 25w REG 8,2 76224
511040 /var/log/httpd/domains/bbbbb.com.share.bytes
exim 15135 root 26w REG 8,2 3734211
511071 /var/log/httpd/domains/bbbbb.com.share.log
exim 15135 root 27w REG 8,2 4881
511052 /var/log/httpd/domains/bbbbb.com.bytes
exim 15135 root 28w REG 8,2 192146
511072 /var/log/httpd/domains/bbbbb.com.log
exim 15135 root 29w REG 8,2 4881
511052 /var/log/httpd/domains/bbbbb.com.bytes
exim 15135 root 30w REG 8,2 192146
511072 /var/log/httpd/domains/bbbbb.com.log
exim 15135 root 31w REG 8,2 797992
511045 /var/log/httpd/domains/dddd.in.bytes
exim 15135 root 32w REG 8,2 56896838
511073 /var/log/httpd/domains/dddd.in.log
exim 15135 root 33w REG 8,2 797992
511045 /var/log/httpd/domains/dddd.in.bytes
exim 15135 root 34w REG 8,2 56896838
511073 /var/log/httpd/domains/dddd.in.log
exim 15135 root 35w REG 8,2 0
511049 /var/log/httpd/domains/aaaa.com.bytes
exim 15135 root 36w REG 8,2 0
511074 /var/log/httpd/domains/aaaa.com.log
exim 15135 root 37w REG 8,2 0
511049 /var/log/httpd/domains/aaaa.com.bytes
exim 15135 root 38w REG 8,2 0
511074 /var/log/httpd/domains/aaaa.com.log
exim 15135 root 39w REG 8,2 73264
29016529 /var/log/httpd/homedir.log
exim 15135 root 40w REG 8,2 73264
29016529 /var/log/httpd/homedir.log
exim 15135 root 41w REG 8,2 73264
29016529 /var/log/httpd/homedir.log
exim 15135 root 42w REG 8,2 73264
29016529 /var/log/httpd/homedir.log
exim 15135 root 43w REG 8,2 73264
29016529 /var/log/httpd/homedir.log
exim 15135 root 44w REG 8,2 73264
29016529 /var/log/httpd/homedir.log
exim 15135 root 45w REG 8,2 73264
29016529 /var/log/httpd/homedir.log
exim 15135 root 46w REG 8,2 73264
29016529 /var/log/httpd/homedir.log
exim 15135 root 47w REG 8,2 12651
29016526 /var/log/httpd/access_log
exim 15135 root 48w REG 8,2 0
29016530 /var/log/httpd/ssl_request_log
exim 15135 root 49w REG 8,2 0
29016882 /var/log/httpd/ssl_mutex.14811 (deleted)
exim 15135 root 3u IPv4 157531 TCP *:http
(LISTEN)
exim 15135 root 4u IPv4 157536 TCP
*:https (LISTEN)
exim 15136 mail 3u IPv4 157531 TCP *:http
(LISTEN)
exim 15136 mail 4u IPv4 157536 TCP
*:https (LISTEN)

[smtalk]

Ah.. and before lsof, "service apache restart" was done with this result:


Stopping httpd: [FAILED]
Starting httpd: (98)Address already in use: make_sock: could not bind to
address 0.0.0.0:443
no listening sockets available, shutting down
Unable to open logs
[FAILED]


So, only "killall -9 exim" could help in this situation :)

[nobaloney]

If it's exim.pl, couldn't we just do:

killall -9 exim.pl?
Should one of these killall lines be added to the httpd restart command? Or even to the httpd start command?

Jeff

[smtalk]

exim.pl is used by /usr/sbin/exim (and exim.pl isn't a process), because of

perl_startup = do '/etc/exim.pl'


So, it's "included" into exim :) We shouldn't use workarounds, we should use real fixes, don't you think so? :)

[nobaloney]

Sure. Create the fix ;) !

That's the beauty of opensource, you can do it instead of complain about it :D .

Jeff

[selfwebhosting]

Sure. Create the fix ;) !

That's the beauty of opensource, you can do it instead of complain about it :D .

Jeff

Has anyone created a fix?

Before a fix is found, I have to use a cronjob to restart apache every four hours a day!

I am running apache2 and I have commented out the ErrorLog lines in all the virtual_host*.conf template config files.

[nobaloney]

exim.pl IS found in the process list when it's running. And that change I mentioned to the apache script may very well resolve the problem. If it does, it IS the fix, and adding it to the restart script IS the right way to fix it.

But someone needs to test it first.

Jeff

[selfwebhosting]

In my case, it is both exim and MySQL. So how should I modify the restart script? Yes, I am willing to test it.

[smtalk]

jlasman, as you see in lsof - it's exim, and not exim.pl, but in exim (not exim.pl) there are no such paths as /var/log/httpd/domains/aaaa.com.bytes etc. Don't you think so? "killall -9" won't ever be a fix, because it IS really a workaround, I never recommend to use "-9" flag, because it can damage the process (file or something more), if you don't think that it's true - try to kill mysql with "-9" few times a day and you will see some damaged databases. ;)

[selfwebhosting]

Jeff, yes, I'm sure exim is causing the problem. Exim.pl isn't a daemon, but it's used by exim daemon. Here you are lsof ;) As you see - it's using apache's files.



exim 15135 root 7w REG 8,2 8240
29016088 /var/log/httpd/error_log
exim 15135 root 8w REG 8,2 1660
511063 /var/log/httpd/domains/dddd.info.error.log
exim 15135 root 9w REG 8,2 481770
511064 /var/log/httpd/domains/ccccc.com.error.log
...
exim 15135 root 49w REG 8,2 0
29016882 /var/log/httpd/ssl_mutex.14811 (deleted)
exim 15135 root 3u IPv4 157531 TCP *:http
(LISTEN)
exim 15135 root 4u IPv4 157536 TCP
*:https (LISTEN)
exim 15136 mail 3u IPv4 157531 TCP *:http
(LISTEN)
exim 15136 mail 4u IPv4 157536 TCP
*:https (LISTEN)


smtalk, how did you capture the above?

In my case it is definitely MySQL as when the load is high I see a lot of MySQL connections. After clicking "stop" followed by "start" for MySQL, the connection numbers suddenly drop down to 2 or 3. Also when the CPU upload shoots up to 16, I see MySQL is taking up 99&#37; of the CPU.

How can I pin-point which domain/script is causing this problem? Now I am running vtx's modified script for the apache restart section every two hours. But from the DA control panel I see I can manually "stop" and then "start" MySQL. How can I do it automatically, that is, writing a simple script and then execute it using a cronjob? Like smtalk says, it killing MySQL with "-9" flag a few times a day can damage databases, how can I avoid using that flad. If I do it manually a few times a day, would this damage my databases?

[selfwebhosting]

OK, I quit using vtx's modified apache restart script. I now run this crontab every two hours:


15 */2 * * * /sbin/service mysqld restart

Would this create problem for my databases? How can I do a graceful restart rathen than a sudden stop and start?

Then the still remaining question of how to track down which database has been used to jump up my CPU usage to 99% - any tip will be appreciated.

[smtalk]

Firstly, do:

netstat -nap --inet |grep -w 80|grep LISTEN

Or:

netstat -nap --inet |grep -w 443|grep LISTEN

You will see a process and PID of it which is using apache's port. You will see something like:


tcp 0 0 0.0.0.0:80 0.0.0.0:*
LISTEN 1111/exim


Then do lsof for it, and you will see something like my output :):


lsof -p PID


Change PID with a PID of the process.

[nobaloney]

I never recommend to use "-9" flag, because it can damage the process (file or something more), if you don't think that it's true - try to kill mysql with "-9" few times a day and you will see some damaged databases. ;)
Your obfuscating. Of course killing mysqld with -9 can cause database damage, because it just stops a deamon that may be in the middle of writing to the database.

Checking exim.pl carefully the only file it appears to write to is a logfile. While it could damage that file (I don't think it will, since it's a flat file, but it could), that could be resolved by rewriting exim.pl to use the appropriate daemon to write the log entry.

I understand you don't agree with me. What do you recommend?

Jeff

[selfwebhosting]

Hmmm... eventually I find that installing mod_security and adding the lines to the mod_security rule file did the trick, as suggested by AWBS (Advanced Webhost Billing System) forum's security alert posted here:

http://forum.awbs.com/showthread.php?t=6718

Now I do not need to run the MySQL restart cronjob anymore and the CPU load remains under 2 since I did this.

Also I got a reply on AWBS forum to my questions on this issue. Here it is if someone is interested in knowing:

http://forum.awbs.com/showthread.php?p=37144#post37144

[selfwebhosting]

Now the same issue comes back - CPU usage gets high every two hours and I have reboot MySQL service every two hours.

[smtalk]

Anyone are still experiencing issue with exim+apache? :)

[bengrubb]

Anyone are still experiencing issue with exim+apache? :)
Yes

I am also getting this same problem:

[Fri Aug 17 00:11:02 2007] [notice] SIGHUP received. Attempting to restart
[Fri Aug 17 00:11:03 2007] [notice] Apache/1.3.37 (Unix) mod_ssl/2.8.28 OpenSSL/0.9.7e PHP/4.4.7 mod_perl/1.29 FrontPage/5.0.2.2510 configured -- resuming normal operations
[Fri Aug 17 00:11:03 2007] [notice] suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
[Fri Aug 17 00:11:03 2007] [notice] Accept mutex: sysvsem (Default: sysvsem)
[Fri Aug 17 00:14:01 2007] [notice] caught SIGTERM, shutting down
[Fri Aug 17 00:14:07 2007] [notice] Apache/1.3.37 (Unix) mod_ssl/2.8.28 OpenSSL/0.9.7e PHP/4.4.7 mod_perl/1.29 FrontPage/5.0.2.2510 configured -- resuming normal operations
[Fri Aug 17 00:14:07 2007] [notice] suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
[Fri Aug 17 00:14:07 2007] [notice] Accept mutex: sysvsem (Default: sysvsem)
[Fri Aug 17 01:36:52 2007] [error] [client 64.246.0.110] File does not exist: /var/www/html/robots.txt
[Fri Aug 17 03:42:01 2007] [notice] caught SIGTERM, shutting down
[Fri Aug 17 03:42:05 2007] [notice] Apache/1.3.37 (Unix) mod_ssl/2.8.28 OpenSSL/0.9.7e PHP/4.4.7 mod_perl/1.29 FrontPage/5.0.2.2510 configured -- resuming normal operations
[Fri Aug 17 03:42:05 2007] [notice] suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
[Fri Aug 17 03:42:05 2007] [notice] Accept mutex: sysvsem (Default: sysvsem)
[Fri Aug 17 13:45:02 2007] [notice] caught SIGTERM, shutting down
[Fri Aug 17 13:45:07 2007] [notice] Apache/1.3.37 (Unix) mod_ssl/2.8.28 OpenSSL/0.9.7e PHP/4.4.7 mod_perl/1.29 FrontPage/5.0.2.2510 configured -- resuming normal operations
[Fri Aug 17 13:45:07 2007] [notice] suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
[Fri Aug 17 13:45:07 2007] [notice] Accept mutex: sysvsem (Default: sysvsem
[Fri Aug 17 18:28:01 2007] [notice] caught SIGTERM, shutting down

-- About here is when I became aware that websites hosted on this particular server were not responding to http requests. --

[Fri Aug 17 18:28:08 2007] [notice] Apache/1.3.37 (Unix) mod_ssl/2.8.28 OpenSSL/0.9.7e PHP/4.4.7 mod_perl/1.29 FrontPage/5.0.2.2510 configured -- resuming normal operations
[Fri Aug 17 18:28:08 2007] [notice] suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
[Fri Aug 17 18:28:08 2007] [notice] Accept mutex: sysvsem (Default: sysvsem)
[Fri Aug 17 18:42:02 2007] [notice] caught SIGTERM, shutting down
[Fri Aug 17 18:42:08 2007] [notice] Apache/1.3.37 (Unix) mod_ssl/2.8.28 OpenSSL/0.9.7e PHP/4.4.7 mod_perl/1.29 FrontPage/5.0.2.2510 configured -- resuming normal operations
[Fri Aug 17 18:42:08 2007] [notice] suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
[Fri Aug 17 18:42:08 2007] [notice] Accept mutex: sysvsem (Default: sysvsem)
[Fri Aug 17 18:50:01 2007] [notice] caught SIGTERM, shutting down
[Fri Aug 17 18:50:04 2007] [notice] Apache/1.3.37 (Unix) mod_ssl/2.8.28 OpenSSL/0.9.7e PHP/4.4.7 mod_perl/1.29 FrontPage/5.0.2.2510 configured -- resuming normal operations
[Fri Aug 17 18:50:04 2007] [notice] suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
[Fri Aug 17 18:50:04 2007] [notice] Accept mutex: sysvsem (Default: sysvsem)
[Sat Aug 18 00:11:02 2007] [notice] SIGHUP received. Attempting to restart
[Sat Aug 18 00:11:06 2007] [notice] Apache/1.3.37 (Unix) mod_ssl/2.8.28 OpenSSL/0.9.7e PHP/4.4.7 mod_perl/1.29 FrontPage/5.0.2.2510 configured -- resuming normal operations
[Sat Aug 18 00:11:06 2007] [notice] suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
[Sat Aug 18 00:11:06 2007] [notice] Accept mutex: sysvsem (Default: sysvsem)
[Sat Aug 18 00:13:02 2007] [notice] caught SIGTERM, shutting down
[Sat Aug 18 00:13:08 2007] [notice] Apache/1.3.37 (Unix) mod_ssl/2.8.28 OpenSSL/0.9.7e PHP/4.4.7 mod_perl/1.29 FrontPage/5.0.2.2510 configured -- resuming normal operations
[Sat Aug 18 00:13:08 2007] [notice] suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
[Sat Aug 18 00:13:08 2007] [notice] Accept mutex: sysvsem (Default: sysvsem)
[Sat Aug 18 06:03:53 2007] [error] [client 66.249.67.116] File does not exist: /var/www/html/robots.txt
-- About here is when the httpd service was restarted manually --
[Sat Aug 18 18:36:22 2007] [notice] caught SIGTERM, shutting down
[Sat Aug 18 18:36:28 2007] [notice] Apache/1.3.37 (Unix) mod_ssl/2.8.28 OpenSSL/0.9.7e PHP/4.4.7 mod_perl/1.29 FrontPage/5.0.2.2510 configured -- resuming normal operations
[Sat Aug 18 18:36:28 2007] [notice] suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
[Sat Aug 18 18:36:28 2007] [notice] Accept mutex: sysvsem (Default: sysvsem)
[Sat Aug 18 20:39:02 2007] [notice] SIGHUP received. Attempting to restart
[Sat Aug 18 20:39:05 2007] [notice] Apache/1.3.37 (Unix) mod_ssl/2.8.28 OpenSSL/0.9.7e PHP/4.4.7 mod_perl/1.29 FrontPage/5.0.2.2510 configured -- resuming normal operations
[Sat Aug 18 20:39:05 2007] [notice] suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
[Sat Aug 18 20:39:05 2007] [notice] Accept mutex: sysvsem (Default: sysvsem)
[Sat Aug 18 20:41:01 2007] [notice] caught SIGTERM, shutting down
[Sat Aug 18 20:41:07 2007] [notice] Apache/1.3.37 (Unix) mod_ssl/2.8.28 OpenSSL/0.9.7e PHP/4.4.7 mod_perl/1.29 FrontPage/5.0.2.2510 configured -- resuming normal operations
[Sat Aug 18 20:41:07 2007] [notice] suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
[Sat Aug 18 20:41:07 2007] [notice] Accept mutex: sysvsem (Default: sysvsem)
[Sat Aug 18 20:48:01 2007] [notice] caught SIGTERM, shutting down
[Sat Aug 18 20:48:06 2007] [notice] Apache/1.3.37 (Unix) mod_ssl/2.8.28 OpenSSL/0.9.7e PHP/4.4.7 mod_perl/1.29 FrontPage/5.0.2.2510 configured -- resuming normal operations
[Sat Aug 18 20:48:06 2007] [notice] suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
[Sat Aug 18 20:48:06 2007] [notice] Accept mutex: sysvsem (Default: sysvsem)


P.S This hanging has only been occurring in the past couple of days.
I am using Debian 3.1

[kobe]

When trying to restart I get this error:
Starting httpd: Syntax error on line 19 of /etc/httpd/conf/httpd.conf:
API module structure 'php5_module' in file /usr/lib/apache/libphp5.so is garbled - expected signature 41503230 but saw 41503232 - perhaps this is not an Apache module DSO, or was compiled for a different Apache version?

Can anyone help me out?
Thanks

[smtalk]

kobe, try to rebuild PHP5.

[kobe]

kobe, try to rebuild PHP5.
I believe my partner ended up doing a fresh install. He said it started but now it's not up.

I am trying to reboot but I still can't get httpd to start.

You suggest a rebuild?

[smtalk]

Yes, do this:


cd /usr/local/directadmin/custombuild
./build clean
./build php n

[kobe]

My partner said there is an issue with php_admin_flag?
So I should rebuild and then see what happens (remember he just reinstalled)?

And all of the errors are coming from /usr/local/directadmin/data/users:
Syntax error on line 30 of /usr/local/directadmin/data/users/admin/httpd.conf:
Invalid command 'php_admin_flag', perhaps misspelled or defined by a module not included in the server configuration

[smtalk]

You have php5 as cgi. http://www.directadmin.com/features.php?id=828

[kobe]

You have php5 as cgi. http://www.directadmin.com/features.php?id=828
Ok, thanks for the heads up. I turned all the options OFF as stated in the link.

Now do you suggest a rebuild, even though it was a fresh install?
I still can't fire up httpd.

[kobe]

OK, i ran the rebuild and get this error:

*** There was an error while trying to configure php. Check the configure/ap2/configure.php5 file

[smtalk]

And few lines before this error? Try to execute "./build update" before the other commands.

[kobe]

new error when building PHP5


ext/mysqli/.libs/mysqli_nonapi.o(.text+0xade): In function `zif_mysqli_get_charset':
/usr/local/directadmin/custombuild/php-5.2.5/ext/mysqli/mysqli_nonapi.c:372: undefined reference to `mysql_get_character_set_info'
collect2: ld returned 1 exit status
make: *** [sapi/cli/php] Error 1

[kobe]

this is the complete error a few lines above......please help.....

ext/mysqli/.libs/mysqli_nonapi.o(.text+0xade): In function `zif_mysqli_get_charset':
/usr/local/directadmin/custombuild/php-5.2.5/ext/mysqli/mysqli_nonapi.c:372: undefined reference to `mysql_get_character_set_info'
collect2: ld returned 1 exit status
make: *** [sapi/cli/php] Error 1

*** The make has failed, do you want to try to make again? (y,n):