Borek hack on server

R

Rick L.

Verified User
Joined
Jun 24, 2006
Messages
23
Location
Netherlands
I found strange entries in my logfiles. Somebody seems to install something using the following script: http://yenzero.com/wp-admin/borek.txt

It starts with omg your box got owned. secure ur **** better. if you dont know how, why are you admin of this box? so I don't think it's good stuff ;)

In my logs I found the following entries:

% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 11745 100 11745 0 0 24035 0 --:--:-- --:--:-- --:--:-- 43127
Died at sess_rdav631df3a1ddfaa34s1x1wwo521459 line 24.
Died at sess_rdav631df3a1ddfaa34s1x1wwo521459 line 24.
rm: borek.txt*: No such file or directory
lwp-download: not found
mv: borek.txt: No such file or directory
Died at sess_rdav631df3a1ddfaa34s1x1wwo521459 line 24.
Died at sess_rdav631df3a1ddfaa34s1x1wwo521459 line 24.
lynx: not found
Died at sess_rdav631df3a1ddfaa34s1x1wwo521459 line 24.
Died at sess_rdav631df3a1ddfaa34s1x1wwo521459 line 24.
borek.txt 11 kB 47 kBps
Can't open perl script "sess_adav631df3a1ddfaa34s1x1wwo521459": No such file or directory
GET: not found
--14:11:13-- http://yenzero.com/wp-admin/borek.txt
=> `borek.txt.1'
Resolving yenzero.com... done.
Connecting to yenzero.com[216.12.200.18]:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 11,745 [text/plain]

0K .......... . 100% 47.01 KB/s

14:11:13 (47.01 KB/s) - `borek.txt.1' saved [11745/11745]

Died at sess_rdav631df3a1ddfaa34s1x1wwo521459 line 24.
Died at sess_rdav631df3a1ddfaa34s1x1wwo521459 line 24.
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 11745 100 11745 0 0 24027 0 --:--:-- --:--:-- --:--:-- 43304
Died at sess_rdav631df3a1ddfaa34s1x1wwo521459 line 24.
rm: borek.txt*: No such file or directory
Died at sess_rdav631df3a1ddfaa34s1x1wwo521459 line 24.
lwp-download: not found
mv: borek.txt: No such file or directory
Died at sess_rdav631df3a1ddfaa34s1x1wwo521459 line 24.
Died at sess_rdav631df3a1ddfaa34s1x1wwo521459 line 24.
lynx: not found
Died at sess_rdav631df3a1ddfaa34s1x1wwo521459 line 24.
Died at sess_rdav631df3a1ddfaa34s1x1wwo521459 line 24.
borek.txt 11 kB 46 kBps
GET: not found
--14:12:15-- http://yenzero.com/wp-admin/borek.txt
=> `borek.txt.1'
Resolving yenzero.com... done.
Connecting to yenzero.com[216.12.200.18]:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 11,745 [text/plain]

0K .......... . 100% 47.01 KB/s

14:12:16 (47.01 KB/s) - `borek.txt.1' saved [11745/11745]

Died at sess_rdav631df3a1ddfaa34s1x1wwa521459 line 24.
Died at sess_rdav631df3a1ddfaa34s1x1wwa521459 line 24.
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 11745 100 11745 0 0 24023 0 --:--:-- --:--:-- --:--:-- 43304
Died at sess_rdav631df3a1ddfaa34s1x1wws521459 line 24.
Died at sess_rdav631df3a1ddfaa34s1x1wws521459 line 24.
rm: borek.txt*: No such file or directory
lwp-download: not found
mv: rename borek.txt to sess_rdav631df3a1ddfaa34s1x1wwd521459: No such file or directory
Can't open perl script "sess_rdav631df3a1ddfaa34s1x1wwd521459": No such file or directory
Can't open perl script "sess_rdav631df3a1ddfaa34s1x1wwd521459": No such file or directory
lynx: not found
Can't open perl script "sess_rdav631df3a1ddfaa34s1x1wwf521459": No such file or directory
Can't open perl script "sess_rdav631df3a1ddfaa34s1x1wwf521459": No such file or directory
borek.txt 11 kB 47 kBps
Can't open perl script "sess_adav631df3a1ddfaa34s1x1wwg521459": No such file or directory
GET: not found
Can't open perl script "sess_rdav631df3a1ddfaa34s1x1wwh521459": No such file or directory
Can't open perl script "sess_rdav631df3a1ddfaa34s1x1wwh521459": No such file or directory
--14:12:17-- http://yenzero.com/wp-admin/borek.txt
=> `borek.txt.1'
Resolving yenzero.com... done.
Connecting to yenzero.com[216.12.200.18]:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 11,745 [text/plain]

0K .......... . 100% 47.01 KB/s

14:12:18 (47.01 KB/s) - `borek.txt.1' saved [11745/11745]

Died at sess_rdav631df3a1ddfaa34s1x1wwz521459 line 24.
Died at sess_rdav631df3a1ddfaa34s1x1wwz521459 line 24.
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 11745 100 11745 0 0 3255 0 0:00:03 0:00:03 --:--:-- 4728
Died at sess_rdav631df3a1ddfaa34s1x1wwx521459 line 24.
rm: borek.txt*: No such file or directory
Died at sess_rdav631df3a1ddfaa34s1x1wwx521459 line 24.
lwp-download: not found
mv: rename borek.txt to sess_rdav631df3a1ddfaa34s1x1wwc521459: No such file or directory
Can't open perl script "sess_rdav631df3a1ddfaa34s1x1wwc521459": No such file or directory
Can't open perl script "sess_rdav631df3a1ddfaa34s1x1wwc521459": No such file or directory
lynx: not found
Can't open perl script "sess_rdav631df3a1ddfaa34s1x1wwv521459": No such file or directory
Can't open perl script "sess_rdav631df3a1ddfaa34s1x1wwv521459": No such file or directory
borek.txt 11 kB 47 kBps
GET: not found
Can't open perl script "sess_rdav631df3a1ddfaa34s1x1wwn521459": No such file or directory
Can't open perl script "sess_rdav631df3a1ddfaa34s1x1wwn521459": No such file or directory
--14:12:22-- http://yenzero.com/wp-admin/borek.txt
=> `borek.txt.1'
Resolving yenzero.com... done.
Connecting to yenzero.com[216.12.200.18]:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 11,745 [text/plain]

0K .......... . 100% 33.83 KB/s

14:12:23 (33.83 KB/s) - `borek.txt.1' saved [11745/11745]

Died at sess_rdav631df3a1ddfaa34s1x1wwo521459 line 24.
Died at sess_rdav631df3a1ddfaa34s1x1wwo521459 line 24.
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 11745 100 11745 0 0 20103 0 --:--:-- --:--:-- --:--:-- 31041
Died at sess_rdav631df3a1ddfaa34s1x1wwo521459 line 24.
Died at sess_rdav631df3a1ddfaa34s1x1wwo521459 line 24.
rm: borek.txt*: No such file or directory
lwp-download: not found
mv: borek.txt: No such file or directory
Died at sess_rdav631df3a1ddfaa34s1x1wwo521459 line 24.
Died at sess_rdav631df3a1ddfaa34s1x1wwo521459 line 24.
lynx: not found
Died at sess_rdav631df3a1ddfaa34s1x1wwo521459 line 24.
Died at sess_rdav631df3a1ddfaa34s1x1wwo521459 line 24.
borek.txt 11 kB 47 kBps
Can't open perl script "sess_adav631df3a1ddfaa34s1x1wwo521459": No such file or directory
GET: not found
--14:12:25-- http://yenzero.com/wp-admin/borek.txt
=> `borek.txt.1'
Resolving yenzero.com... done.
Connecting to yenzero.com[216.12.200.18]:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 11,745 [text/plain]

0K .......... . 100% 33.93 KB/s

14:12:25 (33.93 KB/s) - `borek.txt.1' saved [11745/11745]

Died at sess_rdav631df3a1ddfaa34s1x1wwo521459 line 24.
Died at sess_rdav631df3a1ddfaa34s1x1wwo521459 line 24.
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 11745 100 11745 0 0 20136 0 --:--:-- --:--:-- --:--:-- 31225
Died at sess_rdav631df3a1ddfaa34s1x1wwo521459 line 24.
rm: borek.txt*: No such file or directory
Died at sess_rdav631df3a1ddfaa34s1x1wwo521459 line 24.
lwp-download: not found
mv: borek.txt: No such file or directory
Died at sess_rdav631df3a1ddfaa34s1x1wwo521459 line 24.
Died at sess_rdav631df3a1ddfaa34s1x1wwo521459 line 24.
lynx: not found
Died at sess_rdav631df3a1ddfaa34s1x1wwo521459 line 24.
Died at sess_rdav631df3a1ddfaa34s1x1wwo521459 line 24.
borek.txt 11 kB 47 kBps
GET: not found
[Sat Mar 24 14:21:34 2007] [error] [client 65.54.*.*] File does not exist: /var/www/html/robots.txt
[Sat Mar 24 14:21:34 2007] [warn] [client 65.54.*.*] mod_include: Options +Includes (or IncludesNoExec) wasn't set, INCLUDES filter removed
--14:30:40-- http://yenzero.com/wp-admin/borek.txt
=> `borek.txt.1'
Resolving yenzero.com... done.
Connecting to yenzero.com[216.12.200.18]:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 11,745 [text/plain]

0K .......... . 100% 46.82 KB/s

14:30:41 (46.82 KB/s) - `borek.txt.1' saved [11745/11745]

Died at sess_rdav631df3a1ddfaa34s1x1wwa521459 line 24.
Died at sess_rdav631df3a1ddfaa34s1x1wwa521459 line 24.
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 11745 100 11745 0 0 24049 0 --:--:-- --:--:-- --:--:-- 43483
Died at sess_rdav631df3a1ddfaa34s1x1wws521459 line 24.
Died at sess_rdav631df3a1ddfaa34s1x1wws521459 line 24.
rm: borek.txt*: No such file or directory
lwp-download: not found
mv: rename borek.txt to sess_rdav631df3a1ddfaa34s1x1wwd521459: No such file or directory
Can't open perl script "sess_rdav631df3a1ddfaa34s1x1wwd521459": No such file or directory
Can't open perl script "sess_rdav631df3a1ddfaa34s1x1wwd521459": No such file or directory
lynx: not found
Can't open perl script "sess_rdav631df3a1ddfaa34s1x1wwf521459": No such file or directory
Can't open perl script "sess_rdav631df3a1ddfaa34s1x1wwf521459": No such file or directory
borek.txt 11 kB 47 kBps
Can't open perl script "sess_adav631df3a1ddfaa34s1x1wwg521459": No such file or directory
GET: not found
Can't open perl script "sess_rdav631df3a1ddfaa34s1x1wwh521459": No such file or directory
Can't open perl script "sess_rdav631df3a1ddfaa34s1x1wwh521459": No such file or directory
--14:30:42-- http://yenzero.com/wp-admin/borek.txt
=> `borek.txt.1'
Resolving yenzero.com... done.
Connecting to yenzero.com[216.12.200.18]:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 11,745 [text/plain]

0K .......... . 100% 47.01 KB/s

14:30:43 (47.01 KB/s) - `borek.txt.1' saved [11745/11745]

Died at sess_rdav631df3a1ddfaa34s1x1wwz521459 line 24.
Died at sess_rdav631df3a1ddfaa34s1x1wwz521459 line 24.
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 11745 100 11745 0 0 24031 0 --:--:-- --:--:-- --:--:-- 43304
Died at sess_rdav631df3a1ddfaa34s1x1wwx521459 line 24.
rm: borek.txt*: No such file or directory
Died at sess_rdav631df3a1ddfaa34s1x1wwx521459 line 24.
lwp-download: not found
mv: rename borek.txt to sess_rdav631df3a1ddfaa34s1x1wwc521459: No such file or directory
Can't open perl script "sess_rdav631df3a1ddfaa34s1x1wwc521459": No such file or directory
Can't open perl script "sess_rdav631df3a1ddfaa34s1x1wwc521459": No such file or directory
lynx: not found
Can't open perl script "sess_rdav631df3a1ddfaa34s1x1wwv521459": No such file or directory
Can't open perl script "sess_rdav631df3a1ddfaa34s1x1wwv521459": No such file or directory
borek.txt 11 kB 2239 Bps
GET: not found
Can't open perl script "sess_rdav631df3a1ddfaa34s1x1wwn521459": No such file or directory
Can't open perl script "sess_rdav631df3a1ddfaa34s1x1wwn521459": No such file or directory
--14:30:49-- http://yenzero.com/wp-admin/borek.txt
=> `borek.txt.1'
Resolving yenzero.com... done.
Connecting to yenzero.com[216.12.200.18]:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 11,745 [text/plain]

0K .......... . 100% 47.01 KB/s

14:30:50 (47.01 KB/s) - `borek.txt.1' saved [11745/11745]

Died at sess_rdav631df3a1ddfaa34s1x1wwo521459 line 24.
Died at sess_rdav631df3a1ddfaa34s1x1wwo521459 line 24.
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 11745 100 11745 0 0 24028 0 --:--:-- --:--:-- --:--:-- 43127
Died at sess_rdav631df3a1ddfaa34s1x1wwo521459 line 24.
Died at sess_rdav631df3a1ddfaa34s1x1wwo521459 line 24.
rm: borek.txt*: No such file or directory
lwp-download: not found
mv: borek.txt: No such file or directory
Died at sess_rdav631df3a1ddfaa34s1x1wwo521459 line 24.
Died at sess_rdav631df3a1ddfaa34s1x1wwo521459 line 24.
lynx: not found
Died at sess_rdav631df3a1ddfaa34s1x1wwo521459 line 24.
Died at sess_rdav631df3a1ddfaa34s1x1wwo521459 line 24.
borek.txt 11 kB 47 kBps
Can't open perl script "sess_adav631df3a1ddfaa34s1x1wwo521459": No such file or directory
GET: not found
--14:30:52-- http://yenzero.com/wp-admin/borek.txt
=> `borek.txt.1'
Resolving yenzero.com... done.
Connecting to yenzero.com[216.12.200.18]:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 11,745 [text/plain]

0K .......... . 100% 31.34 KB/s

14:30:52 (31.34 KB/s) - `borek.txt.1' saved [11745/11745]

Died at sess_rdav631df3a1ddfaa34s1x1wwo521459 line 24.
Died at sess_rdav631df3a1ddfaa34s1x1wwo521459 line 24.
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 11745 100 11745 0 0 23939 0 --:--:-- --:--:-- --:--:-- 42951
Died at sess_rdav631df3a1ddfaa34s1x1wwo521459 line 24.
rm: borek.txt*: No such file or directory
Died at sess_rdav631df3a1ddfaa34s1x1wwo521459 line 24.
lwp-download: not found
mv: borek.txt: No such file or directory
Died at sess_rdav631df3a1ddfaa34s1x1wwo521459 line 24.
Died at sess_rdav631df3a1ddfaa34s1x1wwo521459 line 24.
lynx: not found
Died at sess_rdav631df3a1ddfaa34s1x1wwo521459 line 24.
Died at sess_rdav631df3a1ddfaa34s1x1wwo521459 line 24.
borek.txt 11 kB 47 kBps
GET: not found


[Sat Mar 24 15:24:18 2007] [error] [client 130.89.*.*] Invalid method in request cisco
[Sat Mar 24 15:24:18 2007] [error] [client 130.89.*.*] Invalid method in request 130.89.175.16:7534
[Sat Mar 24 15:31:17 2007] [error] [client 65.55.*.*] File does not exist: /var/www/html/robots.txt
[Sat Mar 24 15:31:17 2007] [warn] [client 65.55.*.*] mod_include: Options +Includes (or IncludesNoExec) wasn't set, INCLUDES filter removed
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 11745 100 11745 0 0 24067 0 --:--:-- --:--:-- --:--:-- 43483
Died at sess_rdav631df3a1ddfaa34s1x1wws521459 line 24.
rm: borek.txt*: No such file or directory
[root@s003:/var/log/httpd] #
Click to expand...

After this the Perl-scripts will start up again and it can only be killed using the -9 option. But when the PID is killed, another is starting directly. So I'm afraid that is is not possible to restart Apache anymore.

Can somebody help me with this or does anybody know a solution?
 
Last edited:
I've created a new thread for this post as it didn't belong in a thread about Apache2 failing for an unknown reason.

Hopefully the new topic will attract helpful replies.

Jeff
 
Thanks Jeff.

Additional information: an unknown perl script is running sometimes on the server. It will cause problems with restarting Apache: some processes won't be killed by the default boot-script. Only killing the PIDs with the '-9' option makes it possible to start Apache again.

I have deleted the file borek.txt in the /tmp directory and killed the PID again and it didn't start again automaticly. But it's still a big question what is going on.

Ik hope someone can help me with this.
 
Its unfortunate that your server was compromised but you really do need to do something about your security.
 
Last edited:
Rick L. said:
Thanks Jeff.

Additional information: an unknown perl script is running sometimes on the server. It will cause problems with restarting Apache: some processes won't be killed by the default boot-script. Only killing the PIDs with the '-9' option makes it possible to start Apache again.

I have deleted the file borek.txt in the /tmp directory and killed the PID again and it didn't start again automaticly. But it's still a big question what is going on.

Ik hope someone can help me with this.
Click to expand...

Rick try smtalk's script Easy Linux Security and for starters close /tmp with this script :)

Maybe that will help somewhat
 
Thanks for your reply!

That script will not work for me because it doesn't support FreeBSD at the moment. Of course /tmp is mounted with noexec, but it was still possible to run the script.
 
Mounting /tmp noexec is a good idea but it doesn't keep crackers from running
Code: 
perl /tmp/scriptname
Jeff
 
What I have done is set up a script that searches for processes running as apache that should not be there and kill them with -9.

perl code
Code: 
@ps = `ps aux | grep ^apache | grep -v /usr/sbin/httpd`;
chomp(@ps);

foreach $ps(@ps){

        ($user,$pid) = split(/\s+/, $ps);
        `kill -9 $pid`;

}

The grep -v excludes the known apache run processes that you want.
This takes care of most php exploits. Your mileage may vary.
 
Nice script! Do you run it regular using cronjobs?

jlasman said:
Mounting /tmp noexec is a good idea but it doesn't keep crackers from running
Code: 
perl /tmp/scriptname
Jeff
Click to expand...

Is there a way to prevent it? Or to make it impossible to use commands like wget as a non-root user?
 
You can run it as a cron job but if you want it to catch processes quicker than once a minute you can add:

Code: 
while(1>0){

insert previous script here

}

Might want to run it with a nice level of -19 or add a sleep code to it so it only check every 5 seconds or something like that. That way it is not doing a ps as fast as the server will allow it. I am sure a real programmer can add some stuff to make it better.
 
Rick L. said:
Nice script! Do you run it regular using cronjobs?



Is there a way to prevent it? Or to make it impossible to use commands like wget as a non-root user?
Click to expand...

If you want to make wget and other similar command run only as root, then do something similar to this:
chown root /usr/bin/wget
chmod 744 /usr/bin/wget
that will make sure that only the user root has execute permissions for the executable /usr/bin/wget.
 
ps aux | grep ^apache | grep -v /usr/sbin/httpd give me no results
but
ps aux | grep ^apache | grep -v /usr/sbin/httpd2 gives me
Code: 
apache   10309  0.1  8.5 33404 20984 ?       S    11:27   0:08 /usr/sbin/httpd -k start -DSSL
apache   21285  0.1  8.4 33024 20632 ?       S    12:01   0:05 /usr/sbin/httpd -k start -DSSL
apache   30444  0.1  8.2 32660 20136 ?       S    12:27   0:02 /usr/sbin/httpd -k start -DSSL
apache   32179  0.1  8.2 32588 20100 ?       S    12:33   0:01 /usr/sbin/httpd -k start -DSSL
apache    1169  0.3  8.2 32720 20272 ?       S    12:37   0:02 /usr/sbin/httpd -k start -DSSL
apache    1222  0.3  8.5 33380 20944 ?       S    12:39   0:01 /usr/sbin/httpd -k start -DSSL
apache    1226  0.3  8.1 32396 19904 ?       S    12:39   0:01 /usr/sbin/httpd -k start -DSSL
apache    2933  0.3  9.7 36344 23824 ?       S    12:43   0:01 /usr/sbin/httpd -k start -DSSL
apache    2934  0.3 10.1 37156 24740 ?       S    12:43   0:01 /usr/sbin/httpd -k start -DSSL
apache    3004  0.3  6.9 29484 17072 ?       S    12:46   0:00 /usr/sbin/httpd -k start -DSSL

Does this mean these processes should not be there?

grep 'SIGTERM' /var/log/httpd/error_log* gives

/var/log/httpd/error_log:[Mon May 07 00:16:00 2007] [notice] caught SIGTERM, shutting down
/var/log/httpd/error_log:[Tue May 08 00:11:06 2007] [warn] child process 32650 still did not exit, sending a SIGTERM
/var/log/httpd/error_log:[Tue May 08 00:11:08 2007] [warn] child process 32650 still did not exit, sending a SIGTERM
/var/log/httpd/error_log:[Tue May 08 00:11:10 2007] [warn] child process 32650 still did not exit, sending a SIGTERM
/var/log/httpd/error_log:[Tue May 08 00:18:01 2007] [notice] caught SIGTERM, shutting down
/var/log/httpd/error_log:[Wed May 09 00:17:01 2007] [notice] caught SIGTERM, shutting down
/var/log/httpd/error_log:[Wed May 09 08:59:00 2007] [notice] caught SIGTERM, shutting down
/var/log/httpd/error_log.1:[Mon Apr 30 00:17:02 2007] [notice] caught SIGTERM, shutting down
/var/log/httpd/error_log.1:[Mon Apr 30 11:40:47 2007] [notice] caught SIGTERM, shutting down
/var/log/httpd/error_log.1:[Mon Apr 30 11:43:00 2007] [notice] caught SIGTERM, shutting down
/var/log/httpd/error_log.1:[Mon Apr 30 12:08:00 2007] [notice] caught SIGTERM, shutting down
/var/log/httpd/error_log.1:[Mon Apr 30 14:39:54 2007] [notice] caught SIGTERM, shutting down
/var/log/httpd/error_log.1:[Mon Apr 30 14:46:10 2007] [notice] caught SIGTERM, shutting down
/var/log/httpd/error_log.1:[Mon Apr 30 14:53:31 2007] [notice] caught SIGTERM, shutting down
/var/log/httpd/error_log.1:[Tue May 01 00:17:02 2007] [notice] caught SIGTERM, shutting down
/var/log/httpd/error_log.1:[Tue May 01 04:27:02 2007] [notice] caught SIGTERM, shutting down
/var/log/httpd/error_log.1:[Wed May 02 00:17:02 2007] [notice] caught SIGTERM, shutting down
/var/log/httpd/error_log.1:[Thu May 03 00:16:02 2007] [notice] caught SIGTERM, shutting down
/var/log/httpd/error_log.1:[Fri May 04 00:17:01 2007] [notice] caught SIGTERM, shutting down
/var/log/httpd/error_log.1:[Sat May 05 00:17:02 2007] [notice] caught SIGTERM, shutting down
/var/log/httpd/error_log.1:[Sat May 05 16:09:00 2007] [notice] caught SIGTERM, shutting down
/var/log/httpd/error_log.1:[Sun May 06 00:17:01 2007] [notice] caught SIGTERM, shutting down
/var/log/httpd/error_log.2:[Mon Apr 23 00:18:01 2007] [notice] caught SIGTERM, shutting down
/var/log/httpd/error_log.2:[Tue Apr 24 00:17:00 2007] [notice] caught SIGTERM, shutting down
/var/log/httpd/error_log.2:[Tue Apr 24 16:49:02 2007] [notice] caught SIGTERM, shutting down
/var/log/httpd/error_log.2:[Tue Apr 24 17:08:01 2007] [notice] caught SIGTERM, shutting down
/var/log/httpd/error_log.2:[Tue Apr 24 17:16:01 2007] [notice] caught SIGTERM, shutting down
/var/log/httpd/error_log.2:[Wed Apr 25 00:17:00 2007] [notice] caught SIGTERM, shutting down
/var/log/httpd/error_log.2:[Wed Apr 25 12:55:01 2007] [notice] caught SIGTERM, shutting down
/var/log/httpd/error_log.2:[Wed Apr 25 14:29:01 2007] [notice] caught SIGTERM, shutting down
/var/log/httpd/error_log.2:[Thu Apr 26 00:19:01 2007] [notice] caught SIGTERM, shutting down
/var/log/httpd/error_log.2:[Fri Apr 27 00:17:02 2007] [notice] caught SIGTERM, shutting down
/var/log/httpd/error_log.2:[Fri Apr 27 16:38:04 2007] [notice] caught SIGTERM, shutting down
/var/log/httpd/error_log.2:[Sat Apr 28 00:18:02 2007] [notice] caught SIGTERM, shutting down
/var/log/httpd/error_log.2:[Sun Apr 29 00:19:01 2007] [notice] caught SIGTERM, shutting down
/var/log/httpd/error_log.3:[Mon Apr 16 00:16:03 2007] [notice] caught SIGTERM, shutting down
/var/log/httpd/error_log.3:[Mon Apr 16 10:49:01 2007] [notice] caught SIGTERM, shutting down
/var/log/httpd/error_log.3:[Tue Apr 17 00:17:00 2007] [notice] caught SIGTERM, shutting down
/var/log/httpd/error_log.3:[Wed Apr 18 00:17:03 2007] [notice] caught SIGTERM, shutting down

i can't figure out why apache keeps restarting. Could the last post in this thread be the answer?
http://directadmin.com/forum/showthread.php?t=14432

I checked th logs in detail after noticing the following in httpd error logs
[Mon May 07 11:39:00 2007] [error] [client 220.196.58.37] request failed: error reading the headers
sh: line 1: /usr/bin/wget: Permission denied
sh: line 1: edit: command not found
sh: line 1: get: command not found
[Tue May 08 20:52:59 2007] [error] [client 67.18.113.196] File does not exist: /home/res00978/domains/sharedip/404.shtml
[Tue May 08 20:52:59 2007] [error] [client 67.18.113.196] File does not exist: /var/www/html/a1b2c3d4e5f6g7h8i9
[Tue May 08 20:53:00 2007] [error] [client 67.18.113.196] File does not exist: /home/admin/domains/sharedip/a1b2c3d4e5f6g7h8i9
[Tue May 08 20:53:00 2007] [error] [client 67.18.113.196] File does not exist: /home/admin/domains/sharedip/404.shtml
[Tue May 08 20:53:00 2007] [error] [client 67.18.113.196] File does not exist: /home/admin/domains/sharedip/a1b2c3d4e5f6g7h8i9
[Tue May 08 20:53:00 2007] [error] [client 67.18.113.196] File does not exist: /home/admin/domains/sharedip/404.shtml
[Tue May 08 20:53:00 2007] [warn] [client 67.18.113.196] mod_include: Options +Includes (or IncludesNoExec) wasn't set, INCLUDES filter removed
[Tue May 08 20:53:00 2007] [error] [client 67.18.113.196] File does not exist: /home/stepper22/domains/sharedip/a1b2c3d4e5f6g7h8i9
[Tue May 08 20:53:00 2007] [error] [client 67.18.113.196] File does not exist: /home/stepper22/domains/sharedip/404.shtml
[Tue May 08 20:53:00 2007] [error] [client 67.18.113.196] script '/home/res00978/domains/sharedip/adxmlrpc.php' not found or unable to stat
[Tue May 08 20:53:00 2007] [error] [client 67.18.113.196] File does not exist: /home/res00978/domains/sharedip/404.shtml
[Tue May 08 20:53:00 2007] [error] [client 67.18.113.196] script '/home/admin/domains/sharedip/adxmlrpc.php' not found or unable to stat
[Tue May 08 20:53:00 2007] [error] [client 67.18.113.196] File does not exist: /home/admin/domains/sharedip/404.shtml
[Tue May 08 20:53:00 2007] [error] [client 67.18.113.196] script '/home/admin/domains/sharedip/adxmlrpc.php' not found or unable to stat
[Tue May 08 20:53:00 2007] [error] [client 67.18.113.196] File does not exist: /home/admin/domains/sharedip/404.shtml
[Tue May 08 20:53:00 2007] [error] [client 67.18.113.196] script '/var/www/html/adxmlrpc.php' not found or unable to stat
[Tue May 08 20:53:00 2007] [warn] [client 67.18.113.196] mod_include: Options +Includes (or IncludesNoExec) wasn't set, INCLUDES filter removed

I have mod_security installed and I have locked to root the usual suspects, wget etc
I am running Apache 2.059
 
Which one is normal for you, httpd or httpd2? grep -v the one that is normal.

Apache restarts every time a domain is added.
 
floyd said:
What I have done is set up a script that searches for processes running as apache that should not be there and kill them with -9.

perl code
Code: 
@ps = `ps aux | grep ^apache | grep -v /usr/sbin/httpd`;
chomp(@ps);

foreach $ps(@ps){

        ($user,$pid) = split(/\s+/, $ps);
        `kill -9 $pid`;

}

The grep -v excludes the known apache run processes that you want.
This takes care of most php exploits. Your mileage may vary.
Click to expand...

I have a similar problem where perl scripts are masquerading as apache, could I adapt this script to find them and kill them? I have spent the better part of 2 days reading and looking though forums and sites - trying to find a way to locate the file or user that's creating these processes. The USER is apache but the COMMAND is perl. Should I replace the string '/usr/bin/httpd' with 'per'l ?
 
Last edited:
This script catches everything that is run as the user apache except for /usr/sbin/httpd.

You will probably want to have exclude some other processes as well. This is just a starting point.

But as it is it will catch all perl scripts running as the user apache. When suexec is used there is no reason to have a perl script running as a apache.

I got tired of trying to find all the exploited scripts and just decided to kill any processes that were started by exploited scripts.
 
floyd said:
This script catches everything that is run as the user apache except for /usr/sbin/httpd.

You will probably want to have exclude some other processes as well. This is just a starting point.

But as it is it will catch all perl scripts running as the user apache. When suexec is used there is no reason to have a perl script running as a apache.

I got tired of trying to find all the exploited scripts and just decided to kill any processes that were started by exploited scripts.
Click to expand...

Sweet, that gave me a better idea of what this script was doing :) ive been watching top and can see the apache and perl, but with ps aux its actually listing 'apache' and '[syst]' so i refied the command to be

Code: 
ps aux | grep apache | grep syst

apache   14715  0.0  0.0  9484 3180 ?        S    May07   0:00 [syst]
apache    1197  0.0  0.0  9572 3180 ?        S    10:40   0:00 [syst]
apache   29241 32.8  0.0  9380 4124 ?        R    14:33  19:01 [syst]
apache   30402 33.0  0.0 10180 4124 ?        R    14:33  18:56 [syst]
apache   30935 32.3  0.0 10084 4120 ?        R    14:34  18:26 [syst]
apache   31642 32.1  0.0  8468 4124 ?        R    14:34  18:14 [syst]

and i can isolate just the abuser scripts. Before i start sicking an automated process killer on the system, I wanted to make sure what I was doing was a good idea. :)
 
Keep looking at ps aux | grep apache | grep -v /usr/sbin/httpd and you will get an idea of what is normal and what is not. In the this line I excluded /usr/sbin/httpd because we already know that is supposed to be running. This will tell you everything else that might run as apache.

Sendmail will run as apache if a user is sending mail via a php script so you may want to exclude /usr/sbin/sendmail and /usr/sbin/exim.
 
so what would be the perl script to actually display all the scripts that are sending mail through apache at the given moment?
 
@amkDave: sometimes worms change their command line to hide themselfs (for example "/usr/sbin/httpd -k start -DSSL"), the only way to make sure that a process is really httpd is to look at /proc/<pid>/exe. This can be done with this command:
Code: 
$ ps auxww |grep apache |awk '{print $2}' |sudo xargs -i{ ls -l /proc/{/exe
ls: cannot access /proc/10829/exe: No such file or directory
lrwxrwxrwx 1 root root 0 2009-04-17 08:03 /proc/11128/exe -> /usr/sbin/httpd
lrwxrwxrwx 1 root root 0 2009-04-17 08:03 /proc/11129/exe -> /usr/sbin/httpd
lrwxrwxrwx 1 root root 0 2009-04-17 08:03 /proc/11130/exe -> /usr/sbin/httpd
The first one (10829) is just the "grep" command, sometimes it will show up. You could ignore the grep command, but again some worms hide themselfs as "grep".
What you really want to find is "/proc/<pid>/exe -> /tmp/borek.txt".

@matthewventura: you don't need a perl script to do that. You could simply run "sudo lsof -i TCP:25".
 
Hmmm i have no borek.txt but a tx.txt :S


Code: 
grep 'tx.txt' /var/log/httpd/error_log*
/var/log/httpd/error_log.1:mv: cannot stat `tx.txt': No such file or directory
/var/log/httpd/error_log.3:chmod: cannot access `tx.txt': No such file or directory
/var/log/httpd/error_log.3:Could not open input file: tx.txt
/var/log/httpd/error_log.3:chmod: cannot access `tx.txt': No such file or directory
/var/log/httpd/error_log.3:Could not open input file: tx.txt
/var/log/httpd/error_log.3:chmod: cannot access `tx.txt': No such file or directory
/var/log/httpd/error_log.3:Could not open input file: tx.txt
/var/log/httpd/error_log.3:chmod: cannot access `tx.txt': No such file or directory
/var/log/httpd/error_log.3:Could not open input file: tx.txt
/var/log/httpd/error_log.3:Could not open input file: tx.txt
/var/log/httpd/error_log.4:chmod: cannot access `tx.txt': No such file or directory
/var/log/httpd/error_log.4:Could not open input file: tx.txt
/var/log/httpd/error_log.4:chmod: cannot access `tx.txt': No such file or directory
/var/log/httpd/error_log.4:Could not open input file: tx.txt
/var/log/httpd/error_log.4:chmod: cannot access `tx.txt': No such file or directory
/var/log/httpd/error_log.4:Could not open input file: tx.txt
/var/log/httpd/error_log.4:chmod: cannot access `tx.txt': No such file or directory
/var/log/httpd/error_log.4:Could not open input file: tx.txt
/var/log/httpd/error_log.4:chmod: cannot access `tx.txt': No such file or directory
/var/log/httpd/error_log.4:--01:57:45--  http://www.cr578.com/tx.txt
/var/log/httpd/error_log.4:tx.txt: Permission denied
/var/log/httpd/error_log.4:Cannot write to `tx.txt' (Permission denied).
/var/log/httpd/error_log.4:chmod: cannot access `tx.txt': No such file or directory
/var/log/httpd/error_log.4:--01:59:07--  http://www.cr578.com/tx.txt
/var/log/httpd/error_log.4:tx.txt: Permission denied
/var/log/httpd/error_log.4:Cannot write to `tx.txt' (Permission denied).
/var/log/httpd/error_log.4:chmod: cannot access `tx.txt': No such file or directory
/var/log/httpd/error_log.4:--04:29:31--  http://www.cr578.com/tx.txt
/var/log/httpd/error_log.4:tx.txt: Permission denied

Also i see some strange service httpdse running and have a huge load, after killing the process the load decrease to normal. Also i have search for httpdse and tx.txt but i found nothing also the /tmp /dec/shm /var/tmp folders are clean.

Code: 
ps auxww |grep apache |awk '{print $2}' |sudo xargs -i{ ls -l /proc/{/exe
lrwxrwxrwx 1 root root 0 Apr 17 12:14 /proc/18141/exe -> /usr/sbin/httpd
lrwxrwxrwx 1 root root 0 Apr 17 12:14 /proc/18433/exe -> /usr/sbin/httpd
lrwxrwxrwx 1 root root 0 Apr 17 12:14 /proc/18437/exe -> /usr/sbin/httpd
lrwxrwxrwx 1 root root 0 Apr 17 12:15 /proc/25098/exe -> /usr/sbin/httpd
lrwxrwxrwx 1 root root 0 Apr 17 12:31 /proc/25920/exe -> /usr/sbin/httpd
lrwxrwxrwx 1 root root 0 Apr 17 12:31 /proc/25953/exe -> /usr/sbin/httpd
lrwxrwxrwx 1 root root 0 Apr 17 12:31 /proc/26697/exe -> /usr/sbin/httpd
lrwxrwxrwx 1 root root 0 Apr 17 12:31 /proc/26704/exe -> /usr/sbin/httpd
lrwxrwxrwx 1 root root 0 Apr 17 12:31 /proc/26705/exe -> /usr/sbin/httpd
lrwxrwxrwx 1 root root 0 Apr 17 12:31 /proc/27077/exe -> /usr/sbin/httpd
lrwxrwxrwx 1 root root 0 Apr 17 12:14 /proc/27080/exe -> /usr/sbin/httpd
lrwxrwxrwx 1 root root 0 Apr 17 12:14 /proc/27081/exe -> /usr/sbin/httpd
lrwxrwxrwx 1 root root 0 Apr 17 12:14 /proc/27083/exe -> /usr/sbin/httpd
lrwxrwxrwx 1 root root 0 Apr 17 12:14 /proc/27084/exe -> /usr/sbin/httpd
lrwxrwxrwx 1 root root 0 Apr 17 12:14 /proc/27085/exe -> /usr/sbin/httpd
lrwxrwxrwx 1 root root 0 Apr 17 12:14 /proc/27086/exe -> /usr/sbin/httpd
lrwxrwxrwx 1 root root 0 Apr 17 12:14 /proc/27087/exe -> /usr/sbin/httpd
lrwxrwxrwx 1 root root 0 Apr 17 12:14 /proc/27089/exe -> /usr/sbin/httpd
lrwxrwxrwx 1 root root 0 Apr 17 12:14 /proc/27095/exe -> /usr/sbin/httpd
lrwxrwxrwx 1 root root 0 Apr 17 12:14 /proc/27100/exe -> /usr/sbin/httpd
lrwxrwxrwx 1 root root 0 Apr 17 12:14 /proc/27102/exe -> /usr/sbin/httpd
lrwxrwxrwx 1 root root 0 Apr 17 12:14 /proc/27105/exe -> /usr/sbin/httpd
lrwxrwxrwx 1 root root 0 Apr 17 12:14 /proc/27106/exe -> /usr/sbin/httpd
lrwxrwxrwx 1 root root 0 Apr 17 12:14 /proc/27107/exe -> /usr/sbin/httpd
lrwxrwxrwx 1 root root 0 Apr 17 12:14 /proc/27109/exe -> /usr/sbin/httpd
lrwxrwxrwx 1 root root 0 Apr 17 12:14 /proc/27111/exe -> /usr/sbin/httpd
lrwxrwxrwx 1 root root 0 Apr 17 12:14 /proc/27112/exe -> /usr/sbin/httpd
lrwxrwxrwx 1 root root 0 Apr 17 12:14 /proc/27113/exe -> /usr/sbin/httpd
lrwxrwxrwx 1 root root 0 Apr 17 12:14 /proc/27114/exe -> /usr/sbin/httpd
lrwxrwxrwx 1 root root 0 Apr 17 12:14 /proc/27115/exe -> /usr/sbin/httpd
lrwxrwxrwx 1 root root 0 Apr 17 12:14 /proc/27116/exe -> /usr/sbin/httpd
lrwxrwxrwx 1 root root 0 Apr 17 12:14 /proc/27117/exe -> /usr/sbin/httpd
lrwxrwxrwx 1 root root 0 Apr 17 12:14 /proc/27119/exe -> /usr/sbin/httpd
lrwxrwxrwx 1 root root 0 Apr 17 12:14 /proc/27124/exe -> /usr/sbin/httpd
lrwxrwxrwx 1 root root 0 Apr 17 12:14 /proc/27125/exe -> /usr/sbin/httpd
lrwxrwxrwx 1 root root 0 Apr 17 12:14 /proc/27126/exe -> /usr/sbin/httpd
lrwxrwxrwx 1 root root 0 Apr 17 12:14 /proc/27128/exe -> /usr/sbin/httpd
lrwxrwxrwx 1 root root 0 Apr 17 12:14 /proc/27129/exe -> /usr/sbin/httpd
lrwxrwxrwx 1 root root 0 Apr 17 12:31 /proc/27801/exe -> /usr/sbin/httpd
lrwxrwxrwx 1 root root 0 Apr 17 12:31 /proc/27803/exe -> /usr/sbin/httpd
lrwxrwxrwx 1 root root 0 Apr 17 12:31 /proc/27807/exe -> /usr/sbin/httpd
lrwxrwxrwx 1 root root 0 Apr 17 12:31 /proc/27808/exe -> /usr/sbin/httpd
lrwxrwxrwx 1 root root 0 Apr 17 12:31 /proc/28006/exe -> /usr/sbin/httpd
lrwxrwxrwx 1 root root 0 Apr 17 12:31 /proc/28044/exe -> /usr/sbin/httpd
ls: /proc/28046/exe: No such file or directory

Some one has a good solution to find the hack?
 
You must log in or register to reply here.
Back
Top