wpad.dat in Apache error log

Thanks SirDice:
Maybe this will aide you.
I did not change the log format:
Code:
    LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
    LogFormat "%h %l %u %t \"%r\" %>s %b" common

From the error log:
Code:
[Wed Nov 16 00:15:48 2011] [error] [client My public servers static ip address] File does not exist: /usr/home/username/public_html/wpad.dat
[Wed Nov 16 00:15:48 2011] [error] [client My public servers static ip address] File does not exist: /usr/home/username/public_html/wpad.dat
[Wed Nov 16 00:15:48 2011] [error] [client My public servers static ip address] File does not exist: /usr/home/username/public_html/wpad.dat
[Wed Nov 16 00:15:48 2011] [error] [client My public servers static ip address] File does not exist: /usr/home/username/public_html/wpad.dat
[Wed Nov 16 00:15:48 2011] [error] [client My public servers static ip address] File does not exist: /usr/home/username/public_html/wpad.dat
[Wed Nov 16 00:15:48 2011] [error] [client My public servers static ip address] File does not exist: /usr/home/username/public_html/wpad.dat
[Wed Nov 16 00:22:24 2011] [error] [client My public servers static ip address] File does not exist: /usr/home/username/public_html/wpad.dat

Corresponding From the access log:
Code:
My public servers static ip address - - [16/Nov/2011:00:15:48 -0600] "GET /wpad.dat HTTP/1.1" 404 1010 "-" "SeaPort/3.0"
My public servers static ip address - - [16/Nov/2011:00:15:48 -0600] "GET /wpad.dat HTTP/1.1" 404 1010 "-" "SeaPort/3.0"
My public servers static ip address - - [16/Nov/2011:00:15:48 -0600] "GET /wpad.dat HTTP/1.1" 404 1010 "-" "SeaPort/3.0"
My public servers static ip address - - [16/Nov/2011:00:15:48 -0600] "GET /wpad.dat HTTP/1.1" 404 1010 "-" "SeaPort/3.0"
My public servers static ip address - - [16/Nov/2011:00:15:48 -0600] "GET /wpad.dat HTTP/1.1" 404 1010 "-" "SeaPort/3.0"
My public servers static ip address - - [16/Nov/2011:00:15:48 -0600] "GET /wpad.dat HTTP/1.1" 404 1010 "-" "SeaPort/3.0"

Fromm Google: SeaPort is Microsoft SeaPort Search Enhancement Process.
http://www.brighthub.com/computing/windows-platform/articles/25609.aspx
I don't have this installed on my desktop.
 
Don't read to much into those user-agent strings. I can create any user-agent I want. It's just a string the HTTP client sends to the server.

The requests aren't originating on your desktop so it's no use looking there. All those requests come from the server itself.

I see you only posted a limited process list. Keep in mind the offending application may be running on some other account besides root. Could you post a full list?
 
Interesting. Note the time stamps for all six requests are identical. Look back through the log. Are all the requests for wpad.dat from SeaPort, or do some have a different user agent?

My feeling is this is some webmin or PHP exploit looking for a bigger vulnerability. Check the webmin logs and other logs to see what else was happening at that time. Install ports-mgmt/portaudit if you don't have it already. Deinstall anything unnecessary.
 
Thank you wblock:
Please see below:

Code:
"My servers public IP address" - - [16/Nov/2011:02:03:32 -0600] "GET /wpad.dat HTTP/1.1" 404 999 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.2 (KHTML, like Gecko) Chrome/15.0.874.120 Safari/535.2"
"My servers public IP address" - - [16/Nov/2011:02:03:32 -0600] "GET /wpad.dat HTTP/1.1" 404 999 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.2 (KHTML, like Gecko) Chrome/15.0.874.120 Safari/535.2"
"My servers public IP address" - - [16/Nov/2011:02:03:34 -0600] "GET /wpad.dat HTTP/1.1" 404 1010 "-" "-"
"My servers public IP address" - - [16/Nov/2011:02:03:35 -0600] "GET /wpad.dat HTTP/1.1" 404 1010 "-" "System.Net.AutoWebProxyScriptEngine/2.0.50727.5448"

Check out the third entry below from 157.55.16.221 a request for /wpad.dat. That's an ip for microsoft.com.
Code:
157.55.16.221 - - [16/Nov/2011:02:00:45 -0600] "GET /customer_testimonials.php?cPath=80&products_id=2216&&testimonial_id=24 HTTP/1.1" 200 5392 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
157.55.16.221 - - [16/Nov/2011:02:00:45 -0600] "GET /customer_testimonials.php?cPath=94_206&products_id=9041&&testimonial_id=6 HTTP/1.1" 200 5579 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
157.55.16.221 - - [16/Nov/2011:02:00:56 -0600] "GET /"My servers public IP address" - - [16/Nov/2011:02:03:32 -0600] "GET /wpad.dat HTTP/1.1" 404 999 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.2 (KHTML, like Gecko) Chrome/15.0.874.120 Safari/535.2"
"My servers public IP address" - - [16/Nov/2011:02:03:32 -0600] "GET /wpad.dat HTTP/1.1" 404 999 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.2 (KHTML, like Gecko) Chrome/15.0.874.120 Safari/535.2"
"My servers public IP address" - - [16/Nov/2011:02:03:34 -0600] "GET /wpad.dat HTTP/1.1" 404 1010 "-" "-"
"My servers public IP address" - - [16/Nov/2011:02:03:35 -0600] "GET /wpad.dat HTTP/1.1" 404 1010 "-" "System.Net.AutoWebProxyScriptEngine/2.0.50727.5448"customer_testimonials.php?cPath=196_203&testimonial_id=22&&testimonial_id=38 HTTP/1.1" 200 6129 "-" "Mozilla/5.0
 
The customer_testimonials URL is actually on your server, right? So bingbot gets that URL from your server. Three minutes later, something claiming to be a Microsoft Javascript web thing runs on the server and looks for wpad.dat.

I don't know anything about server-side Javascript, or why it would wait for three minutes. But that's what it looks like, a Javascript blob that runs on the server. Whether that's even possible with what's installed, or it's just camouflage for an exploit... don't know.

Make sure your Windows system is actually off. Default is just to sleep, and I swear I've caught the Ethernet LEDs on at least one notebook flashing when it was "off", which I assumed to be it sort-of waking up to get yet another huge update.
 
Thanks wblock:

Yes, customer_testimonials.php is on the server. It's been there since early 2009 and the error messages just started so I don't think that has anything to do with it.

No server-side Javascript installed on the server.

No Windows system on the server.
 
Do you have any proxy server running on that server (or a proxy setting activated in e.g. Apache itself)? If it's running on the public IP address it may intercept/redirect HTTP requests from your LAN to 'itself'.
 
DutchDaemon said:
Do you have any proxy server running on that server (or a proxy setting activated in e.g. Apache itself)? If it's running on the public IP address it may intercept/redirect HTTP requests from your LAN to 'itself'.

That's an interesting idea, and reminds me of something I noticed but then forgot earlier: the server IP address in those requests ought to be 127.0.0.1, but it's showing up as the outside address instead.
 
Exactly, that's why I suspect that the original requests come from 'outside', rather than from a local process babbling over lo0 (excluding the customary redirect to localhost:3128 for most transparent Squid proxy setups -- some people do run these setups on one and the same interface, binding the proxy to the public IP address).
 
Thank you DutchDaemon and wblock:

In the /etc/hosts file I have:
Code:
::1                     localhost localhost.my_fqdn.com
127.0.0.1               localhost localhost.my_fqdn.com
192.168.1.10            localhost  mail.my_fqdn.com my_fqdn.com

I have great difficulty understanding networking and I did not setup this part of the server. Could this explain the server requests from my domains IP address? If so, what to do about it as it clouds the issue we are trying to get at.
 
Showing what's in /etc/rc.conf might be helpful. It could be something to do with a firewall, or possibly software that has been installed outside of ports. The person who set it up is not available to answer questions?
 
wblock:
No, not available. Networking was set up a couple years ago.

/etc/rc.conf:
Code:
# -- sysinstall generated deltas -- # Sun Jan  3 12:40:52 2010
# Created: Sun Jan  3 12:40:52 2010
# Enable network daemons for user convenience.
# Please make all changes to this file, not to /etc/defaults/rc.conf.
# This file now contains just the overrides from /etc/defaults/rc.conf.
hostname="my_fqdn.com"
ifconfig_bge0="inet 192.168.1.10 netmask 255.255.255.0 broadcast 192.168.1.255"
defaultrouter="192.168.1.1"
moused_enable="YES"
sshd_enable="YES"
apache22_enable="YES"
apche22_http_accept_enable="YES"
mysql_enable="YES"
ntpdate_enable="YES"
ntpdate_flags="north-america.pool.ntp.org"
postfix_enable="YES"
dovecot_enable="YES"
sendmail_enable="NO"
sendmail_submit_enable="NO"
sendmail_outbound_enable="NO"
sendmail_msp_queue_enable="NO"
clear_tmp_enable="YES"  # Clear /tmp at startup, added 8/30/2011
blanktime="no"
# Starts webmin
webmin_enable="YES"
 
Please also post the output of [cmd=]sockstat -l4[/cmd] (in
Code:
 tags, this time ;))
 
DutchDaemon: I am happy to do that and thank you for your interest.

Code:
USER     COMMAND    PID   FD PROTO  LOCAL ADDRESS         FOREIGN ADDRESS
postfix  local      16532 15 udp4   *:18252               *:*
postfix  smtpd      16528 6  tcp4   *:25                  *:*
dovecot  pop3-login 16526 4  tcp4   *:110                 *:*
dovecot  pop3-login 16526 5  tcp4   *:995                 *:*
dovecot  pop3-login 16521 4  tcp4   *:110                 *:*
dovecot  pop3-login 16521 5  tcp4   *:995                 *:*
www      httpd      16519 3  tcp4 6 *:80                  *:*
www      httpd      16519 4  tcp4   *:*                   *:*
www      httpd      16519 5  tcp4 6 *:443                 *:*
www      httpd      16519 6  tcp4   *:*                   *:*
www      httpd      16518 3  tcp4 6 *:80                  *:*
www      httpd      16518 4  tcp4   *:*                   *:*
www      httpd      16518 5  tcp4 6 *:443                 *:*
www      httpd      16518 6  tcp4   *:*                   *:*
dovecot  pop3-login 16516 4  tcp4   *:110                 *:*
dovecot  pop3-login 16516 5  tcp4   *:995                 *:*
www      httpd      16458 3  tcp4 6 *:80                  *:*
www      httpd      16458 4  tcp4   *:*                   *:*
www      httpd      16458 5  tcp4 6 *:443                 *:*
www      httpd      16458 6  tcp4   *:*                   *:*
www      httpd      16419 3  tcp4 6 *:80                  *:*
www      httpd      16419 4  tcp4   *:*                   *:*
www      httpd      16419 5  tcp4 6 *:443                 *:*
www      httpd      16419 6  tcp4   *:*                   *:*
www      httpd      16418 3  tcp4 6 *:80                  *:*
www      httpd      16418 4  tcp4   *:*                   *:*
www      httpd      16418 5  tcp4 6 *:443                 *:*
www      httpd      16418 6  tcp4   *:*                   *:*
www      httpd      16410 3  tcp4 6 *:80                  *:*
www      httpd      16410 4  tcp4   *:*                   *:*
www      httpd      16410 5  tcp4 6 *:443                 *:*
www      httpd      16410 6  tcp4   *:*                   *:*
www      httpd      16402 3  tcp4 6 *:80                  *:*
www      httpd      16402 4  tcp4   *:*                   *:*
www      httpd      16402 5  tcp4 6 *:443                 *:*
www      httpd      16402 6  tcp4   *:*                   *:*
www      httpd      16349 3  tcp4 6 *:80                  *:*
www      httpd      16349 4  tcp4   *:*                   *:*
www      httpd      16349 5  tcp4 6 *:443                 *:*
www      httpd      16349 6  tcp4   *:*                   *:*
www      httpd      16199 3  tcp4 6 *:80                  *:*
www      httpd      16199 4  tcp4   *:*                   *:*
www      httpd      16199 5  tcp4 6 *:443                 *:*
www      httpd      16199 6  tcp4   *:*                   *:*
www      httpd      16192 3  tcp4 6 *:80                  *:*
www      httpd      16192 4  tcp4   *:*                   *:*
www      httpd      16192 5  tcp4 6 *:443                 *:*
www      httpd      16192 6  tcp4   *:*                   *:*
root     httpd      5049  3  tcp4 6 *:80                  *:*
root     httpd      5049  4  tcp4   *:*                   *:*
root     httpd      5049  5  tcp4 6 *:443                 *:*
root     httpd      5049  6  tcp4   *:*                   *:*
root     perl       26631 6  tcp4   *:10000               *:*
root     perl       26631 7  udp4   *:10000               *:*
dovecot  imap-login 2198  4  tcp4   *:143                 *:*
dovecot  imap-login 2198  5  tcp4   *:993                 *:*
dovecot  imap-login 2197  4  tcp4   *:143                 *:*
dovecot  imap-login 2197  5  tcp4   *:993                 *:*
dovecot  imap-login 2196  4  tcp4   *:143                 *:*
dovecot  imap-login 2196  5  tcp4   *:993                 *:*
root     dovecot    2191  6  tcp4   *:143                 *:*
root     dovecot    2191  7  tcp4   *:993                 *:*
root     dovecot    2191  8  tcp4   *:110                 *:*
root     dovecot    2191  9  tcp4   *:995                 *:*
root     sshd       1116  4  tcp4   *:22                  *:*
root     master     1038  12 tcp4   *:25                  *:*
root     syslogd    636   7  udp4   *:514                 *:*
 
rc.conf doesn't set up a publically-addressable IP address at all. So I'm going to guess that HTTP traffic is being forwarded by the default router, and that the mystery requests are also coming from there.
 
Ok, so the server is running on a 192.168.1.0/24 address. Your server doesn't have a public IP address so there must be something in front of it, some firewall or router perhaps?

I'm guessing there's a slight configuration error that makes every request from the internet appear on the webserver to come from that one public IP address. A NAT configured the wrong way around would do that.
 
Thanks SirDice:
I have a Comcast Business class router with a static IP address then a Cisco RVS4000 4-Port Gigabit Security Router. What would you like to know about the configuration or set up? I'll do my best to provide. I can supply a screen capture if that's more convenient.
 
Can you access the website from the outside, I mean to test?

Try and get a page on your website that doesn't exist but can easily be found in the access log. Something like /thisdoesnotexist.html.

Then search for it in the logs and see what the source address is.
 
Thanks SirDice:
Yes, I am doing that from my cellphone now. Well conceived experiment by the way.
I tried two files. One the infamous wpad.dat.
Code:
[Fri Nov 18 08:22:03 2011] [error] [client 208.54.37.202] File does not exist: /usr/home/usrname/public_html/wpad.dat
[Fri Nov 18 08:22:46 2011] [error] [client 208.54.37.202] File does not exist: /usr/home/usrname/public_html/nohelp.html
208.54.37.202 is my cellphone ip.
 
Ok, that means there's no NAT happening, at least not on traffic that's originating on the internet. That's good. The more things we can rule out the better.

Still leaves us with the question where it does come from :\

Which of the two boxes, the cisco or the comcast, has your public IP address assigned to?
 
SirDice:
Good question. Now my ignorance will show. I believe the Comcast router has the public IP address. In the Comcast router configuration window under
Code:
Gateway Summary/ Network/Internet Settings/WAN Internet IP Address 	My public IP address ###.###.###.###

However in the Cisco router under
Code:
Setup/WAN/Internet Connection Type/Static IP/Internet IP Address/ My public IP address ###.###.###.###

So I don't really know for sure.
 
If I run this command: [CMD=]"ipconfig /displaydns"[/CMD] on my desktop pc in Windows Powershell one of the dns entries is this:
Code:
    Record Name . . . . . : wpad.my_fqdn.com
    Record Type . . . . . : 1
    Time To Live  . . . . : 4237
    Data Length . . . . . : 4
    Section . . . . . . . : Answer
    A (Host) Record . . . : my IP address

Where would this come from?
Code:
Record Name . . . . . : wpad.my_fqdn.com

Could this be something that was configured at my domain name registration site?
 
Possible. Seems like something that should not be available outside the LAN, but DNS registrars vary in quality. Try dig(1) on wpad.my_fqdn.com on the FreeBSD system. If named running on the FreeBSD server, you should use an upstream DNS server for the query:
% dig @upstream-dns-server wpad.my_fqdn.com
 
Back
Top