IPFW Checking a website for IP-address changes

M

mrrc

I need to check a website for changes to its IP address.
If the address has changed, I need to update the existing firewall rule, replacing the previous value with the new address.

The script can be run through a scheduled cron job.

So, here's how I see it:

1) When running the script, read the IP address from the existing ipfw rule:

ipfw show | grep 12000
12000 175 46077 allow ip from any to XXX.XXX.XXX.YYY keep-state :default
Click to expand...

2) Check if the site's current IP address has changed:

host site.lovechange.com
site.lovechange.com is an alias for s1000x.cdn.ngenix.net.
s1000x.cdn.ngenix.net has the address XXX.XXX.XXX.YYY
Click to expand...

3) If the IP address from the check hasn't changed, terminate the script.

4) If the IP address changed during the host check and doesn't match XXX.XXX.XXX.YYY, execute:
ipfw delete 12000
ipfw add 12000 allow ip from any to XXX.XXX.XXX.YYY
Click to expand...

If anyone uses a similar ready-made script, I'd be happy to help with the implementation.
Perhaps there is a simpler standard solution to the problem?
 
[...] is an alias for s1000x.cdn.ngenix.net.
Click to expand...

As this seems to be a cdn, this won't work - depending on various factors (physical location, congestion within the cdn, etc pp) that site might (will) be served from other endpoints and the PTR record will point to a completely different host/IP.

What exactly is the goal here? I suspect there's a much simpler solution than relying on fluctuating DNS records.

And FTR: PF (and I guess ipfw too?) can handle hostnames instead of IPs - at startup/reload it will resolve the hostname and use the current IP for the resulting rules. (of course DNS *must* be available at boot for that host, or loading the ruleset will fail)
 
Use a table. Add/remove IP addresses to/from the table, not rewrite an entire rule. Also makes it easier to compare.
 
sko said:
And FTR: PF (and I guess ipfw too?) can handle hostnames instead of IPs - at startup/reload it will resolve the hostname and use the current IP for the resulting rules. (of course DNS *must* be available at boot for that host, or loading the ruleset will fail)
Click to expand...
Yes, I know.If DNS is unavailable for any reason at startup, that will be a problem. Now, if the ipfw could periodically recheck the DNS name rule to make sure it's still up-to-date, that would be great. But the ipfw can't do that?
sko said:
As this seems to be a cdn, this won't work - depending on various factors (physical location, congestion within the cdn, etc pp) that site might (will) be served from other endpoints and the PTR record will point to a completely different host/IP.

What exactly is the goal here? I suspect there's a much simpler solution than relying on fluctuating DNS records.
Click to expand...
You may be right. But address changes are relatively rare, and I would be satisfied with the proposed solution in the form of a verification script.The goal is to keep the domain address up-to-date and accessible from a closed network.
SirDice said:
Use a table. Add/remove IP addresses to/from the table, not rewrite an entire rule. Also makes it easier to compare.
Click to expand...
Yes, thank you, that's a reasonable idea. But is there a working example of a script that would perform the check described?
 
mrrc said:
But is there a working example of a script that would perform the check described?
Click to expand...
Not that I'm aware of, but I'm not a ipfw(8) user so never looked for one.

But the general gist is to query the contents of the table (sort the results), then use drill -Q mysite.example.com (sort those results too), and compare both outputs. Should be relatively easy to script. You're going to want to sort the results as the order of both queries could be different while having the same list of IP addresses.
 
mrrc said:
But is there a working example of a script that would perform the check described?
Click to expand...
Maybe this way:

In pf.conf :
Code: 
table <pulsingsite> file "/etc/pf/pulsingsite"

In /etc/pf/pulsingsite :
Code: 
pulsingsite.xyz

And cronjob :
Code: 
*/5 * * * * pfctl -t pulsingsite -T replace -f /etc/pf/pulsingsite
 
A website may have more than one ip address. I use this script, written in 5 minutes on my knees. The resulting ip list is entered into the ipfw table.
Code: 
#!/bin/sh
FILE="/usr/local/etc/zsites/sites.hosts"
IPFILE="/usr/local/etc/zsites/sites"
TMPFILE="/usr/local/etc/zsites/sitestmp"

echo -n >$IPFILE

cat $FILE|grep -v ^#|\
while read host;\
     do dig +short $host| grep -v '\.$'>>$TMPFILE
done
cat $TMPFILE | sort -u >$IPFILE
rm $TMPFILE
 
mrrc said:
Yes, thank you, that's a reasonable idea. But is there a working example of a script that would perform the check described?
Click to expand...

A simple one-liner using ipfw table add and the uri should suffice. The uri gets resolved and then added to the table (if ipfw works like pfctl - never used ipfw, just took a glimpse at the available commands for ipfw(8)). If something is already in a table, it gets ignored - so you can just run this periodically via cron.
 
freebsd_lg said:
on my knees
Click to expand...
A very simple and working solution, just what I needed!
You can check several problematic sites at the same time.
Below, I've added a table cleanup and rewrite. It runs on cron every half hour, and everything works, thanks.
It would also be good idea to check if the entries in dyn-ip.list haven't changed, so there's no point in resetting the table.

Code: 
#!/bin/sh
FILE="/root/scripts/sites.hosts"
IPFILE="/root/scripts/dyn-ip.list"
TMPFILE="/root/scripts/sitestmp"

echo -n >$IPFILE

cat $FILE|grep -v ^#|\
while read host;\
     do dig +short $host| grep -v '\.$'>>$TMPFILE
done
cat $TMPFILE | sort -u >$IPFILE
rm $TMPFILE

/sbin/ipfw table dyn-ip flush
/bin/cat /root/scripts/dyn-ip.list | while read ip; do
/sbin/ipfw table dyn-ip add $ip
done
 
I had a very similar task recently.
It was required to enable a service by hostname using ipfw, but the hostname looks like behind CDN. It has about 10 IPs who changing over time. Also the hostname returns 2 IPs.
So I used cron to run every hour " /sbin/ipfw table N add hostname.com"
This just collected all IPs in the table because DNS returned different IPs over time.
Then I used the Table in an ipfw rule to allow the hostname.
 
mrrc said:
/sbin/ipfw table dyn-ip flush
/bin/cat /root/scripts/dyn-ip.list | while read ip; do
/sbin/ipfw table dyn-ip add $ip
done[/CODE]
Click to expand...
It will be much faster this way.
Code: 
IPFWFULES="/root/scripts/ipfwrules"
cat IPFILE\
while read ip;\
     do echo table dyn-ip add $ip>>$IPFWFULES
done
ipfw -nq $IPFWFULES&&ipfw table dyn-ip flush&&ipfw -q $IPFWFULES&&logger dyn-ip table updated
 
im said:
I had a very similar task recently.
Click to expand...
A simple and excellent idea. Perfect for problematic sites that frequently change their IP-addresses. For example, like crl.globalsign.com
The only thing is that if there are no changes, such a launch will generate messages like:
ipfw: Adding record failed: record already exists
Click to expand...
 
freebsd_lg said:
It will be much faster this way.
Click to expand...
Apparently, I made a mistake somewhere.
./dyn-ip-hosts.sh
./dyn-ip-hosts.sh: 18: Syntax error: "do" unexpected

Code: 
#!/bin/sh
FILE="/root/scripts/sites.hosts"
IPFILE="/root/scripts/dyn-ip.list"
TMPFILE="/root/scripts/sitestmp"
IPFWFULES="/root/scripts/ipfwrules"

echo -n >$IPFILE

cat $FILE|grep -v ^#|\
while read host;\
        do dig +short $host| grep -v '\.$'>>$TMPFILE
done
cat $TMPFILE | sort -u >$IPFILE
rm $TMPFILE

cat IPFILE\
while read ip;\
        do echo ipfw table dyn-ip add $ip>>$IPFWFULES
done
ipfw -nq $IPFWFULES&&ipfw table dyn-ip flush&&ipfw -q $IPFWFULES&&logger dyn-ip table updated
 
And this is a better way:
Code: 
while read ip; do
 {...}
done < $IPFILE
This doesn't use a subshell.

This does:
Code: 
cat $IPFILE | 
while read ip; do
  {...}
done
The subshell is going to pose a problem if you want to count the number of lines for example. Because it's a subshell any variable that's modified will be gone when the subshell ends (parent shell doesn't know about variables from a child process).
 
Also, don't do this:
Code: 
TMPFILE="/root/scripts/sitestmp"
{...}
rm $TMPFILE

Use mktemp(1) to create a unique and random named temp file.
 
SirDice said:
And this is a better way:
Click to expand...
Unfortunately, I'm not a programmer.Based on the general script code I provided above (with recommendations from freebsd_lg), what adjustments should be made for optimal performance?And so that the table list is cleared and updated only when the IP address of one of the monitored sites changes.
 
mrrc said:
what adjustments should be made for optimal performance
Click to expand...
It's not about performance.

Code: 
dice@molly:~/temp % cat while_subshell.sh
#!/bin/sh

count=0
cat multiline.txt |
while read line; do
        count=$((count+1))
done

echo Number of lines: $count
Code: 
dice@molly:~/temp % cat while_nosubshell.sh
#!/bin/sh

count=0
while read line; do
        count=$((count+1))
done < multiline.txt

echo Number of lines: $count

Code: 
dice@molly:~/temp % cat multiline.txt
line 1
line 2
some more
dice@molly:~/temp % sh while_subshell.sh
Number of lines: 0
dice@molly:~/temp % sh while_nosubshell.sh
Number of lines: 3
Note how one script counts 0 lines while the other correctly counted 3.
 
mrrc said:
The only thing is that if there are no changes, such a launch will generate messages like: ipfw: Adding record failed: record already exists
Click to expand...
Just FYI:
I am using MAILTO="" line in my cron before the command to suppress unwanted output/mail.
Also you can wrap the command as a separate script and redirect all unwanted output inside of a script, including errors to /dev/null. And then call the script from cron.
 
You must log in or register to reply here.
Back
Top