Traffic Shaping - Bandwidth per Port?

I saw the recent thread about live bandwidth monitoring and learned about several tools I was previously unaware of. However, I have a slightly more complex need and am hopeful somebody will have an idea of how I may accomplish this.

I'm planning on implementing traffic shaping on my mail server running FreeBSD 7.1. I'm doing this because my boss sent a 10 meg file to 17 people last week - everybody was complaining to me about how slow it was to check or send mail. While not common, neither is a situation like this uncommon. From my understanding, traffic shaping should eliminate this problem.

To this end, I've written down everything that leaves the box over the DSL line to the internet and prioritized the traffic for each item.

Code:
1 - Answers to DNS requests
2 - Serving web pages
3 - RBL checks
4 - Antivirus update checks
5 - Mail being retrieved by user via IMAP/POP3
6 - Outgoing e-mail
7 - Responding to pings
8 - Traffic from administration laptop
9 - Traffic from WiFi hotspot router

I intend to allow any of these to borrow from the root queue as needed. If I understand things correctly, this should allow everything to function correctly, even if one item is saturating the link.

The problem I'm having is deciding on how much bandwidth to reserve for each item. There's a total of 512k upstream available on the DSL line, which obviously needs to be sliced up in an efficient manner. I can see the total bandwidth used at any given time as well as the peak by using systat -ifstat 1 (very cool), but it's not broken down or limited by port. Looking at the man page for systat, it doesn't look like this is possible with this tool.

I found ntop online, but it has a ton of dependencies that I really don't think I need. (After I get the traffic shaping policy done, I doubt I'll even use it again.) I can dump the packets on a given port with tcpdump, but there's no method that I can see of measuring the current or average data rate. I thought of writing something that would exploit tcpdump to get what I'm looking for (which I'll probably try before installing something with as many dependencies as ntop), but feel that there must be something like this already.



I have two questions:

Does my plan sound like a good one to accomplish the goal of not allowing one item to monopolize the link?

Is there a tool available, preferably either built into the kernel or as a port with minimal dependencies, that can tell me how much bandwidth is used by a given port during a span of time?
 
If you are using dummynet for the traffic shaping, then you can use relative weights. The simplest method is to create a single pipe with the max bandwidth of the connection. Then create individual queues for each protocol/port. For the weight of each queue, just use the percentage that you want them to have, and make sure the totals of all weights is 100.

For example:
Code:
ipfw pipe 1 config bw 512Kbits/s
ipfw queue 1 config pipe 1 weight 10
ipfw queue 2 config pipe 1 weight 20
ipfw queue 3 config pipe 1 weight 30
ipfw queue 4 config pipe 1 weight 40
Any traffic that goes through queue 1 will have a minimum of 10% of the bandwidth available.

Any traffic that goes through queue 2 will have a minimum of 20% of the bandwidth available.

And so on.

If there is no other traffic currently, then any of the queues can "expand" to use up to the max bandwidth available. The queues set the minimum, the pipe sets the maximum. And the queues share the pipe.

Then you just write the ipfw rules to send traffic through the appropriate pipe:
Code:
# Give outgoing SMTP traffic lowest priority
ipfw add queue 1 tcp from $ip to any 25 out xmit $nic
ipfw add queue 1 tcp from any 25 to $ip in recv $nic established

# Give incoming IMAP/POP3 traffic low priority
ipfw add queue 2 tcp from any to $ip 110,143 in recv $nic
ipfw add queue 2 tcp from $ip 110,143 to any out xmit $nic established

# Give incoming HTTP/S traffic high priority
ipfw add queue 3 tcp from any to $ip 80,443 in recv $nic
ipfw add queud 3 tcp from $ip 80,443 to any out xmit $nic established

# Give DNS traffic highest priority
ipfw add queue 4 udp from $ip to any 53 out xmit $nic
ipfw add queue 4 udp from any 53 to $ip in recv $nic

And so on.

Since it's ADSL, you'll want to create two separate pipes, one for incoming traffic, one for outgoing traffic. And have separate sets of queues for each pipe. And assign traffic to the queues based on the direction of the traffic.

If you're not using ipfw/dummynet, then ignore all the above. ;)
 
With pf, look into ALTQ and the CBQ scheduler. You'll have to install a custom kernel though.
 
Thank you everybody for your time and advice.

I saw those ports mentioned in the other thread referenced above, but it didn't look like they were able to do what I needed from the screenshots. I looked at them closer and tried both. iftop looks good, but does not seem to work on my system (FreeBSD 7.1) - refreshes the IPs of where traffic is going to/coming from, but all the numbers stay at 0 even when I know traffic is being used. trafshow works great and I'm able to specify tcpdump-style filters. I can tell that I'm going to be using it in the future, but there's no peak/average usage that I've been able to find so it won't work for this purpose. :(

The firewall on my box is currently using pf as a firewall. Should I look at changing to ipfw instead? I like the idea of giving each queue a weighting rather than assigning a set minimum amount of bandwidth. (Though I guess it would be simple enough to calculate it out... I'm just being lazy. ;) :e )

I was actually reading about ALTQ and CBQ before I posted this and that was the route I was planning to go. (It's why I recorded everything leaving the box and trying to determine how much bandwidth I should reserve for each; I was using this page for reference.) I did not know that I'd have to install a custom kernel in order to use it though; seems like a lot of hassle to enable a relatively simple feature. I've never done this in BSD before, only in the Linux world, and have to admit that I'm more than a little apprehensive to do it for the first time, especially on a live box...



I guess the next step is to decide whether I want to change firewalls, install a new kernel, or simply tell the users to live with it the way it is. (This project has rapidly grown in scope... :O ) I'm going to start with the FreeBSD handbook for information to aid in this decision - any advice on where else to start reading or advice on what route to go?
 
If you already have PF installed and running and working the way you want it to (minus the bandwidth shaping), then I'd suggest to stick with PF.

Compiling a custom kernel just for ALTQ is as simple as reading /usr/src/sys/conf/NOTES and /usr/src/sys/i386/conf/NOTES for the ALTQ/CBQ options needed and:
Code:
# cd /usr/src/sys/i386/conf
# cp GENERIC MYKERNEL
# ee MYKERNEL
<edit the ident line to use MYKERNEL instead of GENERIC>
<add to the end the "options ALTQ*" lines that are needed>
# config MYKERNEL
# cd ../compile/MYKERNEL
# make cleandepend && make depend && make && make KODIR=/boot/mykernel install
# nextboot -k mykernel
# shutdown -r now
KODIR=/boot/mykernel installs your custom kernel as /boot/mykernel and leaves the original, working kernel as /boot/kernel.

nextboot is used to boot from your custom kernel on the next boot only. If things fail horribly, just reboot, and it will come up with the original, working kernel.

If everything works correctly using the custom kernel, then you can make it the default:
Code:
# cd /boot
# rm -rf kernel.old
# mv kernel kernel.old      (or kernel.works or whatever)
# mv mykernel kernel

Then, you can edit your PF rules to use ALTQ.
 
Thank you so much for your help Phoenix. From your post, it sounds like it's easier than doing a Linux kernel and shouldn't be a big deal.

I went to look at the files you mentioned and (this is embarrassing) discovered that I must have not installed source code with my OS. I went with a minimalist approach this time as opposed to my last <and first> FreeBSD box where I installed everything possible and couldn't keep up with updates. Anyways, I got the source code installed from the original FreeBSD 7.1 install DVD and am ready to start the config/build process. I do have a question before I start though.

I've read that one may remove the unneeded parts from the kernel by deleting the device lines from the config file, thereby shrinking and making the kernel more efficient/stable. Is the difference something that would justify the time required for an average mail/web/DNS server? The current stability is fantastic and CPU loads very low with the standard generic kernel, which is why I have to wonder if I should bother or not. I've stripped down the kernel on more than one Slackware box, but these are doing video surveillance and are under a constant high load and therefore are more likely to benefit from marginal increases in performance.

Also, should I look at applying any kernel patches before proceeding? I'm subscribed to the security mailing list and have seen a few bugs that require a kernel rebuild, but none sound like they affect this machine given how it's being used. (I know these questions are expanding the project of implementing traffic shaping even more, but I don't want to have to go back and repeat things because I didn't think ahead.)
 
Back
Top