Bandwidth restrictions + Postifx

[xy16644]

Where I host my server I only have 512k UPLOAD speeds. So when someone sends a large email (usually with attachments) it slows the entire network down for others that are (say) browsing the internet.

Is there a way in FreeBSD or Postfix to restrict the amount of bandwidth used when delivering emails? It would be nice to restrict it to 128k for SMTP.

Can this be done? I had a look in Webmin under Postfix but I couldn't find anything.

Thank you!

[DutchDaemon]

altq, operated from pf.conf.

[Ruler2112]

What DD said. If you want to use ALTQ, you need to compile a custom kernel with it enabled. (This isn't nearly as hard as it sounds.)

[xy16644]

Thanks everyone.

I haven't ever compiled a kernel before so this is something I am going to have to look into!!

I thought there might be a simple option to restrict outgoing SMTP traffic but I think I have much (interesting) reading ahead of me to achieve this.

[DutchDaemon]

The SMTP protocol has zero knowledge of bandwidth, I'm afraid -- wrong OSI level ;)

You'll have to shape network traffic on the box itself, or on the router behind it (if it has the tools). PF/ALTQ is great at this.

[xy16644]

What I have done (just to get something in place) is to use my router to limit the upload speed to 128kb (out of 512kb). This was quick and painless to do. Only "problem" with doing it this way is it restricts the upload speed of EVERYTHING from my servers IP.

If I did this on FreeBSD using the methods you mentioned in this thread, can I restrict the upload speed JUST for SMTP (port 25) but allow everything else to be able to use the full 512kb for uploads?

[DutchDaemon]

ALTQ can shape traffic based on ports, protocols, hosts, networks, etc. It's just a matter of attaching the right queue to the right PF rule. I posted some links in this thread (http://forums.freebsd.org/showthread.php?t=10297).

[xy16644]

Thanks DD. Boy, do I have much reading to do about ALTQ and compiling kernels! ;-)

[Ruler2112]

I had to do much the same thing (http://forums.freebsd.org/showthread.php?t=7219) recently for pretty much the same problem. My boss would send a huge attachment to two dozen people and people then weren't able to retrieve their e-mail because the connection was monopolized sending the messages out. I have some rules posted on here (http://forums.freebsd.org/showthread.php?t=7536), though I remember they needed a little tweaking when I went to apply them. (IIRC, the order was off - a bunch of rules needed to be moved up closer to the beginning of the file.) There's also straight-forward discussion on how to enable ALTQ and compile a kernel in one of those threads; worked perfectly for me.

Let me know if you need them and I'll post my current pf rule set.

[xy16644]

Many thanks for the offer Ruler2112. I am going to try get to this on the weekend!

I am really pleased with the new router I bought but I can't wait to get this all setup on FreeBSD as its far more powerful/flexible.

[xy16644]

Today I started reading about compiling my very own kernel so I could have ALTQ support so I could limit the outgoing bandwidth of SMTP to stop people saturating my internet connection when they send large attachments.

The first thing I did was to check that my network card is supported by ALTQ at http://man.freebsd.org/altq/4. I have Broadcom Gigabit Ethernet card (bge) so it is supported.

The next thing I need to do is add the support for ALTQ into my kernel. This is where I may need a guiding hand...

I have been reading about compiling the kernel in the book "Absolute FreeBSD" but there were one or two parts that I didn't quite get. The first thing I did was:

cd /usr/src/sys/i386/conf
cp GENERIC ALPHA (the name of my custom kernel)

I then added the following lines to the bottom of the ALPHA kernel:

options ALTQ
options ALTQ_CBQ # Class Bases Queuing (CBQ)
options ALTQ_RED # Random Early Detection (RED)
options ALTQ_RIO # RED In/Out
options ALTQ_HFSC # Hierarchical Packet Scheduler (HFSC)
options ALTQ_PRIQ # Priority Queuing (PRIQ)
options ALTQ_NOPCC # Required for SMP build

And backed up the current kernel:

cp -Rp /boot/kernel /boot/kernel.good

Hopefully I've done that bit correctly. This is were I wasn't too sure of what to do next. In the book it says:

cd /usr/src
make KERNCONF=ALPHA kernel

Will the above compile the kernel and install it? How does it know where to find my ALPHA kernel, do I need to specify the full path to it?

Am I on the right track?? :stud

[DutchDaemon]

The syntax is a matter of semantics. I usually use:

cd /usr/src
make buildkernel KERNCONF=ALPHA
make installkernel KERNCONF=ALPHA

which is the same as

cd /usr/src
make kernel KERNCONF=ALPHA

and

cd /usr/src
make KERNCONF=ALPHA kernel

The path to the kernel configuration file is derived from your platform. If you're on a 32-bit system, the make process will automatically use /usr/src/sys/i386/conf, and if you're on an amd64 system, it will use /usr/src/sys/amd64/conf. See uname -m

The make commands you see above are part of a full OS/kernel (re)build, so if you want to see them in a meaningful context, see 'the 11 steps' in /usr/src/Makefile.

[xy16644]

Thank you DD! Its good to know that those 3 commands mean the same thing. I had a look at the 11 steps. I often wondered what "make buildworld" was when I read about it on the forum.

So am I correct in saying that I just need to run:

cd /usr/src
make KERNCONF=ALPHA kernel


Reboot and my new kernel will be installed and ALTQ will then be available?

[DutchDaemon]

Yep.

[xy16644]

Cheers, I'll give it a try tomorrow morning!

Am I correct in saying that when I run my machine with a custom kernel, I can't run this to keep my system up to date with:

freebsd-update fetch
freebsd-update install


Will I come across any other difficulties in the future when running a custom kernel like upgrading from 8.0 to 8.1 or even 9.0?

[DutchDaemon]

Please search the forums. This has been discussed several times, at length. E.g. http://forums.freebsd.org/showpost.php?p=63290&postcount=10

See also: http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/updating-upgrading-freebsdupdate.html

The freebsd-update utility can automatically update the GENERIC kernel only. If a custom kernel is in use, it will have to be rebuilt and reinstalled after freebsd-update finishes installing the rest of the updates. However, freebsd-update will detect and update the GENERIC kernel in /boot/GENERIC (if it exists), even if it is not the current (running) kernel of the system. Note: It is a good idea to always keep a copy of the GENERIC kernel in /boot/GENERIC.

Your previous (pre-ALTQ) kernel will end up as /boot/kernel.old. Move it to /boot/GENERIC, and you should be set. Make sure to also 'freebsd-update' the sources ('src' in freebsd-update.conf's Components), so you can rebuild your custom kernel based on the latest patch level.

[xy16644]

Thanks again DD for taking the time to respond.

I now have my very first working custom kernel!!!

alpha# uname -a
FreeBSD alpha.domain.com 8.0-RELEASE-p2 FreeBSD 8.0-RELEASE-p2 #0: Sun Jan 24 09:14:42 GMT 2010 xy16644@alpha.domain.com:/usr/obj/usr/src/sys/ALPHA i386

When I now run:

pfctl -s info

It doesn't say that ALTQ is disabled.

Next step, reading up on ALTQ and getting it to work with SMTP...

[DutchDaemon]

Good luck. There are a few threads about it on the forums. If you run into trouble, open a new topic, or add to the existing ones.