How to set up FreeBSD 8.2 RELEASE i386 server in Amazon Cloud

[vand777]

In this tutorial I'll describe how to install FreeBSD 8.2 RELEASE i386 in Amazon EC2, patch all security holes and install packages built on your build server.

Many thanks to Colin Percival who created AMI instances of FreeBSD in Amazon Cloud (http://www.daemonology.net/freebsd-on-ec2/).

This tutorial will explain how to install FreeBSD 8.2 RELEASE (32 bit) on t1.micro instance. Instances of this family provide a small amount of consistent CPU resources and allow you to burst CPU capacity when additional cycles are available. They are well suited for lower throughput applications and web sites that consume significant compute cycles periodically.

Micro Instance:

613 MB memory
Up to 2 EC2 Compute Units (for short periodic bursts)
EBS storage only
32-bit or 64-bit platform
I/O Performance: Low
API name: t1.micro


Good news are that if you sign up with Amazon EC2 now, then you'll get t1.micro instance for free for 1 year (conditions apply (http://aws.amazon.com/free/)).

As you can see, this instance type does not offer too much CPU power. That's why we'll be using a build server to compile kernel, world and ports, and then upload the binaries to our server in Amazon Cloud. I usually install it locally on my workstation/laptop in virtual machine.

[vand777]

Step 1. Prepare build server

1. Download (ftp://ftp.freebsd.org/pub/FreeBSD/releases/i386/ISO-IMAGES/8.2/FreeBSD-8.2-RELEASE-i386-disc1.iso) and install on your build server FreeBSD 8.2 RELEASE (32 bit).

You have to use i386 version of FreeBSD because this version is used on t1.micro instances in Amazon Cloud (http://aws.amazon.com/ec2/).

2. Patch the system:

I would not recommend to do binary update because we will still need to compile and install world for jails.

Binary update is very simple and quick (see below):

# freebsd-update fetch
# freebsd-update install
# reboot
# uname -a
FreeBSD aws.build.server 8.2-RELEASE-p3 FreeBSD 8.2-RELEASE-p3 #0: Tue Sep 27 18:07:27 UTC 2011 root@i386-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC i386


But we are not afraid of challenges and will update the system through source compilation.

a) Let's synchronise sources.


# cd /usr/share/examples/cvsup/
# cp standard-supfile /etc/freebsd-supfile


The list CVSup mirror sites can be found at http://www.freebsd.org/doc/handbook/cvsup.html#CVSUP-MIRRORS.

Choose the nearest to your location and then amend the following line in /etc/freebsd-supfile:

*default host=CHANGE_THIS.FreeBSD.org


Now synchronise the sources:

# csup /etc/freebsd-supfile


b) Compile/install kernel (GENERIC kernel is all we need for our task) and world:

# cd /usr/src
# make cleanworld && make cleandir
# make buildworld
# make buildkernel
# make installkernel
# reboot


The following should be done in "Single User" mode:


# adjkerntz -i
# mount -a -t ufs
# mergemaster -p
# cd /usr/src
# make installworld
# make delete-old
# mergemaster
# reboot
# uname -a
FreeBSD aws.build.server 8.2-RELEASE-p5 FreeBSD 8.2-RELEASE-p5 #0: Fri Dec 30 21:43:31 GMT 2011 vand777@aws.build.server:/usr/obj/usr/src/sys/GENERIC i386


Because we are not upgrading between major releases we do not need to recompile all ports, or install the proper compat libs, or delete old libs (make delete-old-libs).

[vand777]

3. Install the latest sources for ports:

# portsnap fetch extract


4. Prepare environment for jails.

Let's assume that your build server has only 1 interface (let's say em0) and the IP address on it is 192.168.199.200.

a) Make sure that your build server acts as gateway. The following line must be present in /etc/rc.conf:

gateway_enable="YES"


If it was not there, then add it and restart the machine.

b) Make sure that syslogd listens on em0 only. The following lines must be present in /etc/rc.conf:

syslogd_enable="YES"
syslogd_flags="-b 192.168.199.200"


Do not forget to change 192.168.199.200 to whatever IP address you have assigned to em0 interface.

Then restart syslogd:

# /etc/rc.d/syslogd restart


c) Make sure that you have enabled sshd during the installation. The following line should be present in /etc/rc.conf:

sshd_enable="YES"


d) Make sure that sshd listens on em0 only. Add the following line to /etc/ssh/sshd_config:

ListenAddress 192.168.199.200


Do not forget to change 192.168.199.200 to whatever IP address you have assigned to em0 interface.

Then restart sshd:

# /etc/rc.d/sshd restart


Check that now other services are listening on *.* local address.

# sockstat -4
USER COMMAND PID FD PROTO LOCAL ADDRESS FOREIGN ADDRESS
root sshd 1070 3 tcp4 192.168.199.200:22 *:*
root syslogd 1045 6 udp4 192.168.199.200:514 *:*
vand777 sshd 1017 3 tcp4 192.168.199.200:22 192.168.199.50:49742
root sshd 1014 3 tcp4 192.168.199.200:22 192.168.199.50:49742
root sendmail 905 4 tcp4 127.0.0.1:25 *:*


e) Assign IP addresses for jails on lo0 interface. Let's assume that all jails will have addresses in 192.168.100.0/24 and we will need only 2 jails: "base_system" and "jail_1" (with IP addresses 192.168.100.2/32 and 192.168.100.3/32).

Jail "base_system" will be used to compile kernel, world and ports for main server in Amazon Cloud. Other jails ("jail_1" etc, or whatever other names you will be using) are used to compile ports for jails on our main server.

Add the following lines to /etc/rc.conf:

ifconfig_lo0_alias0="inet 192.168.100.1/24"
ifconfig_lo0_alias1="inet 192.168.100.2/32"
ifconfig_lo0_alias2="inet 192.168.100.3/32"


Now execute the following commands:

# ifconfig lo0 inet 192.168.100.1/24 alias
# ifconfig lo0 inet 192.168.100.2/32 alias
# ifconfig lo0 inet 192.168.100.3/32 alias

[vand777]

f) Install sysutils/ezjail port:

# cd /usr/ports/sysutils/ezjail
# make install clean
# rehash


g) Install world for jails:

# ezjail-admin update -i


h) Install source code for ports for jails:

# portsnap -p /usr/jails/basejail/usr/ports extract


5. Create and start jails

a) Create "base_system" jail:

# ezjail-admin create base_system 192.168.100.2
# cp /etc/resolv.conf /usr/jails/base_system/etc/


b) Create "jail_1" jail (change "jail_1" to more meaningful name :-) ):

# ezjail-admin create jail_1 192.168.100.3
# cp /etc/resolv.conf /usr/jails/jail_1/etc/


c) Start the jails:

# echo 'ezjail_enable="YES"' >> /etc/rc.conf
# /usr/local/etc/rc.d/ezjail.sh start


You should see now the list of running jails with their ids:

# jls
JID IP Address Hostname Path
1 192.168.100.2 jail_1 /usr/jails/ns
2 192.168.100.1 base_system /usr/jails/base_system


Change root passwords in jails:

# jexec 1 csh
jail_1# passwd
jail_1# exit
# jexec 2 csh
base_system# passwd
base_system# exit

[vand777]

5. Setup firewall:

Create /etc/pf.conf:

ext_if="em0"
int_if="lo0"

jails="192.168.100.0/24"

table <firewall> { self }

scrub in all

nat on $ext_if inet from $jails to !<firewall> -> ($ext_if)

block log all

# ssh
pass in on $ext_if proto tcp to ($ext_if) port ssh flags S/SA keep state

# pass out this server
pass out proto { tcp, udp, icmp } all keep state

# pass out jails
pass in on $int_if proto { tcp, udp, icmp } from $jails to !<firewall> flags S/S

antispoof for $ext_if
antispoof for $int_if


Enable PF firewall:

# echo 'pf_enable="YES"' >> /etc/rc.conf
# echo 'pf_rules="/etc/pf.conf"' >> /etc/rc.conf
# echo 'pflog_enable="YES"' >> /etc/rc.conf
# /etc/rc.d/pf start


Reconnect via SSH.

6. Setup SFTP for file transfers between build server and server at Amazon:

a) add user:

# adduser
Username: sftp_user
Full name: SFTP User
Uid (Leave empty for default):
Login group [sftp_user]:
Login group is sftp_user. Invite sftp_user into other groups? []:
Login class [default]:
Shell (sh csh tcsh nologin) [sh]: nologin
Home directory [/home/sftp_user]:
Home directory permissions (Leave empty for default):
Use password-based authentication? [yes]:
Use an empty password? (yes/no) [no]:
Use a random password? (yes/no) [no]:
Enter password: your_password
Enter password again: your_password
Lock out the account after creation? [no]:
Username : sftp_user
Password : ****
Full Name : SFTP User
Uid : 1002
Class :
Groups : sftp_user
Home : /home/sftp_user
Home Mode :
Shell : /usr/sbin/nologin
Locked : no
OK? (yes/no): yes
adduser: INFO: Successfully added (sftp_user) to the user database.
Add another user? (yes/no): no
Goodbye!


b) Change permissions on home directory and create necessary folders:

# chown root:sftp_user /home/sftp_user
# chmod 450 /home/sftp_user
# mkdir /home/sftp_user/ftp
# chmod 750 /home/sftp_user/ftp
# mkdir /home/sftp_user/ftp/pub
# chmod 770 /home/sftp_user/ftp/pub


c) change /etc/ssh/sshd_config:

Remove or comment the below line:

Subsystem sftp /usr/libexec/sftp-server


Then add the following lines:

Subsystem sftp internal-sftp

Match User sftp_user
ChrootDirectory /home/sftp_user/ftp
AllowTCPForwarding no
X11Forwarding no
ForceCommand internal-sftp


Then restart ssh:

# /etc/rc.d/sshd restart


Now you should be able to connect via SFTP. On Windows box I use WinSCP (http://winscp.net/eng/index.php) for these purposes.

Congratulations! Our build server is ready. Now let's setup FreeBSD 8.2 RELEASE (32 bit) in Amazon Cloud.

[vand777]

Step 2. Setup FreeBSD 8.2 RELEASE (32 bit) in Amazon Cloud

1. Create a new account at Amazon EC2 or log in under existing name. Launch AWS Management Console.

2. Click on "EC2" tab, then click on "Instances", then click on "Launch Instance".

3. In the "Request Instances Wizard" click on the "Community AMIs" tab and search for ami-b55f99dc. When found, click on the "Select" button.

4. Choose "Micro (t1.micro)" as instance type and click on the "Continue" button. Then again click on the "Continue" button.

5. Specify the instance name on the next screen and click on the "Continue" button.

6. Create a new Key Pair or choose from existing Key Pairs (if you already have any).

7. Create a new Security Group. Do not forget to add "SSH" rule to enable ssh connections to your server.

8. Click on the "Launch" button and then the "Close" button.

Your new instance should appear in the list now. Wait until its state changes to "Running". Then right click on it and choose "Connect" from the popup menu. You will see instructions and IP address for ssh connectivity.

Connect via ssh with username root and using the key you downloaded previously.

This guide (http://docs.amazonwebservices.com/AmazonEC2/gsg/2007-01-19/putty.html) might help if you've never used putty (http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html) before.

Congratulations! You are in!

[vand777]

Step 3. Setup SFTP for file transfers

1. Add user:

# adduser
Username: sftp_user
Full name: SFTP User
Uid (Leave empty for default):
Login group [sftp_user]:
Login group is sftp_user. Invite sftp_user into other groups? []:
Login class [default]:
Shell (sh csh tcsh nologin) [sh]: nologin
Home directory [/home/sftp_user]:
Home directory permissions (Leave empty for default):
Use password-based authentication? [yes]:
Use an empty password? (yes/no) [no]:
Use a random password? (yes/no) [no]:
Enter password: your_password
Enter password again: your_password
Lock out the account after creation? [no]:
Username : sftp_user
Password : ****
Full Name : SFTP User
Uid : 1002
Class :
Groups : sftp_user
Home : /home/sftp_user
Home Mode :
Shell : /usr/sbin/nologin
Locked : no
OK? (yes/no): yes
adduser: INFO: Successfully added (sftp_user) to the user database.
Add another user? (yes/no): no
Goodbye!


2. Change permissions on home directory and create necessary folders:

# chown root:sftp_user /home/sftp_user
# chmod 450 /home/sftp_user
# mkdir /home/sftp_user/ftp
# chmod 750 /home/sftp_user/ftp
# mkdir /home/sftp_user/ftp/pub
# chmod 770 /home/sftp_user/ftp/pub


3. Change /etc/ssh/sshd_config:

Remove or comment the below line:

Subsystem sftp /usr/libexec/sftp-server


Then add the following lines:

Subsystem sftp internal-sftp

Match User sftp_user
ChrootDirectory /home/sftp_user/ftp
AllowTCPForwarding no
X11Forwarding no
ForceCommand internal-sftp


Then restart ssh:

# /etc/rc.d/sshd restart


Now you should be able to connect to you server in Amazon Cloud via SFTP.

[vand777]

Step 4. Patch the world's source code and binaries and the source code for kernel

In order to launch FreeBSD 8.2 in Amazon Cloud several patches to the source code were made. All these patches can be found in /root/ec2-bits. Source tree in /usr/src reflects these modifications. The safest way to patch the system would be to apply patches from security advisories (http://www.freebsd.org/security/advisories.html) to the existing source tree, then copy it to the build server, build world and kernel, and then move the binaries back to the server in Amazon Cloud, and then install new kernel.

World will be patched directly as described in security advisories (http://www.freebsd.org/security/advisories.html), so we will not need to install the new world. However we will need the new world's binaries if we setup jails.

Let's start with the first security advisory for FreeBSD 8.2 RELEASE.

1. FreeBSD-SA-11:01.mountd (http://security.freebsd.org/advisories/FreeBSD-SA-11:01.mountd.asc)

# mkdir /tmp/patches
# cd /tmp/patches
# fetch http://security.FreeBSD.org/patches/SA-11:01/mountd.patch
# fetch http://security.FreeBSD.org/patches/SA-11:01/mountd.patch.asc
# cd /usr/src
# patch < /tmp/patches/mountd.patch
# cd /usr/src/usr.sbin/mountd
# make obj && make depend && make && make install
# rm /tmp/patches/*.*


2. FreeBSD-SA-11:02.bind (http://security.freebsd.org/advisories/FreeBSD-SA-11:02.bind.asc)

# cd /tmp/patches
# fetch http://security.FreeBSD.org/patches/SA-11:02/bind.patch
# fetch http://security.FreeBSD.org/patches/SA-11:02/bind.patch.asc
# cd /usr/src
# patch < /tmp/patches/bind.patch
# rm /tmp/patches/*.*


We will compile it later because there is another security advisory (http://security.freebsd.org/advisories/FreeBSD-SA-11:06.bind.asc) related to bind.

3. FreeBSD-SA-11:04.compress (http://security.freebsd.org/advisories/FreeBSD-SA-11:04.compress.asc)

# cd /tmp/patches
# fetch http://security.FreeBSD.org/patches/SA-11:04/compress.patch
# fetch http://security.FreeBSD.org/patches/SA-11:04/compress.patch.asc
# cd /usr/src
# patch < /tmp/patches/compress.patch
# cd /usr/src/usr.bin/compress
# make obj && make depend && make && make install
# cd /usr/src/usr.bin/gzip
# make obj && make depend && make && make install
# rm /tmp/patches/*.*


4. FreeBSD-SA-11:05.unix (http://security.freebsd.org/advisories/FreeBSD-SA-11:05.unix.asc)

# cd /tmp/patches
# fetch http://security.FreeBSD.org/patches/SA-11:05/unix2.patch
# fetch http://security.FreeBSD.org/patches/SA-11:05/unix2.patch.asc
# cd /usr/src
# patch < /tmp/patches/unix2.patch
# rm /tmp/patches/*.*

[vand777]

5. FreeBSD-SA-11:06.bind (http://security.freebsd.org/advisories/FreeBSD-SA-11:06.bind.asc)

# cd /tmp/patches
# fetch http://security.FreeBSD.org/patches/SA-11:06/bind8.patch
# fetch http://security.FreeBSD.org/patches/SA-11:06/bind8.patch.asc
# cd /usr/src
# patch < /tmp/patches/bind8.patch
# cd /usr/src/lib/bind/
# make obj && make depend && make && make install
# cd /usr/src/usr.sbin/named
# make obj && make depend && make && make install
# rm /tmp/patches/*.*


6. FreeBSD-SA-11:07.chroot (http://security.freebsd.org/advisories/FreeBSD-SA-11:07.chroot.asc)

# cd /tmp/patches
# fetch http://security.FreeBSD.org/patches/SA-11:07/chroot8.patch
# fetch http://security.FreeBSD.org/patches/SA-11:07/chroot8.patch.asc
# cd /usr/src
# patch < /tmp/patches/chroot8.patch
# rm /tmp/patches/*.*


7. FreeBSD-SA-11:08.telnetd (http://security.freebsd.org/advisories/FreeBSD-SA-11:08.telnetd.asc)

# cd /tmp/patches
# fetch http://security.FreeBSD.org/patches/SA-11:08/telnetd.patch
# fetch http://security.FreeBSD.org/patches/SA-11:08/telnetd.patch.asc
# cd /usr/src
# patch < /tmp/patches/telnetd.patch
# cd /usr/src/lib/libtelnet
# make obj && make depend && make && make install
# cd /usr/src/libexec/telnetd
# make obj && make depend && make && make install
# rm /tmp/patches/*.*


8. FreeBSD-SA-11:09.pam_ssh (http://security.freebsd.org/advisories/FreeBSD-SA-11:09.pam_ssh.asc)

# cd /tmp/patches
# fetch http://security.FreeBSD.org/patches/SA-11:09/pam_ssh.patch
# fetch http://security.FreeBSD.org/patches/SA-11:09/pam_ssh.patch.asc
# cd /usr/src
# patch < /tmp/patches/pam_ssh.patch
# cd /usr/src/lib/libpam/modules/pam_ssh
# make obj && make depend && make && make install
# rm /tmp/patches/*.*


9. FreeBSD-SA-11:10.pam (http://security.freebsd.org/advisories/FreeBSD-SA-11:10.pam.asc)

# cd /tmp/patches
# fetch http://security.FreeBSD.org/patches/SA-11:10/pam.patch
# fetch http://security.FreeBSD.org/patches/SA-11:10/pam.patch.asc
# cd /usr/src
# patch < /tmp/patches/pam.patch
# cd /usr/src/lib/libpam
# make obj && make depend && make && make install
# rm /tmp/patches/*.*


10. Check (http://www.freebsd.org/security/advisories.html) whether there are any new security advisories after FreeBSD-SA-11:10.pam (http://security.freebsd.org/advisories/FreeBSD-SA-11:10.pam.asc) and apply them in the same manner.

At this point you have achieved the following:

Your source tree in /usr/src is up-to-date and still has all necessary EC2 patches from Colin Percival.
Your world has been patched.
You kernel is still vulnerable as we have not compile and install it yet.

[vand777]

Step 5. Recompile kernel and world

We do not compile kernel, world or ports on the server in Amazon Cloud as it can take ages on t1.micro instance to complete it. Let's move the source tree to our build server and compile kernel and world on it.

1. Pack the source tree and prepare it for transfer.

# tar -cpvzf /home/sftp_user/ftp/pub/src.tar.gz /usr/src


2. Download (via WinSCP) src.tar.gz from the server in Amazon Cloud to your local computer and then upload it to your build server.

3. Connect via ssh to our build server.

4. Now we have src.tar.gz file in /home/sftp_user/ftp/pub/ directory on the build server. Unpack source tree:

# cd /usr/jails/base_system/
# tar -xpvzf /home/sftp_user/ftp/pub/src.tar.gz


5. Go to the "base_system" jail:

# jls
JID IP Address Hostname Path
1 192.168.100.3 jail_1 /usr/jails/jail_1
2 192.168.100.2 base_system /usr/jails/base_system

# jexec 2 csh


6. Compile world and kernel:

base_system# cd /usr/src
base_system# make cleanworld && make cleandir
base_system# make buildworld
base_system# make buildkernel KERNCONF=XEN


7. Pack the binaries and prepare for transfer.

base_system# exit
# tar -cpvzf /home/sftp_user/ftp/pub/obj.tar.gz /usr/jails/base_system/usr/obj


8. Download (via WinSCP) obj.tar.gz from your build server to your local computer and then upload it to the server in Amazon Cloud.

[vand777]

Step 6. Install updated kernel and move new world's binaries to the server in Amazon Cloud

Now we have binaries for new kernel and world sitting in /home/sftp_user/ftp/pub/obj.tar.gz on the server in Amazon Cloud.

1. Connect via ssh to the server in Amazon Cloud and unpack it:

# rm -R /usr/obj
# cd /
# tar -xpzf /home/sftp_user/ftp/pub/obj.tar.gz
# cd /usr/jails/base_system/usr/
# mv obj /usr/
# rm -R /usr/jails/base_system


2. Install new kernel (actually it will be installed in a wrong place but will fix it shortly):

# cd /usr/src
# make installkernel KERNCONF=XEN


Enable read-write on the /boot/grup partition in /etc/fstab file ("-" means that the line to be deleted, "+" adds line):

-/dev/da0 /boot/grub ext2fs ro 2 0
+/dev/da0 /boot/grub ext2fs rw 2 0


And reboot to apply these changes:

# reboot


After reboot:

# rm /boot/grub/boot/kernel/kernel
# rm /boot/grub/boot/kernel/kernel.*
# cd /boot/kernel
# mv ./*.* /boot/grub/boot/kernel/
# mv kernel /boot/grub/boot/kernel/


Before the next reboot, let's make sure that you server can act as gateway:

# echo 'gateway_enable="YES"' >> /etc/rc.conf


Also change partition /boot/grup back to readonly mode in /etc/fstab:

-/dev/da0 /boot/grub ext2fs rw 2 0
+/dev/da0 /boot/grub ext2fs ro 2 0


From network performance point of view, it is important to do the following:

# echo 'net.inet.tcp.tso=0' >> /etc/sysctl.conf


Cleanup some mess after yourself:

# rm /home/sftp_user/ftp/pub/obj.tar.gz
# rm /home/sftp_user/ftp/pub/src.tar.gz


Now you can reboot:

# reboot


Check that you are running the new kernel:

# uname -a
FreeBSD ip-XXX-XXX-XXX-XXX 8.2-RELEASE FreeBSD 8.2-RELEASE #0: Sat Dec 31 19:40:37 UTC 2011 vand777@base_system:/usr/obj/usr/src/sys/XEN i386


Congratulations! Your system is now patched and is up-to-date. Let's configure the system.

[vand777]

Step 7. Configure your server in Amazon Cloud

Let's assume that you will have 2 jails: ns and mx.

1. Assign IP addresses for jails on lo0 interface. Let's assume that all jails will have addresses in 192.168.102.0/24.

Add the following lines to /etc/rc.conf:

ifconfig_lo0_alias0="inet 192.168.102.1/24"
ifconfig_lo0_alias1="inet 192.168.102.2/32"
ifconfig_lo0_alias2="inet 192.168.102.3/32"


Now execute the following commands:

# ifconfig lo0 inet 192.168.102.1/24 alias
# ifconfig lo0 inet 192.168.102.2/32 alias
# ifconfig lo0 inet 192.168.102.3/32 alias


2. Make sure that syslogd listens on 192.168.102.1 only. The following lines must be present in /etc/rc.conf:

syslogd_enable="YES"
syslogd_flags="-b 192.168.102.1"


Then restart syslogd:

# /etc/rc.d/syslogd restart


3. Add new user (do not forget to assign it to wheel group to enable su via ssh):

# adduser
Username: vand777
Full name: vand777
Uid (Leave empty for default):
Login group [vand777]:
Login group is vand777. Invite vand777 into other groups? []: wheel
Login class [default]:
Shell (sh csh tcsh nologin) [sh]:
Home directory [/home/vand777]:
Home directory permissions (Leave empty for default):
Use password-based authentication? [yes]:
Use an empty password? (yes/no) [no]:
Use a random password? (yes/no) [no]:
Enter password:
Enter password again:
Lock out the account after creation? [no]:
Username : vand777
Password : *****
Full Name : vand777
Uid : 1002
Class :
Groups : vand777 wheel
Home : /home/vand777
Home Mode :
Shell : /bin/sh
Locked : no
OK? (yes/no): yes
adduser: INFO: Successfully added (andrey) to the user database.
Add another user? (yes/no): no
Goodbye!


4. Make sure that sshd listens on 192.168.102.1 (if you are planning to setup security/openvpn later) and xn0 only. In AWS Management Console you can find out your internal address on xn0 (click on the instance and then look at the "Private DNS" field)

Add the following lines to /etc/ssh/sshd_config (but replace XXX.XXX.XXX.XXX with the value from the "Private DNS" field):

ListenAddress XXX.XXX.XXX.XXX
ListenAddress 192.168.102.1


Also disable login for root user:

-PermitRootLogin yes
+PermitRootLogin no


Change password for root:

# passwd


Then restart sshd:

# /etc/rc.d/sshd restart


Before you close the existing ssh session please check that you can open a new ssh session as the user you just added.

It is more convenient to setup ssh authorisation via a DSA/RSA key than to type the password all the times. It will be also more secure. You can find out how to do this on http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/openssh.html (section "15.10.6 ssh-keygen")

Check that now other services are listening on *.* local address.

# sockstat -4
USER COMMAND PID FD PROTO LOCAL ADDRESS FOREIGN ADDRESS
root sshd 1042 3 tcp4 192.168.102.1:22 *:*
root sshd 1021 3 tcp4 XXX.XXX.XXX.XXX:22 *:*
root syslogd 999 6 udp4 192.168.102.1:514 *:*
root sshd 952 3 tcp4 XXX.XXX.XXX.XXX:22 YYY.YYY.YYY.YYY:64619
root sendmail 843 4 tcp4 127.0.0.1:25 *:*

[vand777]

5. Setup firewall:

Create /etc/pf.conf:

ext_if="xn0"
int_if="lo0"

jails="192.168.102.0/24"

table <firewall> { self }

scrub in all

nat on $ext_if inet from $jails to !<firewall> -> ($ext_if)

block log all

# ssh
pass in on $ext_if proto tcp to ($ext_if) port ssh flags S/SA keep state

# pass out this server
pass out proto { tcp, udp, icmp } all keep state

# pass out jails
pass in on $int_if proto { tcp, udp, icmp } from $jails to !<firewall> flags S/S

antispoof for $ext_if
antispoof for $int_if


Enable PF firewall:

# echo 'pf_enable="YES"' >> /etc/rc.conf
# echo 'pf_rules="/etc/pf.conf"' >> /etc/rc.conf
# echo 'pflog_enable="YES"' >> /etc/rc.conf
# /etc/rc.d/pf start


Close your disconnected SSH session.

Now let's return back to our build server to create packages we will need for our server.

[vand777]

Step 8. Build packages for server in Amazon Cloud

Connect via SSH to your build server. I'll show you how to create packages for base system (will be using jail "base_system"). In the same manner you can create packages for your jails (just use other jails on build server for these purposes to avoid conflicts).

1. Switch to jail "base_system":

# jls
JID IP Address Hostname Path
1 192.168.100.3 jail_1 /usr/jails/jail_1
2 192.168.100.2 base_system /usr/jails/base_system

# jexec 2 csh
base_system#


2. Build ports-mgmt/portmaster:

base_system# cd /usr/ports/ports-mgmt/portmaster
base_system# make install clean
base_system# rehash
base_system# mkdir /tmp/packages
base_system# cd /tmp/packages
base_system# pkg_info | grep portmaster
portmaster-3.11 Manage your ports without external databases or languages
base_system# pkg_create -zvRb portmaster-3.11


Note: switch -R creates also packages this package depends on. Not helpful for building ports-mgmt/portmaster but will be useful when you try to install ports with dependencies.

3. Build ports-mgmt/portaudit:

base_system# cd /usr/ports/ports-mgmt/portaudit
base_system# make install clean
base_system# rehash
base_system# cd /tmp/packages
base_system# pkg_info | grep portaudit
portaudit-0.5.17 Checks installed ports against a list of security vulnerabi
base_system# pkg_create -zvRb portaudit-0.5.17


4. Build sysutils/ezjail:

base_system# cd /usr/ports/sysutils/ezjail
base_system# make install clean
base_system# rehash
base_system# cd /tmp/packages
base_system# pkg_info | grep ezjail
ezjail-3.1 A framework to easily create, manipulate and run FreeBSD ja
base_system# pkg_create -zvRb ezjail-3.1


5. Build whatever other ports you will need on your server in Amazon Cloud. And then move them to /home/sftp_user/ftp/pub:

base_system# exit
# mv /usr/jails/base_system/tmp/packages/*.* /home/sftp_user/ftp/pub


And then upload them to your server in Amazon Cloud.

6. Connect to your server in Amazon Cloud and install packages:

# cd /home/sftp_user/ftp/pub/
# pkg_add portaudit-0.5.17.tgz
# pkg_add portmaster-3.11.tgz
# pkg_add ezjail-3.1.tgz
# rehash
# pkg_info
ezjail-3.1 A framework to easily create, manipulate and run FreeBSD ja
portaudit-0.5.17 Checks installed ports against a list of security vulnerabi
portmaster-3.11 Manage your ports without external databases or languages


7. Install world for jails:

# ezjail-admin update -i


Congratulations!

Now you server is ready. You can create jails (http://erdgeist.org/arts/software/ezjail/), install and configure more packages etc.

[vand777]

Reserved

[vand777]

Reserved2

[vand777]

Reserved3

[vand777]

Reserved4

[vand777]

Reserved5

[vand777]

Sorry for long tutorial. I usually do many more things for basic FreeBSD setup (like setting up openntp, openvpn, keys for ssh, postfix reporting status of each jail to my email etc). But I realised that the tutorial is becoming too long. :-)

Anyway, I hope I've answered the main question of this topic: "How to setup, patch and use FreeBSD server in Amazon Cloud".

Thanks for reading this HOWTO!

[mrab54]

The Private DNS (internal IP) changes when you reboot your instance. If you hard coded that IP into sshd_config as a ListenAddress, you will have an unpleasant surprise the next time to reboot your vm.

Do you have any suggestions on how to work around that? Thanks for the tutorial, btw :)

[vand777]

I always use internal dns name. You can find it when you click on the instance in AWS Management Console.

[vand777]

BTW, FreeBSD 9.0 RELEASE is now available on all Windows instances (http://www.daemonology.net/blog/2012-01-16-FreeBSD-now-on-all-EC2-instance-types.html)!!!

[mrab54]

If you simply "reboot" your vm from the AWS console, the vm will keep the same internal DNS and IP. But, if you "stop" and "start" your vm, it will be assigned new internal DNS and IP.

[vand777]

If you simply "reboot" your vm from the AWS console, the vm will keep the same internal DNS and IP. But, if you "stop" and "start" your vm, it will be assigned new internal DNS and IP.

Thanks! Good to know!

You can set up ssh to listen on an external IP, remove the firewall rule in AWS Console (SSH to this external IP) but keep the relevant PF rule active. In this case nobody will be able to connect to your external IP unless you add the relevent firewall rule in AWS Console (which you will do only if something goes wrong with ssh listening internal DNS/IP).

I always do this on my servers in EC2. Just in case...

[vand777]

You can set up ssh to listen on an external IP, remove the firewall rule in AWS Console (SSH to this external IP) but keep the relevant PF rule active. In this case nobody will be able to connect to your external IP unless you add the relevent firewall rule in AWS Console (which you will do only if something goes wrong with ssh listening internal DNS/IP).

I always do this on my servers in EC2. Just in case...

I was wrong. The above will not help.

What happens when you perform the following actions?

Reboot - When you perform a reboot, the same virtual machine instance is rebooted. The original virtual machine instance that was provisioned to you is never returned back to Amazon. The public IP address will not change.
Assign, reassign, remove an Elastic IP address - An instance can only have one public IP address at any given time. When an instance is assigned an Elastic IP, the EIP becomes its new public IP address and its previous public IP address (if one has already been assigned to it) will be released. For example, if you launch an instance and later assign an Elastic IP to it, the original public IP address of the instance will be replaced by the Elastic IP address. Later, if you disassociate the Elastic IP from the instance, a new public IP address will be assigned to the instance. The original public IP address will not be reassigned to the instance again.
Relaunch - When you relaunch a server, the running instance is terminated and a new instance is launched in its place. The new instance will have new and different public and private IP addresses than its predecessor because it's a different virtual machine that's been allocated to you.
Stop and Restart - When you stop a server, the associated instance is actually terminated. Therefore, when you restart the server, another virtual machine instance will be provisioned to you, so it will have new and different public and private IP addresses.

[vand777]

In order to protect from the problems caused by change in the internal IP, I'm planning to re-configure my Amazon server as follows:

I'll add IP 192.168.102.1/24 to my lo0 interface.
ssh will listen on this IP address only.
I'll add the following redirect rule in my PF configuration:

local_host = "192.168.102.1"
...
rdr on $ext_if proto tcp to ($ext_if) port ssh -> $local_host port ssh


In theory it should work and be a bullet-proof solution.

[shadowbq]

dave @ http://forums.freebsd.org/archive/index.php/t-254.html

So, once you create an EBS Volume and attach it to your running instance, tail your /var/log/messages and you will see the device name assigned by the kernel:


Jun 10 15:55:37 yourhost kernel: xbd2: 10240MB <Virtual Block Device> at device/vbd/2080 on xenbusb_front0
Jun 10 15:55:37 yourhost kernel: xbd2: attaching as da2
Jun 10 15:55:37 yourhost kernel: GEOM: new disk da2


..in this case, it's /dev/da2, So...

# newfs /dev/da2
# mkdir /ebs
# mount /dev/da2 /ebs

...now you have new storage you can access at /ebs.

Don't forget to add an entry in /etc/fstab if you want to mount it at boot:

/dev/da2 /ebs ufs rw 0 2