The Deep End | Paul Venezia » A (late) look at FreeBSD 6.1

September 06, 2006 | Comments: (0)

A (late) look at FreeBSD 6.1

TAGS: ./configure && make && make install, Systems

Although I've run BSD-based production servers for 15 years or so, I find that I tend to get rusty since they basically just sit there doing their thing until there's a hardware failure. Being the proactive fellow that I am, I tend to fire hardware before it can quit, so I decided to take the weekend and build a FreeBSD 6.1-RELEASE server to replace one that had been running 4.9-RELEASE for years.

This server does just about everything, from handling a massive mail volume and the associated filters and virus scanning duties, to mailing lists served via mailman, to hosting over 80 domains for both DNS, mail and Web hosting. This upgrade would be major indeed, upgrading to PHP5, MySQL5, Apache 2.0, and on and on. Also, the disk in this server is standard ATA/133 PATA drives using software RAID. What follows are brief notes on my migration, some FreeBSD basics, and things I wish I'd known at the time.

Installation-

Boot from the bootonly ISO, standard install, construct your partitions on one of the drives (ad0), set the MBR, basically all the defaults, and select the Developer package set. Let the installer do its thing, but don't bother installing any specific ports yet. When it's all over, set up a local user, root password, timezone, and the like. Then, before rebooting, configure the RAID.

RAID1 with gmirror-

Before rebooting the box, type Alt-F4 and get to a shell. Type

sysctl kern.geom.debugflags=16

to remove the mount checks, and then

gmirror label -v -b round-robin gm0 /dev/ad0

will set the mirror up on ad0. Now,

echo geom_mirror_load="YES" > /boot/loader.conf

to instruct the bootloader to head for the mirror, and now, edit /etc/fstab, replacing "/dev/ad0" with "/dev/mirror/gm0" to mount the RAID device instead of the raw device on boot. If all is well, reboot. Following the initial boot, assume root and enter

gmirror insert gm0 /dev/ad2

which will make /dev/ad2 part of the gm0 mirror.

gmirror status will show you the resync status and tell you when the array has completed the rebuild. Also, gstat will show you how hard the mirror's working, and which disks are in use, measured in usecs.

If you're anything like me, one of the first things you'll do is install bonnie from ports and test the mirror's I/O. I found a particularly nasty IRQ problem this way, which resulted in 5.5MB/s writes to the mirror. Fixing that brought the performance into the 26MB/s write, 80MB/s read territory since reads are striped from each disk. Quite nice.

Installing ports

After the first boot, cvsup you ports tree and src trees, then install portupgrade. I found very little change to the src tree, but plenty of ports updates. I dislike the prompting in the installer to install ports directly from there -- I'd much rather do it following the first boot, though portupgrade makes life lots easier.

The FreeBSD Kernel

I wanted to run pf as the firewall, since it's the slickest firewall available on any OS. To do this, cd /usr/src/sys/i386/conf and cp GENERIC HOSTNAME, substituting the system's hostname for HOSTNAME. Use the SMP kernel file if it's a multi-CPU system.

Add these lines to the file:


device pf
device pflog
device pfsync
options ALTQ


and recompile the kernel with make buildkernel from the /usr/src directory. Install the kernel with make installkernel, and reboot.

pf

I really really like pf. The tables structure and configuration file syntax bring happiness to my heart, as does the use of variables within the config, as seen in this example:


ext_if="fxp0"
loop="lo0"
table <smtpblock> { 10.0.0.0/8 }
tcp_services = "{ 25, 53, 20, 21, 22, 80, 443, 110, 143, 993, 995 }"
udp_services = "{ 53 }"
block all
pass quick on $loop all
block drop in quick on $ext_if inet proto tcp from <smtpblock> to ($ext_if) port 25
pass in on $ext_if inet proto tcp from any to ($ext_if) port $tcp_services flags S/SA keep state
pass in on $ext_if inet proto udp from any to ($ext_if) port $udp_services
pass out quick on $ext_if inet proto { tcp, udp, icmp } all keep state
antispoof for $loop
antispoof for $ext_if


That's it, a full configuration with a table (no NAT).

Make sure that you have


pf_enable="YES"
pf_rules="/etc/pf.conf"
pf_flags=""


in /etc/rc.conf, and you're all set. Some handy pf commands are

pfctl -s rules	Show the current rules
pfctl -sa	Show the current rules, connection tables, and statistics
pfctl -vvsa	Show the current rules, connection tables, and statistics, with extra verbosity
pfctl -vvsTable	Show the currently defined table statistics
pfctl -t smtpblock -T show	Show the entries in table smtpblock.
pfctl -t smtpblock -T add -f /tmp/bl	Add a list of IPs from file /tmp/bl to the table
pfctl -s rules	Show the current rules

There's much more to pf, and to this build, but it'll have to wait for another entry. I hope to have time to detail more of the migration, including gotchas encountered when moving between versions of common services, system-level changes and so on.

Suffice it to say, the server was rebuilt and put in place in a few hours' time, and I'm sitting back enjoying the knowledge that aside from minor patches, I won't have to touch it again for another few years.

Posted by Paul Venezia on September 6, 2006 11:23 AM

RATE THIS ARTICLE: