Systems Administration Problem Solvers
www.tediosity.com
All Things FreeBSD
To upgrade the version of Perl installed on a FreeBSD server:
vi /etc/make.conf
Add this line (adjust for Perl version):
DEFAULT_VERSIONS+=perl5=5.22
Execute on command line (adjust for Perl version):
pkg set -o lang/perl5.16:lang/perl5.22
Execute on command line (adjust for Perl version):
portupgrade -o lang/perl5.22 -f perl-5.16.\*
Execute on command line:
portupgrade -rf perl*
AMD64 Kernel FreeBSD machines are good about autotuning. i386, on the other hand, needs to be adjusted manually.
1. Rebuild your kernel
a. Disable/wipe all drivers you are not using or at least those you are highly unlikely to ever use.
b. Add: options KVA_PAGES=512
c. Recompile/Install new kernel
2. Add these parameters to /boot/loader.conf
a. vm.kmem_size=”1024M”
b. vm.kmem_size_max=”2048M”
c. vfs.zfs.arc_max=”256M”
d. vfs.zfs.vdev.cache.size=”40M”
This will get the machine online without it crashing with vm.kmem errors. ZFS will bring an untuned i386 machine down to its knees with kernel panics quickly!
Adjust the above variables to your tastes. My test platform is a Dual Xeon with 4GB of RAM.
This is what your rc.conf should contain to configure LAGG with FreeBSD:
### LAGG NFS Interface ###
ifconfig_bce3=”mtu 9000 up”
ifconfig_bce0=”mtu 9000 up”
cloned_interfaces=”lagg0″
ifconfig_lagg0=”laggproto roundrobin laggport bce3 laggport bce0″
ipv4_addrs_lagg0=”10.10.40.10/24″
Change the bce* interfaces to whatever your server is using for the network interfaces.
The “mtu 9000 up” is correct!
This script will replicate a ZFS pool to another FreeBSD machine. The sync process is quick, after the initial copy, and depending upon how much data changed.
Download this shell script: http://www.tediosity.com/zfsrep.sh
This script was written by another author for Solaris and I have fixed it to work on FreeBSD.
I chose to use: /root/zfsrep as the script location.
mkdir -p /root/zfsrep/zfsrep.snapshots
touch /root/zfsrep/zfsrep.log
cp zfsrep.sh /root/zfsrep/
vi zfsrep.sh and modify the e-mail address and location of the script (if you are not using /root/zfsrep)
Initial run:
/root/zfsrep/zfsrep.sh sinit nfs/datastore nfs/datastore 10.10.30.20
Subsequent runs:
/root/zfsrep/zfsrep.sh sync nfs/datastore nfs/datastore 10.10.30.20
Create a cronjob and forget about it.
If you make any changes, error fixes, or enhancements please e-mail them to me!! I love seeing other people’s creativity and putting their ideas to work in a production environment.
email: admin -at- tediosity.com
zfs list -t snapshot
Example Output:
nas1# zfs list -t snapshot
NAME USED AVAIL REFER MOUNTPOINT
nfs/datastore@rep-init-20110113013713 18K – 21K –
nfs/datastore@rep20110113013825 311M – 311M –
nfs/datastore@rep20110113015314 0 – 68K –
If you have setup LAGG on FreeBSD and are noting that one NIC is getting all of the traffic.
Switch to roundrobin and check results.
The primary (“MASTER”) NIC will always receive the incoming traffic. With roundrobin all of the traffic will exit all NICs assigned to the LAGG.
FreeBSD recently introduced a disk replication setup: HAST.
This is FreeBSD’s answer to DRDB for Linux.
Some very good blog posts, with scripting, can be found here:
My issues with HAST are its feeling of instability and the addition of numerous points of failure… sloppy design? The system, essentially, adds “virtual harddrives”. These “virtual harddrives (devices)” sit on top of the drives to be mirrored, the HAST devices receive the data, and then distribute the data to real harddrives on the primary server and to the secondary server. I found this to be a nightmare – more Linux style than FreeBSD.
I would not consider putting HAST in production with ZFS. ZFS is created on top of the “virtual harddrives (devices)”. That is 3 layers (ZFS + HAST devices + actual Harddrives). If HAST messes us, then your ZFS tank disappears. If you restart the HAST daemon then you have to make sure to export your ZFS first. If you do not export the ZFS pool, then ZFS will lock up and a hard reboot is needed. Then on reboot HAST has to be live first, and then your ZFS pool is reimported. All of the aforementioned means downtime. It my lab experiments I simply removed it from the servers and have deemed it not suitable for 24x7x365 applications.
For a ZFS pool replication script visit this post: http://www.tediosity.com/2011/05/31/zfs-replication-on-freebsd/
exit 255 at /usr/local/bin/spamd line 2588. /usr/local/etc/rc.d/sa-spamd: WARNING: failed to start spamd
The above error can be resolved by typing:
sa-update
Follow all these steps in this exact order…this is from hard-won experience!!!
1. cvsup the correct /usr/src – example cvsup files can usually be found in /root/sup. Normally we follow the security releases e.g. RELENG 6_2 rather than stable e.g. RELENG_6
2. read /usr/src/UPDATING to make sure nothing bad is going to happen.
3. rm -rf /usr/obj or weird bad shit can happen. This is #1 cause of failed installworlds.
4. check /etc/make.conf is sensible…usually it is but for a new box it won’t be.
5. mergemaster -p – otherwise installworld can fail with new users and groups, or worse it doesn’t boot.
6. make -j16 buildworld
7. check kernel config file, usually you can use the one from the previous release, not always. Using WWW here.
8. make buildkernel KERNCONF=WWW
9. make installkernel KERNCONF=WWW
10. [OPTIONAL] reboot to see if the kernel is OK. This is a 2-edged sword, because if you don’t do it, you are screwed if there is a problem with the kernel and you’ve already done installworld. If you do it, the kernel comes up incompatible with userland, and in the worst case, tcp/ip is broken!
11. make installworld
12. mergemaster – take care not to overwrite /etc/ssh/sshd_config and lock yourself out of the box!
13. reboot and pray to the FSM!
Edit /etc/rc.conf
The primary IP of the machine will be defined by a line that looks similar to this:
ifconfig_rl0=”inet 192.168.1.2 netmask 255.255.255.0″
To add additional IPs you simply:
ifconfig_rl0_alias0=”inet 192.168.1.3 netmask 255.255.255.255″
ifconfig_rl0_alias1=”inet 192.168.1.4 netmask 255.255.255.255″