Upgrading FreeBSD 5.2.1 with DA to FreeBSD 5.3

[rhoekman]

(WORK IN PROGRESS!)

Disclaimer: I'm not responsable if this howto renders your machine inoperable! Use at your own risk! This is also not for the inexperienced even though I use pictures ;)

wdv asked me to write a little howto on upgrading FreebSD 5.2.1 running DA to 5.3 so here we go:

Preperation:

MAKE A FULL SYSTEM BACKUP FIRST! :)

SSH: (thanks to Chrysalis)

If you use openssh in the FreeBSD base then 5.3 by default disables password authentication, so edit /etc/ssh/sshd_config before you do the reboot after make installworld, and change the following 2 lines or you will be locked out if you relying on password login. You dont need to do this if you use openssh-portable port.

PasswordAuthentication yes
PermitEmptyPasswords no

DNS:
To avoid dns to be chrooted when you have to reboot the machine put this in your /etc/rc.conf

named_chrootdir=""

You also need to make sure that there are 2 groups present in /etc/group:

proxy:*:1049:
authpf:*:1050:

the last number is the group ID on my system so check the file and pick a follow up number that is free on your machine

Optional preparation:

As FreeBSD 5.3 is fully cleaned of debug and test codes it now makes sense to optimize it when the kernel and the system compiles for you architecture.

In /etc/make.conf I put this in:

CPUTYPE=p4
COPTFLAGS= -O2 -pipe -funroll-loops -ffast-math

Please be aware that you use the right architecture when you specify CPUTYPE!

More information on what all the flags do can be found here:

http://silverwraith.com/papers/freebsd-kernel.php

Commencing the upgrade:

This is also something that is subscribed very well in the FreeBSD Handbook here:

http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/cvsup.html

I consider myself a very lazy admin and I found a perfect script that uses all that is described in the handbook http://lvl.sourceforge.net/readme.php?txt=update/README

This script uses CVSup so if you do not have CVSup installed please do so by using:

pkg_add -r cvsup-without-gui

now get the script: http://prdownloads.sourceforge.net/lvl/update.tar.gz?download

unpack it and launch it with ./update.sh -m

It will check for latest version and permissions then it will prompt you with a nice menu. If you follow the instructions in the exact order your system will be upgraded to 5.3 RELEASE!

http://www.siteserver.nl/screenshots/1.gif
http://www.siteserver.nl/screenshots/2.gif
http://www.siteserver.nl/screenshots/3.gif
http://www.siteserver.nl/screenshots/4.gif
http://www.siteserver.nl/screenshots/5.gif
http://www.siteserver.nl/screenshots/6.gif

(if you make a mistake you can clear the setting using ./update.sh -C restart again with ./update.sh -m)

- After probably 2 hours you will be prompted for a reboot. So reboot.

- After the reboot run the script again with ./update -m

- After this it will install the compiled system files.

The tricky part is that mergemaster will be called after installing the system files. Here you are on you own because you have to review every file that needs merging or not! This usually takes about 20 minutes.

Depending if your machine took all the beating it should work.

Now it is time to rebuild DA (custom apache)

cd /usr/local/directadmin/customapache/
./build clean
./build update
./build all

Portupgrading (thanks to Chrysalis)

Using portupgrade to recompile all ports with the new libraries and compiler:

After upgrading to 5.3 all multi threading ports are supposed to be rebuilt but since there is no easy way to tell which are the correct ports to do just rebuild all your ports, I used the following command (requires the sysutils/portupgrade port).

'portupgrade -afvrR', this will reinstall all currently installed ports (can take a long while, took nearly 4 hours on my celeron 2ghz). Note tho that if you have built ports with custom options then you need to either add the options to /usr/local/etc/pkgtools.conf in the 'make args' section or recompile the port yourself manually.


--
personal note: Though everything is running smoothly here at my end. I'm not really happy with the enormous change this upgrade has taken on the machine. A lot of files during mergemaster still need reviewing. A full fresh install is advised if you have the chance!

[rhoekman]

I'm aware that some issues will arise and that the above is not all to clear at some points. In my defense I had a very busy schedule this week and look at the time! :D

Please submit you findings so we can iron out pitfalls!

[Chrysalis]

good guide and that app you found on sourceforge will help those who are lazy or inexperienced. However I believe 5.3 uses the same filesystem as 5.0,5.1 and 5.2.1 which is UFS2 so you should already be using it if you upgraded from an older 5.x.

After upgrading to 5.3 all multi threading ports are supposed to be rebuilt but since there is no easy way to tell which are the correct ports to do just rebuild all your ports, I used the following command (requires the sysutils/portupgrade port).

'portupgrade -afvrR', this will reinstall all currently installed ports (can take a long while, took nearly 4 hours on my celeron 2ghz). Note tho that if you have built ports with custom options then you need to either add the options to /usr/local/etc/pkgtools.conf in the 'make args' section or recompile the port yourself manually.

Also one more thing, if you use openssh in the FreeBSD base then 5.3 by default disables password authentication, so edit /etc/ssh/sshd_config before you do the reboot after make installworld, and change the following 2 lines or you will be locked out if you relying on password login. You dont need to do this if you use openssh-portable port.

PasswordAuthentication yes
PermitEmptyPasswords no

[bjseiler]

""Also one more thing, if you use openssh in the FreeBSD base then 5.3 by default disables password authentication, so edit /etc/ssh/sshd_config before you do the reboot after make installworld, and change the following 2 lines or you will be locked out if you relying on password login. You dont need to do this if you use openssh-portable port. ""

PAY ATTENTION TO THIS ONE

Some of us don't read the updating file like we should ........ and had to take a late night jaunt to the data center to add users :-(

[rhoekman]

Originally posted by Chrysalis
good guide and that app you found on sourceforge will help those who are lazy or inexperienced. However I believe 5.3 uses the same filesystem as 5.0,5.1 and 5.2.1 which is UFS2 so you should already be using it if you upgraded from an older 5.x.


You are right I'll correct that in the note. Don't want to alarm people. I forgot that I CVSupped from 4.8 to 5.1 so my machine is still using UFS. Thanks for your input!

[wdv]

Did someone else already use this how to to upgrade from 5.2.1 to 5.3?

Would be nice to know someone else did it succesfully by following this tutorial :)

By the way, this mergemaster stuff and portupgrading stuff isn't gonna be fun, is it?

[Chrysalis]

I have upgraded 2 boxes to 5.3 (one runs DA) and I believe rhoekman has done at least 1 box, this guide will work if you follow it and take your time don't rush through it.

[Chrysalis]

I have found a possible minor bug wih bind 9.3.0 and FreeBSD 5.3, when I send the HUP singal to bind instead of reloading the config it dies, has anyone else got this problem?

[rhoekman]

Originally posted by wdv
By the way, this mergemaster stuff and portupgrading stuff isn't gonna be fun, is it?

No this is the peskiest task that awaits you.. ;)

Originally posted by Chrysalis

I have found a possible minor bug wih bind 9.3.0 and FreeBSD 5.3, when I send the HUP singal to bind instead of reloading the config it dies, has anyone else got this problem?

Everything was recompiled with portupgrade on your machine if I remember correctly? If so it should be ok.. I'm not sure is running named -u bind and then doing the HUP causing problems? (just thinking out loud here)

[wKkaY]

thanks for the guide!

i'd just like to make a comment regarding SSH: you will still be able to login with passwords. if you're using SecureCRT, just set the authentication mode to 'Keyboard Interactive'. no changes are needed with the openssh client and putty clients.

[Chrysalis]

rhoekman yep it was recompiled with portupgrade and running as bind user.

[rhoekman]

I'm starting to have the same problems here with BIND. Newly created dns entries are not loaded and even a restart via the DA interface won't work. I had that a directadmin process turned into a zombie state but this was when I restarted the httpd via the DA interface.

Restarting the named manually didn't work to get the new records propagated thru the DNS so I rebooted the machine and it worked.

My guess is that something is locking the processes. Might be DA itself because it's still the only one that is not recompiled with the new libraries or it is something else..

It also could be that because we bypass the chrooted in rc.conf. I still get some errors about it in the logs that it cannot create some ndc keys.. maybe it's related to the locking of processes.. don't know yet..

[jlasman]

I haven't had the time to get involved with keeping up-to-date on DirectAdmin with FreeBSD, so I've refrained from responding to FreeBSD-based posts, but I thought I might be able to give you some helpful information, so I'm responding to this post.

DA writes the zone file to /var/named/, using the template system, then adds an entry to /etc/named.conf if required, then restarts named.

I'm not sure how it restarts named, but I'm guessing it does it by calling /etc/rc.d/init.d/named restart because that would work on every OS DA runs on.

named (the Berkley Internet Name Daemon, which is where the name "BIND" comes from) writes its startup and failure messages to the standard logging system; I'm not sure what that is in FreeBSD. (On linux systems the messages end up in /var/log/messages.)

So those are the places to look.

Jeff

[Chrysalis]

On FreeBSD that startup srcipt is /usr/local/etc/rc.d/named

I am able to start/stop named fine with DA but named won't react to HUP in the normal way instead it dies. This I believe is linked to 5.3 directly as I have seen mailings on the FreeBSD mailing list about it.

Now about the startup script, the DA script tries to run the binary /usr/sbin/named so if you are using the ports version of named like me then this happens when you run the DA start script.

Starting Named: daemon: /usr/sbin/named: No such file or directory

So really there is 2 options.

1 - Edit the DA script to be compatible with the new named.

2 - Copy the file /etc/rc.d/named to /usr/local/etc/rc.d

I did number 2, this will solve the issue with Directadmin unable to start/stop named via the web interface.

rhoekman I believe you are encountering the same bug, you added the new domains and then DA probably sent a HUP signal to the named process to make it reread the config file, this would normally work but instead the process probably died.

---edit---

rndc.key used to be ok in just /usr/local/etc/rndc.key but since 5.3 the rc.d script checks if the file exists in /etc/namedb (or the chrooted dir if running chrooted) and if it doesnt exist it creates a new file, this is probably linked to your rndc errors. I just simply copied the rndc.key file from /usr/local/etc to fix this problem.

[rhoekman]

@Chrysalis:

Thanks! Copying that named startup script fixes all those. Wondering though about the SIGHUP bug you've read on the mailing list. Can you tell me if it is an official bug like the FreeBSD team has put it on the fix list or is it a bug that only upgraders (5.2.1 to 5.3) experience?

[Chrysalis]

from what I have read its from user's experience only and it hasnt been put on a bug list.

With any luck this shouldnt affect directadmin that much because it checks every minute for processes that are down so what would probably happen is it sends the HUP named crashes then within a minute the process is restarted and of course it will read the new config file. So all you need to do is fix the rndc stuff and move the startup script and you all set, its messy but should work.

[wdv]

Can anyone tell me whether or not this named HUP bug has been fixed yet?

[map]

Originally posted by Chrysalis
[B]On FreeBSD that startup srcipt is /usr/local/etc/rc.d/named

So really there is 2 options.

1 - Edit the DA script to be compatible with the new named.

2 - Copy the file /etc/rc.d/named to /usr/local/etc/rc.d

I did number 2, this will solve the issue with Directadmin unable to start/stop named via the web interface.



It is asking for trouble having a system with a dying named server every time a user changes a DNS value.

So I did step number 1 (edit /usr/local/etc/rc.d/named). The problem is that the ndc has been replaced with rndc tool (also you must run rndc_confgen -a).
Fast solution and resolves the HUP problem because the script uses now /usr/sbin/rndc reload! I think this solution only applies to FreeBSD 5.3!

[Chrysalis]

I am not sure who to balame for the current named issues in 5.3, be it named devs or FreeBSD devs but I found a sort of workaround for the changes not been applied issue, I added a cron entry that restarts named every 10 mins, assuming the restart is quick this shouldnt cause an issue for people relying on the nameserver.

1 - Create file called cyclenamed.sh

#!/bin/sh
#
# cycle named process automatically script
/etc/rc.d/named restart

2 - make executable by root user only

chown root cyclenamed.sh
chmod 700 cyclenamed.sh

3 - add new line to /etc/crontab

*/10 * * * * root /root/cyclenamed.sh >/dev/null 2>&1

replace /root/cyclenamed.sh with full path to the script you made.

[thoroughfare]

Hi Chrysalis,

I'm probably missing something but I just wondered if there's a reason why:

*/10 * * * * root /etc/rc.d/named restart >/dev/null 2>&1

..wouldn't work just as well, and more efficiently than writing a bash script and crontabbing the bash script?

(I'm not picking fault, I'm just curious if there's a gap in my knowledge).

Matt

[thoroughfare]

Thanks for the how-to rhoekman, it was the first time I'd upgraded a FreeBSD system and it went without a hitch (well I'm running portupgrade now, fingers crossed ;).

Thanks again,
Matt :)

[Chrysalis]

yeah it would, I just got used to doing many things that way, because sometimes I add stuff to scripts so in future that script might do more, but right now its just 1 command.

[jlasman]

Barring any incompatibilities with your version of the cron daemon, I don't see any reason why you shouldn't be able to call the entire process from crond, rather than write a script.

Jeff

[rhoekman]

Originally posted by thoroughfare
Thanks for the how-to rhoekman, it was the first time I'd upgraded a FreeBSD system and it went without a hitch (well I'm running portupgrade now, fingers crossed ;).

Thanks again,
Matt :)

You're welcome.. Hope it will work out for you, probably will ;)

Rick

[thoroughfare]

Hey Rick,

Four boxes now upgrading using your method :-D

One won't come back up but I'm waiting on serial console access to look into that.

Thanks again,
Matt