FreeBSD 5.3 Released

[rhoekman]

Ok, it's not officially released (it will be tommorow :)) but it's already on the mirror's and I couldn't resist. Right at the moment I'm upgrading my secondary DNS without DA to see if there's anything you should look out for. If that is going well I'll take the plunge when they officially anounce it and upgrade the DA machine. I'll keep you posted if there's anything that you should be aware of when upgrading..

---- REPORT BELOW ----

1. Biggest change is from Bind 8 to 9 and it's location and that it runs in a chrooted environment. This could break DA installations.

2. Minor: Firewall (PF) is enabled in the kernel by default. It got me a little alarmed because the upgrades exits with an error when the 'Proxy' user does not exist. It needs this to run the firewall in userland I guess.

The release notes have more information on it:

20040928:
If enabled, the default is now to run named in a chroot
"sandbox." For users with existing configurations in
/etc/namedb the migration should be simple. Upgrade your
world as usual, then after installworld but before
mergemaster do the following:

If named is running: /etc/rc.d/named stop
cd /etc
mv namedb namedb.bak
mkdir -p /var/named/etc/namedb
cp -Rp namedb.bak/* /var/named/etc/namedb/
mergemaster (with your usual options)
If using the generated localhost* files:
cd /var/named/etc/namedb
/bin/sh make-localhost
rm -f localhost-v6.rev localhost.rev
/etc/rc.d/syslogd restart
/etc/rc.d/named start

If you are using a custom configuration, or if you have
customised the named_* variables in /etc/rc.conf[.local]
then you may have to adjust the instructions accordingly.
It is suggested that you carefully examine the new named
variables in /etc/defaults/rc.conf and the options in
/var/named/etc/namedb/named.conf to see if they might
now be more suitable.

20040308:
The packet filter (pf) is now installed with the base system. Make
sure to run mergemaster -p before installworld to create required
user account ("proxy"). If you do not want to build pf with your
system you can use the NO_PF knob in make.conf.
Also note that pf requires "options PFIL_HOOKS" in the kernel. The
pf system consists of the following three devices:
device pf # required
device pflog # optional
device pfsync # optional

--- Damage report ---

I'm glad I did this on a server on location and not remote. There is also a change in OpenSSH and the protocol. I've had only SSH2 access and they changed it to default now (was SSH1 and SSH2) but I encounterd problems with the key and the protocol and had to dive in on the console.

Some of the damage I need to fix:

- OpenSSH is not accepting SSH2 but SSH1 and only if you use TIS authentication and this even if config is set on SSH2. This part is a riddle at moment.
fixed: This was probably due to a certificate error/corruption.

note: Dag-Erling Smorgrav has updated OpenSSH 3.8p1 to change some configuration defaults: the server no longer accepts protocol version 1 nor password authentication by default.

- Nameserver (BIND 9) is not running (and configured)
fixed: I have it up and running after running mergemaster and pointing to the old /etc/named

--- Uneffected ---

Also usefull is to report stuff that still worked after the upgrade. Note that it was not a DA box I upgraded so I post what DA uses:

mysql-server-4.0.18_1
apache+mod_ssl-1.3.31+2.8.17_3
php4-4.3.8_2
vm-pop3d-1.1.6_1

--- Effected Libraries and Programs ---

I ran NMAP and it started to yell at me saying that something was wrong with:

libpcre

Now I believe that Exim uses this lib with perl to use regular expressions (filtering etc)

So my guess is we have a broken Exim after the upgrade. Major issue.

NMAP worked after recompile so I hope we just have to recompile Exim and uses the new libs. This needs proper testing.

[Chrysalis]

I have been following this release with interest I think you right about bind 9 been chrooted might break directadmin.

Here is what I think on certian points.

1 - Bind9 not chrooted works fine with directadmin, chrooted as you say will probably cause problems the workaround would be to use the variable to install the base version not chrooted or to use bind9 from ports.

2 - Freebsd 5.3 version jumps some lib files which will stop a load of pre compiled binaries from working, workaround apart from recompiling is I believe to install the compat4x libs, I hope I am right here.

3 - If you upgrading via source then there is going to be a risk anyway of binaries like sendmail overwriting exim stuff, workaround is probably to either reinstall exim etc. after or to instruct sendmail to not be compiled when upgrading userland.

[rhoekman]

It is finished now and I will have to come back with the damage report ;) (see initial post as I'll update)

To your points.

1. Bind 9 is part of the base install now so we have to either figure out how DA will use Bind 9 in sandbox mode. I do not want to break to much of FreeBSD default installations mainly to avoid maintance time on production servers.

2. compat4.x is needed by default if you want to run DA properly. But indeed the libs are big concern even with the existing symlinks they will probably need some fixing.

3. You can prevent this by skipping the config files when it ask you to merge the files. I think..

[wdv]

I am installing a new server right now, but I think I'll stick with 5.2.1 for now till DA works ok out of the box with 5.3

edit: What are your thoughts about this?

[wdv]

Hmm, I think it will not release tommorow, the RC's haven't even been released yet.

[rhoekman]

Originally posted by wdv
I am installing a new server right now, but I think I'll stick with 5.2.1 for now till DA works ok out of the box with 5.3

edit: What are your thoughts about this?

I think that would be very wise.. I do not feel very confident with this release mainly because of the DNS changes and libraries that probably will cause havoc on a DA machine. On the production machine I'll just keep on patching the system until I know exactly what has changed. For now it's a no go.

[jmstacey]

Correct. Last I checked it was beta7 and its been there since the 3rd of this month or thereabouts. At least on the ftp server I was looking at.

[blueice]

Who of the mirrors have the final release?
I have try some but i have see only the last beta.
Check also this:
http://www.freebsd.org/releases/5.3R/schedule.html
This document is updated yesterday and sayfor today schedule:
5.3-RC1 tier-1 platform images released and uploaded
The final released around 22-23 October.

Christos

[rhoekman]

Sorry for the misleading subject of the post the one that I have installed now is in fact 5.3 BETA7 Release. At the time I thought I had the real thing. According to http://www.freebsd.org/releases/5.3R/schedule.html it was out on the 16th and uploaded to the CVSup trees but they changed it and changed the release date already. My bad.

Anyway the Beta is showing that there are some fundamental changes that will be in the final release. So my report isn't entirely useless :)

[wdv]

Your report isn't useless at all!

I think it would be nice if we think of an upgrade method together, as 5.2.1 will EOL 31-12-2004.

[rhoekman]

Thanks wdv I'll keep reporting stuff that I might think we need to fix after we upgrade.

[Chrysalis]

yes its far from useless and I think it would help DA even more if they could make it compatible before 5.3 is released.

Not only is 5.2.1 EOL soon, also 5.3 is the milestone for the new STABLE branch meaning lots of users will be switching over including myself.

[Geffy]

looks like Oct 25th or later at the moment :P

its nice to see you following it though

[jmstacey]

Originally posted by Chrysalis
yes its far from useless and I think it would help DA even more if they could make it compatible before 5.3 is released.

Not only is 5.2.1 EOL soon, also 5.3 is the milestone for the new STABLE branch meaning lots of users will be switching over including myself.

So will 5.3 also become production ready alongside 4.10?
What are the major benefits of 5.x above 4.x? (Other than the EOL of the 4.x line next year or two?)

[existenz]

5.x is designed from the ground up as a all new multithreadable os. It was also the first version of FBSD to support 64bit processors. They added a new filesystems USF2 (the will require you to reformat the drive to take advantage of) , new scheduler and tons of other new pieces of technology since the branch was broken from 4.x series.

I am sure freebsd.org has a full list of all the new features!

[Chrysalis]

Originally posted by jmstacey
So will 5.3 also become production ready alongside 4.10?
What are the major benefits of 5.x above 4.x? (Other than the EOL of the 4.x line next year or two?)

Yes its the reason I have boxes at 5.2.1 now as I know 5 will soon be STABLE.

For a short period there will be 2 STABLE branches and I think they said there will probably even be a 4.11 release.

As for whats new over 4.x well there are lots of base changes such as new gcc and perl no longer in the base, the kernel should now support -O2 compiling for better speed, it will work a LOT better with new hardware especially hyperthreading hardware. PF will be included in the base as an alternative to IPFW, there are many more changes as well but they the ones I can think off on hand.

The chroot problem with bind can be fixed with an extra line in make.conf telling the buildworld to not chroot bind but by default unless you do this it will be chrooted.

[apryan]

5.3 was out 3 days ago. They are slowly putting it out on FTP as 5.3RC1. 5.3-STABLE is working good on my system so far.

[blueice]

Originally posted by apryan
5.3 was out 3 days ago. They are slowly putting it out on FTP as 5.3RC1. 5.3-STABLE is working good on my system so far.
Please check before you write.
5.3 stable is NOT released.
The last version released is the 5.3 RC1 and have some bugs.
This is NOT the 5.3-stable, it is just a pre-release.
Regards,
Christos

[rhoekman]

Okay I found this in my mailbox today which confirms some of my findings and a more detailed summary of the effected libraries:

--

The FreeBSD Release Engineering Team is proud to announce the
availability of FreeBSD 5.3-RC1. It is intended for early adopters and
those wishing to help find and/or fix bugs. This will likely be the
only Release Candidate before the final release of 5.3. The schedule
can be found at http://www.freebsd.org/releases/5.3R/schedule.html. Be
sure to check the "Known Issues" below as there are known problems that
are still being worked on at this time.

IMPORTANT:
BIND 9.3.0 has replaced BIND 8.x as the default name server.

IMPORTANT:
Several libraries have had their version numbers bumped in order to
maintain FreeBSD 4.x compatibility. Any programs that rely on these
libraries should be rebuilt. The /etc/libmap.conf facility can be
used to help this migration. In particular, libm.so.2 should be
mapped to libm.so.3 while the migration is in progress. The libraries
that changed are:

libm.so.2 -> libm.so.3
libhistory.so.4 -> libhistory.so.5
libopie.so.2 -> libopie.so.3
libpcap.so.2 -> libpcap.so.3
libreadline.so.4 -> libreadline.so.5


Other fixes and enhancements since BETA7:

- Added support for nForce2, nForce3, and ICH3 sound chips
- Fixed LOR in the socket code
- VM_KMEM_SIZE_MAX and VM_KMEM_SIZE_SCALE are now tunables
- Fixed security hole in syscons related to invalid coordinates
- Interface renaming events are now logged
- PFIL_HOOKS are no longer an option and exist by default
- Fixed problem with threads sometimes ignoring signals
- Many fixes to gvinum
- Fix timecounting on sparc64 SMP
- Many fixes to the 4BSD scheduler and infrastructure
- Fix pflogd to handle the pflog module being unloaded
- Fix rare locking bug in sendfile
- Fix locking in the nge driver
- Increase NKPT so that amd64 and i386+PAE can boot with more than
8GB of RAM
- Many fixes to thread support
- Fix breakpoint handing on i386 and amd64 for kernel GDB
- Many fixes for the THR thread library
- Fix IP multicast locking when the stack is running under Giant
- Fix locking in the sis, bfe, and ndis drivers
- Fix possible crash in linux ptrace
- Remove the FreeBSD keyword from all rc.d scripts
- Disable MTU feedback on IPv6 packets to fix NFS over IPv6 problems
- Many ATA driver fixes
- Many VM fixes for i386 and amd64
- Fix 2.88MB floppy support
- Fix locking in bpf, pfil, and IPv6 routing
- Fix the isp driver to work with i386+PAE
- Fix locking that sometimes resulted in deadlock in the TCP code
- Fix jumbo frame handling for the re driver
- Fix the msdos filesystem code to not panic on corrupt filesystems.
- Fix compiling the NDIS module into the kernel
- Fix permission handling on multicast sockets for non-root users
- Fix locking for i4b driver
- Fix byteorder problem in the dc driver on big-endian machines
- Many gstipe/gmirror/graid fixes
- Correctly set BIOS packet mode in the i386 bootblocks
- Update the em driver to support the PRO/1000 GT card, plus many bug
fixes including the common 'wedge on heavy transmit' problem.
- Fix locking in the dc driver, add ALTQ support
- Fix stability problems with UMA
- Fix a potential panic in ethernet entropy harvesting

Known issues in this release:
- Panic in sodealloc() under heavy load. A fix is being tested now.
- Poor performance of the de and re drivers. Fixes are being tested
now.

Availability:

For people wishing to upgrade older systems using cvsup(1) and the
procedure described in src/UPDATING the CVS tag to use is RELENG_5
at this point. Note that like all RELENG_X branches this is an
active development branch. We do not recommend those branches for
normal use (for normal use RELENG_X_Y branches are more appropriate,
e.g. RELENG_4_10 is the current stable branch).

As of this writing the following are available on ftp.freebsd.org
along with some of the mirror sites:

alpha: will be available shortly
amd64: all images available
i386: all images available
ia64: all images available
pc98: miniinst available
sparc64: all images available

MD5s for the builds that are complete at this time are:

MD5 (5.3-RC1-amd64-bootonly.iso) = a3955df1c4b168b30a1a17c04c3fc5d8
MD5 (5.3-RC1-amd64-disc2.iso) = 2777e3a6451d14c29ee936cfd9bd4b43
MD5 (5.3-RC1-amd64-miniinst.iso) = 3fc1dbf1319f94b24aa3aabcd980a097
MD5 (5.3-RC1-amd64-disc1.iso) = 393ee3ef7a3dde13a2dd88ffef792e42

MD5 (5.3-RC1-i386-bootonly.iso) = 3e493c494e14b97816229f595e5c271d
MD5 (5.3-RC1-i386-disc2.iso) = a6cca3b873382739f449b721b1e9506d
MD5 (5.3-RC1-i386-miniinst.iso) = b1035ec5102624f499dc81dc964a73b1
MD5 (5.3-RC1-i386-disc1.iso) = 27746b48459f76414f1730ede5fafa28

MD5 (5.3-RC1-ia64-bootonly.iso) = 8ef8b734d3953b15585002d01e875ff7
MD5 (5.3-RC1-ia64-disc1.iso) = dcaa386cfbe160518fd6ba7a5921303a
MD5 (5.3-RC1-ia64-disc2.iso) = 444103efe5fcc435e734b626d6865219
MD5 (5.3-RC1-ia64-miniinst.iso) = 69e9203505c01d53cf5ecb9ba77c351f

MD5 (5.3-RC1-pc98-disc2.iso) = 3e5deebd2a373c03474f1781e0ab8837
MD5 (5.3-RC1-pc98-miniinst.iso) = 343d1ae111672eecb9545ffdc8a17795

MD5 (5.3-RC1-sparc64-bootonly.iso) = 3c7b5dbfccf95f48e1aa32b07c8c498c
MD5 (5.3-RC1-sparc64-disc1.iso) = a1ec5b9157bd6dba88de3cadd1f93f98
MD5 (5.3-RC1-sparc64-disc2.iso) = e0d51a59d9d1736c8e77ec98795a1009
MD5 (5.3-RC1-sparc64-miniinst.iso) = b797cbd788c4d29723c6f08c608d81e5
_______________________________________________
freebsd-announce@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-announce
To unsubscribe, send any mail to "freebsd-announce-unsubscribe@freebsd.org"

[Chrysalis]

rhoekman where do I sign up to get the list of these changes for each new release?

I downloaded RC1 and spent ages looking for changes since beta 7 and then you posted this :D

[rhoekman]

Look at the bottom of the post I've left it there just for that reason ;) Just in case here it is: http://lists.freebsd.org/mailman/listinfo/freebsd-announce

[Chrysalis]

ahh thanks, need to open my eyes more.

[wdv]

Any updates on this? I think it'll be well worth the trouble to upgrade to FBSD 5.3

[Rick]

How about DA support ETA? :)

:D

[rhoekman]

I have installed it on my secondary nameserver without DA and it is running fine. But I'm going to setup a backup machine with DA running and make the switch probably at the end of this week at night. I'm not expecting any big problems though the DNS changes on 5.3 concerns me a little but shouldn't be hard to fix.

FYI: For those of you that are taking the plunge, bare in mind that you need to rebuild DA after you upgrade to 5.3!

[Geffy]

think you have to rebuild everything after updating to 5.3 as the threading model is all different

[wheat]

How does one go about rebuilding DA? I don't see any sources for it.

[jlasman]

DA is sold as statically compiled software; we do NOT get source code.

And I doubt we ever will.

My guess is that if DA is to be supported on FreeBSD 5.3, then DA staff will have to make it available.

Jeff

[bjseiler]

Originally posted by rhoekman
I have installed it on my secondary nameserver without DA and it is running fine. But I'm going to setup a backup machine with DA running and make the switch probably at the end of this week at night. I'm not expecting any big problems though the DNS changes on 5.3 concerns me a little but shouldn't be hard to fix.

FYI: For those of you that are taking the plunge, bare in mind that you need to rebuild DA after you upgrade to 5.3!

You mean for people going from 4.x to 5.x and not from 5.1 or 5.2.1 to 5.3 correct? Is there enough of a difference between 5.x and 5.3 that the data files will all get busted?

[Chrysalis]

I emailed directadmin and got this back

"I've updated the setup.sh to handle the new version, but havn't had a chance to test it out yet. I'm hoping all the 5.x binaries will continue to work on 5.3.

Thank you,

John"

So I would guess just try it and see if it works. (from older 5.x not from 4.x to 5.x, 5.x needs new binaries and licence type.)

[California]

So let me see if I have this right...

To upgrade my server from 4.9 to 5.3 I need to :
[list=1]
Backup everything
Upgrade from 4.9 to 5.3 via source or binary upgrade
Change my DA license
Reinstall DA
Rebuild customapache
Rebuild all custom ports
Restore all config files from /etc that were customized.
[/list=1]

Is that about it??

[jmstacey]

Thats the general idea. Although I'm not sure if any of the configuration files might need a little tweaking since, for example bind is a newer version.

[California]

Ahh yes...

I had forgotten about the new bind. The FreeBSD migration guide says to, among other things:


On systems running named(8), its configuration files need to be moved into a chroot(8) area in /var/named. If any files exist in /var/named, they should be backed up at this point.


I haven't tried chroot'ing things in DA.

[Chrysalis]

Bind9.3.0 functions fine in DA but if you run it chrooted (freebsd 5.3 default) then you will run into problems without a symlink.

[Chrysalis]

I got 5.3 compiling now on my test DA box and will be going bed as it takes a few hours to finish, when I wake up and finish off I will report how I got on using DA on 5.3.

[Chrysalis]

5.3 is up and running on my test DA server, few points first before I say what happened.

1 - before the last reboot I made the following change in /etc/rc.conf.

named_chrootdir=""

this disables named chrooting.

2 - I have named installed from ports anyway, so disabled named base build in make.conf.

3 - I have openssh installed from ports so disabled it in make.conf for base build, this also meant I didnt need to worry about password login been disabled.

4 - I disabled base build of sendmail in make.conf so exim files wouldnt be overwritten.

Directadmin is functioning normally from what I can tell, apache is up and mails work and I can access the control panel, although I will be recompiling it to be on the safe side this is a good sign.

named is down dispite the above, at the moment I am recompiling all my ports after this I will look into it.

[Chrysalis]

had a look at named, it was just an issue of having to chown all the files back to bind:bind and then it simply started fine. All directadmin daemons are up I didnt even need to recompile them.

[rhoekman]

Is Exim running ok?

[Chrysalis]

yes exim is fine

[rhoekman]

Okay, I have upgraded a production server running FreebSD 5.2.1 with DA to FreeBSD 5.3. It is as far as I know running everything without errors!

edit: Too good to be true indeed. If you use whois.cart and try to to access it you httpd will exit with signal 11. Don't know what is going on.. yet.

[Chrysalis]

well I dont use use whois.cart so wouldnt have known about that sorry, also the server I upgraded isnt heavily used for webhosting its my testing server. But glad your upgrade went mostly well.

[wdv]

Originally posted by rhoekman
Okay, I have upgraded a production server running FreebSD 5.2.1 with DA to FreeBSD 5.3. It is as far as I know running everything without errors!

edit: Too good to be true indeed. If you use whois.cart and try to to access it you httpd will exit with signal 11. Don't know what is going on.. yet.

I'd greatly appreciate it if you could post an how-to :)

[rhoekman]

I'll first have to fix this whois.cart thing. It is very nasty. It kills httpd when using Zend. Don't know if it is a FreeBSD, DA (php+zend) or whois.cart issue. I do not get any lines in my logs to see what's causing it. It just kills the deamon.

[rhoekman]

Ok, it's combination of things. I forgot to rebuild Zend using "./build zend" after I did a "./build all". Now I have a stable httpd. There is still an issue with the new Zend and whois.cart but this is not DA related. Please bare with me as I want whois.cart fixed before I post a howto..

fixed: patching whois.cart and entering a new license key into whois.cart solved it.