Home
Reading
Searching
Subscribe
Sponsors
Statistics
Posting
Contact
Spam
Lists
Links
About
Hosting
Filtering
Features Download
Marketing
Archives
FAQ
Blog
 
Gmane
From: Witold Baryluk <baryluk <at> smp.if.uj.edu.pl>
Subject: Debian GNU/kFreeBSD on Xen (with Linux Lenny as dom0)
Newsgroups: gmane.linux.debian.ports.bsd
Date: Wednesday 9th December 2009 03:07:54 UTC (over 8 years ago)
Hi,

I am highly interested in kfreebsd port, and even few times
tested it, it is going better and better every day.
Thanks.


I wanted to ask if anybody is going to look at kfreeBSD Xen [1]
support in Squeuzee? (like xen-tools cross-installer,
or easly available kernels for i386 and amd64 for Linux [as .deb]
so they can be easly booted, network setuped, and so on).

Two weeks ago i managed to run such setup:
 dom0:
  * 2x Intel Xeon II 460 MHZ with 2GB
  * Debian GNU/Linux i386 Lenny
  * Linux 2.6.26-2-xen-686
  * Xen version 3.2-1

 domU:
  * FreeBSD-current from May 2009 (prebuild bziped 100MB image [2a])
  * swap, root and usr disk image as files on ext3 partition
  * booting working correclty
  * working xen console
  * bridged networking working
  * SSH server workin
  * portsnap working and compiling few things

========================================================================
========================================================================
========================================================================
[email protected]:~# cat /etc/xen/freebsd-current.cfg
bootloader = "/usr/lib/xen-3.2-1/bin/pygrub"
memory = 128
name = "freebsd_current"
#vif = [ 'mac=00:bd:c4:13:00:01,bridge=xenbr0' ]
#vif = [ 'type=ioemu, mac=00:16:3e:8a:c1:52, bridge=br0, model=ne2k_pci' ]
vif = [ 'mac=00:16:3e:8a:c1:52,bridge=br0' ]

disk = [
		'file:/usr/xen-domains/freebsd_current/root-head-r192577.img,0xCA00,w',
		'file:/usr/xen-domains/freebsd_current/usr2.img,0xCA20,w',
		'file:/usr/xen-domains/freebsd_current/swap.img,0xCA10,w',
#        'phy:/dev/hosting_5_D1/XEN_freebsd_current_root,0xCA00,w'
#        'phy:/dev/hosting_5_D1/XEN_freebsd_current_swap,0xCA10,w'
		]

#on_crash = 'destroy'
on_crash  = 'preserve'
on_shutdown = 'destroy'
on_reboot = 'restart'

extra = ""
extra += ",vfs.root.mountfrom=ufs:xbd0,kern.hz=100,boot_verbose=1"
[email protected]:~#
========================================================================
========================================================================
========================================================================

After few experiments with finding proper vif parameters, and disk ids, and
extra
flags freebsd is very stable. (Othere configurations panics kfreebsd,
on boot or on network setup):

========================================================================
========================================================================
========================================================================
[email protected]:~# xm create freebsd_current.cfg -c

    pyGRUB  version 0.6
 ┌────────────────────────────────────────────────────────────────────────┐
 │ FreeBSD                                                               
│
 │                                                                       
│
 │                                                                       
│
 │                                                                       
│
 │                                                                       
│
 │                                                                       
│
 │                                                                       
│
 │                                                                       
│
 └────────────────────────────────────────────────────────────────────────┘
     Use the ^ and v keys to select which entry is highlighted.
     Press enter to boot the selected OS. 'e' to edit the
     commands before booting, 'a' to modify the kernel arguments
     before booting, or 'c' for a command line.







Started domain freebsd_current
                              WARNING: loader(8) metadata is missing!
GDB: no debug ports present
KDB: debugger backends: ddb
KDB: current backend: ddb
Copyright (c) 1992-2009 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
	The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 8.0-CURRENT #0: Fri May 22 15:45:51 UTC 2009
   
[email protected]:/home/adrian/work/freebsd/xen/obj-head/home/adrian/work/freebsd/xen/svn/head/sys/XEN
WARNING: WITNESS option enabled, expect reduced performance.
Xen reported: 450.122 MHz processor.
Timecounter "ixen" frequency 1000000000 Hz quality 0
CPU: Pentium II/Pentium II Xeon/Celeron (450.12-MHz 686-class CPU)
  Origin = "GenuineIntel"  Id = 0x653  Stepping = 3
 
Features=0x183fbff

Instruction TLB: 4 KB pages, 4-way set associative, 32 entries
Instruction TLB: 4 MB pages, fully associative, 2 entries
Data TLB: 4 KB pages, 4-way set associative, 64 entries
2nd-level cache: 512 KB, 4-way set associative, 32 byte line size
1st-level instruction cache: 16 KB, 4-way set associative, 32 byte line
size
Data TLB: 4 MB Pages, 4-way set associative, 8 entries
1st-level data cache: 16 KB, 4-way set associative, 32 byte line size
real memory  = 134217728 (128 MB)
Physical memory chunk(s):
0x000000000064c000 - 0x0000000007d7dfff, 124985344 bytes (30514 pages)
avail memory = 123322368 (117 MB)
APIC: Using the MPTable enumerator.
SMP: Added CPU 0 (BSP)
ULE: setup cpu 0
cpu=0 irq=0 vector=0
cpu=0 irq=0 vector=1
Event-channel device installed.
random: 
kbd0 at kbdmux0
mem: 
Pentium Pro MTRR support enabled
nfslock: pseudo-device
null: 
io: 
Grant table initialized
xenbus0:  on motherboard
xc0:  on motherboard
npx0: INT 16 interface
Device configuration finished.
procfs registered
Timecounters tick every 10.000 msec
lo0: bpf attached
xbd0: 512MB  at device/vbd/51712 on xenbus0
GEOM: new disk xbd0
GEOM_LABEL: Label for provider xbd0 is ufsid/4a16ce286b8b4567.
xbd2: 2000MB  at device/vbd/51744 on xenbus0
GEOM: new disk xbd2
xbd1: 512MB  at device/vbd/51728 on xenbus0
xn0:  at device/vif/0 on xenbus0
xn0: bpf attached
xn0: Ethernet address: 00:16:3e:8a:c1:52
GEOM_LABEL: Label for provider xbd2 is ufsid/4aafb30fe1ddc483.
GEOM: new disk xbd1
WARNING: WITNESS option enabled, expect reduced performance.
flowtable cleaner started
Trying to mount root from ufs:xbd0
warning: no time-of-day clock registered, system time will not be set
accurately
start_init: trying /sbin/init
No suitable dump device was found.
Entropy harvesting: interrupts ethernet point_to_point kickstart.
GEOM_LABEL: Label ufsid/4a16ce286b8b4567 removed.
/dev/xbd0: FILE SYSTEM CLEAN; SKIPPING CHECKS
/dev/xbd0: clean, 320834 free (258 frags, 40072 blocks, 0.0% fragmentation)
GEOM_LABEL: Label for provider xbd0 is ufsid/4a16ce286b8b4567.
GEOM_LABEL: Label ufsid/4a16ce286b8b4567 removed.
GEOM_LABEL: Label ufsid/4aafb30fe1ddc483 removed.
kldload: can't load zfs: No such file or directory
/etc/rc: WARNING: Unable to load kernel module zfs
Starting Network: lo0 xn0.
add net default: gateway 10.0.1.1
/etc/rc: WARNING: Dump device does not exist.  Savecore not run.

Tue Sep 15 15:37:10 UTC 2009

FreeBSD/i386 (freebsd-current) (xc0)

login: root
Password:
Sep 15 15:37:16 freebsd-current login: ROOT LOGIN (root) ON xc0
Last login: Tue Sep 15 15:36:52 on xc0
Copyright (c) 1992-2009 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
	The Regents of the University of California. All rights reserved.

FreeBSD 8.0-CURRENT (XEN) #0: Fri May 22 15:45:51 UTC 2009

Welcome to FreeBSD!

Before seeking technical support, please use the following resources:

o  Security advisories and updated errata information for all releases are
   at http://www.FreeBSD.org/releases/
- always consult the ERRATA section
   for your release first as it's updated frequently.

o  The Handbook and FAQ documents are at http://www.FreeBSD.org/ and,
   along with the mailing lists, can be searched by going to
   http://www.FreeBSD.org/search/. 
If the doc distribution has
   been installed, they're also available formatted in /usr/share/doc.

If you still have a question or problem, please take the output of
`uname -a', along with any relevant error messages, and email it
as a question to the [email protected] mailing list.  If you are
unfamiliar with FreeBSD's directory layout, please refer to the hier(7)
manual page.  If you are not familiar with manual pages, type `man man'.

You may also use sysinstall(8) to re-enter the installation and
configuration utility.  Edit /etc/motd to change this login announcement.

freebsd-current# cat /etc/rc.conf
hostname="freebsd-current"
ifconfig_xn0="inet 10.0.1.147 netmask 255.255.255.0"
defaultrouter="10.0.1.1"
zfs_enable="YES"
freebsd-current# uptime
 3:32PM  up 8 days,  1:30, 1 user, load averages: 0.05, 0.01, 0.00
freebsd-current#
========================================================================
========================================================================
========================================================================

So i have also working minimal SSH server on FreeBSD on Xen on
Debian/Linux.

Still there is few problems:
  * It is FreeBSD, not Debian kFreeBSD
  * I have no knowledge how to build image manually:
    it will need: building ufs file system on i386 linux, mounting it,
    and then performing (cross)debootstrap on different architecture,
    something which can be very fragile.
    mayby we would need some prebuild images, or delegate image
    creation to speciall domU kfreebsd only for this purpose (building
images),
    so debootstrap inside it can be run without problem (including ufs,
swap or zfs,
    file system creation and data manipulation).
  * It is not very well supported.
  * Some bugs:
     - after "poweroff" or "reboot" inside freebsd guest, virtual
       machine doesn't really reboot or destroy,
       I need to manually perform "xm destroy freebsd_current;
       xm create freebsd_current.cfg"
     - clock is constantly changing back to ~ "Tue Sep 15 15:31:59 UTC
2009",
       even after changing it manually to some sane value, with
       `date 200912090303`, it goes back to ~ "Tue Sep 15 15:31:42 UTC
2009",
       after no more than one minute, in fact it goes back even without
changing,
       anything:
        freebsd-current# date
        Tue Sep 15 15:32:11 UTC 2009
        freebsd-current# date
        Tue Sep 15 15:32:11 UTC 2009
        freebsd-current# date
        Tue Sep 15 15:32:12 UTC 2009
        freebsd-current# date
        Tue Sep 15 15:32:12 UTC 2009
        freebsd-current# date
        Tue Sep 15 15:31:08 UTC 2009

       sometimes clock will increse monotonically for few minutes (no more
than 15:40),
       and then for no reason go back. I haveno ntp client or server
running.
       This makes `make` compilation process very problematic.

       dmesg says
        warning: no time-of-day clock registered, system time will not be
set accurately
       but this happens only once. there is no informations about clock
going backward.
       (there are also broadcast messages from `date` to all consoles,
after manual
       changes).
     - zfs is not working at all.
       # zfs
          internal error: failed to initialize ZFS library
       (but /lib/lib{zfs,zpool}.so.1 exists)
       # truss zfs
          ....
          open("/dev/zfs",O_RDWR,05001302571)		 ERR#2 'No such file or
directory'
          modfind(0x280c8089,0x2,0x28058579,0x28092000,0x258,0x280c8089)
ERR#2 'No such file or directory'
          kldload("zfs")					 ERR#2 'No such file or directory'
          open("/dev/zfs",O_RDWR,05001302571)		 ERR#2 'No such file or
directory'
          ....
       # find / -iname '*zfs*'
          /boot/zfs
          /boot/zfsboot
          /boot/gptzfsboot
          /etc/periodic/daily/404.status-zfs
          /etc/rc.d/zfs
          /etc/zfs
          /lib/libzfs.so.1
          /sbin/zfs
          /usr/lib/libzfs.a
          /usr/lib/libzfs_p.a
          /usr/lib/libzfs.so
          /usr/share/man/man8/zfs.8.gz
       Where is kernel module?
     - dtrace is also not working.
       # dtrace -l
          dtrace: failed to initialize dtrace: DTrace device not available
on system
       (again /lib/libdtrace.so.1 exists)
       # truss dtrace -l
          ....
          open("/dev/dtrace/dtrace",O_RDWR,00)		 ERR#2 'No such file or
directory'
          open("/dev/dtrace/fasttrap",O_RDWR,00)		 ERR#2 'No such file or
directory'
          ....


Next week I would like to try running the same (or newer) image,
inside 2.6.33-git+pv_ops (it is new dom0 support from scratch for Linux),
(Xen for 2.6.26 in Lenny is really backported from 2.6.18 and pretty old).
Hopefully xen_dom0_pv_ops will be included in 2.6.34 :) [4]

Currently it isn't easy to run dom0 in testing or unstable, in
Lenny it is just working.

I will be also testing ZFS inside guest, and trying export it via NFS4 back
to the
host dom0 :) [3] Then I would like to test performance, and Jails.

I'm also curious if live migration is working.

I would also appreciate any clues how to run kfreebsd directly
on Xen (based on mini.iso ?), not freebsd-current (additionally outdate).

BTW. I'm only interested in Xen paravirtualized setup,
i don't have hardware virtualization support, so KVM / HVM
is no go for me. I also like Xen approach, and for it have
very good performance and companion tools which makes
maintaining VM very easy.


PS. Apparently few mentioned problems, are already being solved:
o kern/140313  xen        [xen] [panic] FreeBSD8 RC2 as PV domU crashes
during c
o kern/135421  xen        [xen] FreeBSD Xen PVM DomU network failure -
netfronc.
o kern/135179  xen        [xen] Xen domU does not properly reboot
o kern/135178  xen        [xen] Xen domU outgoing data transfer stall when
TSO i
o kern/135069  xen        [xen] FreeBSD-current/Xen SMP doesn't function at
all 
o kern/135008  xen        [xen] FreeBSD-current/Xen timecounter jumps
o kern/134926  xen        [xen] [panic] FreeBSD-current Xen DomU networking
pani

Usefull links:
  [1] http://wiki.freebsd.org/FreeBSD/Xen
  [2] http://wiki.freebsd.org/AdrianChadd/XenImages
  [2a] http://people.freebsd.org/~adrian/xen/root-head-r192577.img.gz
  [3] http://wiki.freebsd.org/AdrianChadd/XenHackery
  [4] http://wiki.xensource.com/xenwiki/XenParavirtOps
  [5] http://lists.freebsd.org/mailman/listinfo/freebsd-xen

-- 
Witold Baryluk
JID: witold.baryluk // jabster.pl
 
CD: 2ms