From 1c123a75e9b17858b783fbc6533417bdfa9794eb Mon Sep 17 00:00:00 2001 From: Johannes Stoelp Date: Mon, 19 Dec 2022 18:54:41 +0100 Subject: Migrate pages with assets according to zola best practice https://www.getzola.org/documentation/content/overview/#asset-colocation --- content/2021-12-02-toying-with-virtio.md | 373 ------------------------------- 1 file changed, 373 deletions(-) delete mode 100644 content/2021-12-02-toying-with-virtio.md (limited to 'content/2021-12-02-toying-with-virtio.md') diff --git a/content/2021-12-02-toying-with-virtio.md b/content/2021-12-02-toying-with-virtio.md deleted file mode 100644 index c2ff031..0000000 --- a/content/2021-12-02-toying-with-virtio.md +++ /dev/null @@ -1,373 +0,0 @@ -+++ -title = "QEMU virtio configurations" - -[taxonomies] -tags = ["linux", "qemu", "virtio"] -+++ - -For my own reference I wanted to document some minimal [`virtio`][virtio] -device configurations with qemu and the required Linux kernel configuration to -enable those devices. - -The devices we will use are `virtio console`, `virtio blk` and `virtio net`. - -To make use of the virtio devices in qemu we are going to build and boot into -busybox based [`initramfs`][initramfs]. - -## Build initramfs - -For the initramfs there is not much magic, we will grab a copy of busybox, -configure it with the default config (`defconfig`) and enable static linking as -we will use it as rootfs. - -For the `init` process we will use the one provided by busybox but we have to -symlink it to `/init` as during boot, the kernel will extract the cpio -compressed initramfs into `rootfs` and look for the `/init` file. If that's not -found the kernel will fallback to an older mechanism an try to mount a root -partition (which we don't have). -> Optionally the init binary could be specified with the `rdinit=` kernel boot -> parameter. - -We populate the `/etc/inittab` and `/etc/init.d/rcS` with a minimal -configuration to mount the `proc`, `sys` and `dev` filesystems and drop into a -shell after the boot is completed. \ -Additionally we setup `/etc/passwd` and `/etc/shadow` with an entry for the -`root` user with the password `1234`, so we can login via the virtio console -later. - -```sh -{{ include_range(path="content/2021-12-02-toying-with-virtio/build_initramfs.sh", start=31, end=67) }} -``` - -The full build script is available under [build_initramfs.sh][build-initramfs]. - -## Virtio console - -To enable support for the virtio console we enable the kernel configs shown -below. -The pci configurations are enabled because in qemu the virtio console front-end -device (the one presented to the guest) is attached to the pci bus. - -```sh -{{ include_range(path="content/2021-12-02-toying-with-virtio/build_kernel.sh", start=32, end=38) }} -``` - -The full build script is available under [build_kernel.sh][build-kernel]. - -To boot-up the guest we use the following qemu configuration. - -```sh -qemu-system-x86_64 \ - -nographic \ - -cpu host \ - -enable-kvm \ - -kernel ./linux-$(VER)/arch/x86/boot/bzImage \ - -append "earlyprintk=ttyS0 console=ttyS0 root=/dev/ram0 ro" \ - -initrd ./initramfs.cpio.gz \ - -device virtio-serial-pci \ - -device virtconsole,chardev=vcon,name=console.0 \ - -chardev socket,id=vcon,ipv4=on,host=localhost,port=2222,server,telnet=on,wait=off -``` - -The important parts in this configuration are the last three lines. - -The `virtio-serial-pci` device creates the serial bus where the virtio console -is attached to. - -The `virtconsole` creates the virtio console device exposed to the guest -(front-end). The `chardev=vcon` option specifies that the chardev with -`id=vcon` is attached as back-end to the virtio console. -The back-end device is the one we will have access to from the host running the -emulation. - -The chardev back-end we configure to be a `socket`, running a telnet server -listening on port 2222. The `wait=off` tells qemu that it can directly boot -without waiting for a client connection. - -After booting the guest we are dropped into a shell and can verify that our -device is being detected properly. -```sh -root@virtio-box ~ # ls /sys/bus/virtio/devices/ -virtio0 -root@virtio-box ~ # cat /sys/bus/virtio/devices/virtio0/virtio-ports/vport0p0/name -console.0 -``` - -In `/etc/inittab`, we already configured to spawn `getty` on the first -hypervisor console `/dev/hvc0`. This will effectively run `login(1)` over the -serial console. - -From the host we can run `telnet localhost 2222` and are presented with a login shell to the guest. - -As we already included to launch `getty` on the first hypervisor console -`/dev/hvc0` in `/etc/inittab`, we can directly connect to the back-end chardev -and login to the guest with `root:1234`. - -```sh -> telnet -4 localhost 2222 -Trying 127.0.0.1... -Connected to localhost. -Escape character is '^]'. - -virtio-box login: root -Password: -root@virtio-box ~ # -``` - -## Virtio blk - -To enable support for the virtio block device we enable the kernel configs -shown below. -First we enable general support for block devices and then for virtio block -devices. Additionally we enable support for the `ext2` filesystem because we -are creating an ext2 filesystem to back the virtio block device. - -```sh -{{ include_range(path="content/2021-12-02-toying-with-virtio/build_kernel.sh", start=40, end=47) }} -``` - -The full build script is available under [build_kernel.sh][build-kernel]. - -Next we are creating the ext2 filesystem image. This we'll do by creating an -`128M` blob and format it with ext2 afterwards. Then we can mount the image -via a `loop` device and populate the filesystem. -```sh -{{ include_range(path="content/2021-12-02-toying-with-virtio/build_ext2.sh", start=3, end=7) }} -``` - -Before booting the guest we will attach the virtio block device to the VM. -Therefore we add the `-drive` configuration to our previous qemu invocation. - -```sh -qemu-system-x86_64 \ - ... - -drive if=virtio,file=fs.ext2,format=raw -``` - -The `-drive` option is a shortcut for a `-device (front-end) / -blockdev -(back-end)` pair. - -The `if=virtio` flag specifies the interface of the front-end device to be -`virtio`. - -The `file` and `format` flags configure the back-end to be a disk image. - -After booting the guest we are dropped into a shell and can verify a few -things. First we check if the virtio block device is detected, then we check if -we have support for the ext2 filesystem and finally we mount the disk. - -```sh -root@virtio-box ~ # ls -l /sys/block/ -lrwxrwxrwx 1 root 0 0 Dec 3 22:46 vda -> ../devices/pci0000:00/0000:00:05.0/virtio1/block/vda - -root@virtio-box ~ # cat /proc/filesystems -... - ext2 - -root@virtio-box ~ # mount -t ext2 /dev/vda /mnt -EXT2-fs (vda): warning: mounting unchecked fs, running e2fsck is recommended -ext2 filesystem being mounted at /mnt supports timestamps until 2038 (0x7fffffff) - -root@virtio-box ~ # cat /mnt/hello -world -``` - -## Virtio net - -To enable support for the virtio network device we enable the kernel configs -shown below. -First we enable general support for networking and TCP/IP and then enable the -core networking driver and the virtio net driver. - -```sh -{{ include_range(path="content/2021-12-02-toying-with-virtio/build_kernel.sh", start=49, end=62) }} -``` - -The full build script is available under [build_kernel.sh][build-kernel]. - -For the qemu device emulation we already decided on the front-end device, which -will be our virtio net device. \ -On the back-end we will choose the [`user`][qemu-user-net] option. This enables -a network stack implemented in userspace based on [libslirp][libslirp], which -has the benefit that we do not need to setup additional network interfaces and -therefore require any privileges. Fundamentally, [libslirp][libslirp] works by -replaying [Layer 2][osi-2] packets received from the guest NIC via the socket -API on the host ([Layer 4][osi-4]) and vice versa. User networking comes with a -set of limitations, for example -- Can not use `ping` inside the guest as `ICMP` is not supported. -- The guest is not accessible from the host. - -With the guest, qemu and the host in the picture this looks something like the -following. -``` -+--------------------------------------------+ -| host | -| +-------------------------+ | -| | guest | | -| | | | -| | user | | -| +------+------+-----------+ | -| | | eth0 | kernel | | -| | +--+---+ | | -| | | | | -| | +-----v--------+ | | -| | | nic (virtio) | | | -| +--+---+-----+--------+------+--+ | -| | | Layer 2 qemu | | -| | | (eth frames) | | -| | +----v-----+ | | -| | | libslirp | | | -| | +----+-----+ | | -| | | Layer 4 | | -| | | (socket API) | user | -+--+---------+--v---+--------------+---------+ -| | eth0 | kernel | -| +------+ | -+--------------------------------------------+ -``` - -The user networking implements a virtually NAT'ed sub-network with the address -range `10.0.2.0/24` running an internal dhcp server. By default, the dhcp -server assigns the following IP addresses which are interesting to us: -- `10.0.2.2` host running the qemu emulation -- `10.0.2.3` virtual DNS server -> The netdev options `net=addr/mask`, `host=addr`, `dns=addr` can be used to -> re-configure the sub-network (see [network options][qemu-nic-opts]). - -With the details of the sub-network in mind we can add some additional setup to -the initramfs which performs the basic network setup. - -We add the virtual DNS server to `/etc/resolv.conf` which will be used by the -libc resolver functions. - -Additionally we assign a static ip to the `eth0` network interface, bring the -interface up and define the default route via the host `10.0.2.2`. - -```sh -{{ include_range(path="content/2021-12-02-toying-with-virtio/build_initramfs.sh", start=69, end=85) }} -``` - -The full build script is available under [build_initramfs.sh][build-initramfs]. - -Before booting the guest we will attach the virtio net device and configure to -use the user network stack. -Therefore we add the `-nic` configuration to our previous qemu invocation. - -```sh -qemu-system-x86_64 \ - ... - -nic user,model=virtio-net-pci -``` - -The `-nic` option is a shortcut for a `-device (front-end) / -netdev -(back-end)` pair. - -After booting the guest we are dropped into a shell and can verify a few -things. First we check if the virtio net device is detected. Then we check if -the interface got configured and brought up correctly. - -```sh -root@virtio-box ~ # ls -l /sys/class/net/ -lrwxrwxrwx 1 root 0 0 Dec 4 16:56 eth0 -> ../../devices/pci0000:00/0000:00:03.0/virtio0/net/eth0 -lrwxrwxrwx 1 root 0 0 Dec 4 16:56 lo -> ../../devices/virtual/net/lo - - -root@virtio-box ~ # ip -o a -2: eth0 inet 10.0.2.15/24 scope global eth0 ... - -root@virtio-box ~ # ip route -default via 10.0.2.2 dev eth0 -10.0.2.0/24 dev eth0 scope link src 10.0.2.15 -``` - -We can resolve out domain and see that the virtual DNS gets contacted. - -```sh -root@virtio-box ~ # nslookup memzero.de -Server: 10.0.2.3 -Address: 10.0.2.3:53 - -Non-authoritative answer: -Name: memzero.de -Address: 46.101.148.203 -``` - -Additionally we can try to access a service running on the host. Therefore we -run a simple http server on the host (where we launched qemu) with the -following command `python3 -m http.server --bind 0.0.0.0 1234`. This will -launch the server to listen for any incoming address at port `1234`. - -From within the guest we can manually craft a simple http `GET` request and -send it to the http server running on the host. For that we use the IP address -`10.0.2.2` which the dhcp assigned to our host. - -```sh -root@virtio-box ~ # echo "GET / HTTP/1.0" | nc 10.0.2.2 1234 -HTTP/1.0 200 OK -Server: SimpleHTTP/0.6 Python/3.9.7 -Date: Sat, 04 Dec 2021 16:58:56 GMT -Content-type: text/html; charset=utf-8 -Content-Length: 917 - - - - - -Directory listing for / - - -

Directory listing for /

-
- -
- - -``` - -## Appendix: Workspace - -To re-produce the setup and play around with it just grab a copy of the -following files: -- [Dockerfile][dockerfile] -- [Makefile][makefile] -- [build_initramfs.sh][build-initramfs] -- [build_kernel.sh][build-kernel] -- [build_ext2.sh][build-ext2] - -Then run the following steps to build everything. The prefix `[H]` and `[C]` -indicate whether this command is run on the host or inside the container -respectively. -```sh -# To see all the make targets. -[H] make help - -# Build docker image, start a container with the current working dir -# mounted. On the first invocation this takes some minutes to build -# the image. -[H]: make docker - -# Build kernel and initramfs. -[C]: make - -# Build ext2 fs as virtio blkdev backend. -[H]: make ext2 - -# Start qemu guest. -[H]: make run -``` - -[build-initramfs]: https://git.memzero.de/blog/tree/content/2021-12-02-toying-with-virtio/build_initramfs.sh?h=main -[build-kernel]: https://git.memzero.de/blog/tree/content/2021-12-02-toying-with-virtio/build_kernel.sh?h=main -[build-ext2]: https://git.memzero.de/blog/tree/content/2021-12-02-toying-with-virtio/build_ext2.sh?h=main -[makefile]: https://git.memzero.de/blog/tree/content/2021-12-02-toying-with-virtio/Makefile?h=main -[dockerfile]: https://git.memzero.de/blog/tree/content/2021-12-02-toying-with-virtio/Dockerfile?h=main -[initramfs]: https://www.kernel.org/doc/Documentation/filesystems/ramfs-rootfs-initramfs.txt -[virtio]: http://docs.oasis-open.org/virtio/virtio/v1.1/virtio-v1.1.pdf -[qemu-nic-opts]: https://www.qemu.org/docs/master/system/invocation.html#hxtool-5 -[qemu-user-net]: https://www.qemu.org/docs/master/system/devices/net.html#using-the-user-mode-network-stack -[libslirp]: https://gitlab.com/qemu-project/libslirp -[osi-2]: https://osi-model.com/data-link-layer -[osi-4]: https://osi-model.com/transport-layer -- cgit v1.2.3