From 82e9ac4163b46b59e121194f84ac370818482923 Mon Sep 17 00:00:00 2001 From: johannst Date: Thu, 15 Jul 2021 21:20:14 +0200 Subject: use proper date fmt in content file names that zola can automatically can derive the date --- content/20191027-kernel-debugging-qemu.md | 226 ------------------------------ 1 file changed, 226 deletions(-) delete mode 100644 content/20191027-kernel-debugging-qemu.md (limited to 'content/20191027-kernel-debugging-qemu.md') diff --git a/content/20191027-kernel-debugging-qemu.md b/content/20191027-kernel-debugging-qemu.md deleted file mode 100644 index 7a97fbc..0000000 --- a/content/20191027-kernel-debugging-qemu.md +++ /dev/null @@ -1,226 +0,0 @@ -+++ -title = "Linux Kernel debugging with QEMU" -date = 2019-10-27 - -[taxonomies] -tags = ["linux", "qemu"] -+++ - -**EDIT**: -- 2021-07-15: Added `Appendix: Dockerfile for Kernel development` and updated - busybox + Kernel versions. - -The other evening while starring at some Linux kernel code I thought, let me -setup a minimal environment so I can easily step through the code and examine -the state. - -I ended up creating: -- a [Linux kernel][linux-kernel] with minimal configuration -- a minimal [ramdisk][initrd] to boot into which is based on [busybox][busybox] - -In the remaing part of this article we will go through each step by first -building the kernel, then building the initrd and then running the kernel using -[QEMU][qemu] and debugging it with [GDB][gdb]. - -## $> make kernel - -Before building the kernel we first need to generate a configuration. As a -starting point we generate a minimal config with the `make tinyconfig` make -target. Running this command will generate a `.config` file. After generating -the initial config file we customize the kernel using the merge fragment flow. -This allows us to merge a fragment file into the current configuration by -running the `scripts/kconfig/merge_config.sh` script. - -Let's quickly go over some customizations we do. -The following two lines enable support for gzipped initramdisks: -```config -CONFIG_BLK_DEV_INITRD=y -CONFIG_RD_GZIP=y -``` -The next two configurations are important as they enable the binary loaders for -[ELF][binfmt-elf] and [script #!][binfmt-script] files. -```config -CONFIG_BINFMT_ELF=y -CONFIG_BINFMT_SCRIPT=y -``` - -> Note: In the cursed based configuration `make menuconfig` we can search for -> configurations using the `/` key and then select a match using the number keys. -> After selecting a match we can check the `Help` to get a description for the -> configuration parameter. - -Building the kernel with the default make target will give us the following two -files: -- `vmlinux` statically linked kernel (ELF file) containing symbol information for debugging -- `arch/x86_64/boot/bzImage` compressed kernel image for booting - -Full configure & build script: -```sh -{{ include(path="content/20191027-kernel-debugging-qemu/build_kernel.sh") }} -``` - -## $> make initrd - -Next step is to build the initrd which we base on [busybox][busybox]. Therefore -we first build the busybox project in its default configuration with one -change, we enable following configuration to build a static binary so it can be -used stand-alone: -```sh -sed -i 's/# CONFIG_STATIC .*/CONFIG_STATIC=y/' .config -``` - -One important step before creating the final initrd is to create an init -process. This will be the first process executed in userspace after the kernel -finished its initialization. We just create a script that drops us into a -shell: -```sh -cat < init -#!/bin/sh - -mount -t proc none /proc -mount -t sysfs none /sys - -exec setsid cttyhack sh -EOF -``` -> By default the kernel looks for `/sbin/init` in the root file system, but the -> location can optionally be specified with the [`init=`][kernel-param] kernel -> parameter. - -Full busybox & initrd build script: -```sh -{{ include(path="content/20191027-kernel-debugging-qemu/build_initrd.sh") }} -``` - -## Running QEMU && GDB - -After finishing the previous steps we have all we need to run and debug the -kernel. We have `arch/x86/boot/bzImage` and `initramfs.cpio.gz` to boot the -kernel into a shell and we have `vmlinux` to feed the debugger with debug -symbols. - -We start QEMU as follows, thanks to the `-S` flag the CPU will freeze until we -connected the debugger: -```sh -# -S freeze CPU until debugger connected -> qemu-system-x86_64 \ - -kernel ./linux-5.3.7/arch/x86/boot/bzImage \ - -nographic \ - -append "earlyprintk=ttyS0 console=ttyS0 nokaslr init=/init debug" \ - -initrd ./initramfs.cpio.gz \ - -gdb tcp::1234 \ - -S -``` - -Then we can start GDB and connect to the GDB server running in QEMU (configured -via `-gdb tcp::1234`). From now on we can start to debug through the -kernel. -```sh -> gdb linux-5.3.7/vmlinux -ex 'target remote :1234' -(gdb) b do_execve -Breakpoint 1 at 0xffffffff810a1a60: file fs/exec.c, line 1885. -(gdb) c -Breakpoint 1, do_execve (filename=0xffff888000060000, __argv=0xffffffff8181e160 , __envp=0xffffffff8181e040 ) at fs/exec.c:1885 -1885 return do_execveat_common(AT_FDCWD, filename, argv, envp, 0); -(gdb) bt -#0 do_execve (filename=0xffff888000060000, __argv=0xffffffff8181e160 , __envp=0xffffffff8181e040 ) at fs/exec.c:1885 -#1 0xffffffff81000498 in run_init_process (init_filename=) at init/main.c:1048 -#2 0xffffffff81116b75 in kernel_init (unused=) at init/main.c:1129 -#3 0xffffffff8120014f in ret_from_fork () at arch/x86/entry/entry_64.S:352 -#4 0x0000000000000000 in ?? () -(gdb) -``` - ---- - -## Appendix: Try to get around `` - -When debugging the kernel we often face following situation in gdb: -```text -(gdb) frame -#0 do_execveat_common (fd=fd@entry=-100, filename=0xffff888000120000, argv=argv@entry=..., envp=envp@entry=..., flags=flags@entry=0) at fs/exec.c - -(gdb) info args -fd = -filename = 0xffff888000060000 -argv = -envp = -flags = -file = 0x0 -``` -The problem is that the Linux kernel requires certain code to be compiled with -optimizations enabled. - -In this situation we can "try" to reduce the optimization for single compilation -units or a subtree (try because, reducing the optimization could break the -build). To do so we adapt the Makefile in the corresponding directory. -```make -# fs/Makefile - -# configure for single compilation unit -CFLAGS_exec.o := -Og - -# configure for the whole subtree of where the Makefile resides -ccflags-y := -Og -``` - -After enabling optimize for debug experience `-Og` we can see the following now -in gdb: -```txt -(gdb) frame -#0 do_execveat_common (fd=fd@entry=-100, filename=0xffff888000120000, argv=argv@entry=..., envp=envp@entry=..., flags=flags@entry=0) at fs/exec.c - -(gdb) info args -fd = -100 -filename = 0xffff888000120000 -argv = {ptr = {native = 0x10c5980}} -envp = {ptr = {native = 0x10c5990}} -flags = 0 - -(gdb) p *filename -$3 = {name = 0xffff888000120020 "/bin/ls", uptr = 0x10c59b8 "/bin/ls", refcnt = 1, aname = 0x0, iname = 0xffff888000120020 "/bin/ls"} - -(gdb) ptype filename -type = struct filename { - const char *name; - const char *uptr; - int refcnt; - struct audit_names *aname; - const char iname[]; -} -``` - -## Appendix: `Dockerfile` for Kernel development - -The following `Dockerfile` provides a development environment with all the -required tools and dependencies, to re-produce all the steps of building and -debugging the Linux kernel. -```dockerfile -{{ include(path="content/20191027-kernel-debugging-qemu/Dockerfile") }} -``` - -Save the listing above in a file called `Dockerfile` and build the docker image -as follows. -```sh -docker build -t kernel-dev -``` -> Optionally set `DOCKER_BUILDKIT=1` to use the newer image builder. - -Once the image has been built, an interactive container can be launched as -follows. -```sh -# Some options for conveniene: -# -v : Mount host path to guest path. -# --rm Remove the container after exiting. - -docker run -it kernel-dev -``` - -[linux-kernel]: https://www.kernel.org -[initrd]: https://www.kernel.org/doc/html/latest/admin-guide/initrd.html -[busybox]: https://busybox.net -[qemu]: https://www.qemu.org -[gdb]: https://www.gnu.org/software/gdb -[binfmt-elf]: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/fs/binfmt_elf.c -[binfmt-script]: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/fs/binfmt_script.c -[kernel-param]: https://www.kernel.org/doc/html/latest/admin-guide/kernel-parameters.html -- cgit v1.2.3