{"id":1088,"date":"2025-09-18T10:16:24","date_gmt":"2025-09-18T10:16:24","guid":{"rendered":"https:\/\/cloudspert.com\/?p=1088"},"modified":"2025-09-18T10:16:24","modified_gmt":"2025-09-18T10:16:24","slug":"containers-deep-dive-part-2","status":"publish","type":"post","link":"https:\/\/cloudspert.com\/?p=1088","title":{"rendered":"Containers: Deep dive part 2"},"content":{"rendered":"<figure><img decoding=\"async\" alt=\"\" src=\"https:\/\/cdn-images-1.medium.com\/max\/1024\/0*99bZyF-VzkYW2Rdg.jpg\" \/><\/figure>\n<p>In the last blog post, we explored what containers are and how they are created in the Linux kernel. If you haven\u2019t read it yet, I encourage you to start there\u00a0first.<\/p>\n<p><a href=\"https:\/\/medium.com\/@abdellahtdj\/containers-deep-dive-part-1-dd5a56743a65\">https:\/\/medium.com\/@abdellahtdj\/containers-deep-dive-part-1-dd5a56743a65<\/a><\/p>\n<h3>Requirement<\/h3>\n<p>In this series of articles, you\u2019ll need 2 Linux VMs with a distribution of your choice. I\u2019ll be using a <strong><em>Ubuntu<\/em><\/strong> 22.04 distribution.<\/p>\n<p>Lets goooo\u00a0!<\/p>\n<h3>What is\u00a0chroot?<\/h3>\n<p><strong>Chroot<\/strong> (short for <em>change root<\/em>) is a system call that changes the apparent root directory (\/) for a running process and its children. This means the process will be restricted to a specified directory subtree and will not be able to access files outside of\u00a0it<\/p>\n<p>Think of chroot like putting a process in a playpen and\u00a0saying:<\/p>\n<blockquote><p><em>\u201cHey little process, from now on, this directory is your whole world. You see nothing beyond it. Have\u00a0fun!\u201d<\/em><\/p><\/blockquote>\n<p>Having a separate filesystem is the first step to creating your own environment. When using tools like Docker or Podman, we typically pull images first, which include the application and its libraries installed within their own filesystem. This avoids the need to install or download these components directly on the host\u2019s root filesystem.<\/p>\n<p>The first we need to download a root filesystem; for simplicity, we can go with Alpine minirootfs<\/p>\n<pre>root@testing:~\/blog\/containers# wget http:\/\/dl-cdn.alpinelinux.org\/alpine\/v3.22\/releases\/x86_64\/alpine-minirootfs-3.22.0-x86_64.tar.gz<br \/>root@testing:~\/blog\/containers# mkdir alpine-container<br \/>root@testing:~\/blog\/containers# tar xfz alpine-minirootfs-3.22.0-x86_64.tar.gz -C alpine-container\/<\/pre>\n<p>Once the extraction finishes, we should have a minimal root filesystem that we can use to launch applications within it using chroot. In the example below, we&rsquo;re running the sh command from the Alpine container, not the one on the host system. The -l option is used to instruct sh to behave as a login shell\u2014this means it reads login-related startup files (such as \/etc\/profile), and sets up environment variables like PATH, USER, and\u00a0others.<\/p>\n<pre>root@testing:~\/blog\/containers#chroot alpine-container\/ \/bin\/sh -l <br \/>testing:\/# cat \/etc\/os-release <br \/>NAME=\"Alpine Linux\"<br \/>ID=alpine<br \/>VERSION_ID=3.22.0<br \/>PRETTY_NAME=\"Alpine Linux v3.22\"<br \/>HOME_URL=\"https:\/\/alpinelinux.org\/\"<br \/>BUG_REPORT_URL=\"https:\/\/gitlab.alpinelinux.org\/alpine\/aports\/-\/issues\"<\/pre>\n<p>The host OS we\u2019re running is Ubuntu 22.04. After launching a new Bash process with chroot, the sh process can only see the filesystem of the Alpine image we downloaded. This means we can run commands that exist only on Alpine, such as the apk package\u00a0manager.<\/p>\n<p>But what happens when we try to list processes using ps on the new container<\/p>\n<pre>testing:\/# ps aux <br \/>PID   USER     TIME  COMMAND<br \/>testing:\/# ls \/dev\/<br \/>null<\/pre>\n<p>Yes, it\u2019s expected that there are no processes or devices in the output\u200a\u2014\u200anot even the shell we ran the command from. But\u00a0why?<\/p>\n<p>In Linux, process information is exposed via the \/proc pseudo-filesystem, a virtual interface provided by the kernel. It is not a standard disk-backed filesystem but a dynamic, in-memory representation of process and system data. Since the \/proc filesystem has not been explicitly mounted within the chroot environment, the \/proc directory is empty, resulting in ps returning no output. The same goes for \/dev and \/sys, which are used to access device nodes (e.g., disks, terminals) and kernel parameters and hardware, respectively.<\/p>\n<pre>testing:\/# mount -t proc proc \/proc<br \/>testing:\/# mount -t devtmpfs dev \/dev\/<br \/>testing:\/# mount -t sys sys \/sys<br \/>testing:\/# ps aux <br \/>PID   USER     TIME  COMMAND<br \/>    1 root     25:34 \/lib\/systemd\/systemd --system --deserialize 59<br \/>    2 root      0:24 [kthreadd]<br \/>    3 root      0:00 [rcu_gp]<br \/>    4 root      0:00 [rcu_par_gp]<br \/>    5 root      0:00 [slub_flushwq]<br \/>    6 root      0:00 [netns]<br \/>    8 root      0:00 [kworker\/0:0H-ev]<br \/>   10 root      0:00 [mm_percpu_wq]<br \/>   11 root      0:00 [rcu_tasks_rude_]<br \/>   12 root      0:00 [rcu_tasks_trace]<br \/><\/pre>\n<p>Nope, we\u2019re not there yet. Another important aspect when dealing with containers is user management. It\u2019s generally discouraged to run containers as the root user. Let\u2019s try to find another user (other than root) to switch\u00a0to.<\/p>\n<pre>testing:\/# cat \/etc\/passwd <br \/>root:x:0:0:root:\/root:\/bin\/sh<br \/>bin:x:1:1:bin:\/bin:\/sbin\/nologin<br \/>daemon:x:2:2:daemon:\/sbin:\/sbin\/nologin<br \/>lp:x:4:7:lp:\/var\/spool\/lpd:\/sbin\/nologin<br \/>sync:x:5:0:sync:\/sbin:\/bin\/sync<br \/>shutdown:x:6:0:shutdown:\/sbin:\/sbin\/shutdown<br \/>halt:x:7:0:halt:\/sbin:\/sbin\/halt<br \/>mail:x:8:12:mail:\/var\/mail:\/sbin\/nologin<br \/>news:x:9:13:news:\/usr\/lib\/news:\/sbin\/nologin<br \/>uucp:x:10:14:uucp:\/var\/spool\/uucppublic:\/sbin\/nologin<br \/>cron:x:16:16:cron:\/var\/spool\/cron:\/sbin\/nologin<br \/>ftp:x:21:21::\/var\/lib\/ftp:\/sbin\/nologin<br \/>sshd:x:22:22:sshd:\/dev\/null:\/sbin\/nologin<br \/>games:x:35:35:games:\/usr\/games:\/sbin\/nologin<br \/>ntp:x:123:123:NTP:\/var\/empty:\/sbin\/nologin<br \/>guest:x:405:100:guest:\/dev\/null:\/sbin\/nologin<br \/>nobody:x:65534:65534:nobody:\/:\/sbin\/nologin<\/pre>\n<p>Hmm, there are no other users on this system besides root. The rest are system users, which we can\u2019t login with. To fix this, we need to create our own user inside the container.<\/p>\n<pre>testing:\/# echo \"container::1001:10001:user:\/home\/container:\/bin\/sh\" &gt;&gt; \/etc\/passwd<br \/>testing:\/# mkdir -p \/home\/container<br \/>testing:\/# chown -R 1001:1001 \/home\/container\/<br \/>testing:\/# echo \"container:x:1001:\" &gt;&gt; \/etc\/group<br \/>testing:\/# su - container<br \/>testing:~$ echo \"Hello from inside container\" &gt; hello<br \/>testing:~$ ls -alh hello <br \/>-rw-r--r--    1 container container      28 Jul 25 07:23 hello<\/pre>\n<p>Since we have access to the container\u2019s filesystem from the host, we can see that the user we created exists only inside the container. The group ID, however, corresponds to a different group on the host system (microk8s).<\/p>\n<pre>root@testing:~\/blog\/containers# ls -alh alpine-container\/home\/container\/hello <br \/>-rw-r--r-- 1 1001 microk8s 28 Jul 25 07:23 alpine-container\/home\/container\/hello<br \/>root@testing:~\/blog\/containers# grep 1001 \/etc\/group<br \/>microk8s:x:1001:<\/pre>\n<blockquote><p><strong>Note:<\/strong><\/p><\/blockquote>\n<blockquote><p>You\u2019re probably wondering why the files in my pseudo-container aren\u2019t deleted. Normally, when you stop a Docker container, any changes you\u2019ve made are lost. That\u2019s expected behavior, Docker containers are built on layered images, and any changes happen on a temporary overlay layer. This could be a whole blog post by itself, but let\u2019s not worry about it for\u00a0now.<\/p><\/blockquote>\n<p>Are we finished yet? Not quite. The container we created has an isolated filesystem, but it\u2019s on the same network as the host, shares the host\u2019s hostname configuration, and can see all the processes running on the host. In other words, it still has significant visibility into the host environment. So, our job isn\u2019t done yet; we\u2019ll address these issues in the next blog\u00a0post.<\/p>\n<h3>Bey!!!!!<\/h3>\n<p><img decoding=\"async\" src=\"https:\/\/medium.com\/_\/stat?event=post.clientViewed&amp;referrerSource=full_rss&amp;postId=5e4bf1d813f6\" width=\"1\" height=\"1\" alt=\"\" \/><\/p>","protected":false},"excerpt":{"rendered":"<p>In the last blog post, we explored what containers are and how they are created in the Linux kernel. If you haven\u2019t read it yet, I encourage you to start there\u00a0first. https:\/\/medium.com\/@abdellahtdj\/containers-deep-dive-part-1-dd5a56743a65 Requirement In this series of articles, you\u2019ll need 2 Linux VMs with a distribution of your choice. I\u2019ll be using a Ubuntu 22.04 distribution. Lets goooo\u00a0! What is\u00a0chroot? Chroot (short for change root) is a system call that changes the apparent root directory (\/) for a running process and its children. This means the process will be restricted to a specified directory subtree and will not be able to access files outside of\u00a0it Think of chroot like putting a process in a playpen and\u00a0saying: \u201cHey little process, from now on, this directory is your whole world. You see nothing beyond it. Have\u00a0fun!\u201d Having a separate filesystem is the first step to creating your own environment. When using tools like Docker or Podman, we typically pull images first, which include the application and its libraries installed within their own filesystem. This avoids the need to install or download these components directly on the host\u2019s root filesystem. The first we need to download a root filesystem; for simplicity, we can go with Alpine minirootfs root@testing:~\/blog\/containers# wget http:\/\/dl-cdn.alpinelinux.org\/alpine\/v3.22\/releases\/x86_64\/alpine-minirootfs-3.22.0-x86_64.tar.gzroot@testing:~\/blog\/containers# mkdir alpine-containerroot@testing:~\/blog\/containers# tar xfz alpine-minirootfs-3.22.0-x86_64.tar.gz -C alpine-container\/ Once the extraction finishes, we should have a minimal root filesystem that we can use to launch applications within it using chroot. In the example below, we&rsquo;re running the sh command from the Alpine container, not the one on the host system. The -l option is used to instruct sh to behave as a login shell\u2014this means it reads login-related startup files (such as \/etc\/profile), and sets up environment variables like PATH, USER, and\u00a0others. root@testing:~\/blog\/containers#chroot alpine-container\/ \/bin\/sh -l testing:\/# cat \/etc\/os-release NAME=\u00a0\u00bbAlpine Linux\u00a0\u00bbID=alpineVERSION_ID=3.22.0PRETTY_NAME=\u00a0\u00bbAlpine Linux v3.22&Prime;HOME_URL=\u00a0\u00bbhttps:\/\/alpinelinux.org\/\u00a0\u00bbBUG_REPORT_URL=\u00a0\u00bbhttps:\/\/gitlab.alpinelinux.org\/alpine\/aports\/-\/issues\u00a0\u00bb The host OS we\u2019re running is Ubuntu 22.04. After launching a new Bash process with chroot, the sh process can only see the filesystem of the Alpine image we downloaded. This means we can run commands that exist only on Alpine, such as the apk package\u00a0manager. But what happens when we try to list processes using ps on the new container testing:\/# ps aux PID USER TIME COMMANDtesting:\/# ls \/dev\/null Yes, it\u2019s expected that there are no processes or devices in the output\u200a\u2014\u200anot even the shell we ran the command from. But\u00a0why? In Linux, process information is exposed via the \/proc pseudo-filesystem, a virtual interface provided by the kernel. It is not a standard disk-backed filesystem but a dynamic, in-memory representation of process and system data. Since the \/proc filesystem has not been explicitly mounted within the chroot environment, the \/proc directory is empty, resulting in ps returning no output. The same goes for \/dev and \/sys, which are used to access device nodes (e.g., disks, terminals) and kernel parameters and hardware, respectively. testing:\/# mount -t proc proc \/proctesting:\/# mount -t devtmpfs dev \/dev\/testing:\/# mount -t sys sys \/systesting:\/# ps aux PID USER TIME COMMAND 1 root 25:34 \/lib\/systemd\/systemd &#8211;system &#8211;deserialize 59 2 root 0:24 [kthreadd] 3 root 0:00 [rcu_gp] 4 root 0:00 [rcu_par_gp] 5 root 0:00 [slub_flushwq] 6 root 0:00 [netns] 8 root 0:00 [kworker\/0:0H-ev] 10 root 0:00 [mm_percpu_wq] 11 root 0:00 [rcu_tasks_rude_] 12 root 0:00 [rcu_tasks_trace] Nope, we\u2019re not there yet. Another important aspect when dealing with containers is user management. It\u2019s generally discouraged to run containers as the root user. Let\u2019s try to find another user (other than root) to switch\u00a0to. testing:\/# cat \/etc\/passwd root:x:0:0:root:\/root:\/bin\/shbin:x:1:1:bin:\/bin:\/sbin\/nologindaemon:x:2:2:daemon:\/sbin:\/sbin\/nologinlp:x:4:7:lp:\/var\/spool\/lpd:\/sbin\/nologinsync:x:5:0:sync:\/sbin:\/bin\/syncshutdown:x:6:0:shutdown:\/sbin:\/sbin\/shutdownhalt:x:7:0:halt:\/sbin:\/sbin\/haltmail:x:8:12:mail:\/var\/mail:\/sbin\/nologinnews:x:9:13:news:\/usr\/lib\/news:\/sbin\/nologinuucp:x:10:14:uucp:\/var\/spool\/uucppublic:\/sbin\/nologincron:x:16:16:cron:\/var\/spool\/cron:\/sbin\/nologinftp:x:21:21::\/var\/lib\/ftp:\/sbin\/nologinsshd:x:22:22:sshd:\/dev\/null:\/sbin\/nologingames:x:35:35:games:\/usr\/games:\/sbin\/nologinntp:x:123:123:NTP:\/var\/empty:\/sbin\/nologinguest:x:405:100:guest:\/dev\/null:\/sbin\/nologinnobody:x:65534:65534:nobody:\/:\/sbin\/nologin Hmm, there are no other users on this system besides root. The rest are system users, which we can\u2019t login with. To fix this, we need to create our own user inside the container. testing:\/# echo \u00ab\u00a0container::1001:10001:user:\/home\/container:\/bin\/sh\u00a0\u00bb &gt;&gt; \/etc\/passwdtesting:\/# mkdir -p \/home\/containertesting:\/# chown -R 1001:1001 \/home\/container\/testing:\/# echo \u00ab\u00a0container:x:1001:\u00a0\u00bb &gt;&gt; \/etc\/grouptesting:\/# su &#8211; containertesting:~$ echo \u00ab\u00a0Hello from inside container\u00a0\u00bb &gt; hellotesting:~$ ls -alh hello -rw-r&#8211;r&#8211; 1 container container 28 Jul 25 07:23 hello Since we have access to the container\u2019s filesystem from the host, we can see that the user we created exists only inside the container. The group ID, however, corresponds to a different group on the host system (microk8s). root@testing:~\/blog\/containers# ls -alh alpine-container\/home\/container\/hello -rw-r&#8211;r&#8211; 1 1001 microk8s 28 Jul 25 07:23 alpine-container\/home\/container\/helloroot@testing:~\/blog\/containers# grep 1001 \/etc\/groupmicrok8s:x:1001: Note: You\u2019re probably wondering why the files in my pseudo-container aren\u2019t deleted. Normally, when you stop a Docker container, any changes you\u2019ve made are lost. That\u2019s expected behavior, Docker containers are built on layered images, and any changes happen on a temporary overlay layer. This could be a whole blog post by itself, but let\u2019s not worry about it for\u00a0now. Are we finished yet? Not quite. The container we created has an isolated filesystem, but it\u2019s on the same network as the host, shares the host\u2019s hostname configuration, and can see all the processes running on the host. In other words, it still has significant visibility into the host environment. So, our job isn\u2019t done yet; we\u2019ll address these issues in the next blog\u00a0post. Bey!!!!!<\/p>\n","protected":false},"author":3,"featured_media":531,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[1],"tags":[],"class_list":["post-1088","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-uncategorized","entry","has-media"],"jetpack_featured_media_url":"https:\/\/cloudspert.com\/wp-content\/uploads\/2025\/01\/WhatsApp-Image-2025-01-08-a-14.53.47_f6ab599f-1.jpg","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/cloudspert.com\/index.php?rest_route=\/wp\/v2\/posts\/1088","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/cloudspert.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/cloudspert.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/cloudspert.com\/index.php?rest_route=\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/cloudspert.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=1088"}],"version-history":[{"count":0,"href":"https:\/\/cloudspert.com\/index.php?rest_route=\/wp\/v2\/posts\/1088\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/cloudspert.com\/index.php?rest_route=\/wp\/v2\/media\/531"}],"wp:attachment":[{"href":"https:\/\/cloudspert.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=1088"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/cloudspert.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=1088"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/cloudspert.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=1088"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}