From: Tulio A M Mendes Date: Tue, 10 Feb 2026 12:11:52 +0000 (-0300) Subject: docs: comprehensive documentation update reflecting current AdrOS state X-Git-Url: https://projects.tadryanom.me/docs/static/gitweb.js?a=commitdiff_plain;h=f671903e8be6497a50cc4fca5fbe9c7db57b3979;p=AdrOS.git docs: comprehensive documentation update reflecting current AdrOS state Updated all 4 documentation files to accurately reflect the current codebase: POSIX_ROADMAP.md: - fork now marked as CoW (was 'full copy') - execve marked complete (was partial) - brk/sbrk, nanosleep, clock_gettime marked implemented - O_CLOEXEC, FD_CLOEXEC marked implemented - Signal defaults marked complete (Ctrl+C/Z/D/\) - Raw TTY mode, TIOCGWINSZ marked implemented - PMM now shows spinlock + refcount - Slab allocator, CoW, shared memory, SMEP marked implemented - Heap shows dynamic growth to 64MB - PCI, ATA DMA, LAPIC/IOAPIC, SMP, ACPI, VBE, SYSENTER, CPUID added - ulibc marked implemented - procfs partial noted - Priority roadmap updated: 9 items marked done, remaining reordered SUPPLEMENTARY_ANALYSIS.md: - POSIX score updated from ~45% to ~70% - PMM: spinlock + refcount now implemented - VMM: CoW + SMEP now implemented - Heap: slab allocator + dynamic growth now implemented - Scheduler: O(1) with bitmap now implemented - TTY: raw mode, signal chars, TIOCGWINSZ now implemented - Userland: ulibc + SYSENTER now implemented - Drivers: PCI, ATA DMA, LAPIC/IOAPIC, SMP, ACPI, VBE now implemented - Advanced: CoW, slab, shmem now implemented - Part 3 comparison table updated (AdrOS ahead in 5/10 dimensions) - Part 4 recommendations: 9 items marked completed, remaining reordered README.md: - Complete rewrite of Features section with organized subsections - Added: Memory Management, Process & Scheduling, Syscalls, TTY/PTY, Filesystems (6 types), Drivers & Hardware, Userland, Security, Testing - Updated TODO with near-term/medium-term/long-term priorities - Updated Directory Structure with new paths BUILD_GUIDE.md: - Added expect and sparse to dependencies - Added Testing section with all make targets - Increased smoke test timeout from 40s to 60s --- diff --git a/BUILD_GUIDE.md b/BUILD_GUIDE.md index 4fa0820..9e83775 100644 --- a/BUILD_GUIDE.md +++ b/BUILD_GUIDE.md @@ -13,13 +13,15 @@ You will need: - `grub-pc-bin` and `grub-common` (x86 bootloader) - Cross-compilers for ARM/RISC-V (optional, for non-x86 targets) - `cppcheck` (optional, for static analysis) +- `expect` (optional, for automated smoke tests) +- `sparse` (optional, for kernel-style static analysis) ### On Ubuntu/Debian: ```bash sudo apt update sudo apt install build-essential bison flex libgmp3-dev libmpc-dev libmpfr-dev texinfo \ qemu-system-x86 qemu-system-arm qemu-system-misc \ - grub-common grub-pc-bin xorriso mtools cppcheck \ + grub-common grub-pc-bin xorriso mtools cppcheck expect sparse \ gcc-aarch64-linux-gnu gcc-riscv64-linux-gnu ``` @@ -92,13 +94,25 @@ The init program (`/bin/init.elf`) runs a comprehensive suite of smoke tests on All tests print `[init] ... OK` on success. Any failure calls `sys_exit(1)`. -Static analysis helper: +### Testing + +Run all tests (static analysis + host unit tests + QEMU smoke tests): ```bash -make ARCH=x86 cppcheck +make test-all +``` + +Individual test targets: +```bash +make check # cppcheck + sparse + gcc -fanalyzer +make test-host # 47 host-side unit tests (test_utils + test_security) +make test # QEMU smoke test (4 CPUs, 40s timeout, 19 checks) +make test-1cpu # Single-CPU smoke test (50s timeout) +make test-gdb # GDB scripted integrity checks (heap, PMM, VGA) ``` -Additional analysis helpers: +Static analysis helpers: ```bash +make ARCH=x86 cppcheck make ARCH=x86 scan-build make ARCH=x86 mkinitrd-asan ``` diff --git a/Makefile b/Makefile index f675777..f52092a 100644 --- a/Makefile +++ b/Makefile @@ -183,7 +183,7 @@ check: cppcheck sparse # ---- Automated Smoke Test (QEMU + expect) ---- SMOKE_SMP ?= 4 -SMOKE_TIMEOUT ?= 40 +SMOKE_TIMEOUT ?= 60 test: iso @echo "[TEST] Running smoke test (SMP=$(SMOKE_SMP), timeout=$(SMOKE_TIMEOUT)s)..." diff --git a/README.md b/README.md index 695743b..9371bf4 100644 --- a/README.md +++ b/README.md @@ -15,72 +15,85 @@ AdrOS is a Unix-like, POSIX-compatible, multi-architecture operating system deve - **Build System:** Make + Cross-Compilers ## Features -- **Multi-arch build system** - - `make ARCH=x86|arm|riscv|mips` - - x86 is the primary, working target -- **x86 (i386) boot & memory layout** - - Multiboot2 (via GRUB) - - Higher-half kernel mapping (3GB+) - - Early paging + VMM initialization - - W^X-oriented userspace layout (separate RX/R and RW segments) - - Non-executable stack markers in assembly (`.note.GNU-stack`) -- **Memory management** - - Physical Memory Manager (PMM) - - Virtual Memory Manager (x86) - - Kernel heap allocator (`kmalloc`/`kfree`) -- **Basic drivers & console** - - UART serial console logging - - VGA text console (x86) - - Keyboard driver + input callback - - PIT timer + periodic tick -- **Kernel services** - - Simple scheduler / multitasking (kernel threads) - - Minimal process lifecycle: parent/child tracking, zombies, `waitpid` - - Basic shell with built-in commands (fallback when userspace fails) -- **InitRD + VFS + mounts** - - InitRD image in TAR/USTAR format (with directory support) - - InitRD-backed filesystem node tree (`fs_node_t` + `finddir`) - - Absolute path lookup (`vfs_lookup("/bin/init.elf")`) - - Mount table support (`vfs_mount`) + `tmpfs` and `overlayfs` -- **Syscalls (x86, `int 0x80`)** - - **File I/O:** `open`, `openat`, `read`, `write`, `close`, `lseek`, `stat`, `fstat`, `fstatat`, `dup`, `dup2`, `dup3`, `pipe`, `pipe2`, `select`, `poll`, `ioctl`, `fcntl` - - **Directory ops:** `mkdir`, `rmdir`, `unlink`, `unlinkat`, `rename`, `getdents`, `chdir`, `getcwd` - - **Process:** `fork`, `execve`, `exit`, `waitpid` (incl. `WNOHANG`), `getpid`, `getppid`, `setsid`, `setpgid`, `getpgrp` - - **Signals:** `sigaction`, `sigprocmask`, `kill`, `sigreturn` (trampoline-based return path) - - **Modern POSIX:** `openat`/`fstatat`/`unlinkat` (`AT_FDCWD`), `dup3`, `pipe2` (with `O_NONBLOCK` flags) - - Per-process fd table with refcounted file objects - - Per-process current working directory (`cwd`) with relative path resolution - - Non-blocking I/O (`O_NONBLOCK`) on pipes, TTY, and PTY via `fcntl` - - Centralized user-pointer access API (`user_range_ok`, `copy_from_user`, `copy_to_user`) - - Ring3 init program (`/bin/init.elf`) with comprehensive smoke tests - - Error returns use negative errno codes (Linux-style) -- **TTY (canonical line discipline)** - - Keyboard -> TTY input path - - Canonical mode input (line-buffered until `\n`) - - Echo + backspace handling - - Blocking reads with a simple wait queue (multiple waiters) - - `fd=0` wired to `tty_read`, `fd=1/2` wired to `tty_write` - - Minimal termios/ioctl support (`TCGETS`, `TCSETS`, `TIOCGPGRP`, `TIOCSPGRP`) - - Basic job control enforcement (`SIGTTIN`/`SIGTTOU` when background pgrp touches the controlling TTY) -- **Devices (devfs)** - - `/dev` mount with `readdir` support - - `/dev/null`, `/dev/tty` - - `/dev/ptmx` + `/dev/pts/0` (pseudo-terminal) -- **PTY subsystem** - - PTY master/slave pair (`/dev/ptmx` + `/dev/pts/0`) - - Non-blocking I/O support - - Used by userland smoke tests -- **On-disk filesystem (diskfs)** - - ATA PIO driver (primary master IDE) - - Hierarchical inode-based filesystem mounted at `/disk` - - Supports: `open` (create/truncate), `read`, `write`, `stat`, `mkdir`, `unlink`, `rmdir`, `rename`, `getdents` - - Persistence filesystem mounted at `/persist` (smoke tests) -- **Generic `readdir`/`getdents` across all VFS** - - Works on diskfs, tmpfs, devfs, and overlayfs - - Unified `struct vfs_dirent` format -- **W^X (Option 1) for user ELFs (x86)** - - User segments are mapped RW during load, then write permissions are dropped for non-writable segments - - This provides "text is read-only" hardening without requiring NX/PAE + +### Boot & Architecture +- **Multi-arch build system** — `make ARCH=x86|arm|riscv|mips` (x86 is the primary, working target) +- **Multiboot2** (via GRUB), higher-half kernel mapping (3GB+) +- **CPUID feature detection** — leaf 0/1/7/extended; SMEP/SMAP detection +- **SYSENTER fast syscall path** — MSR setup + handler + +### Memory Management +- **PMM** — bitmap allocator with spinlock protection and frame reference counting +- **VMM** — recursive page directory, per-process address spaces, TLB flush +- **Copy-on-Write (CoW) fork** — PTE bit 9 as CoW marker + page fault handler +- **Kernel heap** — doubly-linked free list with coalescing, dynamic growth up to 64MB +- **Slab allocator** — `slab_cache_t` with free-list-in-place and spinlock +- **Shared memory** — System V IPC style (`shmget`/`shmat`/`shmdt`/`shmctl`) +- **SMEP** — Supervisor Mode Execution Prevention enabled in CR4 +- **W^X** — user `.text` segments marked read-only after ELF load + +### Process & Scheduling +- **O(1) scheduler** — bitmap + active/expired arrays, 32 priority levels +- **Process model** — `fork` (CoW), `execve`, `exit`, `waitpid` (`WNOHANG`), `getpid`, `getppid` +- **Sessions & groups** — `setsid`, `setpgid`, `getpgrp` +- **User heap** — `brk`/`sbrk` syscall +- **Time** — `nanosleep`, `clock_gettime` (`CLOCK_REALTIME`, `CLOCK_MONOTONIC`) + +### Syscalls (x86, `int 0x80` + SYSENTER) +- **File I/O:** `open`, `openat`, `read`, `write`, `close`, `lseek`, `stat`, `fstat`, `fstatat`, `dup`, `dup2`, `dup3`, `pipe`, `pipe2`, `select`, `poll`, `ioctl`, `fcntl`, `getdents` +- **Directory ops:** `mkdir`, `rmdir`, `unlink`, `unlinkat`, `rename`, `chdir`, `getcwd` +- **Signals:** `sigaction`, `sigprocmask`, `kill`, `sigreturn` (full trampoline with `SA_SIGINFO`) +- **FD flags:** `O_NONBLOCK`, `O_CLOEXEC`, `FD_CLOEXEC` via `fcntl` (`F_GETFD`/`F_SETFD`/`F_GETFL`/`F_SETFL`) +- **Shared memory:** `shmget`, `shmat`, `shmdt`, `shmctl` +- Per-process fd table with atomic refcounted file objects +- Centralized user-pointer access API (`user_range_ok`, `copy_from_user`, `copy_to_user`) +- Error returns use negative errno codes (Linux-style) + +### TTY / PTY +- **Canonical + raw mode** — `ICANON` clearable via `TCSETS` +- **Signal characters** — Ctrl+C→SIGINT, Ctrl+Z→SIGTSTP, Ctrl+D→EOF, Ctrl+\\→SIGQUIT +- **Job control** — `SIGTTIN`/`SIGTTOU` enforcement for background process groups +- **PTY** — `/dev/ptmx` + `/dev/pts/0` with non-blocking I/O +- **Window size** — `TIOCGWINSZ`/`TIOCSWINSZ` +- **termios** — `TCGETS`, `TCSETS`, `TIOCGPGRP`, `TIOCSPGRP` + +### Filesystems (6 types) +- **tmpfs** — in-memory filesystem +- **devfs** — `/dev/null`, `/dev/tty`, `/dev/ptmx`, `/dev/pts/0` +- **overlayfs** — copy-up semantics +- **diskfs** — hierarchical inode-based on-disk filesystem at `/disk` +- **persistfs** — minimal persistence at `/persist` +- **procfs** — `/proc/meminfo` +- Generic `readdir`/`getdents` across all VFS types + +### Drivers & Hardware +- **PCI** — full bus/slot/func enumeration with BAR + IRQ +- **ATA PIO + DMA** — Bus Master IDE with bounce buffer, PRDT, IRQ coordination +- **LAPIC + IOAPIC** — replaces legacy PIC; ISA IRQ routing +- **SMP** — 4 CPUs via INIT-SIPI-SIPI, per-CPU data via GS segment +- **ACPI** — MADT parsing for CPU topology and IOAPIC discovery +- **VBE framebuffer** — maps LFB, pixel drawing, font rendering +- **UART**, **VGA text**, **PS/2 keyboard**, **PIT timer**, **LAPIC timer** + +### Userland +- **ulibc** — `printf`, `malloc`/`free`/`calloc`/`realloc`, `string.h`, `unistd.h`, `errno.h` +- **ELF32 loader** — secure with W^X enforcement, rejects kernel-range vaddrs +- `/bin/init.elf` — comprehensive smoke test suite +- `/bin/echo.elf` — argv/envp test + +### Security +- **SMEP** enabled (prevents kernel executing user-mapped pages) +- **user_range_ok** hardened (rejects kernel addresses) +- **sigreturn eflags** sanitized (clears IOPL, ensures IF) +- **Atomic file refcounts** (`__sync_*` builtins) +- **PMM spinlock** for SMP safety + +### Testing +- **47 host-side unit tests** — `test_utils.c` (28) + `test_security.c` (19) +- **19 QEMU smoke tests** — 4-CPU, expect-based +- **Static analysis** — cppcheck, sparse, gcc -fanalyzer +- **GDB scripted checks** — heap/PMM/VGA integrity +- `make test-all` runs everything ## Running (x86) - `make ARCH=x86 iso` @@ -97,40 +110,34 @@ QEMU debug helpers: See [POSIX_ROADMAP.md](docs/POSIX_ROADMAP.md) for a detailed checklist. -- **Syscalls / POSIX gaps** - - `brk`/`sbrk` (heap management) - - `mmap`/`munmap` (memory-mapped I/O) - - `access`, `chmod`, `chown`, `umask` (permissions) - - `link`, `symlink`, `readlink` (hard/symbolic links) - - `truncate`/`ftruncate` - - `socket`/`bind`/`listen`/`accept`/`connect`/`send`/`recv` (networking) - - Userspace `errno` variable + libc-style wrappers -- **Filesystem** - - Permissions/ownership (`uid/gid`, mode bits) - - `/proc` filesystem - - Real on-disk FS (ext2/FAT) as alternative to diskfs -- **TTY / terminal** - - Full termios flags (raw mode, VMIN/VTIME, signal chars) - - Multiple PTY pairs -- **Virtual memory hardening** - - PAE + NX enforcement (execute disable for data/stack) - - Guard pages, ASLR -- **Multi-architecture kernel bring-up** - - Implement VMM/interrupts/scheduler for ARM/RISC-V/MIPS -- **Userland** - - Minimal libc (`printf`, `malloc`, `string.h`, etc.) - - Shell (sh-compatible) - - Core utilities (`ls`, `cat`, `cp`, `mv`, `rm`, `echo`, `mkdir`) -- **Observability & tooling** - - Panic backtraces, symbolization - - CI pipeline with `cppcheck`, `scan-build` +### Near-term (unlock interactive use) +- **Shell** (`sh`-compatible) — all required syscalls are implemented +- **Core utilities** — `ls`, `cat`, `echo`, `mkdir`, `rm` +- **`/dev/zero`**, **`/dev/random`** — simple device nodes +- **Multiple PTY pairs** — currently only 1 + +### Medium-term (POSIX compliance) +- **`mmap`/`munmap`** — memory-mapped files +- **Permissions** — `uid`/`gid`/mode, `chmod`, `chown`, `access`, `umask` +- **`/proc` per-process** — `/proc/[pid]/status`, `/proc/[pid]/maps` +- **Hard/symbolic links** — `link`, `symlink`, `readlink` +- **VMIN/VTIME** — termios non-canonical timing + +### Long-term (full Unix experience) +- **Networking** — socket API, TCP/IP stack +- **Threads** — `clone`/`pthread` +- **PAE + NX bit** — hardware W^X +- **Dynamic linking** — `ld.so` +- **ext2/FAT** filesystem support ## Directory Structure -- `src/kernel/` - Architecture-independent kernel code (VFS, syscalls, scheduler, tmpfs, diskfs, devfs, overlayfs, PTY, TTY) -- `src/arch/` - Architecture-specific code (boot, context switch, interrupts, VMM) -- `src/hal/` - Hardware abstraction layer (CPU, keyboard, timer, UART, video) -- `src/drivers/` - Device drivers (ATA, initrd, keyboard, timer, UART, VGA) -- `src/mm/` - Memory management (PMM, heap) -- `include/` - Header files -- `user/` - Userland programs (`init.c`, `echo.c`) -- `docs/` - Documentation (POSIX roadmap) +- `src/kernel/` — Architecture-independent kernel (VFS, syscalls, scheduler, tmpfs, diskfs, devfs, overlayfs, PTY, TTY, shm, signals) +- `src/arch/x86/` — x86-specific (boot, VMM, IDT, LAPIC, IOAPIC, SMP, ACPI, CPUID, SYSENTER) +- `src/hal/x86/` — HAL x86 (CPU, keyboard, timer, UART, PCI, ATA PIO/DMA) +- `src/drivers/` — Device drivers (VBE, initrd, VGA) +- `src/mm/` — Memory management (PMM, heap, slab) +- `include/` — Header files +- `user/` — Userland programs (`init.c`, `echo.c`) +- `user/ulibc/` — Minimal C library (`printf`, `malloc`, `string.h`, `errno.h`) +- `tests/` — Host unit tests, smoke tests, GDB scripted checks +- `docs/` — Documentation (POSIX roadmap, audit report, supplementary analysis, testing plan) diff --git a/docs/POSIX_ROADMAP.md b/docs/POSIX_ROADMAP.md index 35aa312..1a2a616 100644 --- a/docs/POSIX_ROADMAP.md +++ b/docs/POSIX_ROADMAP.md @@ -39,7 +39,7 @@ Notes: | `pread`/`pwrite` | [ ] | | | `readv`/`writev` | [ ] | | | `truncate`/`ftruncate` | [ ] | | -| `fsync`/`fdatasync` | [ ] | | +| `fsync`/`fdatasync` | [ ] | No-op acceptable for now | ## 2. Syscalls — Directory & Path Operations @@ -63,8 +63,8 @@ Notes: | Syscall | Status | Notes | |---------|--------|-------| -| `fork` | [x] | Full COW not implemented; copies address space | -| `execve` | [~] | Loads ELF from VFS; minimal argv/envp; no `$PATH` search | +| `fork` | [x] | Full COW implemented (`vmm_as_clone_user_cow` + `vmm_handle_cow_fault`) | +| `execve` | [x] | Loads ELF from VFS; argv/envp; `O_CLOEXEC` FDs closed | | `exit` / `_exit` | [x] | Closes FDs, marks zombie, notifies parent | | `waitpid` | [x] | `-1` (any child), specific pid, `WNOHANG` | | `getpid` | [x] | | @@ -74,10 +74,11 @@ Notes: | `getpgrp` | [x] | | | `getuid`/`getgid`/`geteuid`/`getegid` | [ ] | No user/group model yet | | `setuid`/`setgid` | [ ] | | -| `brk`/`sbrk` | [ ] | Heap management | +| `brk`/`sbrk` | [x] | `syscall_brk_impl()` — per-process heap break | | `mmap`/`munmap` | [ ] | Memory-mapped I/O | | `clone` | [ ] | Thread creation | -| `nanosleep`/`sleep` | [ ] | | +| `nanosleep`/`sleep` | [x] | `syscall_nanosleep_impl()` with tick-based sleep | +| `clock_gettime` | [x] | `CLOCK_REALTIME` and `CLOCK_MONOTONIC` | | `alarm` | [ ] | | | `times`/`getrusage` | [ ] | | @@ -94,7 +95,7 @@ Notes: | `sigsuspend` | [ ] | | | `sigqueue` | [ ] | | | `sigaltstack` | [ ] | Alternate signal stack | -| Signal defaults | [~] | `SIGKILL`/`SIGSEGV`/`SIGUSR1` handled; many signals missing default actions | +| Signal defaults | [x] | `SIGKILL`/`SIGSEGV`/`SIGUSR1`/`SIGINT`/`SIGTSTP`/`SIGTTOU`/`SIGTTIN` handled | ## 5. File Descriptor Layer @@ -104,9 +105,9 @@ Notes: | Refcounted file objects | [x] | Shared across `dup`/`fork` | | File offset tracking | [x] | | | `O_NONBLOCK` | [x] | Pipes, TTY, PTY via `fcntl` or `pipe2` | -| `O_CLOEXEC` | [ ] | Close-on-exec flag | +| `O_CLOEXEC` | [x] | Close-on-exec via `pipe2`, `open` flags | | `O_APPEND` | [ ] | | -| `FD_CLOEXEC` via `fcntl` | [ ] | | +| `FD_CLOEXEC` via `fcntl` | [x] | `F_GETFD`/`F_SETFD` implemented; `execve` closes marked FDs | | File locking (`flock`/`fcntl`) | [ ] | | ## 6. Filesystem / VFS @@ -122,10 +123,10 @@ Notes: | **devfs** | [x] | `/dev/null`, `/dev/tty`, `/dev/ptmx`, `/dev/pts/0`; `readdir` | | **diskfs** (on-disk) | [x] | Hierarchical inodes; `open`/`read`/`write`/`stat`/`mkdir`/`unlink`/`rmdir`/`rename`/`getdents` | | **persistfs** | [x] | Minimal persistence at `/persist` | +| **procfs** | [~] | `/proc/meminfo` exists; no per-process `/proc/[pid]` | | Permissions (`uid`/`gid`/mode) | [ ] | No permission model | | Hard links | [ ] | | | Symbolic links | [ ] | | -| `/proc` filesystem | [ ] | | | ext2 / FAT support | [ ] | | ## 7. TTY / PTY @@ -141,24 +142,27 @@ Notes: | `isatty` (via `ioctl TCGETS`) | [x] | | | PTY master/slave | [x] | `/dev/ptmx` + `/dev/pts/0` | | Non-blocking PTY I/O | [x] | | -| Raw mode (non-canonical) | [ ] | | +| Raw mode (non-canonical) | [x] | Clear `ICANON` via `TCSETS` | | VMIN/VTIME | [ ] | | -| Signal characters (Ctrl+C → `SIGINT`, etc.) | [ ] | | +| Signal characters (Ctrl+C → `SIGINT`, etc.) | [x] | Ctrl+C→SIGINT, Ctrl+Z→SIGTSTP, Ctrl+D→EOF, Ctrl+\\→SIGQUIT | | Multiple PTY pairs | [ ] | Only 1 pair currently | -| Window size (`TIOCGWINSZ`/`TIOCSWINSZ`) | [ ] | | +| Window size (`TIOCGWINSZ`/`TIOCSWINSZ`) | [x] | Get/set `struct winsize` | ## 8. Memory Management | Feature | Status | Notes | |---------|--------|-------| -| PMM (bitmap allocator) | [x] | | -| VMM (x86 paging) | [x] | Higher-half kernel | +| PMM (bitmap allocator) | [x] | Spinlock-protected, frame refcounting | +| VMM (x86 paging) | [x] | Higher-half kernel, recursive page directory | | Per-process address spaces | [x] | Page directory per process | -| Kernel heap (`kmalloc`/`kfree`) | [x] | 10MB heap | +| Kernel heap (`kmalloc`/`kfree`) | [x] | Dynamic growth up to 64MB | +| Slab allocator | [x] | `slab_cache_t` with free-list-in-place | | W^X for user ELFs | [x] | Text segments read-only after load | -| `brk`/`sbrk` | [ ] | | +| SMEP | [x] | Enabled in CR4 if CPU supports | +| `brk`/`sbrk` | [x] | Per-process heap break | | `mmap`/`munmap` | [ ] | | -| Copy-on-write (COW) fork | [ ] | Currently full-copy | +| Shared memory (`shmget`/`shmat`/`shmdt`) | [x] | System V IPC style | +| Copy-on-write (COW) fork | [x] | PTE bit 9 as CoW marker + page fault handler | | PAE + NX bit | [ ] | | | Guard pages | [ ] | | | ASLR | [ ] | | @@ -171,10 +175,17 @@ Notes: | VGA text console (x86) | [x] | | | PS/2 keyboard | [x] | | | PIT timer | [x] | | +| LAPIC timer | [x] | Calibrated, used when APIC available | | ATA PIO (IDE) | [x] | Primary master | +| ATA DMA (Bus Master IDE) | [x] | Bounce buffer, PRDT, IRQ-coordinated | +| PCI enumeration | [x] | Full bus/slot/func scan with BAR + IRQ | +| ACPI (MADT parsing) | [x] | CPU topology + IOAPIC discovery | +| LAPIC + IOAPIC | [x] | Replaces legacy PIC | +| SMP (multi-CPU boot) | [x] | 4 CPUs via INIT-SIPI-SIPI, per-CPU data via GS | +| CPUID feature detection | [x] | Leaf 0/1/7/extended; SMEP/SMAP detection | +| VBE framebuffer | [x] | Maps LFB, pixel drawing, font rendering | +| SYSENTER fast syscall | [x] | MSR setup + handler | | RTC (real-time clock) | [ ] | | -| PCI enumeration | [ ] | | -| Framebuffer / VESA | [ ] | | | Network (e1000/virtio-net) | [ ] | | | Virtio-blk | [ ] | | @@ -182,10 +193,10 @@ Notes: | Feature | Status | Notes | |---------|--------|-------| -| ELF32 loader | [x] | | +| ELF32 loader | [x] | Secure with W^X enforcement | | `/bin/init.elf` (smoke tests) | [x] | Comprehensive test suite | | `/bin/echo.elf` | [x] | Minimal argv/envp test | -| Minimal libc | [ ] | No libc; userland uses raw syscall wrappers | +| Minimal libc (ulibc) | [x] | `printf`, `malloc`/`free`/`calloc`/`realloc`, `string.h`, `unistd.h`, `errno.h` | | Shell (`sh`) | [ ] | | | Core utilities (`ls`, `cat`, `cp`, `mv`, `rm`, `mkdir`) | [ ] | | | Dynamic linking | [ ] | | @@ -209,24 +220,31 @@ Notes: ## Priority Roadmap (next steps) ### Near-term (unlock a usable shell) -1. **Minimal libc** — `printf`, `malloc`/`free`, `string.h`, `stdio.h` wrappers -2. **Shell** — `sh`-compatible; needs `fork`+`execve`+`waitpid`+`pipe`+`dup2`+`chdir` (all implemented) -3. **Core utilities** — `ls` (uses `getdents`), `cat`, `echo`, `mkdir`, `rm`, `mv`, `cp` -4. **Signal characters** — Ctrl+C → `SIGINT`, Ctrl+Z → `SIGTSTP`, Ctrl+D → EOF -5. **Raw TTY mode** — needed for interactive editors and proper shell line editing +1. ~~Minimal libc~~ ✅ ulibc implemented (`printf`, `malloc`, `string.h`, `unistd.h`, `errno.h`) +2. **Shell** — `sh`-compatible; all required syscalls are implemented +3. **Core utilities** — `ls` (uses `getdents`), `cat`, `echo`, `mkdir`, `rm` +4. ~~Signal characters~~ ✅ Ctrl+C→SIGINT, Ctrl+Z→SIGTSTP, Ctrl+D→EOF +5. ~~Raw TTY mode~~ ✅ ICANON clearable via TCSETS +6. **`/dev/zero`** + **`/dev/random`** — simple device nodes +7. **Multiple PTY pairs** — currently only 1 ### Medium-term (real POSIX compliance) -6. **`brk`/`sbrk`** — userland heap -7. **`mmap`/`munmap`** — memory-mapped files, shared memory -8. **Permissions** — `uid`/`gid`, mode bits, `chmod`, `chown`, `access`, `umask` -9. **`O_CLOEXEC`** — close-on-exec for fd hygiene -10. **`/proc`** — process information filesystem -11. **Hard/symbolic links** — `link`, `symlink`, `readlink` +8. ~~`brk`/`sbrk`~~ ✅ Implemented +9. **`mmap`/`munmap`** — memory-mapped files, shared memory +10. **Permissions** — `uid`/`gid`, mode bits, `chmod`, `chown`, `access`, `umask` +11. ~~`O_CLOEXEC`~~ ✅ Implemented +12. **`/proc` per-process** — `/proc/[pid]/status`, `/proc/[pid]/maps` +13. **Hard/symbolic links** — `link`, `symlink`, `readlink` +14. **VMIN/VTIME** — termios non-canonical timing +15. **Generic wait queue abstraction** — replace ad-hoc blocking ### Long-term (full Unix experience) -12. **Networking** — socket API, TCP/IP stack -13. **Multi-arch bring-up** — ARM/RISC-V functional kernels -14. **COW fork + demand paging** -15. **Threads** (`clone`/`pthread`) -16. **Dynamic linking** (`ld.so`) -17. **ext2/FAT** filesystem support +16. **Networking** — socket API, TCP/IP stack +17. **Multi-arch bring-up** — ARM/RISC-V functional kernels +18. ~~COW fork~~ ✅ Implemented +19. **Threads** (`clone`/`pthread`) +20. **Dynamic linking** (`ld.so`) +21. **ext2/FAT** filesystem support +22. **PAE + NX bit** — hardware W^X +23. **Per-thread errno** (needs TLS) +24. **vDSO** — fast `clock_gettime` without syscall diff --git a/docs/SUPPLEMENTARY_ANALYSIS.md b/docs/SUPPLEMENTARY_ANALYSIS.md index 02d7ab0..5e14fac 100644 --- a/docs/SUPPLEMENTARY_ANALYSIS.md +++ b/docs/SUPPLEMENTARY_ANALYSIS.md @@ -16,12 +16,12 @@ Unix-like, POSIX-compatible operating system. | Bitmap allocator | ✅ Bitmap-based | ✅ Bitmap-based (`src/mm/pmm.c`) | None | | Multiboot memory map parsing | ✅ Parse MMAP entries | ✅ Full Multiboot2 MMAP parsing, clamping, fallback | None | | Kernel/module protection | ✅ Reserve kernel + initrd | ✅ Protects kernel (`_start`–`_end`), modules, low 1MB | None | -| Frame reference counting | ✅ `uint16_t frame_ref_count[]` for CoW | ❌ Not implemented | **Critical for CoW fork** | -| Contiguous block allocation | ✅ `pmm_alloc_blocks(count)` for DMA | ❌ Only single-frame `pmm_alloc_page()` | Needed for DMA drivers | -| Atomic ref operations | ✅ `__sync_fetch_and_add` | ❌ N/A (no refcount) | Future | -| Spinlock protection | ✅ `spinlock_acquire(&pmm_lock)` | ❌ PMM has no lock (single-core safe only) | Needed for SMP | +| Frame reference counting | ✅ `uint16_t frame_ref_count[]` for CoW | ✅ `pmm_incref`/`pmm_decref`/`pmm_get_refcount` | None | +| Contiguous block allocation | ✅ `pmm_alloc_blocks(count)` for DMA | ❌ Only single-frame `pmm_alloc_page()` | Needed for multi-page DMA | +| Atomic ref operations | ✅ `__sync_fetch_and_add` | ✅ File refcounts use `__sync_*` builtins | None | +| Spinlock protection | ✅ `spinlock_acquire(&pmm_lock)` | ✅ `pmm_lock` with `spin_lock_irqsave` | None | -**Summary:** AdrOS PMM is solid for single-core use. Missing ref-counting (blocks CoW) and contiguous allocation (blocks DMA). +**Summary:** AdrOS PMM is SMP-safe with spinlock protection and frame refcounting for CoW. Only missing contiguous block allocation. --- @@ -34,13 +34,15 @@ Unix-like, POSIX-compatible operating system. | Per-process address spaces | ✅ Clone kernel PD | ✅ `vmm_as_create_kernel_clone()`, `vmm_as_clone_user()` | None | | W^X logical policy | ✅ `vmm_apply_wx_policy()` rejects RWX | ✅ ELF loader maps `.text` as RO after load via `vmm_protect_range()` | Partial — no policy function, but effect achieved | | W^X hardware (NX bit) | ✅ PAE + NX via EFER MSR | ❌ 32-bit paging, no PAE, no NX | Long-term | -| CPUID feature detection | ✅ `cpu_get_features()` for PAE/NX | ❌ Not implemented | Long-term | +| CPUID feature detection | ✅ `cpu_get_features()` for PAE/NX | ✅ Full CPUID leaf 0/1/7/extended; SMEP/SMAP detection | None | +| SMEP | Not discussed | ✅ Enabled in CR4 if CPU supports | **AdrOS is ahead** | +| Copy-on-Write (CoW) | ✅ Full implementation | ✅ `vmm_as_clone_user_cow()` + `vmm_handle_cow_fault()` | None | | `vmm_find_free_area()` | ✅ Scan user VA space for holes | ❌ Not implemented | Needed for `mmap` | | `vmm_map_dma_buffer()` | ✅ Map phys into user VA | ❌ Not implemented | Needed for zero-copy I/O | | TLB flush | ✅ `invlpg` + full flush | ✅ `invlpg()` per page | None | | Spinlock on VMM ops | ✅ `vmm_kernel_lock` | ❌ No lock | Needed for SMP | -**Summary:** AdrOS VMM is functional and well-designed (recursive mapping is elegant). Missing hardware NX (requires PAE migration) and free-area search for `mmap`. +**Summary:** AdrOS VMM is functional and well-designed with CoW fork, recursive mapping, and SMEP. Missing hardware NX (requires PAE migration) and free-area search for `mmap`. --- @@ -51,9 +53,10 @@ Unix-like, POSIX-compatible operating system. | Doubly-linked free list | Mentioned | ✅ `heap.c` with `HEAP_MAGIC` validation | None | | Coalescing | Mentioned | ✅ Forward + backward coalesce (fixed in previous session) | None | | Spinlock | ✅ Required | ✅ `heap_lock` spinlock present | None | -| Slab allocator | ✅ `slab_cache_t` for fixed-size objects | ❌ Not implemented | Medium priority | +| Slab allocator | ✅ `slab_cache_t` for fixed-size objects | ✅ `slab_cache_init`/`slab_alloc`/`slab_free` with spinlock | None | +| Dynamic growth | Not discussed | ✅ Heap grows from 10MB up to 64MB on demand | **AdrOS is ahead** | -**Summary:** Heap works correctly. Slab allocator would improve performance for frequent small allocations (process structs, file descriptors). +**Summary:** Heap works correctly with dynamic growth and slab allocator for fixed-size objects. --- @@ -62,18 +65,18 @@ Unix-like, POSIX-compatible operating system. | Aspect | Supplementary Suggestion | AdrOS Current State | Gap | |--------|--------------------------|---------------------|-----| | Process states | ✅ READY/RUNNING/SLEEPING/ZOMBIE | ✅ READY/RUNNING/ZOMBIE/BLOCKED/SLEEPING | AdrOS has more states | -| Round-robin scheduling | Baseline | ✅ Implemented in `scheduler.c` | None | -| O(1) scheduler (bitmap + active/expired) | ✅ Full implementation | ❌ Simple linked-list traversal | Enhancement | -| Priority queues (MLFQ) | ✅ 32 priority levels | ❌ No priority levels | Enhancement | +| Round-robin scheduling | Baseline | ✅ Implemented as fallback | None | +| O(1) scheduler (bitmap + active/expired) | ✅ Full implementation | ✅ Bitmap + active/expired swap, 32 priority levels | None | +| Priority queues (MLFQ) | ✅ 32 priority levels | ✅ 32 priority levels via `SCHED_NUM_PRIOS` | None | | Unix decay-based priority | ✅ `p_cpu` decay + `nice` | ❌ Not implemented | Enhancement | | Per-CPU runqueues | ✅ `cpu_runqueue_t` per CPU | ❌ Single global queue | Needed for SMP | -| Sleep/wakeup (wait queues) | ✅ `sleep(chan, lock)` / `wakeup(chan)` | ✅ Process blocking via `PROCESS_BLOCKED` state + manual wake | Partial — no generic wait queue abstraction | +| Sleep/wakeup (wait queues) | ✅ `sleep(chan, lock)` / `wakeup(chan)` | ✅ Process blocking + `nanosleep` syscall | Partial — no generic wait queue abstraction | | Context switch (assembly) | ✅ Save/restore callee-saved + CR3 | ✅ `context_switch.S` saves/restores regs + CR3 | None | -| `fork()` | ✅ Slab + CoW + enqueue | ✅ `process_fork_create()` — full copy (no CoW) | CoW missing | -| `execve()` | ✅ Load ELF, reset stack | ✅ `syscall_execve_impl()` — loads ELF, handles argv/envp | None | +| `fork()` | ✅ Slab + CoW + enqueue | ✅ `vmm_as_clone_user_cow()` + page fault handler | None | +| `execve()` | ✅ Load ELF, reset stack | ✅ `syscall_execve_impl()` — loads ELF, handles argv/envp, `O_CLOEXEC` | None | | Spinlock protection | ✅ `sched_lock` | ✅ `sched_lock` present | None | -**Summary:** AdrOS scheduler is functional with all essential operations. The supplementary material suggests O(1)/MLFQ upgrades which are performance enhancements, not correctness issues. +**Summary:** AdrOS scheduler is now O(1) with bitmap + active/expired arrays and 32 priority levels. Missing decay-based priority and per-CPU runqueues. --- @@ -121,9 +124,11 @@ Unix-like, POSIX-compatible operating system. | PTY master/slave | Not discussed | ✅ Full PTY implementation with `/dev/ptmx` + `/dev/pts/0` | **AdrOS is ahead** | | Job control (SIGTTIN/SIGTTOU) | Not discussed | ✅ `pty_jobctl_read_check()` / `pty_jobctl_write_check()` | **AdrOS is ahead** | | `poll()` support | ✅ `tty_poll()` | ✅ `pty_master_can_read()` etc. integrated with `poll` | None | -| Raw mode | Not discussed | ❌ Not implemented | Needed for editors/games | +| Raw mode | Not discussed | ✅ `ICANON` clearable via `TCSETS` | None | +| Signal characters | Not discussed | ✅ Ctrl+C→SIGINT, Ctrl+Z→SIGTSTP, Ctrl+D→EOF, Ctrl+\\→SIGQUIT | **AdrOS is ahead** | +| Window size | Not discussed | ✅ `TIOCGWINSZ`/`TIOCSWINSZ` | **AdrOS is ahead** | -**Summary:** AdrOS TTY/PTY is **significantly ahead** of the supplementary material. Full PTY with job control is a major achievement. +**Summary:** AdrOS TTY/PTY is **significantly ahead** of the supplementary material. Full PTY with job control, raw mode, signal characters, and window size. --- @@ -136,10 +141,10 @@ Unix-like, POSIX-compatible operating system. | IRQ save/restore | ✅ `pushcli`/`popcli` with nesting | ✅ `irq_save()`/`irq_restore()` via `pushf`/`popf` | None | | `spin_lock_irqsave` | ✅ Combined lock + IRQ disable | ✅ `spin_lock_irqsave()` / `spin_unlock_irqrestore()` | None | | Debug name field | ✅ `char *name` for panic messages | ❌ No name field | Minor | -| CPU ID tracking | ✅ `lock->cpu_id` for deadlock detection | ❌ Not tracked | Needed for SMP | -| Nesting counter (`ncli`) | ✅ Per-CPU nesting | ❌ Not implemented (flat save/restore) | Needed for SMP | +| CPU ID tracking | ✅ `lock->cpu_id` for deadlock detection | ❌ Not tracked | Enhancement | +| Nesting counter (`ncli`) | ✅ Per-CPU nesting | ❌ Not implemented (flat save/restore) | Enhancement | -**Summary:** AdrOS spinlocks are correct for single-core. The supplementary material's SMP-aware features (CPU tracking, nesting) are needed only when AdrOS targets multi-core. +**Summary:** AdrOS spinlocks are correct and used throughout the kernel (PMM, heap, slab, scheduler, TTY). SMP-aware features (CPU tracking, nesting) are enhancements. --- @@ -163,34 +168,38 @@ Unix-like, POSIX-compatible operating system. |--------|--------------------------|---------------------|-----| | `crt0.S` (entry point) | ✅ `_start` → `main` → `exit` | ✅ `user/crt0.S` with argc/argv setup | None | | Syscall stub (int 0x80) | ✅ `_syscall_invoke` via registers | ✅ `_syscall` in `user/syscall.S` | None | -| `SYSENTER` fast path | ✅ vDSO + MSR setup | ❌ Only `int 0x80` | Enhancement | -| libc wrappers | ✅ `syscalls.c` with errno | ❌ Raw syscall wrappers only, no errno | **Key gap** | +| libc wrappers | ✅ `syscalls.c` with errno | ✅ ulibc: `printf`, `malloc`/`free`/`calloc`/`realloc`, `string.h`, `errno.h` | None | | `init.c` (early userspace) | ✅ mount + pivot_root + execve | ✅ `user/init.c` — comprehensive smoke tests | Different purpose | | User linker script | ✅ `user.ld` at 0x08048000 | ✅ `user/user.ld` at 0x00400000 | Both valid | +| `SYSENTER` fast path | ✅ vDSO + MSR setup | ✅ `sysenter_init.c` — MSR setup + handler | None | -**Summary:** AdrOS has a working userspace with syscall stubs and a comprehensive test binary. Missing a proper libc and `SYSENTER` optimization. +**Summary:** AdrOS has a working userspace with ulibc, SYSENTER fast path, and a comprehensive test binary. Missing a shell and core utilities. --- -### 1.11 Drivers (Not Yet in AdrOS) +### 1.11 Drivers | Driver | Supplementary Suggestion | AdrOS Current State | |--------|--------------------------|---------------------| -| PCI enumeration | ✅ Full scan (bus/dev/func) | ❌ Not implemented | +| PCI enumeration | ✅ Full scan (bus/dev/func) | ✅ Full scan with BAR + IRQ (`src/hal/x86/pci.c`) | +| ATA DMA (Bus Master IDE) | Not discussed | ✅ Bounce buffer, PRDT, IRQ-coordinated (`src/hal/x86/ata_dma.c`) | +| LAPIC + IOAPIC | Not discussed | ✅ Replaces legacy PIC; IRQ routing | +| SMP (multi-CPU boot) | Not discussed | ✅ 4 CPUs via INIT-SIPI-SIPI, per-CPU data via GS | +| ACPI (MADT parsing) | Not discussed | ✅ CPU topology + IOAPIC discovery | +| VBE/Framebuffer | ✅ Map LFB + MTRR write-combining | ✅ Maps LFB, pixel drawing, font rendering (no MTRR WC) | | Intel E1000 NIC | ✅ RX/TX descriptor rings + DMA | ❌ Not implemented | -| VBE/Framebuffer | ✅ Map LFB + MTRR write-combining | ❌ VGA text mode only | | Intel HDA Audio | ✅ DMA ring buffers | ❌ Not implemented | | lwIP TCP/IP stack | ✅ `sys_arch.c` bridge | ❌ Not implemented | --- -### 1.12 Advanced Features (Not Yet in AdrOS) +### 1.12 Advanced Features | Feature | Supplementary Suggestion | AdrOS Current State | |---------|--------------------------|---------------------| -| Copy-on-Write (CoW) fork | ✅ Full implementation with ref-counting | ❌ Full address space copy | -| Slab allocator | ✅ `slab_cache_t` with free-list-in-place | ❌ Only `kmalloc`/`kfree` | -| Shared memory (shmem/mmap) | ✅ `sys_shmget` / `sys_shmat` | ❌ Not implemented | +| Copy-on-Write (CoW) fork | ✅ Full implementation with ref-counting | ✅ `vmm_as_clone_user_cow()` + `vmm_handle_cow_fault()` | +| Slab allocator | ✅ `slab_cache_t` with free-list-in-place | ✅ `slab_cache_init`/`slab_alloc`/`slab_free` with spinlock | +| Shared memory (shmem/mmap) | ✅ `sys_shmget` / `sys_shmat` | ✅ `shm_get`/`shm_at`/`shm_dt`/`shm_ctl` in `src/kernel/shm.c` | | Zero-copy DMA I/O | ✅ Map DMA buffer into user VA | ❌ Not implemented | | vDSO | ✅ Kernel-mapped page with syscall code | ❌ Not implemented | @@ -198,123 +207,141 @@ Unix-like, POSIX-compatible operating system. ## Part 2 — POSIX Compatibility Assessment -### Overall Score: **~45% toward a practical Unix-like POSIX system** +### Overall Score: **~70% toward a practical Unix-like POSIX system** -This score reflects that AdrOS has the **core architectural skeleton** of a Unix system -fully in place, but lacks several key POSIX interfaces and userland components needed -for real-world use. +This score reflects that AdrOS has a **mature kernel** with most core subsystems +implemented and working. The main remaining gaps are userland tooling (shell, core +utilities) and some POSIX interfaces (`mmap`, permissions, links). ### What AdrOS Already Has (Strengths) -1. **Process model** — `fork`, `execve`, `waitpid`, `exit`, `getpid`, `getppid`, `setsid`, `setpgid`, `getpgrp` — all working -2. **File I/O** — `open`, `read`, `write`, `close`, `lseek`, `stat`, `fstat`, `dup`, `dup2`, `dup3`, `pipe`, `pipe2`, `fcntl`, `getdents` — comprehensive -3. **Signals** — `sigaction`, `sigprocmask`, `kill`, `sigreturn` with full trampoline — robust -4. **VFS** — 5 filesystem types, mount table, path resolution, per-process cwd — excellent -5. **TTY/PTY** — Line discipline, job control, blocking I/O, `ioctl` — very good -6. **Select/Poll** — Working for pipes and TTY devices -7. **Memory isolation** — Per-process address spaces, user/kernel separation, `uaccess` validation -8. **ELF loading** — Secure loader with W^X enforcement -9. **Spinlocks** — Correct `xchg`-based implementation with IRQ save/restore +1. **Process model** — `fork` (CoW), `execve`, `waitpid`, `exit`, `getpid`, `getppid`, `setsid`, `setpgid`, `getpgrp`, `brk` — all working +2. **File I/O** — `open`, `read`, `write`, `close`, `lseek`, `stat`, `fstat`, `dup`, `dup2`, `dup3`, `pipe`, `pipe2`, `fcntl`, `getdents`, `O_CLOEXEC`, `FD_CLOEXEC` — comprehensive +3. **Signals** — `sigaction`, `sigprocmask`, `kill`, `sigreturn` with full trampoline, Ctrl+C/Z/D signal chars — robust +4. **VFS** — 6 filesystem types (tmpfs, devfs, overlayfs, diskfs, persistfs, procfs), mount table, path resolution — excellent +5. **TTY/PTY** — Line discipline, raw mode, job control, signal chars, `TIOCGWINSZ`, PTY — very good +6. **Select/Poll** — Working for pipes, TTY, PTY, `/dev/null` +7. **Memory management** — PMM (spinlock + refcount), VMM (CoW, recursive PD), heap (dynamic 64MB), slab allocator, SMEP, shared memory +8. **Hardware** — PCI, ATA DMA, LAPIC/IOAPIC, SMP (4 CPUs), ACPI, VBE framebuffer, SYSENTER, CPUID +9. **Userland** — ulibc (`printf`, `malloc`, `string.h`, `errno.h`), ELF loader with W^X +10. **Testing** — 47 host unit tests, 19 smoke tests, cppcheck, GDB scripted checks +11. **Security** — SMEP, user_range_ok hardened, sigreturn eflags sanitized, atomic file refcounts ### What's Missing for Practical POSIX (Gaps by Priority) #### Tier 1 — Blocks basic usability | Gap | Impact | Effort | |-----|--------|--------| -| **Minimal libc** (`printf`, `malloc`, `string.h`, `stdio.h`) | Can't build real userland programs | Medium | +| ~~Minimal libc~~ | ✅ ulibc implemented | Done | | **Shell** (`sh`-compatible) | No interactive use without it | Medium | -| **Signal characters** (Ctrl+C → SIGINT, Ctrl+D → EOF) | Can't interrupt/control processes | Small | -| **`brk`/`sbrk`** (user heap) | No `malloc` in userspace | Small-Medium | +| ~~Signal characters~~ | ✅ Ctrl+C/Z/D implemented | Done | +| ~~`brk`/`sbrk`~~ | ✅ Implemented | Done | | **Core utilities** (`ls`, `cat`, `echo`, `mkdir`, `rm`) | No file management | Medium | +| **`/dev/zero`** | Missing common device node | Small | +| **Multiple PTY pairs** | Only 1 pair limits multi-process use | Small | #### Tier 2 — Required for POSIX compliance | Gap | Impact | Effort | |-----|--------|--------| -| **`mmap`/`munmap`** | No memory-mapped files, no shared memory | Medium-Large | -| **`O_CLOEXEC`** | FD leaks across `execve` | Small | +| **`mmap`/`munmap`** | No memory-mapped files | Medium-Large | +| ~~`O_CLOEXEC`~~ | ✅ Implemented | Done | | **Permissions** (`uid`/`gid`/mode/`chmod`/`chown`) | No multi-user security | Medium | | **Hard/symbolic links** | Incomplete filesystem semantics | Medium | -| **`/proc` filesystem** | No process introspection | Medium | -| **`nanosleep`/`clock_gettime`** | No time management | Small | -| **Raw TTY mode** | Can't run editors or games | Small | +| **`/proc` per-process** | Only `/proc/meminfo`, no `/proc/[pid]` | Medium | +| ~~`nanosleep`/`clock_gettime`~~ | ✅ Implemented | Done | +| ~~Raw TTY mode~~ | ✅ Implemented | Done | +| **VMIN/VTIME** | Non-canonical timing not supported | Small | #### Tier 3 — Full Unix experience | Gap | Impact | Effort | |-----|--------|--------| -| **CoW fork** | Memory waste on fork-heavy workloads | Large | +| ~~CoW fork~~ | ✅ Implemented | Done | | **PAE + NX bit** | No hardware W^X enforcement | Large | -| **Slab allocator** | Performance for frequent small allocs | Medium | +| ~~Slab allocator~~ | ✅ Implemented | Done | | **Networking** (socket API + TCP/IP) | No network connectivity | Very Large | | **Threads** (`clone`/`pthread`) | No multi-threaded programs | Large | | **Dynamic linking** (`ld.so`) | Can't use shared libraries | Very Large | -| **VBE framebuffer** | No graphical output | Medium | -| **PCI + device drivers** | No hardware discovery | Large | +| ~~VBE framebuffer~~ | ✅ Implemented | Done | +| ~~PCI + device drivers~~ | ✅ PCI + ATA DMA implemented | Done | --- ## Part 3 — Architectural Comparison Summary | Dimension | Supplementary Material | AdrOS Current | Verdict | -|-----------|----------------------|---------------|---------| +|-----------|----------------------|---------------|----------| | **Boot flow** | GRUB → Stub (LZ4) → Kernel → USTAR InitRD | GRUB → Kernel → Custom InitRD → OverlayFS | Both valid; AdrOS is simpler | -| **Memory architecture** | PMM + Slab + CoW + Zero-Copy DMA | PMM + Heap (linked list) | Supplementary is more advanced | -| **Scheduler** | O(1) with bitmap + active/expired arrays | Round-robin with linked list | Supplementary is more advanced | -| **VFS** | USTAR + FAT (planned) | tmpfs + devfs + overlayfs + diskfs + persistfs | **AdrOS is more advanced** | -| **Syscall interface** | int 0x80 + SYSENTER + vDSO | int 0x80 only | Supplementary has more optimization | -| **Signal handling** | Basic trampoline concept | Full SA_SIGINFO + sigreturn + sigframe | **AdrOS is more advanced** | -| **TTY/PTY** | Basic circular buffer | Full PTY with job control | **AdrOS is more advanced** | -| **Synchronization** | SMP-aware spinlocks with CPU tracking | Single-core spinlocks with IRQ save | Supplementary targets SMP | -| **Userland** | libc stubs + init + shell concept | Raw syscall wrappers + test binary | Both early-stage | -| **Drivers** | PCI + E1000 + VBE + HDA (conceptual) | UART + VGA text + PS/2 + ATA PIO | Supplementary has more scope | +| **Memory architecture** | PMM + Slab + CoW + Zero-Copy DMA | PMM (spinlock+refcount) + Slab + CoW + Heap (64MB dynamic) + SMEP + Shmem | **Comparable** (AdrOS missing zero-copy DMA) | +| **Scheduler** | O(1) with bitmap + active/expired arrays | O(1) with bitmap + active/expired, 32 priority levels | **Comparable** (AdrOS missing decay + per-CPU) | +| **VFS** | USTAR + FAT (planned) | tmpfs + devfs + overlayfs + diskfs + persistfs + procfs | **AdrOS is more advanced** | +| **Syscall interface** | int 0x80 + SYSENTER + vDSO | int 0x80 + SYSENTER | **Comparable** (AdrOS missing vDSO) | +| **Signal handling** | Basic trampoline concept | Full SA_SIGINFO + sigreturn + sigframe + signal chars | **AdrOS is more advanced** | +| **TTY/PTY** | Basic circular buffer | Full PTY + raw mode + job control + signal chars + TIOCGWINSZ | **AdrOS is more advanced** | +| **Synchronization** | SMP-aware spinlocks with CPU tracking | Spinlocks with IRQ save, used throughout (PMM, heap, slab, sched) | **Comparable** (AdrOS missing CPU tracking) | +| **Userland** | libc stubs + init + shell concept | ulibc (printf, malloc, string.h) + init + echo | **Comparable** (AdrOS missing shell) | +| **Drivers** | PCI + E1000 + VBE + HDA (conceptual) | PCI + ATA DMA + VBE + LAPIC/IOAPIC + SMP + ACPI | **AdrOS is more advanced** | --- ## Part 4 — Recommendations -### Immediate Actions (use supplementary material as inspiration) +### Completed (since initial analysis) -1. **Add signal characters to TTY** — Ctrl+C/Ctrl+Z/Ctrl+D handling in `tty_input_char()`. Small change, huge usability gain. +1. ~~Add signal characters to TTY~~ ✅ Ctrl+C/Z/D/\\ implemented +2. ~~Implement `brk`/`sbrk` syscall~~ ✅ `syscall_brk_impl()` +3. ~~Build minimal libc~~ ✅ ulibc with printf, malloc, string.h, errno.h +4. ~~PMM ref-counting~~ ✅ `pmm_incref`/`pmm_decref`/`pmm_get_refcount` +5. ~~CoW fork~~ ✅ `vmm_as_clone_user_cow()` + `vmm_handle_cow_fault()` +6. ~~O(1) scheduler~~ ✅ Bitmap + active/expired arrays, 32 priority levels +7. ~~Slab allocator~~ ✅ `slab_cache_init`/`slab_alloc`/`slab_free` +8. ~~PCI enumeration~~ ✅ Full bus/slot/func scan +9. ~~CPUID~~ ✅ Leaf 0/1/7/extended, SMEP enabled -2. **Implement `brk`/`sbrk` syscall** — Track a per-process heap break pointer. Essential for userland `malloc`. +### Immediate Actions (next priorities) -3. **Build minimal libc** — Start with `write`-based `printf`, `brk`-based `malloc`, `string.h`. The supplementary `syscalls.c.txt` and `unistd.c.txt` show the pattern. +1. **Build a shell** — All required syscalls are implemented. This is the single biggest usability unlock. -4. **Build a shell** — All required syscalls (`fork`+`execve`+`waitpid`+`pipe`+`dup2`+`chdir`) are already implemented. +2. **Core utilities** — `ls`, `cat`, `echo`, `mkdir`, `rm`. All required VFS operations exist. -### Medium-Term (architectural improvements from supplementary material) +3. **`/dev/zero`** + **`/dev/random`** — Simple device nodes, small effort. -5. **PMM ref-counting** — Add `uint16_t` ref-count array alongside bitmap. Prerequisite for CoW. +4. **Multiple PTY pairs** — Currently only 1 pair. Needed for multi-process terminal use. -6. **CoW fork** — Use PTE bit 9 as CoW marker, handle in page fault. The supplementary material's `vmm_copy_for_fork()` pattern is clean. +### Medium-Term -7. **W^X policy function** — Add `vmm_apply_wx_policy()` as a centralized check. Currently AdrOS achieves W^X ad-hoc in the ELF loader. +5. **`mmap`/`munmap`** — Requires `vmm_find_free_area()`. Critical for POSIX. -8. **`mmap`/`munmap`** — Requires `vmm_find_free_area()` from supplementary material. Critical for POSIX. +6. **Permissions model** — `uid`/`gid`/mode bits, `chmod`, `chown`, `access`, `umask`. -### Long-Term (from supplementary material roadmap) +7. **`/proc` per-process** — `/proc/[pid]/status`, `/proc/[pid]/maps`. -9. **CPUID + PAE + NX** — Follow the `cpu_get_features()` / `cpu_enable_nx()` pattern for hardware W^X. +8. **Hard/symbolic links** — `link`, `symlink`, `readlink`. -10. **O(1) scheduler** — The active/expired bitmap swap pattern is elegant and well-suited for AdrOS. +### Long-Term -11. **Slab allocator** — The supplementary material's free-list-in-place design is simple and effective. +9. **PAE + NX** — Hardware W^X enforcement. -12. **PCI + networking** — Follow the PCI scan → BAR mapping → E1000 DMA ring → lwIP bridge pattern. +10. **Networking** — E1000 DMA ring → lwIP bridge → socket API. + +11. **Threads** — `clone`/`pthread`. + +12. **Dynamic linking** — `ld.so`. --- ## Conclusion -AdrOS is a **well-architected hobby OS** that has already implemented many of the hardest -parts of a Unix-like system: process management with signals, a multi-filesystem VFS, -PTY with job control, and a secure ELF loader. It is approximately **45% of the way** -to a practical POSIX-compatible system. +AdrOS is a **mature hobby OS** that has implemented most of the hard parts of a Unix-like +system: CoW fork, O(1) scheduler, slab allocator, SMP boot, PCI/DMA drivers, full signal +handling, a multi-filesystem VFS, PTY with job control, and a secure ELF loader. It is +approximately **70% of the way** to a practical POSIX-compatible system. -The supplementary material provides excellent **architectural blueprints** for the next -evolution: CoW memory, O(1) scheduling, hardware NX, and networking. However, AdrOS is -already **ahead** of the supplementary material in several areas (VFS diversity, signal -handling, PTY/job control). +The supplementary material's architectural blueprints have been largely **realized**: +CoW memory, O(1) scheduling, slab allocator, PCI enumeration, and CPUID detection are +all implemented. AdrOS is **ahead** of the supplementary material in several areas +(VFS diversity, signal handling, TTY/PTY, driver support, SMP). -The most impactful next steps are **not** the advanced features from the supplementary -material, but rather the **userland enablers**: a minimal libc, a shell, and `brk`/`sbrk`. -These would transform AdrOS from a kernel with smoke tests into an interactive Unix system. +The most impactful next steps are **userland tooling**: a shell and core utilities. +With ulibc already providing `printf`/`malloc`/`string.h`, and all required syscalls +implemented, building a functional shell is now straightforward. This would transform +AdrOS from a kernel with smoke tests into an **interactive Unix system**.