From 0310a1541c66276ae91efd10ea7e921b616c8aa8 Mon Sep 17 00:00:00 2001 From: Tulio A M Mendes Date: Mon, 27 Apr 2026 14:13:09 -0300 Subject: [PATCH] docs: update all documentation with recent kernel/userspace/test changes MIME-Version: 1.0 Content-Type: text/plain; charset=utf8 Content-Transfer-Encoding: 8bit README.md: - Host tests: 69 → 212 (test_utils 63, test_security 38, host_utils 111) - Battery checks: 33 → 152 (120 smoke + SMP=1/2 + multi-disk + VFS + ping) - GDB checks: 6 → 10 (PID state, scheduler bitmap, mount count, frame refcount) - W^X description: updated for elf32_reprotect_segments() + boundary page handling - tests/ directory description updated TESTING_PLAN.md: - All test counts updated - Pure function list expanded with tar_parse_octal, mount prefix/normalize, vfs_check_permission, elf32_validate, signal mask logic, parse_symbolic - Makefile targets updated SYSCALL_TEST_COVERAGE.md: - Date updated to 2026-04-27 - Battery: 33 → 152, Host: 69 → 212 POSIX_ROADMAP.md: - Added host unit tests (212), test battery (152), GDB checks (10) rows - Host utility test harness: 68 → 111 AUDIT_REPORT.md: - Added post-audit fix table: ELF W^X reprotect, vfs_lookup_initrd, heap corruption handling, SHM UAF, procfs UAF, ext2 partial inode, rq_remove_if_queued, execve_copy_user_str, frame refcount overflow TIER6_PLAN.md / FULL_POSIX_AUDIT.md: - Test counts and commit references updated --- README.md | 16 ++++++++-------- docs/AUDIT_REPORT.md | 14 ++++++++++++++ docs/FULL_POSIX_AUDIT.md | 2 +- docs/POSIX_ROADMAP.md | 5 ++++- docs/SYSCALL_TEST_COVERAGE.md | 10 +++++----- docs/TESTING_PLAN.md | 14 ++++++++++---- docs/TIER6_PLAN.md | 2 +- 7 files changed, 43 insertions(+), 20 deletions(-) diff --git a/README.md b/README.md index 1414033a..a94cae23 100644 --- a/README.md +++ b/README.md @@ -34,7 +34,7 @@ AdrOS is a Unix-like, POSIX-compatible, multi-architecture operating system deve - **`mmap`/`munmap`** — anonymous mappings, shared memory backing, and file-backed (fd) mappings - **SMEP** — Supervisor Mode Execution Prevention enabled in CR4 - **SMAP** — Supervisor Mode Access Prevention enabled in CR4 (bit 21) -- **W^X** — user `.text` segments marked read-only after ELF load; NX on data segments +- **W^X** — user `.text` segments mapped RW for load + relocations, then re-protected to RO+X after `elf32_reprotect_segments()`; NX on data segments; boundary pages shared between .text/.data stay writable - **Guard pages** — 32KB user stack with unmapped guard page below (triggers SIGSEGV on overflow); kernel stacks use dedicated guard-paged region at `0xC8000000` - **ASLR** — TSC-seeded xorshift32 PRNG randomizes user stack base by up to 1MB per `execve` - **vDSO** — kernel-updated shared page mapped read-only into every user process at `0x007FE000` @@ -163,11 +163,11 @@ AdrOS is a Unix-like, POSIX-compatible, multi-architecture operating system deve - **PMM spinlock** for SMP safety ### Testing -- **69 host-side tests** — `test_utils.c` (28) + `test_security.c` (19) + `test_host_utils.sh` (22 cross-compiled utility tests) +- **212 host-side tests** — `test_utils.c` (63: itoa/atoi, path_normalize, align, tar_parse_octal, mount prefix/normalize, VFS permission, ELF validation) + `test_security.c` (38: user_range_ok, bitmap, eflags, signal mask logic, chmod symbolic parsing) + `test_host_utils.sh` (111 cross-compiled utility tests) - **120 QEMU smoke tests** — 4-CPU expect-based (file I/O, signals, memory mgmt, IPC, devices, procfs, networking, epoll, epollet, inotify, aio, nanosleep, CoW fork, readv/writev, fsync, flock, posix_spawn, TSC precision, gettimeofday, mprotect, getrlimit/setrlimit, uname, LZ4, lazy PLT, execve, clone, pivot_root, dlopen/dlsym/dlclose, execveat, futex, sigaltstack, socket API, mqueue, semaphores, chown, mount/umount2) -- **33-check test battery** — multi-disk ATA (hda+hdb+hdd), VFS mount, ping, diskfs ops, clone, socket API, mqueue, semaphores, futex, sigaltstack, chown, mount/umount2 (`make test-battery`) +- **152-check test battery** — 120 smoke patterns + SMP=1 boot (12) + SMP=2 boot (6) + multi-disk ATA (hda+hdb+hdd) + VFS mount + ping + diskfs ops (`make test-battery`) +- **10 GDB scripted checks** — heap/PMM/VGA integrity, PID 1 state, scheduler bitmap, mount count, frame refcount - **Static analysis** — cppcheck, sparse, gcc -fanalyzer -- **GDB scripted checks** — heap/PMM/VGA integrity - `make test-all` runs everything ## Running @@ -204,7 +204,7 @@ QEMU debug helpers: See [POSIX_ROADMAP.md](docs/POSIX_ROADMAP.md) for a detailed checklist. -**All 31 planned POSIX tasks are complete**, plus 60 additional features (91 total). The kernel has **141 syscalls** with **124 tested** (87.9% coverage) by the 120 smoke tests. The kernel covers **~98%** of the core POSIX interfaces needed for a practical Unix-like system. All 120 smoke tests, 33 battery checks, and 69 host tests pass clean. ARM64, RISC-V 64, and MIPS32 boot on QEMU. +**All 31 planned POSIX tasks are complete**, plus 60 additional features (91 total). The kernel has **141 syscalls** with **124 tested** (87.9% coverage) by the 120 smoke tests. The kernel covers **~98%** of the core POSIX interfaces needed for a practical Unix-like system. All 120 smoke tests, 152 battery checks, and 212 host tests pass clean. ARM64, RISC-V 64, and MIPS32 boot on QEMU. Rump Kernel integration is in progress — prerequisites (condition variables, TSC nanosecond clock, IRQ chaining) are implemented and the `rumpuser` hypercall scaffold is in place. @@ -216,8 +216,8 @@ Rump Kernel integration is in progress — prerequisites (condition variables, T | Syscalls tested by fulltest | 124 (87.9%) | | Syscalls without test | 17 (12.1%) | | Smoke tests | 120 | -| Battery checks | 33 | -| Host tests | 69 | +| Battery checks | 152 | +| Host tests | 212 | **17 untested syscalls:** `shmctl`, `set_thread_area` (indirect via clone), `accept`, `connect`, `send`, `recv`, `sendto`, `recvfrom`, `fdatasync`, `getaddrinfo`, `sendmsg`, `recvmsg`, `aio_suspend`, `setsockopt`, `getsockopt`, `getpeername`, `wait4` @@ -248,7 +248,7 @@ For **100% POSIX compliance**, the following categories are still missing: - `user/` — Userland programs (52 commands: `init.c`, `sh.c`, `cat.c`, `ls.c`, `echo.c`, `cp.c`, `mv.c`, `grep.c`, `sed.c`, `awk.c`, `find.c`, `which.c`, `ps.c`, `top.c`, `kill.c`, `mount.c`, etc. + `ldso.c`, `fulltest.c`, `pie_main.c`) - `user/doom/` — DOOM port (doomgeneric engine + AdrOS platform adapter) - `user/ulibc/` — Minimal C library (`printf`, `malloc`, `string.h`, `errno.h`, `pthread.h`, `signal.h`, `stdio.h`, `stdlib.h`, `ctype.h`, `math.h`, `sys/mman.h`, `sys/ioctl.h`, `sys/uio.h`, `time.h`, `linux/futex.h`) -- `tests/` — Host unit tests, smoke tests, GDB scripted checks +- `tests/` — Host unit tests (212: utils 63 + security 38 + utilities 111), smoke tests (120), battery tests (152), GDB scripted checks (10), serial input test - `tools/` — Build tools (`mkinitrd` — produces USTAR archives with LZ4 Frame compression) - `docs/` — Documentation (POSIX roadmap, audit report, supplementary analysis, testing plan) - `third_party/lwip/` — lwIP TCP/IP stack (vendored) diff --git a/docs/AUDIT_REPORT.md b/docs/AUDIT_REPORT.md index a0fc4e6a..09a7df0b 100644 --- a/docs/AUDIT_REPORT.md +++ b/docs/AUDIT_REPORT.md @@ -328,3 +328,17 @@ SMEP enabled via CR4, heap grows dynamically to 64MB, waitpid NULL guard. **1 additional MODERATE fixed**: `schedule()` now calls `context_switch` BEFORE `spin_unlock_irqrestore`, preventing the timer-fired race window. **Remaining**: 4 MODERATE (open): hardcoded heap VA (1.2), x86 registers leak in interrupts.h (1.3), itoa no buffer size (2.6), itoa UB for INT_MIN (2.7), fd bounds (3.5), kfree doesn't zero (4.2). + +**Additional fixes (post-audit):** + +| Fix | Severity | Description | +|-----|----------|-------------| +| ELF W^X reprotect | CRITICAL | `elf32_reprotect_segments()` re-protects text segments to RO+X AFTER relocations; boundary pages shared with data segments stay writable | +| `vfs_lookup_initrd` | HIGH | ELF loader and execve use `vfs_lookup_initrd()` (bypasses mount table) so `pivot_root` doesn't break binary lookups | +| Heap corruption handling | HIGH | On bad magic/double-free, kernel marks process ZOMBIE and calls `schedule()` instead of infinite `hal_cpu_idle()` loop | +| SHM UAF | HIGH | `shm_at()` rejects IPC_RMID'd segments with `-EIDRM`; `shm_get()` skips marked_rm segments | +| Procfs UAF | CRITICAL | Dedicated `g_pid_cmdline` pool prevents `/proc//cmdline` from overwriting status entries | +| ext2 partial inode | HIGH | `ext2_file_write` calls `ext2_write_inode()` after each `i_blocks` increment for on-disk consistency | +| `rq_remove_if_queued` | HIGH | Scans ALL priority queues in both active/expired runqueues (not just `p->priority`) | +| `execve_copy_user_str` | HIGH | Upfront `user_range_ok()` check before byte-by-byte copy loop | +| Frame refcount overflow | MODERATE | `frame_refcount[]` changed from `uint16_t` to `uint32_t` to prevent overflow | diff --git a/docs/FULL_POSIX_AUDIT.md b/docs/FULL_POSIX_AUDIT.md index 630b20f5..d63bdfc0 100644 --- a/docs/FULL_POSIX_AUDIT.md +++ b/docs/FULL_POSIX_AUDIT.md @@ -1,7 +1,7 @@ # AdrOS — Full POSIX/Unix Compatibility Audit & Porting Analysis **Date:** 2026-03-14 (updated after Tier 1-5 implementation) -**Original commit:** 2deaf85 — **Current state reflects commit aa5474a (Tiers 1-5 complete)** +**Original commit:** 2deaf85 — **Current state reflects commit 0e463dd (Tiers 1-5 complete, tests expanded)** --- diff --git a/docs/POSIX_ROADMAP.md b/docs/POSIX_ROADMAP.md index 631a37ef..6aae5f9c 100644 --- a/docs/POSIX_ROADMAP.md +++ b/docs/POSIX_ROADMAP.md @@ -296,6 +296,9 @@ Notes: |---------|--------|-------| | ELF32 loader | [x] | Secure with W^X + ASLR; supports `ET_EXEC` + `ET_DYN` + `PT_INTERP` | | `/sbin/fulltest` (smoke tests) | [x] | Comprehensive test suite (120 checks: file I/O, signals, memory, IPC, devices, procfs, networking, epoll, epollet, inotify, aio, nanosleep, CoW fork, readv/writev, fsync, flock, posix_spawn, TSC precision, gettimeofday, mprotect, getrlimit/setrlimit, uname, LZ4, lazy PLT, execve, clone, pivot_root, dlopen/dlsym/dlclose, execveat, futex, sigaltstack, socket API, mqueue, semaphores, chown, mount/umount2) | +| Host unit tests | [x] | 212 tests: `test_utils.c` (63: itoa/atoi, path_normalize, align, tar_parse_octal, mount prefix/normalize, VFS permission, ELF validation) + `test_security.c` (38: user_range_ok, bitmap, eflags, signal mask, chmod symbolic) + `test_host_utils.sh` (111 utility tests) | +| Test battery | [x] | 152 checks: 120 smoke patterns + SMP=1 boot (12) + SMP=2 boot (6) + multi-disk ATA + VFS mount + ping + diskfs | +| GDB scripted checks | [x] | 10 checks: heap/PMM/VGA integrity, PID 1 state, scheduler bitmap, mount count, frame refcount | | `/bin/echo` | [x] | argv/envp test | | `/bin/sh` | [x] | POSIX sh-compatible shell; builtins, pipes, redirects, `$PATH` search | | `/bin/cat` | [x] | | @@ -442,7 +445,7 @@ Notes: 85. ~~MIPS32 bring-up (QEMU Malta boot + UART)~~ ✅ 86. ~~SysV `/sbin/init` (inittab, runlevels, respawn)~~ ✅ 87. ~~50+ userland POSIX utilities (cp, mv, sed, awk, grep, find, etc.)~~ ✅ -88. ~~Host utility test harness (68 cross-platform tests)~~ ✅ +88. ~~Host utility test harness (111 cross-platform tests)~~ ✅ 89. ~~Native toolchain (GCC 13.2 + Binutils 2.42, Canadian cross for i686-adros)~~ ✅ 90. ~~`mount` syscall — runtime filesystem mounting~~ ✅ 91. ~~USTAR InitRD format with LZ4 Frame compression~~ ✅ diff --git a/docs/SYSCALL_TEST_COVERAGE.md b/docs/SYSCALL_TEST_COVERAGE.md index 1c531912..10dba6dd 100644 --- a/docs/SYSCALL_TEST_COVERAGE.md +++ b/docs/SYSCALL_TEST_COVERAGE.md @@ -1,10 +1,10 @@ # AdrOS — Syscall Test Coverage & POSIX Compliance Analysis -**Date:** 2026-04-26 +**Date:** 2026-04-27 **Kernel syscalls:** 141 (defined in `include/syscall.h`) **Fulltest smoke tests:** 120 -**Test battery checks:** 33 -**Host tests:** 69 +**Test battery checks:** 152 +**Host tests:** 212 --- @@ -255,8 +255,8 @@ These syscalls are implemented and tested, but have known limitations: | Syscalls tested by fulltest | 124 (87.9%) | | Syscalls without test | 17 (12.1%) | | Smoke tests | 120 | -| Battery checks | 33 | -| Host tests | 69 | +| Battery checks | 152 | +| Host tests | 212 | | Test:syscall ratio | ~0.85:1 (not 1:1) | | POSIX-mandated features missing | ~55+ | | Major missing categories | Threads, TCP loopback, mkfifo/mknod, chroot, getgroups/setgroups, POSIX timers, msync, fchmod, pathconf, socketpair | diff --git a/docs/TESTING_PLAN.md b/docs/TESTING_PLAN.md index 58cd3434..3a6d7a7c 100644 --- a/docs/TESTING_PLAN.md +++ b/docs/TESTING_PLAN.md @@ -6,9 +6,9 @@ All testing layers are **implemented and operational**: - **Static analysis** (`make check`): cppcheck + sparse + gcc -fanalyzer - **QEMU smoke tests** (`make test`): expect-based, 120 checks (file I/O, signals, memory, IPC, devices, procfs, networking, epoll, epollet, inotify, aio, nanosleep, CLOCK_REALTIME/CLOCK_MONOTONIC, /dev/urandom, /proc/cmdline, CoW fork, readv/writev, fsync, truncate, getuid/getgid, chmod, flock, times, gettid, posix_spawn, TSC ns precision, SIGSEGV, gettimeofday, mprotect, getrlimit/setrlimit, uname, LZ4 initrd decomp, lazy PLT, execve, clone, pivot_root, dlopen/dlsym/dlclose, execveat, futex, sigaltstack, socket API, mqueue, semaphores, chown, mount/umount2), 4-CPU SMP, 120s timeout -- **Test battery** (`make test-battery`): 33 checks across QEMU scenarios — multi-disk ATA, VFS mount, ping, diskfs, clone, socket API, mqueue, semaphores, futex, sigaltstack, chown, mount/umount2 -- **Host unit tests** (`make test-host`): 69 tests — `test_utils.c` (28) + `test_security.c` (19) + `test_host_utils.sh` (22 cross-compiled utility tests) -- **GDB scripted checks** (`make test-gdb`): heap/PMM/VGA integrity validation +- **Test battery** (`make test-battery`): 152 checks across QEMU scenarios — 120 smoke patterns + SMP=1 boot (12 checks) + SMP=2 boot (6 checks) + multi-disk ATA (hda+hdb+hdd) + VFS mount + ICMP ping + diskfs ops +- **Host unit tests** (`make test-host`): 212 tests — `test_utils.c` (63: itoa/atoi, path_normalize, align, tar_parse_octal, mount prefix/normalize, VFS permission, ELF validation) + `test_security.c` (38: user_range_ok, bitmap, eflags, signal mask logic, chmod symbolic parsing) + `test_host_utils.sh` (111 cross-compiled utility tests) +- **GDB scripted checks** (`make test-gdb`): 10 checks — heap integrity, PMM bitmap sanity, VGA mapping, PID 1 state, scheduler runqueue bitmap, mount count, frame 0 refcount - **Full suite** (`make test-all`): runs check + test-host + test - **Multi-arch build verification**: `make ARCH=arm`, `make ARCH=riscv`, and `make ARCH=mips` compile and boot on QEMU @@ -84,6 +84,12 @@ Some kernel functions are pure computation with no hardware dependency: - `path_normalize_inplace` (critical for security) - `align_up`, `align_down` - Bitmap operations +- `tar_parse_octal` (initrd header parsing) +- `path_is_mountpoint_prefix`, `normalize_mountpoint` (VFS mount logic) +- `vfs_check_permission` (file permission checks) +- `elf32_validate` (ELF header validation) +- Signal mask logic (KILL/STOP always deliverable, sigpending = pending & blocked) +- `parse_symbolic` (chmod symbolic mode parsing) These can be compiled and run on the host with `gcc -m32` and a minimal test harness. No need for Unity — a simple `assert()` + `main()` is sufficient for a kernel project. @@ -112,7 +118,7 @@ To run manually: boot AdrOS with `-vga std`, then execute `/bin/doom.elf` from t ```makefile make check # cppcheck + sparse + gcc -fanalyzer make test # QEMU + expect automated smoke test (120 checks incl. ICMP ping, epoll, epollet, inotify, aio, CoW fork, flock, posix_spawn, gettimeofday, mprotect, uname, LZ4, lazy PLT, clone, pivot_root, dlopen/dlsym/dlclose, execveat, futex, sigaltstack, socket API, mqueue, semaphores, chown, mount/umount2) -make test-battery # Full test battery: multi-disk ATA, VFS mount, ping, diskfs, clone, socket API, mqueue, semaphores, futex, sigaltstack, chown, mount/umount2 (33 checks) +make test-battery # Full test battery: 120 smoke + SMP=1 (12) + SMP=2 (6) + multi-disk ATA + VFS mount + ping + diskfs (152 checks) make test-host # Host-side unit tests for pure functions make test-gdb # QEMU + GDB scripted checks (optional) make test-all # All of the above diff --git a/docs/TIER6_PLAN.md b/docs/TIER6_PLAN.md index 8013dced..daba8058 100644 --- a/docs/TIER6_PLAN.md +++ b/docs/TIER6_PLAN.md @@ -2,7 +2,7 @@ **Date:** 2026-03-14 **Prerequisite:** Tiers 1-5 complete (commit aa5474a), audit updated (commit f3a652e) -**Current state:** ~99% POSIX, 141 syscalls, 124/141 tested (87.9%), 120 smoke tests, 33 battery checks, all ulibc headers complete +**Current state:** ~99% POSIX, 141 syscalls, 124/141 tested (87.9%), 120 smoke tests, 152 battery checks, 212 host tests, all ulibc headers complete --- -- 2.43.0