From: Tulio A M Mendes Date: Thu, 12 Feb 2026 02:27:09 +0000 (-0300) Subject: docs: comprehensive documentation update reflecting all 15 implemented features ... X-Git-Url: https://projects.tadryanom.me/?a=commitdiff_plain;h=49882677e94ab6dfa7b6510d5190f9e4a8472da8;p=AdrOS.git docs: comprehensive documentation update reflecting all 15 implemented features (threads, networking, dynamic linking, shell, core utils, permissions, symlinks, PAE+NX, procfs, multi-PTY, VMIN/VTIME) --- diff --git a/BUILD_GUIDE.md b/BUILD_GUIDE.md index 9e83775..cfff7a1 100644 --- a/BUILD_GUIDE.md +++ b/BUILD_GUIDE.md @@ -2,7 +2,7 @@ This guide explains how to build and run AdrOS on your local machine (Linux/WSL). -AdrOS is a Unix-like, POSIX-compatible OS kernel. See [POSIX_ROADMAP.md](docs/POSIX_ROADMAP.md) for the full compatibility checklist. +AdrOS is a Unix-like, POSIX-compatible OS kernel with threads, networking (TCP/IP via lwIP), dynamic linking infrastructure, and a POSIX shell. See [POSIX_ROADMAP.md](docs/POSIX_ROADMAP.md) for the full compatibility checklist. ## 1. Dependencies @@ -72,16 +72,23 @@ Generated outputs/artifacts: Syscall return convention note: - The kernel follows a Linux-style convention: syscalls return `0`/positive values on success, and `-errno` (negative) on failure. -- Userland (`user/init.c`) uses a `__syscall_fix()` helper that converts negative returns to `-1` and sets a global `errno`. -- A full libc-style per-thread `errno` is not yet implemented. +- Userland ulibc uses a `__syscall_ret()` helper that converts negative returns to `-1` and sets a per-thread `errno`. +- Per-thread `errno` is supported via `set_thread_area` + TLS. -### Userland smoke tests +### Userland programs +The following ELF binaries are bundled in the initrd: +- `/bin/init.elf` — comprehensive smoke test suite (19+ checks) +- `/bin/echo` — argv/envp test +- `/bin/sh` — POSIX sh-compatible shell +- `/bin/cat`, `/bin/ls`, `/bin/mkdir`, `/bin/rm` — core utilities + +### Smoke tests The init program (`/bin/init.elf`) runs a comprehensive suite of smoke tests on boot, covering: - File I/O (`open`, `read`, `write`, `close`, `lseek`, `stat`, `fstat`) - Overlay copy-up, `dup2`, `pipe`, `select`, `poll` - TTY/ioctl, job control (`SIGTTIN`/`SIGTTOU`) -- PTY (`/dev/ptmx` + `/dev/pts/0`) -- Signals (`sigaction`, `kill`, `sigreturn`) +- PTY (`/dev/ptmx` + `/dev/pts/N`) +- Signals (`sigaction`, `kill`, `sigreturn`, `SA_SIGINFO`) - Session/process groups (`setsid`, `setpgid`, `getpgrp`) - `isatty`, `O_NONBLOCK` (pipes + PTY), `fcntl` - `pipe2`/`dup3` with flags @@ -91,6 +98,7 @@ The init program (`/bin/init.elf`) runs a comprehensive suite of smoke tests on - `getdents` across multiple FS types (diskfs, devfs, tmpfs) - `fork` (100 children), `waitpid` (`WNOHANG`), `execve` - `SIGSEGV` handler +- diskfs mkdir/unlink/getdents All tests print `[init] ... OK` on success. Any failure calls `sys_exit(1)`. diff --git a/README.md b/README.md index 9371bf4..c391e40 100644 --- a/README.md +++ b/README.md @@ -4,15 +4,16 @@ AdrOS is a Unix-like, POSIX-compatible, multi-architecture operating system developed for research and academic purposes. The goal is to build a secure, monolithic kernel from scratch, eventually serving as a platform for security testing and exploit development. ## Architectures Targeted -- **x86** (32-bit & 64-bit) -- **ARM** (32-bit & 64-bit) -- **MIPS** -- **RISC-V** (32-bit & 64-bit) +- **x86** (32-bit, PAE) — primary, fully functional target +- **ARM** (64-bit) — build infrastructure only +- **MIPS** — build infrastructure only +- **RISC-V** (64-bit) — build infrastructure only ## Technical Stack -- **Language:** C/C++ and Assembly +- **Language:** C and Assembly - **Bootloader:** GRUB2 (Multiboot2 compliant) - **Build System:** Make + Cross-Compilers +- **TCP/IP:** lwIP (lightweight IP stack, NO_SYS=1 mode) ## Features @@ -21,30 +22,37 @@ AdrOS is a Unix-like, POSIX-compatible, multi-architecture operating system deve - **Multiboot2** (via GRUB), higher-half kernel mapping (3GB+) - **CPUID feature detection** — leaf 0/1/7/extended; SMEP/SMAP detection - **SYSENTER fast syscall path** — MSR setup + handler +- **PAE paging with NX bit** — hardware W^X enforcement on data segments ### Memory Management - **PMM** — bitmap allocator with spinlock protection and frame reference counting -- **VMM** — recursive page directory, per-process address spaces, TLB flush +- **VMM** — PAE recursive page directory, per-process address spaces (PDPT + 4 PDs), TLB flush - **Copy-on-Write (CoW) fork** — PTE bit 9 as CoW marker + page fault handler - **Kernel heap** — doubly-linked free list with coalescing, dynamic growth up to 64MB - **Slab allocator** — `slab_cache_t` with free-list-in-place and spinlock - **Shared memory** — System V IPC style (`shmget`/`shmat`/`shmdt`/`shmctl`) +- **`mmap`/`munmap`** — anonymous mappings + shared memory backing - **SMEP** — Supervisor Mode Execution Prevention enabled in CR4 -- **W^X** — user `.text` segments marked read-only after ELF load +- **W^X** — user `.text` segments marked read-only after ELF load; NX on data segments ### Process & Scheduling - **O(1) scheduler** — bitmap + active/expired arrays, 32 priority levels - **Process model** — `fork` (CoW), `execve`, `exit`, `waitpid` (`WNOHANG`), `getpid`, `getppid` +- **Threads** — `clone` syscall with `CLONE_VM`/`CLONE_FILES`/`CLONE_THREAD`/`CLONE_SETTLS` +- **TLS** — `set_thread_area` via GDT entry 22 (user GS segment, ring 3) - **Sessions & groups** — `setsid`, `setpgid`, `getpgrp` +- **Permissions** — per-process `uid`/`gid`; `chmod`, `chown` - **User heap** — `brk`/`sbrk` syscall - **Time** — `nanosleep`, `clock_gettime` (`CLOCK_REALTIME`, `CLOCK_MONOTONIC`) ### Syscalls (x86, `int 0x80` + SYSENTER) - **File I/O:** `open`, `openat`, `read`, `write`, `close`, `lseek`, `stat`, `fstat`, `fstatat`, `dup`, `dup2`, `dup3`, `pipe`, `pipe2`, `select`, `poll`, `ioctl`, `fcntl`, `getdents` -- **Directory ops:** `mkdir`, `rmdir`, `unlink`, `unlinkat`, `rename`, `chdir`, `getcwd` -- **Signals:** `sigaction`, `sigprocmask`, `kill`, `sigreturn` (full trampoline with `SA_SIGINFO`) +- **Directory ops:** `mkdir`, `rmdir`, `unlink`, `unlinkat`, `rename`, `chdir`, `getcwd`, `symlink`, `readlink`, `chmod`, `chown` +- **Signals:** `sigaction` (`SA_SIGINFO`), `sigprocmask`, `kill`, `sigreturn` (full trampoline) - **FD flags:** `O_NONBLOCK`, `O_CLOEXEC`, `FD_CLOEXEC` via `fcntl` (`F_GETFD`/`F_SETFD`/`F_GETFL`/`F_SETFL`) - **Shared memory:** `shmget`, `shmat`, `shmdt`, `shmctl` +- **Threads:** `clone`, `gettid`, `set_thread_area` +- **Networking:** `socket`, `bind`, `listen`, `accept`, `connect`, `send`, `recv`, `sendto`, `recvfrom` - Per-process fd table with atomic refcounted file objects - Centralized user-pointer access API (`user_range_ok`, `copy_from_user`, `copy_to_user`) - Error returns use negative errno codes (Linux-style) @@ -53,18 +61,25 @@ AdrOS is a Unix-like, POSIX-compatible, multi-architecture operating system deve - **Canonical + raw mode** — `ICANON` clearable via `TCSETS` - **Signal characters** — Ctrl+C→SIGINT, Ctrl+Z→SIGTSTP, Ctrl+D→EOF, Ctrl+\\→SIGQUIT - **Job control** — `SIGTTIN`/`SIGTTOU` enforcement for background process groups -- **PTY** — `/dev/ptmx` + `/dev/pts/0` with non-blocking I/O +- **PTY** — `/dev/ptmx` + `/dev/pts/N` (up to 8 dynamic pairs) with non-blocking I/O - **Window size** — `TIOCGWINSZ`/`TIOCSWINSZ` -- **termios** — `TCGETS`, `TCSETS`, `TIOCGPGRP`, `TIOCSPGRP` +- **termios** — `TCGETS`, `TCSETS`, `TIOCGPGRP`, `TIOCSPGRP`, `VMIN`/`VTIME` +- **Wait queues** — generic `waitqueue_t` abstraction for blocking I/O -### Filesystems (6 types) +### Filesystems (7 types) - **tmpfs** — in-memory filesystem -- **devfs** — `/dev/null`, `/dev/tty`, `/dev/ptmx`, `/dev/pts/0` +- **devfs** — `/dev/null`, `/dev/zero`, `/dev/random`, `/dev/urandom`, `/dev/console`, `/dev/tty`, `/dev/ptmx`, `/dev/pts/N` - **overlayfs** — copy-up semantics -- **diskfs** — hierarchical inode-based on-disk filesystem at `/disk` +- **diskfs** — hierarchical inode-based on-disk filesystem at `/disk` with symlinks - **persistfs** — minimal persistence at `/persist` -- **procfs** — `/proc/meminfo` -- Generic `readdir`/`getdents` across all VFS types +- **procfs** — `/proc/meminfo` + per-process `/proc/[pid]/status`, `/proc/[pid]/maps` +- Generic `readdir`/`getdents` across all VFS types; symlink following in path resolution + +### Networking +- **E1000 NIC** — Intel 82540EM driver (MMIO, IRQ via IOAPIC) +- **lwIP TCP/IP stack** — IPv4, static IP (10.0.2.15 via QEMU user-net) +- **Socket API** — `socket`/`bind`/`listen`/`accept`/`connect`/`send`/`recv`/`sendto`/`recvfrom` +- **Protocols** — TCP (`SOCK_STREAM`) + UDP (`SOCK_DGRAM`) ### Drivers & Hardware - **PCI** — full bus/slot/func enumeration with BAR + IRQ @@ -74,15 +89,31 @@ AdrOS is a Unix-like, POSIX-compatible, multi-architecture operating system deve - **ACPI** — MADT parsing for CPU topology and IOAPIC discovery - **VBE framebuffer** — maps LFB, pixel drawing, font rendering - **UART**, **VGA text**, **PS/2 keyboard**, **PIT timer**, **LAPIC timer** +- **E1000 NIC** — Intel 82540EM Ethernet controller ### Userland -- **ulibc** — `printf`, `malloc`/`free`/`calloc`/`realloc`, `string.h`, `unistd.h`, `errno.h` -- **ELF32 loader** — secure with W^X enforcement, rejects kernel-range vaddrs +- **ulibc** — `printf`, `malloc`/`free`/`calloc`/`realloc`, `string.h`, `unistd.h`, `errno.h`, `pthread.h` +- **ELF32 loader** — secure with W^X; supports `ET_EXEC` + `ET_DYN` + `PT_INTERP` (dynamic linking) +- **Shell** — `/bin/sh` (POSIX sh-compatible with builtins, pipes, redirects) +- **Core utilities** — `/bin/cat`, `/bin/ls`, `/bin/mkdir`, `/bin/rm`, `/bin/echo` - `/bin/init.elf` — comprehensive smoke test suite -- `/bin/echo.elf` — argv/envp test + +### Dynamic Linking (infrastructure) +- **Kernel-side** — `PT_INTERP` detection, interpreter loading at `0x40000000`, `ET_DYN` support +- **ELF types** — `Elf32_Dyn`, `Elf32_Rel`, `Elf32_Sym`, auxiliary vector (`AT_PHDR`, `AT_ENTRY`, `AT_BASE`) +- **Relocation types** — `R_386_RELATIVE`, `R_386_32`, `R_386_GLOB_DAT`, `R_386_JMP_SLOT` +- Userspace `ld.so` not yet implemented (kernel infrastructure ready) + +### Threads +- **`clone` syscall** — `CLONE_VM`, `CLONE_FILES`, `CLONE_SIGHAND`, `CLONE_THREAD`, `CLONE_SETTLS` +- **TLS via GDT** — `set_thread_area` sets GS-based TLS segment (GDT entry 22, ring 3) +- **`gettid`** — per-thread unique ID +- **ulibc `pthread.h`** — `pthread_create`, `pthread_join`, `pthread_exit`, `pthread_self` +- Thread-group IDs (tgid) for POSIX `getpid()` semantics ### Security - **SMEP** enabled (prevents kernel executing user-mapped pages) +- **PAE + NX bit** — hardware W^X on data segments - **user_range_ok** hardened (rejects kernel addresses) - **sigreturn eflags** sanitized (clears IOPL, ensures IF) - **Atomic file refcounts** (`__sync_*` builtins) @@ -110,34 +141,28 @@ QEMU debug helpers: See [POSIX_ROADMAP.md](docs/POSIX_ROADMAP.md) for a detailed checklist. -### Near-term (unlock interactive use) -- **Shell** (`sh`-compatible) — all required syscalls are implemented -- **Core utilities** — `ls`, `cat`, `echo`, `mkdir`, `rm` -- **`/dev/zero`**, **`/dev/random`** — simple device nodes -- **Multiple PTY pairs** — currently only 1 - -### Medium-term (POSIX compliance) -- **`mmap`/`munmap`** — memory-mapped files -- **Permissions** — `uid`/`gid`/mode, `chmod`, `chown`, `access`, `umask` -- **`/proc` per-process** — `/proc/[pid]/status`, `/proc/[pid]/maps` -- **Hard/symbolic links** — `link`, `symlink`, `readlink` -- **VMIN/VTIME** — termios non-canonical timing - -### Long-term (full Unix experience) -- **Networking** — socket API, TCP/IP stack -- **Threads** — `clone`/`pthread` -- **PAE + NX bit** — hardware W^X -- **Dynamic linking** — `ld.so` +All 15 planned implementation tasks are complete. Remaining future work: + +### Future enhancements +- **Userspace `ld.so`** — full dynamic linker with relocation processing +- **Shared libraries (.so)** — `dlopen`/`dlsym`/`dlclose` +- **Futex** — efficient thread synchronization primitive +- **Multi-arch bring-up** — ARM/RISC-V functional kernels - **ext2/FAT** filesystem support +- **ASLR** — address space layout randomization +- **vDSO** — fast `clock_gettime` without syscall +- **DNS resolver** + `/etc/hosts` +- **RTC driver** — real-time clock ## Directory Structure -- `src/kernel/` — Architecture-independent kernel (VFS, syscalls, scheduler, tmpfs, diskfs, devfs, overlayfs, PTY, TTY, shm, signals) -- `src/arch/x86/` — x86-specific (boot, VMM, IDT, LAPIC, IOAPIC, SMP, ACPI, CPUID, SYSENTER) -- `src/hal/x86/` — HAL x86 (CPU, keyboard, timer, UART, PCI, ATA PIO/DMA) +- `src/kernel/` — Architecture-independent kernel (VFS, syscalls, scheduler, tmpfs, diskfs, devfs, overlayfs, PTY, TTY, shm, signals, networking, threads) +- `src/arch/x86/` — x86-specific (boot, VMM, IDT, LAPIC, IOAPIC, SMP, ACPI, CPUID, SYSENTER, ELF loader) +- `src/hal/x86/` — HAL x86 (CPU, keyboard, timer, UART, PCI, ATA PIO/DMA, E1000 NIC) - `src/drivers/` — Device drivers (VBE, initrd, VGA) - `src/mm/` — Memory management (PMM, heap, slab) - `include/` — Header files -- `user/` — Userland programs (`init.c`, `echo.c`) -- `user/ulibc/` — Minimal C library (`printf`, `malloc`, `string.h`, `errno.h`) +- `user/` — Userland programs (`init.c`, `echo.c`, `sh.c`, `cat.c`, `ls.c`, `mkdir.c`, `rm.c`) +- `user/ulibc/` — Minimal C library (`printf`, `malloc`, `string.h`, `errno.h`, `pthread.h`) - `tests/` — Host unit tests, smoke tests, GDB scripted checks - `docs/` — Documentation (POSIX roadmap, audit report, supplementary analysis, testing plan) +- `third_party/lwip/` — lwIP TCP/IP stack (vendored) diff --git a/docs/POSIX_ROADMAP.md b/docs/POSIX_ROADMAP.md index 1a2a616..b80a2b7 100644 --- a/docs/POSIX_ROADMAP.md +++ b/docs/POSIX_ROADMAP.md @@ -19,23 +19,23 @@ Notes: |---------|--------|-------| | `open` | [x] | Supports `O_CREAT`, `O_TRUNC`; works on diskfs, devfs, tmpfs, overlayfs | | `openat` | [x] | `AT_FDCWD` supported; other dirfd values return `ENOSYS` | -| `read` | [x] | Files, pipes, TTY, PTY; `O_NONBLOCK` returns `EAGAIN` | -| `write` | [x] | Files, pipes, TTY, PTY; `O_NONBLOCK` returns `EAGAIN` | +| `read` | [x] | Files, pipes, TTY, PTY, sockets; `O_NONBLOCK` returns `EAGAIN` | +| `write` | [x] | Files, pipes, TTY, PTY, sockets; `O_NONBLOCK` returns `EAGAIN` | | `close` | [x] | Refcounted file objects | | `lseek` | [x] | `SEEK_SET`, `SEEK_CUR`, `SEEK_END` | -| `stat` | [x] | Minimal `struct stat` (mode/type/size/inode) | +| `stat` | [x] | `struct stat` with mode/type/size/inode/uid/gid/nlink | | `fstat` | [x] | | | `fstatat` | [x] | `AT_FDCWD` supported | | `dup` | [x] | | | `dup2` | [x] | | -| `dup3` | [x] | Flags parameter (currently only `flags=0` accepted) | +| `dup3` | [x] | Flags parameter | | `pipe` | [x] | In-kernel ring buffer | | `pipe2` | [x] | Supports `O_NONBLOCK` flag | -| `select` | [x] | Minimal (pipes, TTY) | -| `poll` | [x] | Minimal (pipes, TTY, `/dev/null`) | -| `ioctl` | [x] | `TCGETS`, `TCSETS`, `TIOCGPGRP`, `TIOCSPGRP` | -| `fcntl` | [x] | `F_GETFL`, `F_SETFL` (for `O_NONBLOCK`) | -| `getdents` | [x] | Generic across all VFS (diskfs, tmpfs, devfs, overlayfs) | +| `select` | [x] | Pipes, TTY, sockets | +| `poll` | [x] | Pipes, TTY, PTY, `/dev/null`, sockets | +| `ioctl` | [x] | `TCGETS`, `TCSETS`, `TIOCGPGRP`, `TIOCSPGRP`, `TIOCGWINSZ`, `TIOCSWINSZ` | +| `fcntl` | [x] | `F_GETFL`, `F_SETFL`, `F_GETFD`, `F_SETFD` | +| `getdents` | [x] | Generic across all VFS (diskfs, tmpfs, devfs, overlayfs, procfs) | | `pread`/`pwrite` | [ ] | | | `readv`/`writev` | [ ] | | | `truncate`/`ftruncate` | [ ] | | @@ -52,9 +52,11 @@ Notes: | `rename` | [x] | diskfs; handles same-type overwrite | | `chdir` | [x] | Per-process `cwd` | | `getcwd` | [x] | | -| `link` | [ ] | Hard links | -| `symlink` | [ ] | Symbolic links | -| `readlink` | [ ] | | +| `link` | [x] | Hard links (stub — returns `ENOSYS` for cross-fs) | +| `symlink` | [x] | Symbolic links in diskfs | +| `readlink` | [x] | | +| `chmod` | [x] | Set mode bits on VFS nodes | +| `chown` | [x] | Set uid/gid on VFS nodes | | `access` | [ ] | Permission checks | | `umask` | [ ] | | | `realpath` | [ ] | Userland (needs libc) | @@ -64,19 +66,21 @@ Notes: | Syscall | Status | Notes | |---------|--------|-------| | `fork` | [x] | Full COW implemented (`vmm_as_clone_user_cow` + `vmm_handle_cow_fault`) | -| `execve` | [x] | Loads ELF from VFS; argv/envp; `O_CLOEXEC` FDs closed | +| `execve` | [x] | Loads ELF from VFS; argv/envp; `O_CLOEXEC` FDs closed; `PT_INTERP` support | | `exit` / `_exit` | [x] | Closes FDs, marks zombie, notifies parent | | `waitpid` | [x] | `-1` (any child), specific pid, `WNOHANG` | | `getpid` | [x] | | | `getppid` | [x] | | +| `gettid` | [x] | Returns per-thread ID | | `setsid` | [x] | | | `setpgid` | [x] | | | `getpgrp` | [x] | | -| `getuid`/`getgid`/`geteuid`/`getegid` | [ ] | No user/group model yet | +| `getuid`/`getgid` | [x] | Per-process uid/gid | | `setuid`/`setgid` | [ ] | | | `brk`/`sbrk` | [x] | `syscall_brk_impl()` — per-process heap break | -| `mmap`/`munmap` | [ ] | Memory-mapped I/O | -| `clone` | [ ] | Thread creation | +| `mmap`/`munmap` | [x] | Anonymous mappings + shared memory | +| `clone` | [x] | Thread creation with `CLONE_VM`/`CLONE_FILES`/`CLONE_THREAD`/`CLONE_SETTLS` | +| `set_thread_area` | [x] | GDT-based TLS via GS segment (GDT entry 22, ring 3) | | `nanosleep`/`sleep` | [x] | `syscall_nanosleep_impl()` with tick-based sleep | | `clock_gettime` | [x] | `CLOCK_REALTIME` and `CLOCK_MONOTONIC` | | `alarm` | [ ] | | @@ -86,7 +90,7 @@ Notes: | Syscall | Status | Notes | |---------|--------|-------| -| `sigaction` | [x] | Installs handlers; `sa_flags` minimal | +| `sigaction` | [x] | Installs handlers; `SA_SIGINFO` supported | | `sigprocmask` | [x] | Block/unblock signals | | `kill` | [x] | Send signal to process/group | | `sigreturn` | [x] | Trampoline-based return from signal handlers | @@ -95,16 +99,16 @@ Notes: | `sigsuspend` | [ ] | | | `sigqueue` | [ ] | | | `sigaltstack` | [ ] | Alternate signal stack | -| Signal defaults | [x] | `SIGKILL`/`SIGSEGV`/`SIGUSR1`/`SIGINT`/`SIGTSTP`/`SIGTTOU`/`SIGTTIN` handled | +| Signal defaults | [x] | `SIGKILL`/`SIGSEGV`/`SIGUSR1`/`SIGINT`/`SIGTSTP`/`SIGTTOU`/`SIGTTIN`/`SIGQUIT` handled | ## 5. File Descriptor Layer | Feature | Status | Notes | |---------|--------|-------| | Per-process fd table | [x] | Up to `PROCESS_MAX_FILES` entries | -| Refcounted file objects | [x] | Shared across `dup`/`fork` | +| Refcounted file objects | [x] | Shared across `dup`/`fork`/`clone` with atomic refcounts | | File offset tracking | [x] | | -| `O_NONBLOCK` | [x] | Pipes, TTY, PTY via `fcntl` or `pipe2` | +| `O_NONBLOCK` | [x] | Pipes, TTY, PTY, sockets via `fcntl` or `pipe2` | | `O_CLOEXEC` | [x] | Close-on-exec via `pipe2`, `open` flags | | `O_APPEND` | [ ] | | | `FD_CLOEXEC` via `fcntl` | [x] | `F_GETFD`/`F_SETFD` implemented; `execve` closes marked FDs | @@ -115,18 +119,18 @@ Notes: | Feature | Status | Notes | |---------|--------|-------| | VFS mount table | [x] | Up to 8 mounts | -| `vfs_lookup` path resolution | [x] | Absolute + relative (via `cwd`) | +| `vfs_lookup` path resolution | [x] | Absolute + relative (via `cwd`); follows symlinks | | `fs_node_t` with `read`/`write`/`finddir`/`readdir` | [x] | | | `struct vfs_dirent` (generic) | [x] | Unified format across all FS | | **tmpfs** | [x] | In-memory; dirs + files; `readdir` | | **overlayfs** | [x] | Copy-up; `readdir` delegates to upper/lower | -| **devfs** | [x] | `/dev/null`, `/dev/tty`, `/dev/ptmx`, `/dev/pts/0`; `readdir` | -| **diskfs** (on-disk) | [x] | Hierarchical inodes; `open`/`read`/`write`/`stat`/`mkdir`/`unlink`/`rmdir`/`rename`/`getdents` | +| **devfs** | [x] | `/dev/null`, `/dev/zero`, `/dev/random`, `/dev/urandom`, `/dev/console`, `/dev/tty`, `/dev/ptmx`, `/dev/pts/N` | +| **diskfs** (on-disk) | [x] | Hierarchical inodes; full POSIX ops; symlinks | | **persistfs** | [x] | Minimal persistence at `/persist` | -| **procfs** | [~] | `/proc/meminfo` exists; no per-process `/proc/[pid]` | -| Permissions (`uid`/`gid`/mode) | [ ] | No permission model | -| Hard links | [ ] | | -| Symbolic links | [ ] | | +| **procfs** | [x] | `/proc/meminfo` + per-process `/proc/[pid]/status`, `/proc/[pid]/maps` | +| Permissions (`uid`/`gid`/mode) | [x] | `chmod`, `chown`; mode bits stored in VFS nodes | +| Hard links | [~] | `link` syscall exists (stub for cross-fs) | +| Symbolic links | [x] | `symlink`, `readlink`; followed by VFS lookup | | ext2 / FAT support | [ ] | | ## 7. TTY / PTY @@ -135,17 +139,17 @@ Notes: |---------|--------|-------| | Canonical input (line-buffered) | [x] | | | Echo + backspace | [x] | | -| Blocking reads + wait queue | [x] | | -| `TCGETS`/`TCSETS` | [x] | Minimal termios | +| Blocking reads + wait queue | [x] | Generic `waitqueue_t` abstraction | +| `TCGETS`/`TCSETS` | [x] | Full termios with `c_cc[NCCS]` | | `TIOCGPGRP`/`TIOCSPGRP` | [x] | | | Job control (`SIGTTIN`/`SIGTTOU`) | [x] | Background pgrp enforcement | | `isatty` (via `ioctl TCGETS`) | [x] | | -| PTY master/slave | [x] | `/dev/ptmx` + `/dev/pts/0` | +| PTY master/slave | [x] | `/dev/ptmx` + `/dev/pts/N` (dynamic, up to 8 pairs) | | Non-blocking PTY I/O | [x] | | | Raw mode (non-canonical) | [x] | Clear `ICANON` via `TCSETS` | -| VMIN/VTIME | [ ] | | +| VMIN/VTIME | [x] | Non-canonical timing with `c_cc[VMIN]`/`c_cc[VTIME]` | | Signal characters (Ctrl+C → `SIGINT`, etc.) | [x] | Ctrl+C→SIGINT, Ctrl+Z→SIGTSTP, Ctrl+D→EOF, Ctrl+\\→SIGQUIT | -| Multiple PTY pairs | [ ] | Only 1 pair currently | +| Multiple PTY pairs | [x] | Up to `PTY_MAX_PAIRS=8`, dynamic `/dev/pts/N` | | Window size (`TIOCGWINSZ`/`TIOCSWINSZ`) | [x] | Get/set `struct winsize` | ## 8. Memory Management @@ -153,17 +157,17 @@ Notes: | Feature | Status | Notes | |---------|--------|-------| | PMM (bitmap allocator) | [x] | Spinlock-protected, frame refcounting | -| VMM (x86 paging) | [x] | Higher-half kernel, recursive page directory | -| Per-process address spaces | [x] | Page directory per process | +| VMM (x86 PAE paging) | [x] | Higher-half kernel, recursive page directory, PAE mode | +| Per-process address spaces | [x] | PDPT + 4 PDs per process | | Kernel heap (`kmalloc`/`kfree`) | [x] | Dynamic growth up to 64MB | | Slab allocator | [x] | `slab_cache_t` with free-list-in-place | | W^X for user ELFs | [x] | Text segments read-only after load | | SMEP | [x] | Enabled in CR4 if CPU supports | | `brk`/`sbrk` | [x] | Per-process heap break | -| `mmap`/`munmap` | [ ] | | +| `mmap`/`munmap` | [x] | Anonymous mappings, shared memory backing | | Shared memory (`shmget`/`shmat`/`shmdt`) | [x] | System V IPC style | | Copy-on-write (COW) fork | [x] | PTE bit 9 as CoW marker + page fault handler | -| PAE + NX bit | [ ] | | +| PAE + NX bit | [x] | PAE paging with NX (bit 63) on data segments | | Guard pages | [ ] | | | ASLR | [ ] | | @@ -185,66 +189,101 @@ Notes: | CPUID feature detection | [x] | Leaf 0/1/7/extended; SMEP/SMAP detection | | VBE framebuffer | [x] | Maps LFB, pixel drawing, font rendering | | SYSENTER fast syscall | [x] | MSR setup + handler | +| E1000 NIC (Intel 82540EM) | [x] | MMIO-based, IRQ-driven, lwIP integration | | RTC (real-time clock) | [ ] | | -| Network (e1000/virtio-net) | [ ] | | | Virtio-blk | [ ] | | -## 10. Userland +## 10. Networking | Feature | Status | Notes | |---------|--------|-------| -| ELF32 loader | [x] | Secure with W^X enforcement | -| `/bin/init.elf` (smoke tests) | [x] | Comprehensive test suite | -| `/bin/echo.elf` | [x] | Minimal argv/envp test | -| Minimal libc (ulibc) | [x] | `printf`, `malloc`/`free`/`calloc`/`realloc`, `string.h`, `unistd.h`, `errno.h` | -| Shell (`sh`) | [ ] | | -| Core utilities (`ls`, `cat`, `cp`, `mv`, `rm`, `mkdir`) | [ ] | | -| Dynamic linking | [ ] | | -| `$PATH` search in `execve` | [ ] | | +| E1000 NIC driver | [x] | Intel 82540EM, MMIO, IRQ 11 via IOAPIC | +| lwIP TCP/IP stack | [x] | NO_SYS=1, IPv4, static IP (10.0.2.15) | +| `socket` | [x] | `AF_INET`, `SOCK_STREAM` (TCP), `SOCK_DGRAM` (UDP) | +| `bind`/`listen`/`accept` | [x] | TCP server support | +| `connect`/`send`/`recv` | [x] | TCP client support | +| `sendto`/`recvfrom` | [x] | UDP support | +| DNS resolver | [ ] | | +| `/etc/hosts` | [ ] | | +| `getaddrinfo` | [ ] | Userland (needs libc) | -## 11. Networking (future) +## 11. Threads & TLS | Feature | Status | Notes | |---------|--------|-------| -| `socket` | [ ] | | -| `bind`/`listen`/`accept` | [ ] | | -| `connect`/`send`/`recv` | [ ] | | -| TCP/IP stack | [ ] | | -| UDP | [ ] | | -| DNS resolver | [ ] | | -| `/etc/hosts` | [ ] | | -| `getaddrinfo` | [ ] | Userland (needs libc) | +| `clone` syscall | [x] | `CLONE_VM`, `CLONE_FILES`, `CLONE_THREAD`, `CLONE_SETTLS`, `CLONE_SIGHAND` | +| `gettid` syscall | [x] | Returns per-thread unique ID | +| `set_thread_area` | [x] | GDT entry 22, ring 3 data segment for user TLS via GS | +| Thread-group ID (tgid) | [x] | `getpid()` returns tgid for POSIX compliance | +| Shared address space | [x] | Threads share addr_space; thread reap doesn't destroy it | +| `CLONE_PARENT_SETTID` | [x] | Writes child tid to parent address | +| `CLONE_CHILD_CLEARTID` | [x] | Stores address for futex-wake on thread exit | +| ulibc `pthread.h` | [x] | `pthread_create`, `pthread_join`, `pthread_exit`, `pthread_self` | +| Per-thread errno | [x] | Via `set_thread_area` + TLS | +| Futex | [ ] | Required for efficient pthread_join/mutex | + +## 12. Dynamic Linking + +| Feature | Status | Notes | +|---------|--------|-------| +| `ET_DYN` ELF support | [x] | Kernel ELF loader accepts position-independent executables | +| `PT_INTERP` detection | [x] | Kernel reads interpreter path from ELF | +| Interpreter loading | [x] | `elf32_load_interp` loads ld.so at `INTERP_BASE=0x40000000` | +| ELF auxiliary vector types | [x] | `AT_PHDR`, `AT_PHENT`, `AT_PHNUM`, `AT_ENTRY`, `AT_BASE`, `AT_PAGESZ` defined | +| ELF relocation types | [x] | `R_386_RELATIVE`, `R_386_32`, `R_386_GLOB_DAT`, `R_386_JMP_SLOT` defined | +| `Elf32_Dyn`/`Elf32_Rel`/`Elf32_Sym` | [x] | Full dynamic section structures in `elf.h` | +| Userspace `ld.so` | [ ] | Stub; full relocation processing not yet implemented | +| Shared libraries (.so) | [ ] | Requires `ld.so` + `dlopen`/`dlsym` | + +## 13. Userland + +| Feature | Status | Notes | +|---------|--------|-------| +| ELF32 loader | [x] | Secure with W^X; supports `ET_EXEC` + `ET_DYN` + `PT_INTERP` | +| `/bin/init.elf` (smoke tests) | [x] | Comprehensive test suite (19+ checks) | +| `/bin/echo` | [x] | argv/envp test | +| `/bin/sh` | [x] | POSIX sh-compatible shell; builtins, pipes, redirects | +| `/bin/cat` | [x] | | +| `/bin/ls` | [x] | Uses `getdents` | +| `/bin/mkdir` | [x] | | +| `/bin/rm` | [x] | | +| Minimal libc (ulibc) | [x] | `printf`, `malloc`/`free`/`calloc`/`realloc`, `string.h`, `unistd.h`, `errno.h`, `pthread.h` | +| `$PATH` search in `execve` | [ ] | Shell does VFS lookup directly | --- -## Priority Roadmap (next steps) - -### Near-term (unlock a usable shell) -1. ~~Minimal libc~~ ✅ ulibc implemented (`printf`, `malloc`, `string.h`, `unistd.h`, `errno.h`) -2. **Shell** — `sh`-compatible; all required syscalls are implemented -3. **Core utilities** — `ls` (uses `getdents`), `cat`, `echo`, `mkdir`, `rm` -4. ~~Signal characters~~ ✅ Ctrl+C→SIGINT, Ctrl+Z→SIGTSTP, Ctrl+D→EOF -5. ~~Raw TTY mode~~ ✅ ICANON clearable via TCSETS -6. **`/dev/zero`** + **`/dev/random`** — simple device nodes -7. **Multiple PTY pairs** — currently only 1 - -### Medium-term (real POSIX compliance) -8. ~~`brk`/`sbrk`~~ ✅ Implemented -9. **`mmap`/`munmap`** — memory-mapped files, shared memory -10. **Permissions** — `uid`/`gid`, mode bits, `chmod`, `chown`, `access`, `umask` -11. ~~`O_CLOEXEC`~~ ✅ Implemented -12. **`/proc` per-process** — `/proc/[pid]/status`, `/proc/[pid]/maps` -13. **Hard/symbolic links** — `link`, `symlink`, `readlink` -14. **VMIN/VTIME** — termios non-canonical timing -15. **Generic wait queue abstraction** — replace ad-hoc blocking - -### Long-term (full Unix experience) -16. **Networking** — socket API, TCP/IP stack -17. **Multi-arch bring-up** — ARM/RISC-V functional kernels -18. ~~COW fork~~ ✅ Implemented -19. **Threads** (`clone`/`pthread`) -20. **Dynamic linking** (`ld.so`) -21. **ext2/FAT** filesystem support -22. **PAE + NX bit** — hardware W^X -23. **Per-thread errno** (needs TLS) -24. **vDSO** — fast `clock_gettime` without syscall +## Priority Roadmap (remaining work) + +### All 15 planned features are now implemented ✅ + +1. ~~`/dev/zero`, `/dev/random`, `/dev/urandom`, `/dev/console`~~ ✅ +2. ~~Multiple PTY pairs (dynamic `/dev/pts/N`)~~ ✅ +3. ~~VMIN/VTIME termios~~ ✅ +4. ~~Shell (`sh`-compatible)~~ ✅ +5. ~~Core utilities (`cat`, `ls`, `mkdir`, `rm`)~~ ✅ +6. ~~Generic wait queue abstraction~~ ✅ +7. ~~`/proc` per-process (`/proc/[pid]/status`, `/proc/[pid]/maps`)~~ ✅ +8. ~~Permissions (`chmod`, `chown`, `getuid`, `getgid`)~~ ✅ +9. ~~Symbolic links (`symlink`, `readlink`)~~ ✅ +10. ~~PAE + NX bit~~ ✅ +11. ~~Per-thread errno + `set_thread_area` stub~~ ✅ +12. ~~Networking (E1000 + lwIP + socket syscalls)~~ ✅ +13. ~~Socket syscalls (`socket`/`bind`/`listen`/`accept`/`connect`/`send`/`recv`/`sendto`/`recvfrom`)~~ ✅ +14. ~~Threads (`clone`/`pthread`)~~ ✅ +15. ~~Dynamic linking infrastructure (`PT_INTERP`, `ET_DYN`, ELF relocation types)~~ ✅ + +### Future enhancements (beyond the 15 planned tasks) +- **Userspace `ld.so`** — full dynamic linker with relocation processing +- **Shared libraries (.so)** — `dlopen`/`dlsym`/`dlclose` +- **Futex** — efficient thread synchronization primitive +- **Multi-arch bring-up** — ARM/RISC-V functional kernels +- **ext2/FAT** filesystem support +- **ASLR** — address space layout randomization +- **vDSO** — fast `clock_gettime` without syscall +- **`O_APPEND`** — append mode for file writes +- **File locking** — `flock`/`fcntl` advisory locks +- **`pread`/`pwrite`/`readv`/`writev`** — scatter/gather I/O +- **`sigaltstack`** — alternate signal stack +- **DNS resolver** + `/etc/hosts` +- **RTC driver** — real-time clock for wall-clock time +- **`alarm`/`setitimer`** — timer signals