OS Basics — Processes, Threads, Memory, FDs

Why this session exists

A program can be alive yet unable to obtain a descriptor, runnable yet unable to obtain CPU, or within its process RSS expectation yet exceed a container memory limit. Diagnose the resource and ownership boundary before changing the application. This lesson develops processes, threads, virtual memory and file descriptors, then interprets ps, top, htop, lsof and strace evidence. There is no promise to diagnose every incident in five minutes.

Prerequisites

Use a Linux shell, understand basic Python and distinguish your own process from a service managed by another owner. Commands that attach debuggers, inspect other users' processes or alter host limits require authorization. The executed exercise here is a bounded in-memory model: no process exhaustion, memory pressure, server or network probe.

The office-manager model: ownership and sharing

Imagine CPU cores as desks, processes as employees and threads as collaborators who share an office's documents. The scheduler chooses runnable tasks, not merely the number of named applications. A thread blocked on I/O is different from one waiting in a run queue. More concurrency may overlap waiting; it cannot manufacture CPU cores or bypass a cgroup CPU quota.

A process normally has a virtual address space, descriptor table and credentials. Threads share the address space and open descriptors but retain their own execution state, stack and thread-local storage. Shared memory does not mean every variable is safely visible without synchronization: language memory models, data races and locks still matter. Separate processes can deliberately share mappings and inherited resources; isolation is not a claim that they can never affect each other.

On Linux the scheduler operates on tasks, represented by task_struct. Sharing is controlled by the creation mechanism and flags; fork, clone, clone3 and library implementations are not interchangeable promises across every architecture or libc. exec replaces a process image; it does not create a new process by itself.

Process versus thread: select by failure and communication needs

For a shared-cache HTTP service, threads avoid copying the cache and make communication cheap, but a memory-corruption crash can terminate the whole process. For independent CPU workers, processes provide stronger address-space isolation and let failures be restarted separately, at the cost of serialization/IPC and duplicated state. A process crash can still disrupt its dependents through locks, queues or shared services.

Python's usual GIL-enabled CPython build limits simultaneous Python bytecode execution within a process; native extensions can release the GIL and free-threaded builds have a different contract. Do not infer CPU parallelism merely from seeing four kernel threads. Measure workload throughput and the runtime actually deployed. There is no universal “threads take 10 microseconds, fork takes one millisecond” cost.

How the model evolved

The useful lineage is the interface, not an unreliable release-date list: Unix exposed processes and descriptors; POSIX specified portable process/thread interfaces; Linux added namespace and cgroup controls; readiness APIs such as epoll and asynchronous interfaces such as io_uring support different I/O styles. Modern implementations change internal data structures. The memory-mapping diagram below intentionally does not assert a linked-list implementation for all kernels, nor call Unix the first multi-user OS.

Kernel state and virtual memory

A virtual address identifies a location in a process address space. Mappings define valid ranges and permissions; page tables translate virtual pages to physical frames. The CPU normally uses a TLB to cache translations, so it does not walk the full page table on every load. Page size and usable address width depend on architecture and configuration: 4 KiB and 48-bit addressing are common examples, not universal constants.

A page fault can mean a missing resident page, first-touch zero-fill, copy-on-write, or an access/permission problem. A minor fault can be resolved without disk I/O; a major fault requires I/O. An invalid access may deliver a signal rather than resume successfully. “Page fault” is not synonymous with “swap read” or “bug.”

A fault, a wake-up and a scheduling decision are different events

Trace one thread through a read: running → blocked waiting for I/O → runnable after completion → running when selected. A different thread may run while it sleeps. A nonblocking read that returns immediately can enter and leave the kernel without selecting a different task. Conversely, timer preemption can switch tasks without an explicit application syscall. Count mode transitions and scheduling switches separately.

Linux sched(7) defines policies for runnable threads; it does not promise one fixed round-robin quantum for every application. Normal-policy nice values influence relative preference, whereas real-time policies have different rules. CPU affinity restricts eligible CPUs, and a cgroup quota restricts the available CPU-time budget. Neither a lower nice value nor more workers creates additional budget. We do not change priorities, affinity or quotas in this exercise.

For a hypothetical fair-class cgroup with cpu.max set to 50000 100000, no burst allowance and no tighter ancestor limit, the ordinary budget is 50 ms of aggregate CPU time per 100 ms period: approximately half of one CPU over full periods. Two parallel workers can consume that budget in 25 ms of wall time, then be throttled; it is not a promise of smooth half-speed execution. Kernel mode work, contention and scheduler details complicate observed timing. This is quota arithmetic, not a host measurement.

Now separate three memory events. A TLB miss may merely require a hardware page-table walk, with no page fault. First touch of a valid anonymous mapping or a copy-on-write write can fault and be handled without reading disk. A file-backed page absent from memory can need I/O and produce a major fault. Writing a genuinely read-only mapping is not permission to allocate a writable copy: the access can fail. The getrusage(2) counters distinguish faults serviced without I/O from faults requiring I/O; they do not tell you that every major fault is swap.

Worked classification: a child writes a private inherited page: copy-on-write can give it a private frame while the parent keeps the old contents. A child increments an inherited file offset: both descriptor entries still reference the same open-file description. These are different sharing contracts even though both arise after fork. A thread waking from an I/O wait becomes eligible, not guaranteed to run immediately. If an incident has low CPU but rising latency, inspect waits and throttling before adding workers; a context-switch count alone cannot prove the cause.

RSS, VSZ, page cache and OOM

VSZ/VmSize describes virtual address-space size; RSS describes resident pages attributed to the process. A large reserved region with little resident memory can be normal. It does not prove the absence of a virtual-address leak, and rising RSS alone does not prove unreachable-memory leakage: caches, allocator retention and changing workload also raise it. Inspect trends under a controlled workload, maps/smaps when appropriate, and allocator/application evidence.

Linux documents some /proc/PID/status memory counters as approximate/inaccurate; use appropriate detailed accounting when precision matters. Shared pages can appear in multiple RSS totals. PSS apportions shared pages and is useful for comparing processes; container accounting also includes categories beyond a single process RSS. A cgroup can reach memory.max even when the host has available memory. memory.events distinguishes counters such as max, oom and oom_kill; compare their changes, not a single label. An EMFILE failure is not evidence of an OOM kill, and Kubernetes does not universally relabel descriptor exhaustion as OOM.

The page cache means a warm file read can be served from RAM. Benchmark warm and cold conditions intentionally; do not drop host caches on a shared machine just to make a benchmark look rigorous. CPU quotas, memory pressure and cache locality can explain high latency without a leak.

File descriptors: integers versus open-file descriptions

open() normally returns the lowest available nonnegative descriptor. Descriptors 0, 1 and 2 conventionally mean stdin, stdout and stderr, but can be closed or rebound. A descriptor indexes a process table; the referenced open-file description contains state such as an offset and file-status flags. dup creates another descriptor referring to the same open-file description. After fork, the child has its own descriptor table whose entries refer to the same descriptions as the parent's inherited entries. Explicit clone flags can instead share a table.

Closing descriptor 4 removes that table reference; it does not necessarily release the underlying object while a duplicate or child reference remains. Threads sharing a table must coordinate closure, because another thread may otherwise use a recycled descriptor number. FD_CLOEXEC is a descriptor flag; set it atomically during creation where the API supports it to avoid inheritance races.

The slogan “everything is a file” means useful common interfaces, not that every object supports every operation. An ordinary signal handler is not a descriptor; signalfd is a specific descriptor-based mechanism. epoll supports many descriptor types, but adding a regular file can fail with EPERM; readiness of a regular file is not the same as nonblocking disk I/O.

Limits and the two exhaustion errors

RLIMIT_NOFILE is one greater than the maximum descriptor number that may be newly opened, not simply a count of live handles. Sparse/high-numbered descriptors and limits lowered after descriptors were opened require care. It has a soft limit enforced for the process and a hard limit constraining unprivileged soft-limit increases. Linux nr_open constrains the maximum limit value; system-wide open-file accounting is separate. EMFILE indicates the process descriptor limit; ENFILE indicates the system-wide open-file limit for relevant calls. Defaults vary by shell, service manager and container setup. Raising a limit without a memory/admission budget can turn a bounded failure into a larger one.

ulimit -n reports the current shell's limit, not necessarily a running daemon's. /proc/PID/limits inspects the target. lsof -p PID | wc -l counts more than descriptors: headers, mapped files, cwd and other records make it unsuitable as an exact FD count. /proc/PID/fd is the better descriptor inventory, still subject to races and permissions.

Read an incident's evidence before changing limits

This is illustrative output, not a measurement of the host:

PID    NLWP  %CPU  RSS(kB)  VSZ(kB)  STAT
4242   16    195   204800   41943040  Rl
 
/proc/4242/limits:
Max open files            1024                 4096                 files
 
/proc/4242/fd: 1008 entries (sampled while process is running)

195% CPU can represent roughly two CPUs under a tool's aggregate convention; check the display mode and quota before interpreting it. Sixteen threads do not imply sixteen running CPUs. About 200 MiB RSS beside 40 GiB VSZ says reservations dominate virtual size, not that all 40 GiB is resident. With 1008 descriptors against a soft limit of 1024, the immediate descriptor margin is small. Ask whether the population is expected concurrent sockets or grows monotonically after requests finish.

For an authorized target, /proc/PID/task/ contains one subdirectory per Linux thread, named by its kernel thread ID. That count describes tasks, not how many can execute Python bytecode simultaneously. Do not confuse a language-level thread identifier with a portable kernel TID. For top/htop, correlate per-thread CPU, task state, memory and time. Linux load average includes runnable tasks and uninterruptible sleepers; high load with low CPU can suggest I/O waits, but CPU quotas, sampling windows and other causes remain. D state means uninterruptible sleep, not proof of one specific disk fault. Voluntary/involuntary context-switch rates describe behavior, not a standalone diagnosis that there are “too many threads.”

A read-only diagnostic sequence for an authorized target PID is:

# Replace PID with the one authorized target; reference only, not executed here.
cat /proc/PID/limits
ls -l /proc/PID/fd
lsof -p PID
ps -p PID -o pid,ppid,nlwp,stat,pcpu,rss,vsz,comm

Classify descriptor targets, sample over time and follow their ownership through normal and error paths. A pile of CLOSE_WAIT sockets often warrants investigating application closure after peer EOF; a stable population of established long-lived sockets may be legitimate capacity. Compare process limits with system-wide usage before tuning. Context managers and finally blocks fix ownership paths; a higher limit does not fix a leak.

Why syscalls differ from ordinary calls

A syscall crosses a hardware-enforced privilege boundary through an architecture-defined entry path. The kernel must validate user arguments, perform the operation and restore execution state. This often costs more than a tiny user-space call, but exact costs depend on the operation, CPU, mitigations and cache state. A syscall is not necessarily a scheduler context switch, and not every transition flushes every TLB entry.

Buffering amortizes small reads/writes; epoll batches readiness events; mmap exposes file-backed pages but still incurs setup and page-fault work. io_uring can reduce transitions through shared submission/completion rings, with syscall avoidance depending on mode and configuration. It is false that every io_uring submission is automatically syscall-free or that mmap eliminates all kernel interaction. Some library services use a vDSO fast path rather than a syscall.

strace: interpret a bounded owned-process trace

strace -f -e trace=openat,network ./myapp launches the named program under tracing and follows children. -p PID attaches to an existing authorized process. Trace output can expose paths, payloads and credentials and tracing can disturb timings, so bound duration and collect only the categories needed.

openat(AT_FDCWD, "/missing/config.json", O_RDONLY) = -1 ENOENT
socket(AF_INET, SOCK_STREAM|SOCK_CLOEXEC, IPPROTO_TCP) = 4
connect(4, ..., 16) = -1 EINPROGRESS
... readiness notification ...
getsockopt(4, SOL_SOCKET, SO_ERROR, [0], ...) = 0
close(4) = 0

ENOENT points to a path/namespace issue. EINPROGRESS on a nonblocking socket is not the final failure: readiness plus SO_ERROR completes the result. With HTTPS, socket reads/writes normally contain TLS records, not the plaintext HTTP GET shown in the old lesson. A network program need not use exactly socket/connect/sendto/recvfrom/close; read/write/sendmsg, asynchronous I/O and reuse change the trace. No live HTTP fetch or security probing is needed to learn this distinction.

Hands-on: bounded descriptor, fork and memory model

This program makes table/reference ownership visible without creating child processes, opening files until exhaustion or allocating 500 MiB. It is a model, not a Linux kernel implementation. It preserves the old exercise's questions—limits, cleanup, inherited FDs, shared offsets, thread sharing and reserved versus resident pages—with deterministic assertions.

from dataclasses import dataclass
 
@dataclass
class Description:
    offset: int = 0
    refs: int = 1
 
class FDTable:
    def __init__(self, limit=8):
        self.limit = limit
        self.entries = {}
 
    def attach(self, description, duplicate=False):
        fd = next((i for i in range(self.limit) if i not in self.entries), None)
        if fd is None:
            raise OSError(24, 'EMFILE model')
        if duplicate:
            description.refs += 1
        self.entries[fd] = description
        return fd
 
    def open(self):
        return self.attach(Description())
 
    def dup(self, fd):
        return self.attach(self.entries[fd], duplicate=True)
 
    def close(self, fd):
        self.entries.pop(fd).refs -= 1
 
    def fork(self):
        child = FDTable(self.limit)
        child.entries = self.entries.copy()
        for description in child.entries.values():
            description.refs += 1
        return child
 
p = FDTable(limit=4)
a = p.open()
b = p.dup(a)
description = p.entries[a]
child = p.fork()
assert description.refs == 4
child.entries[b].offset += 7
assert p.entries[a].offset == 7
p.close(a)
p.close(b)
assert description.refs == 2  # child keeps the underlying object alive
child.close(a)
child.close(b)
assert description.refs == 0
thread_view = p  # model threads sharing the SAME table
fd = p.open()
thread_view.close(fd)
assert fd not in p.entries
opened = []
try:
    for _ in range(5):
        opened.append(p.open())
except OSError as error:
    assert error.errno == 24 and len(opened) == 4
else:
    raise AssertionError('limit not enforced')
finally:
    for fd in opened:
        p.close(fd)
assert not p.entries
reserved_pages = 1024
resident_pages = {0, 7, 8}
page_bytes = 4096  # explicit scenario assumption
assert reserved_pages * page_bytes == 4194304
assert len(resident_pages) * page_bytes == 12288
print('descriptor ownership, cleanup and memory accounting: PASS')

Anatomy, oracle and the original unbounded lab

The fork table is a copy, but its values are shared descriptions. Each descriptor reference contributes to refs; four references remain after duplication plus fork, then two after the parent closes both. The shared offset changes through either reference. The thread view is the table itself, so closure affects the same table. A limit of four permits four descriptors and fails the fifth, and cleanup returns to zero. The arithmetic distinguishes a 4 MiB reservation from three resident 4 KiB pages.

The original fdleak loop and 500 MiB allocation were neither safe defaults nor a proof of a real leak. To reproduce actual OS behavior later, use a disposable subprocess with its own lowered soft limit, a fixed small allocation cap, exact child PID and finally cleanup. Do not lower a service's limits, fork repeatedly, or run an unbounded leak on a shared host. Actual /proc counts can include the temporary descriptor used for enumeration. A forked Python child using os._exit may also lose buffered print output; absence of a printed line is not absence of the child.

Concurrency design and resource controls

A thread-per-connection model is straightforward but spends stack/address-space resources and can accumulate runnable or blocked work. An event loop multiplexes nonblocking I/O efficiently, but a blocking call or long CPU task delays other callbacks. Cooperative switching still has overhead; “no context switches” is too strong. A bounded worker pool can offload work, provided queues, deadlines and admission are also bounded. Under GIL-enabled Python, choose processes/native work where CPU parallelism requires them.

Work a capacity example: assume 1024 descriptor limit, 20 reserved descriptors and four per active operation. The descriptor-only ceiling is floor((1024-20)/4) = 251, not a throughput guarantee. Sharing log handles and pooling DB connections changes the formula; memory, CPU and dependency limits can become binding first. Connection count and request concurrency are also different with keep-alive and multiplexing.

Namespaces change the view of process IDs, mounts, networking and other resources. Cgroups account for and constrain resource use; they do not alone form a complete security boundary. Inspect the actual container/service-manager configuration rather than assuming a Dockerfile or Kubernetes securityContext.sysctls can set every process rlimit. Mount, user and network isolation remain distinct from consumption limits.

Production scenarios: failure evidence, not folklore

• Socket leak after an error path: descriptor count continues rising after traffic falls; owned socket states and allocation/closure paths narrow the suspect code. Fix the missing cleanup, then verify a bounded repeated-error test stabilizes. This is not a claim about a particular CDN incident.
• Container OOM versus EMFILE: record the process errno, termination signal and cgroup event deltas. An OOM killer event and an open() returning errno 24 require different remedies. Raising memory requests does not raise FD limits; raising FD limits does not add memory.
• Periodic latency spikes: correlate request latency with run-queue/CPU throttling, major-fault rates, storage latency, GC and connection-pool queues. Major faults can arise from file-backed I/O, not just swap. /proc/PID/stat has fault counters; /proc/PID/status is not a universal majflt/pgpgin source. Use pidstat -r, appropriate proc data or existing metrics when authorized.
• Many idle connections: budget descriptors, kernel buffers, app objects and event-loop delay. Ten million connections and a fixed RAM figure are not a transferable tuning recipe. Outbound tuple limits and accepted inbound connections are different concerns.

Recall and worked answers

1. Process versus thread

A normal process has its own address space/table; threads share them while retaining per-thread execution state. Explicit shared mappings and inherited open-file descriptions qualify the simplified rule. Synchronization is necessary even when communication is cheap.

2. RSS versus VSZ and alerting

VSZ is mapped/reserved virtual size; RSS is resident memory attributed to the process. Track resident growth together with cgroup usage, pressure, limits and workload. Neither a large VSZ nor rising RSS alone proves a leak or predicts every OOM decision.

3. What is a page fault?

A memory access requires kernel fault handling for absent backing, permissions or copy-on-write. Minor faults need not read disk, major faults do; invalid mappings can signal an error. It may be invisible to program logic while affecting latency.

4. Diagnose EMFILE in four steps

Read the target's /proc/PID/limits, inventory /proc/PID/fd, classify with lsof/symlink targets, and compare samples with active work and completed-request cleanup. ulimit in your shell is not the target limit and lsof | wc -l is not exact. Only then choose a cleanup or capacity correction.

5. Suitable workloads

Threads suit shared-memory concurrent I/O with careful synchronization. Separate processes suit isolated workers and many Python CPU workloads. An event loop suits many mostly waiting connections; a bounded offload pool handles blocking or CPU work without freezing the loop.

6. What does fork cost and inherit?

It creates a child with logically separate address space, copied descriptor table and shared underlying open-file descriptions. Linux copy-on-write avoids eagerly copying every data page, but page-table duplication and task setup still cost work: it is not universally O(1) in process size. In a multithreaded parent only the calling thread is replicated, making safe post-fork actions important.

7. Why are descriptors useful?

They provide common handles for files, pipes and sockets, plus specific event mechanisms. Not every descriptor supports every operation or epoll registration, and a signal handler itself is not a descriptor. Uniformity is a useful interface, not literal identity of all resources.

8. What does the strace command do?

-f -e trace=openat,network ./myapp launches and follows relevant calls and child activity; -p is the attach form. It can reveal missing files and connection errors, but changes timing and can expose sensitive bytes. HTTPS normally hides HTTP plaintext from the socket-level trace.

9. Containers: view versus consumption

Namespaces give isolated views; cgroups control/account resources. A container still shares a host kernel and needs capabilities, filesystem and other security controls. Inspect the target's actual limits instead of assuming a base image changes runtime rlimits.

10. Investigate the thirty-second p95 spike

First align timestamps and compare latency with CPU/run-queue/throttling, faults and storage/pool metrics. Second use an authorized bounded profile during the spike to separate executing from waiting. Third inspect a narrowly filtered owned-process syscall summary if needed, then change one hypothesis and remeasure. GC, rotation and refresh jobs are hypotheses, not a claim that most incidents have one predetermined cause.

Explain-out-loud test

Explain why a child can keep a socket alive after its parent closes it, why a large reservation is not resident use, and how a blocked event loop differs from a CPU-saturated worker pool. Name the measurement that would falsify each diagnosis.

Where this connects next

Networking uses the same descriptor table; load balancers need connection and queue budgets; containers add namespaces/cgroups; observability needs task, memory, descriptor and pressure metrics. Performance and incident response build on the distinction between executing, runnable, waiting and resource-exhausted states.

Sources and execution boundary

Linux man-pages and kernel cgroup-v2 documentation were captured on 2026-10-08; their saved footer/version and raw hashes delimit the review. The 2026-10-10 review rechecked those saved passages and captured scheduler, syscall, getrusage and process-task documentation for the added state-transition/accounting explanations. The deterministic ownership model and arithmetic were executed. No actual leak, fork, debugger attachment, host tuning, global cache drop or network fetch was run as a lesson exercise.

• fork(2)
• open(2)
• getrlimit(2)
• pthreads(7)
• proc_pid_task(5) and syscall(2)
• proc_pid_status(5)
• epoll(7)
• Kernel cgroup v2

Original resource trail and reading purpose

The original session allocated 90 minutes for intuition, visuals, practice, production reasoning and recall. Treat that as optional pacing, not a completion or diagnostic-speed guarantee. Read the mechanisms here before attempting separately authorized service work.

These original companion destinations are retained for discovery, not used as reviewed claim evidence. Video identity, attribution, duration, playback and external reachability have not been verified in this pass; search links are searches, not specific videos. Historical promotional claims and inconsistent labels are not repeated as facts.

• Original video/search destination 1 — kernel orientation.
• Original video/search destination 2 — diagnostic-tool orientation; original title and description conflict.
• Original video/search destination 3 — process/CLI practice.
• man 7 credentials — processes, users, groups — process identity.
• Linux Performance · Brendan Gregg — performance investigation methods.
• Linux Kernel Memory Management — historical memory-management background, not current kernel internals.

For reviewed propositions use the primary sources above and the source-qualified mechanisms, not the optional resource labels. Next: S061 networking. Prerequisite refreshers: S001 environment and S003 command line. Hub: The 6-Month Learning Plan. Navigation indicates a route, not acceptance of another lesson.