What Happens When a Process is Launched in Windows: Key Components, DLLs, APIs, and Drivers Involved

When a process is launched in Windows, a series of complex operations involving multiple components, APIs, DLLs, and drivers occurs to ensure the program is loaded and executed properly. Here’s a detailed breakdown:

1. Process Creation Overview

When you launch a process (e.g., double-clicking an .exe file), the following high-level steps take place:

1. User Mode:
   • The request to create a process is initiated by a user-mode program (e.g., Windows Explorer or a command line).
   • API calls like CreateProcess or ShellExecute are invoked, depending on how the process is started.
2. Kernel Mode:
   • The request is handed over to the Windows kernel, which handles low-level management like memory allocation, thread creation, and process scheduling.
   • The Windows NT Kernel (ntoskrnl.exe), along with specific drivers, is responsible for setting up the environment for the new process.

2. Detailed Breakdown of Process Creation

Step 1: User Mode – API Calls (CreateProcess)

The process creation begins with a user-mode API call:

• The most common API for creating a new process is CreateProcess (part of kernel32.dll).
• CreateProcess is a wrapper for lower-level system calls, responsible for creating the process and the primary thread.

Other involved DLLs:

• kernel32.dll: Provides the core Windows API functions for process creation.
• ntdll.dll: Contains the native Windows API functions that interact with the Windows kernel.
• advapi32.dll: Used for processes launched with additional security requirements (e.g., for impersonation or privilege setting).

Step 2: Kernel Transition (NtCreateProcess and NtCreateThread)

Once the CreateProcess API is called, it invokes lower-level native system calls in ntdll.dll. These native calls are critical in transitioning from user-mode to kernel-mode to facilitate process creation.

• NtCreateProcess and NtCreateThread are called, which communicate with the Windows kernel.

Step 3: Kernel Mode Operations (Process Manager)

After receiving the process creation request, the Windows Kernel (ntoskrnl.exe) and Process Manager are involved. The kernel carries out several key operations:

1. Allocation of Process Object:
   • The Process Manager creates an internal process object, which represents the new process. This object includes details like the process ID (PID), handle table, and security descriptor.
2. Virtual Memory Setup:
   • The kernel sets up a virtual address space for the new process. It creates the appropriate page tables for memory management.
3. Thread Creation:
   • The primary thread for the process is created using the system call NtCreateThread.
   • The thread is placed in a ready state but doesn’t run yet.
4. Handle and Security Setup:
   • The process’s security context (user permissions, tokens) is set up using SeAssignSecurity, and system resources (like file handles) are inherited from the parent process.

Step 4: Image Loading (ntdll.dll, psapi.dll)

Once the process object is created, the Image Loader loads the executable file into memory.

1. Executable File Loading:
   • The Windows PE Loader (Part of ntoskrnl.exe) loads the executable (EXE) into the process’s virtual address space. The Portable Executable (PE) format is used to define the structure of the executable.
   • It maps sections of the EXE file into memory (code, data, etc.).
2. Loading of Required DLLs:
   • The loader loads necessary DLLs (Dynamic Link Libraries) that the program depends on. This includes the Windows core libraries:
     • kernel32.dll: Provides APIs for memory management, file handling, and process control.
     • ntdll.dll: Provides the interface to the Windows kernel.
     • user32.dll: If the process is GUI-based, this is used for user interface elements.
     • gdi32.dll: For graphics-related functions in GUI applications.
     • Other Application-Specific DLLs: Any other DLLs specified by the application.
3. Library Linking and Base Address Fixups:
   • The loader resolves dynamic linking by finding the addresses of functions in each loaded DLL and modifying the executable’s memory to call the correct functions.
   • Relocation of code may occur if libraries need to be loaded at different base addresses than expected.

Step 5: Execution of the Process

Once all required components are loaded and the process environment is set up:

• The primary thread’s instruction pointer is set to the entry point of the executable (the start function defined in the PE header).
• The process scheduler places the thread into the runnable state, and the process starts execution.

3. Involved Drivers and Components

Windows uses several low-level components and drivers during the process launch:

File System Drivers (e.g., ntfs.sys, fltMgr.sys):

• The file system drivers handle reading the executable and its dependencies (DLLs) from disk.
• Filter drivers (such as antivirus drivers) may intercept the executable loading to scan for malicious code before execution.

Memory Manager (Mm.sys):

• The Memory Manager is responsible for allocating memory for the process. It loads the executable file into the process’s virtual memory space.

I/O Manager (I/O subsystem):

• The I/O Manager handles any input/output requests made by the process during startup. This includes file access, inter-process communication (IPC), and interaction with hardware devices.

Security Subsystem (lsass.exe, sspicli.dll):

• The Local Security Authority (LSASS) enforces security policies. It ensures that the process is running under the correct user security context (based on access tokens).
• If the process requires elevated privileges (e.g., administrator), it prompts the user or verifies credentials.

4. Other APIs and System Calls

Several APIs and system calls are used throughout the process creation cycle:

• RtlCreateUserProcess: A lower-level function used by CreateProcess to create a new process object.
• ZwCreateProcess: A kernel-mode system call that is part of process creation.
• PsCreateSystemThread: A kernel-mode function responsible for creating the initial system thread within a process.

5. Execution Context Initialization

Once the process starts executing, it initializes several components:

1. Environment Variables:
   • The new process inherits the parent process’s environment variables or initializes its own.
2. Loader Lock:
   • The loader uses a loader lock to ensure that the process only accesses shared libraries in a thread-safe way.
3. Startup Code Execution:
   • After initialization, the C Runtime (CRT) startup code (_start function in C programs) is invoked, which eventually calls the main function.

Conclusion

When a process is launched in Windows, it involves several key components and operations, including user-mode API calls like CreateProcess, kernel-mode interactions with the process manager, memory manager, and I/O manager, and the loading of the executable and its dependencies by the PE loader. Kernel drivers (like ntfs.sys, fltMgr.sys) and core system DLLs (kernel32.dll, ntdll.dll) are also heavily involved in setting up the process environment and security context.