Showing posts with label Understanding Windows Drivers. Show all posts
Showing posts with label Understanding Windows Drivers. Show all posts

Monday, May 8, 2017

Windows I/O Model

A prerequisite for understanding this chapter is understanding the Driver and the Device object stack in the Windows Operating System. Please refer to the blog (Device Object and Driver Stack (Part 1).
In this blog, we start with understanding the Windows IO Model. Subsequently, we discuss the I/O system and its interaction with the layered driver structure to service a request. A client creates the request (can be an application, any kernel subsystem like PnP manager or another driver), and the destined Driver processes them. Kernel subsystems structure these requests into packets, aptly called I/O Request Packets (IRPs). Finally, we brief how the IO moves from an application through the system and the device stack to completion.

In the next blog, we will look at the associated code and the structures.

The Windows I/O Model

The Windows I/O model governs how the data flows to and from the peripheral device. The model encompasses more than just data transfers - it operates as a general packet-based mechanism that handles the communication between the clients who issue the request and the device stack. Clients include the applications, kernel subsystems such as the PnP manager, and the drivers.


All I/O requests to drivers are sent as I/O request packets (IRPs). In addition to the data transfer (Read, Write, Control), IRPs are also used to carry Plug and Play requests, power management requests, WMI requests, notifications of change in device status, and queries about device and driver resources.


An application typically accesses the drivers synchronously. From the Driver's perspective, however, Windows I/O is inherently asynchronous. For example, if the read operation takes time, the Driver starts the necessary procedure and informs the I/O manager that the request is pending. In the interim, the Driver is free to process additional requests.

For synchronous operation, the I/O manager waits until the Driver finishes the read operation before returning the data to the application.

For an asynchronous operation, when the Driver informs the I/O manager that the request is pending, the I/O manager informs the application and returns the thread to the application's control. While the request is satisfied, the application can use the thread for other purposes.

Eventually, the device signals the Driver that the read operation is complete. The Driver informs the I/O manager, which notifies the application, and the application can then process the data.

The I/O manager defines a set of standard routines, some required and others optional, that drivers can support. All drivers follow a relatively consistent implementation model, given the differences among the peripheral devices and differing functionality required of bus, function, filter and file system drivers.

I/O Manager

The kernel subsystems section in the blog briefly described the I/O manager: (Windows Driver Fundamentals)

I/O system defines the orderly framework or model within which I/O are delivered to the device drivers. As mentioned earlier, I/Os are represented by an I/O request packet (IRP), travelling from one I/O system component to another.

An IRP is a data structure that contains information describing an I/O request. The I/O manager creates an IRP in the memory to represent an I/O operation, passing a pointer to the IRP to the correct Driver and disposing the packet when the I/O operation is complete.

A driver receives an IRP, performs the operation the IRP specifies and passes the IRP back to the I/O manager, either because the requested I/O operation is complete or because it must pass to another driver for further processing.

In addition to creating and disposing of all the IRPs, the I/O manager supplies code common to different drivers that the drivers can call to carry out their I/O processing. By consolidating common tasks in the I/O manager, individual drivers become simpler and more compact.

I/O manager also manages buffer for the I/O requests, provides timeout support for the drivers and records installable file systems loaded into the operating system.

The uniform modular interface that the Driver presents allows the I/O manager to call any driver without requiring special knowledge of its structure or internal details.

Drivers can also call each other (using an I/O manager) to achieve layered, independent processing of an I/O request.t.

I/O Requests (IRPs)

The following are the most common types of the I/O requests issued from the application:

  • Write Requests: These requests pass data to the Driver that must write to the device.
  • Read Requests: These requests pass a buffer to the Driver to read and fill the data from the device.
  • Device I/O control requests: These requests are used to communicate with drivers for any purposes other than reading or writing data. For some devices, I/O control requests are the most common request type
The standard Windows functions that applications use to issue these I/O requests are WriteFile, ReadFile, and DeviceIoControl respectively.
We will examine the IRPs, including different I/O transfer types, in detail in the upcoming blogs. For now, this general information will suffice for understanding.

How a Device Stack Handles IRPs.

When a client sends an I/O request to a device, the I/O manager packages the request and related information such as data buffers into an IRP. The I/O manager locates the pointer to the appropriate dispatch routine in the stack's top device object, calls the routine, and passes the IRP to the associated Driver for processing.



Let us see the general path that an I/O request follows from an application through the system and the device stack. Once we discuss the IRPs in detail in later blogs, we will discuss in detail the processing of IRPs through the KMDF drivers.

I/O request Flow:

Follow figure 1.
  1. An application issues an I/O request by calling a Windows API function.
  2. The Windows API calls the kernel-mode subsystems, which call the corresponding I/O processing function of the I/O manager.
  3. The I/O manager builds an IRP that describes the I/O request and sends it to the top Driver in the kernel-mode device stack for the target device.
  4. If the IRP is not complete, the kernel-mode Driver sends it to the next lower Driver in the device stack. The driver processes the IRP and, in turn, sends it to the next lower Driver and so on down the device stack until a driver completes the IRP.
  5. Any driver that sends an IRP down the stack can register an I/O completion callback to have the opportunity to process the IRP again after lower drivers have completed it.
  6. Finally, the IRP reaches a driver that completes it.
  7. After the IRP is complete, the I/O manager calls the I/O completion callbacks that drivers set as the request passed down the device stack, in reverse order.
  8. The Driver that completes the IRP doesn't require an I/O completion callback because it already knows that the request is complete and has completed its processing.
  9. The I/O manager retrieves the I/O status from the IRP, translates the status to a Windows error code, returns completion information, and copies data to the requesting application's data buffer.
  10. The I/O manager then returns control to the Windows API.
  11. The Windows API returns the results of the operation to the requesting application.

References

I am following the given resources:

Wednesday, December 14, 2016

Device Objects and Driver Stack

Understanding Device Objects and Driver Stack is of utmost importance for understanding the Windows Driver Architecture. In this Windows Driver tutorial we will try to build our initial understanding of the Device Objects, Driver Objects and Device Stack.

Let us revisit the Figure below given in our previous blog Windows Driver Architecture.

Figure 1: Driver Stack
We mentioned that not all the drivers communicate directly with the driver. For given I/O request, there are several drivers layered as stack that participate in the request.
Driver Stack
Elaborating on the statement above and the figure given, the windows driver model classifies drivers broadly into following three kinds:
  1. Bus Driver - Bottom of the stack and managers the bus where the device is attached. Most bus drivers are provided by Microsoft such as PCI, SCSI, USB, etc. Other bus drivers can be provided by IHVs and OEMs.
  2. Function Driver - Main driver of the device and drives an individual device. It is typically written by the device vendor.
  3. Filter Driver - Optional driver that filters I/O requests for a device, a class of devices or a bus.
The following figure shows the relationship between the bus driver, the function driver and the filter drivers for a device.
Figure 2: Driver Stack
Let us look them in detail:
Bus Driver
A Bus driver drives an individual I/O bus device and provides per-slot functionality that is device independent.
Microsoft provides bus drivers for mist buses, such as PCI, SCSI, USB, etc. Other bus drivers can be provided by IHVs and OEMs. Bus drivers are required drivers => there is one bus driver for each type of bus on a machine.
The primary responsibility of a bus driver are to:
  • Enumerate the device on its bus.
  • Respond to Plug and Play and Power management IRPs.
  • Multiplex access to the bus.
  • Generically administer the devices on the bus.
Bus drivers are essentially function drivers that also enumerate children. During enumeration, a bus driver identifies the devices on its bus and creates device objects for them.
For details please follow:
https://msdn.microsoft.com/en-us/library/windows/hardware/ff540704%28v=vs.85%29.aspx?f=255&MSPPError=-2147217396
Function Driver
Main driver of a device. A function driver is typically written by the device vendor and is required. The PnP manager loads at most one function driver for a device.
A function driver provides the operational interface for its device. Typically the function driver handles reads and writes to the device and manages device power policy.
For details please follow:
https://msdn.microsoft.com/en-us/library/windows/hardware/ff546516%28v=vs.85%29.aspx?f=255&MSPPError=-2147217396
Filter Driver
Filter Driver filters I/O request for a device, a class of device or a bus. These are optional drivers that add value to or modify the behavior of a device.
Bus Filter Drivers:
Typically add value to a bus and are supplied by Microsoft or system OEM. A bus filter driver could for example, implement proprietary enhancements to standard bus hardware.
Lower-Level Filter Driver:
Typically modify the behavior of device hardware.
A lower-level device filter driver monitors and/or modifies I/O requests to a particular device. Typically such filters redefine hardware behavior to match expected specifications.
For example, a lower-level class filter driver for mouse devices could provide acceleration, performing a nonlinear conversion of mouse movement data.
Upper-Level Filter Driver:
An upper-level device filter driver adds value for a particular device. For example, an upper-level device filter driver for a keyboard could enforce additional security checks.
For details please visit:
https://msdn.microsoft.com/en-us/library/windows/hardware/ff545890(v=vs.85).aspx

A good example to follow and understand the Driver stack is mentioned in MSDN:
https://msdn.microsoft.com/en-us/library/windows/hardware/ff565644(v=vs.85).aspx
Now, having established understanding of the Driver Stack, lets discuss Device Objects and how they are related to Driver Stack.
Device Objects and the Driver Stack
During system startup, the PnP Manager starts with the lowest level bus and loads the bus driver. A bus driver identifies the devices on its bus and creates device objects for them. Then, with similar iterative method, it creates device object stack for the device. More of it when we discuss Device Tree but I think we get the idea.
So, each driver has an associated device object to represent the driver's participation in the processing of I/O requests for that device. Thus, like the driver stack, these device object are arranged in a stack called as device stack.
The point to remember is that each device stack represents a device participation and is associated with a single device. A set of drivers or driver stack can service multiple device stack.
Thus, like our drivers, we have three kinds of Device Objects:

  1. Physical Device Object (PDO) - Represents a device on a bus to a bus driver.
  2. Functional Device Object (FDO) - Represents a device to a function driver.
  3. Filter Device Object (Filter DO) - Represents a device to a filter driver.


Example: In the Figure below, we have two device, each with its own device stack - That are serviced by a single set of drivers.

Figure 3: Device Object and Driver Stack
Let us discuss the components given in the figure 3 in detail:
Bus Driver and Physical device Object
The bottom of the stack is a Physical Device Object (PDO), which is associated with a bus driver.
A bus driver creates a PDO for each device that it enumerates on the bus and requests resources for those device. The PnP manager uses that information to assign resources to each device. Each device has its own PDO.
A PDO represents the device to the bus driver as well as to other kernel-mode system components such as power manager, the PnP manager, and the I/O manager.
Function Driver and Functional Device Object
The core of the device stack is the functional device object (FDO)which is associated with a function driver.
The function driver creates an FDO and attaches it to the device stack. It translates the Windows abstraction of the device into the actual commands that are required to transfer data to and from a real device.
Filter Drivers and Filter Device objects
Device stacks can have multiple filter device objects (filter DOs), which can be placed above or below the FDO.
Each filter DO is associated with a filter driver. Filter drivers are optional, but often present. The usual purpose of a filter driver is to modify some of the I/O requests as they pass through the device stack.
For example, the filter drivers can be used to encrypt or decrypt read and write requests. Filter Drivers can also be used for purposes that do not require modification of I/O requests, such as tracking requests, and reporting the information back to a monitoring application. Sample of the same we will soon see in our upcoming blog series.
After having understanding of the layout of the Driver stack and Device stack, we will discuss how this arrangement actually works in our next blog.

Thursday, November 24, 2016

Windows Driver Architecture

Let's familiarize ourselves with concepts and terms used extensively with Windows Drivers. I have shared the links to the sources of information for details and tried to keep it brief here to help us speed up.

Let's start with defining drivers.
From the MSDN, "In the most fundamental sense, a driver is a software component that lets the operating system and a device communicate with each other."

The statement itself helps us deduce a lot about drivers:

  • It's software.
  • It sits between the operating system and the connected device.
  • It serves to communicate between the OS and the hardware.

So, the Windows kernel does not directly interact with the device. It depends on drivers to perform hardware-specific tasks, e.g. detecting the attached devices, mediating communication with them, and exposing them to the application by providing an interface.

Essentially, in the larger context, the driver has two interfaces, one which communicates with the OS and the other that knows how to communicate with the device hardware.

Think of it as follows:


Expanding the definition



It will make drivers pretty complex and cumbersome to implement if it's going to do all the business of detecting the device attached to it, handle all its primary functions, and even auxiliary functions like encrypting and decrypting the flow of data to and from the device.

So, Let's divide the work between several drivers and implement them in a stack, as follows:



Implemented as Driver Stack



For a given I/O request, several drivers, layered in a stack, participate in the request. Thus, not all the drivers communicate directly with the device. Keep this figure in mind, and we will discuss and look into it more in our section "Device Objects and the Device Stack".

To better understand how the driver works, let's try to understand how drivers fit in the system.

Core Windows Architecture

This section follows closely with the information given in the book "Developing Drivers with the Microsoft Windows Driver Foundation by Penny Orwick."

Windows system architecture is a layered architecture with applications at the top and hardware at the bottom divided into User Mode and Kernel Mode.

All Input/output (I/O) requests are packet-driven I/O, which utilizes I/O request packets (IRPs) and asynchronous I/O. These are passed between the system and the driver and from one driver to another in the stack.


The following figure is an oversimplified architectural diagram of Windows, showing how drivers integrate into the overall architecture.




Let's look into the key components given in the diagram. They are essential, and as we progress through our understanding of the drivers, we will see how the driver integrates into the overall architecture. 

Applications and Windows API

Consider getting a copy of "Windows via C/C++by Jeffrey Richter, Christophe Nasarre". The book covers the building blocks of the Windows application layer and how the application uses the resources provided by the operating system. The book helps to understand how the applications and drivers interact. It's a great book and is available in the Kindle version.

Applications run in the user mode and cannot directly call in the services of drivers that run in kernel mode (though a bit of generalization since UMDF drivers run in User mode).

Windows provides a documented interface to the application called Windows API or simply WinAPI. These APIs define the interface between the operating system and an application. The application uses these APIs to issue I/O requests, which use appropriate Windows components to pass the request to its kernel-mode component.

User Mode and Kernel Mode.

Windows operating system divides into two main operating modes: User mode and Kernel Mode.

User Mode
Kernel Mode
Applications, System defined processes, and DLLs.
Core Operating system components: Executive services, Kernel Driver, Kernel, and Hardware Abstraction Layer.
Runs in a restricted environment and cannot directly access hardware or reference memory.
Full and unrestricted access to the underlying hardware. Can execute any CPU instruction and reference any memory address.
The application has a private virtual address space and a private handle table. Because an application's virtual address space is private, one application cannot alter data that belongs to another application. 
All code that runs in kernel mode shares a single virtual address space. This means that a kernel-mode driver is not isolated from other drivers and the operating system itself.
The crash is limited to application.
Crash can cause system fault or BSOD.

Kernel Subsystem

NTOSKRNL.EXE contains the kernel subsystem that makes up the low-level kernel-mode portion. These subsystems handle much of the core Windows functionality like I/O, object management, power management, security and process management. There are several kernel subsystems in Windows operating system like:


Let's look into some subsystems and how they interact with the drivers:

I/O Manager
IO Manager facilitates communication between applications and the concerned driver. The I/O manager receives I/O requests from user mode, translates them into IRPs and passes them to the appropriate driver. It later gets a completed IRP from the driver and passes and data back to user mode and to the application that issued the request.

PnP Manager
Handles Plug and Play tasks such as enumerating the devices attached to the bus, constructing devices stack for the device, and handling the process of adding or removing the devices while the system is running.
Plug and Play service running in the user mode forms a significant portion of PnP Manager. It handles the often complex tasks of installing the appropriate drivers, notifying services and applications of the arrival of new devices, and displaying GUI to the user.

Power Manager
Power Manager handles the power events (power-off, stand-by, hibernate, etc.) generated during the computer's power state changes.

Drivers and Devices

The drivers provide the interface between the kernel subsystem and their devices. When the user application wants to initiate the communication or data transfer to the end device, it calls in well-defined APIs by the Windows OS. It, in turn, invokes the appropriate kernel subsystems that receive the request and structures data into a well-defined IRP structure. Later, the driver receives these I/O requests and processes and communicates with the devices. Any data output from the device passes back to the kernel subsystem that generated the request. 
Drivers also handle I/O requests directly generated from kernel subsystems such as PnP IRPs and Power IRPs to handle tasks such as preparing the device for removal or hibernation.

Let's dive into the "Device Objects and the Device Stack" in the upcoming blog.

Sources referred and followed:

Wednesday, November 5, 2014

Understanding Windows Drivers and Kernel Internals.

In this blog series, I will discuss the fundamentals of developing device drivers for the Microsoft Windows operating system. I will cover essential concepts such as the history and architecture of the Windows driver model, the structure and initialization of driver development, and the WDF (Windows Driver Framework) and WDM (Windows Driver Model) frameworks. I will also provide an overview of key topics such as memory management, I/O operations, and synchronization in Windows drivers. This series will provide a solid foundation for those interested in writing device drivers for the Windows operating system. I have categorized the topics I plan to cover in the following sections/parts:



Part 1: Windows Driver Overview

We need to know essential Windows operating system concepts to understand its internal workings. The topics covered are fundamental to Windows OS Design and its core architecture. We develop drivers based on it, and these concepts apply to WDM and WDF-based drivers. I will cover the WDF-based topics in-depth in the later parts.

Topics covered under Part 1:


Introduction: Windows Driver and its History.

What is a Device Driver? How is the Device Driver implemented in Windows OS, and the high-level view of the Windows Driver Architecture?

History gives a good perspective of how we reached here. It primarily holds in the case of the Windows operating system. During the old days of DOS, the kernel was free land, and developers wrote their hardware drivers in real-mode assembly code. Our drivers now conform to Windows driver architecture and have come a long way to the highly structured, object-oriented, event-driven WDF driver model.


Windows Driver Architecture

We take a high-level view of the Windows Architecture from the driver's perspective and cover essential concepts.


Device Object and Driver Stack

  • Device And Driver Layering: Device Objects and Device Stack.
  • Kinds of Devices: PnP Devices and Legacy Devices. 
  • Recursive Enumeration and Device Tree: How the system finds and loads the kernel driver: 


Windows I/O Model

The topic covers how the data communicates between driver layers or device stacks.
Windows adopted a standard communication mechanism between different components of Windows Driver Architecture. Windows OS kernel handles data as request packets or I/O request packets. These IRPs are not just for data transfer but do much more.


WDM and WDF Framework

This topic will cover an overview of the WDM and WDF frameworks. It is better to understand these frameworks before we build drivers.


Part 2: Driver Structure and Initialization: Building and Installation.

We review the latest build environment and necessary tools based on the WDK(Windows Driver Kit) 11 and Visual Studio 2019. We check on the essentials needed to run our kernel driver and write a simple "HelloWorld" driver that can be loaded and unloaded on our Windows Machine. With the fundamentals covered, we will make this driver handle specific tasks as an operating system extension. 

We discuss the following to help us build a reliable driver: 

  • Driver Entry Points and Callbacks
  • Function Return Values
  • Termination and Exception Handling
  • C++ Usage
  • Tracing and Diagnose
  • Windows Kernel Debugging
  • Driver Signing
  • Driver Verifier
  • Kernel Libraries

Part 3: WDF Fundamentals

The WDF Framework and Object Model

Although WDF (Windows Driver Framework) and WDM (Windows Driver Model) serve the same purpose of handling communication between Windows OS and the driver, WDF means to supersede WDM. It provides an abstraction over the WDM that simplifies implementing robust, secure, and efficient drivers. It enables developers to focus on their hardware requirements rather than the complexities of the operating system. This section looks at the framework, its architecture, and how it affects how we write our driver. We discuss WDF Object Model as a foundation for understanding the driver.


Memory Management for Windows Drivers

Drivers allocate memory to store internal data, buffer data during I/O operations, and share the memory with other kernel-mode and user-mode components. Understanding the memory management techniques used by Windows is critical for a driver writer as it affects the decisions regarding scheduling and synchronization.

This topic discusses memory allocation and usage in kernel-mode drivers. It describes the types of memory available for driver use and appropriate techniques for allocating and using each type.


Scheduling, Thread Context, IRQLs, and Interrupts

Windows is a multitasking operating system that can run in a symmetric multiprocessor environment. The driver code executes in the context of one thread or the other.

This topic presents thread scheduling, context, and the processor's current interrupt request level (IRQL).


Locks, Deadlocks, and Synchronization.

We use synchronization mechanisms to protect shared memory locations in kernel-mode drivers for Windows. We must determine when synchronization is required and how to use each type.


User-mode Interactions.

Kernel-mode drivers interact with user-mode components by sharing data, objects, and handles, passing notification and synchronization information, and using settings and properties supplied by a user.


References: