The term “fingerprinting” in the context of the NMAP, or Network Mapper, has a specific meaning. It refers to the process of determining the operating system running on a remote machine. This is done by examining the characteristics of the packets that the system sends in response to various probes. “Fingerprinting” in this sense is analogous to forensic fingerprinting, where unique identifying characteristics are used to determine identity.
NMAP is an open-source tool used for network exploration and security auditing. It can discover hosts and services on a network, creating a map of the system. Besides network mapping, determining the operating system of the devices in a network is also very useful for specialists seeking to troubleshoot or secure a network (Nmap.org).
The way NMAP performs fingerprinting is based on a test suite of around a dozen TCP/IP protocols. It sends out specific probe packets to the target, and then records every aspect of the responses, such as the TCP initial sequence number (ISN) sampling, TCP options support and ordering, ICMP messages quirk and integrity, and response to invalid requests.
A machine’s response to these probes can unveil a lot of information about the type of operating system it’s running, including both its generic category (e.g., Unix, Linux, Windows, etc.) and often its specific version (e.g., Windows XP SP1, Ubuntu 10.04 LTS, etc.). Responses are then compared to a database of known operating system fingerprints to make the identification as accurate as possible.
Taking IP packets as an example, they have various attributes like header length, fragment flags, and time to live (TTL) values, which could be unique depending on the operating system. Therefore, by studying the peculiarities of these attributes in received packets, NMAP can infer the operating system of the source host.
TCP/IP Stack Fingerprinting is the passive collection of configuration attributes from a remote device during standard layer 4 network traffic exchange. The combination of parameters may be used to infer the remote machine’s operating system (or its TCP/IP Stack), or incorporated into a device fingerprint. This technique elucidates differences between differently created TCP/IP stacks and detects the presence of traffic shapers and proxies (SANS Institute).
In a nutshell, fingerprinting in NMAP is a process of detecting and identifying the operating system running on a networked device based on various network attributes and responses.
Sources:
Nmap.org. “Operating System Detection,” https://nmap.org/book/osdetect.html
SANS Institute. “TCP/IP Fingerprinting,” https://www.sans.org/blog/tcp-ip-fingerprinting/
Word count: 398.