Hardware Timestamping for Network Packets on Linux
When microseconds matter — in financial trading systems, precision time protocol (PTP) daemons, or high-frequency network telemetry — software-level packet timestamps are simply not good enough. The kernel's software clock introduces jitter from interrupt handling, scheduler delays, and CPU power states. Hardware timestamping network infrastructure solves this by recording packet arrival and departure times directly inside the NIC, before the packet ever touches kernel code.
Why Software Timestamps Fall Short
Linux assigns a software timestamp to each packet when the kernel's networking stack processes it. Depending on driver implementation and system load, this can occur hundreds of microseconds after the packet actually arrived at the wire. On a busy server, that delay is unpredictable. Applications relying on SO_TIMESTAMP or SO_TIMESTAMPNS receive timestamps that reflect when the CPU noticed the packet — not when the packet crossed the physical layer. For latency measurement, one-way delay calculation, or IEEE 1588 PTP synchronization, this imprecision is unacceptable.
How Hardware Timestamping Works
Modern NICs — Intel 82599, X550, XL710, Mellanox ConnectX series, and others — contain dedicated hardware clocks synchronized to a free-running counter. When a packet's preamble or start-of-frame delimiter is detected on the wire, the NIC latches the current counter value directly into the packet descriptor. This timestamp travels with the packet through the DMA ring buffer and is available to the driver before any software processing occurs.
On the transmit side, the NIC timestamps the packet the moment it is placed on the wire, not when the application called sendmsg(). This two-sided precision allows accurate round-trip and one-way latency measurements at nanosecond resolution. The hardware clock itself can be disciplined via PTP to a grandmaster clock, keeping all nodes in a cluster synchronized to within tens of nanoseconds.
Enabling Hardware Timestamping on Linux
Linux exposes hardware timestamping through the SO_TIMESTAMPING socket option, introduced in kernel 2.6.30. You must first verify that your NIC and driver support it using ethtool:
ethtool -T eth0This prints the supported timestamp modes. Look for SOF_TIMESTAMPING_TX_HARDWARE and SOF_TIMESTAMPING_RX_HARDWARE in the output. To enable hardware timestamps on a socket in C:
int flags = SOF_TIMESTAMPING_RX_HARDWARE |
SOF_TIMESTAMPING_TX_HARDWARE |
SOF_TIMESTAMPING_RAW_HARDWARE;
setsockopt(fd, SOL_SOCKET, SO_TIMESTAMPING, &flags, sizeof(flags));Timestamps are retrieved from the ancillary data of recvmsg() via the SCM_TIMESTAMPING control message, which delivers a struct timespec[3] array. The third element contains the raw hardware timestamp.
Hardware Timestamping with PFQ and Packet Filtering
Tools like PFQ (Packet Filtering Queue) are built for high-throughput packet capture and filtering on Linux. PFQ operates at the driver level using kernel bypass techniques, making it an ideal layer at which to exploit hardware timestamping network capabilities. When PFQ captures packets directly from the NIC ring buffer, the hardware timestamp embedded in each descriptor is preserved and exposed to userspace with no additional kernel processing overhead.
Combining PFQ's network queue architecture with hardware timestamps gives you both the throughput of zero-copy capture and the precision of NIC-level timing. This pairing is particularly valuable for passive monitoring probes that must record accurate inter-packet gaps or detect sub-microsecond bursts without dropping packets under load. The pfq-lang functional DSL can then be used to filter, aggregate, or route timestamped packets to specific queues for parallel processing across CPU cores.
Synchronizing the NIC Hardware Clock
A hardware timestamp is only as useful as the clock it references. Linux provides the phc2sys and ptp4l utilities from the linuxptp package to synchronize the NIC's Physical Hardware Clock (PHC) to a PTP grandmaster or GPS reference. Once synchronized, all hardware timestamps are traceable to UTC with sub-100ns accuracy on well-configured networks.
ptp4l -i eth0 -m -f /etc/ptp4l.conf
phc2sys -s eth0 -c CLOCK_REALTIME -wRunning both daemons ensures that the PHC tracks the network grandmaster while the system clock tracks the PHC, giving all timestamp consumers — kernel, userspace, and PFQ-based tools — a coherent and accurate time reference.
Transmit Timestamps and Error Queues
Transmit hardware timestamping requires reading from the socket error queue after sending. When SOF_TIMESTAMPING_TX_HARDWARE is set, the kernel places the looped-back timestamp into the error queue. Applications poll this queue with MSG_ERRQUEUE in a recvmsg() call immediately after transmission. This is the mechanism used by PTP implementations to measure precise send times without modifying the transmit path.
One important caveat: only one outstanding TX timestamp request is supported per socket at a time. High-rate senders must drain the error queue between sends or risk missing timestamps. For packet filtering and network queue scenarios where TX rates are high, batching and careful queue management are essential.
Practical Considerations and NIC Support
Not all NICs implement hardware timestamping network features equally. Some support only RX timestamps; others support both directions. Virtual NICs in cloud environments rarely expose PHC devices at all. Always verify support with ethtool -T and check that the /dev/ptpN device exists for your interface. Driver quality matters too — some drivers have bugs where timestamps are occasionally zero or duplicated across packets. Testing with testptp from the kernel source tree helps validate clock behavior before deploying in production.
For latency-critical Linux networking deployments, hardware timestamping is not optional — it is the foundation on which accurate measurement, PTP synchronization, and high-fidelity packet analysis all depend.