High-resolution network monitoring with ping

Return Values

The ping command also has a return value, in addition to the RTT, that can assume all kinds of values, because pong (the answer to ping) can contain different messages: from A for communication with destination network administratively prohibited to Z for communication with destination host administratively prohibited. In other cases, you could have corrupted data and ICMP error messages.

The ping programs distinguish between cases but is unfortunately not very specific; even the very useful version from the iputils package only utilizes three cases, as you can see from the man page: "If ping does not receive any reply packets, it terminates with code 1. For other errors, it terminates with code 2. In all other cases it exits with code 0."

In short, the iputils ping returns code 1 for a timeout or loss, 2 for all other errors, and 0 if the ping returns without error. Therefore, lost pings (e.g., through wire breaks) can be identified by a return value of 1, and other errors by a return value of 2.

Deviations from this rule occur in practice: if the network of the target computer is unreachable, ping reports Network is unreachable with a return value of 2. Thus, users cannot tell wire breaks from other errors. Interruptions, however, such as a spy box inserted along the path, will always cause a non-zero return value.

Success Quota

The Pinger program creates exponential moving averages for the return values and RTTs, expressed as percentages, so that a return value of 0 is usually close to 100 percent, and code 1 or 2 errors, as in the case of wire breaks, are well above 0 percent. An advantage compared with typical monitoring programs such as MRTG [6] is that Pinger determines values every second, whereas MRTG and the like usually only check every five minutes (300 seconds).

Moreover, Linux users can identify wire breaks on the local computer from kernel messages, as Listing 2 shows. ethtool gives you the current Ethernet status.

Listing 2

Kernel Messages

 

Other computer components are similar; for example, the keyboard has timeout flag 0x40 on port 0x64 – even PS/2 keyboards, for which the kernel does not provide any messages.

Finding Switches and Bridges

In practice, Pinger can be used to find a switch or a transparent bridge on the gigabit LAN. Such a device is barely noticeable on the network because it does not change the data packets when it forwards them. Yet an attacker could use it to sniff, evaluate, or manipulate the traffic. Pure sniffing goes undetected on the basis of data only.

However, metadata associated with data is on the network, such as the latency of a connection. A switch means additional latency, with a delay that increases with the size of the package, because it only forwards a packet after receiving and processing it completely. Therefore, at least the packet reception time is detectable as a longer packet at run time. For monitoring, you would theoretically want to use packets that are as large as possible and unfragmented. In practice, 44-byte pings have been shown to do the trick, as a test with a gigabit LAN switch shows.

Figure 1 shows the RTT between two Linux PCs. A D-Link DGS-1008D digital gigabit switch is initially between the computers. In the afternoon, that switch is removed and replaced by a 4cm adapter. The diagram plots the RTTs of pings sent every second (in milliseconds) over time, after exponential smoothing with a factor of 0.001, to create approximately 1,000-second mean values.

Figure 1: Experimentally determined RTT values. The drop (left) occurred when a gigabit switch was replaced by a short adapter.

The absence of the gigabit switch is clearly shown by the 30µs shorter RTTs. One thousand pings are all it takes, and they can run in the background during normal operations.

Shortly before noon on the following day, I replaced the adapter with a Netgear GS105 switch that suddenly increased the RTT by more than 20µs. The TP-LINK TL-SG1024 switch, however, shows no significant difference in RTTs, at least in the 1,000-second mean values for the first hours compared with the adapter. Then, the increase is just 15µs, which means the switch is initially significantly closer to the theoretical 352ns packet reception time limit than the others.

Here, the RTTs vary only by a few microseconds; large ping packets might deliver clearer results. You do not need jumbo frames and huge pings; pings with a size of 1,500 bytes prove to be perfectly okay. The packet reception time is an easily measurable 12µs. The RTT difference is 32µs, a change from 228 to 260µs, as Figure 2 shows.

Figure 2: Large ping packets reveal noticeable run-time delays, even on fast switches.

On closer inspection, Figure 1 reveals a daily rhythm, caused by RTT values that increase with the load average of the participating computers and the network load. For example, on the last day of the test, the load average increased by one, and the RTT increased by 5µs on one computer. Minimum values occur at night between 2 and 3am.

You can easily compute these influences in a first approximation; to determine the net RTT for a load average and a network load of 0, thus resulting in significantly more accurate values. Because same-day procedures almost always return the same daily averages, this case can make do without such corrections and thus ensure high resolution, as shown in the next example.

Incidentally, alternatives to using more pings for higher resolution include pinging with real-time threads under Linux-RT (preemption) using a soft real-time or RTAI (real-time application interface) with a hard real-time, and taking differential measurements (i.e., pinging on two different connections) and evaluating the differences in the RTT values.

Buy this article as PDF

Express-Checkout as PDF
Price $2.95
(incl. VAT)

Buy Linux Magazine

SINGLE ISSUES
 
SUBSCRIPTIONS
 
TABLET & SMARTPHONE APPS
Get it on Google Play

US / Canada

Get it on Google Play

UK / Australia

Related content

  • Security Lessons: Bufferbloat

    An abundance of buffers hides the Internet’s dirty little secret.

  • Command Line: Network Diagnostic Tools

    Linux has the right tools to track down network errors and open the way for data packets.

  • Charly's Column

    If you do not receive a response to a ping, or if the response is seriously delayed, you might like to take this as a warning. But who wants to ping all day? You need a ping-based monitoring utility like Smokeping.

  • High Availability VPN

    IPSec prevents many of the clever tricks high-availability products employ. We’ll show you a solution that provides transparent backup for IPSec connections.

  • Charly's Column

    A touchy LAN that plays like a movie diva can spoil any admin’s day. Ethtool to the rescue!

comments powered by Disqus

Direct Download

Read full article as PDF:

Price $2.95

News