Show dropped packets per interface on Linux

Server Management Service

Are you trying to show dropped packets per interface on Linux?

This guide is for you.

We can use the ip command or netstat command or ethtool command to show dropped packets statistics per network interface on Linux.

Here at Ibmi Media, as part of our Server Management Services, we regularly help our Customers to check the dropped packets statistics for Linux servers.

In this context, you shall learn ways to show dropped packets per interface on Linux.

To show dropped packets per interface on Linux using the netstat:

Generally, the netstat command is mostly obsolete. Replacement for netstat is done by ss and ip command.

However, netstat is still available on older Linux distros, which are in productions.

We use the following syntax of netstat command:

netstat -i
netstat --interfaces

For displaying the summary statistics for each protocol, we can run the following commands:

netstat -s
netstat --statistics

Outputs:

Ip:
Forwarding: 1
101759568 total packets received
70289211 forwarded
0 incoming packets discarded
31287093 incoming packets delivered
136164545 requests sent out
22 outgoing packets dropped
220 reassemblies required
110 packets reassembled ok
2364 fragments received ok
3345 fragments failed
4728 fragments created
Icmp:
295517 ICMP messages received
6 input ICMP message failed
ICMP input histogram:
destination unreachable: 145
timeout in transit: 187
echo requests: 289750
echo replies: 5435
298725 ICMP messages sent
0 ICMP messages failed
ICMP output histogram:
destination unreachable: 3408
echo requests: 5567
echo replies: 289750
IcmpMsg:
InType0: 5435
InType3: 145
InType8: 289750
InType11: 187
OutType0: 289750
OutType3: 3408
OutType8: 5567
Tcp:
19006 active connection openings
14619 passive connection openings
2268 failed connection attempts
393 connection resets received
1 connections established
2215735 segments received
2511500 segments sent out
6067 segments retransmitted
182 bad segments received
13173 resets sent
Udp:
28543977 packets received
63 packets to unknown port received
287687 packet receive errors
22106848 packets sent
287687 receive buffer errors
0 send buffer errors
UdpLite:
TcpExt:
10 invalid SYN cookies received
2264 resets received for embryonic SYN_RECV sockets
42 packets pruned from receive queue because of socket buffer overrun
14095 TCP sockets finished time wait in fast timer
21 packetes rejected in established connections because of timestamp
16908 delayed acks sent
13 delayed acks further delayed because of locked socket
Quick ack mode was activated 4346 times
756194 packet headers predicted
441344 acknowledgments not containing data payload received
618096 predicted acknowledgments
TCPSackRecovery: 87
Detected reordering 418 times using SACK
TCPDSACKUndo: 1
14 congestion windows recovered without slow start after partial ack
TCPLostRetransmit: 3994
TCPSackFailures: 1
121 fast retransmits
8 retransmits in slow start
TCPTimeouts: 5158
TCPLossProbes: 789
TCPLossProbeRecovery: 66
TCPSackRecoveryFail: 3
TCPBacklogCoalesce: 8617
TCPDSACKOldSent: 4359
TCPDSACKOfoSent: 1
TCPDSACKRecv: 127
3870 connections reset due to unexpected data
244 connections reset due to early user close
487 connections aborted due to timeout
TCPDSACKIgnoredNoUndo: 33
TCPSackShifted: 37
TCPSackMerged: 115
TCPSackShiftFallback: 731
TCPRcvCoalesce: 225465
TCPOFOQueue: 29252
TCPOFOMerge: 1
TCPChallengeACK: 193
TCPSYNChallenge: 186
TCPAutoCorking: 26574
TCPFromZeroWindowAdv: 8
TCPToZeroWindowAdv: 8
TCPWantZeroWindowAdv: 37
TCPSynRetrans: 647
TCPOrigDataSent: 1526711
TCPACKSkippedSynRecv: 153
TCPKeepAlive: 53
TCPDelivered: 1539034
TCPAckCompressed: 2559
IpExt:
InNoRoutes: 16
InBcastPkts: 4
InOctets: 92596603587
OutOctets: 263001759492
InBcastOctets: 310
InNoECTPkts: 121775194
InECT1Pkts: 1
InECT0Pkts: 51506
InCEPkts: 25

And for showing tcp stats we can use the following:

netstat --statistics --tcp
netstat -s -t

After that, we can use the following commands to display udp stats

netstat --statistics --udp
netstat -s -u

How to show dropped packets statistics per network interface on Linux using the ip ?

Now, let us see how to display link device stats using the ip command.

The following commands are used:

ip -s link
ip -s link show {interface}
ip -s link show eth0

In this example the display link stats for wg0:

ip -s link show wg0
4: wg0: mtu 1420 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
link/none
RX: bytes packets errors dropped overrun mcast
1889086196 11451163 8413 62869 0 0
TX: bytes packets errors dropped carrier collsns
56342032204 41609374 0 5685 0 0

It is clear that TX is Transmit and RX is Receive. Wireguard creates the wg0 interface.

So either Wireguard or firewall dropping packets as per policy.

Queries the specified network device for NIC – and driver-specific statistics with ethtool

Pass the -S or –statistics option to display stats.

We will use the following syntax:

ethtool -S {device}
ethtool -S eth0

NIC statistics:

rx_queue_0_packets: 94804582
rx_queue_0_bytes: 92123064799
rx_queue_0_drops: 0
rx_queue_0_xdp_packets: 0
rx_queue_0_xdp_tx: 0
rx_queue_0_xdp_redirects: 0
rx_queue_0_xdp_drops: 0
rx_queue_0_kicks: 1499
tx_queue_0_packets: 94616365
tx_queue_0_bytes: 93565559918
tx_queue_0_xdp_tx: 0
tx_queue_0_xdp_tx_drops: 0
tx_queue_0_kicks: 40246533

Another option is to directly query the /proc/net/dev file either using the cat command or column command:

cat /proc/net/dev
column -t /proc/net/dev

We will see sumthin similar given below:

Inter-| Receive | Transmit
face |bytes packets errs drop fifo frame compressed multicast|bytes packets errs drop fifo colls carrier compressed
eth0: 92123116754 94805122 0 0 0 0 0 0 93565689124 94617058 0 0 0 0 0 0
wg0: 1889086196 11451163 8413 62869 0 8413 0 0 56342032204 41609374 0 5685 0 0 0 0
lo: 52141452 150908 0 0 0 0 0 0 52141452 150908 0 0 0 0 0 0
tun0: 1650631998 16914416 0 0 0 0 0 0 30143956312 22000354 0 660246 0 0 0 0

How to find out why a Linux server is dropping packets ?

We will be using dropwatch to find our why a Linux server is dropping packets:

1. Building dropwatch

Firstly we will Install required tools, libs and gcc compiler collection on Ubuntu or Debian Linux using the following command:

$ sudo apt-get install libpcap-dev libnl-3-dev libnl-genl-3-dev binutils-dev libreadline6-dev autoconf libtool pkg-config build-essential

After that we will clone the repo and compile it using the following commands:

git clone https://github.com/nhorman/dropwatch
cd dropwatch
./autogen.sh
./configure
make
make install

2. Session:

Making install in src
make[1]: Entering directory '/tmp/dropwatch/src'
make[2]: Entering directory '/tmp/dropwatch/src'
/usr/bin/mkdir -p '/usr/local/bin'
/bin/bash ../libtool --mode=install /usr/bin/install -c dropwatch dwdump '/usr/local/bin'
libtool: install: /usr/bin/install -c dropwatch /usr/local/bin/dropwatch
libtool: install: /usr/bin/install -c dwdump /usr/local/bin/dwdump
make[2]: Nothing to be done for 'install-data-am'.
make[2]: Leaving directory '/tmp/dropwatch/src'
make[1]: Leaving directory '/tmp/dropwatch/src'
Making install in doc
make[1]: Entering directory '/tmp/dropwatch/doc'
make[2]: Entering directory '/tmp/dropwatch/doc'
make[2]: Nothing to be done for 'install-exec-am'.
/usr/bin/mkdir -p '/usr/local/share/man/man1'
/usr/bin/install -c -m 644 dropwatch.1 '/usr/local/share/man/man1'
make[2]: Leaving directory '/tmp/dropwatch/doc'
make[1]: Leaving directory '/tmp/dropwatch/doc'
Making install in tests
make[1]: Entering directory '/tmp/dropwatch/tests'
make[2]: Entering directory '/tmp/dropwatch/tests'
make[2]: Nothing to be done for 'install-exec-am'.
make[2]: Nothing to be done for 'install-data-am'.
make[2]: Leaving directory '/tmp/dropwatch/tests'
make[1]: Leaving directory '/tmp/dropwatch/tests'
make[1]: Entering directory '/tmp/dropwatch'
make[2]: Entering directory '/tmp/dropwatch'
make[2]: Nothing to be done for 'install-exec-am'.
make[2]: Nothing to be done for 'install-data-am'.
make[2]: Leaving directory '/tmp/dropwatch'
make[1]: Leaving directory '/tmp/dropwatch'