Posted in: Linux, Networking, Unix

Networking Performance Troubleshooting in Linux

– ping :
This command is used to test network connectivity. It sends a 32 bytes ICMP (Internet Control Message Protocol) packet to the destination host.
If the destination host is unreachable due to any issue, you will notice packet loss.
ping is also used to check the network latency as it shows the RTT(Round Trip Time)
# ping 192.168.159.127
PING 192.168.159.127 (192.168.159.127) 56(84) bytes of data.
^C
--- 192.168.159.127 ping statistics ---
2 packets transmitted, 0 received, 100% packet loss, time 1010ms


# ping google.com
PING google.com (74.125.193.138) 56(84) bytes of data.
64 bytes from ig-in-f138.1e100.net (74.125.193.138): icmp_seq=1 ttl=128 time=16.8 ms
64 bytes from ig-in-f138.1e100.net (74.125.193.138): icmp_seq=2 ttl=128 time=19.7 ms
64 bytes from ig-in-f138.1e100.net (74.125.193.138): icmp_seq=3 ttl=128 time=21.1 ms
64 bytes from di-in-f138.1e100.net (74.125.193.138): icmp_seq=4 ttl=128 time=22.3 ms
^C
--- google.com ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 4051ms

– ifconfig :
This is another simple command to check the interface along with drops or errors.
# ifconfig eth0
eth0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        inet 172.17.0.1  netmask 255.255.0.0  broadcast 172.17.255.255
        ether 02:42:16:99:1a:3d  txqueuelen 0  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

– curl :
This can be used to check connectivity on specified port
# curl -v telnet://8.8.8.8:53
*   Trying 8.8.8.8:53...
* Connected to 8.8.8.8 (8.8.8.8) port 53 (#0)

– netstat -s :
When used with -s option, netstat shows the detailed statistics for all the protocols. This can be further filtered either for TCP or UDP protocols using -t or -u respectively
# netstat -s

Ip:
    Forwarding: 1
    17733 total packets received
    1 with invalid addresses
    0 forwarded
    0 incoming packets discarded
    17730 incoming packets delivered
    12960 requests sent out
    20 outgoing packets dropped

Icmp:

    129 ICMP messages received
    0 input ICMP message failed
    ICMP input histogram:
        destination unreachable: 106
        timeout in transit: 3
        echo requests: 10
        echo replies: 10
    141 ICMP messages sent
    0 ICMP messages failed
    ICMP output histogram:
        destination unreachable: 107
        echo requests: 24
        echo replies: 10

Tcp:

    107 active connection openings
    0 passive connection openings
    2 failed connection attempts
    1 connection resets received
    0 connections established
    12949 segments received
    11564 segments sent out
    24 segments retransmitted
    0 bad segments received
    10 resets sent

Udp:
    3112 packets received
    85 packets to unknown port received
    0 packet receive errors
    1237 packets sent
    0 receive buffer errors
    0 send buffer errors
    IgnoredMulti: 1455



– traceroute :
traceroute is used to trace the path a packet takes to reach the destination IP-address.  This tool can help you to determine where you are losing packets in the network, helping to identify problems.
# traceroute 192.168.159.128
traceroute to 192.168.159.128 (192.168.159.128), 30 hops max, 60 byte packets
1  jay-virtual-machine (192.168.159.128)  0.214 ms  0.018 ms  0.010 ms




# traceroute www.google.com
traceroute to www.google.com (74.125.193.104), 30 hops max, 60 byte packets
1  _gateway (192.168.159.2)  0.284 ms  0.305 ms  0.202 ms
2  * * *
3  * * *

– tcpdump :
This utility is used to capture and troubleshoot the network traffic issues.
Installing tcpdump –
For Ubuntu and Debian OS :
# sudo apt install tcpdump
For CentOS and Fedora OS :
# sudo yum install tcpdump
Here is an example from Ubuntu system –
# apt install tcpdump
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following packages were automatically installed and are no longer required:
  cpu-checker docker-scan-plugin ibverbs-providers ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaio1 libcacard0 libdaxctl1 libdecor-0-0 libdecor-0-plugin-1-cairo libfdt1
  libgfapi0 libgfrpc0 libgfxdr0 libglusterfs0 libibverbs1 libiscsi7 libndctl6 libpmem1 libpmemobj1 libqrencode4 librados2 librbd1 librdmacm1 libsdl2-2.0-0 libslirp0
  libspice-server1 liburing2 libusbredirparser1 libvirglrenderer1 msr-tools ovmf pass qemu-block-extra qemu-system-common qemu-system-data qemu-system-gui
  qemu-system-x86 qemu-utils qrencode seabios tree uidmap xclip
Use 'apt autoremove' to remove them.
The following NEW packages will be installed:
  tcpdump
0 upgraded, 1 newly installed, 0 to remove and 130 not upgraded.
Need to get 501 kB of archives.
After this operation, 1,407 kB of additional disk space will be used.
Get:1 http://ie.archive.ubuntu.com/ubuntu jammy/main amd64 tcpdump amd64 4.99.1-3build2 [501 kB]
Fetched 501 kB in 4s (115 kB/s)       
Selecting previously unselected package tcpdump.
(Reading database ... 197459 files and directories currently installed.)
Preparing to unpack .../tcpdump_4.99.1-3build2_amd64.deb ...
Unpacking tcpdump (4.99.1-3build2) ...
Setting up tcpdump (4.99.1-3build2) ...
Processing triggers for man-db (2.10.2-1) ...
#


# tcpdump --version
tcpdump version 4.99.1
libpcap version 1.10.1 (with TPACKET_V3)
OpenSSL 3.0.2 15 Mar 2022

There are various useful options available with tcpdump which can help to fetch the required data from the traffic flow which will help to isolate and locate the issue.
1. count (-c) :-
Tcpdump continues to capture packets until it receives an interrupt signal. We can specify the number of packets to be captured with -c i.e. count option.
# tcpdump -c 3 -i ens33

tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on ens33, link-type EN10MB (Ethernet), snapshot length 262144 bytes
15:39:03.049159 ARP, Request who-has _gateway tell 192.168.159.1, length 46
15:39:03.098962 IP jay-virtual-machine.58154 > _gateway.domain: 47775+ [1au] PTR? 2.159.168.192.in-addr.arpa. (55)
15:39:04.046715 ARP, Request who-has _gateway tell 192.168.159.1, length 46
3 packets captured
29 packets received by filter
15 packets dropped by kernel
#
2. interface (-i) :-
With -i you can define the interface/link for which the network traffic needs to be captured.
3.  src and dst :-
with src and dst we can specify the source or destination IP address –
# tcpdump -i ens33 src 192.168.159.1
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on ens33, link-type EN10MB (Ethernet), snapshot length 262144 bytes
16:17:37.547593 ARP, Request who-has _gateway tell 192.168.159.1, length 46
16:17:38.596647 ARP, Request who-has _gateway tell 192.168.159.1, length 46
16:17:39.558136 ARP, Request who-has _gateway tell 192.168.159.1, length 46


# tcpdump -c 3 dst 192.168.159.2
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on ens33, link-type EN10MB (Ethernet), snapshot length 262144 bytes
17:01:46.077680 ARP, Request who-has _gateway tell 192.168.159.1, length 46
4. write (-w) and read (-r) :-
-w is used to write the raw packets to file rather than parsing and printing them out.  They can later be printed with the -r option.
Example –
# tcpdump -i ens33 -w /tmp/result-new.pcap
tcpdump: listening on ens33, link-type EN10MB (Ethernet), snapshot length 262144 bytes
^C297 packets captured
297 packets received by filter
0 packets dropped by kernel
# tcpdump -r /tmp/result-new.pcap
reading from file /tmp/result-new.pcap, link-type EN10MB (Ethernet), snapshot length 262144
16:30:46.664856 IP 192.168.159.1.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 00:50:56:c0:00:08 (oui Unknown), length 300
16:30:46.664857 IP 192.168.159.254.bootps > 192.168.159.1.bootpc: BOOTP/DHCP, Reply, length 300
16:30:50.054881 ARP, Request who-has _gateway tell 192.168.159.1, length 46
5. You can also use “>” to redirect the output to a file –
# tcpdump -i ens33 src 192.168.159.1 > /tmp/res.pcap
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on ens33, link-type EN10MB (Ethernet), snapshot length 262144 bytes
1 packet captured
3 packets received by filter
0 packets dropped by kernel


# less /tmp/res.pcap
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on ens33, link-type EN10MB (Ethernet), snapshot length 262144 bytes
1 packet captured
3 packets received by filter
0 packets dropped by kernel
6. port :-
To specify the port for the packet capture
# tcpdump -i ens33 port 443
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on ens33, link-type EN10MB (Ethernet), snapshot length 262144 bytes
7.  tcp/udp protocol :-
# tcpdump tcp
# tcpdump udp
8. direction (-Q) :-
Choose  send/receive  direction direction for which packets should be captured. Possible values are `in‘, `out‘ and `inout‘.
# tcpdump -Q in
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on ens33, link-type EN10MB (Ethernet), snapshot length 262144 bytes
17:41:56.079744 IP _gateway.domain > jay-virtual-machine.34783: 53368 0/0/1 (58)
17:41:58.100330 IP _gateway.domain > jay-virtual-machine.37836: 57582 0/0/1 (58)
17:41:58.102042 IP _gateway.domain > jay-virtual-machine.36846: 35582 NXDomain 0/0/1 (57)
17:41:58.117978 IP _gateway.domain > jay-virtual-machine.53852: 25628 3/0/1 A 35.224.170.84, A 35.232.111.17, A 34.122.121.32 (106)
17:42:00.115656 IP _gateway.domain > jay-virtual-machine.36846: 35582 NXDomain 0/0/0 (46)
17:42:01.266527 ARP, Reply _gateway is-at 00:50:56:ed:84:cd (oui Unknown), length 46
Back to Top