Monitoring

10 Command Line Tools To Monitor Linux Performance

mm
Written by Santosh Prasad

For every Linux Admin it is very tough job to monitor and debug Linux system performance problem every day. In this article I will show you top 10 frequently used command line monitoring tools that might be useful for you.

1. Top – Linux Process Monitoring

Top command is used to monitor Linux Performance. Top command used to display all the running and active real time process. It display CPU Usage, Memory Status, Swap Usage, Check Size, Buffer Size, Process PID, User, Command and much more. Using this command you can also check memory and cpu utilization of running process.

# top
top - 19:30:38 up 783 days,  5:36,  0 users,  load average: 0.06, 0.03, 0.00
Tasks:  91 total,   1 running,  90 sleeping,   0 stopped,   0 zombie
Cpu(s):  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:   2764752k total,   423272k used,  2341480k free,        0k buffers
Swap:  5529504k total,   263764k used,  5265740k free,   114912k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
    1 root      20   0 21328  200   84 S  0.0  0.0   1:14.62 init
    2 root      20   0     0    0    0 S  0.0  0.0   0:00.00 kthreadd/145172
    3 root      20   0     0    0    0 S  0.0  0.0   0:00.00 khelper/145172
    4 root      20   0     0    0    0 S  0.0  0.0   0:00.00 rpciod/145172/0
    5 root      20   0     0    0    0 S  0.0  0.0   0:00.00 rpciod/145172/1
    6 root      20   0     0    0    0 S  0.0  0.0   0:00.00 rpciod/145172/2
    7 root      20   0     0    0    0 S  0.0  0.0   0:00.00 rpciod/145172/3
    8 root      20   0     0    0    0 S  0.0  0.0   0:00.00 rpciod/145172/4
    9 root      20   0     0    0    0 S  0.0  0.0   0:00.00 rpciod/145172/5
   10 root      20   0     0    0    0 S  0.0  0.0   0:00.00 rpciod/145172/6
   11 root      20   0     0    0    0 S  0.0  0.0   0:00.00 rpciod/145172/7
   12 root      20   0     0    0    0 S  0.0  0.0   0:00.00 nfsiod/145172
  136 root      16  -4 10680    4    0 S  0.0  0.0   0:00.02 udevd
  483 root      20   0  244m 5072  400 S  0.0  0.2  26:11.69 rsyslogd
  736 root      20   0  135m 3680  652 S  0.0  0.1  40:34.74 munin-node
  967 root      20   0  4068    8    4 S  0.0  0.0   0:00.00 mingetty
  997 root      20   0  4068    8    4 S  0.0  0.0   0:00.00 mingetty
 2583 root      20   0  103m    8    4 S  0.0  0.0   0:00.00 inotifywait.sh
 2584 root      20   0 25720  18m  228 S  0.0  0.7   0:17.36 inotifywait

2. VmStat – Virtual Memory Statistics

VmStat command used to monitor virtual memory, kernal threada, disk, system process, etc.. By default it is not installed on the system, you will need to install package called sysstat which include vmstat program.

# vmstat
procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu-----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 0  0 263764 2341680      0 114912    0    0     0     1    0    0  1  0 99  0  0

3. Losf – List Open Files

Lsof command used to display list of all the open files and process. Open files include are network socket, disk files, pipes, devices and processes. Main use of this command is when a disk con not be unmounted and displays the error that files are being used or opened. Using this command you can easily identify which files are in use.

# lsof
COMMAND     PID      USER   FD      TYPE             DEVICE  SIZE/OFF       NODE NAME
init          1      root  cwd       DIR               0,39      4096    2501026 /
init          1      root  rtd       DIR               0,39      4096    2501026 /
init          1      root  txt       REG               0,39    150352    2495060  (deleted)/sbin/init
init          1      root  mem       REG              253,1              2513643 (deleted)/lib64/libnss_ldap.so.2 (stat: No such file or directory)
init          1      root  mem       REG              253,1              2503526 (deleted)/lib64/libnss_files-2.12.so (stat: No such file or directory)
init          1      root  mem       REG              253,1              2501184 (deleted)/lib64/libc-2.12.so (stat: No such file or directory)
init          1      root  mem       REG              253,1              2501476 (deleted)/lib64/libgcc_s-4.4.7-20120601.so.1;55e556dd (stat: No such file or directory)
init          1      root  mem       REG              253,1              2503555 (deleted)/lib64/librt-2.12.so (stat: No such file or directory)
init          1      root  mem       REG              253,1              2501134 (deleted)/lib64/libpthread-2.12.so (stat: No such file or directory)
init          1      root  mem       REG              253,1              2501158 (deleted)/lib64/libdbus-1.so.3.4.0 (stat: No such file or directory)
init          1      root  mem       REG              253,1              2501227 /lib64/libnih-dbus.so.1.0.0 (path dev=0,39)
init          1      root  mem       REG              253,1              2501248 /lib64/libnih.so.1.0.0 (path dev=0,39)
init          1      root  mem       REG              253,1              2501673 (deleted)/lib64/ld-2.12.so (stat: No such file or directory)
init          1      root  mem       REG              253,1              2495060 (deleted)/sbin/init (stat: No such file or directory)
init          1      root    0u      CHR                1,3       0t0    2501528 /dev/null
init          1      root    1u      CHR                1,3       0t0    2501528 /dev/null
init          1      root    2u      CHR                1,3       0t0    2501528 /dev/null
init          1      root    3r     FIFO                0,8       0t0 1485165868 pipe
init          1      root    4w     FIFO                0,8       0t0 1485165868 pipe
init          1      root    5r      DIR               0,10         0          1 inotify
init          1      root    6r      DIR               0,10         0          1 inotify
init          1      root    7u     unix 0xffff88023c4c4e40       0t0 1485165870 @/com/ubuntu/upstart

4. Tcpdump – Network Packet Analyzer

Tcpdump used to analyze network packet or it is also packet sniffer program that is used filter TCP/IP packets that received or transferred on a specific interface over a network.

# tcpdump -i venet0:0
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on venet0:0, link-type LINUX_SLL (Linux cooked), capture size 65535 bytes
19:35:33.847554 IP www.looklinux.com.sicct > ldap01.ldap: Flags [P.], seq 3454527163:3454527261, ack 1861271793, win 1731, length 98
19:35:33.878878 IP ldap01.ldap > www.looklinux.com.sicct: Flags [P.], seq 1:17, ack 98, win 126, length 16
19:35:33.878893 IP www.looklinux.com.sicct > ldap01.ldap: Flags [.], ack 17, win 1731, length 0
19:35:33.878959 IP www.looklinux.com.sicct > ldap01.ldap: Flags [P.], seq 98:251, ack 17, win 1731, length 153
19:35:33.910485 IP ldap01.ldap > www.looklinux.com.sicct: Flags [P.], seq 17:33, ack 251, win 126, length 16
19:35:33.911363 IP www.looklinux.com.31237 > ldap01.ldap: Flags [P.], seq 3375107161:3375107280, ack 3164352562, win 1750, length 119
19:35:33.942867 IP ldap01.ldap > www.looklinux.com.31237: Flags [P.], seq 1:120, ack 119, win 4150, length 119
19:35:33.942892 IP www.looklinux.com.31237 > ldap01.ldap: Flags [.], ack 120, win 1750, length 0
19:35:33.942873 IP ldap01.ldap > www.looklinux.com.31237: Flags [P.], seq 120:136, ack 119, win 4150, length 16
19:35:33.942909 IP www.looklinux.com.31237 > ldap01.ldap: Flags [.], ack 136, win 1750, length 0
19:35:33.950042 IP www.looklinux.com.sicct > ldap01.ldap: Flags [.], ack 33, win 1731, length 0

5. Netstat – Newtwork Statistics

Netstat is used to monitor incoming and outgoing network packets statistics as well as interface statistics. Mostly it is used to monitor network performance and troubleshot network related issue.

# netstat -a | more
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address               Foreign Address             State
tcp        0      0 *:https                     *:*                         LISTEN
tcp        0      0 *:down                      *:*                         LISTEN
tcp        0      0 localhost.locald:cslistener *:*                         LISTEN
tcp        0      0 *:memcache                  *:*                         LISTEN
tcp        0      0 *:http                      *:*                         LISTEN
tcp        0      0 *:81                        *:*                         LISTEN
tcp        0      0 www.looklinux.com:munin    *:*                         LISTEN
tcp        0      0 localhost.localdomain:smtp  *:*                         LISTEN
tcp        0      0 www.looklinux.com:sicct    ldap:ldap                 ESTABLISHED
tcp        0      0 www.looklinux.com:31936    ldap:ldap                 ESTABLISHED
tcp        0      0 www.looklinux.com:31237    ldap:ldap                 ESTABLISHED
tcp        0      0 www.looklinux.com:http     10.12.145.174:55448         TIME_WAIT
tcp        0      0 www.looklinux.com:25575    vsrv:mysql                TIME_WAIT
tcp        0      0 www.looklinux.com:31215    ldap:ldap                 ESTABLISHED
tcp        0      0 www.looklinux.com:25574    vsrv:mysql                TIME_WAIT
tcp        0      0 www.looklinux.com:36582    vsrv:mysql                TIME_WAIT
tcp        0      0 www.looklinux.com:36577    vsrv:mysql                TIME_WAIT
tcp        0      0 www.looklinux.com:http     www.looklinux.com:30355    TIME_WAIT
tcp        0      0 www.looklinux.com:http     www.looklinux.com:19353    TIME_WAIT
tcp        0      0 wwiw.looklinux.com:12227    ldap:ldap                 ESTABLISHED
tcp        0      0 *:memcache                  *:*                         LISTEN
udp        0      0 *:memcache                  *:*
udp        0      0 *:memcache                  *:*
Active UNIX domain sockets (servers and established)

6. Htop – Monitoring Linux Process

Htop is used to monitor real time Linux process. This is similar to Linux top command but it has some rich features. It is user friendly interface to manage process, shortcut keys, vertical and horizontal view of the processes. By default it does not comes with Linux systems, you need to install it using YUM package manager tool.

htop

7. Iotop – Linux Disk I/O Monitoring

Iotop is used to monitor real time Disk I/O and process. This is much similar to top command and Htop command. This tool is much useful for finding the exact process and high used disk Read/Writes of the processes.

# iotop
Total DISK READ: 127.33 K/s | Total DISK WRITE: 0.00 B/s
  TID  PRIO  USER     DISK READ  DISK WRITE  SWAPIN     IO>    COMMAND
 2661 be/2 apache    127.33 K/s    3.75 K/s  0.00 %  1.61 % httpd
 8532 be/2 apache      0.00 B/s    3.75 K/s  0.00 %  0.00 % httpd
10632 be/2 apache      0.00 B/s    3.75 K/s  0.00 %  0.00 % httpd
12724 be/2 apache      0.00 B/s    3.75 K/s  0.00 %  0.00 % httpd
 4540 be/2 apache      0.00 B/s    7.49 K/s  0.00 %  0.00 % httpd
21017 be/2 apache      0.00 B/s    3.75 K/s  0.00 %  0.00 % httpd
21025 be/2 apache      0.00 B/s    3.75 K/s  0.00 %  0.00 % httpd
  564 be/2 apache      0.00 B/s    3.75 K/s  0.00 %  0.00 % httpd
23135 be/2 apache      0.00 B/s    7.49 K/s  0.00 %  0.00 % httpd
24827 be/2 apache      0.00 B/s    3.75 K/s  0.00 %  0.00 % httpd
23927 be/2 apache      0.00 B/s    3.75 K/s  0.00 %  0.00 % httpd
15009 be/2 apache      0.00 B/s    3.75 K/s  0.00 %  0.00 % httpd
19141 be/2 apache      0.00 B/s    3.75 K/s  0.00 %  0.00 % httpd
  757 be/2 apache      0.00 B/s    3.75 K/s  0.00 %  0.00 % httpd
14480 be/2 apache      0.00 B/s    3.75 K/s  0.00 %  0.00 % httpd
19428 be/2 apache      0.00 B/s    3.75 K/s  0.00 %  0.00 % httpd
19281 be/2 apache      0.00 B/s    3.75 K/s  0.00 %  0.00 % httpd
15189 be/2 apache      0.00 B/s    3.75 K/s  0.00 %  0.00 % httpd
 7348 be/2 apache      0.00 B/s    7.49 K/s  0.00 %  0.00 % httpd
27776 be/2 apache      0.00 B/s    3.75 K/s  0.00 %  0.00 % httpd
23770 be/2 apache      0.00 B/s    3.75 K/s  0.00 %  0.00 % httpd
 7400 be/2 apache      0.00 B/s    3.75 K/s  0.00 %  0.00 % httpd

8. lostat – Input/Output Statistics

loStat is used to collect and show system input and output storage device statistics. This is mostly used to trace storage device performance issues including device, remote disks, local disks such as NFS (Network File System).

# iostat
Linux 2.6.32-042stab081.5 (www.looklinux.com)  08/26/2017      _x86_64_        (8 CPU)

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
          14.13    9.29    1.67    0.60    0.00   74.30

Device:            tps   Blk_read/s   Blk_wrtn/s   Blk_read   Blk_wrtn
sdb             120.57      1086.78      3415.39 119123008403 374365353408
sda             118.71      1080.47      3420.61 118431464278 374937409672
md127             0.00         0.00         0.00      12474     159282
md126             0.40         1.92         3.47  210365802  380585504
md125           162.92       227.88      3410.69 24977936773 373850361296
dm-0              0.67         2.13         3.27  233577856  358095816
dm-1            159.99       225.75      3407.43 24744356250 373492265480

9. IPTraf – Monitor Real Time IP LAN

IPTraf is used to monitor real time network (IP LAN) in Linux. It collects lots of information  such as IP traffic that passes over the network, with TCP flag information, ICMP details, TCP/UDP traffic breakdowns, TCP connection packet and byne counts.

10. Monit – Monitor Linux Process and Service

Monit is a free open source. It is web based process monitoring utility that automatically monitors and managers system processes, files, programs, directories, permissions, checksums and filesystems. It also monitor services like MySQL, Apache, Mail, FTP, ProFTP, Nginx and many more.

Thank you! for visiting Look Linux.

If you find this tutorial helpful please share with your friends to keep it alive. For more helpful topic browse my website www.looklinux.com. To become an author at Look Linux Submit Article. Stay connected to Facebook.

About the author

mm

Santosh Prasad

Hi! I'm Santosh and I'm here to post some cool article for you. If you have any query and suggestion please comment in comment section.

Leave a Comment