#90 Days DevOps Challenge#Zero To Hero#Day3💻 Challenge Accepted with #tws by Shubham Londhe 👨‍🏫
1. top
top is one of the most frequently used commands by Linux administrators. At a very high level, the command shows the system uptime, CPU usage, number of threads, memory usage (total, used, free, etc.), a list of running processes, and much more.
shows that there’s a lot of information here, and a lot of options to filter the task list, kill processes, sort the list, and more. For example, we can sort the task list by the %CPU or %MEM columns to easily find out which processes are consuming the most CPU or memory. Use these commands to sort tasks by memory or CPU percentage:
Copytop -o +%MEM
top -o +%CPU
shows the processes sorted by the percentage of memory used in descending order.
Sorting by CPU usage or memory consumption lets us identify processes that could be causing performance issues. We can then easily kill those processes from the same top user interface.
2. vmstat
vmstat tells us everything we need to know about virtual memory. The system starts using virtual memory when it runs out of physical memory. Therefore, to start with, virtual memory will always be zero. Along with virtual memory stats, vmstat gives us a lot more information about system processes, interrupts, block I/O operations, disks, paging, CPU scheduling, and more.
the output of vmstat without any options passed. The command is:
Copyvmstat
As you can see, it's not as dense as the top UI. But with just a few options, we can still get all the information we need. For example, passing the option a gives us the active and inactive memory info The command is:
Copyvmstat -a
Passing the option will give us all the information we need about CPU scheduling, . The command is:
Copyvmstat -s
3. lsof
In Linux, everything depends on files. For example, the network adapters—and even any USB accessory that we plug in—are all controlled using files. So, any issues with hardware, or even software on a Linux machine has to be debugged using files. lsof (or list of open files) is a handy command to quickly see the list of open files and associated processes.
Below shows a truncated list of the lsof command output. This table gives us a lot of information, including the command used to run the process that owns the file, the PID, the user who owns the process, the type and size of the file, and more.
We can use various options with the command to filter files for a particular user, or by files used by a certain port, and so on. These include:
Command to filter files by owner:
lsof -u rootCommand to filter files of processes listening to a certain port:
lsof -i TCP:22
4. tcpdump
To debug network issues, or to check traffic of any specific app or service on machines, use the command, tcpdump. Figure 8 shows a screenshot of the command’s output.
As evident from the screenshot, tcpdump provides source IP, destination IP, type of protocol used, amount of bytes transferred, and more. This information is useful when trying to trace a network call or check for unusual network activity.
This command also comes in handy to check if a package or service is making calls to unauthorized hosts or IP addresses. If so, it also checks whether any unauthorized sensitive data is being shared that shouldn’t be, or see if any extra packages are being downloaded that could result in security risks.
5. netstat
netstat is a command used to identify open ports, connections to external IP addresses, and the connection status. Figure 10 shows a screenshot of the output of the command.
The command provides the protocol used, the local and foreign IP addresses, and also the state of the connection. This helps us find out if a process has all of its ports open, and if they’re being used or not. This can help debug many network problems.
This command also helps debug potential network security issues. There have been many reported instances of security breaches where a network port has been made open on the public internet, leading to an attack. So, whenever we install or deploy a new service, we need to ensure there are no unnecessarily open ports that might result in security risks.
Using netstat, we can also check the network routing table.
6. iostat
As we’ve already mentioned, everything in a Linux computer is controlled using files. These files are written to the disks attached to the computer. Whenever a process runs, along with CPU and memory, the process also consumes disk bandwidth. In other words, every process performs several I/O (input/output) operations every second. And much like other resources, there's a limit to this bandwidth.
iostat is used to monitor such I/O activity on all disks and partitions on a computer. It also provides the CPU utilization for such operations. Using iostat , we can decide if we need to modify system configuration to allow for better or balanced I/O operations.
As you can see, the first section of the output is the average CPU usage divided into sections, including userspace, system space, CPU steal, CPU idle, and I/O wait. The %iowait column tells us if the CPU is wasting a lot of time waiting for I/O operations to complete. A high number here would indicate that the I/O is slow, or is being held up in some other process.
The next section lists all the devices attached to the computer, and the corresponding TPS (transfers per second), kilobytes read and write numbers, and much more. These numbers will tell us if further tuning is required to improve I/O performance.
7. iotop
Just as we would use top for monitoring processes, we can use iotop for monitoring all threads that are performing I/O operations and the bandwidth they consume.
The top left section of this table gives us the total and actual disk read, while the top right section gives the total and actual disk writes. Under these sections, we have a list of threads running on the computer along with the thread ID (TID), the priority of the thread, the owner (user) of the thread, disk read and write bandwidth, percentage of time spent on swap in, percentage of time spent on I/O wait, as well as the command for the thread.
On the other hand, you will also notice that not all these threads perform I/O operations. To filter the list to show only such threads, use the option o or only, as shown below:
Copyiotop -o
or
Copyiotop --only
This command helps us to see the list of threads that take up too much of the I/O bandwidth and cause other threads to wait on I/O, thereby halting other processes or users.
Summary
There are powerful commands available in Linux to monitor and debug any kind of performance issues. However, merely knowing the syntax and the options of these commands is not enough. We also need to know which command to use in a given scenario. This list should provide a headstart for Linux system administrators to better understand their computers.
Thank You Shubham Londhe Sir for supporting and guidance :)
Subscribe to my newsletter
Read articles from directly inside your inbox. Subscribe to the newsletter Akash Raj and don't miss out.
SUBSCRIBE Akash Raj
©2023 Akash Raj @dreamdevopsakash
Archive·Privacy policy·Terms
Powered by Hashnode - Home for tech writers and readers