While The Planet is here to act as “remote
hands and eyes” for connectivity issues that might keep you from
accessing your server, as system
owner/administrator, it is your responsibility to troubleshoot and
resolve
load issues with your system.
Whenever you run into problems with your server slowing down or acting
sluggishly, you will need to take steps to first identify what is
causing the load, and then address those particular services.
‘top’ is the linux command used to view the server’s current To view the server’s local resource utilization stats, issue the following from the root prompt: top Example output: 11:30:03 up 56 days, 19:22, 1 user, load average: 37.37, 36.70, 36.57 First, examine the memory statistics from the beginning of the “top” output. Mem: 2055252k av, 2021508k You Swap: 2040244k av,
Of the total swap space available on the
utilization statistics (cpu and memory usage), breaking down each
process and its utilization state.
384 processes: 347 sleeping, 37 running, 0 zombie, 0 stopped
CPU states: cpu user
nice system irq softirq
iowait idle
total 65.3% 0.0%
34.3% 0.0% 0.3%
0.0% 0.0%
cpu00 61.5% 0.0%
37.1% 0.0% 1.2%
0.0% 0.0%
cpu01 66.6% 0.0%
33.0% 0.0% 0.0%
0.0% 0.2%
cpu02 65.9% 0.0%
34.0% 0.0% 0.0%
0.0% 0.0%
cpu03 67.0% 0.0%
32.9% 0.0% 0.0%
0.0% 0.0%
Mem: 2055252k av, 2034780k used, 20472k
free, 0k shrd, 86680k
buff
1482172k actv, 279112k in_d, 32636k in_c
Swap: 2040244k av, 60308k used, 1979936k
free
1064912k cached
PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME CPU COMMAND
22084 mysql 25 0 117M
117M 2120 R 3.6 5.8
64:00 2 mysqld
29038 mysql 25 0 117M
117M 2120 R 3.6 5.8
29:14 3 mysqld
9262 mysql 25 0 117M
117M 2120 R 3.4 5.8
28:09 0 mysqld
29883 mysql 25 0 117M
117M 2120 R 3.3 5.8
508:34 3 mysqld
22080 mysql 25 0 117M
117M 2120 R 3.3 5.8
198:59 0 mysqld
28351 mysql 25 0 117M
117M 2120 R 3.1 5.8
165:01 0 mysqld
19583 mysql 25 0 117M
117M 2120 R 3.0 5.8
201:11 0 mysqld
19820 mysql 25 0 117M
117M 2120 R 2.8 5.8
268:32 2 mysqld
22087 mysql 25 0 117M
117M 2120 R 2.8 5.8
167:27 0 mysqld
29957 mysql 25 0 117M
117M 2120 R 2.7 5.8
166:42 2 mysqld
30021 mysql 25 0 117M
117M 2120 R 2.7 5.8
201:15 2 mysqld
29873 mysql 25 0 117M
117M 2120 R 2.6 5.8
736:01 0 mysqld
There is a common misconception that the load average is the number to
watch. Load average is calculated as the number of processes in
the run queue at 1, 5 and 15 minute averages. It is completely
realistic to think that you can have a high load average and still have
a high idle time percentage. In cases such as these, the CPU(s)
are handling these processes before they’re having to wait on each
other to finish executing. You can also have a load average of
1.0 that is consuming the entire CPU, leaving next to nothing for the
system, much less any services or users that may be logged in. For these
reasons, it’s important to look at more than just the load average when
deciding how busy your server is. A combination of factors,
including how busy your hard drives are and how much network traffic
there is, can influence CPU performance. A complete picture of
the server will take these factors into consideration.
The processes are sorted, by default, from the top down, with the top process utilizing the most CPU cycles. In this example, you can see that mysql is using a great deal of the CPU. However, it is not unusual to see a system responding
quite slowly when the CPU does not appear to be under much stress.
Memory
is a key factor, and if your CPU is not being taxed, is likely the
place you should begin your investigation. Top can be sorted by hitting M or P (to sort by Memory or Processor, respectively).
In this example, if we hit M to sort by Memory, the output changes:
11:30:28 up 56 days, 19:22, 1 user, load average: 35.64, 36.33, 36.44
371 processes: 334 sleeping, 37 running, 0 zombie, 0 stopped
CPU states: cpu user
nice system irq softirq
iowait idle
total 66.6% 0.0%
32.8% 0.1% 0.3%
0.0% 0.0%
cpu00 67.1% 0.0%
31.9% 0.4% 0.4%
0.0% 0.0%
cpu01 64.5% 0.0%
35.4% 0.0% 0.0%
0.0% 0.0%
cpu02 66.3% 0.0%
33.6% 0.0% 0.0%
0.0% 0.0%
cpu03 68.6% 0.0%
30.4% 0.0% 0.8%
0.0% 0.0%
Mem: 2055252k av, 2021508k used, 33744k
free, 0k shrd, 86692k
buff
1466364k actv, 278952k in_d, 32636k in_c
Swap: 2040244k av, 60308k used, 1979936k
free
1065040k cached
PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME CPU COMMAND
29859 mysql 15 0 117M
117M 2120 S 0.0 5.8
0:49 0 mysqld
29860 mysql 15 0 117M
117M 2120 S 0.0 5.8
0:01 2 mysqld
29861 mysql 20 0 117M
117M 2120 S 0.0 5.8
0:00 2 mysqld
29862 mysql 15 0 117M
117M 2120 S 0.0 5.8
0:00 0 mysqld
29863 mysql 15 0 117M
117M 2120 S 0.0 5.8
0:00 2 mysqld
29864 mysql 15 0 117M
117M 2120 S 0.0 5.8
0:00 3 mysqld
29865 mysql 15 0 117M
117M 2120 S 0.0 5.8
0:26 2 mysqld
29866 mysql 15 0 117M
117M 2120 S 0.0 5.8
0:23 0 mysqld
29867 mysql 15 0 117M
117M 2120 S 0.0 5.8
0:00 0 mysqld
29868 mysql 15 0 117M
117M 2120 S 0.0 5.8
0:20 1 mysqld
29869 mysql 15 0 117M
117M 2120 S 0.0 5.8
2:56 1 mysqld
29873 mysql 25 0 117M
117M 2120 R 2.9 5.8
736:04 0 mysqld
29883 mysql 25 0 117M
117M 2120 R 3.4 5.8
508:36 0 mysqld
used, 33744k free, 0k
shrd, 86692k buff 1466364k actv, 278952k
in_d, 32636k in_c
can see that the total physical memory is 2055252k (roughly 2Gb), and
that 2021508k are used. This indicates that most of the physical memory
(RAM) has already been used, and the system is now dipping into your
swap space on the HDD (hard drive). If you examine the Swap usage,
you’ll see
60308k used, 1979936k
free
1065040k cached
HDD (again, roughly 2Gb), the system is currently only using roughly
60k. However, each time the swap is accessed, the hard drive is
accessed. This, in conjunction with all of the mysql processes (which,
as databases, have heavy IO operations on the drive already), means
that with all of the physical memory gone, and the hard drive being
accessed quite frequently, the system is going to perform slowly
- even if the CPU is relatively idle. As the swap space is used more
heavily, you can expect to see the IOWAIT rise as well, further slowing
the system.
While this example deals mainly with mysql,
there are any number of processes that could be causing load. httpd
(apache) is another fairly common culprit. If your symptoms show either
httpd or mysql as using a high number of system resources, you will
need to take steps to optimize those services - taking into account
your traffic expectations, coding, how your sites are configured, and
so forth, and also with regard to how many resources you have to use
(Memory and CPU).
http://support.theplanet.com/knowledgebase/users/kb.php?id=10140&category_id=0&sid2=
Please note that this is not meant to be an all-inclusive guide on
load, merely a primer on how to begin identifying problems. You might
also find “ps” useful:
Again, as system administrator, load
issues and optimization are ultimately your responsibility. If you would like The
Planet to troubleshoot your load issues, or optimize services causing
said issues, billable Administrative Time will be required.
If you’d like more information on Administrative Time, you can reference this article:
http://support.theplanet.com/knowledgebase/users/kb.php?id=10521&category_id=0&sid2=