问题描述:
10月25日上午滨州网通的工程师报告oss应用系统运行缓慢,具体操作是通过oss系统查询话单,很长时间才能返回结果,严重影响了客户的正常使用。
问题处理:
1.登陆数据库主机,用sar命令看到idle的值持续为0,cpu的资源已经耗尽:
bz_db1# sar 2 4
sunos kest 5.8 generic_108528-19 sun4u 10/26/04
10:56:46 %usr %sys %wio %idle
10:56:48 1 4 95 0
10:56:50 1 5 94 0
10:56:52 0 6 93 0
10:56:54 1 6 93 0
average 1 5 94 0
2.使用top命令看到有两个明显占用cpu利用率过高的进程,以下是top命令的结果:
bz_db1# top
last pid: 1664;load averages: 3.26, 3.24, 3.69
159 processes: 152 sleeping, 2 running, 2 zombie, 1 stopped, 2 on cpu
cpu states: 1.5% idle, 72.5% user, 17.9% kernel, 8.0% iowait, 0.0% swap
memory: 2.0g real, 233m free, 2.0g swap in use, 3.4g swap free
pid username thr pr nce size res state time flts cpu command
27420 oracle 1 10 0 1.3g 1.2g cpu01 22.9h 2 31.94% oracle
27418 oracle 1 10 0 1.3g 1.2g run 23.0h 6 26.86% oracle
5943 oracle 1 59 0 1.3g 1.2g sleep 25:26 37 4.92% oracle
6295 oracle 1 55 0 1.3g 1.2g run 25:14 74 4.90% oracle
7778 oracle 1 43 0 1.3g 1.2g sleep 11:43 110 4.86% oracle
13270 oracle 1 59 0 1.3g 1.2g sleep 210.6h 0 0.96% oracle
13056 oracle 1 48 0 1.3g 1.2g sleep 303:30 0 0.37% oracle
10653 root 1 58 0 2560k 1624k cpu00 0:00 0 0.32% top
18827 oracle 1 58 0 1.3g 1.2g sleep 18.4h 0 0.31% oracle
12748 oracle 258 58 0 1.3g 1.2g sleep 555:14 0 0.21% oracle
10634 oracle 1 59 0 1.3g 1.2g sleep 0:01 0 0.21% oracle
28458 oracle 1 58 0 1.3g 1.2g sleep 535:02 0 0.18% oracle
13075 oracle 1 59 0 1.3g 1.2g sleep 326:33 0 0.15% oracle
13173 oracle 1 58 0 1.3g 1.2g sleep 593:07 0 0.13% oracle
4927 oracle 1 59 0 1.3g 1.2g sleep 33.4h 0 0.11% oracle
可以看到这两个进程号分别是27420和27418.
3.捕获占用cpu利用率过高的sql语句:
以下用到了我总结的sql语句:
sql>set line 240
sql>set verify off
sql>column sid format 999
sql>column pid format 999
sql>column s_# format 999
sql>column username format a9 heading "ora user"
sql>column program format a29
sql>column sql format a60
sql>column osname format a9 heading "os user"
sql>select p.pid pid,s.sid sid,p.spid spid,s.username username,
s.osuser osname,p.serial# s_#,p.terminal,p.program program,
p.background,s.status,rtrim(substr(a.sql_text, 1, 80)) sql
from v$process p, v$session s,v$sqlarea a where p.addr = s.paddr