首页 » ORACLE » oracle僵死Process m000 died,J000 died

oracle僵死Process m000 died,J000 died

今天无意执行了一个脚本,后来发现kill 了j000和m000的进程,oracle被僵死在那里,j000是job的守护进程,m000是MMON进程启动的从属平行进程,都是轻量级但不容忽视!

[oracle@orazhang bdump]$ echo $ORACLE_SID
orcl
[oracle@orazhang bdump]$ sqlplus / as sysdba

SQL*Plus: Release 10.2.0.1.0 - Production on 星期三 6月 15 14:04:33 2011

Copyright (c) 1982, 2005, Oracle.  All rights reserved.

Connected to an idle instance.

idle> exit
Disconnected
[oracle@orazhang bdump]$ ps -ef|grep ora_
oracle    3453     1  0 Jun13 ?        00:00:00 ora_pmon_orcl
oracle    3455     1  0 Jun13 ?        00:00:00 ora_psp0_orcl
oracle    3457     1  0 Jun13 ?        00:00:00 ora_mman_orcl
oracle    3459     1  0 Jun13 ?        00:00:01 ora_dbw0_orcl
oracle    3461     1  0 Jun13 ?        00:00:02 ora_lgwr_orcl
oracle    3463     1  0 Jun13 ?        00:00:00 ora_ckpt_orcl
oracle    3465     1  0 Jun13 ?        00:00:03 ora_smon_orcl
oracle    3467     1  0 Jun13 ?        00:00:00 ora_reco_orcl
oracle    3469     1  0 Jun13 ?        00:00:01 ora_cjq0_orcl
oracle    3471     1  0 Jun13 ?        00:00:00 ora_mmon_orcl
oracle    3473     1  0 Jun13 ?        00:00:00 ora_mmnl_orcl
oracle    3475     1  0 Jun13 ?        00:00:00 ora_d000_orcl
oracle    3477     1  0 Jun13 ?        00:00:00 ora_s000_orcl
oracle    3485     1  0 Jun13 ?        00:00:00 ora_arc0_orcl
oracle    3487     1  0 Jun13 ?        00:00:00 ora_arc1_orcl
oracle    3491     1  0 Jun13 ?        00:00:00 ora_qmnc_orcl
oracle    3673     1  0 Jun13 ?        00:00:00 ora_q000_orcl
oracle    3677     1  0 Jun13 ?        00:00:00 ora_q001_orcl
oracle   29600 29164  0 14:04 pts/0    00:00:00 grep ora_

[oracle@orazhang bdump]$ tail alert_orcl.log -n 100
Thread 1 advanced to log sequence 20
Thread 1 opened at log sequence 20
Current log# 5 seq# 20 mem# 0: /u01/app/oracle/oradata/ORCL/onlinelog/redo5.log
Current log# 5 seq# 20 mem# 1: /u01/app/oracle/flash_recovery_area/ORCL/onlinelog/redo5.log
.........省略......

Wed Jun 15 13:59:57 2011
Process J000 died, see its trace file
Wed Jun 15 13:59:57 2011
kkjcre1p: unable to spawn jobq slave process
Wed Jun 15 13:59:57 2011
Errors in file /u01/app/oracle/admin/orcl/bdump/orcl_cjq0_3469.trc:

Process J000 died, see its trace file
Wed Jun 15 14:00:03 2011
kkjcre1p: unable to spawn jobq slave process
Wed Jun 15 14:00:03 2011
Errors in file /u01/app/oracle/admin/orcl/bdump/orcl_cjq0_3469.trc:

Wed Jun 15 14:00:55 2011
Process m000 died, see its trace file
Wed Jun 15 14:00:55 2011
ksvcreate: Process(m000) creation failed
Wed Jun 15 14:01:39 2011
Process J000 died, see its trace file
Wed Jun 15 14:01:39 2011
kkjcre1p: unable to spawn jobq slave process
Wed Jun 15 14:01:39 2011
Errors in file /u01/app/oracle/admin/orcl/bdump/orcl_cjq0_3469.trc:

Wed Jun 15 14:01:56 2011
Process m000 died, see its trace file
Wed Jun 15 14:01:56 2011
ksvcreate: Process(m000) creation failed
Wed Jun 15 14:02:57 2011
Process m000 died, see its trace file
Wed Jun 15 14:02:57 2011
ksvcreate: Process(m000) creation failed
Wed Jun 15 14:03:58 2011
Process m000 died, see its trace file
Wed Jun 15 14:03:58 2011
ksvcreate: Process(m000) creation failed
Wed Jun 15 14:04:59 2011
Process m000 died, see its trace file
Wed Jun 15 14:04:59 2011
ksvcreate: Process(m000) creation failed
[oracle@orazhang bdump]$
[oracle@orazhang bdump]$ cat /u01/app/oracle/admin/orcl/bdump/orcl_cjq0_3469.trc
/u01/app/oracle/admin/orcl/bdump/orcl_cjq0_3469.trc
Oracle Database 10g Enterprise Edition Release 10.2.0.1.0 - Production
With the Partitioning, OLAP and Data Mining options
ORACLE_HOME = /u01/app/oracle/product/10.2.0/db_1
System name:    Linux
Node name:      orazhang
Release:        2.6.18-53.el5xen
Version:        #1 SMP Wed Oct 10 17:06:12 EDT 2007
Machine:        i686
Instance name: orcl
Redo thread mounted by this instance: 1
Oracle process number: 10
Unix process pid: 3469, image: oracle@orazhang (CJQ0)

*** SERVICE NAME:(SYS$BACKGROUND) 2011-06-15 13:59:57.946
*** SESSION ID:(162.1) 2011-06-15 13:59:57.946
*** 2011-06-15 13:59:57.946
Process J000 is dead (pid=29552, state=3):
*** 2011-06-15 14:00:03.954
Process J000 is dead (pid=29556, state=3):
*** 2011-06-15 14:01:39.960
Process J000 is dead (pid=29574, state=3):

[oracle@orazhang bdump]$ ipcs -sa

------ Shared Memory Segments --------
key        shmid      owner      perms      bytes      nattch     status
0x00000000 0          oracle    640        1258295296 18         dest
0x00000000 32769      oracle    600        393216     2          dest
0x00000000 65538      oracle    600        393216     2          dest
0x00000000 98307      oracle    600        393216     2          dest
0x00000000 131076     oracle    600        393216     2          dest
0x00000000 163845     oracle    600        393216     2          dest
0x00000000 196614     oracle    600        393216     2          dest
0x00000000 229383     oracle    600        393216     2          dest
0x00000000 262152     oracle    600        393216     2          dest
0x00000000 294921     oracle    600        393216     2          dest
0x00000000 393226     oracle    600        393216     2          dest
0x00000000 360459     oracle    600        393216     2          dest

------ Semaphore Arrays --------
key        semid      owner      perms      nsems
0xf5d4b884 98304      oracle    640        154

------ Message Queues --------
key        msqid      owner      perms      used-bytes   messages

[root@orazhang ~]# killall -9 -r "ora_*"
[root@orazhang ~]# ps -ef|grep ora_
root     29973 29760  0 15:06 pts/0    00:00:00 grep ora_
[root@orazhang ~]# ipcs -as

------ Shared Memory Segments --------
key        shmid      owner      perms      bytes      nattch     status
0x00000000 32769      oracle    600        393216     2          dest
0x00000000 65538      oracle    600        393216     2          dest
0x00000000 98307      oracle    600        393216     2          dest
0x00000000 131076     oracle    600        393216     2          dest
0x00000000 163845     oracle    600        393216     2          dest
0x00000000 196614     oracle    600        393216     2          dest
0x00000000 229383     oracle    600        393216     2          dest
0x00000000 262152     oracle    600        393216     2          dest
0x00000000 294921     oracle    600        393216     2          dest
0x00000000 393226     oracle    600        393216     2          dest
0x00000000 360459     oracle    600        393216     2          dest

------ Semaphore Arrays --------
key        semid      owner      perms      nsems
0xf5d4b884 98304      oracle    640        154

------ Message Queues --------
key        msqid      owner      perms      used-bytes   messages

sql > startup
ORACLE instance started.

Total System Global Area 1258291200 bytes
Fixed Size                  1219184 bytes
Variable Size             385877392 bytes
Database Buffers          855638016 bytes
Redo Buffers               15556608 bytes
Database mounted.
Database opened.
打赏

目前这篇文章有1条评论(Rss)评论关闭。

  1. Nathaniel Suganuma | #1
    2011-12-21 at 16:59

    Many thanks for sharing this first-class write-up. Very interesting ideas! (as always, btw)