0209天津rac问题总结1

 
一、查看rac两结点的alert日志文件，进行对比发现报错信息起始于WedJan2102:51:512009。 
alert_FUJITSU2.log 
WedJan2102:51:512009 
Errorsinfile/oracle/admin/FUJITSU/bdump/fujitsu2_dbw0_26749.trc: 
ORA-01157: 无法标识/锁定数据文件17-请参阅DBWR跟踪文件 
ORA-01110: 数据文件17:'/dev/md/oraset/rdsk/d104' 
ORA-27041: 无法打开文件 
SVR4Error:13:Permissiondenied 
Additionalinformation:2 
WedJan2102:51:512009 
Errorsinfile/oracle/admin/FUJITSU/bdump/fujitsu2_dbw0_26749.trc: 
ORA-01186: 文件17验证测试失败 
ORA-01157: 无法标识/锁定数据文件17-请参阅DBWR跟踪文件 
ORA-01110: 数据文件17:'/dev/md/oraset/rdsk/d104' 
WedJan2102:51:512009 
File17notverifiedduetoerrorORA-01157 
WedJan2102:51:512009 
Errorsinfile/oracle/admin/FUJITSU/bdump/fujitsu2_dbw0_26749.trc: 
ORA-01157: 无法标识/锁定数据文件17-请参阅DBWR跟踪文件 
ORA-01110: 数据文件17:'/dev/md/oraset/rdsk/d104' 
ORA-27041: 无法打开文件 
SVR4Error:13:Permissiondenied 
Additionalinformation:2 
WedJan2102:51:512009 
Errorsinfile/oracle/admin/FUJITSU/bdump/fujitsu2_dbw0_26749.trc: 
ORA-01186: 文件17验证测试失败 
ORA-01157: 无法标识/锁定数据文件17-请参阅DBWR跟踪文件 
ORA-01110: 数据文件17:'/dev/md/oraset/rdsk/d104' 
File17notverifiedduetoerrorORA-01157 
…… 
WedJan2103:08:462009 
Errorsinfile/oracle/admin/FUJITSU/bdump/fujitsu2_dbw0_14533.trc: 
ORA-01186:??17?????? 
ORA-01157:????/??????17-???DBWR???? 
ORA-01110:????17:'/dev/md/oraset/rdsk/d104' 
WedJan2103:08:462009 
File17notverifiedduetoerrorORA-01157 
WedJan2103:08:462009 
Thread2openedatlogsequence2 
Currentlog#4seq#2mem#0:/dev/md/oraset/rdsk/d87 
Currentlog#4seq#2mem#1:/dev/md/oraset/rdsk/d88 
Successfulopenofredothread2 
WedJan2103:08:462009 
MTTRadvisoryisdisabledbecauseFAST_START_MTTR_TARGETisnotset 
WedJan2103:08:462009 
SMON:enablingcacherecovery 
WedJan2103:08:472009 
SuccessfullyonlinedUndoTablespace5. 
WedJan2103:08:472009 
SMON:enablingtxrecovery 
WedJan2103:08:472009 
DatabaseCharactersetisZHS16CGB231280 
replication_dependency_trackingturnedoff(noasyncmultimasterreplicationfound) 
StartingbackgroundprocessQMNC 
QMNCstartedwithpid=25,OSid=14849 
WedJan2103:08:492009 
Completed:ALTERDATABASEOPEN 
WedJan2103:09:382009 
Errorsinfile/oracle/admin/FUJITSU/bdump/fujitsu2_mmon_14576.trc: 
ORA-00600: 内部错误代码,参数:[ktte_append_file_info-1],[11],[15],[11],[15],[],[],[] 
WedJan2103:09:382009 
Tracedumpingisperformingid=[cdmp_20090121030938] 
WedJan2103:13:482009 
RestartingdeadbackgroundprocessMMON 
MM,OSid=23704 
WedJan2103:13:512009 
Errorsinfile/oracle/admin/FUJITSU/bdump/fujitsu2_mmon_23704.trc: 
ORA-00600: 内部错误代码,参数:[ktte_append_file_info-1],[11],[15],[11],[15],[],[],[] 
…… 
WedJan2118:49:052009 
Errorsinfile/oracle/admin/FUJITSU/bdump/fujitsu2_mmon_27174.trc: 
ORA-00600: 内部错误代码,参数:[ktte_append_file_info-1],[11],[15],[11],[15],[],[],[] 
WedJan2118:49:052009 
Tracedumpingisperformingid=[cdmp_20090121184905] 
WedJan2118:50:152009 
Error:KGXGNabortstheinstance(6) 
WedJan2118:50:152009 
Errorsinfile/oracle/admin/FUJITSU/bdump/fujitsu2_lmon_5214.trc: 
ORA-29702: 簇组服务操作中出现错误 
LMON:terminatinginstanceduetoerror29702 
WedJan2118:50:152009 
Systemstatedumpismadeforlocalinstance 
SystemStatedumpedtotracefile/oracle/admin/FUJITSU/bdump/fujitsu2_diag_5181.trc 
WedJan2118:50:152009 
Tracedumpingisperformingid=[cdmp_20090121185015] 
WedJan2118:50:192009 
InstanceterminatedbyLMON,pid=5214 
WedJan2118:55:572009 
StartingORACLEinstance(normal) 
alert_FUJITSU1.log 
WedJan2102:48:092009 
altertablespacexczhadddatafile'/dev/md/oraset/rdsk/d104'size29998M 
WedJan2102:51:512009 
Completed:altertablespacexczhadddatafile'/dev/md/oraset/rdsk/d104'size29998M 
WedJan2102:53:432009 
ALTERSYSTEMSETservice_names='FUJITSU'SCOPE=MEMORYSID='FUJITSU1'; 
WedJan2102:54:132009 
StartingbackgroundprocessEMN0 
EMN0startedwithpid=27,OSid=24039 
…… 
WedJan2118:50:402009 
Errorsinfile/oracle/admin/FUJITSU/bdump/fujitsu1_lmon_5185.trc: 
ORA-29702: 簇组服务操作中出现错误 
LMON:terminatinginstanceduetoerror29702 
WedJan2118:50:402009 
Errorsinfile/oracle/admin/FUJITSU/bdump/fujitsu1_lms2_5226.trc: 
ORA-29702: 簇组服务操作中出现错误 
WedJan2118:50:422009 
Systemstatedumpismadeforlocalinstance 
SystemStatedumpedtotracefile/oracle/admin/FUJITSU/bdump/fujitsu1_diag_5181.trc 
WedJan2118:50:432009 
Tracedumpingisperformingid=[cdmp_20090121185042] 
WedJan2118:50:442009 
InstanceterminatedbyLMON,pid=5185 
WedJan2118:55:022009 
StartingORACLEinstance(normal)