Instance Hangs data gather

• If the instance is hanging and you can't log in to any instance, please do one of these 2 options instead:

1) Use "sqlplus -prelim / as sysdba" to get into sqlplus on one instance, and run the oradebug statements 2-3 times - 1 minute apart:

SQL> oradebug setmypid
SQL> oradebug unlimit
SQL> oradebug -g all hanganalyze 3
SQL> oradebug -g all dump systemstate 267

This only needs to be done on one instance; the "-g all" will kick off collection on all the other instances.

-or-

2) Capture systemstate dumps from each node with an OS debugger (gdb):

ps -ef |grep diag
gdb $ORACLE_HOME/bin/oracle
print ksudss(266)

This must be done on each node.

RAC/CRS Troubleshooting

• Document 289690.1 Data Gathering for Troubleshooting CRS Issues 
• Document 556679.1 Data Gathering for Troubleshooting RAC Issues
• Document 135714.1 Script to Collect RAC Diagnostic Information (racdiag.sql)

• Document 399482.1 How to recreate OCR/Voting disk accidentally deleted 
• Document 265769.1 Troubleshooting CRS Reboots 
• Document 357808.1 Diagnosability for CRS / EVM / RACG 
• Document 559365.1 Using Diagwait as a diagnostic to get more information for diagnosing Oracle Clusterware Node evictions 
• Document 341788.1 Recommendation for the Real Application Cluster Interconnect and Jumbo Frames 
• Document 279793.1 How to Restore a Lost Voting Disk in 10g 
• Document 219361.1 Troubleshooting ORA-29740 in a RAC Environment 

• Document 339939.1 Running Cluster Verification Utility to Diagnose Install 
• Document 316817.1 CLUSTER VERIFICATION UTILITY FAQ 
• Document 428681.1 OCR / Vote disk Maintenance Operations: (ADD/REMOVE/REPLACE/MOVE), including moving from RAW Devices to Block Devices 
• Document 363254.1 Applying one-off Oracle Clusterware patches in a mixed version home environment 
• Document 332257.1 Using Oracle Clusterware with Vendor Clusterware FAQ
• Document 759895.1 The ONS Daemon Explained In RAC/CRS environment
• Document 783456.1 CRS Diagnostic Data Gathering: A Summary of Common tools and their Usage