Thursday, February 4, 2016

Oracle® 12c Database Configuration Assistant hangs on Oracle® Linux 7 in VirtualBox 5

I recently installed Oracle® Linux 7.2 on my VirtualBox 5.0.14 installation on Windows 7 in order to have a private playground for evaluating software. Afterwards I installed Oracle® 12c database version 12.1.0.2 on my new virtual Linux server. This went fine until I fired up Database Configuration Assistant (DBCA) in order to create a playground database. DBCA just hanged forever without consuming noticeable CPU. I kept it running for an entire night, but it never started fully up. It kept hanging at the splash screen as shown below:



Investigations

The log file $ORACLE_BASE/cfgtoollogs/dbca/trace.log_OraDB12Home1_<timestamp> showed the following:

[main] [ 2016-02-04 10:06:33.737 CET ] [Host.<init>:1090]  Begin tracing..
[main] [ 2016-02-04 10:06:34.971 CET ] [UIHost.createHelpSet:485]  HelpSetParseExceptionjava.lang.NullPointerException
[main] [ 2016-02-04 10:06:36.703 CET ] [Host.checkIfBigClusterAndHubNode:1710]  Not a cluster environment: exiting BigCluster Check
[main] [ 2016-02-04 10:06:36.708 CET ] [InventoryUtil.getOUIInvSession:349]  setting OUI READ level to ACCESSLEVEL_READ_LOCKLESS
[main] [ 2016-02-04 10:06:36.708 CET ] [InventoryUtil.isCRSHome:386]  Homeinfo /u01/app/oracle/product/12.1.0/dbhome_1,1
[main] [ 2016-02-04 10:06:36.922 CET ] [Host.validateGridHome:3878]  Validation false
[main] [ 2016-02-04 10:06:36.922 CET ] [Host.startOperation:2395]  Source db null
[main] [ 2016-02-04 10:06:36.922 CET ] [Host.startOperation:2396]  GDB Name null
[main] [ 2016-02-04 10:06:36.957 CET ] [Host.startOperation:2397]  MgmtDB sid -MGMTDB
[main] [ 2016-02-04 10:06:36.957 CET ] [Host.startOperation:2398]  MgmtDB name _mgmtdb
[main] [ 2016-02-04 10:06:36.998 CET ] [OracleHome.getVersion:991]  OracleHome.getVersion called.  Current Version: null
[main] [ 2016-02-04 10:06:37.000 CET ] [InventoryUtil.getOUIInvSession:349]  setting OUI READ level to ACCESSLEVEL_READ_LOCKLESS
[main] [ 2016-02-04 10:06:37.000 CET ] [OracleHome.getVersion:1010]  Homeinfo /u01/app/oracle/product/12.1.0/dbhome_1,1
[main] [ 2016-02-04 10:06:37.181 CET ] [OracleHome.getVersion:1038]  OracleHome.server.getVersion Version: 12.1.0.2.0
[main] [ 2016-02-04 10:06:37.182 CET ] [OracleHome.getVersion:1059]  Current Version From Inventory: 12.1.0.2.0
[main] [ 2016-02-04 10:06:37.182 CET ] [OracleHome.getVersion:991]  OracleHome.getVersion called.  Current Version: 12.1.0.2.0
[main] [ 2016-02-04 10:06:37.182 CET ] [OracleHome.getVersion:1059]  Current Version From Inventory: 12.1.0.2.0
[main] [ 2016-02-04 10:06:37.183 CET ] [CommonUtils.createPasswordFile:1243]  calling new orapwd for 11.1 or higher
[main] [ 2016-02-04 10:06:37.183 CET ] [OracleHome.getVersion:991]  OracleHome.getVersion called.  Current Version: 12.1.0.2.0
[main] [ 2016-02-04 10:06:37.184 CET ] [OracleHome.getVersion:1059]  Current Version From Inventory: 12.1.0.2.0
[main] [ 2016-02-04 10:06:37.184 CET ] [OracleHome.getVersion:991]  OracleHome.getVersion called.  Current Version: 12.1.0.2.0
[main] [ 2016-02-04 10:06:37.184 CET ] [OracleHome.getVersion:1059]  Current Version From Inventory: 12.1.0.2.0
[main] [ 2016-02-04 10:06:37.184 CET ] [CommonUtils.getPasswordFileCreateCmd:1182]  for new orapwd for 11.1 or higher
[main] [ 2016-02-04 10:06:37.189 CET ] [OracleHome.getVersion:991]  OracleHome.getVersion called.  Current Version: 12.1.0.2.0
[main] [ 2016-02-04 10:06:37.190 CET ] [OracleHome.getVersion:1059]  Current Version From Inventory: 12.1.0.2.0
[main] [ 2016-02-04 10:06:37.190 CET ] [OracleHome.getVersion:991]  OracleHome.getVersion called.  Current Version: 12.1.0.2.0
[main] [ 2016-02-04 10:06:37.190 CET ] [OracleHome.getVersion:1059]  Current Version From Inventory: 12.1.0.2.0
[main] [ 2016-02-04 10:06:37.191 CET ] [CommonUtils.getPasswordFileCreateCmd:1213]  /u01/app/oracle/product/12.1.0/dbhome_1/bin/orapwd
[main] [ 2016-02-04 10:06:37.191 CET ] [CommonUtils.getPasswordFileCreateCmd:1213]  file=/u01/app/oracle/product/12.1.0/dbhome_1/dbs/orapwDBUA0636958
[main] [ 2016-02-04 10:06:37.191 CET ] [CommonUtils.getPasswordFileCreateCmd:1213]  force=y
[main] [ 2016-02-04 10:06:37.191 CET ] [CommonUtils.getPasswordFileCreateCmd:1213]  format=12
[main] [ 2016-02-04 10:06:37.192 CET ] [OsUtilsBase.execProg:2123]  beginning execProg with input array.
[main] [ 2016-02-04 10:06:37.377 CET ] [OsUtilsBase.execProg:2160]  finished execProg with input array. Status:0
[main] [ 2016-02-04 10:06:37.378 CET ] [OracleHome.initOptionsStopOnError:1356]  Initializing Database Options with  for dummy sid=DBUA0636958 using initfile=/u01/app/oracle/product/12.1.0/dbhome_1/dbs/initDBUA0636958.ora using pwdfile=/u01/app/oracle/product/12.1.0/dbhome_1/dbs/orapwDBUA0636958
[main] [ 2016-02-04 10:06:37.398 CET ] [OracleHome.getVersion:991]  OracleHome.getVersion called.  Current Version: 12.1.0.2.0
[main] [ 2016-02-04 10:06:37.399 CET ] [OracleHome.getVersion:1059]  Current Version From Inventory: 12.1.0.2.0
[main] [ 2016-02-04 10:06:37.399 CET ] [OracleHome.getVersion:991]  OracleHome.getVersion called.  Current Version: 12.1.0.2.0
[main] [ 2016-02-04 10:06:37.400 CET ] [OracleHome.getVersion:1059]  Current Version From Inventory: 12.1.0.2.0
[main] [ 2016-02-04 10:06:37.400 CET ] [OracleHome.getVersion:991]  OracleHome.getVersion called.  Current Version: 12.1.0.2.0
[main] [ 2016-02-04 10:06:37.401 CET ] [OracleHome.getVersion:1059]  Current Version From Inventory: 12.1.0.2.0
[main] [ 2016-02-04 10:06:37.402 CET ] [SQLPlusEngine.getCmmdParams:225]  m_home 12.1.0.2.0
[main] [ 2016-02-04 10:06:37.402 CET ] [SQLPlusEngine.getCmmdParams:226]  version > 112 true
[main] [ 2016-02-04 10:06:37.403 CET ] [SQLEngine.getEnvParams:612]  NLS_LANG: AMERICAN_AMERICA.AL32UTF8
[main] [ 2016-02-04 10:06:37.421 CET ] [SQLEngine.initialize:358]  Execing SQLPLUS/SVRMGR process...
[main] [ 2016-02-04 10:06:37.467 CET ] [SQLEngine.initialize:395]  m_bReaderStarted: false
[main] [ 2016-02-04 10:06:37.467 CET ] [SQLEngine.initialize:399]  Starting Reader Thread...
[main] [ 2016-02-04 10:06:42.068 CET ] [OracleHome.initOptionsStopOnError:1370]  executing: startup nomount pfile='/u01/app/oracle/product/12.1.0/dbhome_1/dbs/initDBUA0636958.ora'


The 'ps -ef' command also confirms that sqlplus is stuck in "startup nomount"




Some unsuccessful attempts

- Tried with CentOS 7.1 instead of Oracle® Linux 7.2. Replicated the issue 100%.
- Tried to run DBCA as part of the Oracle® 12c Database installation (Software and starter database) and after a software only install. Same problem in both cases. 

Searching for an answer

I found a similar issue described in https://community.oracle.com/message/13249304#13249304, but no solution was offered, 

The solution

After various unsuccessful attempts to fix this at Linux level, I turned my attention to VirtualBox. After trying to tweak several options, I found the culprit to be the Paravirtualization Interface for acceleration. Changing that from Default to None fixed the issue.




I did not try all the various options for Paravirtualization Interface, but in my setup both with Default and Legacy I faced issues with DBCA.







15 comments:

  1. Thank you! Thank you! Thank you! Thank you! Thank you! Thank you! Thank you! Thank you! Thank you! Thank you! Thank you!

    This issue was driving me absolutely crazy and you are the only one in the whole internet with the solution!

    Carlos F. Turienzo

    ReplyDelete
    Replies
    1. Glad to be of assistance :) I also struggled with that issue for a few days until I started looking at the virtualizer.

      Delete
  2. Audun, you've just gave us an amazing lesson on troubleshooting. Thanks for sharing. We were stuck dealing with this problem for many hours without being able of getting any solution in the short term. Best regards!

    Irving Perez

    ReplyDelete
  3. Thank you so much!! I have been moving in circles trying to fix this issue. This did the trick!

    ReplyDelete
  4. THANK YOU, after many days and returning back to searching, this solved the issue! Really couldn't find anywhere

    ReplyDelete
  5. Hi Audun, Thanks so much for sharing the fix, appreciate it very much

    ReplyDelete
  6. This comment has been removed by the author.

    ReplyDelete
  7. Thanks a lot Audun. After a week of running in circles, trying oel 7, oel 6, mixing oracle GI 12 and GI 11, I find your workaround and it made the miracle. It affects not only RDBMS, but GI install also.
    As I read here https://patrickhurley.wordpress.com/2015/11/30/freeze-of-oracle-dbca-and-startup-nomount-running-on-linux-on-virtualbox-5/, It seems to be related to AMD processors.
    Thanks again

    ReplyDelete
  8. This comment has been removed by the author.

    ReplyDelete
  9. Thank you so much sir... yes troubleshooting is amazing... issue is resolved for me.

    ReplyDelete
  10. Hi Sir,

    I am trying to create db using dbca but it hangs at nomount stage.
    My os is windows 2008 R2 Enterprise Sp1. Please help me regarding the same.

    ReplyDelete