Wednesday, November 23, 2016

Customize Fedora 25 in a few minutes using Ansible

Instead of spending a few hours after installing Fedora 25 on adding the software you want and tweaking the options you prefer, why not use Ansible to get you up and running in minutes?

I assume you have already downloaded Fedora 25 Workstation from the official site at https://getfedora.org/, and that you have installed it, and that your personal user is allowed to run sudo (hint: being a member of the wheel group).


After logon to Fedora 25, download my Ansible playbook sample from https://github.com/avnes/blogsamples/blob/master/fedora.yml 

The rest of this article assumes you saved the file to ~/fedora.yml

Open ~/fedora.yml in a text editor (for instance vi or Gedit) and have look at what software it is installing, and what software it removes. Change it if you like.

When you are ready to play, run the following commands as your personal user:
sudo dnf install -y ansible
ansible-playbook ~/fedora.yml --connection=local --ask-become-pass

When promoted for a password, type in your personal password, and the playbook will then be executed with sudo rights. Both the sudo command and the ansible.playbook command will prompt you for your personal password.

Afterwards you should see a similar output as below, though changed would be higher than 0. In my case of is 0 because I have already run the playbook a few times already, so no changes.




Wednesday, November 9, 2016

How to install Vagrant with Oracle® VM VirtualBox and libvirt support on Antergos Linux

Installation

I recently needed to install Vagrant with support for Oracle® VM VirtualBox and libvirt for my dixie project at GitHub. Some of the virtualbox dependencies was found in this excellent post over at forum.antergos.com. Then I wrapped it into a script, and the here is the part of the script that installs and configures it:

if [ $(uname -r | grep ARCH | wc -l) -gt 0 ]; then
  sudo pacman -S --needed --noconfirm vagrant
  sudo pacman -S --needed --noconfirm libvirt
  sudo pacman -S --needed --noconfirm linux-headers
  sudo pacman -S --needed --noconfirm virtualbox virtualbox-guest-iso
  sudo pacman -S --needed --noconfirm vde2 net-tools virtualbox-ext-vnc virtualbox-host-modules-arch
  sudo pacman -S --needed --noconfirm ansible rsync
  sudo su -c "modprobe vboxdrv" || echo "Reboot your computer and try again"
  if [ ! -f "/etc/modules-load.d/virtualbox.conf" ]; then
    sudo touch /etc/modules-load.d/virtualbox.conf
  fi

  if [ $(grep vboxdrv /etc/modules-load.d/virtualbox.conf | wc -l) -eq 0 ]; then
    sudo su -c 'echo "vboxdrv" >> /etc/modules-load.d/virtualbox.conf'
  fi

  if [ $(grep vboxnetadp /etc/modules-load.d/virtualbox.conf | wc -l) -eq 0 ]; then
    sudo su -c 'echo "vboxnetadp" >> /etc/modules-load.d/virtualbox.conf'
  fi
  if [ $(grep vboxnetflt /etc/modules-load.d/virtualbox.conf | wc -l) -eq 0 ]; then
    sudo su -c 'echo "vboxnetflt" >> /etc/modules-load.d/virtualbox.conf'
  fi
  if [ $(grep vboxpci /etc/modules-load.d/virtualbox.conf | wc -l) -eq 0 ]; then
    sudo su -c 'echo "vboxpci" >> /etc/modules-load.d/virtualbox.conf'
  fi
fi

Install Ruby with gem support on Red Hat® Enterprise Linux

This post is heavily influenced by http://collectiveidea.com/blog/archives/2011/10/31/install-ruby-193-with-libyaml-on-centos/, so all credit to that author.

The problem

Installing a Ruby gem with gem install on  Red Hat® Enterprise Linux 6.8 returns errors indicating problems with libyaml:

gem install bundler
/usr/local/lib/ruby/1.9.1/yaml.rb:84:in `<top (required)>':
It seems your ruby installation is missing psych (for YAML output).
To eliminate this warning, please install libyaml and reinstall your ruby.

The investigation

Trying to reinstall libyaml and libyaml-devel followed by a reinstall of ruby did not solve the issue.

The solution

After investigating this issue for a while, and not able to solve it through yum, I considered to either install and use rvm for this, or to download and compile the source. I found  http://collectiveidea.com/blog/archives/2011/10/31/install-ruby-193-with-libyaml-on-centos/ where the author had good success with the latter, so I did the same, but cleaning up all by Ruby and libyaml stuff from yum first:

yum remove -y ruby libyaml libyaml-devel rubygem-rake rubygems rubygems-devel

# Download and install libyaml
cd /tmp
wget http://pyyaml.org/download/libyaml/yaml-0.1.7.tar.gz
tar xzvf yaml-0.1.7.tar.gz
cd yaml-0.1.7
./configure --prefix=/usr/local
make
make install

# Download and install Ruby
cd /tmp
wget http://ftp.ruby-lang.org/pub/ruby/2.3/ruby-2.3.1.tar.gz
tar xzvf ruby-2.3.1.tar.gz
cd ruby-2.3.1
./configure --prefix=/usr/local --enable-shared --disable-install-doc --with-opt-dir=/usr/local/lib
make install

# Install your gems
gem install bundler

Monday, October 10, 2016

Oracle® XE Configuration Assistant hangs on Fedora Linux in VirtualBox 5

The problem

I was installing Oracle® XE on Fedora Linux 23 on Oracle® VirtualBox. I noticed that the database configuration assistant was just hanging forever on my desktop PC with an AMD CPU. I was not able to reproduce the same issue on a desktop PC with an Intel CPU.

The cause

I have previously encountered a similar problem with Oracle® 12c Database Configuration Assistant that hangs on Oracle® Linux 7 in VirtualBox 5, and it turned out that the solution was the same for this issue.

The solution

I found the culprit to be the Paravirtualization Interface for acceleration. Changing that from Default to None fixed the issue.



The above screen dumps were made on a virtual session for Centos 7, but it is the same workaround for Fedora Linux and Oracle® Linux running in VirtualBox too.

Friday, September 9, 2016

Development In a X Integrated Environment

It's been almost 7.5 years since I last released anything through Open Source development. At that time I was using SourceForge and storing the source code in CVS.

Today I made a comeback. I started a project on GitHub called dixie, which is short for Development In a X Integrated Environment.

It is partially to have a practical task for learning Vagrant, Ansible and Kickstart, but in the end I think it can result in a pretty usable product too.

Using a Windows server as slave node for Hudson

Installed Cygwin64, and on the Select Packages screen I manually had to choose to install openssh and cygrunsrv as they are not picked by default. 

Created local Windows user sshd on SLAVESRV1. Unchecked "Change password at first login". Checked "Password never expires".

Created local Windows user cyg_server on SLAVESRV1. Unchecked "Change password at first login". Checked "Password never expires".

Added local Windows user cyg_server to local Administrators group.

Created local Windows user hudson on SLAVESRV1. Unchecked "Change password at first login". Checked "Password never expires".

Added local hudson Windows user to local 'Remote Desktop Users' group.

Started "Cygwin64 Terminal" with "Run as Administrator"

Initially I had some issue with "/bin/bash: Operation not permitted", but then I found https://cygwin.com/ml/cygwin/2016-03/msg00097.html with a solution:
Edited the file /bin/ssh-host-config, and after the line CSIH_SCRIPT=/usr/share/csih/cygwin-service-installation-helper.sh I added the following:
editrights -a SeAssignPrimaryTokenPrivilege -u cyg_server
editrights -a SeCreateTokenPrivilege -u cyg_server
editrights -a SeTcbPrivilege -u cyg_server
editrights -a SeServiceLogonRight -u cyg_server

Ran the ssh-host-config script
*** Query: you have the required privileges) (yes/no) yes
*** Query: Should StrictModes be used? (yes/no) no
*** Query: Should privilege separation be used? (yes/no) yes
*** Query: (Say "no" if it is already installed as a service) (yes/no) yes
*** Query: Enter the value of CYGWIN for the daemon: [] binmode ntsec
*** Info: 'SLAVESRV1+cyg_server' will only be used by registered services.
*** Query: Do you want to use a different name? (yes/no) no
*** Query: Please enter the password for user 'SLAVESRV1+cyg_server':

Started Windows service 'CYGWIN sshd'

Verified that SSH was started on port 22 with: netstat -an | grep LISTEN

Logged on to Windows as hudson through Remote Desktop

As hudson user started Cygwin. This ensured /home/hudson folder was created.

Now I was finally ready to add the SLAVESRV1 as a node from the Hudson master using the SLAVESRV1+hudson user for SSH connections. It was colleague that discovered that we needed to prefix the user with SLAVESRV1+ since it was a local user and not a domain user.

Tuesday, August 2, 2016

Remote deployment failed (oracle.jdevimpl.deploy.common.Jsr88RemoteDeployer) with Oracle® JDeveloper

The problem

Using JDeveloper 12.1.3, I had created a dummy BPMN Process with just a User Task bound to a human task. For this task I had also auto-generated an input form. Deploying the task form went fine, but when trying to deploy the BPMN process I got the following error:


[08:30:27 AM] Deployment cancelled.
[08:30:27 AM] Taskflow deployment failed to deploy to server. Remote deployment failed
[08:30:27 AM] Deployment cancelled.
[08:30:27 AM] ----  Deployment incomplete  ----.
[08:30:27 AM] Remote deployment failed (oracle.jdevimpl.deploy.common.Jsr88RemoteDeployer)

The cause

After trying various things, I discovered the problem was my proxy settings in Oracle® JDeveloper.

The solution

Tools -> Preferences -> Web Browser and Proxy -> Proxy Settings, and then I set it to No Proxy. 
Now I could deploy the process without any errors. 

I assume it would also have worked if I had set it to Manual Proxy Settings, and then added my hostname to the exception list.

Wednesday, June 29, 2016

The art of creating efficient algorithms

How often do we think outside the box to create a creative and efficient algorithm when we develop software? I bet answer is "probably not often enough". I have not been coding in Python for close to two years. In fact I have not been coding much for the past two years in any language apart from a few Java utilities and the odd shell scripts to help me get things done in a predicable and automated manner. Hence this is not a post about how to create good algorithms, but rather some reflections on code performance during my re-entry to Python programming. So please forgive me for any false assumptions and poor code quality.

While reading a book on Python 3 recently, I came across an interesting anecdote about the German mathematician Carl Friedrich Gauss, who at the age of 8 was given a homework assignment to sum up all the numbers between 1 and 100. To his teacher’s astonishment Carl Friedrich answered 5050 a few seconds later, and it was the teacher had to go home and check if the answer was correct.

Instead of adding every number between 1 and 100, the young Carl Friedrich quickly realized that there were 50 pairs of unequal numbers that would sum up to 100:
99 + 1
98 + 2
And so on, all the way down to 51 + 49
And since he had not included the number 50 yet, the answer was (50 pairs * 100) + 50 = 5050

So the youngster had by himself discovered a pretty efficient algorithm for summing up integers. Not bad for an 8 year old! According to Wikipedia this story is contested, but nevertheless it is a very good example of a creative solution for optimizing an algorithm.

Naturally I needed to test the same logic on a computer, so I wrote a small Python script to check the human way of thinking (the count()method) and the young Carl Friedrich' way of thinking (the f_count() method):

import time

class FunnyMath(object):

    def __init__(self,num=0):
        self.num = num
   
    def count(self,num=0):
        start = time.time()
        self.num = num;
        for i in range(1,num):
            self.num += i
        end = time.time()
        print("Traditional approach took {} seconds to get the result: {}".format(round((end-start),2),int(self.num)))
    
    def f_count(self,num=0):
        start = time.time()
        self.num = num**2/2.0 + num/2.0
        end = time.time()
        print("Creative approach took {} seconds to get the result: {}".format(round((end-start),2),int(self.num)))
    
def main():
    fm = FunnyMath()
    num = 100000000
    fm.f_count(num)
    fm.count(num)
   
if __name__ == "__main__":
    main()

How long it takes to run will of course depend on your hardware resources. I tested this script on two different laptops.

On the "fast" laptop I got the following output:
Creative approach took 0.0 seconds to get the result: 5000000050000000
Traditional approach took 14.88 seconds to get the result: 5000000050000000

On the "slow" laptop I got the following output:
Creative approach took 0.0 seconds to get the result: 5000000050000000
Traditional approach took 27.76 seconds to get the result: 5000000050000000

So it is an interesting example that shows that some algorithms can and should be optimized. Not only did f_count() perform better. It also had fewer lines of code.

 But with a rewrite I could also optimize the slow count() method:

import time

class FunnyMath(object):

    def __init__(self,num=0):
        assert type(num) is int, "num was not an integer {}".format(num)
        self.num = num
   
    def count(self):
        num = self.num
        for i in range(1,num):
            num += i
        return int(num)
        
    def f_count(self):
        return int(self.num**2/2.0 + self.num/2.0)
    
def main():
    fm = FunnyMath(num = 100000000)
    start = time.time() 
    num = fm.f_count()
    end = time.time()
    print("Creative approach took {} seconds to get the result: {}".format(round((end-start),2),num))
    
    start = time.time()
    num = fm.count()
    end = time.time()
    print("Traditional approach took {} seconds to get the result: {}".format(round((end-start),2),num))
    
if __name__ == "__main__":
    main()

Then I ran it on the "fast" laptop and got the following output:
Creative approach took 0.0 seconds to get the result: 5000000050000000
Traditional approach took 5.98 seconds to get the result: 5000000050000000

A full rewrite was not really necessary, but I wanted a cleaner and more compact code for my class. The observant reader might have spotted the real problem with the first definition of the count() method:

def count(self,num=0):
        start = time.time()
        self.num = num;
        for i in range(1,num):
            self.num += i
        end = time.time()
        print("Traditional approach took {} seconds to get the result: {}".format(round((end-start),2),int(self.num)))
 
For each iteration in the loop I was updating the class variable num instead of just using a local variable called num, so the same performance gain for the count() method could have been achieved with a small rewrite:

def count(self,num=0):
        start = time.time()
        num = num;
        for i in range(1,num):
            num += i
        end = time.time()
        print("Traditional approach took {} seconds to get the result: {}".format(round((end-start),2),int(num)))

When I ran that on the "fast" laptop I got the the following output:
Traditional approach took 5.92 seconds to get the result: 5000000050000000

The reason for this performance gain is explained in the Python wiki.

Still, the count() method would always be slower than the f_count() method.

Wednesday, May 11, 2016

weblogic.store.PersistentStoreFatalException: [Store:280019]There was an error while writing to a storage

The problem

We discovered that an Oracle® WebLogic managed server was in status FAILED on a development environment. The apparent reason for this was the following error:

weblogic.store.PersistentStoreFatalException: [Store:280019]There was an error while writing to a storage

The cause

The can be multiple reasons or the above error, but the logical guess was that our /data mount point had run out of disk space, because that mount point was where our domain configuration (and persistent stores) were saved. That theory turned out to be wrong. The /data mount point was only 12% utilized on a 100GB partition, so there was plenty of available disk.

Then I tried to create an empty file on the mount point:

# touch /data/dummy
# touch: cannot touch `/data/dummy': Read-only file system


When checking the /var/log/messages file, I found the reason for that:

May 11 07:37:45 <hostname> kernel: JBD2: I/O error detected when updating journal superblock for dm-25-8.
May 11 07:37:45 <hostname> kernel: EXT4-fs error (device dm-25): ext4_journal_start_sb: Detected aborted journal
May 11 07:37:45 <hostname> kernel: EXT4-fs (dm-25): Remounting filesystem read-only

Comparing the commands cat /etc/fstab and ls -l /dev/mapper/ | grep dm-25 also confirmed that device dm-25 was indeed the source for the /data mount point.

The solution

After shutting down everything as cleanly as I could, I checked if there were still processes trying to use the /data mount point:

lsof | grep "/data"

It turned out that there was a few processes I was unable to shutdown cleanly, so I had to kill those. 

Then I unmounted the /data mount point:

umount /data


Finally I fixed the issue by running fsck and mounting the partition again. 

Caution: Running fsck on a partition can in worst case scenario be destructive, so make sure you have a valid backup!

fsck /data
mount /data

Now I could start up Oracle® WebLogic Server without any issues.

Thursday, April 28, 2016

RDA-10913: Unknown profile "FM12c_SoaMin" when running Remote Diagnostic Agent on Oracle® SOA Suite 12.1.3

The problem

You are trying to collect information about the Oracle® SOA Suite 12.1.3 using the Remote Diagnostic Agent (RDA), but it fails with a RDA-10913: Unknown profile "FM12c_SoaMin" error, like shown below:

source $DOMAIN_HOME/bin/setDomainEnv.sh
cd $SOA_ORACLE_HOME
cd ../oracle_common/rda
./rda.sh -s soa_issue -p FM12c_SoaMin

RDA-00013: Error reported by the command "setup":
 RDA-00014: Request error "Setup":
  RDA-10913: Unknown profile "FM12c_SoaMin"

The cause

There is something wrong with the RDA supplied with  Oracle® SOA Suite 12.1.3, and this is how I discovered that:

The command ./rda.sh -V told me I was running RDA 8.02, which looked fine.

By reading the full output from ./rda.sh -V I could also see that the required modules where installed, according to Oracle Support Doc ID 391983.1.

Using ./rda.sh -h told me which options RDA is accepting, And in particular I noted the -L flag:
-L     List the available modules, profiles, and conversion groups

./rda.sh -L | grep Soa
  FM11g_Soa                        Oracle SOA Suite 11g problems
  FM11g_SoaMin                     Oracle SOA Suite 11g problems (minimum)

As you can see, the RDA shipped with Oracle® SOA Suite 12.1.3 was only supporting Oracle® SOA Suite 11g diagnostics.

The solution

  1. Downloaded the latest version of the RDA tool for Oracle® Fusion Middleware. At the time of writing this, the patch number was 22783073.
  2. Used opatch apply to install the patch I just downloaded.
  3. Verified which SOA profiles were now available by running:
    ./rda.sh -L | grep Soa
    OFM_SoaMax                       OFM SOA Suite problems (11g/12c+) : full
    OFM_SoaMin                       OFM SOA Suite problems (11g/12c+) : minimum

     
  4. Ran the RDA with the new module for Oracle® SOA Suite 11g and 12c:
    ./rda.sh -s soa_issue -p OFM_SoaMin

Monday, March 14, 2016

Oracle® WebLogic managed server reaches ADMIN state during startup due to name collision

The problem

When starting a Oracle® WebLogic managed server it halts when reaching the ADMIN state. 
Looking at the managed server's out file I saw the following error:

<Mar 15, 2016 8:45:40 AM UTC> <Error> <Deployer> <BEA-149231> <Unable to set the activation state to true for the application "MyDataSource".
weblogic.application.ModuleException: weblogic.common.ResourceException: Failed to bind remote object (ClusterableRemoteRef(6058417782957188722S:my-server:soa_domain:soa_server1 null)/289        [weblogic.jdbc.common.internal.RemoteDataSource]) to replica aware stub at MY_DS(ClusterableRemoteRef(6058417782957188722S:my-server:soa_domain:soa_server1 [6058417782957188722S:my-server:soa_domain:soa_server1/288])/288        [weblogic.jdbc.common.internal.RemoteDataSource])
        at weblogic.jdbc.module.JDBCModule.activate(JDBCModule.java:411)
        at weblogic.application.internal.flow.ModuleListenerInvoker.activate(ModuleListenerInvoker.java:114)
        at weblogic.application.internal.flow.ModuleStateDriver$2.next(ModuleStateDriver.java:195)
        at weblogic.application.internal.flow.ModuleStateDriver$2.next(ModuleStateDriver.java:190)
        at weblogic.application.utils.StateMachineDriver.nextState(StateMachineDriver.java:42)
        Truncated. see log file for complete stacktrace
Caused By: weblogic.common.ResourceException: Failed to bind remote object (ClusterableRemoteRef(6058417782957188722S:my-server:soa_domain:soa_server1 null)/289   [weblogic.jdbc.common.internal.RemoteDataSource]) to replica aware stub at MY_DS(ClusterableRemoteRef(6058417782957188722S:my-server:


The cause

The error above says I have a problem starting the MyDataSource data source due to a name collision. Logging into the WebLogic console, and looking at my data sources, I realized I had two data sources with the same JNDI location. The reason was that I had prepared a new data source long time ago, and forgotten all about it. During the startup of the managed server, my new data source had started up, but the old one could of course not start since it had the same JNDI  location.

The solution

  1. Removed the old and obsolete data source with the conflicting JNDI location.
  2. Checked the managed server's out file for other errors, and there were none. 
  3. Resumed startup of the managed server so the server state changed from ADMIN to RUNNING. 



Thursday, March 10, 2016

Oracle® SOA Suite 12.1.3 startup fails with CoherenceException due to ensureWKAAddresses

The problem

After a fresh install of Oracle® SOA Suite 12.1.3 with BPM and BAM enabled, both the SOA server and the BAM server failed to startup due to the following exception:

<Mar 9, 2016 9:49:34 AM UTC> <Error> <weblogic-coherence-integration> <BEA-000012> <Server bam_server1 is configured with localhost as the Unicast Listen Address which is an error in Coherence Production mode. A generated Coherence WKA list will not operate correctly across multiple machines.>
Mar 09, 2016 9:49:36 AM oracle.dms.servlet.DMSServletFilter setEagerlySetContextValues
INFO: The setting that controls the eager fetching of some types of execution context data has been set to true.
<Mar 9, 2016 9:49:37 AM UTC> <Critical> <WebLogicServer> <BEA-000362> <Server failed. Reason:

There are 1 nested errors:

weblogic.cacheprovider.coherence.CoherenceException:
    at weblogic.cacheprovider.coherence.CoherenceClusterManager.ensureWKAAddresses(CoherenceClusterManager.java:510)
    at weblogic.cacheprovider.coherence.CoherenceClusterManager.configureClusterService(CoherenceClusterManager.java:236)
    at weblogic.cacheprovider.CacheProviderServerService.bootCoherenceFromWLSCluster(CacheProviderServerService.java:225)
    at weblogic.cacheprovider.CacheProviderServerService.initCoherence(CacheProviderServerService.java:94)
    at weblogic.cacheprovider.CacheProviderServerService.initialize(CacheProviderServerService.java:71)
    at weblogic.cacheprovider.CacheProviderServerService.start(CacheProviderServerService.java:65)
    at weblogic.server.AbstractServerService.postConstruct(AbstractServerService.java:78)
    at sun.reflect.GeneratedMethodAccessor34.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at org.glassfish.hk2.utilities.reflection.ReflectionHelper.invoke(ReflectionHelper.java:1017)
    at org.jvnet.hk2.internal.ClazzCreator.postConstructMe(ClazzCreator.java:388)
    at org.jvnet.hk2.internal.ClazzCreator.create(ClazzCreator.java:430)
    at org.jvnet.hk2.internal.SystemDescriptor.create(SystemDescriptor.java:456)
    at org.glassfish.hk2.runlevel.internal.AsyncRunLevelContext.findOrCreate(AsyncRunLevelContext.java:225)
    at org.glassfish.hk2.runlevel.RunLevelContext.findOrCreate(RunLevelContext.java:82)
    at org.jvnet.hk2.internal.Utilities.createService(Utilities.java:2488)
    at org.jvnet.hk2.internal.ServiceHandleImpl.getService(ServiceHandleImpl.java:98)
    at org.jvnet.hk2.internal.ServiceHandleImpl.getService(ServiceHandleImpl.java:87)
    at org.glassfish.hk2.runlevel.internal.CurrentTaskFuture$QueueRunner.oneJob(CurrentTaskFuture.java:1162)
    at org.glassfish.hk2.runlevel.internal.CurrentTaskFuture$QueueRunner.run(CurrentTaskFuture.java:1147)
    at weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl.run(SelfTuningWorkManagerImpl.java:548)
    at weblogic.work.ExecuteThread.execute(ExecuteThread.java:311)
    at weblogic.work.ExecuteThread.run(ExecuteThread.java:263)

>
<Mar 9, 2016 9:49:37 AM UTC> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to FAILED.>
<Mar 9, 2016 9:49:37 AM UTC> <Error> <WebLogicServer> <BEA-000383> <A critical service failed. The server will shut itself down.>
<Mar 9, 2016 9:49:37 AM UTC> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to FORCE_SHUTTING_DOWN.>

The cause

As stated in the exception, the issue is that the servers are listening on localhost instead of the actual hostname while the Oracle® WebLogic Server is running in Production mode.

The solution

There are multiple solutions to this one, but the two most obvious ones are to either run Oracle® WebLogic Server is running in Development mode, or to listen to the actual hostname instead of localhost. I would not recommend the former. Not unless it is a single user development system.

Run the hostname command to find the actual hostname. The hostname command works on both Linux, Unix and Windows.
Edit the managed servers through the  Oracle® WebLogic Server console, and set the value returned from the hostname command as the Listen Address,

Tuesday, March 1, 2016

Could not initialize class sun.awt.X11GraphicsEnvironment for frevvo module in Oracle® BPM Suite 12.2.1

The problem

BPM Composer in Oracle® BPM Suite 12.2.1 fails when open enterprise maps, and the SOA server's out file says:

<Mar 1, 2016 7:31:43 AM UTC> <Error> <HTTP> <BEA-101017> <[ServletContext@881844748[app:frevvo module:/frevvo path:null spec-version:3.1]] Root cause of ServletException.
java.lang.NoClassDefFoundError: Could not initialize class sun.awt.X11GraphicsEnvironment
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:264)
    at java.awt.GraphicsEnvironment.createGE(GraphicsEnvironment.java:103)
    at java.awt.GraphicsEnvironment.getLocalGraphicsEnvironment(GraphicsEnvironment.java:82)
    at sun.awt.X11FontManager.isHeadless(X11FontManager.java:511)
    Truncated. see log file for complete stacktrace
>

The cause

The JVM was not running headless, and it did not have a DISPLAY variable set to point to a X server.

The solution

Added -Djava.awt.headless=true to the soa server, and restarted it.

Friday, February 12, 2016

Error "ORA-01882 timezone region not found" when creating new domain in Oracle® WebLogic

The problem

During domain configuration I received an error saying ORA-01882 timezone region not found when I clicked on the Get RCU Configuration button.

The cause

This is because your operating system is either using a invalid timezone value, or using the Etc/UTC or Universal timezone. The two latter examples are due to a bug when connecting to the database. 

The solution

  1. Exit the domain configuration assistant
  2. Set a timezone value accepted by the database: TZ=UTC; export TZ
  3. Start the domain configuration assistant again by running config.sh
  4. To prevent the same issue from happening again (assuming you hit the bug), you should add the corrected TZ to the software owner's profile, e.g. in $HOME/.bash_profile if you are using the Bourne Again Shell. However, if your Linux server clock is configured with an invalid timezone, you should fix it at Linux level by using a IANA compliant timezone.

Thursday, February 11, 2016

OPatch failed with error code 73

The problem

I came across a sandbox server which had an interesting problem with oraInventory. When running opatch I received the following error:

Inventory load failed... OPatch cannot load inventory for the given Oracle Home.
Possible causes are:
   Oracle Home dir. path does not exist in Central Inventory
   Oracle Home is a symbolic link
   Oracle Home inventory is corrupted
LsInventorySession failed: Oracle Home inventory cannot be loaded.

OPatch failed with error code 73



The cause

The immediate cause was that /u01/app/oraInventory was empty besides the "locks" directory that got created when I ran $ORACLE_HOME/OPatch/opatch lsinventory.

/etc/oraInst.loc pointed to the /u01/app/oraInventory directroy, and so did $ORACLE_HOME/oraInst.loc.
I looked for other copies of oraInventory on the server, but there were none.

I can only speculate on why /u01/app/oraInventory was empty. Perhaps someone deleted the contents by mistake, or by some misguided effort to free up disk space.

The solution

Had this been a production system it would have made sense to check the file system backup, and do a restore of /u01/app/oraInventory. But his was just a sandbox, and no file system backup existed. 

The solution was to run $ORACLE_HOME/oui/bin/attachHome.sh to populate the oraInventory again. 


Now the command $ORACLE_HOME/OPatch/opatch lsinventory worked fine.

Friday, February 5, 2016

Server subsystem failed. Reason: A MultiException has 10 exceptions when starting Oracle® WebLogic managed server

The problem

After starting the Oracle® WebLogic AdminServer in a sandbox environment, I attempted to start the managed servers, but got the following errors in the $DOMAIN_HOME/servers/<ms-server>/logs/<ms-server>.out log file:

<Feb 5, 2016 8:31:03 AM UTC> <Critical> <WebLogicServer> <BEA-000386> <Server subsystem failed. Reason: A MultiException has 10 exceptions.  They are:
1. java.lang.AssertionError: Assertion violated
2. java.lang.IllegalStateException: Unable to perform operation: post construct on weblogic.ldap.EmbeddedLDAP
3. java.lang.IllegalArgumentException: While attempting to resolve the dependencies of weblogic.security.PreSecurityService errors were found
4. java.lang.IllegalStateException: Unable to perform operation: resolve on weblogic.security.PreSecurityService
5. java.lang.IllegalArgumentException: While attempting to resolve the dependencies of weblogic.security.SecurityService errors were found
6. java.lang.IllegalStateException: Unable to perform operation: resolve on weblogic.security.SecurityService
7. java.lang.IllegalArgumentException: While attempting to resolve the dependencies of weblogic.jndi.internal.RemoteNamingService errors were found
8. java.lang.IllegalStateException: Unable to perform operation: resolve on weblogic.jndi.internal.RemoteNamingService
9. java.lang.IllegalArgumentException: While attempting to resolve the dependencies of weblogic.jndi.internal.ForeignJNDIManagerService errors were found
10. java.lang.IllegalStateException: Unable to perform operation: resolve on weblogic.jndi.internal.ForeignJNDIManagerService

A MultiException has 10 exceptions.  They are:
1. java.lang.AssertionError: Assertion violated
2. java.lang.IllegalStateException: Unable to perform operation: post construct on weblogic.ldap.EmbeddedLDAP
3. java.lang.IllegalArgumentException: While attempting to resolve the dependencies of weblogic.security.PreSecurityService errors were found
4. java.lang.IllegalStateException: Unable to perform operation: resolve on weblogic.security.PreSecurityService
5. java.lang.IllegalArgumentException: While attempting to resolve the dependencies of weblogic.security.SecurityService errors were found
6. java.lang.IllegalStateException: Unable to perform operation: resolve on weblogic.security.SecurityService
7. java.lang.IllegalArgumentException: While attempting to resolve the dependencies of weblogic.jndi.internal.RemoteNamingService errors were found
8. java.lang.IllegalStateException: Unable to perform operation: resolve on weblogic.jndi.internal.RemoteNamingService
9. java.lang.IllegalArgumentException: While attempting to resolve the dependencies of weblogic.jndi.internal.ForeignJNDIManagerService errors were found
10. java.lang.IllegalStateException: Unable to perform operation: resolve on weblogic.jndi.internal.ForeignJNDIManagerService

        at org.jvnet.hk2.internal.Collector.throwIfErrors(Collector.java:88)
        at org.jvnet.hk2.internal.ClazzCreator.resolveAllDependencies(ClazzCreator.java:269)
        at org.jvnet.hk2.internal.ClazzCreator.create(ClazzCreator.java:413)
        at org.jvnet.hk2.internal.SystemDescriptor.create(SystemDescriptor.java:456)
        at org.glassfish.hk2.runlevel.internal.AsyncRunLevelContext.findOrCreate(AsyncRunLevelContext.java:225)
        Truncated. see log file for complete stacktrace
Caused By: java.lang.AssertionError: Assertion violated
        at weblogic.utils.Debug.assertion(Debug.java:58)
        at weblogic.server.channels.ChannelService.getServerChannel(ChannelService.java:2049)
        at weblogic.server.channels.ChannelService.getServerChannel(ChannelService.java:1987)
        at weblogic.protocol.ServerChannelManager.findServerChannel(ServerChannelManager.java:79)
        at weblogic.ldap.EmbeddedLDAP.findLdapURL(EmbeddedLDAP.java:1082)
        Truncated. see log file for complete stacktrace
>


The solution

This turned out to be a side effect of a mismatch between the domain binaries and the domain version number in $DOMAIN_HOME/config/config.xml. I have written about that here: http://audunnes.blogspot.dk/2016/02/the-domain-version-122100-is-greater.html

The domain version 12.2.1.0.0 is greater than the release version 12.1.3.0 of this server when starting an Oracle® WebLogic managed server.

The problem

After starting the Oracle® WebLogic AdminServer in a sandbox environment, I attempted to start the managed servers, but got the following errors in the $DOMAIN_HOME/servers/<ms-server>/logs/<ms-server>.out log file:

<Error> <Configuration Management> <BEA-150001> <An error occurred while connecting to the Administration Server to bootstrap through the URL: http://localhost:7001/bea_wls_management_internal2/Bootstrap,
 user: weblogic.
weblogic.management.configuration.ConfigurationException: [Management:141252]The domain version 12.2.1.0.0 is greater than the release version 12.1.3.0 of this server.
        at weblogic.management.provider.internal.BootStrapHelper.getBootStrapStruct(BootStrapHelper.java:147)
        at weblogic.management.provider.internal.RuntimeAccessImpl.initialize(RuntimeAccessImpl.java:444)
        at weblogic.management.provider.internal.RuntimeAccessService.start(RuntimeAccessService.java:70)
        at weblogic.server.AbstractServerService.postConstruct(AbstractServerService.java:78)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.glassfish.hk2.utilities.reflection.ReflectionHelper.invoke(ReflectionHelper.java:1017)
        at org.jvnet.hk2.internal.ClazzCreator.postConstructMe(ClazzCreator.java:388)
        at org.jvnet.hk2.internal.ClazzCreator.create(ClazzCreator.java:430)
        at org.jvnet.hk2.internal.SystemDescriptor.create(SystemDescriptor.java:456)
        at org.glassfish.hk2.runlevel.internal.AsyncRunLevelContext.findOrCreate(AsyncRunLevelContext.java:225)
        at org.glassfish.hk2.runlevel.RunLevelContext.findOrCreate(RunLevelContext.java:82)
        at org.jvnet.hk2.internal.Utilities.createService(Utilities.java:2488)
        at org.jvnet.hk2.internal.ServiceHandleImpl.getService(ServiceHandleImpl.java:98)
        at org.jvnet.hk2.internal.ServiceLocatorImpl.getService(ServiceLocatorImpl.java:606)
        at org.jvnet.hk2.internal.ThreeThirtyResolver.resolve(ThreeThirtyResolver.java:77)
        at org.jvnet.hk2.internal.ClazzCreator.resolve(ClazzCreator.java:231)
        at org.jvnet.hk2.internal.ClazzCreator.resolveAllDependencies(ClazzCreator.java:254)
        at org.jvnet.hk2.internal.ClazzCreator.create(ClazzCreator.java:413)
        at org.jvnet.hk2.internal.SystemDescriptor.create(SystemDescriptor.java:456)
        at org.glassfish.hk2.runlevel.internal.AsyncRunLevelContext.findOrCreate(AsyncRunLevelContext.java:225)
        at org.glassfish.hk2.runlevel.RunLevelContext.findOrCreate(RunLevelContext.java:82)
        at org.jvnet.hk2.internal.Utilities.createService(Utilities.java:2488)
        at org.jvnet.hk2.internal.ServiceHandleImpl.getService(ServiceHandleImpl.java:98)
        at org.jvnet.hk2.internal.ServiceHandleImpl.getService(ServiceHandleImpl.java:87)
        at org.glassfish.hk2.runlevel.internal.CurrentTaskFuture$QueueRunner.oneJob(CurrentTaskFuture.java:1162)
        at org.glassfish.hk2.runlevel.internal.CurrentTaskFuture$QueueRunner.run(CurrentTaskFuture.java:1147)

The cause

I was at first a bit puzzled by the message "The domain version 12.2.1.0.0 is greater than the release version 12.1.3.0 of this server." because the AdminServer console footer said it was version 12.1.3.0, and the startup scripts in $DOMAIN_HOME/bin also only pointed to 12.1.3.0 binaries.

I discovered there were 12.2.1.0.0 binaries installed in a new $ORACLE_HOME, but as mentioned, these binaries were not referenced by the scripts in $DOMAIN_HOME/bin.

The fact that 12.2.1.0.0 binaries were available on the file system suggested someone had either planned to create a new 12.2.1.0.0 domain, or planned to upgrade the existing 12.1.3.0 domain.

After discussing this with a colleague it became clear that he had installed the new binaries, and that all he wanted to do with them was to create response files for running the upgrade of a domain on a different server. He only intended to run the information collection part, but it had obviously also touched the $DOMAIN_HOME/config/confix.xml file in the 12.1.3.0 domain, and changed the version parameter to 12.2.1.0.0 as shown below:

$ grep version $DOMAIN_HOME/config/config.xml
<?xml version='1.0' encoding='UTF-8'?>
  <domain-version>12.2.1.0.0</domain-version>
  <configuration-version>12.2.1.0.0</configuration-version>

The solution

  1. Shutdown the entire domain. In my case that meant the AdminServer and the NodeManager.
  2. Took a file system backup of $DOMAIN_HOME
  3. Edited $DOMAIN_HOME/config/config.xml and replaced the two occurences of 12.2.1.0.0 with 12.1.3.0.0
  4. Started the NodeManager, the AdminServer and the managed servers up without any errors.

Post-change verification:

$ grep version config.xml
<?xml version='1.0' encoding='UTF-8'?>
  <domain-version>12.1.3.0.0</domain-version>
  <configuration-version>12.1.3.0.0</configuration-version>

Unexpected EOF in prolog when starting Oracle® WebLogic AdminServer

The problem

The Oracle® WebLogic AdminServer has started up and reached RUNNING state, but I have the habit of checking the AdminServer.out file after startup, and today I saw the following error:

<Error> <Default> <J2EE JMX-47001> <A JAXB error occurred during unmarshalling of "usermessagingconfig.xml".
javax.xml.bind.UnmarshalException
 - with linked exception:
[Exception [EclipseLink-25004] (Eclipse Persistence Services - 2.5.2.v20140319-9ad6abd): org.eclipse.persistence.exceptions.XMLMarshalException
Exception Description: An error occurred unmarshalling the document
Internal Exception: com.ctc.wstx.exc.WstxEOFException: Unexpected EOF in prolog
 at [row,col {unknown-source}]: [1,0]]
        at org.eclipse.persistence.jaxb.JAXBUnmarshaller.handleXMLMarshalException(JAXBUnmarshaller.java:980)
        at org.eclipse.persistence.jaxb.JAXBUnmarshaller.unmarshal(JAXBUnmarshaller.java:588)
        at org.eclipse.persistence.jaxb.JAXBUnmarshaller.unmarshal(JAXBUnmarshaller.java:140)
        at oracle.as.config.persistence.jaxb.JAXBXmlPersistenceManagerImpl.load(JAXBXmlPersistenceManagerImpl.java:190)
        at oracle.as.jmx.framework.util.DefaultConfigObjectMBeanAssociationInfo.loadConfigObject(DefaultConfigObjectMBeanAssociationInfo.java:638)
        at oracle.as.jmx.framework.util.DefaultConfigObjectMBeanAssociationInfo.initializeConfigObjectAndAssociatedMBean(DefaultConfigObjectMBeanAssociationInfo.java:676)
        at oracle.as.jmx.framework.wls.spi.ComponentMBeans.registerMBeans(ComponentMBeans.java:244)
        at oracle.as.jmx.framework.wls.spi.ComponentMBeans.registerDomainLevelMBeans(ComponentMBeans.java:356)
        at oracle.as.jmx.framework.wls.spi.ComponentMBeans.registerRuntimeMBeanServerMBeans(ComponentMBeans.java:373)
        at oracle.as.jmx.framework.wls.spi.JMXFrameworkProviderImpl$3.run(JMXFrameworkProviderImpl.java:1134)
        at java.security.AccessController.doPrivileged(Native Method)
        at oracle.as.jmx.framework.wls.spi.JMXFrameworkProviderImpl.initRuntimeMBeanServer(JMXFrameworkProviderImpl.java:1127)
        at oracle.as.jmx.framework.wls.spi.JMXFrameworkProviderImpl.access$200(JMXFrameworkProviderImpl.java:210)
        at oracle.as.jmx.framework.wls.spi.JMXFrameworkProviderImpl$2.run(JMXFrameworkProviderImpl.java:1080)
        at java.lang.Thread.run(Thread.java:745)
Caused By: Exception [EclipseLink-25004] (Eclipse Persistence Services - 2.5.2.v20140319-9ad6abd): org.eclipse.persistence.exceptions.XMLMarshalException

The cause

The usermessagingconfig.xml in question was located in $DOMAIN_HOME/config/fmwconfig/usermessagingconfig.xml

ls -l $DOMAIN_HOME/config/fmwconfig/usermessagingconfig.xml confirmed that the file had not been update since the 9th of September 2015, but then again this WebLogic server had been running for several months before I restarted it today. This is a sandbox server so it is used very infrequently.

ls -l $DOMAIN_HOME/config/fmwconfig/usermessagingconfig.xml also showed me that the size was 0 bytes. 

I remembered that the mount point where WebLogic stores its domain configuration files ran out of disk space in September last year. At that time it corrupted the EmbeddedLDAP folder, but after clearing the log files and re-creating the EmbeddedLDAP, WebLogic was started up again. In hindsight I realize that the same error with usermessagingconfig.xml must have been in AdminServer.out back then as well, but I obviously failed to notice it at the time. 

The solution

I added the following to $DOMAIN_HOME/config/fmwconfig/usermessagingconfig.xml:

<?xml version="1.0"?>
<MessagingConfiguration xmlns="http://www.oracle.com/ucs/messaging/config"
                        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                        xsi:schemaLocation="http://www.oracle.com/ucs/messaging/config ../config.xsd" version="12.1.3.0.0">


  <Server>
  </Server>

</MessagingConfiguration>

Then restarted the AdminServer, and the error was gone.

Thursday, February 4, 2016

Oracle® 12c Database Configuration Assistant hangs on Oracle® Linux 7 in VirtualBox 5

I recently installed Oracle® Linux 7.2 on my VirtualBox 5.0.14 installation on Windows 7 in order to have a private playground for evaluating software. Afterwards I installed Oracle® 12c database version 12.1.0.2 on my new virtual Linux server. This went fine until I fired up Database Configuration Assistant (DBCA) in order to create a playground database. DBCA just hanged forever without consuming noticeable CPU. I kept it running for an entire night, but it never started fully up. It kept hanging at the splash screen as shown below:



Investigations

The log file $ORACLE_BASE/cfgtoollogs/dbca/trace.log_OraDB12Home1_<timestamp> showed the following:

[main] [ 2016-02-04 10:06:33.737 CET ] [Host.<init>:1090]  Begin tracing..
[main] [ 2016-02-04 10:06:34.971 CET ] [UIHost.createHelpSet:485]  HelpSetParseExceptionjava.lang.NullPointerException
[main] [ 2016-02-04 10:06:36.703 CET ] [Host.checkIfBigClusterAndHubNode:1710]  Not a cluster environment: exiting BigCluster Check
[main] [ 2016-02-04 10:06:36.708 CET ] [InventoryUtil.getOUIInvSession:349]  setting OUI READ level to ACCESSLEVEL_READ_LOCKLESS
[main] [ 2016-02-04 10:06:36.708 CET ] [InventoryUtil.isCRSHome:386]  Homeinfo /u01/app/oracle/product/12.1.0/dbhome_1,1
[main] [ 2016-02-04 10:06:36.922 CET ] [Host.validateGridHome:3878]  Validation false
[main] [ 2016-02-04 10:06:36.922 CET ] [Host.startOperation:2395]  Source db null
[main] [ 2016-02-04 10:06:36.922 CET ] [Host.startOperation:2396]  GDB Name null
[main] [ 2016-02-04 10:06:36.957 CET ] [Host.startOperation:2397]  MgmtDB sid -MGMTDB
[main] [ 2016-02-04 10:06:36.957 CET ] [Host.startOperation:2398]  MgmtDB name _mgmtdb
[main] [ 2016-02-04 10:06:36.998 CET ] [OracleHome.getVersion:991]  OracleHome.getVersion called.  Current Version: null
[main] [ 2016-02-04 10:06:37.000 CET ] [InventoryUtil.getOUIInvSession:349]  setting OUI READ level to ACCESSLEVEL_READ_LOCKLESS
[main] [ 2016-02-04 10:06:37.000 CET ] [OracleHome.getVersion:1010]  Homeinfo /u01/app/oracle/product/12.1.0/dbhome_1,1
[main] [ 2016-02-04 10:06:37.181 CET ] [OracleHome.getVersion:1038]  OracleHome.server.getVersion Version: 12.1.0.2.0
[main] [ 2016-02-04 10:06:37.182 CET ] [OracleHome.getVersion:1059]  Current Version From Inventory: 12.1.0.2.0
[main] [ 2016-02-04 10:06:37.182 CET ] [OracleHome.getVersion:991]  OracleHome.getVersion called.  Current Version: 12.1.0.2.0
[main] [ 2016-02-04 10:06:37.182 CET ] [OracleHome.getVersion:1059]  Current Version From Inventory: 12.1.0.2.0
[main] [ 2016-02-04 10:06:37.183 CET ] [CommonUtils.createPasswordFile:1243]  calling new orapwd for 11.1 or higher
[main] [ 2016-02-04 10:06:37.183 CET ] [OracleHome.getVersion:991]  OracleHome.getVersion called.  Current Version: 12.1.0.2.0
[main] [ 2016-02-04 10:06:37.184 CET ] [OracleHome.getVersion:1059]  Current Version From Inventory: 12.1.0.2.0
[main] [ 2016-02-04 10:06:37.184 CET ] [OracleHome.getVersion:991]  OracleHome.getVersion called.  Current Version: 12.1.0.2.0
[main] [ 2016-02-04 10:06:37.184 CET ] [OracleHome.getVersion:1059]  Current Version From Inventory: 12.1.0.2.0
[main] [ 2016-02-04 10:06:37.184 CET ] [CommonUtils.getPasswordFileCreateCmd:1182]  for new orapwd for 11.1 or higher
[main] [ 2016-02-04 10:06:37.189 CET ] [OracleHome.getVersion:991]  OracleHome.getVersion called.  Current Version: 12.1.0.2.0
[main] [ 2016-02-04 10:06:37.190 CET ] [OracleHome.getVersion:1059]  Current Version From Inventory: 12.1.0.2.0
[main] [ 2016-02-04 10:06:37.190 CET ] [OracleHome.getVersion:991]  OracleHome.getVersion called.  Current Version: 12.1.0.2.0
[main] [ 2016-02-04 10:06:37.190 CET ] [OracleHome.getVersion:1059]  Current Version From Inventory: 12.1.0.2.0
[main] [ 2016-02-04 10:06:37.191 CET ] [CommonUtils.getPasswordFileCreateCmd:1213]  /u01/app/oracle/product/12.1.0/dbhome_1/bin/orapwd
[main] [ 2016-02-04 10:06:37.191 CET ] [CommonUtils.getPasswordFileCreateCmd:1213]  file=/u01/app/oracle/product/12.1.0/dbhome_1/dbs/orapwDBUA0636958
[main] [ 2016-02-04 10:06:37.191 CET ] [CommonUtils.getPasswordFileCreateCmd:1213]  force=y
[main] [ 2016-02-04 10:06:37.191 CET ] [CommonUtils.getPasswordFileCreateCmd:1213]  format=12
[main] [ 2016-02-04 10:06:37.192 CET ] [OsUtilsBase.execProg:2123]  beginning execProg with input array.
[main] [ 2016-02-04 10:06:37.377 CET ] [OsUtilsBase.execProg:2160]  finished execProg with input array. Status:0
[main] [ 2016-02-04 10:06:37.378 CET ] [OracleHome.initOptionsStopOnError:1356]  Initializing Database Options with  for dummy sid=DBUA0636958 using initfile=/u01/app/oracle/product/12.1.0/dbhome_1/dbs/initDBUA0636958.ora using pwdfile=/u01/app/oracle/product/12.1.0/dbhome_1/dbs/orapwDBUA0636958
[main] [ 2016-02-04 10:06:37.398 CET ] [OracleHome.getVersion:991]  OracleHome.getVersion called.  Current Version: 12.1.0.2.0
[main] [ 2016-02-04 10:06:37.399 CET ] [OracleHome.getVersion:1059]  Current Version From Inventory: 12.1.0.2.0
[main] [ 2016-02-04 10:06:37.399 CET ] [OracleHome.getVersion:991]  OracleHome.getVersion called.  Current Version: 12.1.0.2.0
[main] [ 2016-02-04 10:06:37.400 CET ] [OracleHome.getVersion:1059]  Current Version From Inventory: 12.1.0.2.0
[main] [ 2016-02-04 10:06:37.400 CET ] [OracleHome.getVersion:991]  OracleHome.getVersion called.  Current Version: 12.1.0.2.0
[main] [ 2016-02-04 10:06:37.401 CET ] [OracleHome.getVersion:1059]  Current Version From Inventory: 12.1.0.2.0
[main] [ 2016-02-04 10:06:37.402 CET ] [SQLPlusEngine.getCmmdParams:225]  m_home 12.1.0.2.0
[main] [ 2016-02-04 10:06:37.402 CET ] [SQLPlusEngine.getCmmdParams:226]  version > 112 true
[main] [ 2016-02-04 10:06:37.403 CET ] [SQLEngine.getEnvParams:612]  NLS_LANG: AMERICAN_AMERICA.AL32UTF8
[main] [ 2016-02-04 10:06:37.421 CET ] [SQLEngine.initialize:358]  Execing SQLPLUS/SVRMGR process...
[main] [ 2016-02-04 10:06:37.467 CET ] [SQLEngine.initialize:395]  m_bReaderStarted: false
[main] [ 2016-02-04 10:06:37.467 CET ] [SQLEngine.initialize:399]  Starting Reader Thread...
[main] [ 2016-02-04 10:06:42.068 CET ] [OracleHome.initOptionsStopOnError:1370]  executing: startup nomount pfile='/u01/app/oracle/product/12.1.0/dbhome_1/dbs/initDBUA0636958.ora'


The 'ps -ef' command also confirms that sqlplus is stuck in "startup nomount"




Some unsuccessful attempts

- Tried with CentOS 7.1 instead of Oracle® Linux 7.2. Replicated the issue 100%.
- Tried to run DBCA as part of the Oracle® 12c Database installation (Software and starter database) and after a software only install. Same problem in both cases. 

Searching for an answer

I found a similar issue described in https://community.oracle.com/message/13249304#13249304, but no solution was offered, 

The solution

After various unsuccessful attempts to fix this at Linux level, I turned my attention to VirtualBox. After trying to tweak several options, I found the culprit to be the Paravirtualization Interface for acceleration. Changing that from Default to None fixed the issue.




I did not try all the various options for Paravirtualization Interface, but in my setup both with Default and Legacy I faced issues with DBCA.