Oracle® WebLogic managed server reaches ADMIN state during startup due to name collision

The problem

When starting a Oracle® WebLogic managed server it halts when reaching the ADMIN state. 

Looking at the managed server's out file I saw the following error:

<Mar 15, 2016 8:45:40 AM UTC> <Error> <Deployer> <BEA-149231> <Unable to set the activation state to true for the application "MyDataSource".

weblogic.application.ModuleException: weblogic.common.ResourceException: Failed to bind remote object (ClusterableRemoteRef(6058417782957188722S:my-server:soa_domain:soa_server1 null)/289        [weblogic.jdbc.common.internal.RemoteDataSource]) to replica aware stub at MY_DS(ClusterableRemoteRef(6058417782957188722S:my-server:soa_domain:soa_server1 [6058417782957188722S:my-server:soa_domain:soa_server1/288])/288        [weblogic.jdbc.common.internal.RemoteDataSource])

        at weblogic.jdbc.module.JDBCModule.activate(JDBCModule.java:411)

        at weblogic.application.internal.flow.ModuleListenerInvoker.activate(ModuleListenerInvoker.java:114)

        at weblogic.application.internal.flow.ModuleStateDriver$2.next(ModuleStateDriver.java:195)

        at weblogic.application.internal.flow.ModuleStateDriver$2.next(ModuleStateDriver.java:190)

        at weblogic.application.utils.StateMachineDriver.nextState(StateMachineDriver.java:42)

        Truncated. see log file for complete stacktrace

Caused By: weblogic.common.ResourceException: Failed to bind remote object (ClusterableRemoteRef(6058417782957188722S:my-server:soa_domain:soa_server1 null)/289   [weblogic.jdbc.common.internal.RemoteDataSource]) to replica aware stub at MY_DS(ClusterableRemoteRef(6058417782957188722S:my-server:

The cause

The error above says I have a problem starting the MyDataSource data source due to a name collision. Logging into the WebLogic console, and looking at my data sources, I realized I had two data sources with the same JNDI location. The reason was that I had prepared a new data source long time ago, and forgotten all about it. During the startup of the managed server, my new data source had started up, but the old one could of course not start since it had the same JNDI  location.

The solution

1. Removed the old and obsolete data source with the conflicting JNDI location.
2. Checked the managed server's out file for other errors, and there were none. 
3. Resumed startup of the managed server so the server state changed from ADMIN to RUNNING. 

Oracle® SOA Suite 12.1.3 startup fails with CoherenceException due to ensureWKAAddresses

The problem

After a fresh install of Oracle® SOA Suite 12.1.3 with BPM and BAM enabled, both the SOA server and the BAM server failed to startup due to the following exception:

<Mar 9, 2016 9:49:34 AM UTC> <Error> <weblogic-coherence-integration> <BEA-000012> <Server bam_server1 is configured with localhost as the Unicast Listen Address which is an error in Coherence Production mode. A generated Coherence WKA list will not operate correctly across multiple machines.>

Mar 09, 2016 9:49:36 AM oracle.dms.servlet.DMSServletFilter setEagerlySetContextValues

INFO: The setting that controls the eager fetching of some types of execution context data has been set to true.

<Mar 9, 2016 9:49:37 AM UTC> <Critical> <WebLogicServer> <BEA-000362> <Server failed. Reason:

There are 1 nested errors:

weblogic.cacheprovider.coherence.CoherenceException:

    at weblogic.cacheprovider.coherence.CoherenceClusterManager.ensureWKAAddresses(CoherenceClusterManager.java:510)

    at weblogic.cacheprovider.coherence.CoherenceClusterManager.configureClusterService(CoherenceClusterManager.java:236)

    at weblogic.cacheprovider.CacheProviderServerService.bootCoherenceFromWLSCluster(CacheProviderServerService.java:225)

    at weblogic.cacheprovider.CacheProviderServerService.initCoherence(CacheProviderServerService.java:94)

    at weblogic.cacheprovider.CacheProviderServerService.initialize(CacheProviderServerService.java:71)

    at weblogic.cacheprovider.CacheProviderServerService.start(CacheProviderServerService.java:65)

    at weblogic.server.AbstractServerService.postConstruct(AbstractServerService.java:78)

    at sun.reflect.GeneratedMethodAccessor34.invoke(Unknown Source)

    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

    at java.lang.reflect.Method.invoke(Method.java:497)

    at org.glassfish.hk2.utilities.reflection.ReflectionHelper.invoke(ReflectionHelper.java:1017)

    at org.jvnet.hk2.internal.ClazzCreator.postConstructMe(ClazzCreator.java:388)

    at org.jvnet.hk2.internal.ClazzCreator.create(ClazzCreator.java:430)

    at org.jvnet.hk2.internal.SystemDescriptor.create(SystemDescriptor.java:456)

    at org.glassfish.hk2.runlevel.internal.AsyncRunLevelContext.findOrCreate(AsyncRunLevelContext.java:225)

    at org.glassfish.hk2.runlevel.RunLevelContext.findOrCreate(RunLevelContext.java:82)

    at org.jvnet.hk2.internal.Utilities.createService(Utilities.java:2488)

    at org.jvnet.hk2.internal.ServiceHandleImpl.getService(ServiceHandleImpl.java:98)

    at org.jvnet.hk2.internal.ServiceHandleImpl.getService(ServiceHandleImpl.java:87)

    at org.glassfish.hk2.runlevel.internal.CurrentTaskFuture$QueueRunner.oneJob(CurrentTaskFuture.java:1162)

    at org.glassfish.hk2.runlevel.internal.CurrentTaskFuture$QueueRunner.run(CurrentTaskFuture.java:1147)

    at weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl.run(SelfTuningWorkManagerImpl.java:548)

    at weblogic.work.ExecuteThread.execute(ExecuteThread.java:311)

    at weblogic.work.ExecuteThread.run(ExecuteThread.java:263)

>

<Mar 9, 2016 9:49:37 AM UTC> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to FAILED.>

<Mar 9, 2016 9:49:37 AM UTC> <Error> <WebLogicServer> <BEA-000383> <A critical service failed. The server will shut itself down.>

<Mar 9, 2016 9:49:37 AM UTC> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to FORCE_SHUTTING_DOWN.>

The cause

As stated in the exception, the issue is that the servers are listening on localhost instead of the actual hostname while the Oracle® WebLogic Server is running in Production mode.

The solution

There are multiple solutions to this one, but the two most obvious ones are to either run Oracle® WebLogic Server is running in Development mode, or to listen to the actual hostname instead of localhost. I would not recommend the former. Not unless it is a single user development system.

Run the hostname command to find the actual hostname. The hostname command works on both Linux, Unix and Windows.

Edit the managed servers through the  Oracle® WebLogic Server console, and set the value returned from the hostname command as the Listen Address,

Could not initialize class sun.awt.X11GraphicsEnvironment for frevvo module in Oracle® BPM Suite 12.2.1

The problem

BPM Composer in Oracle® BPM Suite 12.2.1 fails when open enterprise maps, and the SOA server's out file says:

<Mar 1, 2016 7:31:43 AM UTC> <Error> <HTTP> <BEA-101017> <[ServletContext@881844748[app:frevvo module:/frevvo path:null spec-version:3.1]] Root cause of ServletException.

java.lang.NoClassDefFoundError: Could not initialize class sun.awt.X11GraphicsEnvironment

    at java.lang.Class.forName0(Native Method)

    at java.lang.Class.forName(Class.java:264)

    at java.awt.GraphicsEnvironment.createGE(GraphicsEnvironment.java:103)

    at java.awt.GraphicsEnvironment.getLocalGraphicsEnvironment(GraphicsEnvironment.java:82)

    at sun.awt.X11FontManager.isHeadless(X11FontManager.java:511)

    Truncated. see log file for complete stacktrace

>

The cause

The JVM was not running headless, and it did not have a DISPLAY variable set to point to a X server.

The solution

Added -Djava.awt.headless=true to the soa server, and restarted it.