Oracle Application Server 10g – Increase JVM Heap Memory Size

Background:

One of my customers still has the old oracle application server (10.1.2.3.0), that is used to host a forms and reports application.

Application Tier is running on Windows server (Windows 2008 R2 64-bit)

Customer has 6-sites, each one has its own application server (no load balancer). All branches are connecting to the same database at the backend.

The reason why customer has 6 application servers, because each server has only around 4GB RAM, and one server cannot handle the end user workload.

The performance of each application server was ok, except at the month-end processing, when the application starts to be very slow (number of concurrent connections starts to exceed 100)

What is Required:?

Lately, customer purchase a powerful server with 96GB RAM. Initially, he moved all the application servers (as VMWare Virtual Machines) to the new server, but increased the memory assigned to each server from 4GB to about 16GB. Everything is running fine, but he didn’t notice any performance enhancement.

I told them that we need to increase the JVM heap size to take advantage from the available memory. They agreed and asked me to proceed.

So, our objective is to increase the memory assigned to the iAS, so as we can consolidate all the application servers into one application server to simplify the administration and enhance the performance.

Solution:

I started reconfiguring one of the application servers as follows:
Using Application Server Control Console, go to the OC4J instance homepage and perform the following steps:
* Stop the OC4J Instance.
* Drill down to the Server Properties page.
* In the Command Line Options area of the Server Properties page, under the heading “Multiple VM Configuration”, set the Java Options.
* For example, enter the following to set the JVM heap sizes to 2048 Megabytes:
-Xmx512m
* Click the Apply button to apply the changes.
* Start the OC4J instance.

(or you may simply modify the opmn.xml file)

What Happened after that?

• I noticed the following:
• If I assigned JVM heap size > 1500 MB, I cannot restart the OC4J Instance. The maximum size that I can assign was 1500MB.
• Connecting to the Application became very slow.
• If the number of concurrent connections exceeds 100, new connections start to hang for long time!.
• At this time, the error.log file has the following errors:

[Sun Oct 06 14:00:06 2019] [warn] [client 192.168.120.189] [ecid: 1570363206:192.168.108.221:50012:2204:24,0] MOD_OC4J_0184: Failed to find an oc4j process for destination: OC4J_BI_Forms
[Sun Oct 06 14:00:06 2019] [error] [client 192.168.120.189] [ecid: 1570363206:192.168.108.221:50012:2204:24,0] MOD_OC4J_0145: There is no oc4j process (for destination: OC4J_BI_Forms) available to service request.
[Sun Oct 06 14:00:06 2019] [error] [client 192.168.120.189] [ecid: 1570363206:192.168.108.221:50012:2204:24,0] MOD_OC4J_0119: Failed to get an oc4j process for destination: OC4J_BI_Forms

Final Solution:

What I did to overcome these issues:

1. I reduced the JVM heap size from 1500 MB to 1024MB.
2. I increased the range of the AJP ports from 100 to 300:

From:

To:

Now, application started to behave normally with reasonable response time, and the hanging issue (after 100 concurrent connections) disappeared.

We are going to monitor for a couple of days, if no issues, then we may implement multiple JVMs for the same OC4J instance to accommodate more concurrent users and assign more memory for the JVM.

The question that I was not able to answer till moment, why I cannot assign more than 1.5GB for one JVM?!

I hope I can answer it very soon!.

Thanks

Ahmed