Oracle Enterprise Manager 13c R5 Disk Full Issue

I have a client who recently installed Oracle Enterprise Manager 13c R5 on Windows Server 2019.

All OEM components are installed on the same server (Database, OMS, Agent,..etc)
The space consumed by all OEM components should’t exceed about 40 GB.
The installation was done on 400GB disk.

Within couple of days, my client complained that the disk is full (about 400 GB, instead of 40 GB).

Investigation:
I got access to the environment and started investigating the issue, and I found something that is a little bit strange:

As you see, there is a huge difference between the real folder size and the actual size on the disk for all folders.

I know that the reason behind this may be the partition allocation unit that you specified when you partitioned the disk.
Assume that the allocation unit (the minimum space that can be allocated by the windows, it is like the database block size in oracle) is 1MB. If you have a file of 1K size, windows will allocate 1M , not 1K.
Assume that you have 30,000 small files , each is 1k, Windows will allocate about 30GB instead of 30M, si, a huge amount of space will be wasted here.

But, how to check the actual allocation unite size of our disk? I know about two methods:

So, it is very clear that our disk was formatted with allocation unit = 2M (default is 4k).
and for sure this is the real cause of the issue here.


Oracle Enterprise Manager has very large amount of small files. ALso, database has large amount of small trace files.

The question now, is how we can reduce the allocation unit size, given that this is a running production environment.

From Windows side, you can change the The question now, is how we can reduce the allocation unit size, given that this is a running production environment. if you do a “Quick Format” of the disk, but in this case we will lose the data!!.
Actually, the data will be still there in disk, and you may try to use some tools to recover it, but this is a risky approach.

You may found some third-part tools that may be able to change the allocation unit size, without loosing the data.
Or you may format and re-install everything again, as this is a new installation (lengthy approach).

I followed the following approach:
1- I asked the client to add a new disk F: (and format it with Allocation Unit = 4K)
2- I stopped all oracle services.
3- I copied the “app” folder from E: to F:
4- I changed the partition letter of partition E to M
5- I changed the partition letter of partition F: to E:
6- I dropped partition M: (You may wait for a couple of days to drop it, just for safety).
7- Start the Oracle Services.

This fixed the issue, and now all the OEM components consume only about 40GB instead of 400GB.


While coping the “app” folder from the old partition to the new one (step #2 above), you may face the copying issue documented in the following post:


https://ahmedfattah.com/2021/07/28/the-file-names-would-be-too-long-for-the-destination-folder/

Or you may encounter “Access is Denied” issue for the Wallet files:

You can fix this issue by granting privileges on these files for the proper user:

At step 7, when you try to start the database, you may face the following access error:

Total System Global Area 2.0670E+10 bytes

Fixed Size                 12578056 bytes

Variable Size            2617245696 bytes

Database Buffers         1.7985E+10 bytes

Redo Buffers               54530048 bytes

ORA-00221: error on write to control file

ORA-00206: error in writing (block 1, # blocks 1) of control file

ORA-00202: control file: ‘E:\APP\ORACLE\19C\ORADATA\PRBOEM\CONTROL01.CTL’

ORA-27041: unable to open file

OSD-04002: unable to open file

O/S-Error: (OS 5) Access is denied.



Again, this is a security settings, change it for the folder “app”, and grant access for the proper user(s):

After that , I was able to start the whole environment successfully and login to the OEM Console, and I was able to safe about 360 GB !.

Thanks
Ahmed

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s