During my present Exadata course in Munich I learned that there is an (invisible) default threshold present on the Exadata Cells that triggers „Accelerated space reclamation“ when 80 % Filesystem Utilization is reached. This is what I observed on a Cell where I consumed 82 % of space:
CellCLI> list alerthistory detail name: 4_1 alertMessage: "File system "/" is 82% full, which is above the 80% threshold. Accelerated space reclamation has started. This alert will be cleared when file system "/" becomes less than 75% full." alertSequenceID: 4 alertShortName: Software alertType: Stateful beginTime: 2012-09-12T13:17:53-04:00 examinedBy: metricObjectName: / notificationState: 0 sequenceBeginTime: 2012-09-12T13:17:53-04:00 severity: critical alertAction: "MS includes a file deletion policy that is triggered when file system utilitization is high. Deletion of files is triggered when file utilization reaches 80%. In particular, the deletion policy is as follows: For the /opt/oracle and /var/log/oracle file systems, files in the ADR base directory, metric history directory, and LOG_HOME directory will be deleted using a policy based on the file modification time stamp. Files older than the number of days set by the metricHistoryDays attribute value will be deleted first, then successive deletions will occur for earlier files, down to files with modification time stamps older than or equal to 10 minutes, or until file system utilization is less than 75%. The renamed alert.log files and ms-odl generation files that are over 5 MB, and older than the successively-shorter age intervals are also deleted. Crash files that are over 5 MB and older than one day will be deleted. For the / file system, files in the home directories (cellmonitor and celladmin), /tmp and /var/spool directories that are over 5 MB and older than one day will be deleted. Try to delete more recent files, or files not being automatically purged, to free up space if needed."
I have not seen that documented, so I thought it might be of interest for the Exadata community. The threshold does not show, though:
CellCLI> list threshold
--Nothing here --
The mentioned metricHistoryDays attribute is documented and defaults to 7 days, which can be modified like this:
CellCLI> list cell attributes metricHistoryDays 7 CellCLI> alter cell metricHistoryDays=3 Cell qr01cel01 successfully altered
As always: Don’t believe it, test it! 🙂
Pickleball spielen 
#1 von vishaldesai am September 14, 2012 - 02:35
Is there any action attribute/script associated with threshold?
#2 von frorablog am September 14, 2012 - 11:57
Hi,
This feature is documented in the Exadata Storage Server Software User’s guide.
#3 von Uwe Hesse am September 17, 2012 - 10:56
frorablog, you are right: It’s documented under ‚Understanding Automated Cell Maintenance‘, while I was looking under ‚Monitoring and Tuning Oracle Exadata Storage Server Software‘ where all else about Metrics & Alerts is listed.
#4 von Yuri Pudovchenko am November 7, 2012 - 09:28
Hello, Uwe!
One more question for you ;).
Whishing to patch my Exadata I’ve got the message from patchmgr:
[ERROR] Need at least 4363 MB free space on cell. Found only 3961 MB.
Question: how to free the space in the storage cell ?
Much thanks !
#5 von Uwe Hesse am November 12, 2012 - 09:35
Yuri,
you may free up space that is consumed by metric- and tracefiles by reducing the default retention period of them.
The retention period of seven days is the default. The retention period can be modified using the metricHistoryDays and diagHistoryDays attributes with the ALTER CELL command. The diagHistoryDays attribute controls the ADR files, and the metricHistoryDays attribute controls the other files.
#6 von sunilbhola am Juni 22, 2013 - 16:53
Hi Yuvi,
You can always cleanup previous applied patch by using the cleanup utility to
./patchmgr -cells /opt/oracle.SupportTools/onecommand/cell_group -cleanup
Regards,
Sunil Bhola
#7 von Uwe Hesse am Juli 9, 2013 - 20:10
Sunil Bhola, thank you for answering, much appreciated 🙂
#8 von Jimmy am Oktober 31, 2013 - 11:24
sorry, I want to get this document Link
I tried but I can not find any information about that filesystem 80%
or any document ‘Understanding Automated Cell Maintenance’,
‘Monitoring and Tuning Oracle Exadata Storage Server Software’
can you help me
#9 von Uwe Hesse am November 8, 2013 - 12:11
Jimmy, unfortunately the corporation decided against making the Exadata documentation available to the public. Therefore you can’t find it on the web. It is shipped together with the Database Machine to the customer. You may get it from MOS also with this patch number 10386736.