How can I limit the size of the chat log database?
The size of the Instant IMtegrity chat log database
imtchatlog9.nsf can grow large very fast, depending on these factors:
- The number of active users using IBM Sametime to chat with each other.
- The number of chats users perform throughout the day.
- The number and size of files users send to each other.
- The number and size of images users send to each other.
- Whether or not you attach the original chat .xml files to the chat log documents.
Save space by not storing source XML log files as attachments
To limit rapid growth of the chat log database we recommend to not attach the original XML log files to the chat log documents since all of the relevant content such as dates, times, names and chat content incl. images get imported into the chat log documents anyway.
The main purpose of the XML log files are simply to act as intermediate files only; they get created by the core Instant IMtegrity chat logging component running inside the IBM Sametime Community server and then imported into native IBM Notes documents by the Instant IMtegrity server task.
To verify that the XML files are not stored as attachments, open the server configuration document in
imtconfig.nsf and make sure that the option Attach XML log files is disabled.
Save space by not storing emoticons and images
New in version 9.3.0
As of Instant IMtegrity version 9.3.0, you can disable logging of emoticons and images. If logging is disabled, emoticons and images will be replaced by a placeholder text.
Distribute the chat logs into multiple smaller databases
New in version 9.3.0
You can now store chat log documents in different databases by using this technique.
Save space by not storing transferred files as attachments
When users transfer files in IM chats, Instant IMtegrity by default captures these transferred files and imports them into the chat log database as attachments.
If you don’t need the content of these files inside the chat log database then you can save space by disabling this feature. To disable storing transferred files as attachments, open the server configuration document and make sure that the option Attach file transfers is disabled.
Transferred files are stored on the file system (as specified in the server configuration document under the option Log IBM Sametime File transfers) and their file names match the chat ID of the chat log file transfer document (viewable under the “Administration” tab in the document).
For example, if the chat ID is
dec736e5-0726-4819-9adf-9182fc1d4ce6 then the file name is
imt9_dec736e5-0726-4819-9adf-9182fc1d4ce6.bin (The original file name is available inside the chat log file transfer document; however files stored on disk use the unique chat log ID to avoid file name clashes due to duplicate files).
Save space by storing transferred files on the server file system using DAOS
If you do need the files transferred by users as attachments in the database then you can still save space by utilizing DAOS (“Domino Attachment and Object Service”). DAOS is available in IBM Domino 8.5.0 and higher.
For details on how to configure and use DAOS see the IBM product documentation.
Enable LZ1 attachment compression, design and document compression
We recommend that you enable these advanced database properties for the Instant IMtegrity chat log database:
- Use LZ1 compression for attachments
- Compress database design
- Compress document data
To verify that these options are enabled, open the
imtchatlog9.nsf database using an IBM Notes Client, open the database properties infobox, select the last tab with the beanie hat icon and make sure that all three options are checked.
It is not enough to simply enable these options. If you enabled any of these options then you additionally have to compact the database for these options to actually take effect.
You can also enable the “Compress database design” and “Compress document data” options directly on the IBM Domino server console using the following console commands:
> load compact imtchatlog9 -n -v
> load compact imtchatlog9 -c
Keep individual chat log databases smaller by disabling replication
By default, each Instant IMtegrity chat log database has a unique replica ID and therefore does not replicate with other chat log databases on other IBM Domino servers. This keeps the size of each individual chat log database small because only chat logs captured on the IBM Domino server where the database resides are imported.
If the databases replicate, all databases contain the chat logs of all other servers, which makes the database convenient to search across all servers, but also drastically increases the database size of each replica by containing all chat logs captured by all other IBM Domino servers.
To make all imtchatlog9.nsf databases replicas of each other, set a domain-wide replica ID.
Manage the size of the chat log database by periodically deleting old documents
To keep the size of the chat log database manageable we recommend that you make use of the document archiving features available in IBM Notes and Domino.
For an overview see the IBM product documentation.
You can either simply delete chat documents older than n days from the chat log database, or first archive them to an archive database on a designated IBM Domino archive server and then delete them from the Instant IMtegrity chat log database.
Configure deletion and archiving
- first create an archiving profile in the Instant IMtegrity chat log database.
- then create a program document in the IBM Domino Directory to run the compact task periodically to execute the archiving profile.
Create an archive profile
- Open the database properties infobox for
- Click the button “Archive Settings”
- Click the “Basics” tab
- Select “Domino Server based on” and select your current IBM Domino server
- To archive to a different archiving server, select that other archive server in “to the Domino server” (If you only want to delete old chat log documents without archiving, the “to the Domino Server” option is irrelevant)
- Click the “Criteria” tab
- Delete any existing sample criteria profiles
- Click “Create…” to create a new archive criteria
- Provide a descriptive name for the criteria, like “Delete chat logs older than 30 days”
Check “Enable this criteria”
If you want to archive documents and then delete them, select the option “Copy documents into my archive application specified here, then clean up this application” and specify an archive database to archive into.
If you only want to delete documents without archiving then, select the option “Just clean up this application without archiving”
- Click the button “Selection Criteria…”
- Select the drop-down option “older than” and provide a number of days (or months)
- Select the option “In all views or folders”
Instant IMtegrity 9.1 or later
If you have the option Expire log documents after n days enabled and/or have an active action rule with the option Set expiration date, you can use the expiration date option provided by the archive selection criteria:
- Click the “Advanced” tab
Uncheck the option “Delete a document only when the criteria can delete all responses as well.”Optional
Check the option “Log all archiving activity into a log application” and specify an archive log database like imtArchivelog.nsf. Enabling this option will make it easier to verify if chat log documents get archived and deleted as expected.
NoteThe “Schedule” tab is not used for server-based deleting and archiving.
When you set up archiving, the archive tool signs the archive settings with your signature; the IBM Domino compact server task then uses this signature to verify that you have the necessary database access rights to archive/delete and then archives/deletes on your behalf. If you don’t have the necessary rights then the compact task will not archive or delete any documents, so make sure that the user who edits/saves the archive settings has sufficient rights to copy and delete documents in the database ACL.
Create a program document to run the compact task
- Open the IBM Domino Directory database
- Expand “Configuration”
- Expand “Servers”
- Click the “Programs” view
- Click the “Add Program” action button
- Set the fields to these values:
- Program name:
- Command line:
- Server to run on: <Your Domino server name>
- Program name:
- Enable the program and schedule it for a convenient time to run, like 6AM each morning.
- Save the program document.
The IBM Domino server will now schedule the compact task to archive/clean up the
imtchatlog9.nsf database every day at 6 AM, deleting and/or archiving all documents older than n days (as specified in the archive settings profile).
To verify that the compact task performs the archiving and/or deletions as expected, check the archive log database
imtArchivelog.nsf as specified in the archive settings.
If you don’t want to wait until the scheduled program document gets executed, you can also run this command manually on the IBM Domino server console:
> load compact imtchatlog9 -a
The console output should be similar to this (here we only clean up document without archiving them):
Archiving documents from imtchatlog9 (Instant IMtegrity 9 Chat Logs)
Archived imtchatlog9, 0 documents were archived and 2521 were deleted
Compacting imtchatlog9 (Instant IMtegrity 9 Chat Logs)
Compacted imtchatlog9, 1812K bytes recovered (3%)
Database compactor process shutdown
RelatedHow much space do I need to store all those chat logs?