Journaling for CommuniGate Pro

The purpose of Journaling is to save copies of sent and received messages, keep them for some time period, and automatically delete them when the period expires. This guarantees you will have a backup in the event of an accidental incoming mail deletion from the user's side.

Compared to other methods of saving copies of messages (via a Rule or Legal Intercept) Journaling has the following advantages:

It's easy to deploy for specific groups of users, or for all users.
The envelope addresses are saved so messages can be restored for right users.
Old messages are automatically deleted so there is minimal impact on storage requirements.

Installation

Make sure you have Perl language installed. Type this in the OS commans shell:

perl -V

For Windows OS a free Perl interpreter may be downloaded from ActiveState.

CommuniGate Pro version 6.0.11 or newer is required.

Download the Journaling script Journaler.pl

Language	CPU	Download
Perl	Any

Store it into CommuniGate Base directory; make it executable.
The script requires CLI.pm Perl module, get one from www.communigate.ru/CGPerl/CLI.pm and store the file one of the @INC directorues or Perl.

cd /var/CommuniGate/
wget http://www.communigate.ru/CGPJournaler/Journaler.pl
chmod +x Journaler.pl
wget http://www.communigate.ru/CGPerl/CLI.pm
cp CLI.pm /usr/share/perl5/

Configuration

Configure the Journaling script:

Open the script in a text editor and change this line:

my $PostmasterPassword='pass';

Specify your postmaster password.

Launch the script:

Open the General page in the Settings section of the WebAdmin Interface and click the Helpers link. Create a Helper with the following settings:

Content Filtering


Log Level:		Program Path:
Time-out:		Auto-Restart:

Note: For Windows OS the Program Path should be as c:\perl\bin\perl.exe c:\CommuniGateFiles\Journaler.pl
Note: In a Cluster environment the above should be done on all nodes which process the message queue, normally on Frontends.

To invoke the Helper create a server-wide (or cluster-wide) Rule in Settings->Mail->Rules page with the arbitrary name/priority and the following contents:

Data	Operation	Parameter



Action	Parameter
	Journaler

On first launch the script will create journals.local domain (shared domain if you're in Dynamic Cluster environment) with some pre-defined settings and account defaults, and with a sample Journal account inside.

Configure the Journal:

Open the journals.local domain, open the j-mydomain sample Journal account settings page. Change the AddPattern, SkipPattern and days to retain values.

Type:	Multi-Mailbox	Access Rights
Created:	4:55:37AM
Real Name:
State/Province:
City:
Unit:
CommuniGate Password:
Phone:

Those values are stored in "st", "l" and "ou" (State/Province, City and Unit, respectively) settings, which initilly are not designed for that, but those fields exist in the account settings by default and convenient to use for our purpose.
Wait for 10 minutes or restart the helper to make the changes take effect.

The AddPattern and SkipPattern are Perl regular expressions where
. matches any one character
.* matches any sequence of characters
\d matches a digit
\. matches the dot character
(abc|def) matches either abc or def
[a-d] matches a range of characters from a to d
Use only lowercase letters.
The data after '#' is a comment; leading and trailing spaces are ignored; empty patterns don't match anthing.

If a message's Return-Path address or any of the envelope recipients match the AddPattern and don't match the SkipPattern then the message is stored into the Journal. Use SkipPattern to exclude messages from certain users from being journaled; however those messages will be journaled if they're sent from/to another user who is journaled.

Examples of patterns:

.*@mydomain\.com: all accounts in "mydomain.com" domain
.*@(mydomain|otherdomain)\.com: all accounts from two domains
admin.*: accounts from all domains (including external) with names starting with "admin" string.

You may create multiple journaling accounts with different AddPattern and SkipPattern to serve different domains or users within same domain. A journaling account must be in the journals.local domain and the account name must start with j- characters.

Restoring messages from Journals

Download the restoring script:

Language	CPU	Download
Perl	Any

Configure the restoring script:

Open the script in a text editor and change these lines:

my $CGServerAddress='127.0.0.1'; #IP or domain name;
my $Password='pass';

Specify the postmaster password and CommuniGate server address.

The script can work over network, i.e. no need to launch it directly from the server machine.

Launch the script:

Usage:

perl jRestore.pl [options] user@domain: Restore messages of the user@domain account.

Options:

--target account@domain/mailbox: Specifies the target account and mailbox where to the messages are copied. The target account must exist; if the target mailbox doesn't exist it will be created automatically. If target is not speciied the messages will be copied into INBOX mailbox of the user@domain account.
Sample:
perl jRestore.pl --target admin@domain.com/abcd user@domain.com
Look for messages from/to user@domain.com account and copy them into "abcd" mailbox of admin@domain.com account.
--journal name: The Journal name, without j- prefix.
If not specified, it will find the right Journal by checking the restored user name against AddPattern and SkipPattern of all Journals.
Sample:
perl jRestore.pl --journal mydomain user@domain.com
Look into "mydomain" Journal only.
--days number: Restore emails not older than this number of days.
If not specified, it will restore everything available.

Depending on the Journal size and server performance the restoring may take long time, so in order to prevent the interruption of the search process it's recommended to launch the sript via 'nohup' and in daemon mode as:

nohup ./jRestore.pl user@mydomain.com &

Only error messages are printed; the information about search progress is written into jRestore.log file which you can watch while the script is running.

Support

To contact authors please email to support@communigate.ru address.