Mailboxes

When an Account is created, its INBOX Mailbox is automatically created. The System and/or Domain Administrator can specify additional Mailboxes to be created at that time.

A user can create a Mailbox using an IMAP, MAPI, or XIMSS mailer application or using the WebUser Interface.

Mailboxes can be "nested": for any Mailbox "A" you can create a sub-Mailbox "B" - in the same way as you can created a file directory inside some other file directory. The CommuniGate Pro Server uses the slash (/) symbol as the hierarchy separator: INBOX/important is the name of the sub-Mailbox important "inside" the INBOX Mailbox.

CommuniGate Pro allows you to store messages in some Mailbox X and at the same time you can create sub-Mailboxes X/Y, X/Z for that Mailbox. This feature is implemented by providing two "invisible" Mailbox entities - one for storing messages, one - for serving as a "directory" for the nested Mailboxes. The "directory" entity is created automatically, as soon as you try to create the first sub-Mailbox. You can, though, create the "directory" entity without creating the "mail storage" entity: use the ABCDEF/ name as the new Mailbox name to create only the directory entity with the ABCDEF name. The name ABCDEF will be listed, but will not be "selectable" - and you will not be able to store messages in the ABCDEF Mailbox. You can later create the regular ABCDEF Mailbox and the "storage" entity for your ABCDEF Mailbox name will be added.

It is impossible to delete the INBOX Mailbox. You can rename the INBOX Mailbox, though. In this case a new empty INBOX Mailbox will be created automatically.

Mailbox names are case-sensitive. Some file systems (NTFS, for example) provide case-insensitive file naming conventions. When these file systems are used for CommuniGate Pro Account/Mailbox storage, the Mailbox names are still case-sensitive, but you cannot create two Mailboxes with names that differ in case only. The INBOX Mailbox name is an exception: it is always a case-insensitive name.

Messages in Mailboxes have individual flags. These flags can be set when the message is being stored in the Mailbox, and they can be updated using Mailbox access protocols and methods, such as IMAP, MAPI, XIMSS, WebUser Interface, Real-Time Applications.

Some flags are set automatically, even when the access protocol used does not support flag modification. For example, the Seen flag is set automatically when the message is being read using the POP protocol RETR command.

Several components (such as Automated Rule, CG/PL programs, etc.) can access message flags by name. They can also use "negative names" to instruct the server to reset a certain flag or to look for messages that do not have that flag set.

Besides the predefined flags CommuniGate Server supports custom flags which can be defined by user. The IMAP names of the custom flags can not contain spaces and these characters: \(){%*" The names are case-insensitive. The custom flags do not have Negative names.

The CommuniGate Pro Server maintains an Access Control List (ACL) for every Mailbox it creates.

The Access Control Lists are used to control the Foreign Mailbox Access feature that allows Account users to access Mailboxes in other Accounts.

A Server Administrator with the All Domains access right has all access rights to all Mailboxes in all Server or Cluster Accounts.

Domain Administrators with the CanViewMailboxes access right have all access rights for all Mailboxes in their Domains.

The Mailbox Account owner can grant certain limited access rights to other users, using the Access Control Lists.

l (Lookup)

If you grant a user the Lookup access right, that user will be able to see this Mailbox when it asks the Server to list all Mailboxes in your Account.

r (Read/Select)

If you grant a user the Read access right, that user will be able to open (select) this Mailbox and see (read) the messages in this Mailbox.

s (Seen)

If you grant a user the Seen access right, that user will be able to mark messages as read (seen). Usually a message is automatically marked as seen when a user reads it. But if this access right is not granted to a user reading the Mailbox, the Mailbox message "seen" status will not be changed.

w (Write/Flags)

If you grant a user the Write access right, that user will be able to set message flags: i.e. to mark messages as answered or "flagged", and to reset the message flags.

d (Delete)

If you grant a user the Delete access right, that user will be able to mark messages as deleted and to compress the Mailbox, removing all its messages marked as deleted.

i (Insert)

If you grant a user the Insert access right, that user will be able to append messages to this Mailbox and to copy messages from other Mailboxes into this one.

p (Post)

This access right is not used by modern mailers.

c (Create)

If you grant a user the Create access right, that user will be able to create new Mailboxes "inside" this Mailbox.

a (Administer)

If you grant a user the Administer access right, that user will be able:

• to modify the Mailbox ACL
• to modify the Mailbox meta-data (such as the Mailbox Class)
• to see Hidden Mailbox messages

When a sub-Mailbox is created, it inherits the ACL of the outer (parent) Mailbox. This means that if you create the INBOX/sales Mailbox, it is created with the same ACL as specified for the INBOX Mailbox.

To delete a foreign Mailbox, a user should have:

• the Create access right for the outer (parent) Mailbox, and
• the Delete access right for the specified Mailbox

To rename a foreign Mailbox, a user should have:

• the Create access right for the outer (parent) Mailbox of the original Mailbox, and
• the Delete access right for the specified original Mailbox, and
• the Create access right for the outer (parent) Mailbox of the new Mailbox (Mailbox name)

To create a foreign Mailbox, a user should have the Create access right for the outer (parent) Mailbox of the Mailbox to be created.

When the target Mailbox is a "top" one in an Account, and thus there is no outer (parent) Mailbox, the "CreateMailboxes" Account Access Right is checked instead.

The Mailbox Access Control Lists can be set and modified using the WebUser Interface, a XIMSS, a MAPI, or a decent IMAP client.

CommuniGate Pro stores received messages in Account Mailboxes. The server supports several Mailbox formats, and the Mailbox type is defined by the Mailbox file (or directory) name extension.

For single-Mailbox Accounts, the Mailbox type is specified when the Account is created.

Each multi-Mailbox Account has a setting that specifies the default type for all new Mailboxes created in this Account. A user can explicitly specify the Mailbox type creating a Mailbox in a multi-Mailbox Account: if the Mailbox name is specified as name.extension, then the Mailbox name of the extension type is created.

The Text Mailbox (.mbox) Format

If a Mailbox file has been copied from an old system, or when it is used as an Legacy INBOX and old applications can add messages to this Mailbox, some messages may have no "comment;" part. CommuniGate Pro allows a user to work with such messages, but it does not store message flags if they were modified, and it does not remember the message UIDs between sessions. The simplest solution is to copy such messages to a different Mailbox and then copy them back to the original Mailbox - the copy operation places the correct information into the From-line.

When a message is being stored in the .mbox-type Mailbox, all message lines are checked. If there is an empty line followed with the line starting with the letters From, the '>' symbol is inserted before the letter F.

The Text Mailboxes become less effective as their size grows. When a Text Mailbox is being opened, it has to be parsed, in order to detect message boundaries and retrieve the UID, flags, and other per-message information. When some messages are being deleted from the middle of a Text Mailbox, the Server has to copy the remaining messages data, compressing the Mailbox. To make these processes more efficient, the CommuniGate Pro server can deal with Mailbox data in large chunks. A special semaphore object limits the number of buffers allocated for large Mailbox processing. Changing this parameter can change the overall large Mailbox access (you may want to increase or decrease it, depending on the OS and file system you use).

To improve Text Mailbox opening speed, the CommuniGate Pro can maintain a Mailbox index (.bdx) file alongside the Text Mailbox Mailbox file. If the index file exists, the Server reads it instead of parsing the entire Mailbox file. CommuniGate Pro automatically creates an index file when it the Mailbox file size exceeds the specified limit. The Server removes the index file if the Mailbox becomes smaller than that limit.
The Index file is created when any message in the Mailbox is modified or deleted. If new messages have been added to the Mailbox, but the Mailbox has not been opened, or it has been only read without any flag modification, the Index file may not be created.

To store Message Attributes, special "invisible" messages are created in the Text Mailbox.

Use the WebAdmin Interface to specify the Text Mailbox Manager settings. Open the General pages in the Settings realm, and find the Text Mailbox Manager panel on the Others page:

Concurrently used large buffers

Use this setting to specify how many concurrent operations (parsing, deletion) the Text Mailbox Manager can perform on large Mailboxes.

Index Mailboxes larger than

Use this setting to specify the minimal size for Mailboxes that need indexing.

The MailDir Mailbox (.mdir) Format

Mailboxes with this extension are file directories. Each Mailbox message is stored as a separate file in the Mailbox directory.

Note:On the Unix platforms, the MailDir Mailboxes implement the shared storage model: if the same message is directed to many Accounts/Mailboxes, only one message file is created, and a hard link to that file is placed into each Mailbox directory.
When a message is removed from all Mailboxes, the file is automatically deleted by the OS.
This feature is disabled for the Accounts that have the "Zap Deleted Messages" Setting enabled.

To store Message Attributes, special files with Ooooo names are created inside the Mailbox directory.

Note: most freeware mail systems use either the mbox-like or mdir-like formats, and designers of those systems make various claims about the advantages of the formats they have selected. It is important to remember that:

• CommuniGate Pro does not use OS or file system features (locks) to provide multi-access to Mailboxes. CommuniGate Pro Mailboxes in all formats can be accessed by several clients and components at the same time, and all synchronization is implemented inside the CommuniGate Pro Mailbox Manager that works in the same way for all Mailbox formats.
• CommuniGate Pro uses efficient mechanisms to parse Mailboxes, so many claims about various Mailbox formats being 'slower' or 'faster' than other formats usually do not apply to CommuniGate Pro installations.
• CommuniGate Pro allows users to create Mailboxes in different formats, even within the same Account.

Note: the Text format is more efficient than the MailDir format in most cases, this is why this format is used as the default one. The MailDir format is recommended only for those Mailboxes that contain many (100 or more) large (1MB or more) messages. If a user has a Proposals Mailbox where she stores all messages with attached documents, each 0.5-5MB in size, then this Mailbox may work faster if it is created in the .mdir format.

The Sliced Mailbox (.mslc) Format

Mailboxes with this extension are file directories. These directories contain dataNNNN files, each file contains one or more messages, in the format similar to one used in Text Mailboxes.

The index.bdx file contains the mailbox index, and includes information for all messages stored in all data files.

As new messages are added to such a Mailbox, they are either appended to the existing data files, or stored in a newly created data file, depending on the message size and the size and number of messages stored in the existing data files.

When messages are removed, data files can be merged to keep the total number of data files low.

The 4th Version Mailbox (.mb4) Format

Mailboxes with this extension are file directories. All files in those directories have .mb4 extensions, or .emb4 if the Mailbox is encrypted.

The files are pseudo-textual: they may be viewed as text, but cannot be edited and may contain non-printable charcaters. The CR+LF combination is used as EOL (end-of-line) regardless of the Server OS convention.

The messages are stored in dataNNNN.mb4 files.

To store Message Flags and Attributes, mailbox index, settings and other data, some special files are created inside the Mailbox directory.

When messages are deleted by user they are only marked for deletion and not displayed to the user, and they may remain in the mailbox files for some time. The marked for deletion messages are physically removed on the server discretion. To hide the contents of the messages being deleted the "Zap Deleted Messages" Setting can be enabled.

The files from the Mailbox directory are binary and must not be converted when moving between Unix and Windows OSes.

Each Mailbox can have a Class attribute. This attribute specifies the type of the information this Mailbox is created for: Calendar, Contacts, Tasks, Notes, etc. If a Mailbox does not contain the Class attribute, it means that it is created to store regular E-mail messages.

The Mailbox Class does not restrict the types of data that can be stored in the Mailbox: E-mail and Contacts messages can be stored in Mailboxes with the Tasks Class, Notes messages can be stored in Calendar Class Mailboxes, etc. The Mailbox Class information is used with the advanced user interfaces (WebUser, MAPI) to present the Mailbox content in the proper format.

When a Mailbox is created with an advanced client interface, the interface can set the Mailbox Class. Mailbox Classes can also be updated using the CommuniGate Pro CLI/API.

Some Mailboxes have special meanings.

The INBOX Mailbox receives all incoming E-mail messages, unless these messages are explicitly directed to other Mailboxes with Rules or using Direct Mailbox Addressing.
When an Account is created, its INBOX Mailbox is created automatically.

Other special Mailboxes do not have fixed names. The Account user or an administrator can specify or modify the name used for any special Mailbox. These names are stored in Account Preferences.

The CommuniGate Pro Server recognizes Special Mailbox names (starting and ending with the $ symbols) as references to special Mailboxes. This feature allows client applications to access special Mailboxes in any Account, without being reconfigured to use the actual name of those special Mailboxes. These Special names do not show up in the Account Mailbox lists.

Trash

This Mailbox is used to store messages to be deleted. Clients can move messages from other Mailboxes to this Mailbox, using it as a "Tash can". Special name: $Trash$

Sent Messages

This Mailbox is used to store copies of sent E-mail messages. Special name: $Sent$

Drafts

This Mailbox is used to store E-mail message drafts. Special name: $Drafts$

Junk

This Mailbox is used to store junk E-mail messages. Special name: $Junk$

History

This Mailbox is used to store "log"-type messages, such as copies of the IM sessions. Special name: $History$

Default Calendar

This Mailbox is used as the Account main Calendar Mailbox. This Mailbox is expected to be of the Calendar class. Special name: $Calendar$

Default Task List

This Mailbox is used as the Account main Tasks Mailbox. This Mailbox is expected to be of the Tasks class. Special name: $Tasks$

Default Contacts

This Mailbox is used as the Account main Contacts Mailbox. This Mailbox is expected to be of the Contacts class. Special name: $Contacts$

Default Notes

This Mailbox is used as the Account main Notes Mailbox. This Mailbox is expected to be of the Notes class. Special name: $Notes$

Each Mailbox can have the Locked attribute. If this attribute is set, the Mailbox cannot be deleted or renamed.

A locked Mailbox can be deleted or renamed together with its parent Mailbox, if the parent Mailbox itself is not locked.

You can specify the Locked attribute for the Mailboxes created using the Account Template. The Mailbox Locked attribute can also be updated using the CommuniGate Pro CLI/API.

Every Account has a setting that specifies the default format for new Mailboxes that can be created in this Account.

The Account user can explicitly specify the storage format for a new Mailbox by adding the format extension to the new Mailbox name. If a user tells the CommuniGate Pro Server to create the newmailbox.mdir Mailbox, the .mdir-formatted Mailbox newmailbox is created.

The CommuniGate Pro Server allows an Account user to subscribe to some Mailboxes. The Account Mailbox subscription is a simple list of Mailbox names. This list is not used by the Server itself - the Server just stores one subscription list for each Account.

Many IMAP mailers use the Account subscription list and show only the Mailboxes the Account is subscribed to. The WebUser Interface can also be configured to show only the subscribed Mailboxes.

You can modify the Account subscription either via a decent IMAP mailer, or using the WebUser Interface.

You can use the Account Mailbox subscription to make some not-so-decent IMAP mailers access foreign Mailboxes: make sure that your IMAP client is configured to use the Account Mailbox subscription, and add the desired foreign Mailbox name into the subscription list.

Note:Some IMAP mailers tend to rebuild Account subscription lists: they empty the subscription, and then subscribe you to all Mailboxes in your own Account.

The Account Mailbox subscription is stored in the Account .info service file.

Many IMAP clients (such as Microsoft Outlook and Outlook Express) and AirSync clients (such as Windows Mobile, Apple iPhone, Nokia PDAs, etc.) cannot handle foreign Mailboxes directly, and they cannot use the Account Mailbox subscription to access foreign Mailboxes.

Mailbox Aliases can be used to let these clients access foreign Mailboxes.

A Mailbox Alias is a name associated with some [foreign] Mailbox name. For example, you can create a Mailbox Alias salesBox for the ~sales/INBOX Mailbox name. You will see the salesBox Mailbox in your client application, but in reality this will be the INBOX Mailbox in the sales Account.

Mailbox Aliases can be created only on the topmost level of the Account Mailbox hierarchy, that means that the Mailbox Alias name cannot contain the slash (/) symbol.

A Mailbox Alias provides access to the Mailbox it is associated with, and to all its sub-mailboxes (subject to Access Control List restrictions).

A Mailbox Alias can contain just a foreign Account name (~accountName). Such an Alias provides access to all accessible Mailboxes in that foreign Account. The Mailbox Alias itself is presented as an unselectable Mailbox name.

The owner of the Account chief has granted "lookup" and other access rights for his Mailboxes INBOX and Pending to the assistant Account.

The user assistant has created the Mailbox Alias boss pointing to ~j.smith.

When the user assistant connects to her Account using any IMAP or XIMSS client, or the WebUser Interface, she sees all her own Mailboxes, the unselectable Mailbox boss, and also the boss/INBOX and boss/Pending Mailboxes.

If the user j.smith creates a new Mailbox Urgent in his Account and grants access rights for that Mailbox to the assistant Account, the user assistant will immediately see the new Mailbox as the boss/Urgent Mailbox.

The CommuniGate Pro Server allows several client applications to connect, open the same Mailbox, and read and modify the Mailbox data at the same time.

The CommuniGate Pro multithreaded design allows the Server to synchronize client activities without using OS-level file locks and it does not require a client to wait till all other clients close the Mailbox.

• several clients can have simultaneous access to the same Mailbox
• new messages can be added to a Mailbox while mailer clients are working with that Mailbox
• messages can be deleted from a Mailbox while mailer clients are working with that Mailbox

Clients accessing the same Mailbox can use the same or different Mailbox access protocols - POP, IMAP, MAPI, AirSync, WebUser, or XIMSS Interface.

Simultaneous Access is supported for all Mailbox types implemented in the CommuniGate Pro software.

This feature allows you to work with your Mailbox from several workstations and devices, and it lets a group of people (i.e. the sales department) process messages in one centralized Mailbox.

The CommuniGate Pro allows an Account user to access Mailboxes in other Accounts.
Access to these foreign Mailboxes (also called shared Mailboxes) is controlled via the Mailbox Access Rights.

To access a Mailbox in a different Account, the Mailbox name should be specified as ~accountname/mailboxname. For example, to access the INBOX Mailbox in the Boss Account, the Mailbox name should be specified as ~Boss/INBOX .

If there are several local Domains on the Server, Mailboxes in a different Domain can be accessed by specifying full Account names. To access the LIST/reports Mailbox in the Account ListMaster in the client.com Domain, the Mailbox name should be specified as ~ListMaster@client.com/LIST/reports.

Account names specified after the "~" sign are processed with the Router, so Account Alias names can be used instead of the real Account names, and all Routing Table rules are applied.

• to let a secretary view and mark messages in your INBOX;
• to let several sales persons see and process a single "sales maildrop" - the INBOX of the sales Account;
• to let several engineers see and process a single "technical support maildrop" - the INBOX of the support Account.

CommuniGate Pro can provide "public" Mailboxes, too. This can be done by creating an Account public, and assigning public Access rights to its Mailboxes. Usually, each group of public Mailboxes is managed by some administrator, who is not required to be a CommuniGate Pro administrator.

A CommuniGate Pro Server administrator should create the public Account, log into that Account using the WebUser Interface or a decent IMAP client, create some public Mailboxes, and grant administration rights to regular users that will administer these public Mailboxes. Those users will then grant access rights to other users, create sub-Mailboxes, and perform other administrative tasks.

For example, a public Mailbox administrator can use Automated Rules to copy certain incoming messages directly into some public Mailbox.

Some IMAP clients (such as Microsoft Outlook and Outlook Express), and most AirSync clients do not support foreign Mailboxes at all. To let those clients access shared Mailboxes in other Accounts, Mailbox Aliases can be used.

On some systems users have direct (login) access to the mail server computer, and some of them get used to Local Mailers - mail, elm, and others. Local Mailers do not use any network protocol to access mailboxes. Instead, those programs read and modify mailbox files directly, via the file system.

The CommuniGate Pro allows you to create Accounts with Legacy INBOX Mailboxes. These Mailboxes are stored not inside the CommuniGate Pro base directory, but in the system directory known to the legacy mailer applications.

Since these INBOX files can be read and modified directly, bypassing the CommuniGate Pro protocols and modules, the Server needs to synchronize its activity with legacy mail applications using OS file locking features - either FileLevel locks or FileRange locks.

On Unix systems the FileLevel locks are known as flock operations, and RangeLevel locks are known as fcntl operations. Check with your OS manual to see which method the legacy mailers use on your system, and configure the CommuniGate Pro Server to use that method. For systems that support only one file locking mechanism (MS Windows, Sun Solaris, and some other systems), selecting either method selects that mechanism.

You should use Legacy Mailboxes only when absolutely necessary, because:

• access to Legacy Mailboxes is less effective as it consumes resources needed for OS file locking.
• local mailer applications do NOT synchronize correctly, and there is always a chance that a local mailer destroys a mailbox. If some of your users have to work via local mailers, warn them about this fact, and ask them to avoid using local mailers and modern (protocol-based) mailers at the same time. These bugs in most local mailers do not show up if messages are only added to the mailbox when a local mailer is active. Local mailers may corrupt mailboxes if messages are deleted from a mailbox during the time a local mailer is active.

If you have to support Local Mailer compatibility for all or some Accounts in a Domain (usually - in the Main Domain), you should specify the Legacy INBOX settings for that Domain.

When you create an Account that has an Legacy INBOX, the Server checks if the Account INBOX file already exists in the specified location and creates one if the Mailbox file is absent.

When you delete an Account that has an Legacy INBOX, the Server does NOT remove the INBOX Mailbox file.

Log records created for Mailbox-related events have the MAILBOX tag.
The level of records being recorded for Mailboxes is specified in Domain settings.

The records contain Account name and Mailbox name. Non-ASCII characters in Mailbox names are encoded in modified UTF-7.

The records level 0 (Crashes) contain messages about critical failures which require attention of the Server administrator.

The records level 1 (Failures) contain messages about failures which can be fixed by the Server automatically or ignored.

The records level 2 (Major & Failures) contain messages about:

• Mailbox opening and closing
• appending a message
• deleting a message
• message flags changing
• message attributes changing

• creation of a Mailbox
• Mailbox deletion
• Mailbox renaming
• changing Mailbox access rights

The records level 3 (Problems) partially duplicate the messages of level 2, but contain more detailed information about the operations performed (size of mail messages, specific names of the flags to be changed, etc.).

The records level 4 (Low Level) and 5 (All Info) contain messages about reading mail, as well as other messages specific to the particular folder format (about operations with the file system, etc.).

Example:

 1) … 2 MAILBOX(user@domain/INBOX) #1 opened by IMAP-000022, 1 open
 2) … 2 MAILBOX(user@domain/INBOX) #2 opened by WEBUSER-000033, 2 open
 3) … 2 MAILBOX(user@domain/INBOX) #3 opened by LOCAL-000001, 3 open
 4) … 3 MAILBOX(user@domain/INBOX) {181} appended [1] @7454: 70+1169 bytes
 5) … 2 MAILBOX(user@domain/INBOX) #3 {181} appended
 6) … 2 MAILBOX(user@domain/INBOX) [17701] stored as {181}
 7) … 2 MAILBOX(user@domain/INBOX) #3 closed, 2 open
 8) … 3 MAILBOX(user@domain/INBOX) #2 flags update (1): +Seen,Deleted
 9) … 2 MAILBOX(user@domain/INBOX) #2 {181} flags updated
10) … 2 MAILBOX(user@domain/INBOX) #2 closed, 1 open
11) … 2 MAILBOX(user@domain/INBOX) #1 {181} deleted
12) … 2 MAILBOX(user@domain/INBOX) #1 closed, 0 open

Interpretation:

 1) the mailbox INBOX of user@domain Account was opened by IMAP-000022 with session #1, there is 1 open session
 2) the mailbox was opned by WEBUSER-000033 with session #2, there are 2 open sessions
 3) the mailbox was opened by LOCAL module with session #3, there are 3 open sessions
 4) a message with ID 181 was appended to the mailbox into file 1 at offset 7454: there are 1169 bytes of data and 70 of metadata
 5) the session #3 appended the message with ID 181 to the mailbox
 6) the message with ID 17701 from Queue was srored into the mailbox with ID 181
 7) the session #3 was closed, there are 2 open sessions
 8) the session #2 requested updating flags of one message: adding flags Seen and Deleted
 9) the session #2 changed flags of message 181
10) the session #2 was closed, there are 2 open sessions
11) the session #1 deleted the message 181
12) the session #1 was closed, there are no more open sessions