Helper Applications

To specify the External Helper program path and other parameters, open the General page in the Settings realm of the WebAdmin Interface and click the Helpers link:

Helper Name
Log Level:	Crashes OnlyFailuresMajor & FailuresProblemsLow LevelAll Info	Program Path:
Time-out:	Never5 sec10 sec15 sec20 sec30 sec45 sec60 sec90 sec2 min3 min5 min10 min15 min20 min30 min40 min45 min60 min	Auto-Restart:	Never5 sec10 sec15 sec20 sec30 sec45 sec60 sec90 sec2 min3 min5 min10 min15 min20 min30 min40 min45 min60 min90 min2 hour(s)3 hour(s)5 hour(s)6 hour(s)

The checkbox next to the Helper name tells the Server to start the specified program as a separate OS process.

Log

Use this setting to specify the type of information the Helper support module should put in the Server Log. Each Helper uses its own tag for its Log record.

Program Path

The file name (path) of the Helper program. If a relative path is specified, it should be relative to the CommuniGate Pro base directory.
If the first setting symbol is $, then the rest of the setting string should be a path relative to the CommuniGate Pro application directory.

Time-out

If the Helper program does not send a response within the specified period of time, the program is stopped.

Auto-Restart

If the Helper program stops, and this option is disabled, all pending requests are rejected. If the Helper program stops when this option is enabled, the Server waits for the specified period of time, restarts the Helper Program and re-sends the requests to it.

For security purposes the names of external programs that are launched by CommuniGate Pro Server must be specified in the Child Processes Manifesto. The Manifesto is a text file Settings/ChildProcs.data from the CommuniGate Pro base directory.
An external program will not be launched if the initial (left) part of its path/name does not match one of the lines from the Manifesto.

In addition to Helper Programs, the Manifesto also applies to other programs that can be run by the Server:

• to CGI programs and scripts
• external Spell Checker programs
• external applications for delivery via PIPE module
• programs launched by 'Execute' action of a Rule

The names of the Programs depend on the Server OS. On Windows OS, program names are case-insensitive; on Unix-like OSes, names are case-sensitive.

Lines of the Manifesto, in addition to the path and name of the Program, can also include the following command line parameters of the Program. Whitespaces matter.

Example for Windows OS:

CGPKAV\CGPKAV.exe
CGPKAS\CGPKAS.exe
c:\perl64\bin\perl authLDAPNewAD.pl

Example for Unix-like OS:

./mmindex-helper
CGPKAV/CGPKAV
CGPKAS/CGPKAS
/usr/local/bin/aspell -a --lang=

Note: by specifying the / and . lines in the Manifesto for Unix-like OSes, you can allow programs to run with any names.

When the Helper program is launched, the Server sends commands to the Helper process via the process standard input. The Server reads the program responses from the process standard output.

Commands and responses are text lines, ending with the EOL symbol(s) used in the Server OS.

Each command starts with a sequence number, and the response produced with the Helper program starts with the same number. This method allows the Helper program to process several requests simultaneously, and it can return responses in any order.

The Helper program can send information responses at any time. An information response starts with the asterisk (*) symbol. The Server ignores information responses, but they can be seen in the Server Log.

The response lines generated with a Helper program should not be larger than 4096 bytes.

Note: communication between the Server and an Helper program takes place via OS pipes, and many programming libraries buffer output data sent to pipes. Check that your Helper program uses some form of the flush command after it sends a response to its standard output, otherwise the response will not reach the Server.

Helper programs are started with the CommuniGate Pro base directory as their current directory.

Helper programs should not write anything to their standard error streams, unless they want to report a reason for the failure before quitting. CommuniGate Pro reads the program standard error stream only after the program has terminated, and if the program writes into that stream while processing Server commands, the program will be suspended by the OS when the standard error pipe buffer is full.

The Interface Version command is used to provide compatibility between different versions of Helper programs and different versions of the CommuniGate Pro Server. The Server sends this command specifying the protocol version it implements:
nnnnnn INTF serverInterfaceVersion
where:

nnnnnn

a unique sequence number for this request

serverInterfaceVersion

the version of the Helper protocol implemented by this version of the CommuniGate Pro Server

The Helper program should return the INTF response and the supported protocol version.
nnnnnn INTF programInterfaceVersion
If the returned number is smaller than the Server protocol version, the Server will use this (older) protocol version.

When the Server shuts down or when it needs to stop the Helper program, it sends the QUIT command, and then closes the process standard input. The Helper program should send the OK response and it should quit within 5 seconds.

Sample session (I: - server commands sent to the program standard input, O: - responses the program writes to its standard output, COMMAND - a Helper-specific command):

O: * My Helper program started
I: 00001 INTF 1
O: 00001 INTF 1
I: 00002 COMMAND parameters
O: 00002 OK
I: 00003 COMMAND parameters
I: 00004 COMMAND parameters
O: * processing 00003 will take some time
O: 00004 ERROR description
O: 00003 OK
I: 00005 QUIT
O: * processed: 5 requests. Quitting.
O: 00005 OK
I: stdin closed

The sample above shows that the Server does not wait for a response before it sends the next command, and that it can accept responses for several pending commands in any order - as long as each command receives a response within the specified time limit.

External Authenticator programs can be used to provide authentication, provisioning, and routing services using external data sources.

The External Authenticator Interface protocol is based on the generic Helper Protocol.

This manual describes the External Authenticator Interface Version 11.

When a user should be authenticated using the clear text method, the Server sends the following command:
nnnnnn VRFY (mode) name@domain password [loginAddress]
where:

nnnnnn

a unique sequence number for this request

mode

the name of the service (IMAP, POP, FTP, etc.) that requested this authentication operation.
This parameter can be absent if the request has been received from an unnamed Server component.
If the service name is specified, it is enclosed into the parenthesis.

name

user Account name

domain

user Account Domain name

password

password string to verify. If the password contains special symbols, the password is encoded as a quoted String.

loginAddress

the network address from which the user logs in.
This parameter can be absent if the password is checked by an internal Server component.
If the parameter is specified, it is enclosed into square brackets.

When a user should be authenticated using a secure SASL method, the following command is sent:
nnnnnn SASL(method) (mode) name@domain password key [loginAddress]
where:

method

the name of the secure SASL method used (CRAM-MD5, APOP)

key

the challenge string sent to the client mailer. If the challenge contains special symbols, it is encoded as a quoted String.

If the password is accepted, the External Authenticator should return a positive response:
nnnnnn OK

If the password was not accepted, a negative response should be returned:
nnnnnn ERROR optional-error-message

If the password is accepted, and there is an authentication response to be returned to the client, a positive response with a quoted string should be returned:
nnnnnn RETURN "authentication-response"

SASL password verification requires that the External Authenticator program correctly implements all supported SASL methods and algorithms. Alternatively, the External Authenticator program can return the user plain text password, making the Server verify the password and calculate necessary authentication responses. The user plain text password should be returned as a quoted string:
nnnnnn PLAIN "plain-text-password"

Sample session (I: - server commands sent to the program standard input, O: - responses the program writes to its standard output):

I: 00001 INTF 1
O: 00001 INTF 1
I: 00010 VRFY user1@domain1.com dsyui134
O: 00010 OK
I: 00011 VRFY (IMAP) user2@domain2.com jskj23#45 [10.0.3.4]
O: 00011 ERROR incorrect password
I: 00012 SASL(CRAM-MD6) user4@domain2.com hdkj547812329394055 <pop-23456@mydomain.com> [10.0.1.4]
I: 00013 VRFY (IMAP) user2@domain2.com "jskj23\"45"
O: 00013 OK
O: 00012 ERROR unsupported SASL method
I: 00014 SASL(DIGEST-MD5) user4@domain2.com 012345 "user:qop:zz:mmm:uri" [10.0.1.4]
O: 00014 RETURN "0123456789AAAA"
I: 00015 SASL(DIGEST-MD5) user4@domain2.com 012345 "user:qop:zz:mmm:uri" [10.0.1.4]
O: 00015 PLAIN "my$$password"

The External Authentication program can be used to retrieve plain text passwords from external databases.

To retrieve a password, the Server sends the following command:
nnnnnn READPLAIN name@domain
where:

name@domain

the full name (address) of the target Account.

The program should return the user plain text password as a quoted string:
nnnnnn PLAIN "plain-text-password"
If the program cannot retrieve the plain text password, it should return the FAILURE response.

The External Authentication program can be used to process unknown names, too. For example, the program can consult an external database, check if the user exists in that database, create an Account, Alias, Group, Mailing List, or Forwarder using the CommuniGate Pro CLI/API, and return a positive response to the Server. In this case, CommuniGate Pro will re-try to open a Domain object with the specified name.

To check an unknown name, the Server sends the following command:
nnnnnn NEW name@domain relayType
where:

relayType

[MAIL] if the command is sent as a part of mail-type routing process,
[SIGNAL] if the command is sent as a part of signal-type routing process,
[ACCESS] if the command is sent as a part of access-type routing process.

name@domain

the full name (address) of the unknown local object.

If the program sends the OK response, the Server tries to find the name object in the domain Domain again.

If the program sends the ROUTED address response, the Server takes the supplied address response and restarts the Router procedure with this address. The routed address gets a "can Relay" attribute, unless it is specified with the [NORELAY] prefix.

If the program sends the FAILURE response, the Server Router returns a "temporary internal error" code (this code causes the SMTP module to return a 4xx error code, not a permanent 5xx error code).

If the program sends any other response, the Server Router returns the "unknown user account" error.

Sample session:

I: 00010 NEW user1@domain1.com [MAIL]
O: 00010 ERROR this account is not known
I: 00011 NEW user2@domain2.com [MAIL]
I: 00012 NEW user3@domain2.com [ACCESS]
O: 00012 OK
O: 00011 ROUTED [NORELAY] userX@domain2.com

The Consult External for Unknown Domain Setting tells the Server to use the External Authenticator program when an unknown name is addressed.

The External Authentication program can be used to assist in address Routing. If an address is routed to the @external domain, the address "local part" is passed to the External Authentication program using the ROUTE command:
nnnnnn ROUTE <address> relayType
where:

relayType

[MAIL] if the command is sent as a part of mail-type routing process,
[SIGNAL] if the command is sent as a part of signal-type routing process,
[ACCESS] if the command is sent as a part of access-type routing process.

address

the local part of the address with the external domain part.

If the program sends the ROUTED address response, the Server takes the supplied address response and restarts the Router procedure with this address. The routed address gets a "can Relay" attribute if the address is specified with the [RELAY] prefix.

If the program sends the FAILURE response, the Server Router returns a "temporary internal error" code (this code causes the SMTP module to return a 4xx error code, not a permanent 5xx error code).

If the program sends any other response, the Server Router returns the "cannot route the address" error.

Sample session:

I: 00010 ROUTE <user1> [MAIL]
O: 00010 ERROR this account is blocked
I: 00011 ROUTE <user2%domain1.dom> [MAIL]
I: 00012 ROUTE <"user3##name"%domain2.dom> [SIGNAL]
O: 00012 FAILURE internal error
O: 00011 ROUTED [RELAY] userX@domain100.dom

The External Authentication program can be used to assist in provisioning operations. If the Consult External on Provision Domain Setting is enabled, the Server sends the following commands to the External Authentication program:

before an Account is created:

nnnnnn PRECREATE [authAccount] accountName@domainName accountType initialSettings
initialSettings is a dictionary.
If this operation fails, the Account is not created.

after an Account is created:

nnnnnn POSTCREATE [authAccount] accountName@domainName accountType initialSettings
If this operation fails, the newly created Account is removed.

before an Account is renamed:

nnnnnn PRERENAME [authAccount] accountName@domainName newAccountName@newDomainName
If this operation fails, the Account is not renamed.

after an Account is renamed:

nnnnnn POSTRENAME [authAccount] accountName@domainName newAccountName@newDomainName
If this operation fails, the renamed Account is renamed back.

before an Account is removed:

nnnnnn PREDELETE [authAccount] accountName@domainName
If this operation fails, the Account is not removed.

after an Account is removed:

nnnnnn POSTDELETE [authAccount] accountName@domainName

before an Account License Class is changed:

nnnnnn PRETYPECHANGE [authAccount] accountName@domainName newClass
If this operation fails, the Account License Class is not changed.

after an Account License Class is changed:

nnnnnn POSTTYPECHANGE [authAccount] accountName@domainName newClass

before an Account settings are updated:

nnnnnn PREUPDATE [authAccount] accountName@domainName newSettings
If this operation fails, the Account settings are not updated.

after an Account settings are updated:

nnnnnn POSTUPDATE [authAccount] accountName@domainName newSettings

before an Account password is updated:

nnnnnn PREPWDCHANGE [authAccount] accountName@domainName newPassword
If this operation fails, the Account password is not updated.

The password is one of Account settings, so the PREUPDATE command will follow this one.

authAccount is the name of the Account performing the provisioning operation. If it is not known, this account name and the square brackets surrounding it are omitted.

The program should either send an OK response, or a FAILURE "errorCode" response.

External Message Filter programs are used for content filtering (anti-virus and anti-spam filtering).

The External Filter Interface protocol is based on the generic Helper Protocol.
This section describes the External Filter protocol version 4.

• The program should process the External Filter requests:
  seqNum FILE fileName

  where fileName is the name of the file the program should scan.

  
  
• If message processing should proceed, the response line should have the following format:
  seqNum [ modifiers ] OK

  where modifiers is zero, one, or several keywords (each with an optional parameter), separated using the space symbol:

  ADDHEADER header-field-text

  header-field-text is a text string to be added to the message header. It should contain one or several RFC822/RFC2822 header fields.
  This (optionally multi-line) header field text should be placed into the response line using the String format.

  MIRRORTO address

  address is a text string to mirror the message to.
  The string should contain exactly one E-mail address. To mirror the message to several addresses, use several MIRRORTO modifiers.
  The address text should be placed into the response line using the String format.

  ADDROUTE address

  This modifier is supported when the Filter is used in Server-wide and Cluster-wide Rules only.
  address is a text string to add to the message "route set", i.e. to the list of addresses this message is to be delivered to.
  The string should contain exactly one E-mail address. To add several addresses, use several ADDROUTE modifiers.
  The address text should be placed into the response line using the String format.

  
  Examples:
  1077 OK
  1078 ADDHEADER "X-SPAM-SCORE: 100\eX-SPAM-FILTER: CGateProSpamFilter(r)" OK
  1079 ADDHEADER "X-SPAM-SCORE: 100" MIRRORTO "spamreport@mydomain.com" MIRRORTO "abuse@mydomain.com" OK
• If a message should be rejected the response line should have the following format:
  seqNum ERROR report

  where report is a text string explaining why the message is rejected.
  This (optionally multi-line) text should be placed into the response line in the CommuniGate Pro String format.

  
  
• If a message should be discarded the response line should have the following format:
  seqNum DISCARD
  
  
• If message processing should be postponed (because of the license limitations, for example), the response line should have the following format:
  seqNum REJECTED report

  where report is a text string explaining why the message processing should be postponed.

  
  
• If the program receives a request it cannot process, it should return the FAILURE response:
  seqNum FAILURE
  If the program has sent the FAILURE response or any unlisted response, the Server places a record into its Log and proceeds as if it has received the OK response.
  
• The program SHOULD be ready to process several requests simultaneously (using several threads). Since the External Filter program is used with the Server-Wide Rules (processed with the Enqueuer Server component), the program should be ready to handle N concurrent requests, when N is the number of Enqueuer "processors" (threads).
  
• The program MAY be implemented as a single-threaded one, so it reads the next request only after the previous request has been processed. But this design can result in severe performance degradation of the entire Server: when a single-threaded External Filter program is scanning a large message, other messages are not being enqueued.
  

If the external program crashes, CommuniGate Pro suspends the Enqueuer process until the external program is restarted.

External RADIUS programs can be used to provide additional checks for authentication operations performed via the RADIUS module, as well as to add additional attributes into RADIUS responses.

The External RADIUS Interface protocol is based on the generic Helper Protocol.
This manual describes the External RADIUS Interface Version 2.

If the External RADIUS program is enabled, it is used after the user password is verified. The Server sends it the following command:
nnnnnn LOGIN name@domain attributes settings
where:

nnnnnn

a unique sequence number for this request

name

user Account name

domain

user Account Domain name

attributes

a dictionary with all request attributes.

settings

a dictionary with the Account settings.

If the login request is accepted, the Helper program should return a positive response:
nnnnnn ACCEPT attributes
where:

nnnnnn

the request sequence number

attributes

a dictionary with the attributes to be added to the RADIUS response.

If the password was not accepted, a negative response should be returned:
nnnnnn REJECT optional-error-message

If the External RADIUS program is enabled, it is used to process the Start, Stop, and Interim-Update accounting requests. The Server sends the following command:
nnnnnn ACCNT command name@domain attributes
where:

nnnnnn

a unique sequence number for this request

command

the accounting command (started, ended, updated)

name

user Account name

domain

user Account Domain name

attributes

a dictionary with request attributes.

The Helper program should return a positive response:
nnnnnn OK
where:

nnnnnn

the request sequence number

The attributes in dictionaries should use the attribute type numeric values as keys (for example 27 for Session-Timeout).

The following attributes are interpreted as 32-bit integer values and they are encoded as numeric strings in dictionaries:
NAS-Port, Service-Type, Framed-Protocol, Framed-Routing, Framed-MTU, Framed-Compression, Login-Service, Login-TCP-Port, Framed-IPX-Network, Session-Timeout, Idle-Timeout, Termination-Action, Framed-AppleTalk-Link, Framed-AppleTalk-Network, Event-Timestamp, NAS-Port-Type, Port-Limit, ARAP-Zone-Access, Password-Retry, Prompt, Tunnel-Type, Tunnel-Medium-Type, Tunnel-Preference, Acct-Interim-Interval, Acct-Delay-Time, Acct-Input-Octets, Acct-Output-Octets, Acct-Authentic, Acct-Session-Time, Acct-Input-Packets, Acct-Output-Packets, Acct-Terminate-Cause, Acct-Link-Count, Acct-Input-Gigawords, Acct-Output-Gigawords.

The following attributes are interpreted as 32-bit IP addresses and they are encoded as aaa.bbb.ccc.ddd strings in dictionaries:
NAS-IP-Address, Framed-IP-Address, Framed-IP-Netmask, Login-IP-Host.

The following attributes are ignored in Helper responses:
User-Name, User-Password, CHAP-Password, State, Proxy-State, EAP-Message, Message-Authenticator, Acct-Status-Type.

All other attribute values are encoded either as a String or as DataBlocks. The Server uses the DataBlocks format for those attribute values that contain bytes outside the hexadecimal 0x20-0x7F range.
The DataBlock format must be used if the value contains binary zero bytes.

If an attribute has multiple values, the attribute value is encoded as an Array.

The following attributes are added to the attribute dictionaries passed to the Helper:

0

the RADIUS protocol request ID. It can be used to detect retransmitted packets (duplicate requests)

secretKey (authentication requests only)

a string value with the RADIUS module "shared secret" setting.

authData (authentication requests only)

a 16-byte DataBlock containing the "authentication data" portion of the RADIUS request.

The Vendor-specific attributes are presented using keys with negative numeric values. The key absolute value is the "VendorID" value. For each VendorID, the associated element is a dictionary. This dictionary keys are Vendor-specific "vendor types", with associated "specific attributes". The "specific attributes" values can be stored as String, DataBlocks, Number, or Arrays of Strings, DataBlocks, and/or Numbers.

Sample session (I: - server commands sent to the program standard input, O: - responses the program writes to its standard output):

I: 00001 INTF 1
O: 00001 OK 1
I: 00002 LOGIN user1@domain1.com {0=#15; 1="User1";4=10.0.0.1;32="NAS 1";31=4992713154;"-311"={9=#777;10="ZZZ";}; authData=[AbndghAbndgh1sjkjkss3T=]; secretKey=a123;} {RealName="User"; NATIP="192.168.1.3";}
O: 00002 ACCEPT {8=192.168.1.3; 9=255.255.255.0; 13=(0,3);}
I: 00003 LOGIN user1@domain1.com {0=#16; 1="uSEr1";32="NAS 2";31=415.5512.12; 8=192.168.1.3; authData=[Abnd278sjkljsljkjksFG=]; secretKey=a123;} {NATIP="10.0.1.114";}
O: 00003 REJECT
I: 00004 ACCNT started user1@domain1.com {0=#17;1="uSEr1";32="NAS 2";31=415.5512.12; 8=192.168.1.3;}
O: 00004 OK

Note: the Server can send several concurrent requests for the same target Account.

Note: the External RADIUS program is called when the target Account is open. In a Dynamic Cluster system this means that External RADIUS programs should run on backend servers, and that External RADIUS programs running on different backend servers will never get requests for the same Account at the same time.

External CDR Processor programs can be used to process CDRs (Call Detail Records) generated with the Signal component when a call is attempted, established, or tiered down. They can also process CDR records generated with CG/PL applications.

The External CDR Processor Interface protocol is based on the generic Helper Protocol.
This manual describes the External CDR Processor Interface Version 1.

If the External CDR Processor program is enabled, the Signal module generated CDRs and sends them to the program.

When a CDR is composed, the Server sends the following command:
nnnnnn CDR cdr_data
where:

nnnnnn

a unique sequence number for this request

cdr_data

CDR data in the Signal component format or in the CG/PL Application format.

when the record is processed, the program should return a positive response:
nnnnnn OK

External Load Balancer programs are used to control software load balancers for the Dynamic Cluster installations. These programs should be installed and enabled on all Cluster members that can act as load balancers. For each "balancing groups" the Cluster Controller selects one of the available load balancers and activates it, while other load balancers work in the "backup" mode.

The External Load Balancer Interface protocol is based on the generic Helper Protocol.
This manual describes the External Load Balancer Interface Version 1.

When the Cluster Controller detects a change in the cluster member belonging to this Load Balancer Group, the program receives the following command:
nnnnnn MEMBERS (ip-address [,ip-address...] ) (ip-address [,ip-address...] )
where:

nnnnnn

a unique sequence number for this request

ip-addresses

Two sets of Network IP addresses. The first one lists all currently available cluster members, the second list - all currently disabled (but running) cluster members. Each set may contain zero or more addresses, separated with the comma (,) symbol. Each address is the "Server WAN IPv4 Address" of the cluster member, as specified in its Network Settings.

When the program starts, or when the Server detects a change in its "Server WAN IPv4 Address" Network Setting, the program receives the following command:
nnnnnn LOCAL ip-address
where:

nnnnnn

a unique sequence number for this request

ip-address

the current "Server WAN IPv4 Address" Network Setting of this Server.

When this load balancer needs to be activated, the program receives the following command:
nnnnnn STARTBALANCER

When the Cluster Controller needs to deactivate this load balancer, the program receives the following command:
nnnnnn STOPBALANCER

External Application Helper programs can be used to provide CG/PL applications with arbitrary data..

The External Application Helper Interface protocol is based on the generic Helper Protocol.
This manual describes the External Application Helper Interface Version 2.

When a client requests data, the Server sends the following command:
nnnnnn USER user@domain REQ requestData
where:

nnnnnn

a unique sequence number for this request

user@domain

the name of the account that owns the context for the request to the helper.

requestData

the request data from the application, which may be a string, array, dictionary or datablock.

When the request is processed, the program should return a positive response:
nnnnnn RESP responseData
where:

responseData

the text presentation of the response data.

Sample session (I: - server commands sent to the program standard input, O: - responses the program writes to its standard output):

I: 00001 INTF 2
O: 00001 INTF 2
I: 00010 USER john@dummy.dom REQ "What time is it?"
O: 00010 RESP "11:18pm"

If the External Application Helper program is not running, any request immediately produces an empty response.

External Banner System programs can be used to provide XIMSS and other clients with "banner" data (advertising information that a client presents to the user).

The External Banner System Interface protocol is based on the generic Helper Protocol.
This manual describes the External Banner System Interface Version 1.

When a client requests a banner, the Server sends the following command:
nnnnnn BANNER bannerType [ accountName@domainName ] [ INFO bannerSetting ] [ PREFS bannerPreference ] [ PARAM paramData ]
where:

nnnnnn

a unique sequence number for this request

bannerType

if the banner type string specified with the client application (specifies the client application and its banner type, for example: samowareEmailTop, myClientLeftBanner).

accountName@domainName

full name of the Account requesting a banner.

bannerSetting

(optional) the BannerInfo Account Setting value.

bannerPreference

(optional) the BannerClass Account Preference value.

paramData

(optional) the text presentation of the parameter object specified with the client application.

When the record is processed, the program should return a positive response:
nnnnnn RESULT resultData
where:

resultData

the text presentation of the banner info.

The program can also return a blocking response:
nnnnnn BLOCK
The specified bannerType is added to the list of "blocked" types. If any client requests a banner of a "blocked" type, an empty response is returned immediately, without calling the External Banner System program.

If the External Banner System program is not running, any banner request immediately produces an empty response.