Forrester View

Archive instant messages and e-mail together

If you archive e-mail, you may need to archive instant messages too. By Erica Rugullies (above) with Connie Moore & Lucy Fossner

Instant messages (IMs) are a cross between conversation and correspondence. They are more permanent than a face-to-face or phone discussion, and more fleeting than an e-mail or letter. But legal experts and regulatory bodies have begun to view and treat IMs as just another form of business communication.

For example, in February 2005, the National Association of Securities Dealers (NASD) fined a research analyst at Fulcrum Global Partners for circulating rumours about a company via IM and phone calls, while simultaneously short-selling that company’s stock. Firms that are concerned enough about their e-mail to archive it should also consider archiving IMs. These organisations include

• Financial services firms

These firms are beholden to the rules and regulations of NASD, the New York Stock Exchange (NYSE), and the Securities and Exchange Commission (SEC). NASD Conduct Rule 3010 explicitly requires that firms archive their brokers’ and dealers’ e-mail and IMs. SEC Rule 17a requires that certain business records and communications be readily accessible for two years and accessible (not necessarily online) for the year after that. Transaction-related communications must also be kept and made accessible for seven years after the event. The NYSE put out a memo in March 2003 stating that SEC Rule 17a-4 explicitly requires the archiving of both e-mail and IMs.

• Companies that must comply with the Sarbanes-Oxley Act

Sarbanes-Oxley does not explicitly mention IMs, e-mail, or other forms of communication. But it has heavy record-keeping requirements for public accounting firms and the customers they serve. The Act says that any client of a public accounting firm may be required to produce documents related to audits or investigations. It is conceivable that these items could include e-mail and IMs.

Export to third-party archive is the best option for IMs

Firms that want to archive their instant messages have three main options: use the basic archiving capabilities of enterprise IM platforms; implement an IM gateway, which provides more advanced archiving features; or export IMs from an IM gateway into a message archiving system.

Enterprise IM platforms

Enterprise IM platforms provide limited internal archiving capabilities. For example, IBM Lotus Instant Messaging and Web Conferencing logs all IMs through a built-in logging API. It doesn’t, however, provide archive management tools. Microsoft Live Communications Server 2005 has a logging mechanism, licenced to Microsoft by IMlogic, that routes messages to an SQL Server database.

With either platform, changing the default logging behaviour requires customisation. In the case of IBM, this means redirecting message logs to a Lotus Notes NSF or relational database, and in the case of Microsoft, it means using the server-side API to create custom post-processing routines. Neither product, when used alone, makes it easy for firms to adhere to the supervision requirements of NASD 3010. Using the capabilities of the IM platform is the least desirable option for firms that are trying to achieve regulatory compliance.

IM Gateways

IM gateways such as Akonix L7 Enterprise and Enforcer, FaceTime IM Auditor, and IMlogic IM Manager, log all IM activity, regardless of the IM network being used. They then transfer log files to a relational database, which is most often Microsoft SQL Server or Oracle. FaceTime is unique in that it provides a Linux appliance called RTG500 and RTShield, which does the traffic monitoring and logging.

These products also provide supervision features to enable reviewers to monitor and act on end-user compliance. FaceTime offers especially advanced features in support of supervision: guaranteed logging and export to the archive; archiving messages in the order they are sent; anti-tampering and checksums; non-repudiation; and guaranteed non-circumvention.

Firms that are trying to achieve regulatory compliance could choose this option, but it does not enable them to consistently manage policy across message formats. IMs are archived in one place, while e-mail are archived in a separate place, and other types of records are stored in a third place, possibly a records management (RM) system.

Message Archiving Systems

The leading IM gateway vendors all provide an export tool and interfaces to message archiving systems from vendors like EMC, IBM, iLumin Software Services, Iron Mountain, Open Text, Veritas Software and Zantaz. Most IM gateway vendors include this export capability in the base price of their products; Akonix Systems charges extra for it.

The IM gateway vendors provide two forms of export: simple mail transport protocol (SMTP) export and XML export. The latter provides a much richer set of metadata, indexing, retrieval and reporting capabilities. IM gateways use XML forms to parse IM conversations into e-mail messages that have SMTP headers. These e-mail are then exported to the archiving system, sometimes via a short stop at a mail server. IM gateways also consume logs from enterprise IM systems, and these logs can also be sent into the archive.

Exporting IMs from an IM gateway into a message archiving system is the recommended approach for firms that are trying to achieve regulatory compliance. It allows firms to archive e-mails, IMs and attachments in a single repository, and to standardise storage management practices. The message archiving vendors provide compression, single instance storage, and integration with write-once/read-many devices.

An integrated approach allows firms to apply storage and retention policies consistently across various types of stored content. Reviewers can perform a single supervision activity, and attorneys can discover all IMs and e-mail that meet a set of criteria, in a single system, with a single set of training requirements. As enterprise content management vendors’ RM solutions begin to provide supervision tools and become better able to handle large volumes of e-mail, the best practice will evolve to archiving IMs and e-mail in those systems rather than in specialised message archives.

Leading IM gateways have a similar approach to archive integration

All of the leading enterprise IM gateways integrate with third-party message archiving systems in the same basic way. A system administrator:

• Creates and schedules exports in the IM gateway

The export process transforms IMs into e-mail messages by applying SMTP headers. IM conversations are stored in the body. Typically, the initiator of the conversation appears in the ‘from’ field, all other participants appear in the ‘to’ field, and the date of initiation appears in the ‘date’ field. The subject field contains a conversation ID number or perhaps just a few words indicating that the message contains an IM conversation. Exports can be scheduled for any time or frequency. In the case of Akonix, they happen continuously.

• Determines which metadata and messages will be exported

When XML integration is used, the system administrator can select specific metadata fields for export—metadata about conversations, users, and events. These metadata elements may include the conversation’s start and end-time; IM network(s) used; participant count; message count; number of messages blocked or flagged; number of files or bytes transferred; number of viruses found; conversation status and reason; user names (screen, network, and full); user IP and SMTP address; and the amount of time users spent in the conversation.

• Filters which messages are exported

Akonix can be configured to archive only those conversations that violate a policy. FaceTime offers filters by multiple views of the same conversation (duplicates), conversation type (internal or external), and network type (all or just selected networks). IMlogic allows administrators to capture conversations based on the IDs of the participants as well as dates and times.

Then the system then takes over and:

• Pulls user information from the enterprise directory

When the IM gateway queries the enterprise directory as part of the export process, exported messages can be enriched with any existing lightweight directory access protocol (LDAP) attributes associated with sender or recipient IDs. This additional metadata appears in the body of the message. If LDAP data is included in exported messages, reviewers and others can search IMs in the message archive on a greater range of fields.

• Sends the message to the mail server or directly to the message archive

In some cases, the IM gateway provider has written to the message archiving vendor’s API and can send XML-formatted messages directly over the network to the message archive. In other cases, the IM gateway sends messages via SMTP to a designated mailbox on the mail server where journaling is turned on. The message archive then picks the message up from the mail server just like it would for any other e-mail, and stores it in the archive where it applies a retention policy.

But legal discovery and supervision experience varies

Investigators who are engaged in legal discoveries need tools to search through vast message stores to find e-mail and IMs that meet specific criteria, while NASD requires that reviewers actively monitor registered reps’ e-mail and IMs for compliance. To meet this requirement, reviewers need tools to search through archives for messages that may be out of compliance with policy. Most message archiving solutions offer some form of supervision capability. But the user experience for reviewers and investigators is inconsistent across these products because:

• Some IM gateways send plain text messages to the archive

When messages are sent into the message archive in plain text format, the message archive has limited metadata on which to index the messages. This limits the ability of reviewers and other searchers to streamline and refine their queries. Akonix provides this level of integration with most of its message archiving partners, and FaceTime and IMlogic provide this level with some of their partners.

• Others send XML files or add customised headers

When messages go into the archive as XML files that are compliant with the message archiving system’s extensible style sheet language (XSL) style sheet, the archiving system gains additional attributes on which users can index and search. These additional attributes can also be added to the headers of the message sent to the archive.

Recommendations

Consolidate e-mail and instant message archives IM archiving should not be handled in isolation, especially when the reason for archiving is regulatory compliance. Assess whether you should be archiving IMs If you archive e-mail, you may also need to archive IMs. Keep all messages in one archive system Archive IMs in the same repository as e-mail to gain the benefits of a consistently-applied policy—a single place in which to perform supervision activities, a reduced number of locations in which to undertake discovery, and reduced training requirements. Add message archiving integration to IM gateway evaluation criteria When making IM gateway purchase decisions, a priority evaluation criterion should be whether, and to what degree, the system integrates with your existing or planned message archiving solution(s). Add IM gateway integration to message archiving evaluation criteria When making message archive solution decisions, a priority evaluation criterion should be how that vendor handles IMs in its system. The focus should be on how easy the system makes it to search and review IMs. This comes down to the level of integration that the message archiving vendor provides with IM gateways. Shortlist tightly integrated solutions For the most integrated indexing and search experience within the message archive, shortlist combinations.