Event correlation

The purpose of event correlation is similar to time correlation, but instead of publishing new events based on time, a triggering event decides when the events should be published. For example, when you print purchase orders in M3 using the program Purchase Order. Print Document (PPS600), multiple database updates for different purchase orders are done for the order header table and the order line table depending on different statuses, but we only want to generate one SyncPurchaseOrder BOD per purchase order. We know that there are no more database updates for this print when the program PPS601 exits. So, the program exit event for PPS601, published by M3 with document name PPS601 and operation Exit, triggers a rule. In turn, this rule triggers the event correlation rule to publish events for the printed purchase orders, based on the already received database events.

Event correlation is currently used for one standard outbound BOD, SyncPurchaseOrder. There are also custom use cases where you can use event correlation.

The event correlation rule subscribes to events published by Event Analytics, with the document name EventCorrelation and the operations Create, Request, and optionally Start. Only events published by Event Analytics rules can trigger event correlation. You need at least two rules to control event correlation: one or more data rules and a triggering rule. Optionally, you can also use an additional start rule.

Event correlation requires these correlation IDs to group events:

The virtual field correlatedEventName of the triggering rule is used to define what document name the events published by the event correlation rule have. For BODs, you should set correlatedEventName to 'M3BEBOD’ because the event correlated events are received by the M3 BOD Processor.

The virtual field correlatedEventOperation of the triggering rule is used to define what operation the events published by the event correlation rule have.

When one or more data events are received by the event correlation rule, a corresponding triggering event is expected to be received. If such an event is not received within a longer period of time, all stored data for this correlationId are automatically deleted.

If too much data is stored for event correlation, no more data is accepted and additional data is lost.

Due to timing issues for different queues in Event Hub, a data event may be received by the event correlation rule after the corresponding triggering event is received and processed. In order not to lose this data, all received triggering events are stored. When a data event is received, and its correlationId is already processed because the triggering event has already been received, and an event for the data event’s correlationId and dataEventId is not already published, a new event is automatically published by the event correlation rule. This way, no data is lost due to timing issues in Event Hub. Stored data for triggering events is automatically deleted after some time.

To explain this process, this diagram shows a horizontal timeline formatted from left to right, showing events in chronological order:

Blue arrows represent received events that have the same correlationId, where the numbers 1, 2, and 3 represent dataEventId values and T represents the triggering event. Purple arrows represent published events, and the smaller red arrows indicate which received event data is used for the published events.

The received event with dataEventId = 3 is received after the triggering event, even though this event is published before the triggering event in M3. A new event is published immediately since this correlationId is already processed, but no event for dataEventId = 3 is published. The last received event is not published because an event for dataEventId = 1 is already published for this correlationId.

The downside to this logic is that you cannot reuse correlationId values. If you must do this, you can create an additional start rule that publishes events with the document name EventCorrelation and the operation Start. This rule must specify the virtual field correlationId. When the event correlation rule receives this event, all stored data for this correlationId is deleted and you can reuse the correlationId value. New data events having this correlationId received after this event are processed when a new triggering event with the same correlationId is received. Use a chronological sequence of a start event, data events, and a triggering event to reuse correlationId values.

To explain this process, this diagram shows the same received events as for the previous image plus some additional events:

Received events marked with S represent start events. An event for the last received event with dataEventId = 1 is now published when the next triggering event is received, as the correlationId is reused.

This table shows an overview of the virtual fields used to control event correlation from data events:

These events must have the publisher EventAnalytics, the document name EventCorrelation, and the operation Create.

This table shows an overview of the virtual fields used to control event correlation from triggering events:

These events must have the publisher EventAnalytics, the document name EventCorrelation, and the operation Request.

See the standard BOD session M3BODs_PurchaseOrder, data rules MPHEAD_PrintPO_StoreKeys and MPLINE_PrintPO_StoreKeys, and triggering rule PPS601_EXIT_CREATE_PurchaseOrder for an example on how to use event correlation in Event Analytics.

This table shows the only virtual field required for optional start events:

These events must have the publisher EventAnalytics, the document name EventCorrelation and the operation Start.