Ledger Import considerations

There are limitations in SSC when processing a large number of records with different journal numbers.

A large number of entries on the work tables, created by ledger import processing, can result in an "out of memory" error in the Journal / Ledger component. Additionally, the SQL statement created to retrieve the data can become too complex and may appear to hang, or time out. This is because the generated SQL select statement is a union over the work tables, and a set of work tables is created per journal.

There are several parameters that affect the number of entries written to the work tables. To minimise the number of work table entries, set 「Report Errors = yes」 and 「Suppress messages for insignificant substituions = yes」. To maximise the number of work table entries, set both parameters to "no".

Minimising the work table entries can still result in an "out of memory" error, due to the complexity of the SQL query. That is, the number of unions generated is related to the number of distinct journals in the payload. For example, 1000 errors over 500 journals may cause issues whereas 1000 errors for a single journal may not.

Therefore it is difficult to proivde guidelines on reducing the possibility of this issue occurring. Simply that it is better to run more imports containing fewer individual journal numbers.