Tuesday, January 14, 2014

Recommended number of rows that can be loaded in a single batch

What is the recommended number of rows that can be loaded in a single batch?

  1. For an initial load, you can use 30,000 rows for a large batch.
  2. For ongoing loads, you can use 20,000 rows for a large batch.
  3. You should not exceed 100,000 rows in a large batch.
  4. Furthermore, you should limit the number of records in the interface tables to those that are being processed. 
  5. For example, if you have determined that the optimal batch size for your system is 19,000 rows per patch and you are going to be running 8 parallel EIM processes, then you should have 152,000 rows in the interface table. 
  6. Under no circumstances should you have more than 250,000 rows in any single interface table, since this will have a tremendous negative performance impact.

NOTE: The number of rows you can load in a single batch may vary depending on your physical machine setup. To reduce demands on system resources and improve performance, you should always try to use smaller batch sizes.