What is the recommended number of rows that can be loaded in a single batch?
- For an initial load, you can use 30,000 rows for a large batch.
- For ongoing loads, you can use 20,000 rows for a large batch.
- You should not exceed 100,000 rows in a large batch.
- Furthermore, you should limit the number of records in the interface tables to those that are being processed.
- For example, if you have determined that the optimal batch size for your system is 19,000 rows per patch and you are going to be running 8 parallel EIM processes, then you should have 152,000 rows in the interface table.
- Under no circumstances should you have more than 250,000 rows in any single interface table, since this will have a tremendous negative performance impact.
NOTE: The number of rows you can load in a single batch may vary depending on your physical machine setup. To reduce demands on system resources and improve performance, you should always try to use smaller batch sizes.