Welcome Guest! Log in
Stambia versions 2.x, 3.x, S17, S18, S19 and S20 are reaching End of Support January, 15th, 2024. Please consider upgrading to the supported Semarchy xDI versions. See Global Policy Support and the Semarchy Documentation.

The Stambia User Community is moving to Semarchy! All the applicable resources have already been moved or are currently being moved to their new location. Read more…


Symptom

When loading data from Postgresql tables you can sometimes experience "Java Heap Space" error during the execution of your process. You may also notice that executions with Postgresql tables in source take a lot of runtime memory.

 

Solution

This is caused by the default behvaiour of the Postgresql jdbc driver, that loads all the source records into the runtimes memory (as oppposed to standard behaviour which consists of loading the lines in batches -> the number of lines in a batch is configured in "SQL Fetch Size" option) as explained in the postgresql doc :

https://jdbc.postgresql.org/documentation/83/query.html#query-with-cursor

To force the driver to respect the fetchSize declared in Stambia templates you might try to switch into transactional mode (in which the fetchSize is respected).

If the template does not allow transactionalMode or for some reason cannot use it, in the Designer S19.0.15 and runtime S17.6.4 we have changed the way the data is selected on postgresql tables to force the driver to send the data in batches. So if you encounter the problem described above you should make sure to develop your mapping with Designer 19.0.15 or newer and run the developped process with runtime 17.6.4 or newer.

 

Knowledge Base

Suggest a new Article!