Welcome Guest! Log in
×

Notice

The forum is in read only mode.
Stambia versions 2.x, 3.x, S17, S18, S19 and S20 are reaching End of Support January, 15th, 2024. Please consider upgrading to the supported Semarchy xDI versions. See Global Policy Support and the Semarchy Documentation.

The Stambia User Community is moving to Semarchy! All the applicable resources have already been moved or are currently being moved to their new location. Read more…

Topic-icon Question Limit number of rows in a mapping

More
09 Jun 2016 00:09 #1 by Shahin
Limit number of rows in a mapping was created by Shahin
Bonjour,
I have a mapping from a DB to a JSON webservice. I would like to only load 100 rows at a time, but I cannot figure out how to do this? Is there a filter that can do this?

Thanks!
More
09 Jun 2016 16:39 #2 by Cyril Dussud
Replied by Cyril Dussud on topic Limit number of rows in a mapping
Hi,

In a Mapping a target WebService is invoked for each value of the repetition key.
So if you have 1000 source rows, the target WebService will be called 1000 time sequentially.

Can you give a little more detail on what you want to accomplish ?
If you want the Mapping to load only 100 rows, you could filter the source
More
09 Jun 2016 17:47 #3 by Shahin
Replied by Shahin on topic Limit number of rows in a mapping
We have product data in Semarchy and need to export that to Netsuite via JSON WS. Our ultimate goal is to do this as fast as possible. In the past, simply running the mapping as is with not filters on the entire data set took days to complete. I was hoping to run the mapping multiple times in parallel to increase the throughput. Is there a better way to increase the performance?

How would I create a filter on the source? I only see a way to filter at the attribute level. Please be specific in your answer and be aware because of the fact I am in the US there are time zone differences.

Thanks for the help.
More
11 Jun 2016 18:55 #4 by mdahlman
Replied by mdahlman on topic Limit number of rows in a mapping
Let's close this thread. I'm working with Shahin.
I know how to query to get N records and then submit them via web services calls in M batches. Basically it's just a SQL Operation bound to the execution of a Delivery.

But I'll open a new thread with a more precise question about an issue I'm seeing with running these sessions in parallel.
More
13 Jun 2016 09:26 #5 by Thomas BLETON
Replied by Thomas BLETON on topic Limit number of rows in a mapping
Thanks Matt, indeed making batches of rows is usually done with SQL.
For your information, we plan to improve our WS features so that one can specify parallelization params.