Rate Limit Strategy

I'm trying to figure out the best way to deal with the Rate Limits. I am developing a Webservice that will use the Podio API to run custom workflows (which pull in information from 3rd party apps). This webservice will be available to many customers. My service will be react to the hooks when users Add new Items or update certain fields. Additionally, these customers will be Importing (1000's of records) at random times. This mass creation of items is going to completely bust my rate limit because hooks will be fired for each created item.

I know one of the strategies is to queue the hooks in a database and then slow down the api calls to not run into the rate limits. While this would work for some of the workflows, most of them are required to run ASAP. My users can't wait 15+ minutes before the Flow is run.

At this point my biggest worry is the importing of items. So I am testing out a system where I have the users upload the records to my database first. I then will throttle the adding of those records to their apps. This will work to the extent that I can make sure there is always a set amount of rate-limit left...I think.

Doing it this way though I'll be using the Create API call to add records. I will actually need to make 3 API calls per imported item because I need to create an item for 3 different Apps...Lead, Property and Contact. If I used the importer this can be taken care of in 1 api call I believe, however I would not be able to throttle the use of the importer.

I'm looking to find out what the best way to handle the API Rate Limit? How have some of you handled this?

Another question. Is the importer's rate limit tied to the "rate-limited" total of other "rate-limited" api-calls like Create? Or do they come out of a separate pool. My understanding is that they come out of the same pool of 1000/hr.

Thanks so much!

Official comment

Domenico Matteo

June 01, 2016 19:11

Hey Scott,

You're correct when you say that the importer's rate-limit is exactly the same "bucket" as the other rate-limited ops, like Create.

However, importing 1000s of items still count as a single (batched) operation, if done using the Importer endpoint. In your case, since you have to import into 3 applications, it will count as 3 operations, but not (3 * number of items).

Hope that helps.

Best,
Domenico

Scott Costello

June 01, 2016 19:55

Thanks for the information Domenico! My issue with the Importer is that it will fire off the "create" webhooks with each item that gets created. This will trigger more api calls. When I add the items using the "Create" api call I can tell Podio not to fire off the hook.

Is there a way I can tell Podio not to fire the hooks when using the importer endpoint? I see it has an attributes parameter.

Rate Limit Strategy

Comments

Didn't find what you were looking for?