Interesting Issue when I hit 71 api calls in 60 seconds
Answered
Background: I'm creating some php scripts to auto-create a Podio Workspace. I need to create 1 WorkSpace, 8 Apps and roughly 150 fields. So that is 159 calls that will be made to the Api in order to create all the apps and fields. I'm sure I could throw some of the Field create scripts into the App create Call to limit my total calls but that isn't the strange problem I'm having.
Issue: I've got everything working great if I run everything one at a time. Now if I start adding the Api calls to one php script, I can run up to 70 api calls and have no issues. As soon as I add the 71st call to the script I start having problems. Let me see if I can explain what is happening...It seems as once the 71st call gets reached, the script starts over again (creating a new workspace, 8 apps and 61 fields). Then it will do it again and again until I seem to hit my rate limit. I take a look at my Workspace Manager and I'll see 7+ workspaces created, each with all the apps and fields.
I look at the Log file and there is no error message. I just see the 70th api call get executed with code 200 and then it jumps back to the very first api call that creates the workspace and starts all over again.
Also it doesn't matter what call I have as the 71st one, so I can't say that it's one particular thing that is causing this.
I don't have any loops in my script where it might cause a reset. It's almost as if after the 70th call the php page just reloads and starts over again. I'm fairly new to php so I wonder if this is a php thing.
Any help would be appreciated because I'm lost here!
-
Hi Scott,
It sounds like something could be wrong with your program, but it's hard to help out without looking at the code. Would you be able to share the relevant parts, e.g. on https://gist.github.com/ ?
Thanks,
Andreas -
Here you go...
The top level script is the Podio-Setup.php
-
Hi Andreas,
I got it all to work with a combination of adding fields through the PodioApp::create() call and the PodioAppField::Create() call. I reduced down the number of api calls to a point where it works. I'm still puzzled as to why I'm having a problem when I pass 70 api calls from one PHP file though. Even though I managed to get my current solution working, I'm afraid if I need to expand what I'm doing I will run into the same issue.
-
Perhaps there is a PHP execution timeout and your browser just reloads the url which starts the script again. I'm a bit puzzled why it would reload, but who knows. Check your PHP settings - make page with just phpinfo(). Also, you could run your script from the command line as it needs no interaction (well, you would have to change to user auth).
Another idea - have you thought of making a "source workspace" somewhere and just clone the source APPs via API?
I'm quite sure there is no "71 API calls in one minute" limit in the API ;)
-
Hi Henrik,
If there was a timeout somewhere in php settings I would think that the number of calls would change because I'm sure speed of execution changes between runs. I'll check that out anyway because you know what happens when you assume :)
Let's talk about your source workspace idea...I like it for a number of reasons. Would the install location have to be a member of the sources corporation? In other words, follow the same rules as if you where to install a pack of apps from a private app market pack?
-
Scott - ok, I thought the 60 seconds in your post title was an exact time.. but best to check, 60 seconds sounds quite a possible timeout somewhere.
About the "source workspace" - what is required is just what is required by cloning, right? So the user who is cloning must have access to both source and target workspaces. No need for the workspaces to be in the same organization.
However I like your method of creating via API much more, so forget about my suggestion. Via API I feel you have more control and it is more reliable (aside from your current issue ;D). For instance if you have a lot of relationships between the apps, cloning between workspaces is tricky. Relationship fields silently drop away while cloning if the target-app of the relationship does not exist in the target workspace.
-
Henrik,
The 60 seconds was just a rough time frame because that is what it seemed like the breaking point was. I didn't really give it much thought because the number of API calls was so consistent that I thought there was a cap or something.
I did some testing with my .htaccess settings for time out and execution time and it didn't seem to have any affect on things. I'm completely at a loss for what is going on. I'll keep running tests.
However...with the combination of creating fields using the PodioApp::Create(), PodioApp::Update() and the PodioAppField::Create(), I have trimmed my Api calls down to 34. This works perfectly and completes the setup of my Workspace, Apps and Fields. The entire process takes 15 seconds according to the Log.
I wish I could just create all the fields in the PodioApp::Create() calls, but because I have calculation fields I need to get the field ID of the fields I need to use in the calc fields first. The only way I figured out how to get those IDs is to use PodioAppField::Create() to create any field i need the ID for first and then use the PodioApp::Update() call to create the rest of the fields. It works but it's not as clean and ordered as I would like.
-
I did some more testing and actually uploaded the php scripts to another host I had (bluehost) and ran some tests. I had ZERO issues executing 125+ api calls. So my conclusion is that my other host (media temple) is the cause of the problem and there is some setting that is behind the issue over there. Perhaps because it's a Wordpress Hosting account they do have some special php settings to keep performance up.
Gonna transfer all my scripts over to my bluehost account and continue over there.
Thanks Henrik and Andreas for helping out
Please sign in to leave a comment.
Comments
9 comments