Very slow fetch

Hello,

I noticed the requests today are very slow today (over 1 minute). I usually fetch 600 results in a loop (6 times 100 results)

It makes a simple request lasting over 7 minutes.

Is this a problem with the server or is it related to my application ?

(I use an API key)

Yes, it’s a problem with the servers, someone was making huge numbers of request in a short period of time while the primary server was doing OS updates, this in turn burns through our ‘burstable CPU credits’ and when these run out everything grinds to a halt and we can do nothing until the CPU credits acrue/reset again. This is spread across 3 API servers. I could throw more resources or scaling at it but really that would just costs me more money :slight_smile: - so we need people to stop spamming the API (which I’m afraid as you have described is exactly what you are doing).

Can you explain more about what you are trying to achieve with your approach? I’m almost certain it’s the wrong way to do whatever it is you’re trying to do. If you really need to do it that way then you need to immediately start using your own API mirror.

I hope not, I have been trying to limit my results in numbers with max results and in area with an encoded polyline in the URL. The app is not public, it’s a student project only, so it’s just me using it.

I chose to use a loop in order to harmonize results. For example if I plan a route from France to Estonia I had a lot of results at the origin but vast areas with no results. It is a problem because the app is trying to calculate a route for electrical vehicle, and if there are huge gaps between results, gaps bigger than the car autonomy, it cannot calculate a route.

I didn’t want to fetch thousands of results as a solution, it’s counter-productive and wastes a lot of data. My solution is to calculate a complex polyline from origin to destination and to divide in segments of 500 km. in order to make smaller fetches limited to 100 results on a distance of 50 km. on each side from the polyline only, in Json.

If your app is ‘EloptioAlfa’ then you have recently hit the API with large batches of simultaneous requests (many hundreds in the space of a few seconds). These look like denial of service attacks to our API.

If you want to implement your own routing I’d recommend you run a mirror of the API otherwise just request a larger bounding box that covers more of your total area with compact=true&verbose=false. You’ll need max results to be higher. Your box can be a rectangle, which typically will cover any sane route between two places. You should ideally cache results when building up your working set especially if you plan to offer multiple route variations. Our API accepts encoded polylines and polygon (polyline= or polygon=) if that helps refine your data gathering mechanism (Encoded Polyline Algorithm Format  |  Google Maps Platform).

Our API is built by volunteers (me) and hosting/data transfer if paid for by me. It’s very important (to me) that users treat it with care.

Looking at the most recent logs I can see 3419 requests within 3 minutes (from your app, if it’s the EloptioAlfa one) averaging 14 requests per second.

My app is not EleptioAlfa, it’s called “Route finder”. Your work is really great and I have no intention of abusing it. I think my app usually does no more than 6 requests per minute, and only when it is prompted to calculate a route. May be 12 once in a while if I use the app twice. Most of the time it’s zero bc the app is mostly idle when I am not working with it.

This is my code for your info : Electrical-vehicle-route-calculator-frontend/map.component.ts at master · Reivilo85k/Electrical-vehicle-route-calculator-frontend · GitHub . The URL query is line 171.

As you can see I use polyline and max results to limit the impact of my app.

I have use verbose and comments true because I intend to display info about the charging stations.

Thanks, glad to hear that’s not your app. Unfortunately they are making things slower for everyone else and not responding to emails.

I’m investigating various strategies and optimizations. You can expect the API to get faster over the new few weeks.

1 Like

By the way, nice app. In your code I think you are firing the queries but then returning without waiting for the results of your Promise. An async method should return a Promise, so that the calling method can await for that promise to finish.

Sorry, I did not push my last commit before sharing my code. The return statement had been removed.

I do have a bug somewhere in my app, it somehow fails to calculate very long distances, and I have been thinking for a long time this piece of code was the culprit. I thought the problem was indeed asynchronous and the array containing the markers created from fetched results was incomplete. I even created a question about if on StackOverflow this week : javascript - How to force callback method to wait for Google Markers creation - Stack Overflow

However it turns out the Promise is correctly returned and all EV charging station markers are created before the route calculation method is launched so I have to look elsewhere, may be in the method selecting which marker is closer to destination on line 375.

Thanks for looking at my code and for the praise, I am counting on this app to help me find my first job as a Junior.

It’s nice work, keep hunting for the bugs and break it down into smaller sections to find any issue you might have. Your demo is perhaps out of date but it isn’t using an API key for it’s OCM requests. The message ‘no route found’ comes up well before the OCM API calls complete (if one call takes 1.5s for instance), you can see this happen in the network tab on chrome devtools - the API response hasn’t been received yet.

I use an API Key, but I did not put it on github for best practice.

On my version the call is complete before the alert unless I am mistaken. At least this is what I assumed because it is not written as “pending” anymore.

I also created logs in console to check if markers were created before the route calculation was called

Capture2

If the call is not complete wouldn’t it still be be mentionned as “pending” or do I misunderstand something ?

Yeah maybe the online heroku demo just behaves differently to your local version. I tried building locally to see if I could fix anything but I don’t have the config for google maps etc.

If you are interested in looking more into it I’d be happy sending you the config details fort a local install
tbh. You can PM me at [email protected].

1 Like

For info I did find my bug. The method computeDistanceBetween() from Google geometry library returns a string variable and not a number.

Later on when the app is selecting the marker closest to destination it assumed 1100 km. was closer than 930 km. because it was a string. A little parseFloat fixed the entire algorithm.

1 Like