AppHarbor gives 504 Gateway Time-out (nginx) while locally it retrieves data within 10-15 seconds...

mormarijn's Avatar

mormarijn

30 May, 2016 07:25 PM

Hey,

My application retrieves certain data calling an external service, which locally finishes within 10 to 15 seconds seconds, but on AppHarbor it times out after 180 seconds. My application is marijngrootjans.apphb.com.

Any idea what might be causing this extreme time-outs?

I'm desperate in solving this issue on short notice. Thanks in advance!

Kind regards,
mormarijn

  1. Support Staff 1 Posted by rune on 31 May, 2016 02:42 AM

    rune's Avatar

    Hi Mormarijn,

    Did this issue crop up recently or have you seen the HTTP 504 when invoking the action? When did you experience this issue first?

    Best,
    Rune

  2. 2 Posted by mormarijn on 31 May, 2016 12:14 PM

    mormarijn's Avatar

    Hey Rune,

    Thanks for the reply.

    I used AutoMapper 3.3.1 before and everything was working fine and fast (< 100ms), but when upgrading my site from scratch I chose to upgrade AutoMapper as well, namely, from 3.3.1 to 4.2.1. Unfortunately, the same mapping which took less than 100ms with version 3.3.1 caused the gateway time-out on AppHarbor when using version 4.2.1. When testing locally, it varied a lot; from < 100ms to > 30s.

    So, long story short: downgrading AutoMapper from 4.2.1 to 3.3.1 solved my issue, but still not sure why the same mapping is done within milliseconds using the older version, whereas it may take up to more than 30 seconds locally and even results in a time-out (180 seconds) on AppHarbor using this newer version.

    Hope to have informed you sufficiently. Thanks!

    Regards,
    mormarijn

  3. Support Staff 3 Posted by rune on 02 Jun, 2016 01:21 AM

    rune's Avatar

    Hi again,

    Ok thanks for providing this information. Sounds like it's an issue with the Automapper version you're using (or perhaps rather how it's being used - perhaps there have been changes that requires changes in your code?). I saw a couple of discussions on v4 performance on the AutoMapper GitHub page, but this one sounds particularly interesting -- a 10x performance increase in the v5-beta is quite significant. It'd be interesting to know how it compares to version 3 as I suspect there might be some change introduced in version 4 that degraded performance.

    In any case - is there a particular reason the library needs to be upgraded to the current stable release? If not then perhaps it might be better to just wait until version 5 is out of beta and stable?

    On a related note: The HTTP 504 is just returned by nginx when the backend worker doesn't respond within the configured timeout (usually 120-180 seconds). One thing you might want to test: Try and scale the application vertically to 2-4x worker units per worker, and then see how it performs on AppHarbor. If it's significantly faster the issue is likely to be CPU bound, and a temporary solution (if you want to use the current version of AutoMapper) might be to keep the app scaled. You can test this without incurring significant charges -- just scale the app down after testing and you'll only be charged the prorated fees for the time the app was scaled.

    Best,
    Rune

  4. Support Staff 4 Posted by rune on 02 Jun, 2016 01:23 AM

    rune's Avatar

    Also, I can recommend installing the Librato add-on to analyze application performance metrics. This can give you very accurate and useful measurements of CPU and memory performance and, perhaps particularly relevant with regards to AutoMapper, the number of handles being used by the application. Perhaps the app is processing a very large number of objects making the memory or CPU the bottleneck, but it's difficult to say without more data available. Let me know if you'd like me to take a look at the performance metrics when it's been installed for a few hours (and exhibited the degraded performance).

    Best,
    Rune

Discussions are closed to public comments.
If you need help with AppHarbor please start a new discussion.

Keyboard shortcuts

Generic

? Show this help
ESC Blurs the current field

Comment Form

r Focus the comment reply box
^ + ↩ Submit the comment

You can use Command ⌘ instead of Control ^ on Mac