Exceptionally long loading times

Hey,

we have a problem with our Indico server events.geomar.de. Since some time the server takes very long to build the pages (sometimes 1min +). The server runs on a Debian 5.10.92-1, with 4 cores and 4GB memory.

As a test we have dumped the database, indico.conf and the archive to our identical development system, here these problems do not occur, even if the server has relatively long response times (2-5sec).

Often the server waits ~1min for JS and FONTS files (see Firefox screenshot two to four).

The difference between the two machines is that the production system is behind a haproxy and the URL events.geomar.de does not point directly to the server.

The problems occur especially with Firefox, Chrome and Opera. With Safari, however, not (see screenshots - all with cache turned off).

Chrome:

Firefox:




Opera:

Safari:

Very strange. I’d check the logs on the haproxy and indico server to see where the bottleneck is. Also check if there are any errors in the nginx/apache log for indico - most of the static files are usually served directly by the webserver bypassing the indico webapp altogether (see the aliases in the webserver config)

I think it could ressource starvation, as we run with 8 GB RAM and have a ram usage ~ 50% on a not heavily loaded system (< 1req/sec). We are actually using nginx instead of apache as this tends to be a bit slimmer. So I would recommend to check resource usage.

Seems strange. We run e.g. this instance on a single VM with 4 cores and 7GB of memory just fine.

As much of the RAM usage is caching of fixed data, I would expect it to be mostly independent of load and size of the site. Load times here are <200msec for all assets and content. We do not observe degradation during peak times as well.

Same problems with haproxy and apache. What was your solution?