HostGator Slowness and server overload

[alexmbn]

Hello I have installed one of the latest version of Arrowchat 1.5.6.

Since that, I have been experiencing a very high level of server overload and database crashes (more than 1 mn to load pages !!!!).
Support from HostGator analysed and told me Arrowchat is the guilt :-).

They have updraged Apache and PHP versions on my server.
I have seen on another thread you mention FAST CGI.

What do you recommand exactly to use Arrowchat the best way on a VPS server for example.

Good tool anyway.

Regards
Patrice

[Jason]

HostGator doesn't configure there servers very well at all, so I have seen a few people have trouble with them.

I would either recommend our push service or using mod_fcgid. If you use mod_fcgid, you MUST have a settings configuration file for that mod (cpanel doesn't automatically create it).

[alexmbn]

Hi, here is the result for arrowchat.

I would appreciate some more information. What is mod_fcgid, what configuration settings ...
For the time being, the arrowchat has been disabled. A pity.

We appreciate your patience.

At this time it appears there is a component that you have installed on morbonet.com that needs an excessive amount of memory to run, causing the container to run out of memory and Apache to crash

The arrowchat component is using around 500mb per process on average and it has multiple processes running at a time. The total memory available for your VPS is 1.3GB meaning that 2 of these processes already use almost all the memory available.

23876 dgoc 25 0 586m 466m 7332 R 49.8 34.7 0:01.80 /usr/bin/php /home/dgoc/public_html/arrowchat/includes/json/receive/receive_buddylist.php
23879 dgoc 23 0 451m 331m 7332 R 49.8 24.7 0:01.33 /usr/bin/php /home/dgoc/public_html/arrowchat/includes/json/receive/receive_core.php

Apache kept constantly crashing while this component was enabled. We had to disable access to it so that it would not keep crashing. It appears that to be able to use this component will need to upgrade to a higher VPS plan that provides more memory as it looks like it's very memory intensive. In the current VPS level, leaving this component enabled will continue to cause crashes as it will run out of memory

Aug 21 12:48:17 vps77 kernel: OOM killed process php (pid=6502, ve=205) exited, free=157779 gen=393775.
Aug 21 12:48:20 vps77 kernel: OOM killed process php (pid=4759, ve=205) exited, free=21338 gen=393776.
Aug 21 12:48:20 vps77 kernel: OOM killed process httpd (pid=4579, ve=205) exited, free=21475 gen=393777.
Aug 21 12:48:20 vps77 kernel: OOM killed process httpd (pid=4610, ve=205) exited, free=21581 gen=393778.
Aug 21 12:48:20 vps77 kernel: OOM killed process php (pid=6929, ve=205) exited, free=57144 gen=393779.
Aug 21 12:48:20 vps77 kernel: OOM killed process httpd (pid=4749, ve=205) exited, free=212270 gen=393780.
Aug 21 12:48:20 vps77 kernel: OOM killed process httpd (pid=4599, ve=205) exited, free=212425 gen=393781.
Aug 21 12:48:20 vps77 kernel: OOM killed process httpd (pid=4609, ve=205) exited, free=212983 gen=393782.
Aug 21 12:48:20 vps77 kernel: OOM killed process leechprotect (pid=4600, ve=205) exited, free=214378 gen=393783.
Aug 21 12:48:20 vps77 kernel: OOM killed process php (pid=4820, ve=205) exited, free=286771 gen=393784.
Aug 21 12:48:20 vps77 kernel: OOM killed process php (pid=6874, ve=205) exited, free=333398 gen=393785.
Aug 21 12:51:09 vps77 kernel: OOM killed process php (pid=10501, ve=205) exited, free=83244 gen=393786.
Aug 21 12:51:09 vps77 kernel: OOM killed process httpd (pid=9190, ve=205) exited, free=83895 gen=393787.
Aug 21 12:51:12 vps77 kernel: OOM killed process php (pid=16817, ve=205) exited, free=93796 gen=393788.
Aug 21 12:51:12 vps77 kernel: OOM killed process php (pid=10461, ve=205) exited, free=186298 gen=393789.
Aug 21 12:51:18 vps77 kernel: OOM killed process php (pid=16825, ve=205) exited, free=45261 gen=393790.
Aug 21 12:51:27 vps77 kernel: OOM killed process php (pid=17257, ve=205) exited, free=88414 gen=393791.
Aug 21 12:51:31 vps77 kernel: OOM killed process php (pid=16883, ve=205) exited, free=20173 gen=393792.
Aug 21 12:51:31 vps77 kernel: OOM killed process httpd (pid=9021, ve=205) exited, free=20111 gen=393793.
Aug 21 12:51:31 vps77 kernel: OOM killed process httpd (pid=9030, ve=205) exited, free=20206 gen=393794.
Aug 21 12:51:31 vps77 kernel: OOM killed process php (pid=17371, ve=205) exited, free=49704 gen=393795.
Aug 21 12:51:31 vps77 kernel: OOM killed process httpd (pid=9004, ve=205) exited, free=49797 gen=393796.
Aug 21 12:51:31 vps77 kernel: OOM killed process php (pid=17334, ve=205) exited, free=90350 gen=393797.
Aug 21 12:51:31 vps77 kernel: OOM killed process httpd (pid=17113, ve=205) exited, free=131022 gen=393798.
Aug 21 12:51:31 vps77 kernel: OOM killed process php (pid=16830, ve=205) exited, free=133014 gen=393799.
Aug 21 12:51:31 vps77 kernel: OOM killed process httpd (pid=9029, ve=205) exited, free=133628 gen=393800.
Aug 21 12:51:31 vps77 kernel: OOM killed process leechprotect (pid=9022, ve=205) exited, free=135023 gen=393801.
Aug 21 12:51:31 vps77 kernel: OOM killed process php (pid=17345, ve=205) exited, free=138745 gen=393802.
Aug 21 12:51:31 vps77 kernel: OOM killed process php (pid=17175, ve=205) exited, free=235147 gen=393803.
Aug 21 12:51:31 vps77 kernel: OOM killed process php (pid=17140, ve=205) exited, free=323796 gen=393804.
Aug 21 12:52:09 vps77 kernel: OOM killed process php (pid=19405, ve=205) exited, free=122183 gen=393805.
Aug 21 12:53:47 vps77 kernel: OOM killed process httpd (pid=19066, ve=205) exited, free=19377 gen=393806.
Aug 21 12:53:47 vps77 kernel: OOM killed process httpd (pid=19112, ve=205) exited, free=19344 gen=393807.
Aug 21 12:53:47 vps77 kernel: OOM killed process httpd (pid=19044, ve=205) exited, free=19439 gen=393808.
Aug 21 12:53:47 vps77 kernel: OOM killed process php (pid=24691, ve=205) exited, free=23723 gen=393809.
Aug 21 12:53:47 vps77 kernel: OOM killed process httpd (pid=19074, ve=205) exited, free=23909 gen=393810.
Aug 21 12:53:47 vps77 kernel: OOM killed process leechprotect (pid=19067, ve=205) exited, free=25426 gen=393811.
Aug 21 12:53:47 vps77 kernel: OOM killed process httpd (pid=19210, ve=205) exited, free=27002 gen=393812.
Aug 21 12:53:48 vps77 kernel: OOM killed process php (pid=24592, ve=205) exited, free=195130 gen=393813.
Aug 21 12:53:48 vps77 kernel: OOM killed process php (pid=24648, ve=205) exited, free=267376 gen=393814.
Aug 21 12:53:48 vps77 kernel: OOM killed process php (pid=24677, ve=205) exited, free=301075 gen=393815.
Aug 21 12:53:48 vps77 kernel: OOM killed process php (pid=24681, ve=205) exited, free=301354 gen=393816.
Aug 21 12:53:48 vps77 kernel: OOM killed process php (pid=24579, ve=205) exited, free=323740 gen=393817.


Regards

[alexmbn]

or at least a minimum configuration of arrowchat to have it going well (backend settings to make it not so load).

Patrice

[alexmbn]

Hi, other information : FCGI already installed...

root@mor [/home/dgoc/public_html]# /usr/local/cpanel/bin/rebuild_phpconf --current | grep -i php5
PHP5 SAPI: fcgi

[Jason]

Hi, other information : FCGI already installed...

root@mor [/home/dgoc/public_html]# /usr/local/cpanel/bin/rebuild_phpconf --current | grep -i php5
PHP5 SAPI: fcgi

Make sure the configuration file is filled out properly.

[alexmbn]

Hi,
ok but
what config file and what to fill out ?
Is there any documentation ?

Cheers

[CAlchin]

Hope you get there; I'm following this thread very closely. I'm on a VPS account with hostgator and had issues running red5 with arrowchat on the container. Red5 kept on running out of memory and crashing.

After switching to the arrowchat push servers the load was lifted and the red5 server never crashes.

I'll look forward to the result as inevitably I may have to drop the push service in the unforeseeable future.

Best of luck!

[alexmbn]

Well, here is a new piece of information : Hostgator support who have been very helpful for years do no accept the fact their configuration is bad / Arrowchat team words : "HostGator doesn't configure there servers very well at all, so I have seen a few people have trouble with them".

I think here David (Arrowchat) is criticising Goliath "HostGator army / Just N°1 VPS in the world".

My point of view is that the best thing would be for Arrowchat to do some cleaning of their code and accept the fact their software is really cool be not really light. It is too heavy, too slow, crashing a VPS4 website every 24h with heavy php routines.

I have FCGI installed. The site has been deeply analysed. The only issue is ... Arrowchat.

And, by the way, stop telling guys with the same issue to take a push service, this is just not a good strategy if you want to stay in the market.

Again, the tool is really good, just needs the arrowchat team to accept webmasters critical and constructive advices and improve it.

Cheers

[Martok]

Interesting. I am with Dreamhost on a standard shared hosting plan (i.e. not VPS) and I have absolutely no problems running Arrowchat at all. VPS is better than standard hosting, so I can't understand why there would be issues running Arrowchat on a VPS plan with Hostgator when a standard shared hosting plan on Dreamhost is fine, unless you either have an awful lot of users using Arrowchat or if the issue is with Hostgator themselves.

[alexmbn]

Hi Martok,

Thank you for your concern.
Actually, I had no problem until last release.
Which one do you have installed ?

Other information is that a shared server might be more flexible adapting to the load (I am not really sure). The VPS has a power configuration and can't increase if sudden server load appears for a short time.

[Martok]

I have the latest version of Arrowchat, v 1.5.61.

With a VPS you will have more resources allocated to your hosting than you would with shared hosting, otherwise no-one would bother to pay more money for less resources.

[alexmbn]

not true.
Ressource allocation is not the same, even if you have more cpu, more bandwith, more everything, the advantage of VPS is to be able to install everything you need for a full service website.
In a shared, less programs, less bandwith, less CPU at first glance, but some flexibility if cpu overloads sometimes because they can allocate global cpu where your site is installed on.

Any ways, this doesn't answer my concern, a pitty. At least if Arrowchat team had a look instead of saying "look at your config". This would have saved a lot of time for all.

Cheers

[MFonti]

I run a dedicated server (medium seize) + phpfox + arrowchat (1 sec poll time) and I can handle more than 2'500 simultaneous users chatting and having fun, but I had to fine tune my.cnf, php.ini and compress+cache every script.

So Jason is not wrong in my humble opinion, but it's not yor fault either, try to minify everything (css+js), try install Nginx and polish your core files code.

Which script are you running on your site? Is it it fully optimized and updated?

[Jason]

I run a dedicated server (medium seize) + phpfox + arrowchat (1 sec poll time) and I can handle more than 2'500 simultaneous users chatting and having fun, but I had to fine tune my.cnf, php.ini and compress+cache every script.

So Jason is not wrong in my humble opinion, but it's not yor fault either, try to minify everything (css+js), try install Nginx and polish your core files code.

Which script are you running on your site? Is it it fully optimized and updated?

Out of curiosity and to potentially help someone else, what php.ini settings did you change that seemed to help?

[MFonti]

Of course!

Openbase_dir enabled
Gzip compression enabled
Memcache enabled
Apache keepalive off
Apache start servers < 10
Safe_mode off
Apache mod_deflate on

Plus the tips written here (some of them are a bit old) ==> http://phplens.com/phpeverywhere/tuning-apache-php

Basically all you need to do is caching internal scripts that are over 500 characters length, help reduce html content size when there is too much javascript.
Caching final output scripts into one file for faster retreive in later requests.

:D

[CAlchin]

Well I think the last update was killing my VPS account. As soon as I disabled arrowchat everything got quicker and the server load was back to how it was before the update.

I'm using the push server as well (which should of relieved the load?). Hostgator are monitoring my container with the intent to get back to me with reports in a few days time. So for now it's no arrowchat for my site until it's determined what the exact problem is/was, but for speculation it's not looking good... :(

I'll post back the findings.

[Jason]

MFonti: Thank you for posting. Much appreciated!

CAlchin: Most likely HostGator won't tell you anything useful. It's still unclear what setting(s) HostGator is using that is causing a slowdown.

[CAlchin]

How about an arrowchat hosting service; whereby maybe a couple of configuration files on the host to build the bridge between database and the hosting service. Small monthly fee similar to the push service to get shot of 99% of the load. :)

[MFonti]

Arrowchat cloud, something similar to Cloudflare :)