Jump to content
CairnTalk

Slow Server


bradl

Recommended Posts

I appreciate your patience (and persistence) as we deal with a slow server. Whenever you see the "server too busy" message it means our web server is loaded beyond a very high trigger point and the forum has stopped processing requests to attempt to lower the server load.

The server load may or may not have anything to do with the forum itself. We are on a shared server and indeed have other software running our public web site and a members-only site for the CTCA. Neither seem to be adversely affected.

I tried turning the forum offline overnight to see if the load tapered down, but it did not. I am not convinced the load has anything to do with our forum (other than we are suffering for it) but the web host has a differing opinion. Their proposed solution is a private server and the cost is prohibitive -- I believe not even close to feasible. There are additional organizational complications.

Regardless I will continue to work to find and reduce the excessive server load so that we can enjoy our forum again. I have to tell you my stomach sinks every time I see that server too busy message. We have a popular forum but on the scale of these things, we are a niche group and thus not really all that big. There is no excuse for this poor performance, so if we have been secretly hacked, or have a misbehaving script somewhere, I desperately want to restore our previous brisk performance and good "web server citizenship."

CAIRNTALK: Questions? Need help? → Support Forum Please do not use PMs for tech support
CRCTC: Columbia River Cairn Terrier Club 

 

 

Link to comment
Share on other sites

Thanks for all your hard work Brad. I know it's not easy!

"as far as i am concerned cairns are the original spirit from which all terriers spring, and all terriers are cairns very deep down inside." pkcrossley

Link to comment
Share on other sites

Thanks Brad for all your hard work. It is very aggravating (slow server message) but now that I read your post here I won't be annoyed by it anymore. I know I have more patience than that... I am not addicted to Cairn talk, i can...stop...any time... i want...! And you don't need to feel bad, it's not your fault!

(null)

Edited by DemiSheep

Our Cairns: Attila (Sprouted 03/09/11), Tessa (Sprouted 01/14/12)

Link to comment
Share on other sites

It's annoying and frustrating because we need our cairn fix! But we know you do whatever you can Brad to keep us in touch. We don't blame you.

We just have to quick read and post when there's a moment to do so.

Edited by Hillscreek
Link to comment
Share on other sites

Bradl - Who do you use for hosting?

I'm a bit of a web nerd ( by trade and obsession ). Maybe I can be of some assistance. I know all about the dedicated and VPS hoopla they will try to sell you, and I know you most likely won't need it unless your site gets traffic like CuteOverload.com ( in which case there would be scores of ads to support such investments in server hardware )

Link to comment
Share on other sites

That's kind of you. Dreamhost (since 2002).

I suspect I'm legitimately crowding the memory usage per user for shared hosting. I think they allow about 90M per user before they start killing processes.

I have a fairly stout (query-heavy) expressionengine multi-site install driving the CTCA members-only site, plus a lightweight public site. Looking at logs the traffic on the public site is almost all around the breeder-referral geolocation activity. Members barely use their members-only site.

Then there is this forum which is by far the source of the vast majority of traffic. When I look at 'top' the main consumers of memory are php5.cgi (fcgi). DH support shows their procwatch script 'renicing' or killing my processes pretty regularly.

Here's 'top' at the moment



top - 09:40:39 up 66 days, 14:53, 2 users, load average: 63.81, 48.48, 38.17
Tasks: 5 total, 1 running, 4 sleeping, 0 stopped, 0 zombie
Cpu(s): 50.8%us, 7.8%sy, 6.2%ni, 34.4%id, 0.4%wa, 0.0%hi, 0.4%si, 0.0%st
Mem: 16435788k total, 15832488k used, 603300k free, 207836k buffers
Swap: 3998712k total, 576636k used, 3422076k free, 6235152k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
4242 myuser 20 0 125m 19m 7692 S 0 0.1 0:00.87 php5.cgi
5642 myuser 20 0 18976 1164 896 R 0 0.0 0:00.01 top
17825 myuser 20 0 118m 10m 6260 S 0 0.1 0:00.10 php5.cgi
62787 myuser 20 0 233m 19m 4860 S 0 0.1 1:10.85 php5.cgi
63255 myuser 20 0 235m 21m 5608 S 0 0.1 1:01.06 php5.cgi

I don't get where all the load is coming from ... I'm not seeing it there (perhaps my processes are being kept low by procwatch and would be higher otherwise -- I dunno).

My emergency fall-back plan if I get the boot is to attempt to move the forum to a different user (which would mean splitting it off to a different or subdomain) so that at least the memory would be divided between two users rather than all run as the same user. I believe the host allows that.

The 'server too busy' messages were because I set the forum to pause if server loads exceeded a high threshold value, in an attempt to prevent it from falling over. But even when I turned the forum offline, the baseline server load did not go down. Something else is going on.

Last night I set the server limit to be absurdly high so we should rarely see the dreaded pink 'too busy' message now. I'll just let the thing slow down or fall over, but at least it will be open.

I'm happy to let you look at a snapshot of the server log to see if you can spot anything nefarious going on. I have a number of notoriously bad 'bots blocked by htaccess and robots.txt, with google (which indexes the site constantly .. literally it's on at virtually all times) set to a very slow crawl rate. So far as I can tell we're not being hammered by a bad bot.

Any ideas are welcome!

CAIRNTALK: Questions? Need help? → Support Forum Please do not use PMs for tech support
CRCTC: Columbia River Cairn Terrier Club 

 

 

Link to comment
Share on other sites

Off the top of my head, I'm thinking if you switched from using PHP5 CGI to Apache module you might do away with server load issues altogether. The module is known to be more efficient at processing heavy load. There are other things like Xcache or eAccelerator that you can also use that help loading issues. I've ran into a lot of load problems with Wordpress installs and weird server configs.

I've actually never been much of a fan of dreamhost because they were usually the source of all the weird config issues. I've moved a few sites from dh because I just couldn't get stability on their servers. I'm also impatient.

Link to comment
Share on other sites

Heh, I'm lazy (about some things) so inertia keeps me there pretty much. (There are also a lot of things like club governance Mailman discussion lists and whatnot all dependent on DH that would be more work to untangle than it's worth.)

I would love to run mod apache. In the early days we did run under mod apache at DH but at some point it was no more ... cgi/fcgi only on shared servers. Mod apache is available on VPS, which I'm trying to avoid for financial and logistical reasons.

In truth I'm reluctant to move anything anywhere at this time, primarily for organizational reasons. I am planning to retire from web administration for the national club during the coming year and I expect (and pray) my successor will move the sites under new hosting etc. for their own convenience. I know that's one of the first things I did when I took over from the last admin over ten years ago.

CAIRNTALK: Questions? Need help? → Support Forum Please do not use PMs for tech support
CRCTC: Columbia River Cairn Terrier Club 

 

 

Link to comment
Share on other sites

Well then it sounds like you've got a plan and I can't really provide much immediate assistance after all ;) If/when the time comes, I don't mind lending a helping hand. If there's anything I've got experience in, its moving complicated stuff from one terrible environment to another LOL.

The only other option I can think of is using Amazon's web services to offset the load. I use AWS for storage and email needs, and its incredibly affordable. I don't know how other companies manage to sell their products after Amazon sends out a web service for it.

  • Like 1
Link to comment
Share on other sites

You're not leaving us, are you Brad? Surely we'll still hear from you frequently, although not about site stuff, just fun Cairn stuff, right?!?

Jandy and my Cairns, Kirby & Phinney 
Link to comment
Share on other sites

  • 2 weeks later...

I'm afraid to jinx it but the load dropped from 65 this morning (a truly absurd number) to single digits around lunchtime and have been there ever since. Nothing I had anything to do with but hope this lasts!

CAIRNTALK: Questions? Need help? → Support Forum Please do not use PMs for tech support
CRCTC: Columbia River Cairn Terrier Club 

 

 

Link to comment
Share on other sites

  • 2 weeks later...

Hey Brad, I just wanted to comment that the server has been working very fast recently over the past week I would say. I have not seen any errors for a while now...

Our Cairns: Attila (Sprouted 03/09/11), Tessa (Sprouted 01/14/12)

Link to comment
Share on other sites

Join the conversation

You can post now and register after. Your post will display after you confirm registration. If you already have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

CairnTalk.net

  • A meeting place and
    online scrapbook for
    Cairn Terrier fanciers.

ctn-no-text-200.png

Disclaimers

  • All posts are the opinion and
    responsibility of the poster.
  • Post content © the author.
×
×
  • Create New...

Important Information

Site Guidelines | We put cookies on your device to help this website work better for you. You can adjust your cookie settings; otherwise we'll assume you're okay to continue.