You'll fight, and you'll shag, and you'll hate each other till it makes you quiver, but you'll never be friends.

Spike ,'Sleeper'


Buffistas Building a Better Board  

Do you have problems, concerns or recommendations about the technical side of the Phoenix? Air them here. Compliments also welcome.

To-do list


Monique - Aug 01, 2003 9:36:52 am PDT #4438 of 10000

Yeah, do we know what was the up?

We're looking into what the deal was with the suspended page right now. Kristen or I will provide a report once we've learned more. I apologize for the difficulties you guys may have experienced today, and I thank you for not hunting one or both of us down. ... Unless you've already sent out the lynching parties, in which case, I should probably be hunting for a hidey-space?


Kristen - Aug 01, 2003 10:07:23 am PDT #4439 of 10000

As I'm sure you noticed, we had a bit of an outage this morning. Some of you may have even seen a suspended page. The short version is that y'all talk too much. The long version is...well, a little longer.

This morning, Monique and I received an email from IH in which they informed us that this site was starting to outgrow a shared hosting environment and that it was bogging down the server. There's a possibility that on the newer, faster server, it won't be such an issue but we were urged to consider dedicated hosting.

As I'm reading the email, the site goes down.

I respond, a little pissed and very perplexed, given the number of conversations we had with them prior to moving in. I ask for some clarification.

IH informs me that, while they did not anticipate any problems with the site, they simply weren't expecting "over 200 concurrent mysql connections at the same time". They had no way of predicting that. They informed us that we are the reason the server has been so slow of late. They also suggested that we modify our code, as they believe the script we're using is the problem.

My response to this is that, while I will discuss it with you [Buffistas], I'm a little confused as to why I'm just hearing about this now. Both Monique and I have had conversations with them in the past few weeks about the slowness and other issues and, at no time, did they tell us it was due to our usage. Yadda yadda yadda. More stuff that really is irrelevant to the issue at hand.

IH mentions that they think if we "optimize the database queries" that will help a lot. It isn't our bandwidth usage that is the issue, it's our resource usage.

So here's the issue. As I have already recommended to ita, we should try and optimize the queries, if possible. Finding other ways to use fewer resources would also be good. The new server may also buy us some time.

But the bottom line is that if we are truly using "200 concurrent mysql connections at the same time" on a Friday morning in August, I don't think there's a shared hosting environment in all the world that's going to be able to host us for very long come the fall. I really am starting to believe that this board needs a dedicated server to exist peacefully on the net.


Jon B. - Aug 01, 2003 10:14:18 am PDT #4440 of 10000
A turkey in every toilet -- only in America!

"200 concurrent mysql connections at the same time"

I find it hard to believe that there are 200 users on at the same time, but maybe I underestimate the number of lurkers. Also, I don't know anything about what constitutes a mysql connection. Is the code opening multiple connections for one user before closing any of them? Can anyone explain the concept in English?


Jessica - Aug 01, 2003 10:19:01 am PDT #4441 of 10000
And then Ortus came and said "It's Ortin' time" and they all Orted off into the sunset

I would guess that every pageview constitutes at least one MySQL connection, but ita would know for sure.


Typo Boy - Aug 01, 2003 10:20:21 am PDT #4442 of 10000
Calli: My people have a saying. A man who trusts can never be betrayed, only mistaken.Avon: Life expectancy among your people must be extremely short.

We have around 1000 registered users total. twenty percent of our users on line simulataneiously would be a lot.

And yeah we should be able not only limit it to one connection per user logged on, but actually open (say) fifty connection permanently and then share them among users queuing requests.


Kristen - Aug 01, 2003 10:20:25 am PDT #4443 of 10000

No, it's not users. I had a definition around somewhere, I think, from before we moved in. I'm not sure if it was at WX or PF.

Is the code opening multiple connections for one user before closing any of them?

It sounds like it is, to me. Then again, I've never looked at the code and probably wouldn't be able to tell even if I had.


Jon B. - Aug 01, 2003 10:22:14 am PDT #4444 of 10000
A turkey in every toilet -- only in America!

I would guess that every pageview constitutes at least one MySQL connection

That's what I thought. But once the page is downloaded, the connection should be closed. I thought that "200 concurrent mysql connections at the same time" would mean that 200 requests for pages are being made at the same time. Which I find hard to believe.


Typo Boy - Aug 01, 2003 10:22:55 am PDT #4445 of 10000
Calli: My people have a saying. A man who trusts can never be betrayed, only mistaken.Avon: Life expectancy among your people must be extremely short.

I understand we probably have multiple queries per page view. But there is no reason they can't be done sequentially using the same connection. That would limit it to one connection per user.

And as I said we could actually have queries use a static pool of connections - using less than one connection per user. BEfore getting that drastic it ought to be comparatively easy to limit us to one connection per user.


Tom Scola - Aug 01, 2003 10:24:10 am PDT #4446 of 10000
Remember that the frontier of the Rebellion is everywhere. And even the smallest act of insurrection pushes our lines forward.

It's possible that there is a bug in the code where MySQL connections aren't properly closed in all circumstances, causing them to accumulate.

This would be consistent with the pattern we've been seeing, where the server gets slower and slower, and finally crashes, but when it comes back up again it is fast again.


Jon B. - Aug 01, 2003 10:24:12 am PDT #4447 of 10000
A turkey in every toilet -- only in America!

No, it's not users.

I understand that Kristen. But I wouldn't expect more than one connection per user at any point in time. Unless I misunderstand what a connection is....

t edit Or what Tom said. I hope he's right cause.... easy fix.