|
|
#1
February 21, 2010, 03:30:22 PM
|
||||
|
||||
Cause of 503s Possibly Established
I have found the pattern: 503s occur daily at 4:00 PM PDT. In UTC (the time the server uses), this is 12:00 midnight. Because servers often have cron jobs that run at this time, I started moving vBulletin cron jobs like cleanup and birthday notices to 9:00 AM UTC (about the quietest part of the day). A week or so ago, FreezeWarp recommended I run all of them one by one to find out whether any of them caused a forum blackout. None of them caused outages and the errors continued promptly at midnight. Just today, I thought about server jobs and why only the forum goes down.
Then it hit me: the forum has a very large database. None of the other sites have one anywhere near its size, thus they only are down for about 10-15 seconds. The daily backups cause the downtime! Hopefully, we'll be able to move these to around 11:00 AM UTC, so they occur after the forum cron jobs and before the activity begins going back up. So, the 503s were actually something good, just not running at the right time. With that, I should point out that the forum will be going down daily for scheduled maintenance at midnight UTC. (I know it's a terrible time. We're working on moving it!) |
|
#2
February 21, 2010, 03:38:06 PM
|
||||
|
||||
|
Good thing I'm not on at that time.
|
|
#3
February 21, 2010, 04:07:43 PM
|
||||
|
||||
|
So what time is UTC in Texas... CST?
|
|
#4
February 21, 2010, 04:09:31 PM
|
||||
|
||||
|
Midnight UTC = 6:00 PM Central. Note that that is only during non-DST. (I HATE DST!)
Last edited by Cat333Pokémon; February 21, 2010 at 04:11:05 PM. Reason: I HATE DST. I put the time wrong first! |
|
#5
February 21, 2010, 04:32:00 PM
|
||||
|
||||
|
Yay no more getting logged off for no reason! I dont know why but the 503 messages make me log off.
|
|
#6
February 21, 2010, 04:42:59 PM
|
||||
|
||||
|
Zorua, are you putting a check in the 'Keep Me Logged In' box?
 Just wondering.And 6 PM sucks for me, because 6 is when I get on VR. O_O Last edited by SK; February 21, 2010 at 04:43:21 PM. |
|
#7
February 21, 2010, 04:49:24 PM
|
||||
|
||||
|
Yes I am. But it still logs me out. I should try clearing cookies.
|
|
#8
February 21, 2010, 04:49:24 PM
|
||||
|
||||
|
Well, I'm at least glad to see that it's not due to us leaking memory again.
|
|
#9
February 21, 2010, 04:51:05 PM
|
||||
|
||||
|
Well I'm to lazy to check what that is my time, but, I'll check later. Good you figured it out!
|
|
#10
February 21, 2010, 04:51:06 PM
|
||||
|
||||
|
Memory leaks were one of the causes, yes.
Yeah, I know that time sucks. That's why I want it moved. If you are getting weird errors with staying logged in, definitely try clearing cookies. |
|
#11
February 21, 2010, 05:25:28 PM
|
||||
|
||||
|
Well, at least we know what's causing 'em. I did notice once that it happened around the same time two nights in a row, but I only thought of it as coincidence.
|
|
#12
February 21, 2010, 06:21:05 PM
|
||||
|
||||
|
Quote:
I understand how annoying logging in to forums is... Hehe. Lux, tell me your time zone and I think I can convert it quick for you. 
Last edited by SK; February 21, 2010 at 06:21:45 PM. |
|
#13
February 22, 2010, 06:53:16 AM
|
||||
|
||||
|
I believe I have no problem with the timing, since I think I'm in school during that time.
|
|
#14
February 22, 2010, 06:57:40 AM
|
||||
|
||||
|
Both of the timings, the old and the new one are fine with me! I would be sleeping during the old one, and at school during the new one!
|
|
#15
February 22, 2010, 11:18:12 AM
|
||||
|
||||
|
I haven't been getting 503s.
|
|
#16
February 22, 2010, 01:21:47 PM
|
||||
|
||||
|
Quote:
Quote:
|
|
#17
February 22, 2010, 01:23:32 PM
|
|||
|
|||
|
Quote:
 ), to save on candle wax because candles were rather expensive. Once the age of digital clocks started showing up, they incorporated DST into their programming.Now, this system has become obsolete thanks to the invention of the light bulb. However, because DST has been hard-coded into most technologies, they can not simply end it, as it would be too costly. *Takes off smarty cap* So basically these 503s are just the natural slowdown that happens when a computer's ram/processor is overloaded, right? It's amazing how simple computer problems truly are... |
|
#18
February 22, 2010, 02:23:09 PM
|
||||
|
||||
|
Some of them seem to be due to the memory overload, but look at what Cat said. The server probably shuts off access to the database during the backup to prevent data from changing during the backup.
|
|
#19
February 22, 2010, 05:04:32 PM
|
|||
|
|||
|
No, servers don't normally do that. I remember that time Will, Alternate, and I were posting during the pre-move backup back in phpBB. It doesn't close.
Also, the memory thing is because the con jobs or whatever he called them must take up a lot of RAM based on what operation they do. |
|
#20
February 22, 2010, 07:38:29 PM
|
||||
|
||||
|
I haven't seen a 503 yet. I think it closes down the server to prevent memory leaks. It's irritating to the users but activates as a protection mechanism.
|
|
#21
February 22, 2010, 08:24:05 PM
|
||||
|
||||
|
What I meant before was that ispCP Omega may be shutting off database and/or site access during its backups, not vBulletin.
|
|
#22
February 22, 2010, 08:46:37 PM
|
||||
|
||||
|
Isn't the vb on the database tho?
|
|
#23
February 22, 2010, 09:09:39 PM
|
||||
|
||||
|
This reminds me of how every Chatzy server always has to crash for 3 minutes at 12:32 PST. Well, at least we found the reason why it kept doing that.
|
|
#24
February 23, 2010, 08:08:31 AM
|
||||
|
||||
|
By the way, if you didn't already know Cat, I experienced a "Database error" yesterday at night, around 9:30 PM. Does it have to do with those backups?
|
|
#25
February 23, 2010, 04:23:08 PM
|
||||
|
||||
|
We're moving servers soon. Don't bother mentioning the last hour of database errors.
Fivebean sucks. |
|
#26
February 23, 2010, 04:30:14 PM
|
|||
|
|||
|
I'm guessing you'll put the message on huge letters in the index... And I'll still not notice it.
|
|
#27
February 23, 2010, 06:13:03 PM
|
||||
|
||||
|
Yeah, I don't want a bunch of noobish mods (you remember who you are...) posting during that time, forcing me to lock the whole forum even from viewing.
|
|
