Post technical questions and issues here. Yes, the layout is slightly different!

What are you searching on?

 

So which user did you ban?


Things were a little rough about 3 hours ago, a couple of bad gateway messages, but it seems smooth and fast right now.

 

Yeah, there was a server issue which was resolved. Things have been better since then.

 

We're still talking to the support people at Vultr to find out A) exactly what they fixed and B) how long it had been a problem.

 

ES is up, might be missing some more recent results since the indexes haven't been updated in a while.

 

I noticed that it was up - fewer errors in the server log. (Just the 'shard failure' message that we normally get).

 

I don't want to jinx anything, but this forum seems to be working exceptionally well today!

ETA: like this message just ...worked! Straight away!

And I can read pages of threads! Even reply to posts within them! This is awesome guys, well done.

ETA: And 'watched threads' loads in under 2 seconds!

*tries 'new threads'*

Practically instant!

More of this sort of thing!

 

Ah, heaven! Working like a gem.

 

So I've noticed something new. For as long as I can remember on these boards, right now (a few minutes after midnight PT) the boards would be down every single night. I'm not referring to the general slowness and errors we've had at all hours for the past few weeks, but only to the specific nightly downtime right at midnight PT that has been going on for years (and has been frequently reported in this thread by people unaware that it's a normal, expected thing). This has been explained in the past as the nightly backup process needing to lock the database or something like that. Now tonight, as well as last night, I'm able to access the boards just fine. Did something change with the backups to where it's no longer necessary to shut down the boards while it runs?

 

So, here's the deal. On the RS forums we use a backup method called "MySQLHotCopy" because RS uses a MySQL setup where each database table is in a separate file and those files have to be locked in order to be copied. This takes down the system while the tables are locked. On the JC boards we have to use a different method (Percona Xtrabackup) because the database is in InnoDB format (all tables are in one HUGE file). Xtrabackup does NOT require that all tables be locked during the copy, so the boards should in theory stay up. However, when we first moved to Xenforo and set this up I'd never used InnoDB before and noticed that after a backup a bunch of files were missing (there are still some indexes and other things stored in single files outside of the huge InnoDB file). Therefore in order to make sure we got a good backup I had it set up to do a MySQLHotCopy as well as the Xtrabackup just to be safe.

Now, during all of the recent unpleasantness I took a full copy of the Boards database for myself so that I could work on troubleshooting the slowness on my own PC without impacting you guys any more than you already were. In the course of my work I discovered that Xtrabackup does indeed back up everything it needs to do a full restore and I simply did not know how to "prepare" the backup correctly to be used. I tested this a bunch of times and sure enough, I could take a backup with Xtrabackup, prepare it for use and turn it back into a working database copy again. This means the MySQLHotCopy on the JC boards is no longer needed, ergo no table locking and no downtime.

TLDR;I learned something new, and the backups now work without locking.

The End.

 

I guess this was a question for me? Sorry that I didn't see it last night.

I'm searching story names, but AO3 hijackes the search result page, even when I type "jedi council" or similar. At this point, I wonder if Google is acting funny in this particular area of the world, since I do some SEO as a part of my job and I have never encountered anything like that before.

 

When you search, follow your search terms with 'site:boards.theforce.net' (no quotes, no space before or after the colon). That should restrict your search to these boards.

 

Cool! Thanks for figuring it out, and for all you've done over the past couple of weeks!

 

Somebody noticed a problem there. I'm just reporting it.

 

When I click on Lit prefix to filter the threads in the Literature forum, the result is only 4 threads. A quick glance of page 1 shows that there are definitely more than four threads with that prefix. Thanks!

 

Yeah, we've been discussing that over in the Forum Upgrade thread - there's a problem with prefixes at the moment.

 

Yep, I just noticed that. Glad you guys are on top of it!

 

how do I access the Senate forum ? every time I click on the icon it just says "no new threads to display"

 

Yeah - tags are a problem right now. I know the US Politics thread has stayed pretty much on page 1 of Community. If you're after different threads than that one you'll have to scroll down through the other threads to find the one you want. You could try the Search Forum function up top - I haven't played with it yet so if it helps let us know here.