main
side
curve
  1. Welcome to the new boards! Details here!

Elasticsearch Has Been Disabled

Discussion in 'Communications' started by Ramza, Dec 6, 2015.

  1. Ramza

    Ramza Administrator Emeritus star 8 VIP - Former Mod/RSA VIP

    Registered:
    Jul 13, 2008
    As you may be aware, in the past couple of weeks we've been experiencing periodic and uncharacteristic amounts of slowdown. After discussion with Grahame and Philip, it's become apparent that at least some of this slowdown is due to the large amount of resources being dedicated to Elasticsearch, the background process that enables our handy on-site search function. Consequently, with an eye towards approving general board stability, we have disabled Elasticsearch, and are exploring alternative search methods that could be viable on the boards (E.g. integrated Google search, the default Xenforo search feature, etc.).

    In practical terms for you, the users, you should expect search to not function properly while we consider alternatives, as Elasticsearch uses a dedicated index that posts made following the move have been sorted under. While we realize this isn't an ideal status quo, we're hoping that there's an improved main board usage experience as the tradeoff, and we hope you understand that this came down to a difficult calculus between having accessible boards and having a functioning search, and thus we chose the former.

    While we're in this period of disabled Elasticsearch, please keep us abreast of errors that may be occurring as a result of that change in this thread. Also, our boards are fairly well-indexed by Google, so using their search engine combined with the tag "site:boards.theforce.net" should be a relatively useful substitute. For example, to search for uses of the phrase "Yarael Poof" on our boards, you can enter
    Code:
    Yarael Poof site:boards.theforce.net
    into Google's search bar. We will also strive to keep you posted on updates as we become aware of them, but keep in mind that we're coordinating this alongside the expected hit from the release of The Force Awakens, so we're not necessarily going to see fast developments on this.

    We apologize for the inconvenience this will obviously result in. If you have more questions, feel free to post them in this thread.
     
    Sudooku, pronker, BigAl6ft6 and 3 others like this.
  2. Boba_Fett_2001

    Boba_Fett_2001 Chosen One star 8

    Registered:
    Dec 11, 2000
    So this affects post history right? Because if I look at mine I see one post I made today but then the next one is from 2012.
     
  3. Darkslayer

    Darkslayer Force Ghost star 6

    Registered:
    Mar 26, 2013
    I noticed the slow down the other day, I was really confused seeing as we were still 2 weeks out from Episode VII. Thanks for the update Ramza
     
  4. Ramza

    Ramza Administrator Emeritus star 8 VIP - Former Mod/RSA VIP

    Registered:
    Jul 13, 2008
    Correct: your post history prior to the move was indexed during the move process, and thus still shows up as it's not part of Elasticsearch functionality, whereas posts made while Elasticsearch was enabled were indexed by Elasticsearch, and thus no longer show up. Getting those posts to show up in your history is something we're also investigating, but for now we're just looking to see if we get the stability improvements we want.
     
    skypadme94 and Boba_Fett_2001 like this.
  5. GrandAdmiralJello

    GrandAdmiralJello Comms Admin ❉ Moderator Communitatis Litterarumque star 10 Staff Member Administrator

    Registered:
    Nov 28, 2000
    Cool. Are we still going down next week, or is possible that disabling indexing might help us stay above water? :p
     
    jcgoble3 likes this.
  6. Ramza

    Ramza Administrator Emeritus star 8 VIP - Former Mod/RSA VIP

    Registered:
    Jul 13, 2008
    Not sure, but it'd be a nice plus. :p We do actually have a working theory now about the crashes that was born out of observations made following turning Elasticsearch off, so who knows?
     
    jcgoble3 likes this.
  7. Admiral Volshe

    Admiral Volshe Force Ghost star 10

    Registered:
    Sep 2, 2012
    Weird observation... mine still show up. In all tabs (postings, recent activity). Is that not supposed to happen? Do we have zombie elasticsearch? :p Or is it just something that *may* not work?
     
    Darkslayer likes this.
  8. Dandelo

    Dandelo Force Ghost star 7

    Registered:
    Aug 25, 2014
    shhh don't tell 'em Volshe :p
     
    Darkslayer and Admiral Volshe like this.
  9. jcgoble3

    jcgoble3 Jedi Grand Master star 5

    Registered:
    Nov 7, 2010
    No, your recent posts, like everyone's, are cut off at the time Elasticsearch was shut off. Only your posts back to around the wee hours of yesterday morning show up; those posts have been indexed by the regular XenForo functions. Everything between the move and early yesterday morning, though, was indexed by Elasticsearch and only Elasticsearch, so it is currently missing. For those of us who have been around since before the move, like me, our pre-move posts do also show up because they were indexed by XenForo during the move process presumably before Elasticsearch was set up.
     
    Admiral Volshe likes this.
  10. Cryogenic

    Cryogenic Force Ghost star 5

    Registered:
    Jul 20, 2005
    This is a real bummer.

    I know you guys are probably trying your best, but it feels like this board is leaking and gradually falling apart.

    The truncated posts have never been fixed (it'll be four years next March).

    And now, there's this.

    Oh, well. Maybe just another bump in the road, on the way to something more special.
     
  11. Ender Sai

    Ender Sai Chosen One star 10

    Registered:
    Feb 18, 2001
    [​IMG]

    THIS IS AN OUTRAGE!
     
    Darkslayer likes this.
  12. Sith-I-5

    Sith-I-5 Force Ghost star 6

    Registered:
    Aug 14, 2002
    I didn't even know that we even had a functioning search! Doesn't seem that long since I was being told that Search wasn't working, so I had been using Google to look for old stuff on here.
     
  13. Admiral Volshe

    Admiral Volshe Force Ghost star 10

    Registered:
    Sep 2, 2012

    That's what I was wondering - thanks. I didn't realise there was a separate index that the XenForo was capable of updating itself. :)
     
  14. PatttyB0123

    PatttyB0123 RSA Latin America star 6 Staff Member Manager

    Registered:
    Apr 2, 2003
    Thanks for the information about the issue.
    I did not have idea how that things work.
     
  15. jcgoble3

    jcgoble3 Jedi Grand Master star 5

    Registered:
    Nov 7, 2010
    I notice that the number of online users is (currently, at least) back down to more normal levels similar to where it typically was prior to the TFA rush. If this persists, then with less load on the server, might it be possible to try turning Elasticsearch back on and see if the server can handle it now? Google is simply not an alternative for the types of searches I perform; I need, at the very least, the ability to filter by user, date, and/or thread and the ability to treat posts individually rather than just scanning the whole webpage, and I've been sorely missing these features.
     
    Gamiel and Darth Downunder like this.
  16. jcgoble3

    jcgoble3 Jedi Grand Master star 5

    Registered:
    Nov 7, 2010
    Hello? Ramza? LAJ_FETT? Anyone?
     
    Tim Battershell likes this.
  17. JoinTheSchwarz

    JoinTheSchwarz JC Head Admin & Community Manager star 9 Staff Member Administrator

    Registered:
    Nov 21, 2002
    Chill out. :p There are some logistical factors involved that we are not ready to announce at the moment.
     
    jcgoble3 likes this.
  18. jcgoble3

    jcgoble3 Jedi Grand Master star 5

    Registered:
    Nov 7, 2010
    OK, thanks. As long as I know I'm not being totally ignored. :p :)
     
    JoinTheSchwarz likes this.
  19. Boba_Fett_2001

    Boba_Fett_2001 Chosen One star 8

    Registered:
    Dec 11, 2000
    Any updates on this?
     
    TK327 and Chewies_bandolier like this.
  20. JoinTheSchwarz

    JoinTheSchwarz JC Head Admin & Community Manager star 9 Staff Member Administrator

    Registered:
    Nov 21, 2002
    Right now we are focused on updating the forum software, so any changes to the search system will have to happen after that.
     
  21. Chewies_bandolier

    Chewies_bandolier Force Ghost star 4

    Registered:
    May 5, 2002
    just pointing out that with some form of function (even limited to the most popular boards), the quality of the discussions could be much better since we're currently just seeing the same things asked ad infinitum, in some cases and it is a major pain in the ass to go back through all the posts and find the ones manually, that have the answer..
     
    Scapro Tyler likes this.
  22. thejeditraitor

    thejeditraitor Chosen One star 6

    Registered:
    Aug 19, 2003
    any info on getting a new search engine or re-implementing the old one since traffic has gone down?
     
    TK327 likes this.
  23. Scapro Tyler

    Scapro Tyler Jedi Knight star 3

    Registered:
    Oct 17, 2015
    I asked the same question in the technical question thread and was told it won't be until a forum upgrade is done.

    I don't know anything about the logistics/etc however I do see an awful lot of threads being deleted with the mods saying "Not appropriate a thread already exists"

    As a new comer that is really hard to do without a search function.
     
    jcgoble3 and thejeditraitor like this.
  24. thejeditraitor

    thejeditraitor Chosen One star 6

    Registered:
    Aug 19, 2003
    yeah it worked ok but wreaked havoc with the site apparently.
     
  25. anakinfansince1983

    anakinfansince1983 Nightsister of Four Realms star 10 Staff Member Manager

    Registered:
    Mar 4, 2011
    I literally just locked a thread and redirected a user to an existing thread, and as such I'll say this: it's not a punishment, it's not meant to be embarrassing, and we get that you can't search for an already-existing thread.

    The modus operandi in locking duplicate threads is a little different now than it was when a quick use of the search function would turn up a thread on the topic--unless, of course, a user is trying to start a thread on "Adam Driver's hair" when there is a thread called "What do you think of Adam Driver's hair?" five threads below it on the first page.

    If you have done as much as reasonably expected (i.e. at least look on the first page or two) and don't find your existing topic, it's possible that we just know of an existing one from memory.