4

A coworker of mine and I are discussing the use (or not) of a full text search solution, namely Elasticsearch in our web application. I am the developer of the web application, and he manages the production server, which is a single Intel NUC running Ubuntu 14.04 , dedicated to running a Rails app (with PostgreSQL) for a small department at my university.

We're currently in disagreement about the choice of software for text search; he says that I could accomplish the task with PostgreSQL's built-in text searching capabilities. However, the additional features of Elasticsearch which are a core part of the software would significantly reduce the complexity of search operations in the web app (as compared to PostgreSQL's text search features), as well as the code needed to develop such features.

I am of the mind that installing Elasticsearch on our production server is not that big of a deal. He is of the mind that the overhead of installing, configuring, managing, upgrading and monitoring is more than that of the code that I would have to write to have the same functionality in PostgreSQL directly.

We're at an impasse. Using Elasticsearch over PostgreSQL's text search would save me quite a bit of development time, at the cost of his needing to install one piece of software on the server. I should note here that the Elasticsearch configuration would be vanilla. He didn't mention security as an issue, so let me just ask:

What workflow is required for a server administrator to install (and maintain) a piece of software? From what I can guess:

  1. Write a small bash script (~5 lines) to install Elasticsearch X.X.X (and be re-usable/automatable)
  2. Incorporate Elasticsearch into start-up services script (re-usable/automatable)
  3. Periodically watch for software updates/security vulnerabilities (are there automated tools for this?)
  4. Update installation if necessary; update script made in step 1

Edit: Elasticsearch can be installed/updated/removed via apt. That obviously simplifies some of the problem. But there are maintenance concerns, which I believe is what my coworker is most worried about.

I really don't think that's terribly complicated; yet, I'm not a system administrator, so I can't know for sure. I just know that the installation on my own Ubuntu machine took less than 3 minutes, and required zero extra configuration to work with our existing Rails app setup.

I would like to know if his not installing Elasticsearch is worth the development time it will take me to implement the same functionality from scratch without a software dependency on the server, and I think that the answer to the question in bold will shed some light on that.

Chris Cirefice
  • 353
  • 1
  • 9
  • PS - Feel free to re-tag... I'm not very familiar with available tags on this site. – Chris Cirefice Jun 09 '16 at 02:57
  • IMO, adding something like ElasticSearch in *production* is a *much* bigger investment than you're making it out to be. Yes, it's fairly simple to tinker with on dev under minimal load for a few hours/days. In production, it really should be on its own server (or better yet, a high-availability cluster), with backups, health monitoring, capacity monitoring etc. Add in things like learning the quirks that reveal themselves six months into using it and it has the potential to be a lot more trouble than it's worth, particularly if you already have something that can do the job decently well. – ceejayoz Jun 09 '16 at 03:06
  • @ceejayoz I get your point - however, I mentioned that we're a small department at a university. Elasticsearch is working great on my own development installation, which I stress-tested for the particular uses that my web app has, and the expected data for 10 years from now based on past data trends (which are linear). In essence, my cloud-managed Ubuntu virtual development environment had zero issues with the installation and usage of Elasticsearch for my use case. I can't imagine that a dedicated server would have more issues. But maybe I'm being naive. – Chris Cirefice Jun 09 '16 at 03:11
  • As a small department there's even less resource than usual for "we've run into a bug/issue in this tech I don't really understand or commonly use". I'd also be concerned as a sysadmin with running multiple memory-heavy things on a small sub-desktop PC like the Intel NUC you describe. – ceejayoz Jun 09 '16 at 03:16
  • @ceejayoz I understand the concern; with one guy managing our tech stack, it's a single point of failure. As for the Intel NUC, it has more hardware than my development env, and ***only*** hosts the web app, which is used by at most ~2500 people at a time (literally every faculty/student in our department). The things they do are trivial CRUD operations, but searching is very important. I believe that the metrics I took for 10 years down the road that held in development will definitely hold in production on a more powerful machine. Then again, there's always Amazon EC2 hosting. – Chris Cirefice Jun 09 '16 at 03:24
  • 1
    There's no reason that maintaining ElasticSearch should be any harder than maintaining PostgreSQL. – Moshe Katz Jun 09 '16 at 17:48
  • @MosheKatz It's not harder, but if they're already maintaining Postgres, it's *additional* work. – ceejayoz Jun 13 '16 at 20:40

1 Answers1

2

It looks like you can add ElasticSearch as a repository on Ubuntu and manage it with apt, which means it would be updated the same way as all the other software on the Ubuntu server.

The steps are outlined here: https://www.elastic.co/guide/en/elasticsearch/reference/current/setup-repositories.html

  1. Add the eleasticsearch public signing key
  2. Add the elasticsearch repo
  3. Update apt
  4. Install or Update elasticsearch using apt like you would anything else on Ubuntu.

Once elasticsearch is available in apt like this, then the security updates to elasticsearch should be handled when security updates are applied through apt like normal. I don't have personal experience with elasticsearch, but I'm not sure what maintenance issues there would be once it is able to be updated by apt.

I do find it a bit odd that the elasticsearch document doesn't include a bash script right there in addition to the info.

jgstew
  • 86
  • 9
  • Yep! I'm aware of this, though I could have mentioned it in my question. However, there are other factors that I don't know about and would like to: e.g., maintenance, monitoring, security issues, etc. Installing/updating is easy because of `apt`. I have a feeling that my coworker is more concerned about the *maintenance* aspect of it, which I am unfamiliar with. I edited my question to reflect this. – Chris Cirefice Jun 09 '16 at 03:13