How to change Elasticsearch max memory size

Question

I have an Apache server with a default configuration of Elasticsearch and everything works perfectly, except that the default configuration has a max size of 1GB.

I don't have such a large number of documents to store in Elasticsearch, so I want to reduce the memory.

I have seen that I have to change the -Xmx parameter in the Java configuration, but I don't know how.

I have seen I can execute this:

bin/ElasticSearch -Xmx=2G -Xms=2G

But when I have to restart Elasticsearch this will be lost.

Is it possible to change max memory usage when Elasticsearch is installed as a service?

What operating system and version? Elasticsearch version? How did you install Elasticsearch? — James Addison, Aug 09 '13 at 18:05
You might want to lower the number of replicas and shards since there is no upside of having them on a 1 node setup, add this to /etc/elasticsearch/elasticsearch.yaml: `index.number_of_shards: 1 index.number_of_replicas: 0` This way you save on memory and cpu by not doing unneccessary work. — neo112, Feb 26 '16 at 19:49

score 169 · Answer 1 · edited Mar 10 '21 at 09:36

169

In ElasticSearch >= 5 the documentation has changed, which means none of the above answers worked for me.

I tried changing ES_HEAP_SIZE in /etc/default/elasticsearch and in /etc/init.d/elasticsearch, but when I ran ps aux | grep elasticsearch the output still showed:

/usr/bin/java -Xms2g -Xmx2g # aka 2G min and max ram

I had to make these changes in:

/etc/elasticsearch/jvm.options

# Xms represents the initial size of total heap space
# Xmx represents the maximum size of total heap space

-Xms1g 
-Xmx1g 
# the settings shipped with ES 5 were: -Xms2g
# the settings shipped with ES 5 were: -Xmx2g

edited Mar 10 '21 at 09:36

bfontaine

18,169
13
73
107

answered Oct 30 '16 at 20:23

BigRon

3,182
3
22
47

5

And for Elasticsearch 5.2 on Mac OS 10.12 I found this file in `/usr/local/opt/elasticsearch/libexec/config/jvm.options` – victorpolko Mar 27 '17 at 01:30
1

I can confirm this works for aws ami instance (installed with RPM), but first set MAX_LOCKED_MEMORY=unlimited in /etc/sysconfig/elasticsearch and bootstrap.memory_lock: true in /etc/elasticsearch/elasticsearch.yml – mork May 04 '17 at 16:49
4

This does not work in Windows - the correct answer is here: https://stackoverflow.com/questions/28798845/how-to-set-memory-limit-to-elasticsearch-in-windows – cbp Aug 25 '17 at 08:23
5

For homebrew installs, file is in here: `/usr/local/etc/elasticsearch/jvm.options` – Milovan Zogovic Nov 09 '17 at 09:22
This works for Rocky 8 (and the entire EL8 family) – Simon Avery Jan 20 '22 at 12:54

score 117 · Answer 2 · edited May 23 '17 at 12:03

117

Updated on Nov 24, 2016: Elasticsearch 5 apparently has changed the way to configure the JVM. See this answer here. The answer below still applies to versions < 5.

tirdadc, thank you for pointing this out in your comment below.

I have a pastebin page that I share with others when wondering about memory and ES. It's worked OK for me: http://pastebin.com/mNUGQCLY. I'll paste the contents here as well:

References:

https://github.com/grigorescu/Brownian/wiki/ElasticSearch-Configuration http://www.elasticsearch.org/guide/reference/setup/installation/

Edit the following files to modify memory and file number limits. These instructions assume Ubuntu 10.04, may work on later versions and other distributions/OSes. (Edit: This works for Ubuntu 14.04 as well.)

/etc/security/limits.conf:

elasticsearch - nofile 65535
elasticsearch - memlock unlimited

/etc/default/elasticsearch (on CentOS/RH: /etc/sysconfig/elasticsearch ):

ES_HEAP_SIZE=512m
MAX_OPEN_FILES=65535
MAX_LOCKED_MEMORY=unlimited

/etc/elasticsearch/elasticsearch.yml:

bootstrap.mlockall: true

edited May 23 '17 at 12:03

Community

1
1

answered Aug 09 '13 at 18:07

James Addison

3,086
1
17
16

4

This looks good. I also have the following advice about determining what ES_HEAP_SIZE should be: The rule of thumb is that the ElasticSearch heap should have around 50% of the available memory on the machine. ElasticSearch uses system caches heavily, so you should leave enough memory for them. (from http://asquera.de/opensource/2012/11/25/elasticsearch-pre-flight-checklist/ ) – Tom Nov 11 '13 at 13:52
17

In CentOS, I see the configs that you mentioned under `/etc/default` sits under `/etc/sysconfig' – Nishant Aug 05 '14 at 08:26
5

On CentOS `/etc/sysconfig/elasticsearch` is the appropriate place to make these changes. The RPM even provides a default version of this file there too. – slm Sep 08 '15 at 20:19
5

This is no longer applicable for Elasticsearch 5, you need this [answer](http://stackoverflow.com/a/40333263/3102766). – tirdadc Nov 22 '16 at 18:58
For a Windows Service, also be aware of registry settings: https://stackoverflow.com/questions/27128620/elasticsearch-reports-default-heap-memory-size-after-setting-environment-variabl – JohnLBevan Jun 30 '17 at 21:47
2

This looks good, but I put my limits in specifically for elasticsearch user in `/etc/security/limits.d/elasticsearch.conf` – radtek Mar 19 '18 at 21:57

score 33 · Answer 3 · answered May 29 '21 at 11:47

33

Create a new file with the extension .options inside /etc/elasticsearch/jvm.options.d and put the options there. For example:

sudo nano /etc/elasticsearch/jvm.options.d/custom.options

and put the content there:

# JVM Heap Size - see /etc/elasticsearch/jvm.options
-Xms2g
-Xmx2g

It will set the maximum heap size to 2GB. Don't forget to restart elasticsearch:

sudo systemctl restart elasticsearch

Now you can check the logs:

sudo cat /var/log/elasticsearch/elasticsearch.log | grep "heap size"

You'll see something like so:

… heap size [2gb], compressed ordinary object pointers [true]

Doc

answered May 29 '21 at 11:47

Farahmand

2,731
1
24
27

3

kudos for mentioning the ".options" filename, this was not in documentation and thus working for me, now it is. Thanks ! – Radek Aug 09 '21 at 16:47
1

Thanks. This worked on Rocky 8, where the previous Centos 7 instructions did not. (Even though the sysconfig file existed) – Simon Avery Jan 20 '22 at 12:51
1

Thanks for that. This approach is only recommended by ES. – VisioN Jun 21 '22 at 11:30

score 28 · Answer 4 · 2015-09-04T00:40:50.727

28

For anyone looking to do this on Centos 7 or with another system running SystemD, you change it in

/etc/sysconfig/elasticsearch

Uncomment the ES_HEAP_SIZE line, and set a value, eg:

# Heap Size (defaults to 256m min, 1g max)
ES_HEAP_SIZE=16g

(Ignore the comment about 1g max - that's the default)

edited Sep 04 '15 at 00:40

answered May 12 '15 at 06:54

I should add: I tried the approach above on ES 1.7 on CentOS 7, and it had no effect – Jonesome Reinstate Monica Nov 24 '15 at 20:27
1

@Jonesome not sure what I can suggest - I am running ES 1.7.1 from the ES RPM Package on CentOS 7 and that's the path that this file exists in, and changing ES_HEAP_SIZE works as intended. – Nov 25 '15 at 04:10
Note for users of EL8 (Rhel, Centos, Rocky, etc). Although sysconfig does exist as per this, changing it does not make the change. The change needs to be made in /etc/elasticsearch/jvm.options as per @farahmand's answer instead. Verification is made by: curl -sS "localhost:9200/_cat/nodes?h=heap*&v" – Simon Avery Jan 20 '22 at 12:53

score 15 · Answer 5 · answered Sep 17 '16 at 12:24

15

Instructions for ubuntu 14.04:

sudo vim /etc/init.d/elasticsearch

Set

ES_HEAP_SIZE=512m

then in:

sudo vim /etc/elasticsearch/elasticsearch.yml

Set:

bootstrap.memory_lock: true

There are comments in the files for more info

answered Sep 17 '16 at 12:24

tread

10,133
17
95
170

score 11 · Answer 6 · edited Jan 06 '18 at 00:55

11

Previous answers were insufficient in my case, probably because I'm on Debian 8, while they were referred to some previous distribution.

On Debian 8 modify the service script normally place in /usr/lib/systemd/system/elasticsearch.service, and add Environment=ES_HEAP_SIZE=8G just below the other "Environment=*" lines.

Now reload the service script with systemctl daemon-reload and restart the service. The job should be done!

edited Jan 06 '18 at 00:55

Danilo Cândido

408
1
5
18

answered Aug 23 '15 at 23:20

claudod

815
8
8

1

thanks, you save my day! yes, this is the correct procedure for Debian 8 – faster2b Apr 25 '16 at 10:46
Is there no other place to configure this but to change a file in `/usr/`? – Martin Schröder Jan 31 '17 at 08:42
Works like a charm on ES 1.7.6 / Ubuntu 16 – Quentin Jul 09 '21 at 04:01

score 6 · Answer 7 · answered Aug 13 '13 at 12:36

6

If you use the service wrapper provided in Elasticsearch's Github repository, found at https://github.com/elasticsearch/elasticsearch-servicewrapper, then the conf file at elasticsearch-servicewrapper / service / elasticsearch.conf controls memory settings. At the top of elasticsearch.conf is a parameter:

set.default.ES_HEAP_SIZE=1024

Just reduce this parameter, say to "set.default.ES_HEAP_SIZE=512", to reduce Elasticsearch's allotted memory.

Note that if you use the elasticsearch-wrapper, the ES_HEAP_SIZE provided in elasticsearch.conf OVERRIDES ALL OTHER SETTINGS. This took me a bit to figure out, since from the documentation, it seemed that heap memory could be set from elasticsearch.yml.

If your service wrapper settings are set somewhere else, such as at /etc/default/elasticsearch as in James's example, then set the ES_HEAP_SIZE there.

answered Aug 13 '13 at 12:36

Scott Rice

2,430
22
18

3

what if u don't have any service at all? (ie just run bin/elasticsearch -d) – Henley Mar 14 '14 at 02:16
[Note: elasticsearch-servicewrapper has been deprecated and much of the functionality added to elasticsearch.](https://github.com/elastic/elasticsearch-servicewrapper) – notapatch Mar 30 '15 at 20:57
@HenleyChiu then just set ES_HEAP_SIZE environment variable before running it. ie `export ES_HEAP_SIZE=1G; bin/elasticsearch -d` – Andrey Regentov Jun 15 '15 at 11:40
In the ES 1.7.x stock elasticsearch.conf (the one installed by the rpm on CentOS), there is no set.default at all. Does this approach work on ES 1.7? – Jonesome Reinstate Monica Nov 24 '15 at 20:29
The above answer is misleading. Set ES_HEAP_SIZE in /etc/sysconfig/elasticsearch – Jonesome Reinstate Monica Nov 24 '15 at 23:44

score 6 · Answer 8 · edited Jan 05 '18 at 23:47

If you installed ES using the RPM/DEB packages as provided (as you seem to have), you can adjust this by editing the init script (/etc/init.d/elasticsearch on RHEL/CentOS). If you have a look in the file you'll see a block with the following:

export ES_HEAP_SIZE
export ES_HEAP_NEWSIZE
export ES_DIRECT_SIZE
export ES_JAVA_OPTS
export JAVA_HOME

To adjust the size, simply change the ES_HEAP_SIZE line to the following:

export ES_HEAP_SIZE=xM/xG

(where x is the number of MB/GB of RAM that you would like to allocate)

Example:

export ES_HEAP_SIZE=1G

Would allocate 1GB.

Once you have edited the script, save and exit, then restart the service. You can check if it has been correctly set by running the following:

ps aux | grep elasticsearch

And checking for the -Xms and -Xmx flags in the java process that returns:

/usr/bin/java -Xms1G -Xmx1G

Hope this helps :)

Don't do it this way. This is not a good place to make these types of changes. This is a giant hack! Might work, but it'll get potentially trashed whenever ES is updated. — slm, Sep 08 '15 at 20:20
I should add: I tried the approach above on ES 1.7 on CentOS 7, and it had no effect. — Jonesome Reinstate Monica, Nov 24 '15 at 20:27

score 6 · Answer 9 · answered Sep 29 '18 at 00:15

6

Elasticsearch will assign the entire heap specified in jvm.options via the Xms (minimum heap size) and Xmx (maximum heap size) settings.
- -Xmx12g
- -Xmx12g
Set the minimum heap size (Xms) and maximum heap size (Xmx) to be equal to each other.
Don’t set Xmx to above the cutoff that the JVM uses for compressed object pointers (compressed oops), the exact cutoff varies but is near 32 GB.
It is also possible to set the heap size via an environment variable
- ES_JAVA_OPTS="-Xms2g -Xmx2g" ./bin/elasticsearch
- ES_JAVA_OPTS="-Xms4000m -Xmx4000m" ./bin/elasticsearch

answered Sep 29 '18 at 00:15

Satyendra Sharma

297
2
4

2

Also no more than 50% of the physical RAM – Achi Even-dar Oct 15 '19 at 12:37
Hi How i set environment variable like this .Variable name is ES_JAVA_OPTS and value : "-Xms2g -Xmx2g" ./bin/elasticsearch like – Manikandan Jun 15 '20 at 07:13
I recommend making the adjustment in the file `jvm.options` https://www.elastic.co/guide/en/elasticsearch/reference/current/important-settings.html#heap-size-settings – Elinaldo Monteiro Jan 12 '21 at 14:48
I need something configurable, so environment variables are perfect, but I can't seem to make it work. I have an environment file that I'm passing to docker with --env-file argument, and I tried adding `ES_JAVA_OPTS="-Xms2g -Xmx2g" ./bin/elasticsearch` there, but that just gives me `Error: Could not find or load main class "-Xms2g`. Any ideas? – lovrodoe Nov 10 '21 at 10:49

score 6 · Answer 10 · answered Apr 06 '21 at 11:30

6

File path to change heap size /etc/elasticsearch/jvm.options

If you are using nano then do sudo nano /etc/elasticsearch/jvm.options and update -Xms and -Xmx accordingly.

(You can use any file editor to edit it)

answered Apr 06 '21 at 11:30

Its_aggarwal

721
7
9

score 4 · Answer 11 · edited Jan 05 '18 at 23:03

4

In elasticsearch path home dir i.e. typically /usr/share/elasticsearch, There is a config file bin/elasticsearch.in.sh. Edit parameter ES_MIN_MEM, ES_MAX_MEM in this file to change -Xms2g, -Xmx4g respectively. And Please make sure you have restarted the node after this config change.

edited Jan 05 '18 at 23:03

Danilo Cândido

408
1
5
18

answered Oct 24 '16 at 04:00

Amit Sharma

281
5
8

score 3 · Answer 12 · answered Mar 27 '22 at 09:55

3

Oneliner for Centos 7 & Elasticsearch 7 (2g = 2GB)

$ echo $'-Xms2g\n-Xmx2g' > /etc/elasticsearch/jvm.options.d/2gb.options

and then

$ service elasticsearch restart

answered Mar 27 '22 at 09:55

okliv

3,909
30
47

score 2 · Answer 13 · answered Nov 02 '21 at 22:31

If you are using docker-compose to run a ES cluster:

Open <your docker compose>.yml file
If you have set the volumes property, you won't lose anything. Otherwise, you must first move the indexes.
Look for this value ES_JAVA_OPTS under environment and change the value in all nodes, the result could be somethig like "ES_JAVA_OPTS=-Xms2g -Xmx2g"
rebuild all nodes docker-compose -f <your docker compose>.yml up -d

score 1 · Answer 14 · answered Apr 15 '15 at 03:07

1

If you use windows server, you can change Environment Variable, restart server to apply new Environment Value and start Elastic Service. More detail in Install Elastic in Windows Server

answered Apr 15 '15 at 03:07

Thoai Ngo

29
1

score 1 · Answer 15 · edited Feb 26 '18 at 13:10

1

In elasticsearch 2.x :

vi /etc/sysconfig/elasticsearch

Go to the block of code

# Heap size defaults to 256m min, 1g max
# Set ES_HEAP_SIZE to 50% of available RAM, but no more than 31g
#ES_HEAP_SIZE=2g

Uncomment last line like

ES_HEAP_SIZE=2g

edited Feb 26 '18 at 13:10

ata

3,398
5
20
31

answered Feb 26 '18 at 11:11

Ankush

499
1
7
17

score 1 · Answer 16 · answered Aug 01 '21 at 14:53

Update elastic configuration in path /etc/elasticsearch/jvm.options

################################################################
## IMPORTANT: JVM heap size
################################################################
##
## The heap size is automatically configured by Elasticsearch
## based on the available memory in your system and the roles
## each node is configured to fulfill. If specifying heap is
## required, it should be done through a file in jvm.options.d,
## and the min and max should be set to the same value. For
## example, to set the heap to 4 GB, create a new file in the
## jvm.options.d directory containing these lines:
##
## -Xms4g
## -Xmx4g
##
## See https://www.elastic.co/guide/en/elasticsearch/reference/current/heap-size.html
## for more information
##
################################################################
-Xms1g
-Xmx1g

These configs mean you allocate 1GB RAM for elasticsearch service.

score 1 · Answer 17 · answered Jan 05 '22 at 20:51

If you use ubuntu 15.04+ or any other distro that uses systemd, you can set the max memory size editing the elasticsearch systemd service and setting the max memory size using the ES_HEAP_SIZE environment variable, I tested it using ubuntu 20.04 and it works fine:

systemctl edit elasticsearch

Add the environement variable ES_HEAP_SIZE with the desired max memory, here 2GB as example:

[Service]
Environment=ES_HEAP_SIZE=2G

Reload systemd daemon

systemd daemon-reload

Then restart elasticsearch

systemd restart elasticsearch

To check if it worked as expected:

systemd status elasticsearch

You should see in the status -Xmx2G:

 CGroup: /system.slice/elasticsearch.service
         └─2868 /usr/bin/java -Xms2G -Xmx2G

Yogesh Yadav · Answer 18 · 2022-03-14T06:52:23.607

1

Elastic Search 7.x and above, tested with Ubuntu 20

Create a file in /etc/elasticsearch/jvm.options.d. The file name must ends with .options

For example heap_limit.options

Add these lines to the file

## Initial memory allocation

-Xms1g

## Maximum memory allocation

-Xmx1g

Restart elastic search service

sudo service elasticsearch restart

or

sudo systemctl restart elasticsearch

edited Mar 14 '22 at 06:52

answered Mar 04 '22 at 14:38

Yogesh Yadav

4,557
6
34
40

With ES 7.17 you no longer need to specify the heap size at all, it automatically computes it. Note this automatic computation does not work (uses default -Xmx1g) when elasticsearch.yml has deprecated settings node.master=true node.data=true. – jamshid May 26 '23 at 14:56

score 0 · Answer 19 · answered Sep 16 '21 at 17:25

window 7 elasticsearch elastic search memories problem elasticsearch-7.14.1\config\jvm.options

add this
-Xms1g -Xmx1g

elasticsearch-7.14.1\config\elasticsearch.yml

uncomment bootstrap.memory_lock: true

and pest https://github.com/elastic/elasticsearch-servicewrapper download service file and pest lasticsearch-7.14.1\bin

bin\elasticsearch.bat enter

How to change Elasticsearch max memory size

19 Answers19

Linked