Kafka server failed to start - java.io.IOException: Map failed

Question

I am not able to start Kafka Server because of the error below.

java.io.IOException: Map failed
    at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:940)
    at kafka.log.AbstractIndex.<init>(AbstractIndex.scala:61)
    at kafka.log.TimeIndex.<init>(TimeIndex.scala:55)
    at kafka.log.LogSegment.<init>(LogSegment.scala:73)
    at kafka.log.Log.loadSegments(Log.scala:267)
    at kafka.log.Log.<init>(Log.scala:116)
    at kafka.log.LogManager$$anonfun$createLog$1.apply(LogManager.scala:365)
    at kafka.log.LogManager$$anonfun$createLog$1.apply(LogManager.scala:361)
    at scala.Option.getOrElse(Option.scala:121)
    at kafka.log.LogManager.createLog(LogManager.scala:361)
    at kafka.cluster.Partition$$anonfun$getOrCreateReplica$1.apply(Partition.scala:109)
    at kafka.cluster.Partition$$anonfun$getOrCreateReplica$1.apply(Partition.scala:106)
    at kafka.utils.Pool.getAndMaybePut(Pool.scala:70)
    at kafka.cluster.Partition.getOrCreateReplica(Partition.scala:105)
    at kafka.cluster.Partition$$anonfun$4$$anonfun$apply$3.apply(Partition.scala:166)
    at kafka.cluster.Partition$$anonfun$4$$anonfun$apply$3.apply(Partition.scala:166)
    at scala.collection.mutable.HashSet.foreach(HashSet.scala:78)
    at kafka.cluster.Partition$$anonfun$4.apply(Partition.scala:166)
    at kafka.cluster.Partition$$anonfun$4.apply(Partition.scala:160)
    at kafka.utils.CoreUtils$.inLock(CoreUtils.scala:213)
    at kafka.utils.CoreUtils$.inWriteLock(CoreUtils.scala:221)
    at kafka.cluster.Partition.makeLeader(Partition.scala:160)
    at kafka.server.ReplicaManager$$anonfun$makeLeaders$4.apply(ReplicaManager.scala:754)
    at kafka.server.ReplicaManager$$anonfun$makeLeaders$4.apply(ReplicaManager.scala:753)
    at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
    at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
    at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:230)
    at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40)
    at scala.collection.mutable.HashMap.foreach(HashMap.scala:99)
    at kafka.server.ReplicaManager.makeLeaders(ReplicaManager.scala:753)
    at kafka.server.ReplicaManager.becomeLeaderOrFollower(ReplicaManager.scala:698)
    at kafka.server.KafkaApis.handleLeaderAndIsrRequest(KafkaApis.scala:148)
    at kafka.server.KafkaApis.handle(KafkaApis.scala:84)
    at kafka.server.KafkaRequestHandler.run(KafkaRequestHandler.scala:62)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.OutOfMemoryError: Map failed
    at sun.nio.ch.FileChannelImpl.map0(Native Method)
    at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:937)
    ... 34 more

Tried the below options . but of no use . please help

Upgraded OS from 32 to 64 Bit.
Increased java heap size to 1 GB.
Uninstalled and installed Apache Kafka

JVM is the problem. I was using JVM 32 Bit and upgrading to 64 Bit solves the issue. Thanks. — Gopi, Mar 27 '17 at 11:10
Ok that's cool. You should post the answer then! It might be helpful to someone else. — Maximilien Belinga, Mar 27 '17 at 11:16

score 15 · Answer 1 · answered Apr 28 '17 at 08:30

15

When this doesn't resolve the problem you could try to increase vm.max_map_count. The default value is 65536 (check this with sysctl vm.max_map_count)

With cat /proc/[kafka-pid]/maps | wc -l you can see how many maps are used.

Increase the setting with :

sysctl -w vm.max_map_count=262144

answered Apr 28 '17 at 08:30

jvwilge

2,474
2
16
21

Worked for us, After increasing max memory-map count we are not facing issue anymore. – Shrikant Salgar Dec 11 '18 at 06:13
1

Is there a setting for windows? – Jaison Apr 08 '20 at 09:11

score 12 · Accepted Answer · edited Sep 18 '17 at 19:57

12

Upgrading the JVM to 64 bit resolved the issue

edited Sep 18 '17 at 19:57

Matt

3,617
2
27
39

answered Mar 27 '17 at 13:56

Gopi

619
2
9
27

score 7 · Answer 3 · answered May 27 '20 at 15:52

7

This helped me : Changing KAFKA_HEAP_OPTS="-Xmx256M -Xms256M" (Originally 512m)

change this to : kafka-server-start script

Thanks !

answered May 27 '20 at 15:52

GAURAV PATEL

71
1
1

1

Surprisingly, DECREASING memory fixed my issue as well. – ccu Nov 13 '20 at 21:57
This worked on my windows setup. – cody.tv.weber Sep 28 '22 at 15:21

score 2 · Answer 4 · answered Apr 23 '20 at 12:48

2

this helped me: change : export KAFKA_HEAP_OPTS="-Xmx512M -Xms512M" (originally 1G)

in kafka-server-start script

answered Apr 23 '20 at 12:48

Dina Goldebrg

21
2

score 1 · Answer 5 · answered Oct 22 '19 at 21:57

I had the same issue on Windows, Kafka takes some memory during the process. So We need to increase the heap to prevent throttling the performance of the application. This can be achieved graphically through JAVA Control Panel.

Inside Runtime Parameters, you can change the size of the memory allocated by JVM:

-Xmx512m that assigns 512MB,
-Xmx1024m that assigns 1GB,
-Xmx2048m that assigns 2GB,
-Xmx3072m that assigns 3GB memory and so on.

Kafka server failed to start - java.io.IOException: Map failed

5 Answers5