[HBase] Current heap configuration for MemStore and BlockCache exceeds the threshold required for successful cluster operation. 에러 발생

눈가락 2019. 12. 5. 16:40

2019. 12. 5. 16:40

에러가 어떻게 발생했는지 적는다.

hbase 튜닝을 하고 있었다. hfile.block.cache.size 값을 40% 주고 싶어서 아래처럼 했다.

hbase-site.xml 에서

<property>

<name>hfile.block.cache.size</name>

<value>40</value>

</property>

사실 0.4 를 넣어야 하는데 실수로 40을 넣어버렸더니 아래 에러가 떴다.

java.lang.RuntimeException: Current heap configuration for MemStore and BlockCache exceeds the threshold required for successful cluster operation. The combined value cannot exceed 0.8. Please check the settings for hbase.regionserver.global.memstore.size and hfile.block.cache.size in your configuration. hbase.regionserver.global.memstore.size is 0.4 hfile.block.cache.size is 40.0

에러가 뜨면서 HMaster 도 HRegionServer 들도 다 죽어버려서 잠깐 당황했지만

당황하지 않고 log 를 보고 원인을 파악.

0.4로 다시 바꾸고 재시동해서 에러를 없앴다.

구체적인 에러 보기

2019-12-05 07:39:15,003 ERROR [main] regionserver.HRegionServer: Failed construction RegionServer

at org.apache.hadoop.hbase.io.util.MemorySizeUtil.checkForClusterFreeHeapMemoryLimit(MemorySizeUtil.java:95)

at org.apache.hadoop.hbase.regionserver.HRegionServer.<init>(HRegionServer.java:569)

at org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:494)

at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)

at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)

at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)

at java.lang.reflect.Constructor.newInstance(Constructor.java:423)

at org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:2924)

at org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:236)

at org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:140)

at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)

at org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:149)

at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:2942)

2019-12-05 07:39:15,006 ERROR [main] master.HMasterCommandLine: Master exiting

java.lang.RuntimeException: Failed construction of Master: class org.apache.hadoop.hbase.master.HMaster.

at org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:2931)

at org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:236)

at org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:140)

at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)

at org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:149)

at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:2942)

Caused by: java.lang.RuntimeException: Current heap configuration for MemStore and BlockCache exceeds the threshold required for successful cluster operation. The combined value cannot exceed 0.8. Please check the settings for hbase.regionserver.global.memstore.size and hfile.block.cache.size in your configuration. hbase.regionserver.global.memstore.size is 0.4 hfile.block.cache.size is 40.0