CDH中yarn内存控制

妖狐艹你老母 2022-02-14 09:22 262阅读 0赞

*  注： 堆java size > 分配内存

# 1, ResourceManager 内存 #

<table> 
 <thead> 
  <tr> 
   <th align="left">－－－－内存(MB)－－－</th> 
   <th align="left"></th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <td align="left">yarn.scheduler.minimum-allocation-mb</td> 
   <td align="left">默认　1024MB</td> 
  </tr> 
  <tr> 
   <td align="left">yarn.scheduler.maximum-allocation-mb</td> 
   <td align="left">默认　 8192MB</td> 
  </tr> 
  <tr> 
   <td align="left">－－－－java堆内存大小－－－</td> 
   <td align="left"></td> 
  </tr> 
  <tr> 
   <td align="left">Java Heap Size of ResourceManager in Bytes</td> 
   <td align="left">默认　1G</td> 
  </tr> 
  <tr> 
   <td align="left">－－－－cpu个数－－－</td> 
   <td align="left"></td> 
  </tr> 
  <tr> 
   <td align="left">yarn.scheduler.minimum-allocation-vcores</td> 
   <td align="left">默认　 1</td> 
  </tr> 
  <tr> 
   <td align="left">yarn.scheduler.maximum-allocation-vcores</td> 
   <td align="left">默认　 4</td> 
  </tr> 
 </tbody> 
</table>

# 2 :ApplicationMaster 内存 #

<table> 
 <thead> 
  <tr> 
   <th align="left">－－－－ApplicationMaster Memory内存－－－</th> 
   <th></th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <td align="left">yarn.app.mapreduce.am.resource.mb</td> 
   <td></td> 
  </tr> 
  <tr> 
   <td align="left">－－－－java堆内存大小－－－</td> 
   <td></td> 
  </tr> 
  <tr> 
   <td align="left">ApplicationMaster Java Maximum Heap Size</td> 
   <td></td> 
  </tr> 
  <tr> 
   <td align="left">－－－ApplicationMaster Virtual CPU Cores－－－</td> 
   <td></td> 
  </tr> 
  <tr> 
   <td align="left">yarn.app.mapreduce.am.resource.cpu-vcores</td> 
   <td>默认　 1</td> 
  </tr> 
 </tbody> 
</table>

# 3, NodeManager内存 #

<table> 
 <thead> 
  <tr> 
   <th align="left">yarn.nodemanager.resource.memory-mb</th> 
   <th align="left">默认　-1　(80%物理内存)</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <td align="left">yarn.nodemanager.resource.cpu-vcores</td> 
   <td align="left">默认　-1</td> 
  </tr> 
  <tr> 
   <td align="left">yarn.nodemanager.resource.cpu-vcores</td> 
   <td align="left">默认　-1</td> 
  </tr> 
  <tr> 
   <td align="left">－－－－java堆内存大小－－－</td> 
   <td align="left"></td> 
  </tr> 
  <tr> 
   <td align="left">Java Heap Size of NodeManager in Bytes</td> 
   <td align="left">默认　1G</td> 
  </tr> 
 </tbody> 
</table>

# 4, map-reduce内存 #

<table> 
 <thead> 
  <tr> 
   <th align="left">mapreduce.map.memory.mb</th> 
   <th align="left">[minimum-allocation-mb , maximum-allocation-mb)</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <td align="left">mapreduce.reduce.memory.mb</td> 
   <td align="left">[minimum-allocation-mb , maximum-allocation-mb) : 约等2x的map内存</td> 
  </tr> 
  <tr> 
   <td align="left">－－－－java堆内存大小－－－</td> 
   <td align="left"></td> 
  </tr> 
  <tr> 
   <td align="left">Map Task Maximum Heap Size</td> 
   <td align="left">Reduce Task Maximum Heap Size</td> 
  </tr> 
 </tbody> 
</table>