yarn启用CGroups

Myth丶恋晨 2022-08-09 12:45 168阅读 0赞

##     ##

## Enabling CGroups ##

## (http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.0/bk\_yarn\_resource\_mgt/content/enabling\_cgroups.html) ##

CGroups is a Linux kernel feature. Currently HDP supports CGroups on RHEL6 and Centos6. HDP does not support the default CGroups on Centos7 or RHEL7. However, Centos7 and RHEL7 are supported if you set up your own CGroups. At this time there is no CGroups equivalent for Windows. CGroups are not enabled by default on HDP. CGroups require that the HDP cluster be Kerberos enabled.

<table style="border:1px solid black; margin-bottom:20px; background:inherit!important"> 
 <tbody style="color:inherit!important; border-style:inherit!important; border-width:inherit!important; background:inherit!important"> 
  <tr style="color:inherit!important; border-style:inherit!important; border-width:inherit!important; background:inherit!important"> 
   <td align="center" style="text-align:center; color:inherit!important; border:1px solid black; padding:5px; background:inherit!important"> <img alt="[Note]" src="http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.0/bk_yarn_resource_mgt/common/images/admon/note.png" style="color:inherit!important; border:inherit!important; background:inherit!important"></td> 
   <th align="left" style="color:white; padding:5px; border:inherit!important; background:gray"> Note</th> 
  </tr> 
  <tr style="color:inherit!important; border-style:inherit!important; border-width:inherit!important; background:inherit!important"> 
   <td align="left" style="color:inherit!important; border:1px solid black; padding:5px; background:inherit!important"> <p style="font-size:10pt; font-family:'Helvetica Neue',Helvetica,Arial,sans-serif; color:inherit!important; border:inherit!important; background:inherit!important"> On RHEL7, the&nbsp;<code style="color:inherit!important; border:inherit!important; background:inherit!important">cpu</code>&nbsp;and&nbsp;<code style="color:inherit!important; border:inherit!important; background:inherit!important">cpuacct</code>&nbsp;controllers are managed together by default. The default directory is&nbsp;<code style="color:inherit!important; border:inherit!important; background:inherit!important">/sys/fs/cgroup/cpu,cpuacct</code>. Due to the parsing logic of the container-executor, the presence of the comma in this path may lead to failures when initializing the NodeManager (when using the&nbsp;<code style="color:inherit!important; border:inherit!important; background:inherit!important">LinuxContainerExecutor</code>).</p> <p style="font-size:10pt; font-family:'Helvetica Neue',Helvetica,Arial,sans-serif; color:inherit!important; border:inherit!important; background:inherit!important"> To avoid this issue, create your own directory (such as&nbsp;<code style="color:inherit!important; border:inherit!important; background:inherit!important">/sys/fs/cgroup/hadoop/cpu</code>) and set the&nbsp;<code style="color:inherit!important; border:inherit!important; background:inherit!important">yarn.nodemanager.linux-container-executor.cgroups.mount</code>&nbsp;property to&nbsp;<code style="color:inherit!important; border:inherit!important; background:inherit!important">true</code>. This will allow the NodeManager to mount the&nbsp;<code style="color:inherit!important; border:inherit!important; background:inherit!important">cpu</code>controller, and YARN will be able to enforce CPU limits.</p> <p style="font-size:10pt; font-family:'Helvetica Neue',Helvetica,Arial,sans-serif; color:inherit!important; border:inherit!important; background:inherit!important"> If you would like to mount CGroups yourself, you should set the&nbsp;<code style="color:inherit!important; border:inherit!important; background:inherit!important">yarn.nodemanager.linux-container-executor.cgroups.mount</code>&nbsp;property to&nbsp;<code style="color:inherit!important; border:inherit!important; background:inherit!important">false</code>&nbsp;and ensure that the hierarchy specified in the&nbsp;<code style="color:inherit!important; border:inherit!important; background:inherit!important">yarn.nodemanager.linux-container-executor.cgroups.hierarchy</code>&nbsp;property exists in the mount location. You must also ensure that there are no commas anywhere in the path names.</p> </td> 
  </tr> 
 </tbody> 
</table>

**Enable CGroups**

On Centos 6, CGroups are not set up by default. Run the following commands to set up CGroups:

yum install libcgroup
    sudo mkdir /cgroup
    mount -t cgroup -o cpu cpu /cgroup

Set the following properties in the `/etc/hadoop/conf/yarn-site.xml` file on the ResourceManager and NodeManager hosts:

**Property:**`yarn.nodemanager.container-executor.class`

**Value:**`org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor`

Example:

<property>
     <name>yarn.nodemanager.container-executor.class</name>
     <value>org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor</value>
    </property>

**Property:**`yarn.nodemanager.container-executor.group`

**Value:**`hadoop`

Example:

<property>
     <name>yarn.nodemanager.container-executor.group</name>
     <value>hadoop</value>
    </property>

**Property:** `yarn.nodemanager.container-executor.resources-handler.class`

**Value:** `org.apache.hadoop.yarn.server.nodemanager.util.CgroupsLCEResourcesHandler`

Example:

<property>
     <name>yarn.nodemanager.container-executor.resources-handler.class</name>
     <value>org.apache.hadoop.yarn.server.nodemanager.util.CgroupsLCEResourcesHandler</value>
    </property>

**Property:**`yarn.nodemanager.container-executor.cgroups.hierarchy`

**Value:**`/yarn`

Example:

<property>
     <name>yarn.nodemanager.container-executor.cgroups.hierarchy</name>
     <value>/yarn</value>
    </property>

**Property:** `yarn.nodemanager.container-executor.cgroups.mount`

**Value:**`true`

Example:

<property>
     <name>yarn.nodemanager.container-executor.cgroups.mount</name>
     <value>true</value>
    </property>

**Property:** `yarn.nodemanager.linux-container-executor.cgroups.mount-path`

**Value:**`/cgroup`

Example:

<property>
     <name>yarn.nodemanager.linux-container-executor.cgroups.mount-path</name>
     <value>/cgroup</value>
    </property>

**Set the Percentage of CPU used by YARN**

Set the percentage of CPU that can be allocated for YARN containers. In most cases, the default value of 100% should be used. If you have another process that needs to run on a node that also requires CPU resources, you can lower the percentage of CPU allocated to YARN to free up resources for the other process.

**Property:** `yarn.nodemanager.resource.percentage-physical-cpu-limit`

**Value:**`100`

Example:

<property>
     <name>yarn.nodemanager.resource.percentage-physical-cpu-limit</name>
     <value>100</value>
    </property>

**Set Flexible or Strict CPU limits**

CPU jobs are constrained with CPU scheduling and CGroups enabled, but by default these are flexible limits. If spare CPU cycles are available, containers are allowed to exceed the CPU limits set for them. With flexible limits, the amount of CPU resources available for containers to use can vary based on cluster usage -- the amount of CPU available in the cluster at any given time.

You can use CGroups to set strict limits on CPU usage. When strict limits are enabled, each process receives only the amount of CPU resources it requests. With strict limits, a CPU process will receive the same amount of cluster resources every time it runs.

Strict limits are not enabled (set to `false`) by default. To enable strict CPU limits, set the following property to `true`.

**Property:** `yarn.nodemanager.linux-container-executor.cgroups.strict-resource-usage`

**Value:**`true`

Example:

<property>
     <name>yarn.nodemanager.linux-container-executor.cgroups.strict-resource-usage</name>
     <value>true</value>
     </property>