Linux查看程序被Killed原因

谁践踏了优雅 2023-07-20 12:27 20阅读 0赞

### `1.查看信息，输出最近被killed的进程` ###

*  `查看被Killed简短信息列表 egrep -i -r 'killed process' /var/log`

[root@ecs-bcb3 ~]# egrep -i -r 'killed process' /var/log 
    /var/log/messages:Jul 12 15:00:24 ecs-bcb3 kernel: Killed process 26399 (python3), UID 1002, total-vm:27455044kB, anon-rss:24933916kB, file-rss:712kB, shmem-rss:0kB
    /var/log/messages:Jul 12 15:00:24 ecs-bcb3 kernel: Killed process 29412 (python3), UID 1002, total-vm:27455044kB, anon-rss:24934124kB, file-rss:780kB, shmem-rss:0kB
    /var/log/messages:Jul 12 15:18:39 ecs-bcb3 kernel: Killed process 25017 (chromedriver), UID 1002, total-vm:218888kB, anon-rss:2312kB, file-rss:184kB, shmem-rss:0kB
    /var/log/messages:Jul 12 15:18:39 ecs-bcb3 kernel: Killed process 30252 (chromedriver), UID 1002, total-vm:201472kB, anon-rss:2032kB, file-rss:104kB, shmem-rss:0kB

*  `查看被Killed详细信息列表 dmesg | egrep -i -B100 'killed process'`

[root@ecs-bcb3 ~]# dmesg | egrep -i -B100 'killed process'
    [19095552.852409] [31573]  1002 31573   131893     2522     166        0             0 chrome
    [19095552.852585] [31597]  1002 31597   144656     4247     168        0           200 chrome
    [19095552.852756] [31599]  1002 31599   148317     2766     183        0             0 chrome
    
    [19095552.869403] [26287]  1004 26287  1975489   186919     547        0             0 java
    [19095552.869548] [27054]   999 27054     1154       23       8        0             0 sh
    [19095552.869700] [27058]   999 27058     1110       16       8        0             0 df
    [19095552.869842] Out of memory: Kill process 7429 (python3) score 357 or sacrifice child
    [19095552.869991] Killed process 7429 (python3), UID 1002, total-vm:26420076kB, anon-rss:23508276kB, file-rss:920kB, shmem-rss:0kB
    ...

*  `查看被Killed简短信息列表 journalctl -xb | egrep -i 'killed process'`

12月 21 15:33:18 ecs-kms kernel: Killed process 2580 (chrome), UID 0, total-vm:862648kB, anon-rss:53044kB, file-rss:156kB, shmem-rss:2692kB
    12月 21 15:33:18 ecs-kms kernel: Killed process 3000 (chrome), UID 0, total-vm:862136kB, anon-rss:49508kB, file-rss:16kB, shmem-rss:2508kB
    12月 21 15:33:18 ecs-kms kernel: Killed process 3165 (chrome), UID 0, total-vm:859320kB, anon-rss:48372kB, file-rss:112kB, shmem-rss:2492kB
    ...

### `2.设定kill优先度` ###

`2.1 完全关掉oom`(不建议)

sysctl vm.overcommit_memory=2
    echo “vm.overcommit_memory=2” >> /etc/sysctl.conf

`2.2 出现 Out Of Memory 之后重启`

sysctl vm.panic_on_oom=1
    sysctl kernel.panic=X
    echo “vm.panic_on_oom=1” >> /etc/sysctl.conf
    echo “kernel.panic=X” >> /etc/sysctl.conf

`2.3 优先考虑kill这个进程`

sudo echo 10>/proc/[PID]/oom_adj

`2.4 尽量不去kill这个进程`

sudo echo -15>/proc/[PID]/oom_adj

`2.5 不会kill这个进程`

sudo echo -17>/proc/[PID]/oom_adj

**`备注`：**`oom_adj的值在-16 到 +15之间，值越高被kill的优先度越高。当该值为-17时，系统将不会杀死指定pid的进程，而-16~15则会使得进程的 /proc/[pid]/oom_adj 值呈指数形式递增，即它们被杀掉的可能性呈指数递增。针对init（进程号为1）这个进程，无论该值设为多少都不会被杀`

### `3.若上述命令被拒绝访问，命令修改成以下方式` ###

bash -c "echo '10' | tee /proc/[PID]/oom_adj"
    bash -c "echo '-15' | tee /proc/[PID]/oom_adj"
    bash -c "echo '-17' | tee /proc/[PID]/oom_adj"