From: Michal Hocko Date: Thu, 28 Jul 2016 22:44:55 +0000 (-0700) Subject: mm, oom: task_will_free_mem should skip oom_reaped tasks X-Git-Tag: v4.8-rc1~116^2~93 X-Git-Url: https://git.kernelconcepts.de/?a=commitdiff_plain;h=696453e66630ad45e644c4571307fa3ebec9a835;p=karo-tx-linux.git mm, oom: task_will_free_mem should skip oom_reaped tasks The 0-day robot has encountered the following: Out of memory: Kill process 3914 (trinity-c0) score 167 or sacrifice child Killed process 3914 (trinity-c0) total-vm:55864kB, anon-rss:1512kB, file-rss:1088kB, shmem-rss:25616kB oom_reaper: reaped process 3914 (trinity-c0), now anon-rss:0kB, file-rss:0kB, shmem-rss:26488kB oom_reaper: reaped process 3914 (trinity-c0), now anon-rss:0kB, file-rss:0kB, shmem-rss:26900kB oom_reaper: reaped process 3914 (trinity-c0), now anon-rss:0kB, file-rss:0kB, shmem-rss:26900kB oom_reaper: reaped process 3914 (trinity-c0), now anon-rss:0kB, file-rss:0kB, shmem-rss:27296kB oom_reaper: reaped process 3914 (trinity-c0), now anon-rss:0kB, file-rss:0kB, shmem-rss:28148kB oom_reaper is trying to reap the same task again and again. This is possible only when the oom killer is bypassed because of task_will_free_mem because we skip over tasks with MMF_OOM_REAPED already set during select_bad_process. Teach task_will_free_mem to skip over MMF_OOM_REAPED tasks as well because they will be unlikely to free anything more. Analyzed by Tetsuo Handa. Link: http://lkml.kernel.org/r/1466426628-15074-9-git-send-email-mhocko@kernel.org Signed-off-by: Michal Hocko Acked-by: Oleg Nesterov Cc: Tetsuo Handa Cc: Vladimir Davydov Cc: David Rientjes Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds --- diff --git a/mm/oom_kill.c b/mm/oom_kill.c index 8ee92fb76968..36d5dd88d990 100644 --- a/mm/oom_kill.c +++ b/mm/oom_kill.c @@ -747,6 +747,16 @@ bool task_will_free_mem(struct task_struct *task) return false; mm = p->mm; + + /* + * This task has already been drained by the oom reaper so there are + * only small chances it will free some more + */ + if (test_bit(MMF_OOM_REAPED, &mm->flags)) { + task_unlock(p); + return false; + } + if (atomic_read(&mm->mm_users) <= 1) { task_unlock(p); return true;