fs/notify: optimize inotify/fsnotify code for unwatched files

I have a _tiny_ microbenchmark that sits in a loop and writes single
bytes to a file.  Writing one byte to a tmpfs file is around 2x slower
than reading one byte from a file, which is a _bit_ more than I expecte.
This is a dumb benchmark, but I think it's hard to deny that write() is
a hot path and we should avoid unnecessary overhead there.

I did a 'perf record' of 30-second samples of read and write.  The top
item in a diffprofile is srcu_read_lock() from fsnotify().  There are
active inotify fd's from systemd, but nothing is actually listening to
the file or its part of the filesystem.

I *think* we can avoid taking the srcu_read_lock() for the common case
where there are no actual marks on the file.  This means that there will
both be nothing to notify for *and* implies that there is no need for
clearing the ignore mask.

This patch gave a 13.1% speedup in writes/second on my test, which is an
improvement from the 10.8% that I saw with the last version.

Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Jan Kara <jack@suse.com>
Cc: Al Viro <viro@zeniv.linux.org.uk>
Cc: Eric Paris <eparis@redhat.com>
Cc: John McCutchan <john@johnmccutchan.com>
Cc: Robert Love <rlove@rlove.org>
Cc: Andi Kleen <ak@linux.intel.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

diff --git a/fs/notify/fsnotify.c b/fs/notify/fsnotify.c
index dd3fb0b17be7cc5d914275c6e83da49e7b8e3f48..d675e76251d368ed183fe42368bc4415f9c3184f 100644 (file)
--- a/fs/notify/fsnotify.c
+++ b/fs/notify/fsnotify.c
@@ -204,6 +204,16 @@ int fsnotify(struct inode *to_tell, __u32 mask, void *data, int data_is,
        else
                mnt = NULL;
 
+       /*
+        * Optimization: srcu_read_lock() has a memory barrier which can
+        * be expensive.  It protects walking the *_fsnotify_marks lists.
+        * However, if we do not walk the lists, we do not have to do
+        * SRCU because we have no references to any objects and do not
+        * need SRCU to keep them "alive".
+        */
+       if (hlist_empty(&to_tell->i_fsnotify_marks) &&
+           (!mnt || hlist_empty(&mnt->mnt_fsnotify_marks)))
+               return 0;
        /*
         * if this is a modify event we may need to clear the ignored masks
         * otherwise return if neither the inode nor the vfsmount care about