Fix data race between zil_commit() and zil_suspend()

[ryao]

Motivation and Context

openzfsonwindows#206 found that it is possible to trip VERIFY(list_is_empty(&lwb->lwb_itxs)) when a zil_commit() is delayed by the scheduler long enough for a parallel zil_suspend() operation to exit zil_commit_impl(). This is a data race. To prevent this, we introduce a zilog->zl_commit_lock rwlock to ensure that all outstanding zil_commit() operations finish before zil_suspend() begins and that subsequent operations fallback to txg_wait_synced() after zil_suspend() has begun.

On PREEMPT_RT Linux kernels, the rw_enter() implementation suffers from writer starvation. This means that a ZIL intensive system can delay zil_suspend() indefinitely. This is a pre-existing problem that affects everything that uses rw locks, so it needs to be addressed in the SPL. However, builds against PREEMPT_RT Linux kernels are currently broken due to a GPL symbol issue (#11097), so we can safely disregard that issue for now.

Description

We modify zil_commit() to grab a read lock for both its suspend check and the full duration of zil_commit_impl(), although not for the duration of txg_wait_synced() when the suspend check shows that ZIL is suspended. We also modify zil_suspend() to grab a write lock before grabbing zilog->zl_lock and release it after it has incremented zilog->zl_suspend. The result is that all outstanding zil_commit() operations finish before zil_suspend() begins and subsequent zil_commit() operations fallback to txg_wait_synced() as expected. This prevents the scheduler from adding arbitrarily long waits to zil_commit() that will cause it to run zil_commit_impl() on a ZIL that is either already suspending or already suspended.

Note that grabbing the write lock before grabbing zilog->zl_lock is intended to prevent a lock inversion deadlock between zil_commit() and zil_suspend() on the two locks.

How Has This Been Tested?

The buildbot can test it.

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Performance enhancement (non-breaking change which improves efficiency)
• Code cleanup (non-breaking change which makes code smaller or more readable)
• Breaking change (fix or feature that would cause existing functionality to change)
• Library ABI change (libzfs, libzfs_core, libnvpair, libuutil and libzfsbootenv)
• Documentation (a change to man pages or other documentation)

Checklist:

• My code follows the OpenZFS code style requirements.
• I have updated the documentation accordingly.
• I have read the contributing document.
• I have added tests to cover my changes.
• I have run the ZFS Test Suite with this change applied.
• All commit messages are properly formatted and contain Signed-off-by.

[ryao]

This appears to be a regression introduced by 1ce23dc.

[ryao]

I have rebased this on master and repushed.

[amotin]

I would call this lock zl_suspend_lock, since that is what it protects.

Also I am worrying about one more contention point per ZIL. Even though in most cases it won't block, it is still a couple of atomics on shared variable. This reminds me about FreeBSD's rms_rlock() primitives, but may be there are some other solutions too.

[ryao]

I would call this lock zl_suspend_lock, since that is what it protects.

I will change the name when I do my next push.

Also I am worrying about one more contention point per ZIL. Even though in most cases it won't block, it is still a couple of atomics on shared variable. This reminds me about FreeBSD's rms_rlock() primitives, but may be there are some other solutions too.

I could not see a better way to close the race.

[ryao]

The latest push rebases on master and addresses both of @amotin's comments.