It is the exciting return of the global checkpoint background
sync. Long, long ago, in snapshot version far, far away we had and only
had a global checkpoint background sync. This sync would fire
periodically and send the global checkpoint from the primary shard to
the replicas so that they could update their local knowledge of the
global checkpoint. Later in time, as we sped ahead towards finalizing
the initial version of sequence IDs, we realized that we need the global
checkpoint updates to be inline. This means that on a replication
operation, the primary shard would piggy back the global checkpoint with
the replication operation to the replicas. The replicas would update
their local knowledge of the global checkpoint and reply with their
local checkpoint. However, this could allow the global checkpoint on the
primary to advance again and the replicas would fall behind in their
local knowledge of the global checkpoint. If another replication
operation never fired, then the replicas would be permanently behind. To
account for this, we added one more sync that would fire when the
primary shard fell idle. However, this has problems:
 - the shard idle timer defaults to five minutes, a long time to wait
   for the replicas to learn of the new global checkpoint
 - if a replica missed the sync, there was no follow-up sync to catch
   them up
 - there is an inherent race condition where the primary shard could
   fall idle mid-operation (after having sent the replication request to
   the replicas); in this case, there would never be a background sync
   after the operation completes
 - tying the global checkpoint sync to the idle timer was never natural

To fix this, we add back a global checkpoint background sync that fires
on a timer. This timer fires every thirty seconds, and is not
configurable (for simplicity). This background sync is smarter in the
sense that it only sends a sync if the global checkpoint on at least one
replica is lagging that of the primary. This necessitates adding the
primary shard tracking its knowledge of the local knowledge of the
global checkpoint on the replicas. When the timer fires, we can compare
the global checkpoint on the primary to its knowledge of the global
checkpoint on the replicas and only send a sync if there is a shard
behind. During replication operations it can be the case that the timer
fires and sends a sync that would be covered by an in-flight
operation. This is okay, the extra sync does not hurt and we do not need
the complexity of optimizing away this duplicate sync.

* master: (67 commits)
  Restoring from snapshot should force generation of a new history uuid (elastic#26694)
  test: Use a single primary shard so that the exception can caught in the same way
  Move pre-6.0 node checkpoint to SequenceNumbers
  Invalid JSON request body caused endless loop (elastic#26680)
  added comment
  fix line length violation
  Moved the check to fetch phase. This basically means that we throw a better error message instead of an AOBE and not adding more restrictions.
  inner hits: Do not allow inner hits that use _source and have a non nested object field as parent
  Separate Painless Whitelist Loading from the Painless Definition (elastic#26540)
  convert more admin requests to writeable (elastic#26566)
  Handle release of 5.6.1
  Allow `InputStreamStreamInput` array size validation where applicable (elastic#26692)
  Update global checkpoint with permit after recovery
  Filter pre-6.0 nodes for checkpoint invariants
  Skip bad request REST test on pre-6.0
  Reenable BWC tests after disabling for backport
  Add global checkpoint tracking on the primary
  [Test] Fix reference/cat/allocation/line_8 test failure
  [Docs] improved description for fs.total.available_in_bytes (elastic#26657)
  Fix discovery-file plugin to use custom config path
  ...

* master:
  Remove assertion from checkpoint tracker invariants
  Upgrade API: fix excessive logging and unnecessary template updates (elastic#26698)

* master:
  [DOCS] Added index-shared4 and index-shared5.asciidoc
  BulkProcessor flush runnable preserves the thread context from creation time (elastic#26718)
  Catch exceptions and inform handler in RemoteClusterConnection#collectNodes (elastic#26725)
  [Docs] Fix name of character filter in example. (elastic#26724)
  Remove parse field deprecations in query builders (elastic#26711)
  elastic#26720: Set the correct bwc version after backport to 6.0
  Remove deprecated type and slop field in MatchQueryBuilder (elastic#26720)
  Refactoring of Gateway*** classes (elastic#26706)
  Make RestHighLevelClient's Request class public (elastic#26627)
  Deguice ActionFilter (elastic#26691)
  aggs: Allow aggregation sorting via nested aggregation.
  Build: Set bwc builds to always set snapshot (elastic#26704)
  File Discovery: Remove fallback with zen discovery (elastic#26667)

This reverts commit 9bc5155.

They can not work right not until this is backported; a primary running
6.x code without this patch will not be sending the global checkpoint
sync yet.

* master:
  Add permission checks before reading from HDFS stream (elastic#26716)
  muted test
  [Docs] Fixed typo of *configuration* (elastic#25058)
  Add azure storage endpoint suffix elastic#26432 (elastic#26568)