Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

box.info.ro can be true in init_storage() on all instances in replicaset #412

Closed
askalt opened this issue Jan 10, 2024 · 1 comment · Fixed by #417
Closed

box.info.ro can be true in init_storage() on all instances in replicaset #412

askalt opened this issue Jan 10, 2024 · 1 comment · Fixed by #417
Assignees
Labels
bug Something isn't working

Comments

@askalt
Copy link

askalt commented Jan 10, 2024

As the result, access grants may be skipped.

This should be documented or the code should account for it.

@DifferentialOrange DifferentialOrange self-assigned this Jan 10, 2024
@DifferentialOrange DifferentialOrange added the bug Something isn't working label Jan 10, 2024
DifferentialOrange added a commit that referenced this issue Jan 23, 2024
This is the preliminary refactoring before solving the original issue.

Part of #412
Part of #415
DifferentialOrange added a commit that referenced this issue Jan 23, 2024
DifferentialOrange added a commit that referenced this issue Jan 23, 2024
This patch add a storage handle to wait till module is bootstrapped.
The handle is based on existing `storage_info` check. The handle is
for using on storages only: it makes no sense to call in on the router.

The current standard pattern is as follows: `crud.init_storage()` is
called after `box.cfg` on both masters and replicas. In case of
Tarantool 1.x and 2.x rw-instances,
`crud.init_storage{wait_until_ready = true}` doesn't introduce anything
new. In case of Tarantool 1.x and 2.x ro-instances, it helps to wait
until persistent part of bootstrap (functions and grants) are
replicated. The main motivation behind this handle is the support of
asynchronous start that will be introduced in next commits.
The asynchronous start is required to properly work with Tarantool 3
instances: all Tarantool 3.x instances start in ro mode.

This patch doesn't change existing behavior for Cartridge roles and
`crud.init_storage()` calls.

Part of #412
Part of #415
DifferentialOrange added a commit that referenced this issue Jan 23, 2024
All Tarantool 3.x instances start in ro mode. Simple
`if not box.info.ro` check isn't enough to properly bootstrap a user
since it may pass before master instance became rw. box.watch is
used to track the moment when instance became rw [1]. Watchers are
supported since Tarantool 2.10+.

This patch doesn't change existing behavior for Cartridge roles and
`crud.init_storage()` calls.

1. https://www.tarantool.io/ru/doc/latest/reference/reference_lua/box_events/

Part of #412
Part of #415
DifferentialOrange added a commit that referenced this issue Jan 23, 2024
Tarantool 3.0 instances start in ro mode. It is possible that masters
are not available for write requests right after test case has been
started.

Cartridge servers run only on Tarantool 1.x and 2.x, so they are
expected to work fine without additional changes: Cartridge helpers
checks are sophisticated enough.

Part of #412
Part of #415
DifferentialOrange added a commit that referenced this issue Jan 23, 2024
The approach is similar to crud bootstrap waiting. Cartridge tests
approach remains the same.

Part of #412
Part of #415
DifferentialOrange added a commit that referenced this issue Jan 23, 2024
The approach is similar to crud bootstrap waiting. Cartridge tests
approach remains the same.

Part of #412
Part of #415
DifferentialOrange added a commit that referenced this issue Jan 23, 2024
Tarantool 3.0 instances start in ro mode. It is possible that masters
are not available for write requests right after test case has been
started.

Cartridge servers run only on Tarantool 1.x and 2.x, so they are
expected to work fine without additional changes: Cartridge helpers
checks are sophisticated enough.

Part of #412
Part of #415
DifferentialOrange added a commit that referenced this issue Jan 23, 2024
The approach is similar to crud bootstrap waiting. Cartridge tests
approach remains the same.

Part of #412
Part of #415
DifferentialOrange added a commit that referenced this issue Jan 23, 2024
Tarantool 3.0 instances start in ro mode. It is possible that masters
are not available for write requests right after test case has been
started.

Cartridge servers run only on Tarantool 1.x and 2.x, so they are
expected to work fine without additional changes: Cartridge helpers
checks are sophisticated enough.

Part of #412
Part of #415
DifferentialOrange added a commit that referenced this issue Jan 23, 2024
The approach is similar to crud bootstrap waiting. Cartridge tests
approach remains the same.

Part of #412
Part of #415
DifferentialOrange added a commit that referenced this issue Jan 23, 2024
All Tarantool 3.x instances start in ro mode. Simple
`if not box.info.ro` check isn't enough to properly bootstrap a user
since it may pass before master instance became rw. box.watch is
used to track the moment when instance became rw [1]. Watchers are
supported since Tarantool 2.10+.

This patch doesn't change existing behavior for Cartridge roles and
`crud.init_storage()` calls.

1. https://www.tarantool.io/ru/doc/latest/reference/reference_lua/box_events/

Part of #412
Part of #415
DifferentialOrange added a commit that referenced this issue Jan 23, 2024
Tarantool 3.0 instances start in ro mode. It is possible that masters
are not available for write requests right after test case has been
started.

Cartridge servers run only on Tarantool 1.x and 2.x, so they are
expected to work fine without additional changes: Cartridge helpers
checks are sophisticated enough.

Part of #412
Part of #415
DifferentialOrange added a commit that referenced this issue Jan 23, 2024
The approach is similar to crud bootstrap waiting. Cartridge tests
approach remains the same.

Part of #412
Part of #415
DifferentialOrange added a commit that referenced this issue Jan 23, 2024
After this patch, it is possible to start a Tarantool 3.0 cluster
from the config. It doesn't yet integrated with common test matrix yet.
The tests are basic and shallow since it would be tested against
the whole crud test suite in the next commits of the patchset.

Part of #412
Part of #415
DifferentialOrange added a commit that referenced this issue Jan 23, 2024
After this patch, all unit and integration tests run also on cluster
started from configuration file.

Part of #412
Part of #415
DifferentialOrange added a commit that referenced this issue Jan 23, 2024
All Tarantool 3.x instances start in ro mode. Simple
`if not box.info.ro` check isn't enough to properly bootstrap a user
since it may pass before master instance became rw. box.watch is
used to track the moment when instance became rw [1]. Watchers are
supported since Tarantool 2.10+.

This patch doesn't change existing behavior for Cartridge roles and
`crud.init_storage()` calls.

1. https://www.tarantool.io/ru/doc/latest/reference/reference_lua/box_events/

Part of #412
Part of #415
DifferentialOrange added a commit that referenced this issue Jan 23, 2024
Tarantool 3.0 instances start in ro mode. It is possible that masters
are not available for write requests right after test case has been
started.

Cartridge servers run only on Tarantool 1.x and 2.x, so they are
expected to work fine without additional changes: Cartridge helpers
checks are sophisticated enough.

Part of #412
Part of #415
DifferentialOrange added a commit that referenced this issue Jan 23, 2024
The approach is similar to crud bootstrap waiting. Cartridge tests
approach remains the same.

Part of #412
Part of #415
DifferentialOrange added a commit that referenced this issue Jan 23, 2024
After this patch, it is possible to start a Tarantool 3.0 cluster
from the config. It doesn't yet integrated with common test matrix yet.
The tests are basic and shallow since it would be tested against
the whole crud test suite in the next commits of the patchset.

Part of #412
Part of #415
DifferentialOrange added a commit that referenced this issue Jan 23, 2024
After this patch, all unit and integration tests run also on cluster
started from configuration file.

Part of #412
Part of #415
DifferentialOrange added a commit that referenced this issue Jan 24, 2024
This is the preliminary refactoring before solving the original issue.

Part of #412
Part of #415
DifferentialOrange added a commit that referenced this issue Jan 24, 2024
DifferentialOrange added a commit that referenced this issue Jan 24, 2024
This patch add a storage handle to wait till module is bootstrapped.
The handle is based on existing `storage_info` check. The handle is
for using on storages only: it makes no sense to call in on the router.

The current standard pattern is as follows: `crud.init_storage()` is
called after `box.cfg` on both masters and replicas. In case of
Tarantool 1.x and 2.x rw-instances,
`crud.init_storage{wait_until_ready = true}` doesn't introduce anything
new. In case of Tarantool 1.x and 2.x ro-instances, it helps to wait
until persistent part of bootstrap (functions and grants) are
replicated. The main motivation behind this handle is the support of
asynchronous start that will be introduced in next commits.
The asynchronous start is required to properly work with Tarantool 3
instances: all Tarantool 3.x instances start in ro mode.

This patch doesn't change existing behavior for Cartridge roles and
`crud.init_storage()` calls.

Part of #412
Part of #415
DifferentialOrange added a commit that referenced this issue Jan 24, 2024
All Tarantool 3.x instances start in ro mode. Simple
`if not box.info.ro` check isn't enough to properly bootstrap a user
since it may pass before master instance became rw. box.watch is
used to track the moment when instance became rw [1]. Watchers are
supported since Tarantool 2.10+.

This patch doesn't change existing behavior for Cartridge roles and
`crud.init_storage()` calls.

1. https://www.tarantool.io/ru/doc/latest/reference/reference_lua/box_events/

Part of #412
Part of #415
DifferentialOrange added a commit that referenced this issue Jan 24, 2024
Tarantool 3.0 instances start in ro mode. It is possible that masters
are not available for write requests right after test case has been
started.

Cartridge servers run only on Tarantool 1.x and 2.x, so they are
expected to work fine without additional changes: Cartridge helpers
checks are sophisticated enough.

Part of #412
Part of #415
DifferentialOrange added a commit that referenced this issue Jan 24, 2024
The approach is similar to crud bootstrap waiting. Cartridge tests
approach remains the same.

Part of #412
Part of #415
DifferentialOrange added a commit that referenced this issue Apr 1, 2024
Support asynchronous bootstrap for crud storages. The main motivation is
compatibility with Tarantool 3 instances, which start in read-only mode.

We do not support async bootstrap for crud routers since they do
not change `box`, thus may start on read-only instances without any
issues.

Part of #412
Part of #415
DifferentialOrange added a commit that referenced this issue Apr 1, 2024
DifferentialOrange added a commit that referenced this issue Apr 1, 2024
Support asynchronous bootstrap for crud storages. The main motivation is
compatibility with Tarantool 3 instances, which start in read-only mode.

We do not support async bootstrap for crud routers since they do
not change `box`, thus may start on read-only instances without any
issues.

Part of #412
Part of #415
DifferentialOrange added a commit that referenced this issue Apr 2, 2024
This is the preliminary refactoring before solving the original issue.

Part of #412
Part of #415
DifferentialOrange added a commit that referenced this issue Apr 2, 2024
DifferentialOrange added a commit that referenced this issue Apr 2, 2024
DifferentialOrange added a commit that referenced this issue Apr 2, 2024
Support asynchronous bootstrap for crud storages. The main motivation is
compatibility with Tarantool 3 instances, which start in read-only mode.

We do not support async bootstrap for crud routers since they do
not change `box`, thus may start on read-only instances without any
issues.

Part of #412
Part of #415
DifferentialOrange added a commit that referenced this issue Apr 3, 2024
Overview

  This release introduces roles for Tarantool 3 configuration (supported
  for versions 3.0.2, 3.1.0 and newer).

Added
* Asynchronous bootstrap support for storages (#412).
* Tarantool 3 roles for setting up crud routers and storages (#415).
* Ability to configure crud through Tarantool 3 roles
  configuration (#415).

Changed
* Explicitly forbid datetime interval conditions (#373).
* Storage initialization is now asynchronous by default for
  Tarantool 3.0+ (#412).
* Additionally check backoff error on storage info fetch (#427).

Fixed
* Working with datetime conditions in case of non-indexed fields or
  non-iterating indexes (#373).
* Precision loss for decimal conditions in case of non-indexed fields or
  non-iterating indexes (#373).
* Passing errors from storages for merger operations (`crud.select`,
  `crud.pairs`, `readview:select`, `readview:pairs`) (#423).
* Working with `nil` operand conditions in case of non-indexed fields or
  non-iterating indexes (#422).
DifferentialOrange added a commit that referenced this issue Apr 4, 2024
Overview

  This release introduces roles for Tarantool 3 configuration (supported
  for versions 3.0.2, 3.1.0 and newer).

Added
* Asynchronous bootstrap support for storages (#412).
* Tarantool 3 roles for setting up crud routers and storages (#415).
* Ability to configure crud through Tarantool 3 roles
  configuration (#415).

Changed
* Explicitly forbid datetime interval conditions (#373).
* Storage initialization is now asynchronous by default for
  Tarantool 3.0+ (#412).
* Additionally check backoff error on storage info fetch (#427).

Fixed
* Working with datetime conditions in case of non-indexed fields or
  non-iterating indexes (#373).
* Precision loss for decimal conditions in case of non-indexed fields or
  non-iterating indexes (#373).
* Passing errors from storages for merger operations (`crud.select`,
  `crud.pairs`, `readview:select`, `readview:pairs`) (#423).
* Working with `nil` operand conditions in case of non-indexed fields or
  non-iterating indexes (#422).
DifferentialOrange added a commit that referenced this issue May 20, 2024
Some read cases were missing in #404, #406 and #408 test stabilization
commits. This patch updates missing read cases.

1. https://github.com/tarantool/crud/actions/runs/7322196516/job/19943422319

Follows #404
Part of #412
Part of #415
DifferentialOrange added a commit that referenced this issue May 20, 2024
Index alias provide stable info on instance nature, yet servers list
index doesn't.

Part of #412
Part of #415
DifferentialOrange added a commit that referenced this issue May 20, 2024
Before this patch, most tests used an assumption that main server is
a router. Yet these are two different concepts.

`main_server` entity was introduced in Cartridge test helpers Cluster
as some server you may refer to for maintenance work, like clusterwide
configuration update. Any cluster server is fine to do such operation:
router or not. Since most cluster configurations start with router
definition, it is often a first server in a list and is chosen to be
main_server as well. But no one guarantees that main_server provides
router API (both for vshard and crud).

Part of #412
Part of #415
DifferentialOrange added a commit that referenced this issue May 20, 2024
Tarantool 3.0 instances start in ro mode. It is possible that masters
are not available for write requests right after test case has been
started.

Cartridge servers run only on Tarantool 1.x and 2.x, so they are
expected to work fine without additional changes: Cartridge helpers
checks are sophisticated enough.

Part of #412
Part of #415
DifferentialOrange added a commit that referenced this issue May 20, 2024
Wait until crud is ready in tests. It doesn't make a lot of sense now
since crud init calls are syncronized (even though some vshard opts can
be async and now we wait for them too), but will be crucial in case we
bootstrap asynchronously.

Part of #412
Part of #415
DifferentialOrange added a commit that referenced this issue May 20, 2024
Wait until replication is finished if test case is a read request from
replica.

Part of #412
Part of #415
DifferentialOrange added a commit that referenced this issue May 20, 2024
Tarantool 3.0 instances start in ro mode. It is possible that masters
are not available for write requests right after test case has been
started.

This patch supports only vshard tests for Tarantool 2.2+. After this
patch, schema init is async and we wait for schema to initialize in
tests.

Part of #412
Part of #415
DifferentialOrange added a commit that referenced this issue May 20, 2024
DifferentialOrange added a commit that referenced this issue May 20, 2024
This patch removes vinyl from test matrices. Since [1], vinyl is
considered experimental and we do not guarantee its work with modules.
Vinyl clusters like to misbehave in tests, often resulting in failing
tests in CI. To improve developer experience, this patch disables them.

This patch do not remove vinyl support since there is no vinyl-specific
code in crud, but we do not guarantee it anymore.

1. tarantool/doc#3523

Part of #412
Part of #415
DifferentialOrange added a commit that referenced this issue May 20, 2024
This is the preliminary refactoring before solving the original issue.

Part of #412
Part of #415
DifferentialOrange added a commit that referenced this issue May 20, 2024
DifferentialOrange added a commit that referenced this issue May 20, 2024
DifferentialOrange added a commit that referenced this issue May 20, 2024
Support asynchronous bootstrap for crud storages. The main motivation is
compatibility with Tarantool 3 instances, which start in read-only mode.

We do not support async bootstrap for crud routers since they do
not change `box`, thus may start on read-only instances without any
issues.

Part of #412
Part of #415
DifferentialOrange added a commit that referenced this issue May 20, 2024
Overview

  This release introduces roles for Tarantool 3 configuration (supported
  for versions 3.0.2, 3.1.0 and newer).

Added
* Asynchronous bootstrap support for storages (#412).
* Tarantool 3 roles for setting up crud routers and storages (#415).
* Ability to configure crud through Tarantool 3 roles
  configuration (#415).

Changed
* Explicitly forbid datetime interval conditions (#373).
* Storage initialization is now asynchronous by default for
  Tarantool 3.0+ (#412).
* Additionally check backoff error on storage info fetch (#427).

Fixed
* Working with datetime conditions in case of non-indexed fields or
  non-iterating indexes (#373).
* Precision loss for decimal conditions in case of non-indexed fields or
  non-iterating indexes (#373).
* Passing errors from storages for merger operations (`crud.select`,
  `crud.pairs`, `readview:select`, `readview:pairs`) (#423).
* Working with `nil` operand conditions in case of non-indexed fields or
  non-iterating indexes (#422).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants