New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Failed to select with datetime conditions #373

Closed

DifferentialOrange opened this issue Sep 20, 2023 · 0 comments · Fixed by #423

Assignees

Labels

Member

DifferentialOrange commented Sep 20, 2023

Simple reproducer is as follows.

Extend playground space format.

--- a/doc/playground.lua
+++ b/doc/playground.lua
@@ -66,6 +66,7 @@ box.once('customers', function()
             {name = 'bucket_id', type = 'unsigned'},
             {name = 'name', type = 'string'},
             {name = 'age', type = 'number'},
+            {name = 'date', type = 'datetime', is_nullable = true},
         }
     })
     box.space.customers:create_index('primary_index', {

Run the updated playground.

./doc/playground.lua

Try to select with datetime condition

tarantool> dt = require('datetime').parse('2022-01-01T00:00:00Z')
---
...

tarantool> crud.select('customers', {{'==', 'date', dt}}, {first = 1})
---
- null
- line: 183
  class_name: SelectError
  err: '...velopment/github/tarantool/crud/crud/compare/filters.lua:636: [string "library"]:4:
    malformed number near ''01T00'''
  file: '...ment/github/tarantool/crud/crud/common/sharding/init.lua'
  stack: "stack traceback:\n\t...ment/github/tarantool/crud/crud/common/sharding/init.lua:183:
    in function <...ment/github/tarantool/crud/crud/common/sharding/init.lua:168>\n\t[C]:
    in function 'xpcall'\n\t.../github/tarantool/crud/.rocks/share/tarantool/errors.lua:145:
    in function <.../github/tarantool/crud/.rocks/share/tarantool/errors.lua:139>\n\t[C]:
    in function 'pcall'\n\tbuiltin/box/console.lua:415: in function 'eval'\n\tbuiltin/box/console.lua:750:
    in function 'repl'\n\tbuiltin/box/console.lua:801: in function 'start'\n\t./doc/playground.lua:150:
    in main chunk"
  str: 'SelectError: ...velopment/github/tarantool/crud/crud/compare/filters.lua:636:
    [string "library"]:4: malformed number near ''01T00'''
...

As a follow up, we also may verify other extended types like decimal, uuid and interval.

DifferentialOrange added bug 2sp labels

DifferentialOrange self-assigned this

LeonidVas added the teamE label

DifferentialOrange added a commit that referenced this issue


          scan: support datetime filters

f9064e4

Before this patch, datetime conditions were supported only in iterators,
but not filters. (Filter is any non-indexed condition or condition
for any non-iterating index. Any condition index expect for the first
one is non-iterating.) This patch introduce datetime comparison support
to filter codegen library, similar to uuid one.

Closes #373

DifferentialOrange mentioned this issue

scan: extention types filters #423

Merged

DifferentialOrange added a commit that referenced this issue


          scan: support datetime filters

fd8ce2d

Before this patch, datetime conditions were supported only in iterators,
but not filters. (Filter is any non-indexed condition or condition
for any non-iterating index. Any condition index expect for the first
one is non-iterating.) This patch introduce datetime comparison support
to filter codegen library, similar to uuid one.

Closes #373

DifferentialOrange added a commit that referenced this issue


          scan: support datetime filters

e861188

Before this patch, datetime conditions were supported only in iterators,
but not filters. (Filter is any non-indexed condition or condition
for any non-iterating index. Any condition index expect for the first
one is non-iterating.) This patch introduce datetime comparison support
to filter codegen library, similar to uuid one.

Closes #373

DifferentialOrange added a commit that referenced this issue


          scan: support datetime filters

8f3f849

Before this patch, datetime conditions were supported only in iterators,
but not filters. (Filter is any non-indexed condition or condition
for any non-iterating index. Any condition index expect for the first
one is non-iterating.) This patch introduce datetime comparison support
to filter codegen library, similar to uuid one.

Closes #373

DifferentialOrange added a commit that referenced this issue


          scan: support datetime filters

3535de5

Before this patch, datetime conditions were supported only in iterators,
but not filters. (Filter is any non-indexed condition or condition
for any non-iterating index. Any condition index expect for the first
one is non-iterating.) This patch introduce datetime comparison support
to filter codegen library, similar to uuid one.

Part of #373

DifferentialOrange added a commit that referenced this issue


          test: check decimal filters

d0d6ce7

Module uses cdata string cast for decimal filters. Later they are
compared to decimal objects and casted back, so everything already works
fine.

Part of #373

DifferentialOrange added a commit that referenced this issue


          scan: support datetime filters

9224e0f

Before this patch, datetime conditions were supported only in iterators,
but not filters. (Filter is any non-indexed condition or condition
for any non-iterating index. Any condition index expect for the first
one is non-iterating.) This patch introduce datetime comparison support
to filter codegen library, similar to uuid one.

Part of #373

DifferentialOrange added a commit that referenced this issue


          scan: properly support decimal filters

4438b0b

Before this patch, decimals were cast to numbers in filters through
`tostring` in codegen. This patch fixes this precision loss.

Part of #373

DifferentialOrange added a commit that referenced this issue


          scan: support datetime filters

258bb09

Before this patch, datetime conditions were supported only in iterators,
but not filters. (Filter is any non-indexed condition or condition
for any non-iterating index. Any condition index expect for the first
one is non-iterating.) This patch introduce datetime comparison support
to filter codegen library, similar to uuid one.

Part of #373

DifferentialOrange added a commit that referenced this issue


          scan: fix decimal filters precision loss

8cf201f

Before this patch, decimals were cast to numbers in filters through
`tostring` in codegen. This patch fixes this precision loss.

Part of #373

DifferentialOrange added a commit that referenced this issue


          crud: fix passing errors from storages for merger

074e78d

Before this patch, returning any errors from storages for merger
operations (`crud.select`, `crud.pairs`, `readview:select`,
`readview:pairs`) had resulted in non-informative
"Invalid merge source 0x556bb6885a00" errors. This patch fixes the issue
by replacing error return with error throw.

Part of #373

DifferentialOrange added a commit that referenced this issue


          scan: forbid interval conditions

2aa73df

As for Tarantool 3.0 and older, datetime intervals are
not comparable [1]. They are also don't supported in indexes. This patch
explicitly forbids to use them in conditions.
1. tarantool/tarantool#7659

Closes #373

DifferentialOrange added a commit that referenced this issue


          crud: fix passing errors from storages for merger

3fa4fbb

Before this patch, returning any errors from storages for merger
operations (`crud.select`, `crud.pairs`, `readview:select`,
`readview:pairs`) had resulted in non-informative
"Invalid merge source 0x556bb6885a00" errors. This patch fixes the issue
by replacing error return with error throw.

Part of #373

DifferentialOrange added a commit that referenced this issue


          scan: forbid interval conditions

01b5976

As for Tarantool 3.0 and older, datetime intervals are
not comparable [1]. They are also don't supported in indexes. This patch
explicitly forbids to use them in conditions.
1. tarantool/tarantool#7659

Closes #373

DifferentialOrange added a commit that referenced this issue


          scan: support datetime filters

c1a52eb

Before this patch, datetime conditions were supported only in iterators,
but not filters. (Any non-indexed condition or condition for any
non-iterating index is a filter. Any condition index except for
the first one is non-iterating.) This patch introduce datetime
comparison support to filter codegen library, similar to uuid one.

Part of #373

DifferentialOrange added a commit that referenced this issue


          scan: fix decimal filters precision loss

46bbfe4

Before this patch, decimals were cast to numbers in filters through
`tostring` in codegen. This patch fixes this precision loss.

Part of #373

DifferentialOrange added a commit that referenced this issue


          crud: fix passing errors from storages for merger

b4ca442

Before this patch, returning any errors from storages for merger
operations (`crud.select`, `crud.pairs`, `readview:select`,
`readview:pairs`) had resulted in non-informative
"Invalid merge source 0x556bb6885a00" errors. This patch fixes the issue
by replacing error return with error throw.

Part of #373

DifferentialOrange added a commit that referenced this issue


          scan: forbid interval conditions

c0a7295

As for Tarantool 3.0 and older, datetime intervals are
not comparable [1]. They are also don't supported in indexes. This patch
explicitly forbids to use them in conditions.
1. tarantool/tarantool#7659

Closes #373

DifferentialOrange added a commit that referenced this issue


          scan: support datetime filters

dd95975

Before this patch, datetime conditions were supported only in iterators,
but not filters. (Any non-indexed condition or condition for any
non-iterating index is a filter. Any condition index except for
the first one is non-iterating.) This patch introduce datetime
comparison support to filter codegen library, similar to uuid one.

Part of #373

DifferentialOrange added a commit that referenced this issue


          scan: fix decimal filters precision loss

9c7ceb2

Before this patch, decimals were cast to numbers in filters through
`tostring` in codegen. This patch fixes this precision loss.

Part of #373

DifferentialOrange added a commit that referenced this issue


          crud: fix passing errors from storages for merger

7449ade

Before this patch, returning any errors from storages for merger
operations (`crud.select`, `crud.pairs`, `readview:select`,
`readview:pairs`) had resulted in non-informative
"Invalid merge source 0x556bb6885a00" errors. This patch fixes the issue
by replacing error return with error throw.

Part of #373

DifferentialOrange added a commit that referenced this issue


          scan: forbid interval conditions

771904f

As for Tarantool 3.0 and older, datetime intervals are
not comparable [1]. They are also don't supported in indexes. This patch
explicitly forbids to use them in conditions.
1. tarantool/tarantool#7659

Closes #373

DifferentialOrange closed this as completed in #423

DifferentialOrange added a commit that referenced this issue


          scan: support datetime filters

bc43a8f

Before this patch, datetime conditions were supported only in iterators,
but not filters. (Any non-indexed condition or condition for any
non-iterating index is a filter. Any condition index except for
the first one is non-iterating.) This patch introduce datetime
comparison support to filter codegen library, similar to uuid one.

Part of #373

DifferentialOrange added a commit that referenced this issue


          scan: fix decimal filters precision loss

dedf184

Before this patch, decimals were cast to numbers in filters through
`tostring` in codegen. This patch fixes this precision loss.

Part of #373

DifferentialOrange added a commit that referenced this issue


          crud: fix passing errors from storages for merger

1f3b28f

Before this patch, returning any errors from storages for merger
operations (`crud.select`, `crud.pairs`, `readview:select`,
`readview:pairs`) had resulted in non-informative
"Invalid merge source 0x556bb6885a00" errors. This patch fixes the issue
by replacing error return with error throw.

Part of #373

DifferentialOrange added a commit that referenced this issue


          scan: forbid interval conditions

f790d54

As for Tarantool 3.0 and older, datetime intervals are
not comparable [1]. They are also don't supported in indexes. This patch
explicitly forbids to use them in conditions.
1. tarantool/tarantool#7659

Closes #373

DifferentialOrange added a commit that referenced this issue


          release 1.5.0

9aa9b02

Overview

  This release introduces roles for Tarantool 3 configuration (supported
  for versions 3.0.2, 3.1.0 and newer).

Added
* Asynchronous bootstrap support for storages (#412).
* Tarantool 3 roles for setting up crud routers and storages (#415).
* Ability to configure crud through Tarantool 3 roles
  configuration (#415).

Changed
* Explicitly forbid datetime interval conditions (#373).
* Storage initialization is now asynchronous by default for
  Tarantool 3.0+ (#412).
* Additionally check backoff error on storage info fetch (#427).

Fixed
* Working with datetime conditions in case of non-indexed fields or
  non-iterating indexes (#373).
* Precision loss for decimal conditions in case of non-indexed fields or
  non-iterating indexes (#373).
* Passing errors from storages for merger operations (`crud.select`,
  `crud.pairs`, `readview:select`, `readview:pairs`) (#423).
* Working with `nil` operand conditions in case of non-indexed fields or
  non-iterating indexes (#422).

DifferentialOrange mentioned this issue

release 1.5.0 #428

Merged

DifferentialOrange added a commit that referenced this issue


          release 1.5.0

99315a5

Overview

  This release introduces roles for Tarantool 3 configuration (supported
  for versions 3.0.2, 3.1.0 and newer).

Added
* Asynchronous bootstrap support for storages (#412).
* Tarantool 3 roles for setting up crud routers and storages (#415).
* Ability to configure crud through Tarantool 3 roles
  configuration (#415).

Changed
* Explicitly forbid datetime interval conditions (#373).
* Storage initialization is now asynchronous by default for
  Tarantool 3.0+ (#412).
* Additionally check backoff error on storage info fetch (#427).

Fixed
* Working with datetime conditions in case of non-indexed fields or
  non-iterating indexes (#373).
* Precision loss for decimal conditions in case of non-indexed fields or
  non-iterating indexes (#373).
* Passing errors from storages for merger operations (`crud.select`,
  `crud.pairs`, `readview:select`, `readview:pairs`) (#423).
* Working with `nil` operand conditions in case of non-indexed fields or
  non-iterating indexes (#422).

DifferentialOrange added a commit that referenced this issue


          release 1.5.0

6cb4f4a

Overview

  This release introduces roles for Tarantool 3 configuration (supported
  for versions 3.0.2, 3.1.0 and newer).

Added
* Asynchronous bootstrap support for storages (#412).
* Tarantool 3 roles for setting up crud routers and storages (#415).
* Ability to configure crud through Tarantool 3 roles
  configuration (#415).

Changed
* Explicitly forbid datetime interval conditions (#373).
* Storage initialization is now asynchronous by default for
  Tarantool 3.0+ (#412).
* Additionally check backoff error on storage info fetch (#427).

Fixed
* Working with datetime conditions in case of non-indexed fields or
  non-iterating indexes (#373).
* Precision loss for decimal conditions in case of non-indexed fields or
  non-iterating indexes (#373).
* Passing errors from storages for merger operations (`crud.select`,
  `crud.pairs`, `readview:select`, `readview:pairs`) (#423).
* Working with `nil` operand conditions in case of non-indexed fields or
  non-iterating indexes (#422).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment