Multi-line / New Lines in NGQL statements cause syntax errors

[bradenwright-opunai]

Describe the bug (required)

New lines in a NGQL statement seem to break it.

Your Environments (required)

• OS: uname -a: Darwin bradens-MacBook-Pro 22.5.0 Darwin Kernel Version 22.5.0: Mon Apr 24 20:53:19 PDT 2023; root:xnu-8796.121.2~5/RELEASE_ARM64_T6020 arm64

How To Reproduce(required)

Add new lines to a statement and run them, for example this file won't work:

braden@bradens-MacBook-Pro nebula-console % cat test.nqql 
CREATE SPACE IF NOT EXISTS `dev` (
  partition_num = 60,
  replica_factor = 1,
   vid_type = FIXED_STRING(70)
);

but this will:

braden@bradens-MacBook-Pro nebula-console % cat test1.ngql 
CREATE SPACE IF NOT EXISTS `dev` ( partition_num = 60, replica_factor = 1, vid_type = FIXED_STRING(70));

Steps to reproduce the behavior:

1. create test.ngql with new lines, for example:

braden@bradens-MacBook-Pro nebula-console % cat test.nqql 
CREATE SPACE IF NOT EXISTS `dev` (
  partition_num = 60,
  replica_factor = 1,
   vid_type = FIXED_STRING(70)
);

2. Run that ngql statement:

braden@bradens-MacBook-Pro nebula-console % ./nebula-console -addr=127.0.0.1 -port 9669 -u root -p $NEBULA_PASSWORD -f test.ngql 
(root@nebula) [(none)]> CREATE SPACE IF NOT EXISTS `dev` (
[ERROR (-1004)]: SyntaxError: syntax error near `('

Fri, 04 Aug 2023 11:39:13 MDT

(root@nebula) [(none)]>   partition_num = 60,
[ERROR (-1004)]: SyntaxError: syntax error near `partition_num'

Fri, 04 Aug 2023 11:39:13 MDT

(root@nebula) [(none)]>   replica_factor = 1,
[ERROR (-1004)]: SyntaxError: syntax error near `replica_factor'

Fri, 04 Aug 2023 11:39:13 MDT

(root@nebula) [(none)]>    vid_type = FIXED_STRING(70)
[ERROR (-1004)]: SyntaxError: syntax error near `vid_type'

Fri, 04 Aug 2023 11:39:13 MDT

(root@nebula) [(none)]> );
[ERROR (-1004)]: SyntaxError: syntax error near `);'

Fri, 04 Aug 2023 11:39:13 MDT

(root@nebula) [(none)]> 


Bye root!
Fri, 04 Aug 2023 11:39:13 MDT

3. Remove the newlines from the statement:

braden@bradens-MacBook-Pro nebula-console % cat test1.ngql
CREATE SPACE IF NOT EXISTS `dev` ( partition_num = 60, replica_factor = 1, vid_type = FIXED_STRING(70));

4. Run test1.ngql statement:

braden@bradens-MacBook-Pro nebula-console % ./nebula-console -addr=127.0.0.1 -port 9669 -u root -p $NEBULA_PASSWORD -f test1.ngql
(root@nebula) [(none)]> CREATE SPACE IF NOT EXISTS `dev` ( partition_num = 60, replica_factor = 1, vid_type = FIXED_STRING(70));
Execution succeeded (time spent 997µs/35.339834ms)

Fri, 04 Aug 2023 11:41:44 MDT

(root@nebula) [(none)]> 


Bye root!
Fri, 04 Aug 2023 11:41:44 MDT

Expected behavior
I'd expect ; to end a statement and white space to be allowed.

Additional context

It becomes an issue b/c our developers create nebula statements in Nebula Studio and then we automate what they create but the statements in Nebula Studio work with newlines, so it creates errors when we try to automate their statements. Plus we want the ngql statements to be organized / more readable.

[bradenwright-opunai]

As a workaround currently I'm doing some process on our files and passing that into nebula-console, but its def a little hacky and more prone to bugs:

./nebula-console -addr=127.0.0.1 -port 9669 -u root -p $CURRENT_NEBULA_PASSWORD -f <(cat prod-setup.sql | sed -e 's/^[ \t]*//' | sed '/^--/ d' | sed '/\/\*.*\*\// d' | sed '/\/\*/,/\*\// d' | tr '\n' ' ' | sed -E 's/;\ /;\n/g' | sed -e 's/^[ \t]*//' | envsubst)

Im using envsubst to replace sensitive variables in the sql

[wey-gu]

Now, my way to mitigate this is via """, after typing """, we could paste multiple-line queries.

@veezhang what do you think, please about this, could we find a way to make it multiple lines aware yet not cause new problems?

@bradenwright-opunai also, it's more than welcome to try make the changes(and PR) and see if there are any regressions!

Thanks!

[bradenwright-opunai]

Thanks for the reply, I'll see what veezhang says, depending on the code/complexity that maybe something I can do. I'm working on it as a consultant, so I can spend a little time on my own but if it gets too involved I'll have to get approval from the team I'm doing the work for.

[veezhang]

@bradenwright-opunai The @wey-gu ’s method is very good! Additionally, you can use \ to support multiline nGQL.

"""
CREATE SPACE IF NOT EXISTS `dev` (
  partition_num = 60,
  replica_factor = 1,
   vid_type = FIXED_STRING(70)
);
"""


CREATE SPACE IF NOT EXISTS `dev` ( \
  partition_num = 60, \
  replica_factor = 1, \
   vid_type = FIXED_STRING(70) \
);

[QingZ11]

Hi, I have noticed that the issue you created hasn’t been updated for nearly a month, so I have to close it for now. If you have any new updates, you are welcome to reopen this issue anytime.

Thanks a lot for your contribution anyway 😊

这个 issue 超过一个月没有更新内容了，这边先行关闭了。如果你有任何更新的内容，可以再打开这个 issue 哈。