Skip to content

Commit

Permalink
details modified (#623)
Browse files Browse the repository at this point in the history
* details modified

* Update 4.nebula-graph-crud.md

* Update 3.connect-to-nebula-graph.md

* Update 4.nebula-graph-crud.md
  • Loading branch information
izhuxiaoqing committed Aug 27, 2021
1 parent 4b2a6dc commit 7f82284
Show file tree
Hide file tree
Showing 7 changed files with 206 additions and 121 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -82,11 +82,11 @@ In the `nebula-graphd.conf` file, when `enable_optimizer` is set to be `true`, P

RBO is a “bottom-up” exploration process. For each rule, the root node of the execution plan (in this case, the `Project` node) is the entry point, and step by step along with the node dependencies, it reaches the node at the bottom to see if it matches the rule.

As shown in the preceding figure, when the `Filter` node is explored, it is found that its children node is `GetNeighbors`, which matches successfully with the pre-defined rules, so a transformation is initiated to integrate the `Filter` node into the `GetNeighbors` node, the `Filter` node is removed, and then the process continues to the next rule. Therefore, when the `GetNeighbor` operator calls interfaces of the Storage layer to get the neighboring edges of a vertex during the execution stage, the Storage layer will directly filter out the unqualified edges internally. Such optimization greatly reduces the amount of data transfer, which is commonly known as filter push down.
As shown in the preceding figure, when the `Filter` node is explored, it is found that its children node is `GetNeighbors`, which matches successfully with the pre-defined rules, so a transformation is initiated to integrate the `Filter` node into the `GetNeighbors` node, the `Filter` node is removed, and then the process continues to the next rule. Therefore, when the `GetNeighbor` operator calls interfaces of the Storage layer to get the neighboring edges of a vertex during the execution stage, the Storage layer will directly filter out the unqualified edges internally. Such optimization greatly reduces the amount of data transfer, which is commonly known as filter pushdown.

!!! Note

Nebula Graph {{ nebula.release }} will not run optimization at default settings.
Nebula Graph {{ nebula.release }} will not run optimization by default.

## Executor

Expand All @@ -98,7 +98,7 @@ Each node of the execution plan has one execution operator node, whose input and

## Source code hierarchy

The source code hierarchy under the nebula-graph repository is as follows
The source code hierarchy under the nebula-graph repository is as follows.

```bash
|--src
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

The persistent data of Nebula Graph have two parts. One is the [Meta Service](2.meta-service.md) that stores the meta-related data.

The other is Storage Service that stores the data, which is run by the nebula-storaged process. This topic will describe the architecture of Storage Service.
The other is the Storage Service that stores the data, which is run by the nebula-storaged process. This topic will describe the architecture of Storage Service.

## Advantages

Expand Down Expand Up @@ -30,15 +30,15 @@ All the nebula-storaged processes consist of a Raft-based cluster. There are thr

- `getNeighbors`: query the in-edge or out-edge of a set of vertices, return the edges and the corresponding properties, and support conditional filtering.

- `insert vertex/edge`insert a vertex or edge and its properties.
- `insert vertex/edge`: insert a vertex or edge and its properties.

- `getProps`get the properties of a vertex or an edge.
- `getProps`: get the properties of a vertex or an edge.

It is this layer that makes the Storage Service a real graph storage. Otherwise, it is just a KV storage.

- Consensus

Below the storage interface is the consensus layer that implements [Multi Group Raft](#multi_group_raft), which ensures the strong consistency and high availability of Storage Service.
Below the storage interface is the consensus layer that implements [Multi Group Raft](#multi_group_raft), which ensures the strong consistency and high availability of the Storage Service.

- Store engine

Expand All @@ -48,15 +48,15 @@ The following will describe some features of Storage Service based on the above

## KVStore

Nebula Graph develops and customizes its built-in KVStore because:
Nebula Graph develops and customizes its built-in KVStore for the following reasons.

- It is a high-performance KVStore.

- It is provided as a (kv) library and can be easily developed for the filtering-pushdown purpose. As a strong-typed database, how to provide Schema during pushdown is the key to efficiency for Nebula Graph.

- It has strong data consistency.

Therefore, Nebula Graph develops its own KVStore with RocksDB as the local storage engine. The advantages are as follows:
Therefore, Nebula Graph develops its own KVStore with RocksDB as the local storage engine. The advantages are as follows.

- For multiple local hard disks, Nebula Graph can make full use of its concurrent capacities through deploying multiple data directories.

Expand All @@ -76,15 +76,15 @@ Nebula Graph stores vertices and edges. Efficient property filtering is critical

Nebula Graph {{ nebula.base20 }} has changed a lot over its releases. The following will introduce the old and new data storage formats and cover their differences.

- Vertex key format
- Vertex format

![The vertex format of storage service](https://docs-cdn.nebula-graph.com.cn/docs-2.0/1.introduction/2.nebula-graph-architecture/storage-vertex-format.png)

|Field|Description|
|:---|:---|
|`Type`|One byte, used to indicate the key type.|
|`PartID`|Three bytes, used to indicate the sharding partition and to scan the partition data based on the prefix when re-balancing the partition.|
|`VertexID`|Used to indicate vertex ID. For an integer VertexID, it occupies eight bytes. However, for a string VertexID, it is changed to `fixed_string` of a fixed length which needs to be specified by users when they create space.|
|`VertexID`|Used to indicate vertex ID. For an integer VertexID, it occupies eight bytes. However, for a string VertexID, it is changed to `fixed_string` of a fixed length which needs to be specified by users when they create the space.|
|`TagID`|Four bytes, used to indicate the tags that vertex relate with.|

- Edge Format
Expand Down Expand Up @@ -244,7 +244,7 @@ When a transfer leadership command is committed, the leader will abandon its lea

### Peer changes

To avoid split-brain, when members in a Raft Group change, an intermediate state is required. In such a state, the quorum of the old group and new group always have an overlap. Thus it prevents the old or new group from making decisions unilaterally. To make it even simpler, in his doctoral thesis Diego Ongaro suggests adding or removing a peer once to ensure the overlap between the quorum of the new group and the old group. Nebula Graph‘s implementation also uses this approach, except that the way to add or remove a member is different. For details, please refer to addPeer/removePeer in the Raft Part class.
To avoid split-brain, when members in a Raft Group change, an intermediate state is required. In such a state, the quorum of the old group and new group always have an overlap. Thus it prevents the old or new group from making decisions unilaterally. To make it even simpler, in his doctoral thesis Diego Ongaro suggests adding or removing a peer once to ensure the overlap between the quorum of the new group and the old group. Nebula Graph also uses this approach, except that the way to add or remove a member is different. For details, please refer to addPeer/removePeer in the Raft Part class.

## Differences with HDFS

Expand Down
147 changes: 120 additions & 27 deletions docs-2.0/2.quick-start/3.connect-to-nebula-graph.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,17 +6,17 @@ Nebula Graph supports multiple types of clients, including a CLI client, a GUI c

You can use supported [clients or console](../20.appendix/6.eco-tool-version.md) to connect to Nebula Graph database.

<!-- TODO cloud service can't be provided together with v2.0.0.
If you don't have a Nebula Graph database yet, we recommend that you try the cloud service. [Nebula Graph Cloud Service](https://www.nebula-cloud.io/) supports on-demand deployment and fast building, and uses Nebula Graph Studio as its default client.
<!-- TODO cloud service cannot be provided together with v{{ nebula.release }}.
If you do not have a Nebula Graph database yet, we recommend that you try the cloud service. [Nebula Graph Cloud Service](https://www.nebula-cloud.io/) supports on-demand deployment and fast building, and uses Nebula Graph Studio as its default client.
-->

## Use Nebula Console to connect to Nebula Graph

### Prerequisites

* You have started the Nebula Graph services. For how to start the services, see [Start and Stop Nebula Graph](./5.start-stop-service.md).
* Nebula Graph services are started. For how to start services, see [Start and Stop Nebula Graph](./5.start-stop-service.md).

* The machine you plan to run Nebula Console on has network access to the Nebula Graph services.
* The machines that run Nebula Console and Nebula Graph services have network access to each other.

### Steps

Expand All @@ -26,11 +26,11 @@ If you don't have a Nebula Graph database yet, we recommend that you try the clo

We recommend that you select the **latest** release.

![Select a Nebula Graph version and click **Assets**](https://docs-cdn.nebula-graph.com.cn/docs-2.0/2.quick-start/nebula-console-releases-1.png "Click Assets to show the available Nebula Graph binary files")
![Select a Nebula Graph version and click **Assets**](../2.quick-start/console.png "Click Assets to show the available Nebula Graph binary files")

2. In the **Assets** area, find the correct binary file for the machine where you want to run Nebula Console and download the file to the machine.

![Click to download the package according to your hardware architecture](https://docs-cdn.nebula-graph.com.cn/docs-2.0/2.quick-start/nebula-console-releases-2-1.png "Click the package name to download it")
![Click to download the package according to your hardware architecture](../2.quick-start/assets.png "Click the package name to download it")

3. (Optional) Rename the binary file to `nebula-console` for convenience.

Expand All @@ -52,21 +52,21 @@ If you don't have a Nebula Graph database yet, we recommend that you try the clo

6. Run the following command to connect to Nebula Graph.

* For Linux or macOS:
* For Linux or macOS:

```bash
$ ./nebula-console -addr <ip> -port <port> -u <username> -p <password>
[-t 120] [-e "nGQL_statement" | -f filename.nGQL]
```
```bash
$ ./nebula-console -addr <ip> -port <port> -u <username> -p <password>
[-t 120] [-e "nGQL_statement" | -f filename.nGQL]
```

* For Windows:
* For Windows:

```powershell
> nebula-console.exe -addr <ip> -port <port> -u <username> -p <password>
[-t 120] [-e "nGQL_statement" | -f filename.nGQL]
```
```powershell
> nebula-console.exe -addr <ip> -port <port> -u <username> -p <password>
[-t 120] [-e "nGQL_statement" | -f filename.nGQL]
```

The description of the parameters is as follows.
The descriptions are as follows.

| Parameter | Description |
| - | - |
Expand All @@ -75,36 +75,129 @@ If you don't have a Nebula Graph database yet, we recommend that you try the clo
| `-port` | Sets the port number of the graphd service. The default port number is 9669.<!-- If you have deployed Nebula Graph in a docker container but Nebula Console is working outside the container, check the [source port](2.deploy-nebula-graph-with-docker-compose.md/#check_the_nebula_graph_service_status_and_port) of any nebula-graphd process and use it for connection. --> |
| `-u/-user` | Sets the username of your Nebula Graph account. Before enabling authentication, you can use any existing username (The default username is `root`). |
| `-p/-password` | Sets the password of your Nebula Graph account. Before enabling authentication, you can use any characters as the password. |
| `-t/-timeout` | Sets an integer-type timeout threshold of the connection. The unit is second. The default value is 120. |
| `-t/-timeout` | Sets an integer-type timeout threshold of the connection. The unit is second. The default value is 120. |
| `-e/-eval` | Sets a string-type nGQL statement. The nGQL statement will be executed once the connection succeeds. The connection stops after the result is returned. |
| `-f/-file` | Sets the path of an nGQL file. The nGQL statements in the file are executed once the connection succeeds. The connection stops after the result is returned. |

Users can use the command of `./nebula-console --help` to get the details of all the parameters and can also find more details in the [Nebula Console Repository](https://github.com/vesoft-inc/nebula-console/tree/v2.0.0-ga).

## Nebula Console export mode
## Nebula Console commands

When the export mode is enabled, Nebula Console will export all the query results into a CSV file. When the export mode is disabled, the export stops. The syntax is as follows.
Nebula Console supports some commands. Users can export all the query results into a CSV file or a DOT file, and import test datasets.

!!! Note

* The following commands are case insensitive.
* The CSV file is stored in the working directory. Run the Linux command `pwd` to show the working directory.
* The commands are case-insensitive.

### Export CSV files

!!! note

- The CSV file is stored in the working directory. Run the Linux command `pwd` to show the working directory.

- This command only takes effect on the next query statement.

The command is as follows.

```ngql
nebula> :CSV <file_name.csv>
```

### Export DOT files

!!! Note

- The DOT file is stored in the working directory. Run the Linux command `pwd` to show the working directory.

- The content of the DOT file can be copied and pasted in the [GraphvizOnline](https://dreampuf.github.io/GraphvizOnline/) to generate a visual execution plan diagram.

- This command only takes effect on the next query statement.

The command is as follows.

```ngql
nebula> :dot <file_name.dot>
```

The example is as follows.

```ngql
nebula> :dot a.dot
nebula> PROFILE FORMAT="dot" GO FROM "player100" OVER follow;
```

### Load test datasets

The name of the test dataset is nba. To check the detailed information on the Schema and data, run the relevant `SHOW` command.

* Enable Nebula Console export mode:
The command is as follows.

```ngql
nebula> :SET CSV <your_file.csv>
nebula> :play nba
```

* Disable Nebula Console export mode:
### Repeat the execution

This command repeats the next command N times, and then prints the average execution time. The command is as follows.

```ngql
nebula> :UNSET CSV
nebula> :repeat N
```

Examples are as follows.

```ngql
nebula> :repeat 3
nebula> GO FROM "player100" OVER follow;
+-------------+
| follow._dst |
+-------------+
| "player101" |
+-------------+
| "player125" |
+-------------+
Got 2 rows (time spent 2602/3214 us)
Fri, 20 Aug 2021 06:36:05 UTC
+-------------+
| follow._dst |
+-------------+
| "player101" |
+-------------+
| "player125" |
+-------------+
Got 2 rows (time spent 583/849 us)
Fri, 20 Aug 2021 06:36:05 UTC
+-------------+
| follow._dst |
+-------------+
| "player101" |
+-------------+
| "player125" |
+-------------+
Got 2 rows (time spent 496/671 us)
Fri, 20 Aug 2021 06:36:05 UTC
Executed 3 times, (total time spent 3681/4734 us), (average time spent 1227/1578 us)
```

### Sleep

The command sleeps for N seconds. It is often used to modify the Schema because the modification of the Schema is implemented asynchronously, and Nebula Graph synchronizes the data in the next heartbeat cycle. The command is as follows:

```ngql
nebula> :sleep N
```

## Disconnect Nebula Console from Nebula Graph

You can use `:EXIT` or `:QUIT` to disconnect from Nebula Graph. For convenience, Nebula Console supports using these commands in lower case without the colon, such as `quit`.
Users can use `:EXIT` or `:QUIT` to disconnect from Nebula Graph. For convenience, Nebula Console supports using these commands in lower case without the colon, such as `quit`.

The example is as follows.

```ngql
nebula> :QUIT
Expand Down
Loading

0 comments on commit 7f82284

Please sign in to comment.