unhealthy when use docker compose with v2.6.0 in M1 mac

[awsl-dbq]

(base) ➜  nebula-docker-compose git:(v2.6.0) docker-compose -f docker-compose.yaml up
Docker Compose is now in the Docker CLI, try `docker compose up`

Creating network "nebula-docker-compose_nebula-net" with the default driver
Creating nebula-docker-compose_metad1_1 ... done
Creating nebula-docker-compose_metad2_1 ... done
Creating nebula-docker-compose_metad0_1 ... done
Creating nebula-docker-compose_storaged2_1 ... done
Creating nebula-docker-compose_storaged0_1 ... done
Creating nebula-docker-compose_storaged1_1 ... done
Creating nebula-docker-compose_graphd1_1   ... done
Creating nebula-docker-compose_graphd_1    ... done
Creating nebula-docker-compose_graphd2_1   ... done
Attaching to nebula-docker-compose_metad1_1, nebula-docker-compose_metad0_1, nebula-docker-compose_metad2_1, nebula-docker-compose_storaged2_1, nebula-docker-compose_storaged1_1, nebula-docker-compose_storaged0_1, nebula-docker-compose_graphd1_1, nebula-docker-compose_graphd2_1, nebula-docker-compose_graphd_1

docker compose ps

(base) ➜  nebula-docker-compose git:(v2.6.0) docker-compose -f docker-compose.yaml ps
              Name                             Command                   State                                                          Ports                                                  
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
nebula-docker-compose_graphd1_1     /usr/local/nebula/bin/nebu ...   Up (healthy)     0.0.0.0:61624->19669/tcp, 0.0.0.0:61625->19670/tcp, 0.0.0.0:61626->9669/tcp                              
nebula-docker-compose_graphd2_1     /usr/local/nebula/bin/nebu ...   Up (healthy)     0.0.0.0:61641->19669/tcp, 0.0.0.0:61642->19670/tcp, 0.0.0.0:61643->9669/tcp                              
nebula-docker-compose_graphd_1      /usr/local/nebula/bin/nebu ...   Up (healthy)     0.0.0.0:61647->19669/tcp, 0.0.0.0:61648->19670/tcp, 0.0.0.0:9669->9669/tcp,:::9669->9669/tcp             
nebula-docker-compose_metad0_1      /usr/local/nebula/bin/nebu ...   Up (unhealthy)   0.0.0.0:61543->19559/tcp, 0.0.0.0:61544->19560/tcp, 0.0.0.0:61542->9559/tcp, 9560/tcp                    
nebula-docker-compose_metad1_1      /usr/local/nebula/bin/nebu ...   Up (unhealthy)   0.0.0.0:61534->19559/tcp, 0.0.0.0:61535->19560/tcp, 0.0.0.0:61533->9559/tcp, 9560/tcp                    
nebula-docker-compose_metad2_1      /usr/local/nebula/bin/nebu ...   Up (unhealthy)   0.0.0.0:61551->19559/tcp, 0.0.0.0:61552->19560/tcp, 0.0.0.0:61553->9559/tcp, 9560/tcp                    
nebula-docker-compose_storaged0_1   /usr/local/nebula/bin/nebu ...   Up (unhealthy)   0.0.0.0:61602->19779/tcp, 0.0.0.0:61600->19780/tcp, 9777/tcp, 9778/tcp, 0.0.0.0:61601->9779/tcp, 9780/tcp
nebula-docker-compose_storaged1_1   /usr/local/nebula/bin/nebu ...   Up (unhealthy)   0.0.0.0:61591->19779/tcp, 0.0.0.0:61592->19780/tcp, 9777/tcp, 9778/tcp, 0.0.0.0:61593->9779/tcp, 9780/tcp
nebula-docker-compose_storaged2_1   /usr/local/nebula/bin/nebu ...   Up (unhealthy)   0.0.0.0:61582->19779/tcp, 0.0.0.0:61580->19780/tcp, 9777/tcp, 9778/tcp, 0.0.0.0:61581->9779/tcp, 9780/tcp

[wey-gu]

@awsl-dbq please checkout the v2.6 branch(we are updating the doc now) as 2.6.0/2.6.1 version is with issues on cgroupv2 and docker desktop's VM introduced cgroupv2 since v4.3.0.

Sorry for this.

docker-compose down
git checkout  v2.6 
docker-compose pull
docker-compose up -d

[awsl-dbq]

@wey-gu. sorry ,I still got unhealth with v2.6
And I find some error logs

*** Aborted at 1643349301 (Unix time, try 'date -d @1643349301') ***
/usr/local/nebula/bin/nebula-metad(_ZN5folly10symbolizer17getStackTraceSafeEPmm+0x31)[0x20ef2b1]
/usr/local/nebula/bin/nebula-metad(_ZN5folly10symbolizer21SafeStackTracePrinter15printStackTraceEb+0x1b)[0x20e711b]
/usr/local/nebula/bin/nebula-metad[0x20e5bf0]
/lib64/libpthread.so.0(+0xf62f)[0x400115b62f]
/usr/local/nebula/bin/nebula-metad[0x20e5167]
/lib64/libpthread.so.0(+0xf62f)[0x400115b62f]
/lib64/libpthread.so.0(__nanosleep+0x2d)[0x400115ae9d]
/usr/local/nebula/bin/nebula-metad(_ZN6nebula6raftex13RaftexService14waitUntilReadyEv+0x2a)[0x18eec1a]
/usr/local/nebula/bin/nebula-metad(_ZN6nebula6raftex13RaftexService5startEv+0x7e)[0x18ef0be]
/usr/local/nebula/bin/nebula-metad(_ZN6nebula7kvstore11NebulaStore4initEv+0x288)[0x18ab578]
/usr/local/nebula/bin/nebula-metad(_Z6initKVSt6vectorIN6nebula8HostAddrESaIS1_EES1_+0x86d)[0xf4abed]
/usr/local/nebula/bin/nebula-metad(main+0x79c)[0xf0e7fc]
/lib64/libc.so.6(__libc_start_main+0xf4)[0x400138a554]
/usr/local/nebula/bin/nebula-metad[0xf49a9d]

here is all stderr.log

logs git:(v2.6) ✗ cat */*stderr.log
I0128 05:55:01.508348     1 MetaDaemon.cpp:262] localhost = "metad0":9559
I0128 05:55:01.558621     1 NebulaStore.cpp:52] Start the raft service...
I0128 05:55:01.567404     1 NebulaSnapshotManager.cpp:25] Send snapshot is rate limited to 8388608 for each part by default
I0128 05:55:01.574983     1 RaftexService.cpp:62] Init thrift server for raft service, port: 9560
I0128 05:55:01.578619    55 RaftexService.cpp:93] Starting the Raftex Service
*** Aborted at 1643349301 (Unix time, try 'date -d @1643349301') ***
/usr/local/nebula/bin/nebula-metad(_ZN5folly10symbolizer17getStackTraceSafeEPmm+0x31)[0x20ef2b1]
/usr/local/nebula/bin/nebula-metad(_ZN5folly10symbolizer21SafeStackTracePrinter15printStackTraceEb+0x1b)[0x20e711b]
/usr/local/nebula/bin/nebula-metad[0x20e5bf0]
/lib64/libpthread.so.0(+0xf62f)[0x400115b62f]
/usr/local/nebula/bin/nebula-metad[0x20e5167]
/lib64/libpthread.so.0(+0xf62f)[0x400115b62f]
/lib64/libpthread.so.0(__nanosleep+0x2d)[0x400115ae9d]
/usr/local/nebula/bin/nebula-metad(_ZN6nebula6raftex13RaftexService14waitUntilReadyEv+0x2a)[0x18eec1a]
/usr/local/nebula/bin/nebula-metad(_ZN6nebula6raftex13RaftexService5startEv+0x7e)[0x18ef0be]
/usr/local/nebula/bin/nebula-metad(_ZN6nebula7kvstore11NebulaStore4initEv+0x288)[0x18ab578]
/usr/local/nebula/bin/nebula-metad(_Z6initKVSt6vectorIN6nebula8HostAddrESaIS1_EES1_+0x86d)[0xf4abed]
/usr/local/nebula/bin/nebula-metad(main+0x79c)[0xf0e7fc]
/lib64/libc.so.6(__libc_start_main+0xf4)[0x400138a554]
/usr/local/nebula/bin/nebula-metad[0xf49a9d]
*** Signal 15 (SIGTERM) (Entered fatal signal handler recursively. We're in trouble.
(safe mode, symbolizer not available)
qemu: uncaught target signal 11 (Segmentation fault) - core dumped
I0128 05:56:03.626678     1 MetaDaemon.cpp:262] localhost = "metad0":9559
I0128 05:56:03.685827     1 NebulaStore.cpp:52] Start the raft service...
I0128 05:56:03.694396     1 NebulaSnapshotManager.cpp:25] Send snapshot is rate limited to 8388608 for each part by default
I0128 05:56:03.702298     1 RaftexService.cpp:62] Init thrift server for raft service, port: 9560
I0128 05:56:03.705076    53 RaftexService.cpp:93] Starting the Raftex Service
I0128 05:55:06.843482     1 MetaDaemon.cpp:262] localhost = "metad1":9559
I0128 05:55:06.885730     1 NebulaStore.cpp:52] Start the raft service...
I0128 05:55:06.893378     1 NebulaSnapshotManager.cpp:25] Send snapshot is rate limited to 8388608 for each part by default
I0128 05:55:06.900025     1 RaftexService.cpp:62] Init thrift server for raft service, port: 9560
I0128 05:55:06.902967    54 RaftexService.cpp:93] Starting the Raftex Service
*** Aborted at 1643349319 (Unix time, try 'date -d @1643349319') ***
/usr/local/nebula/bin/nebula-metad(_ZN5folly10symbolizer17getStackTraceSafeEPmm+0x31)[0x20ef2b1]
/usr/local/nebula/bin/nebula-metad(_ZN5folly10symbolizer21SafeStackTracePrinter15printStackTraceEb+0x1b)[0x20e711b]
/usr/local/nebula/bin/nebula-metad[0x20e5bf0]
/lib64/libpthread.so.0(+0xf62f)[0x400115b62f]
/usr/local/nebula/bin/nebula-metad[0x20e5167]
/lib64/libpthread.so.0(+0xf62f)[0x400115b62f]
/lib64/libpthread.so.0(__nanosleep+0x2d)[0x400115ae9d]
/usr/local/nebula/bin/nebula-metad(_ZN6nebula6raftex13RaftexService14waitUntilReadyEv+0x2a)[0x18eec1a]
/usr/local/nebula/bin/nebula-metad(_ZN6nebula6raftex13RaftexService5startEv+0x7e)[0x18ef0be]
/usr/local/nebula/bin/nebula-metad(_ZN6nebula7kvstore11NebulaStore4initEv+0x288)[0x18ab578]
/usr/local/nebula/bin/nebula-metad(_Z6initKVSt6vectorIN6nebula8HostAddrESaIS1_EES1_+0x86d)[0xf4abed]
/usr/local/nebula/bin/nebula-metad(main+0x79c)[0xf0e7fc]
/lib64/libc.so.6(__libc_start_main+0xf4)[0x400138a554]
/usr/local/nebula/bin/nebula-metad[0xf49a9d]
*** Signal 15 (SIGTERM) (Entered fatal signal handler recursively. We're in trouble.
(safe mode, symbolizer not available)
qemu: uncaught target signal 11 (Segmentation fault) - core dumped
I0128 05:56:01.919839     1 MetaDaemon.cpp:262] localhost = "metad1":9559
I0128 05:56:01.964605     1 NebulaStore.cpp:52] Start the raft service...
I0128 05:56:01.973289     1 NebulaSnapshotManager.cpp:25] Send snapshot is rate limited to 8388608 for each part by default
I0128 05:56:01.980338     1 RaftexService.cpp:62] Init thrift server for raft service, port: 9560
I0128 05:56:01.983345    53 RaftexService.cpp:93] Starting the Raftex Service
I0128 05:55:05.488449     1 MetaDaemon.cpp:262] localhost = "metad2":9559
I0128 05:55:05.532888     1 NebulaStore.cpp:52] Start the raft service...
I0128 05:55:05.540961     1 NebulaSnapshotManager.cpp:25] Send snapshot is rate limited to 8388608 for each part by default
I0128 05:55:05.547499     1 RaftexService.cpp:62] Init thrift server for raft service, port: 9560
I0128 05:55:05.550427    54 RaftexService.cpp:93] Starting the Raftex Service
*** Aborted at 1643349319 (Unix time, try 'date -d @1643349319') ***
/usr/local/nebula/bin/nebula-metad(_ZN5folly10symbolizer17getStackTraceSafeEPmm+0x31)[0x20ef2b1]
/usr/local/nebula/bin/nebula-metad(_ZN5folly10symbolizer21SafeStackTracePrinter15printStackTraceEb+0x1b)[0x20e711b]
/usr/local/nebula/bin/nebula-metad[0x20e5bf0]
/lib64/libpthread.so.0(+0xf62f)[0x400115b62f]
/usr/local/nebula/bin/nebula-metad[0x20e5167]
/lib64/libpthread.so.0(+0xf62f)[0x400115b62f]
/lib64/libpthread.so.0(__nanosleep+0x2d)[0x400115ae9d]
/usr/local/nebula/bin/nebula-metad(_ZN6nebula6raftex13RaftexService14waitUntilReadyEv+0x2a)[0x18eec1a]
/usr/local/nebula/bin/nebula-metad(_ZN6nebula6raftex13RaftexService5startEv+0x7e)[0x18ef0be]
/usr/local/nebula/bin/nebula-metad(_ZN6nebula7kvstore11NebulaStore4initEv+0x288)[0x18ab578]
/usr/local/nebula/bin/nebula-metad(_Z6initKVSt6vectorIN6nebula8HostAddrESaIS1_EES1_+0x86d)[0xf4abed]
/usr/local/nebula/bin/nebula-metad(main+0x79c)[0xf0e7fc]
/lib64/libc.so.6(__libc_start_main+0xf4)[0x400138a554]
/usr/local/nebula/bin/nebula-metad[0xf49a9d]
*** Signal 15 (SIGTERM) (Entered fatal signal handler recursively. We're in trouble.
(safe mode, symbolizer not available)
qemu: uncaught target signal 11 (Segmentation fault) - core dumped
I0128 05:55:58.258803     1 MetaDaemon.cpp:262] localhost = "metad2":9559
I0128 05:55:58.303790     1 NebulaStore.cpp:52] Start the raft service...
I0128 05:55:58.312212     1 NebulaSnapshotManager.cpp:25] Send snapshot is rate limited to 8388608 for each part by default
I0128 05:55:58.319686     1 RaftexService.cpp:62] Init thrift server for raft service, port: 9560
I0128 05:55:58.322674    53 RaftexService.cpp:93] Starting the Raftex Service
I0128 05:56:16.053514     1 StorageDaemon.cpp:125] localhost = "storaged0":9779
I0128 05:56:16.104982     1 MetaClient.cpp:57] Create meta client to "metad2":9559
I0128 05:56:22.956266    48 ThriftClientManager-inl.h:66] resolve "metad1":9559 as "172.27.0.3":9559
I0128 05:56:31.367763    48 ThriftClientManager-inl.h:66] resolve "metad0":9559 as "172.27.0.4":9559
I0128 05:56:39.451017    48 ThriftClientManager-inl.h:66] resolve "metad1":9559 as "172.27.0.3":9559
I0128 05:56:45.939937    48 ThriftClientManager-inl.h:66] resolve "metad2":9559 as "172.27.0.2":9559
E0128 05:56:45.943087    48 MetaClient.cpp:624] Send request to "metad2":9559, exceed retry limit
I0128 05:56:14.839825     1 StorageDaemon.cpp:125] localhost = "storaged1":9779
I0128 05:56:14.889155     1 MetaClient.cpp:57] Create meta client to "metad1":9559
I0128 05:56:20.662170    49 ThriftClientManager-inl.h:66] resolve "metad0":9559 as "172.27.0.4":9559
I0128 05:56:26.806910    49 ThriftClientManager-inl.h:66] resolve "metad1":9559 as "172.27.0.3":9559
I0128 05:56:33.764557    49 ThriftClientManager-inl.h:66] resolve "metad0":9559 as "172.27.0.4":9559
I0128 05:56:41.647446    49 ThriftClientManager-inl.h:66] resolve "metad2":9559 as "172.27.0.2":9559
E0128 05:56:41.650044    49 MetaClient.cpp:624] Send request to "metad2":9559, exceed retry limit
I0128 05:56:10.622653     1 StorageDaemon.cpp:125] localhost = "storaged2":9779
I0128 05:56:10.671337     1 MetaClient.cpp:57] Create meta client to "metad0":9559
I0128 05:56:16.790499    49 ThriftClientManager-inl.h:66] resolve "metad0":9559 as "172.27.0.4":9559
I0128 05:56:23.666523    49 ThriftClientManager-inl.h:66] resolve "metad0":9559 as "172.27.0.4":9559
I0128 05:56:30.757205    49 ThriftClientManager-inl.h:66] resolve "metad2":9559 as "172.27.0.2":9559
I0128 05:56:38.523998    49 ThriftClientManager-inl.h:66] resolve "metad1":9559 as "172.27.0.3":9559
E0128 05:56:38.526479    49 MetaClient.cpp:624] Send request to "metad1":9559, exceed retry limit

[wey-gu]

Is your macOS intel-x86_64 or Apple-arm based, please?

Now the docker desktop simulated x86(qemu-fmt) on M1 is with issues. And native arm linux docker image is not yet supported now.

[awsl-dbq]

Thanks . It's M1, arm64.
And when I try to build it in mac,
I got

CMake Error at cmake/nebula/PlatformCheck.cmake:2 (MESSAGE):
  Only Linux is supported
Call Stack (most recent call first):
  CMakeLists.txt:41 (include)

Does nebula has any way to run on mac m1?

[wey-gu]

Sorry, the nebula code base had an strong assumption that it's running on a linux(not taking other unix-like OS into account).

I (as a M1 user) was hoping for the arm-linux support(thus the docker-desktop will work) for some time, too, it will be supported in future.

While, before that, we will have to do that on a x86 server to play with nebula graph :(, i really feel sorry for that.

Apart from that, you could use its GUI ready-only playground here: https://nebula-graph.io/demo/ or try my personal katacoda env https://siwei.io/cources/ to quicky play with it ;-).

[wey-gu]

Does nebula has any way to run on mac m1?

I think to simulate a x86 VM and running all process inside that VM is one way to go but I didn't try that before.(not like docker desktop, it used qemu to similate each process in an arm-linux machine, which came with communication issues for some reasons that I could not figure out why)

[wey-gu]

While the CLI console for neubla graph (namely nebula-console) comes with ARM binary in its github release, so you could deploy it on a x86 server and connect locally in your M1 mac.

[awsl-dbq]

OK, Thanks