Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feature: support etcd v3, by mocking v2 API #2036

Merged
merged 79 commits into from
Sep 16, 2020
Merged
Show file tree
Hide file tree
Changes from 63 commits
Commits
Show all changes
79 commits
Select commit Hold shift + click to select a range
e76c285
feat: support etcd v3 by mock v2 api
Yiyiyimu Aug 10, 2020
c710b23
fix: lint
Yiyiyimu Aug 10, 2020
c5a200a
avoid lib lbase64 error
Yiyiyimu Aug 10, 2020
913c744
support etcd v3.2, multiversion WIP
Yiyiyimu Aug 11, 2020
a2b93d3
fix travis files
Yiyiyimu Aug 11, 2020
2597b4f
test for ETCDCTL_API position
Yiyiyimu Aug 11, 2020
82fd940
fix: license check typo
Yiyiyimu Aug 11, 2020
e91ccf5
fix: converted index tonumber
Yiyiyimu Aug 11, 2020
a14ff1b
fix typo
Yiyiyimu Aug 11, 2020
3c6480e
only first set return status 201, others return 200
Yiyiyimu Aug 11, 2020
c85d86a
fix v3 watch would not return all changes
Yiyiyimu Aug 16, 2020
c29a210
fix unnecessary change
Yiyiyimu Aug 16, 2020
479573f
fix lint
Yiyiyimu Aug 16, 2020
354219d
retest
Yiyiyimu Aug 17, 2020
e18d3ed
v2/v3 CI; test ETCDCTL_API
Yiyiyimu Aug 17, 2020
c5993dc
use luaposix to set environment variable ETCDCTL_API
Yiyiyimu Aug 17, 2020
d3ecb45
fix license-check
Yiyiyimu Aug 17, 2020
97809d8
env var test
Yiyiyimu Aug 17, 2020
2e771c7
support v3 only
Yiyiyimu Aug 22, 2020
e2d48cb
merge master
Yiyiyimu Aug 22, 2020
330135a
remove etcd v2 in github actions
Yiyiyimu Aug 22, 2020
e8cfaab
fix typo
Yiyiyimu Aug 22, 2020
95e8bed
fix format and doc
Yiyiyimu Aug 22, 2020
bcba1b8
fix format
Yiyiyimu Aug 22, 2020
7a24f70
fix indentation, remove tostring not needed
Yiyiyimu Aug 23, 2020
091dbbf
catch etcd_version_from_cmd failure
Yiyiyimu Aug 26, 2020
4b4d100
make function name readable
Yiyiyimu Aug 26, 2020
b9c5dc3
bugfix: create etcd instance in `timer`, `cosocket` is not valid in `…
membphis Aug 30, 2020
58d98ee
bugfix: create `etcd_cli` object if it is not autorun model.
membphis Aug 30, 2020
781e92a
remove init_etcd, since watch not need create dir first
Yiyiyimu Sep 3, 2020
c2e02e7
Merge branch 'master' of https://github.com/apache/incubator-apisix i…
Yiyiyimu Sep 3, 2020
b4955ea
update etcd to v3.4, for auth correctly working
Yiyiyimu Sep 3, 2020
73c96c8
fix typo
Yiyiyimu Sep 3, 2020
95aba51
move etcd & etcdctl to default path
Yiyiyimu Sep 3, 2020
af6b7f8
fix typo
Yiyiyimu Sep 3, 2020
f938505
init prev_index to 0
Yiyiyimu Sep 3, 2020
661d9a9
readd init_etcd for now, due to effect logic in config_etcd a lot
Yiyiyimu Sep 4, 2020
bd49c6b
fix typo
Yiyiyimu Sep 4, 2020
92bba2f
remove basexx lib
Yiyiyimu Sep 4, 2020
74cd62f
fix lint
Yiyiyimu Sep 4, 2020
03ce792
fix typo
Yiyiyimu Sep 4, 2020
df89d8d
rm init_etcd
Yiyiyimu Sep 4, 2020
0902ff2
fix typo
Yiyiyimu Sep 4, 2020
f20697c
Merge branch 'master' of https://github.com/apache/incubator-apisix i…
Yiyiyimu Sep 8, 2020
f5eb657
Merge branch 'master' of https://github.com/apache/incubator-apisix i…
Yiyiyimu Sep 10, 2020
401c770
init_etcd back, with a tiny base64 lib
Yiyiyimu Sep 10, 2020
5a3ea6b
revert
johzchen Sep 10, 2020
1fe1f22
optimize waitdir
Yiyiyimu Sep 10, 2020
4949580
Merge branch 'simple-etcdv3' of https://github.com/yiyiyimu/incubator…
Yiyiyimu Sep 10, 2020
85e2e0b
Merge branch 'master' of https://github.com/apache/incubator-apisix i…
Yiyiyimu Sep 10, 2020
78da2dd
[WIP]etcd test
Yiyiyimu Sep 10, 2020
0a4dbd2
[WIP]etcd test
Yiyiyimu Sep 10, 2020
9d7d7fe
fix lint
Yiyiyimu Sep 11, 2020
4ea856b
test: delete if needed
Yiyiyimu Sep 11, 2020
091f97b
revert
Yiyiyimu Sep 11, 2020
fbd200c
fix lint
Yiyiyimu Sep 12, 2020
203c3e3
update test cases
johzchen Sep 12, 2020
b0ca069
fix err
johzchen Sep 12, 2020
a4d92bd
delete test case first
johzchen Sep 12, 2020
18ec078
fix: err
johzchen Sep 12, 2020
34ccfbb
try twice watch to skip create info
johzchen Sep 12, 2020
9735d25
fix: lint
johzchen Sep 14, 2020
5f9b586
Merge branch 'master' into simple-etcdv3
moonming Sep 14, 2020
075eb7f
install etcd in a shell script
johzchen Sep 14, 2020
099a86b
fix code style
johzchen Sep 14, 2020
a85203c
add doc
Sep 14, 2020
4edd3a0
add doc
Yiyiyimu Sep 14, 2020
b400194
test: remove the 1999 port listening and reuse 1980~1982.
membphis Sep 15, 2020
13c276e
Merge branch 'master' into simple-etcdv3
moonming Sep 15, 2020
59a59fd
merge
Yiyiyimu Sep 15, 2020
d2e3d01
rm not reached conditions
Yiyiyimu Sep 15, 2020
58e8401
check if etcd above v3.4
Yiyiyimu Sep 15, 2020
3688862
optimize output of etcd version check
Yiyiyimu Sep 15, 2020
fc93f3d
fix typo
Yiyiyimu Sep 15, 2020
ca7324d
compatibility for etcd version
Yiyiyimu Sep 15, 2020
7e33e71
use check_or_version for comparison
Yiyiyimu Sep 15, 2020
c990fe3
update func name
Yiyiyimu Sep 15, 2020
0570fdb
fix: etcd needed version
johzchen Sep 15, 2020
7d30618
Merge branch 'master' into simple-etcdv3
Yiyiyimu Sep 15, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,6 @@ addons:
- libreadline-dev
- libssl-dev
- perl
- etcd
homebrew:
update: true

Expand Down
10 changes: 8 additions & 2 deletions .travis/linux_apisix_current_luarocks_runner.sh
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,12 @@ do_install() {
sudo make install > build.log 2>&1 || (cat build.log && exit 1)
cd ..
rm -rf luarocks-2.4.4

wget https://github.com/etcd-io/etcd/releases/download/v3.4.0/etcd-v3.4.0-linux-amd64.tar.gz
tar xf etcd-v3.4.0-linux-amd64.tar.gz
sudo cp etcd-v3.4.0-linux-amd64/etcd /usr/local/bin/
sudo cp etcd-v3.4.0-linux-amd64/etcdctl /usr/local/bin/
rm -rf etcd-v3.4.0-linux-amd64
}

script() {
Expand All @@ -48,8 +54,8 @@ script() {
sudo service etcd start
sudo service etcd stop
mkdir -p ~/etcd-data
/usr/bin/etcd --listen-client-urls 'http://0.0.0.0:2379' --advertise-client-urls='http://0.0.0.0:2379' --data-dir ~/etcd-data > /dev/null 2>&1 &
etcd --version
etcd --listen-client-urls 'http://0.0.0.0:2379' --advertise-client-urls='http://0.0.0.0:2379' --data-dir ~/etcd-data > /dev/null 2>&1 &
etcdctl version
sleep 5

sudo rm -rf /usr/local/apisix
Expand Down
9 changes: 7 additions & 2 deletions .travis/linux_openresty_mtls_runner.sh
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,11 @@ do_install() {

sudo luarocks install luacheck > build.log 2>&1 || (cat build.log && exit 1)

wget https://github.com/etcd-io/etcd/releases/download/v3.4.0/etcd-v3.4.0-linux-amd64.tar.gz
tar xf etcd-v3.4.0-linux-amd64.tar.gz
sudo cp etcd-v3.4.0-linux-amd64/etcd /usr/local/bin/
sudo cp etcd-v3.4.0-linux-amd64/etcdctl /usr/local/bin/
rm -rf etcd-v3.4.0-linux-amd64

if [ ! -f "build-cache/apisix-master-0.rockspec" ]; then
create_lua_deps
Expand Down Expand Up @@ -92,8 +97,8 @@ script() {
openresty -V
sudo service etcd stop
mkdir -p ~/etcd-data
/usr/bin/etcd --listen-client-urls 'http://0.0.0.0:2379' --advertise-client-urls='http://0.0.0.0:2379' --data-dir ~/etcd-data > /dev/null 2>&1 &
etcd --version
etcd --listen-client-urls 'http://0.0.0.0:2379' --advertise-client-urls='http://0.0.0.0:2379' --data-dir ~/etcd-data > /dev/null 2>&1 &
etcdctl version
sleep 5


Expand Down
10 changes: 8 additions & 2 deletions .travis/linux_openresty_runner.sh
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,12 @@ do_install() {

sudo luarocks install luacheck > build.log 2>&1 || (cat build.log && exit 1)

wget https://github.com/etcd-io/etcd/releases/download/v3.4.0/etcd-v3.4.0-linux-amd64.tar.gz
tar xf etcd-v3.4.0-linux-amd64.tar.gz
sudo cp etcd-v3.4.0-linux-amd64/etcd /usr/local/bin/
sudo cp etcd-v3.4.0-linux-amd64/etcdctl /usr/local/bin/
rm -rf etcd-v3.4.0-linux-amd64

if [ ! -f "build-cache/apisix-master-0.rockspec" ]; then
create_lua_deps

Expand Down Expand Up @@ -127,8 +133,8 @@ script() {
openresty -V
sudo service etcd stop
mkdir -p ~/etcd-data
/usr/bin/etcd --listen-client-urls 'http://0.0.0.0:2379' --advertise-client-urls='http://0.0.0.0:2379' --data-dir ~/etcd-data > /dev/null 2>&1 &
etcd --version
etcd --listen-client-urls 'http://0.0.0.0:2379' --advertise-client-urls='http://0.0.0.0:2379' --data-dir ~/etcd-data > /dev/null 2>&1 &
etcdctl version
sleep 5

./build-cache/grpc_server_example &
Expand Down
10 changes: 8 additions & 2 deletions .travis/linux_tengine_runner.sh
Original file line number Diff line number Diff line change
Expand Up @@ -259,6 +259,12 @@ do_install() {

sudo luarocks install luacheck > build.log 2>&1 || (cat build.log && exit 1)

wget https://github.com/etcd-io/etcd/releases/download/v3.4.0/etcd-v3.4.0-linux-amd64.tar.gz
tar xf etcd-v3.4.0-linux-amd64.tar.gz
sudo cp etcd-v3.4.0-linux-amd64/etcd /usr/local/bin/
sudo cp etcd-v3.4.0-linux-amd64/etcdctl /usr/local/bin/
rm -rf etcd-v3.4.0-linux-amd64
Yiyiyimu marked this conversation as resolved.
Show resolved Hide resolved

git clone https://github.com/iresty/test-nginx.git test-nginx
make utils

Expand All @@ -280,8 +286,8 @@ script() {
openresty -V
sudo service etcd stop
mkdir -p ~/etcd-data
/usr/bin/etcd --listen-client-urls 'http://0.0.0.0:2379' --advertise-client-urls='http://0.0.0.0:2379' --data-dir ~/etcd-data > /dev/null 2>&1 &
etcd --version
etcd --listen-client-urls 'http://0.0.0.0:2379' --advertise-client-urls='http://0.0.0.0:2379' --data-dir ~/etcd-data > /dev/null 2>&1 &
etcdctl version
sleep 5

./build-cache/grpc_server_example &
Expand Down
2 changes: 1 addition & 1 deletion .travis/osx_openresty_runner.sh
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ script() {
export_or_prefix
export PATH=$OPENRESTY_PREFIX/nginx/sbin:$OPENRESTY_PREFIX/luajit/bin:$OPENRESTY_PREFIX/bin:$PATH

etcd --enable-v2=true &
etcd &
sleep 1

./grpc_server_example &
Expand Down
5 changes: 2 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -166,10 +166,9 @@ There are several ways to install the Apache Release version of APISIX:
apisix start
```

**Note**: Apache APISIX does not yet support the v3 protocol of etcd, so you need to enable v2 protocol when starting etcd.
We are doing support for etcd v3 protocol.
**Note**: Apache APISIX would not support the v2 protocol of etcd anymore since APISIX v2.0, so you need to enable v3 protocol when starting etcd, if etcd version is below v3.4.
membphis marked this conversation as resolved.
Show resolved Hide resolved
```shell
etcd --enable-v2=true &
export ETCDCTL_API=3
```

## For Developer
Expand Down
5 changes: 2 additions & 3 deletions README_CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -165,10 +165,9 @@ CentOS 7, Ubuntu 16.04, Ubuntu 18.04, Debian 9, Debian 10, macOS, **ARM64** Ubun
apisix start
```

**注意**:Apache APISIX 现在还不支持 etcd 的 v3 协议,所以启动 etcd 时需要开启 v2 协议的支持。
我们正在做 etcd v3 协议的支持。
**注意**:Apache APISIX 从 v2.0 开始不再支持 etcd v2 协议,如果 etcd 版本低于 v3.4,启动 etcd 时需要开启 v3 协议的支持。
```shell
etcd --enable-v2=true &
export ETCDCTL_API=3
```

## 针对开发者
Expand Down
Loading