Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feature: support raft configuration center #6830

Open
wants to merge 59 commits into
base: 2.x
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 16 commits
Commits
Show all changes
59 commits
Select commit Hold shift + click to select a range
739c947
feature: support raft config center(server)
LegendGasai Jul 14, 2024
52e53c9
feature: support raft config center(client)
LegendGasai Jul 15, 2024
fdb6cc3
bugfix: Add raft and rocksdbjni dependency in pom
LegendGasai Jul 17, 2024
7aed12e
optimize: Add some unit test and fix upload config bug
LegendGasai Jul 17, 2024
5c5ba4a
bugfix: add log and tcc prefix config items
LegendGasai Jul 18, 2024
e048418
optimize: rename db config key and add config unit tests
LegendGasai Jul 26, 2024
8f31636
optimize: rocksdb multi-tenant storage optimization
LegendGasai Jul 28, 2024
119e2d2
bugfix: Fix JVM exception caused by incorrect close of rocksdb's object
LegendGasai Jul 29, 2024
ec10f68
optimize: add configuration version manager in raft mode and add conf…
LegendGasai Aug 6, 2024
9cc03ed
optimize: add shutdown hook for ConfigStoreManager.
LegendGasai Aug 7, 2024
a1ce74c
fix: fix some bugs.
LegendGasai Aug 13, 2024
92468ec
Merge remote-tracking branch 'upstream/2.x' into summer-code-raft-con…
LegendGasai Sep 7, 2024
9b256cb
fix: add a npm dependency
LegendGasai Sep 7, 2024
97df2c7
optimize: add configuration meta in console ui
LegendGasai Sep 8, 2024
841e413
fix: remove some console file
LegendGasai Sep 8, 2024
c8c502b
fix: code style and unit test
LegendGasai Sep 9, 2024
335a0c6
Merge branch '2.x' into summer-code-raft-config-center
xingfudeshi Sep 9, 2024
5b38176
fix: code style and bug fix
LegendGasai Sep 9, 2024
ebc254f
Merge remote-tracking branch 'upstream/2.x' into summer-code-raft-con…
LegendGasai Sep 9, 2024
f02b649
Merge branch 'summer-code-raft-config-center' of github.com:LegGasai/…
LegendGasai Sep 9, 2024
e91d4ad
optimize: improve configuration table style
LegendGasai Sep 9, 2024
48ed38e
fix: code style
LegendGasai Sep 9, 2024
2b7fb8c
Merge branch '2.x' into summer-code-raft-config-center
xingfudeshi Sep 10, 2024
9b8485c
fix: code style
LegendGasai Sep 10, 2024
6e34d21
Merge branch 'summer-code-raft-config-center' of github.com:LegGasai/…
LegendGasai Sep 10, 2024
597e62b
Merge branch '2.x' into summer-code-raft-config-center
xingfudeshi Sep 10, 2024
6a4530d
fix: code style
LegendGasai Sep 10, 2024
fe8599b
Merge branch 'summer-code-raft-config-center' of github.com:LegGasai/…
LegendGasai Sep 10, 2024
3d621f9
Merge branch '2.x' into summer-code-raft-config-center
xingfudeshi Sep 11, 2024
f3fcaf0
Merge branch '2.x' into summer-code-raft-config-center
xingfudeshi Sep 11, 2024
e957210
Merge remote-tracking branch 'upstream/2.x' into summer-code-raft-con…
LegendGasai Sep 11, 2024
1dfb17b
Merge branch '2.x' into summer-code-raft-config-center
xingfudeshi Sep 11, 2024
17f90f9
Merge branch '2.x' into summer-code-raft-config-center
xingfudeshi Sep 11, 2024
afcce0b
optimize: add unit tests and fix some bugs
LegendGasai Sep 11, 2024
7081bfa
Merge remote-tracking branch 'upstream/2.x' into summer-code-raft-con…
LegendGasai Sep 11, 2024
63b5c29
Merge branch 'summer-code-raft-config-center' of github.com:LegGasai/…
LegendGasai Sep 11, 2024
c4cabbb
ci
LegendGasai Sep 11, 2024
366ed39
fix : fix raft test
LegendGasai Sep 12, 2024
d3a2b81
test: fix raft test fail.
LegendGasai Sep 12, 2024
8a26be4
test: fix raft test fail.
LegendGasai Sep 12, 2024
1eecba1
Merge branch '2.x' into summer-code-raft-config-center
LegGasai Sep 12, 2024
d0f85c9
Merge branch 'summer-code-raft-config-center' of github.com:LegGasai/…
LegendGasai Sep 12, 2024
f2d0f8b
test: fix raft test fail.
LegendGasai Sep 12, 2024
8bcbc81
Merge branch '2.x' into summer-code-raft-config-center
funky-eyes Sep 12, 2024
6900a37
Merge branch '2.x' into summer-code-raft-config-center
xingfudeshi Sep 13, 2024
c1f90ba
Merge remote-tracking branch 'upstream/2.x' into summer-code-raft-con…
LegendGasai Sep 13, 2024
235dd76
Merge remote-tracking branch 'upstream/2.x' into summer-code-raft-con…
LegendGasai Sep 14, 2024
dea1df5
Merge branch '2.x' into summer-code-raft-config-center
LegGasai Sep 14, 2024
7a959e0
Merge branch '2.x' into summer-code-raft-config-center
LegGasai Sep 16, 2024
d083127
Merge branch 'summer-code-raft-config-center' of github.com:LegGasai/…
LegendGasai Sep 16, 2024
891f314
optimize: raft stateMachine
LegendGasai Sep 16, 2024
0c87a26
Merge branch '2.x' into summer-code-raft-config-center
lovepoem Sep 17, 2024
92d574f
Merge branch '2.x' into summer-code-raft-config-center
xingfudeshi Sep 18, 2024
dcdfdaa
Merge branch '2.x' into summer-code-raft-config-center
LegGasai Sep 22, 2024
31034b3
Merge branch '2.x' into summer-code-raft-config-center
xingfudeshi Sep 23, 2024
05574be
optimize: code style
LegendGasai Sep 27, 2024
a92f09f
Merge branch '2.x' into summer-code-raft-config-center
xingfudeshi Sep 30, 2024
d8d8178
Merge remote-tracking branch 'upstream/2.x' into summer-code-raft-con…
LegendGasai Oct 9, 2024
92b59e0
Merge branch 'summer-code-raft-config-center' of github.com:LegGasai/…
LegendGasai Oct 9, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,10 @@ dependency-reduced-pom.xml
/distribution/sessionStore/
/distribution/*/sessionStore/
/file_store/
/configStore/
/config/configStore/
/distribution/configStore/
/distribution/*/configStore/

# system ignore
.DS_Store
Expand Down
5 changes: 5 additions & 0 deletions all/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,11 @@
<artifactId>seata-config-spring-cloud</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>org.apache.seata</groupId>
<artifactId>seata-config-raft</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>org.apache.seata</groupId>
<artifactId>seata-core</artifactId>
Expand Down
1 change: 1 addition & 0 deletions build/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -310,6 +310,7 @@
<include>*-pom.xml</include>
<include>**/db_store/**</include>
<include>**/sessionStore/**</include>
<include>**/configStore/**</include>
<include>**/root.data</include>
</includes>
<followSymlinks>false</followSymlinks>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -1017,6 +1017,31 @@ public interface ConfigurationKeys {
*/
String ROCKET_MQ_MSG_TIMEOUT = SERVER_PREFIX + "rocketmqMsgTimeout";

String CONFIG_STORE_PREFIX = FILE_ROOT_PREFIX_CONFIG + "raft" + FILE_CONFIG_SPLIT_CHAR + "db" + FILE_CONFIG_SPLIT_CHAR;

/**
* The constant CONFIG_STORE_TYPE
*/
String CONFIG_STORE_TYPE = CONFIG_STORE_PREFIX + "type";

/**
* The constant CONFIG_STORE_DIR
*/
String CONFIG_STORE_DIR = CONFIG_STORE_PREFIX + "dir";

/**
* The constant CONFIG_STORE_DESTROY_ON_SHUTDOWN
*/
String CONFIG_STORE_DESTROY_ON_SHUTDOWN = CONFIG_STORE_PREFIX + "destroyOnShutdown";

/**
* The constant CONFIG_STORE_NAMESPACE
*/
String CONFIG_STORE_NAMESPACE = CONFIG_STORE_PREFIX + "namespace";
/**
* The constant CONFIG_STORE_DATA_ID
*/
String CONFIG_STORE_DATA_ID = CONFIG_STORE_PREFIX + "dataId";
/**
*
*/
Expand Down Expand Up @@ -1051,4 +1076,5 @@ public interface ConfigurationKeys {
* The constant META_PREFIX
*/
String META_PREFIX = SEATA_FILE_ROOT_CONFIG + FILE_CONFIG_SPLIT_CHAR + FILE_ROOT_REGISTRY + FILE_CONFIG_SPLIT_CHAR + "metadata.";

}
34 changes: 34 additions & 0 deletions common/src/main/java/org/apache/seata/common/Constants.java
Original file line number Diff line number Diff line change
Expand Up @@ -225,6 +225,36 @@ public interface Constants {
*/
String JACKSON_JSON_TEXT_PREFIX = "{\"@class\":";

/**
* The constant APPLICATION_TYPE_KEY
*/
String APPLICATION_TYPE_KEY = "application.type";

/**
* The constant APPLICATION_TYPE_SERVER
*/
String APPLICATION_TYPE_SERVER = "server";

/**
* The constant APPLICATION_TYPE_CLIENT
*/
String APPLICATION_TYPE_CLIENT = "client";

/**
* The constant DEFAULT_STORE_NAMESPACE in raft configuration
*/

String DEFAULT_STORE_NAMESPACE = "default";
/**
* The constant DEFAULT_STORE_DATA_ID in raft configuration
*/
String DEFAULT_STORE_DATA_ID = "seata.properties";

/**
* The constant RAFT_CONFIG_GROUP
*/
String RAFT_CONFIG_GROUP = "config";

/**
* The constant DEAD_LOCK_SQL_STATE
*/
Expand All @@ -235,4 +265,8 @@ public interface Constants {
*/
int DEAD_LOCK_ERROR_CODE = 1213;

/**
* The constant CONFIGURATION_META_FILE_NAME
*/
String CONFIGURATION_META_FILE_NAME = "configuration-meta.yml";
}
Original file line number Diff line number Diff line change
Expand Up @@ -314,4 +314,8 @@ public interface DefaultValues {
String DRUID_LOCATION = "lib/sqlparser/druid.jar";

int DEFAULT_ROCKET_MQ_MSG_TIMEOUT = 60 * 1000;

String DEFAULT_DB_STORE_FILE_DIR = "configStore";

String DEFAULT_DB_TYPE = "rocksdb";
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.seata.common.config;


public class ConfigDataResponse<T> {
xingfudeshi marked this conversation as resolved.
Show resolved Hide resolved

T result;
String errMsg;
Boolean success;
xingfudeshi marked this conversation as resolved.
Show resolved Hide resolved

public T getResult() {
return result;
}

public void setResult(T result) {
this.result = result;
}

public String getErrMsg() {
return errMsg;
}

public void setErrMsg(String errMsg) {
this.errMsg = errMsg;
}

public Boolean getSuccess() {
return success;
}

public void setSuccess(Boolean success) {
this.success = success;
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,8 @@ public class MetadataResponse {

String storeMode;

String configMode;

long term;

public List<Node> getNodes() {
Expand All @@ -50,5 +52,12 @@ public long getTerm() {
public void setTerm(long term) {
this.term = term;
}


public String getConfigMode() {
return configMode;
}

public void setConfigMode(String configMode) {
this.configMode = configMode;
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,8 @@
*/
package org.apache.seata.common.store;

import org.apache.seata.common.util.StringUtils;

/**
* transaction log store mode
*
Expand Down Expand Up @@ -55,6 +57,9 @@ public enum StoreMode {
* @return the store mode
*/
public static StoreMode get(String name) {
if (StringUtils.isEmpty(name)) {
return null;
}
for (StoreMode sm : StoreMode.class.getEnumConstants()) {
if (sm.name.equalsIgnoreCase(name)) {
return sm;
Expand Down
18 changes: 18 additions & 0 deletions common/src/main/java/org/apache/seata/common/util/NumberUtils.java
Original file line number Diff line number Diff line change
Expand Up @@ -60,4 +60,22 @@ public static Long toLong(String str) {
}
return null;
}

public static byte[] longToBytes(long x) {
byte[] result = new byte[8];
for (int i = 7; i >= 0; i--) {
result[i] = (byte)(x & 0xFF);
x >>= 8;
}
return result;
}

public static long bytesToLong(byte[] bytes) {
long result = 0;
for (int i = 0; i < 8; i++) {
result <<= 8;
result |= bytes[i] & 0xFF;
}
return result;
}
}
1 change: 1 addition & 0 deletions config/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -41,5 +41,6 @@
<module>seata-config-etcd3</module>
<module>seata-config-consul</module>
<module>seata-config-spring-cloud</module>
<module>seata-config-raft</module>
</modules>
</project>
5 changes: 5 additions & 0 deletions config/seata-config-all/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,11 @@
<artifactId>seata-config-spring-cloud</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>${project.groupId}</groupId>
<artifactId>seata-config-raft</artifactId>
<version>${project.version}</version>
</dependency>
</dependencies>

</project>
5 changes: 5 additions & 0 deletions config/seata-config-core/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,11 @@
<groupId>org.yaml</groupId>
<artifactId>snakeyaml</artifactId>
</dependency>
<dependency>
<groupId>org.rocksdb</groupId>
<artifactId>rocksdbjni</artifactId>
</dependency>

</dependencies>

</project>
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,11 @@ public enum ConfigType {
/**
* Custom config type
*/
Custom;
Custom,
/**
* Raft config type
*/
Raft;
xingfudeshi marked this conversation as resolved.
Show resolved Hide resolved

/**
* Gets type.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,10 @@ public ConfigurationChangeEvent(String dataId, String newValue) {
this(dataId, DEFAULT_NAMESPACE, null, newValue, ConfigurationChangeType.MODIFY);
}

public ConfigurationChangeEvent(String namespace, String dataId, String newValue) {
this(dataId, namespace, null, newValue, ConfigurationChangeType.MODIFY);
}

public ConfigurationChangeEvent(String dataId, String namespace, String oldValue, String newValue,
ConfigurationChangeType type) {
this.dataId = dataId;
Expand Down
Loading
Loading