- Implementing data analytics APIs against multiple underlying data sources often results in code bloat and tight coupling.
- If you're using Akka Streams for stream processing, you are probably missing a data sourcing component (like Storm's Spouts).
- ODBC/JDBC are antiquated and not designed for streaming large datasets.
Sonicd is a data streaming gateway that abstracts over data source connectors and provides a modern protocol to stream data over WebSockets or over plain TCP.
- PrestoSource: non-blocking streaming connector for Facebook's Presto.
- ElasticSearchSource: non-blocking streaming connector for ElasticSearch
- JdbcSource: JDBC connector for any database with a JDBC driver implementation (tested with Hive, Redshift, H2, MySQL, PostgreSQL).
- ZuoraObjectQueryLanguageSource: Zuora's SOAP API ZOQL streaming connector.
- LocalJsonStreamSource: stream changes in local JSON files.
- KafkaSource: Apache Kafka connector with simple DSL to select/filter streams efficiently.
- Composer: Combine in a single Sonic stream, any of the previous sources.
Check server/src/main/resources/reference.conf for a config reference and docker run -d -v ${CONFIG_DIR}:/etc/sonicd:ro -p 9111:9111 -p 10001:10001 xarxa6/sonicd;.
If intend to use the JDBC source, then you'll want to add JDBC drivers jars to the classpath:
docker run -d -p 9111:9111 -p 10001:10001 -v ${CONFIG_DIR}:/etc/sonicd/ -v ${JDBC_DRIVERS}:/var/lib/sonicd/ xarxa6/sonicdSonicd also provides a CLI to run ad hoc queries. If you have the rust toolchain installed already, then simply cargo install sonic, otherwise install rustup first with curl https://sh.rustup.rs -sSf | sh or check https://www.rustup.rs/.
Check examples folder. For an example in Rust check the cli.
If you would like to contribute to the project, please fork the project, include your changes and submit a pull request back to the main repository.
MIT License