Skip to content

Commit

Permalink
Merge pull request #68 from cpursley/remove-destinations
Browse files Browse the repository at this point in the history
Depreciate Destinations - take 1.
  • Loading branch information
cpursley committed Jun 21, 2024
2 parents 112f5d9 + 7dc4ae3 commit c9b0cbe
Show file tree
Hide file tree
Showing 17 changed files with 106 additions and 441 deletions.
58 changes: 9 additions & 49 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,9 @@
Simple and reliable Postgres [Change Data Capture
(CDC)](https://en.wikipedia.org/wiki/Change_data_capture) in Elixir.

WalEx allows you to listen to change events on your Postgres tables then send them on to [destinations](#destinations) or perform callback-like actions with the data via the [DSL](#elixir-dsl). For example:
WalEx allows you to listen to change events on your Postgres tables then perform callback-like actions with the data via the [Event DSL](#elixir-dsl). For example:

- Stream database changes to an event service like [EventRelay](https://github.com/eventrelay/eventrelay)
- Stream database changes to an external service
- Send a user a welcome email after they create a new account
- Augment an existing Postgres-backed application with business logic
- Send events to third party services (analytics, CRM, [webhooks](#webhooks))
Expand All @@ -27,7 +27,7 @@ by adding `walex` to your list of dependencies in `mix.exs`:
```elixir
def deps do
[
{:walex, "~> 3.8.0"}
{:walex, "~> 4.0.0"}
]
end
```
Expand Down Expand Up @@ -122,21 +122,9 @@ config :my_app, WalEx,
database: "postgres",
publication: "events",
subscriptions: ["user", "todo"],
# optional
destinations: [
# WalEx assumes your module names match this pattern: MyApp.Events.User, MyApp.Events.ToDo, etc
# but you can also specify custom modules like so:
# modules: [MyApp.CustomModule, MyApp.OtherCustomModule],
webhooks: ["https://webhook.site/c2f32b47-33ef-425c-9ed2-f369529a0de8"],
event_relay_topic: "todos"
],
webhook_signing_secret: "9da89f5f8f4717099c698a17c0d3a1869ee227de06c27b18",
event_relay: [
host: "localhost",
port: "50051",
token:
"cmpiNmpFSGhtNVhORFVubDFzUW9OR1JqTlFZOVFFcjRwZWMxS2VWRzJIOnY5NkFRQVFjSVp0TWVmc3hpRl8ydVZuaW9FTC0wX3JrZjhXcTE4MS1EbnVLU1p5VF9OZkpBZGs1SlFuQlNNdVg="
],
# WalEx assumes your module names match this pattern: MyApp.Events.User, MyApp.Events.ToDo, etc
# but you can also specify custom modules like so:
# modules: [MyApp.CustomModule, MyApp.OtherCustomModule],
name: MyApp
```

Expand Down Expand Up @@ -220,11 +208,9 @@ where _name_ field was changed):
]
```

### Destinations
### Event Module

#### Event Module

If your app is named _MyApp_ and you have a subscription called _:user_ (which represents a database table), WalEx assumes you have a module called `MyApp.Events.User` that uses WalEx Event. But you can also define any custom module, just be sure to add it to the _modules_ config under _destinations_.
If your app is named _MyApp_ and you have a subscription called _:user_ (which represents a database table), WalEx assumes you have a module called `MyApp.Events.User` that uses WalEx Event. But you can also define any custom module, just be sure to add it to the _modules_ config.

Note that the result of `events` is a list. This is because WalEx returns a _List_ of _transactions_ for a particular table when there's a change event. Often times this will just contain one result, but it could be many (for example, if you use database triggers to update a column after an insert).

Expand Down Expand Up @@ -312,37 +298,11 @@ defmodule MyApp.Events.User do
end
```

You can optionally [configure](#config) WalEx to automatically send events to the following destinations without needing to know Elixir:

#### Webhooks

Send subscribed events to one or more webhooks. WalEx supports the [Standard Webhooks](https://www.standardwebhooks.com) spec via the [webhoox](https://github.com/cpursley/webhoox) library (which can also be used to receive webhooks in Elixir).

#### EventRelay

If you need something more durable and flexible than webhooks, check out [EventRelay](https://github.com/eventrelay/eventrelay).

In EventRelay, you'll need to create a topic matching what's in the WalEx destinations config. So, if your event_relay_topic is called _todos_ (usually this is the database name), then your topic name in EventRelay should be `todos`. Here's how to do it via grpcurl:

```bash
grpcurl -H "Authorization: Bearer {api_key_token}" -plaintext -proto event_relay.proto -d '{"name": "todos"}' localhost:50051 eventrelay.Topics.CreateTopic
```

#### Coming Soon

More destinations coming. Pull requests welcome!

## Test

You'll need a local Postgres instance running

```bash
MIX_ENV=test mix walex.setup
MIX_ENV=test mix test
```

## Help?

I would love to have your help! I do ask that if you do find a bug, please add a test to your PR that shows the bug and how it was fixed.

Thanks!
```
29 changes: 4 additions & 25 deletions lib/walex/config/config.ex
Original file line number Diff line number Diff line change
Expand Up @@ -8,11 +8,7 @@ defmodule WalEx.Config do
alias WalEx.Config.Registry, as: WalExRegistry

@allowed_config_value ~w(database hostname name password port publication username webhook_signing_secret slot_name durable_slot message_middleware)a
@allowed_config_values ~w(destinations event_relay modules subscriptions)a

@type destinations_t :: [
{:modules, [module]} | {:webhooks, [binary]} | {:event_relay_topic, binary}
]
@allowed_config_values ~w(modules subscriptions)a

@type start_opts :: [
{:database, binary}
Expand All @@ -26,7 +22,6 @@ defmodule WalEx.Config do
| {:slot_name, binary}
| {:durable_slot, boolean}
| {:message_middleware, (term, term -> :ok)}
| {:destinations, destinations_t()}
| {:event_relay, keyword()}
| {:modules, [module]}
| {:subscriptions, [binary()]}
Expand Down Expand Up @@ -71,22 +66,7 @@ defmodule WalEx.Config do

def get_database(app_name), do: get_configs(app_name, :database)

def get_destination(app_name, destination) do
case get_configs(app_name, :destinations) do
destinations when is_list(destinations) and destinations != [] ->
destinations
|> Keyword.get(destination, nil)

_ ->
nil
end
end

def get_event_modules(app_name), do: get_destination(app_name, :modules)

def get_webhooks(app_name), do: get_destination(app_name, :webhooks)

def get_event_relay_topic(app_name), do: get_destination(app_name, :event_relay_topic)
def get_event_modules(app_name), do: get_configs(app_name, :modules)

def add_config(app_name, key, new_values)
when is_list(new_values) and key in @allowed_config_values do
Expand Down Expand Up @@ -131,8 +111,7 @@ defmodule WalEx.Config do

name = Keyword.get(configs, :name)
subscriptions = Keyword.get(configs, :subscriptions, [])
destinations = Keyword.get(configs, :destinations, [])
modules = Keyword.get(destinations, :modules, [])
modules = Keyword.get(configs, :modules, [])
module_names = build_module_names(name, modules, subscriptions)

[
Expand All @@ -147,7 +126,7 @@ defmodule WalEx.Config do
socket_options: Keyword.get(configs, :socket_options, []),
subscriptions: subscriptions,
publication: Keyword.get(configs, :publication),
destinations: Keyword.put(destinations, :modules, module_names),
modules: module_names,
webhook_signing_secret: Keyword.get(configs, :webhook_signing_secret),
event_relay: Keyword.get(configs, :event_relay),
slot_name: Keyword.get(configs, :slot_name) |> parse_slot_name(name),
Expand Down
84 changes: 0 additions & 84 deletions lib/walex/destinations/destinations.ex

This file was deleted.

98 changes: 0 additions & 98 deletions lib/walex/destinations/event_relay.ex

This file was deleted.

Loading

0 comments on commit c9b0cbe

Please sign in to comment.