Loupe

Loupe is query language for Ecto schema inspection in a safe and configurable manner.

You can see this example app to understand how it applies with Ecto.

Important

Until Loupe reaches 1.x.x, it's considered experimental. The syntax will change, APIs will change and structure will too. We'll do our best to respect semantic versioning and avoid big breaking changes but they can happen.

Installation

If available in Hex, the package can be installed by adding loupe to your list of dependencies in mix.exs:

def deps do
  [
    {:loupe, "~> 0.1.0"}
  ]
end

Documentation can be generated with ExDoc and published on HexDocs. Once published, the docs can be found at https://hexdocs.pm/loupe.

Syntax

The basic syntax has the following format

get [quantifier?] [schema][parameters?] where [predicates]

• quantifier is how many records you want. You can provide a positive integer (1, 2, 3 ...), a range (1..10, 10..20, 50..100) or all.
• schema needs to be an alphanumeric indentifier that you registered in the Definition (See Ecto Usage for exmaple).
• parameters is a json inspired map. It takes the format of {key: "value"}. Key is an identifier, but value can be any literal type (another object, string, int, float, boolean, list)
• predicates needs to be a combinaison or operators and boolean operators.

Cool stuff

You can use k and m quantifiers for numbers. Writing get all User where money > 100k translates to get all User where money > 100000.

Operators

The are a couple of basic operators like <, >, <=, >=, =, !=.

But also some textual operators:

• in is used with lists, like age in [18, 21]
• like is used with strings and automatically wraps in %.

You can also use the keyword :empty as a null checker like age :empty.

Textual operators and :empty can be prefixed with not to negate the expression: not like, not in, age not :empty.

For boolean, the binding can be provided as is and prefixed by not for false. Example where active or where not enabled.

Boolean Operators

So far, the syntax supprts and and or and use parenthese to scope the expressions.

Field variant

Recently, support for "field variant" has been added. It's a syntax that allows to "customize" a field. The Ecto implementation uses the variant to query composite fields. Assume you have a composite Postgres field that is Money (like the Money.Ecto.Composite.Type type from the Money lib), you can now do the following to query the amount:

get User where bank_account:amount >= 1k

Path binding

Loupe now supports "Path binding", being able to specify a path (like a json path) on a field. This is used by te Ecto implementation to query json field like below:

get User where role.permissions[posts, access] = "write"
# or
get User where role.permissions["posts", "access"] = "write"

Variables and external identifiers

Query variable

The library allows you to provide external data to you query. Any identifier (unquote alphanumerical and underscore values) provided on the right side of an operator will be output as such. Taking for instance the Ecto implementation, it allows you to provide external parameter to the query.

A good usecase example could be to automatically provide a user_id based from the authenticated user. So you can use it like:

get Posts where author_id = user_id

Then when evaluating the query you make sure to provide the user id by doing

Loupe.Ecto.build_query(query, EctoDefinition, %{}, %{"user_id" => current_user.id})

Note: Variables in query are required. When evaluating if the query uses a variable that is not provided, an error will be raise.

Parameters

This variables can also be used in parameters. Suppose your implementation supports an order_by parameter, you can use it like

get Posts{order_by: {direction: direction, field: field}}

Unlike variables, they don't need to be provided, they are simply extract as such and it's up to you to manipulate them the way you want. For the case of the Ecto implementation, however, they do need to be implemented so they can be extracted in the returning context.

Ecto usage

Create a Definition module

The Definition module is necessary for Loupe to work with your Ecto schema. In this module you define the schemas that are allowed to be queried and the fields that are permitted for querying.

All callbacks accepts a last argument called "assigns". The assigns are provided to you when evaluating the query allowing you to alter the defition. You could, for instance, add a user's role to the assign and use that role to filter out the allowed schemas so that only admins can query Users.

defmodule MyApp.Loupe.Definition do
    @moduledoc """
    Example Ecto definition for the modules defined above.
    """
    @behaviour Loupe.Ecto.Definition

    @schemas %{
      "Post" => Post,
      "User" => User,
      "Role" => Role
    }

    @impl Loupe.Ecto.Definition
    def schemas(%{role: "admin"}), do: @schemas
    def schemas(_), do: Map.take(@schemas, ["Post", "User"])

    @impl Loupe.Ecto.Definition
    def schema_fields(_, %{role: "admin"}), do: :all
    def schema_fields(Post, _), do: {:only, [:title, :body]}
    def schema_fields(User, _), do: {:only, [:email, :posts]}
    def schema_fields(_, _), do: :all

    @impl Loupe.Ecto.Definition
    def scope_schema(schema, _), do: schema
end

Once you have this definition, you can try some queries

{:ok, ast} = Loupe.Language.compile(~s|get all User where age > 18|)
{:ok, ecto_query} = Loupe.Ecto.build_query(ast, MyApp.Loupe.Definition, %{role: "admin"})
Repo.all(ecto_query)

Todo

Here are some things that I would like Loupe to support:

• Sorting a query, current ideas involves
  • get all User order asc inserted_at
  • get all User where age > 10 ordered asc inserted_at.
• Support some more complex fields prefixed by ~ (or whatever syntax, inspired by elixir's sigils) like the examples below
  • get all Product where price = ~99.99$ and have that use the Elixir money lib.
  • get all Item where ratio = ~1/4
• Implement a LiveView UI lib that shows the strucutres as expandable. Being able to click on a User's posts to automatically preload all of its nested Posts.
  • Also have "block" UI module where you can simply create a query from dropdowns in a form for non-power user.
• Make lexer and parser swappable. Right now, you are stuck with the internal structure that I came up with. The idea would be to allow some to swap the syntax for anything they want. For instance, a french team could implement a french query language to give to their normal user.

Contributing

You can see the CONTRIBUTING.md file to know more about the contributing guidelines.

Pull requests are welcome!