Use custom functions to evaluate equality

[daphee]

I'd like to propose to add a syntax to allow Expectation creating functions other than Expect.equal. The case I'd need such a syntax for is to test examples containing floating point numbers, e.g.

cos (2*pi * 90/360)
--> 0  

This would fail because the result is actually something really small but different from 0.
I need to check for equality with a certain tolerance. The test should look like this:

cos (2*pi * 90/360) |> Expect.within (Absolute 0.000000001) 0

instead of:

cos (2*pi * 90/360) |> Expect.equal 0

I think this could be achieved by modifying the syntax to include another special character which marks a line that contains the function to be used, e.g. like this:

cos (2*pi * 90/360)
| Expect.equalWithingThreshold
--> 0  

What do you think? Would this obfuscate the examples too much? Do you maybe have a better idea on how to test examples like that? Is this doable?

[ianmackenzie]

I'd add that floating-point equality is an important case but it would also be great to support fully custom expectations on custom types. For example, in opensolid/geometry the documentation examples often look like

point =
    Point2d.fromCoordinates ( 1, 0 )

Point2d.rotateAround Point2d.origin (degrees 45) point
--> Point2d.fromCoordinates ( 0.707, 0.707 )

and a corresponding test might use

import OpenSolid.Geometry.Expect as Expect

Point2d.fromCoordinates ( 1, 0 )
    |> Point2d.rotateAround Point2d.origin (degrees 45)
    |> Expect.point2dWithin 0.001
        (Point2d.fromCoordinates ( 0.707, 0.707 )

Could this potentially be implemented by adding some more options to elm-verify-examples.json to specify what expectation functions to use by default in a particular file, or perhaps based on a regex match with the result expression or something? (For example if the result expression starts with Point2d.fromCoordinates then I know it's a Point2d and should use Expect.point2dWithin 0.001...)

[stoeffel]

⚠️ This is not a replacement for tests, this tool should be used for improving your documentation. (from the readme)

This is the reason, why I'm against adding special syntax for more complex expectations. Adding special syntax (--> is just a comment) and customised expectations make it hard to understand what's actually going on in the example. It feels like the examples you're describing are actually tests.

What do you think? Would this obfuscate the examples too much? Do you maybe have a better idea on how to test examples like that? Is this doable?

I think you could just do:

cos (2*pi * 90/360)
    |> truncate
--> 0  

[daphee]

I think e.g. this example is perfectly within the scope of the library. Still it isn't possible to verify it without introducing some additional function to truncate the numbers, like in your example.

Doing so would introduce the same amount of clutter than the syntax I proposed does.

Another possibilty would be to define which comparison function should be used in a seperate non-documentation comment above the example, like that:

{- @verify Expect.equalWithingThreshold -}
{-| 
     cos (2*pi * 90/360)
     --> 0  
-}

What do you think about that?

[stoeffel]

I think e.g. this example is perfectly within the scope of the library.

Agreed, but I think it's easier for people new to a library to grok an example if it doesn't hide the code for verifying a result. You can use intermediate definitions to simplify an example and still have a function to verify it. This way it's all explicit and you can separate the function from the example using a line-comment. Would that work for your use-case?
https://github.com/stoeffel/elm-verify-examples/#intermediate-definitions

Separately, I think it's better to keep the verified example as close to actual elm code as possible. The current syntax is totally valid elm and doesn't introduce any new syntax feature.

[gampleman]

I think there are 2 separate issues going on here:

1. Floating point can't be tested in elm-test using Expect.equal. This (as a side-effect of that decision) makes elm-verify-examples unusable for verifying examples for numerical code. Let's keep discussion of that issue in Impossible to test equality of floating point numbers #83.

2. Customising assertions for other stuff. I think @stoeffel has argued quite eloquently that there are good reasons for why that's out of scope. You can always do:

example input >= 3 --> True

or some other way to make a custom predicate, and it will be easier for users to understand what you are saying.

As such, I think we can close this issue. If there is some other angle we feel hasn't been covered, we can of course re-open this discussion.