Feature request: make rule identifier available in test output

[davewongillies]

In the docs, I see that the deny, violation, and warn rules are able to be suffixed, eg deny_myrule. It'd be really great if somehow the identifies would be printed when running conftest test

For the standard output it currently looks like this:

WARN - test.json - main - WARNING: warn_myrule

1 test, 0 passed, 1 warning, 0 failures, 0 exceptions

It'd be helpful to perhaps be:

WARN_MYRULE - test.json - main - WARNING: warn_myrule

1 test, 0 passed, 1 warning, 0 failures, 0 exceptions

And json output currently is:

[
	{
		"filename": "test.json",
		"namespace": "main",
		"successes": 0,
		"warnings": [
			{
				"msg": "WARNING: warn_myrule"
			}
		]
	}
]

Perhaps something like

[
	{
		"filename": "test.json",
		"namespace": "main",
		"successes": 0,
		"warnings": [
			{
				"identifier": "warn_myrule",
				"msg": "WARNING: warn_myrule"
			}
		]
	}
]

[davewongillies]

It'd be interesting also if we could pass in extra parameters into a rule, eg:

warn_myrule[msg, my_thing] {

  msg := "Rule error here"
  my_thing := "Extra stuff"
}

Then when running conftest test -o json

[
	{
		"filename": "test.json",
		"namespace": "main",
		"successes": 0,
		"warnings": [
			{
				"msg": "Rule error here",
				"my_thing": "Extra stuff"
			}
		]
	}
]

[matrixise]

The feature with the name of the rule is really interesting, just for a debugging session.

Thank you

[boranx]

does underscore work with warn? If so, there might be a regression?

git diff
.
.
-warn[msg] {
+warn_myrule[msg] {

./conftest test -p examples/kubernetes/policy examples/kubernetes/deployment+service.yaml
Error: running test: load: loading policies: get compiler: 2 errors occurred:
examples/kubernetes/policy/base_test.rego:12: rego_unsafe_var_error: var warn is unsafe
examples/kubernetes/policy/base_test.rego:50: rego_unsafe_var_error: var warn is unsafe

[BenjaminHerbert]

@boranx
The documentation says: "USING DENY_ AS A PREFIX TO SIMPLIFY TESTING"
So I guess it is not implemented for warn...