Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docs for the audit logfile structured format #34584

Merged

Conversation

albertzaharovits
Copy link
Contributor

@albertzaharovits albertzaharovits commented Oct 18, 2018

This is the docs part of the following work on structured logfile auditing:
#31931
#34475

It documents the format and file name changes.

This PR is geared towards the 6.x branch.
It will be followed by another one PR that will completely forget
the deprecated format (and will describe this as a breaking change,
as has been suggested here #34475 (review))

@albertzaharovits albertzaharovits added >docs General docs changes v7.0.0 :Security/Audit X-Pack Audit logging v6.5.0 labels Oct 18, 2018
@albertzaharovits albertzaharovits self-assigned this Oct 18, 2018
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-security

docs/reference/settings/audit-settings.asciidoc Outdated Show resolved Hide resolved
docs/reference/settings/audit-settings.asciidoc Outdated Show resolved Hide resolved
the `<clustername>_audit.log` file in the logs directory. To maintain
compatibility with releases prior to 6.5.0, another file `<clustername>_access.log`
is also generated next to it. They differ in the output format but the contents
are similar. For systems that had not automatically ingesting the audit file
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/systems that had not automatically ingesting/systems that are not ingesting

Copy link
Member

@jaymode jaymode left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

the `<clustername>_access.log` file with the format for versions prior to 6.5.0.
If the backwards compatible format is not required, it should be disabled.
To do that, change its logger level to `off` in the `log4j2.properties` file
from `ES_PATH_CONF`.
Copy link
Contributor

@lcawl lcawl Oct 23, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest changing this to something like this:
"... in the log4j2.properties file. For more information, see <<configuring-logging-levels>>. "


`xpack.security.audit.logfile.emit_node_id`::
Specifies whether to include the node id as a field in each audit event.
This is available for the new format only.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it worth adding "that is to say, this information does not exist in the <clustername>_access.log file"?

This is available for the new format only.
Unlike <<node-name, node name>>, whose value might change if the administrator
changes the setting in the config file, the node id will persist across cluster
restarts and the administrator has no handle to change it.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The "has no handle to change it" is a bit unclear to me. Can we just say, "cannot change it"?

to 6.5.0. Between the two, the contents are identical, therefore it is recommended
to disable the `<clustername>_access.log` output. For this, turn off the
logger in the `log4j2.properties` config file from `ES_PATH_CONF`. (hint: look
out for a comment in the config file).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This might be too much information for such a high-level introductory page. i.e. since it's covered in the linked logfile page, I think you could just cut off after the "...is also generated" sentence.

Specifies whether to include the node's host name in the local node info. The
default value is `false`.
`xpack.security.audit.logfile.emit_node_name`::
Specifies whether to include the <<node-name, node name>> as a field in each
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This link fails when I build the documentation. Are you trying to link to https://www.elastic.co/guide/en/elasticsearch/reference/master/node.name.html ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

replaced with {ref}/node.name.html[node name]

`xpack.security.audit.logfile.emit_node_id`::
Specifies whether to include the node id as a field in each audit event.
This is available for the new format only.
Unlike <<node-name, node name>>, whose value might change if the administrator
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This "node-name" link also fails. If you let me know what you're trying to link to, I can help fix it

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

replaced with {ref}/node.name.html[node name]

Copy link
Contributor

@lcawl lcawl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've submitted some suggestions, but once those are applied or discarded (per https://help.github.com/articles/incorporating-feedback-in-your-pull-request/#applying-a-suggested-change), it LGTM

x-pack/docs/en/security/auditing/output-logfile.asciidoc Outdated Show resolved Hide resolved
x-pack/docs/en/security/auditing/output-logfile.asciidoc Outdated Show resolved Hide resolved
x-pack/docs/en/security/auditing/output-logfile.asciidoc Outdated Show resolved Hide resolved
docs/reference/settings/audit-settings.asciidoc Outdated Show resolved Hide resolved
docs/reference/settings/audit-settings.asciidoc Outdated Show resolved Hide resolved
lcawl and others added 5 commits October 23, 2018 23:23
Co-Authored-By: albertzaharovits <albert.zaharovits@gmail.com>
Co-Authored-By: albertzaharovits <albert.zaharovits@gmail.com>
Co-Authored-By: albertzaharovits <albert.zaharovits@gmail.com>
Co-Authored-By: albertzaharovits <albert.zaharovits@gmail.com>
Co-Authored-By: albertzaharovits <albert.zaharovits@gmail.com>
@colings86 colings86 added v6.6.0 and removed v6.5.0 labels Oct 25, 2018
@albertzaharovits albertzaharovits merged commit 3f1fec1 into elastic:master Oct 26, 2018
@albertzaharovits albertzaharovits deleted the docs-structured-audit-only branch October 26, 2018 12:19
albertzaharovits added a commit that referenced this pull request Oct 26, 2018
Documents the new structured logfile format for auditing
that was introduced by #31931. Most changes herein
are for 6.x . In 7.0 the deprecated format is gone and a
follow-up PR is in order.
albertzaharovits added a commit that referenced this pull request Oct 26, 2018
Documents the new structured logfile format for auditing
that was introduced by #31931. Most changes herein
are for 6.x . In 7.0 the deprecated format is gone and a
follow-up PR is in order.
jasontedor added a commit to jasontedor/elasticsearch that referenced this pull request Oct 26, 2018
* 'master' of github.com:elastic/elasticsearch:
  Fix line length for org.elasticsearch.common.* files (elastic#34888)
  [ML] Extract common native process base class (elastic#34856)
  Refactor children aggregator into a generic ParentJoinAggregator (elastic#34845)
  [Style] Fix line lengths in action.admin.indices (elastic#34890)
  HLRC - add support for source exists API (elastic#34519)
  [CCR] Retry when no index shard stats can be found (elastic#34852)
  [Docs] audit logfile structured format (elastic#34584)
  [Test] Fix FullClusterRestartIT.testShrink() with copy_settings param (elastic#34853)
  Fix LineLength Check Suppressions: index.fielddata (elastic#34891)
  TEST: Stablize Minio Free Port Search (elastic#34894)
  Delete flaky SettingsBasedHostProviderIT test (elastic#34813)
  [ML] Include message in field_stats for text log files (elastic#34861)
  [TEST] HLRC: Expand failure messages in API checks (elastic#34838)
  Lowercase static final DeprecationLogger instance names (elastic#34887)
kcm pushed a commit that referenced this pull request Oct 30, 2018
Documents the new structured logfile format for auditing
that was introduced by #31931. Most changes herein
are for 6.x . In 7.0 the deprecated format is gone and a
follow-up PR is in order.
@colings86 colings86 added v6.5.0 and removed v6.5.1 labels Nov 2, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
>docs General docs changes :Security/Audit X-Pack Audit logging v6.5.0 v6.6.0 v7.0.0-beta1
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants