diff --git a/docs/settings/reporting-settings.asciidoc b/docs/settings/reporting-settings.asciidoc index f871f72db22c0a..87ee0b9bac099c 100644 --- a/docs/settings/reporting-settings.asciidoc +++ b/docs/settings/reporting-settings.asciidoc @@ -245,6 +245,7 @@ Choose the API method used to page through data during CSV export. Valid options [NOTE] ============ Each method has its own unique limitations which are important to understand. + * Scroll API: Search is limited to 500 shards at the very most. In cases where data shards are unavailable or time out, the export may return partial data. * PIT API: Permissions to read data aliases alone will not work: the permissions are needed on the underlying indices or datastreams. In cases where data shards are unavailable or time out, the export will be empty rather than returning partial data. ============ diff --git a/docs/user/reporting/index.asciidoc b/docs/user/reporting/index.asciidoc index 338b2bc53a55a9..5075caad718134 100644 --- a/docs/user/reporting/index.asciidoc +++ b/docs/user/reporting/index.asciidoc @@ -171,3 +171,5 @@ NOTE: *Public URL* is available only when anonymous access is configured and you include::automating-report-generation.asciidoc[] include::reporting-troubleshooting.asciidoc[] +include::reporting-csv-troubleshooting.asciidoc[leveloffset=+1] +include::reporting-pdf-troubleshooting.asciidoc[leveloffset=+1] \ No newline at end of file diff --git a/docs/user/reporting/reporting-csv-troubleshooting.asciidoc b/docs/user/reporting/reporting-csv-troubleshooting.asciidoc new file mode 100644 index 00000000000000..a0eb782ce3d3b0 --- /dev/null +++ b/docs/user/reporting/reporting-csv-troubleshooting.asciidoc @@ -0,0 +1,90 @@ +[[reporting-troubleshooting-csv]] +== Troubleshooting CSV reports +++++ +CSV +++++ + +The CSV export feature in Kibana makes queries to Elasticsearch and formats the results into CSV. +This feature offers a solution that attempts to provide the most benefit to the most use cases. +However, things could go wrong during export. +Elasticsearch can stop responding, repeated querying can take so long that authentication tokens can time +out, and the format of exported data can be too complex for spreadsheet applications to handle. +Such situations are outside of the control of Kibana. +If the use case becomes complex enough, it's recommended that you create scripts that query Elasticsearch directly, using a scripting language like Python and the public {es} APIs. + +For more advice about common problems, refer to <>. + +[NOTE] +============ +It is recommended that you use CSV reports to export moderate amounts of data only. +The feature enables analysis of data in external tools, but it's not intended for bulk export or to backup {es} data. +If you need to export more than 250 MB of CSV, rather than increasing <>, use +filters to create multiple smaller reports or extract the data you need directly from {es}. + +The following deployment configurations may lead to failed report jobs or incomplete reports: + +* Any shard needed for search is unavailable. +* Data is stored on slow storage tiers. +* Network latency between nodes is high. +* {ccs-cap} is used. + +To export large amounts of data, use {es} APIs directly. +Check out the {ref}/point-in-time-api.html[Point in time API] or {ref}/sql-rest-format.html#_csv[SQL with CSV response data format]. +============ + +[float] +[[reporting-troubleshooting-csv-configure-scan-api]] +=== Configuring CSV export to use the scroll API + +The Kibana CSV export feature collects all of the data from Elasticsearch by using multiple requests to page +over all of the documents. +Internally, the feature uses the {ref}/point-in-time-api.html[Point in time API and +`search_after` parameters in the queries] to do so. +There are some limitations related to the point in time API: + +1. Permissions to read data aliases alone will not work: the permissions are needed on the underlying indices or data streams. +2. In cases where data shards are unavailable or time out, the export will be empty rather than returning partial data. + +Some users may benefit from using the {ref}/paginate-search-results.html#scroll-search-results[scroll API], an +alternative to paging through the data. +The behavior of this API does not have the limitations of point in time API, however it has its own limitations: + +1. Search is limited to 500 shards at the very most. +2. In cases where the data shards are unavailable or time out, the export may return partial data. + +If you prefer the internal implementation of CSV export to use the scroll API, you can configure this in +`kibana.yml`: + +[source,yml] +------------------------------------------- +xpack.reporting.csv.scroll.strategy: scroll +------------------------------------------- + +For more details about CSV export settings, go to <>. + +[float] +[[reporting-troubleshooting-csv-socket-hangup]] +=== Socket hangups + +A "socket hangup" is a generic type of error meaning that a remote service (in this case Elasticsearch or a proxy in Cloud) closed the connection. +Kibana can't foresee when this might happen and can't force the remote service to keep the connection open. +To work around this situation, consider lowering the size of results that come back in each request or increase the amount of time the remote services will +allow to keep the request open. +For example: + +[source,yml] +--------------------------------------- +xpack.reporting.csv.scroll.size: 50 +xpack.reporting.csv.scroll.duration: 2m +--------------------------------------- + +Such changes aren't guaranteed to solve the issue, but give the functionality a better +chance of working in this use case. +Unfortunately, lowering the scroll size will require more requests to Elasticsearch during export, which adds more time overhead, which could unintentionally create more instances of auth token expiration errors. + +[float] +[[reporting-troubleshooting-csv-token-expired]] +=== Token expiration + +To avoid token expirations, use a type of authentication that doesn't expire (such as Basic auth) or run the export using scripts that query Elasticsearch directly. +In a custom script, you have the ability to refresh the auth token as needed, such as once before each query. diff --git a/docs/user/reporting/reporting-pdf-troubleshooting.asciidoc b/docs/user/reporting/reporting-pdf-troubleshooting.asciidoc new file mode 100644 index 00000000000000..9ea3ff6aa3721a --- /dev/null +++ b/docs/user/reporting/reporting-pdf-troubleshooting.asciidoc @@ -0,0 +1,136 @@ +[[reporting-troubleshooting-pdf]] +== Troubleshooting PDF and PNG reports +++++ +PDF/PNG +++++ + +For the most reliable configuration of PDF/PNG {report-features}, consider installing {kib} using <> or using <>. + +For more advice about common problems, refer to <>. + +[float] +[[reporting-diagnostics]] +=== Reporting diagnostics +Reporting comes with a built-in utility to try to automatically find common issues. +When {kib} is running, navigate to the *Report Listing* page, and click *Run reporting diagnostics*. +This will open up a diagnostic tool that checks various parts of the {kib} deployment and comes up with any relevant recommendations. + +If the diagnostic information doesn't reveal the problem, you can troubleshoot further by starting the Kibana server with an environment variable for revealing additional debugging logs. +Refer to <>. + +[float] +[[reporting-troubleshooting-nss-dependency]] +=== Network security service libraries + +You must install Network Security Service (NSS) libraries for {report-features} to work. +Reporting using the Chromium browser relies on these libraries. +Install the appropriate nss package for your distribution. +Refer to <>. + +[float] +[[reporting-troubleshooting-sandbox-dependency]] +=== Chromium sandbox requirements + +Chromium uses sandboxing techniques that are built on top of operating system primitives. +The Linux sandbox depends on user namespaces, which were introduced with the 3.8 Linux kernel. +However, many distributions don't have user namespaces enabled by default or they require the CAP_SYS_ADMIN capability. +If the sandbox is not explicitly disabled in Kibana, either based on operating system detection or with the `xpack.screenshotting.browser.chromium.disableSandbox` setting, Chrome will try to enable the sandbox. +If it fails due to operating system or permissions restrictions, Chrome will crash during initialization. + +Elastic recommends that you research the feasibility of enabling unprivileged user namespaces before disabling the sandbox. +An exception is if you are running Kibana in Docker because the container runs in a user namespace with the built-in seccomp/bpf filters. + +[float] +[[reporting-troubleshooting-text-incorrect]] +=== Text rendered incorrectly in generated reports + +If a report label is rendered as an empty rectangle, no system fonts are available. +Install at least one font package on the system. + +If the report is missing certain Chinese, Japanese or Korean characters, ensure that a system font with those characters is installed. + +[float] +[[reporting-troubleshooting-missing-data]] +=== Missing data in PDF report of data table visualization + +There is currently a known limitation with the data table visualization that only the first page of data rows, which are the only data +visible on the screen, are shown in PDF reports. + +[float] +[[reporting-troubleshooting-pdf-connection-refused]] +=== Connection refused errors + +If PDF or PNG reports are not working due to a "connection refused" or "unable to connect" type of error, ensure that the `kibana.yml` +file uses the setting of `server.host: 0.0.0.0`. +Also verify that no firewall rules or other routing rules prevent local services from accessing this address. +Find out more at <>. + +[float] +[[reporting-troubleshooting-file-permissions]] +=== File permissions + +Ensure that the `headless_shell` binary located in your Kibana data directory is owned by the user who is running Kibana, that the +user has the execute permission, and if applicable, that the filesystem is mounted with the `exec` option. + +[NOTE] +-- +The Chromium binary is located in the Kibana installation directory as `data/headless_shell-OS_TYPE/headless_shell`. +The full path is logged the first time Kibana starts when verbose logging is enabled. +-- + +[float] +[[reporting-troubleshooting-puppeteer-debug-logs]] +=== Puppeteer debug logs + +The Chromium browser that {kib} launches on the server is driven by a NodeJS library for Chromium called Puppeteer. +The Puppeteer library has its own command-line method to generate its own debug logs, which can sometimes be helpful, particularly to figure out if a problem is caused by Kibana or Chromium. +Learn more https://github.com/GoogleChrome/puppeteer/blob/v1.19.0/README.md#debugging-tips[debugging tips]. + +Using Puppeteer's debug method when launching Kibana would look like: +``` +env DEBUG="puppeteer:*" ./bin/kibana +``` +The internal DevTools protocol traffic will be logged via the `debug` module under the `puppeteer` namespace. + +The Puppeteer logs are very verbose and could possibly contain sensitive information. +Handle the generated output with care. + +[float] +[[reporting-troubleshooting-system-requirements]] +=== System requirements + +In Elastic Cloud, the {kib} instances that most configurations provide by default is for 1GB of RAM for the instance. +That is enough for {kib} {report-features} when the visualization or dashboard is relatively simple, such as a single pie chart or a dashboard with a few visualizations. +However, certain visualization types incur more load than others. +For example, a TSVB panel has a lot of network requests to render. + +If the {kib} instance doesn't have enough memory to run the report, the report fails with an error such as `Error: Page crashed!`. +In this case, try increasing the memory for the {kib} instance to 2GB. + +[float] +[[reporting-troubleshooting-maps-ems]] +=== Unable to connect to Elastic Maps Service + +https://www.elastic.co/elastic-maps-service[{ems} ({ems-init})] is a service that hosts tile layers and vector shapes of administrative boundaries. +If a report contains a map with a missing basemap layer or administrative boundary, the {kib} server does not have access to {ems-init}. +Refer to <> for information about how to connect your {kib} server to {ems-init}. + +[float] +[[reporting-manual-chromium-install]] +=== Manually install the Chromium browser for Darwin + +Chromium is not embedded into {kib} for the Darwin (Mac OS) architecture. +When running {kib} on Darwin, {report-features} will download Chromium into the proper area of the {kib} installation path the first time the server starts. +If the server does not have access to the internet, you must download the Chromium browser and install it into the {kib} installation path. + +1. Download the Chromium zip file: + +** For https://commondatastorage.googleapis.com/chromium-browser-snapshots/Mac/901912/chrome-mac.zip[x64] systems +** For https://commondatastorage.googleapis.com/chromium-browser-snapshots/Mac_Arm/901913/chrome-mac.zip[ARM] systems + +2. Copy the zip file into the holding area. Relative to the root directory of {kib}, the path is: + +** `.chromium/x64` for x64 systems +** `.chromium/arm64` for ARM systems + +When {kib} starts, it will automatically extract the browser from the zip file and is then ready for PNG and PDF reports. diff --git a/docs/user/reporting/reporting-troubleshooting.asciidoc b/docs/user/reporting/reporting-troubleshooting.asciidoc index 72b94c02a6ebc3..814c2e48f15d98 100644 --- a/docs/user/reporting/reporting-troubleshooting.asciidoc +++ b/docs/user/reporting/reporting-troubleshooting.asciidoc @@ -1,73 +1,31 @@ -[role="xpack"] [[reporting-troubleshooting]] -== Reporting troubleshooting +== Troubleshooting {report-features} ++++ Troubleshooting ++++ -Having trouble? Here are solutions to common problems you might encounter while using Reporting. +Kibana excels as a data visualization tool. The {report-features} exist to export data as simple reports, however Kibana is not a data export tool. +To export data at a large scale, there are better ways and better architectures for exporting data at scale from Elasticsearch. -* <> -* <> -* <> -* <> -* <> -* <> -* <> -* <> -* <> - -[float] -[[reporting-diagnostics]] -=== Reporting diagnostics -Reporting comes with a built-in utility to try to automatically find common issues. When {kib} is running, -navigate to the Report Listing page, and click *Run reporting diagnostics*. This will open up a diagnostic tool -that checks various parts of the {kib} deployment and come up with any relevant recommendations. - -If the diagnostic information doesn't reveal the problem, you can troubleshoot further by starting the Kibana -server with an environment variable for revealing additional debugging logs. Refer to -<>. - -[float] -[[reporting-troubleshooting-text-incorrect]] -=== Text rendered incorrectly in generated reports - -If a report label is rendered as an empty rectangle, no system fonts are available. Install at least one font package on the system. - -If the report is missing certain Chinese, Japanese or Korean characters, ensure that a system font with those characters is installed. - -[float] -[[reporting-troubleshooting-missing-data]] -=== Missing data in PDF report of data table visualization -There is currently a known limitation with the Data Table visualization that only the first page of data rows, which are the only data -visible on the screen, are shown in PDF reports. - -[float] -[[reporting-troubleshooting-file-permissions]] -=== File permissions -Ensure that the `headless_shell` binary located in your Kibana data directory is owned by the user who is running Kibana, that the -user has the execute permission, and if applicable, that the filesystem is mounted with the `exec` option. - -[NOTE] --- -The Chromium binary is located in the Kibana installation directory as `data/headless_shell-OS_TYPE/headless_shell`. The full path is logged -the first time Kibana starts when verbose logging is enabled. --- +If you have trouble creating simple reports, there are some general solutions to common problems you might encounter while using {report-features}. +For tips related to specific types of reports, refer to <> and <>. [float] [[reporting-troubleshooting-error-messages]] === Error messages -Whenever possible, a Reporting error message tries to be as self-explanatory as possible. Here are some error messages you might encounter, -along with the solution. + +There are some common solutions for error messages that you might encounter in {report-features}. [float] -==== `StatusCodeError: [version_conflict_engine_exception]` -If you are running multiple instances of {kib} in a cluster, the instances share the work of executing report jobs to evenly distribute -the work load. Each instance searches the reporting index for "pending" jobs that the user has requested. It is possible for -multiple instances to find the same job in these searches. Only the instance that successfully updated the job status to -"processing" will actually execute the report job. The other instances that unsuccessfully tried to make the same update will log -something similar to this: +[[reporting-troubleshooting-version-conflict-exception]] +==== Version conflict engine exceptions + +If you are running multiple instances of {kib} in a cluster, the instances share the work of running report jobs to evenly distribute the workload. +Each instance searches the reporting index for "pending" jobs that the user has requested. +It is possible for multiple instances to find the same job in these searches. +Only the instance that successfully updated the job status to "processing" will actually run the report job. +The other instances that unsuccessfully tried to make the same update will log something similar to this: [source,text] -------------------------------------------------------------------------------- @@ -85,103 +43,34 @@ StatusCodeError: [version_conflict_engine_exception] [...]: version conflict, re } -------------------------------------------------------------------------------- -These messages alone don't indicate a problem. They show normal events that happen in a healthy system. +These messages alone don't indicate a problem. +They show normal events that happen in a healthy system. [float] ==== Max attempts reached -There are two primary causes of this error: - -* You're creating a PDF of a visualization or dashboard that spans a large amount of data and Kibana is hitting the `xpack.reporting.queue.timeout` - -* Kibana is hosted behind a reverse-proxy, and the <> are not configured correctly -Create a Markdown visualization and then create a PDF report. If this succeeds, increase the `xpack.reporting.queue.timeout` setting. If the -PDF report fails with "Max attempts reached," check your <>. +There are two primary causes for a "Max attempts reached" error: -[float] -[[reporting-troubleshooting-nss-dependency]] -==== You must install nss for Reporting to work -Reporting using the Chromium browser relies on the Network Security Service libraries (NSS). Install the appropriate nss package for your -distribution. +* You're creating a PDF of a visualization or dashboard that spans a large amount of data and Kibana is hitting the `xpack.reporting.queue.timeout` -[float] -[[reporting-troubleshooting-sandbox-dependency]] -==== Unable to use Chromium sandbox -Chromium uses sandboxing techniques that are built on top of operating system primitives. The Linux sandbox depends on user namespaces, -which were introduced with the 3.8 Linux kernel. However, many distributions don't have user namespaces enabled by default, or they require -the CAP_SYS_ADMIN capability. If the sandbox is not explicitly disabled in Kibana, either based on OS detection or with the -`xpack.screenshotting.browser.chromium.disableSandbox` setting, Chrome will try to enable the sandbox. If it fails due to OS or permissions -restrictions, Chrome will crash during initialization. +* Kibana is hosted behind a reverse-proxy, and the <> are not configured correctly -Elastic recommends that you research the feasibility of enabling unprivileged user namespaces before disabling the sandbox. An exception -is if you are running Kibana in Docker because the container runs in a user namespace with the built-in seccomp/bpf filters. +Create a Markdown visualization and then create a PDF report. +If this succeeds, increase the `xpack.reporting.queue.timeout` setting. +If the +PDF report fails with "Max attempts reached," check your <>. [float] [[reporting-troubleshooting-verbose-logs]] -=== Verbose logs -{kib} server logs have a lot of useful information for troubleshooting and understanding how things work. If you're having any issues at -all, the full logs from Reporting will be the first place to look. In `kibana.yml`: +=== Verbose logging + +{kib} server logs have a lot of useful information for troubleshooting and understanding how things work. +The full logs from {report-features} are a good place to look when you encounter problems. +In `kibana.yml`: [source,yaml] -------------------------------------------------------------------------------- logging.root.level: all -------------------------------------------------------------------------------- -For more information about logging, see <>. - -[float] -[[reporting-troubleshooting-puppeteer-debug-logs]] -=== Puppeteer debug logs -The Chromium browser that {kib} launches on the server is driven by a NodeJS library for Chromium called Puppeteer. The Puppeteer library -has its own command-line method to generate its own debug logs, which can sometimes be helpful, particularly to figure out if a problem is -caused by Kibana or Chromium. See more at https://github.com/GoogleChrome/puppeteer/blob/v1.19.0/README.md#debugging-tips[debugging tips]. - -Using Puppeteer's debug method when launching Kibana would look like: -``` -env DEBUG="puppeteer:*" ./bin/kibana -``` -The internal DevTools protocol traffic will be logged via the `debug` module under the `puppeteer` namespace. - - -The Puppeteer logs are very verbose and could possibly contain sensitive information. Handle the generated output with care. - -[float] -[[reporting-troubleshooting-system-requirements]] -=== System requirements -In Elastic Cloud, the {kib} instances that most configurations provide by default is for 1GB of RAM for the instance. That is enough for -{kib} Reporting when the visualization or dashboard is relatively simple, such as a single pie chart or a dashboard with -a few visualizations. However, certain visualization types incur more load than others. For example, a TSVB panel has a lot of network -requests to render. - -If the {kib} instance doesn't have enough memory to run the report, the report fails with an error such as `Error: Page crashed!` -In this case, try increasing the memory for the {kib} instance to 2GB. - -[float] -[[reporting-troubleshooting-maps-ems]] -=== Unable to connect to Elastic Maps Service - -https://www.elastic.co/elastic-maps-service[{ems} ({ems-init})] is a service that hosts -tile layers and vector shapes of administrative boundaries. -If a report contains a map with a missing basemap layer or administrative boundary, the {kib} server does not have access to {ems-init}. -See <> for information on how to connect your {kib} server to {ems-init}. - -[float] -[[reporting-manual-chromium-install]] -=== Manually install the Chromium browser for Darwin -Chromium is not embedded into {kib} for the Darwin (Mac OS) architecture. When -running {kib} on Darwin, Reporting will download Chromium into the proper area of -the {kib} installation path the first time the server starts. If the server -does not have access to the Internet, you must download the -Chromium browser and install it into the {kib} installation path. - -1. Download the Chromium zip file: - -** For https://commondatastorage.googleapis.com/chromium-browser-snapshots/Mac/901912/chrome-mac.zip[x64] systems -** For https://commondatastorage.googleapis.com/chromium-browser-snapshots/Mac_Arm/901913/chrome-mac.zip[ARM] systems - -2. Copy the zip file into the holding area. Relative to the root directory of {kib}, the path is: - -** `.chromium/x64` for x64 systems -** `.chromium/arm64` for ARM systems - -When {kib} starts, it will automatically extract the browser from the zip file, and is then ready for PNG and PDF reports. +For more information about logging, check out <>. diff --git a/packages/kbn-doc-links/src/get_doc_links.ts b/packages/kbn-doc-links/src/get_doc_links.ts index 751f1ac5ea866d..6d8b18bb5d714b 100644 --- a/packages/kbn-doc-links/src/get_doc_links.ts +++ b/packages/kbn-doc-links/src/get_doc_links.ts @@ -686,7 +686,7 @@ export const getDocLinks = ({ kibanaBranch, buildFlavor }: GetDocLinkOptions): D cloudMinimumRequirements: `${KIBANA_DOCS}reporting-getting-started.html#reporting-on-cloud-resource-requirements`, grantUserAccess: `${KIBANA_DOCS}secure-reporting.html#grant-user-access`, browserSystemDependencies: `${KIBANA_DOCS}secure-reporting.html#install-reporting-packages`, - browserSandboxDependencies: `${KIBANA_DOCS}reporting-troubleshooting.html#reporting-troubleshooting-sandbox-dependency`, + browserSandboxDependencies: `${KIBANA_DOCS}reporting-troubleshooting-pdf.html#reporting-troubleshooting-sandbox-dependency`, }, security: { apiKeyServiceSettings: `${ELASTICSEARCH_DOCS}security-settings.html#api-key-service-settings`,