Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Export Type Registry for Reporting [POC/exploration] #155081

Closed
wants to merge 195 commits into from

Conversation

rshen91
Copy link
Contributor

@rshen91 rshen91 commented Apr 17, 2023

Summary

#152845
This PR will lend itself to closing (but doesn't close this) - #154675

The different export types from ReportingExportTypes plugin will register themselves within ReportingPlugin's export types registry and the registry will have the allowed export types for serverless.

Checklist

Delete any items that are not applicable to this PR.

@rshen91 rshen91 added the WIP Work in progress label Apr 17, 2023
@rshen91 rshen91 self-assigned this Apr 17, 2023
@elastic elastic deleted a comment from kibana-ci Apr 19, 2023
@rshen91 rshen91 requested a review from tsullivan May 8, 2023 12:58
Comment on lines 9 to 12
"configPath": [
"xpack",
"reportingExportTypes"
],
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like this causes the server to crash, since there are no plugins that export a config schema with this path.

For now, we can re-use the configPath from reporting:

Suggested change
"configPath": [
"xpack",
"reportingExportTypes"
],
"configPath": [
"xpack",
"reporting"
],

Comment on lines 50 to 82
/**
* Export types to the central reporting plugin
*/
type CreateFnType = CreateJobFn<any, any>; // can not specify params types because different type of params are not assignable to each other
type RunFnType = any; // can not specify because ImmediateExecuteFn is not assignable to RunTaskFn
const getTypeFns: Array<() => ExportTypeDefinition<CreateFnType | null, RunFnType>> = [
getTypeCsv,
getTypeCsvFromSavedObject,
getTypeCsvFromSavedObjectImmediate,
getTypePng,
getTypePngV2,
getTypePrintablePdf,
getTypePrintablePdfV2,
];
getTypeFns.forEach((getType) => {
if (
getType === getTypeCsv ||
getType === getTypeCsvFromSavedObject ||
getType === getTypeCsvFromSavedObjectImmediate
) {
return (
// setFieldFormats(fieldFormats),
reporting.registerExportType(
getType() as unknown as ExportTypeDefinition<CreateJobFn<any, any>, RunTaskFn<any>>
),
setFieldFormats(fieldFormats)
);
} else {
return reporting.registerExportType(
getType() as unknown as ExportTypeDefinition<CreateJobFn<any, any>, RunTaskFn<any>>
);
}
});
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can simplify this section, and make it more readable without the .forEach:

    /**
     * Export types to the central reporting plugin
     */
    reporting.registerExportType(getTypeCsv());
    reporting.registerExportType(getTypeCsvFromSavedObject());
    // @ts-ignore
    reporting.registerExportType(getTypeCsvFromSavedObjectImmediate());
    reporting.registerExportType(getTypePng());
    reporting.registerExportType(getTypePngV2());
    reporting.registerExportType(getTypePrintablePdf());
    reporting.registerExportType(getTypePrintablePdfV2());

I put the ts-ignore over the getTypeCsvFromSavedObjectImmediate because the definition for that export type is a strange one. But I think it's ok to ignore the TS error for now, since we want to remove that export type: #145937

}

// do nothing
public start({}, {}) {}
Copy link
Member

@tsullivan tsullivan May 8, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Within this lifecycle method, we want to make the setFieldFormats call:

Suggested change
public start({}, {}) {}
public start(_core: CoreStart, plugins: ExportTypesPluginStartDependencies) {
setFieldFormats(plugins.fieldFormats);
}

And ExportTypesPluginStartDependencies will just be:

export interface ExportTypesPluginStartDependencies {
  fieldFormats: FieldFormatsStart;
}

Comment on lines +89 to +105
public getPluginSetupDeps() {
if (!this.pluginSetupDeps) {
throw new Error(`"pluginSetupDeps" dependencies haven't initialized yet`);
}
return this.pluginSetupDeps;
}

/*
* Gives async access to the startDeps
*/
public async getPluginStartDeps() {
if (this.pluginStartDeps) {
return this.pluginStartDeps;
}

return await Rx.firstValueFrom(this.pluginStart$);
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we just need these 2 methods from this class?

- PNGExportTypeDefinition
- PDFExportTypeDefinition

The `ReportingPlugin` calls the `ReportingExportTypesPlugin` and creates an instance of the `ExportTypeRegistry`. These export types use the `ReportingPlugin`'s registerExportType() within the `ExportTypeRegistry`. The `ReportingPlugin` uses the getExportType() method to use the registered export types for exporting.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
The `ReportingPlugin` calls the `ReportingExportTypesPlugin` and creates an instance of the `ExportTypeRegistry`. These export types use the `ReportingPlugin`'s registerExportType() within the `ExportTypeRegistry`. The `ReportingPlugin` uses the getExportType() method to use the registered export types for exporting.
The `ReportingPlugin` creates an instance of the `ExportTypeRegistry` and exposes its `registerExportType()` to other plugins, which `ReportingExportTypesPlugin` uses to register all of the export types. The `ReportingPlugin` uses the getExportType() method to use the registered export types for exporting.

@@ -31,6 +31,7 @@ export function isConfigPath(value: unknown): value is ConfigPath {
* @internal
*/
export interface Config {
kbnConfig: any;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see the confusion here. Reporting has a strange object that gets passed around from a class called ReportingConfig. The reasons we do this go way back, but I believe that passing config around to the parts that need it might be simpler now.

Given the impact of this, I believe we should add a granular step #152845: Refactor ReportingConfig to just use a simple type.

@rshen91 Would you mind if we put this PR on pause while we look at that?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this simplifies the base code quite a lot and hopefully makes things in this PR easier: #157118.

@kibana-ci
Copy link
Collaborator

kibana-ci commented May 9, 2023

💔 Build Failed

Failed CI Steps

Test Failures

  • [job] [logs] FTR Configs #8 / apis home apis sample data apis list in the default space (before install) should return list of sample data sets with installed status
  • [job] [logs] FTR Configs #8 / apis home apis sample data apis list in the default space (before install) should return list of sample data sets with installed status
  • [job] [logs] Jest Integration Tests #4 / checking migration metadata changes on all registered SO types detecting migration related changes in registered types
  • [job] [logs] FTR Configs #4 / dashboard Reporting Dashboard Reporting Screenshots Preserve Layout downloads a PDF file: large dashboard
  • [job] [logs] FTR Configs #4 / dashboard Reporting Dashboard Reporting Screenshots Preserve Layout downloads a PDF file: large dashboard
  • [job] [logs] Jest Tests #5 / data modeling usage data with meta.isDeprecated jobTypes
  • [job] [logs] Jest Tests #5 / data modeling with empty data
  • [job] [logs] Jest Tests #5 / data modeling with metrics and execution times in the data
  • [job] [logs] Jest Tests #5 / data modeling with sparse data
  • [job] [logs] Jest Tests #5 / data modeling with usage data from the reporting/archived_reports es archive
  • [job] [logs] FTR Configs #23 / discover Discover CSV Export Generate CSV: new search generates a report from a new search with data: default
  • [job] [logs] FTR Configs #23 / discover Discover CSV Export Generate CSV: new search generates a report from a new search with data: default
  • [job] [logs] Jest Tests #7 / gets the csv content from job parameters
  • [job] [logs] Jest Tests #5 / Handle request to generate Enqueue Job creates a report object to queue
  • [job] [logs] Jest Tests #5 / Handle request to generate Enqueue Job provides a default kibana version field for older POST URLs
  • [job] [logs] Jest Tests #5 / Handle request to generate generates the download path
  • [job] [logs] Jest Tests #7 / headers fails if it can't decrypt headers
  • [job] [logs] Jest Integration Tests #2 / incompatible_cluster_routing_allocation retries the INIT action with a descriptive message when cluster settings are incompatible
  • [job] [logs] Jest Tests #5 / Incorporate error code stats
  • [job] [logs] Jest Tests #5 / Incorporate execution times
  • [job] [logs] Jest Tests #5 / Incorporate metric stats
  • [job] [logs] FTR Configs #34 / Interactive Setup Functional Tests (Manual configuration without TLS) should configure Kibana successfully
  • [job] [logs] FTR Configs #34 / Interactive Setup Functional Tests (Manual configuration without TLS) should configure Kibana successfully
  • [job] [logs] FTR Configs #38 / Interactive Setup Functional Tests (Manual configuration) should configure Kibana successfully
  • [job] [logs] FTR Configs #38 / Interactive Setup Functional Tests (Manual configuration) should configure Kibana successfully
  • [job] [logs] Jest Tests #5 / license checks with a basic license sets csv_searchsource available to true
  • [job] [logs] Jest Tests #5 / license checks with a basic license sets pdf availability to false
  • [job] [logs] Jest Tests #5 / license checks with gold license sets csv_searchsource available to true
  • [job] [logs] Jest Tests #5 / license checks with gold license sets pdf availability to true
  • [job] [logs] Jest Tests #5 / license checks with no license sets csv_searchsource available to false
  • [job] [logs] Jest Tests #5 / license checks with no license sets pdf availability to false
  • [job] [logs] Jest Tests #5 / license checks with no usage data sets csv_searchsource available to true
  • [job] [logs] Jest Integration Tests #2 / migrating from 7.3.0-xpack which used v1 migrations copies all the document of the previous index to the new one
  • [job] [logs] Jest Integration Tests #2 / migrating from 7.3.0-xpack which used v1 migrations creates the new index and the correct aliases
  • [job] [logs] Jest Integration Tests #2 / migrating from 7.3.0-xpack which used v1 migrations migrates the documents to the highest version
  • [job] [logs] Jest Integration Tests #3 / migration from 7.13 to 7.14+ with many failed action_tasks if mappings are incompatible (reindex required) filters out all outdated action_task_params and action tasks
  • [job] [logs] Jest Integration Tests #4 / migration v2 completes the migration even when a full batch would exceed ES http.max_content_length
  • [job] [logs] Jest Integration Tests #4 / migration v2 fails with a descriptive message when a single document exceeds maxBatchSizeBytes
  • [job] [logs] Jest Integration Tests #4 / migration v2 fails with a descriptive message when maxBatchSizeBytes exceeds ES http.max_content_length
  • [job] [logs] Jest Integration Tests #4 / migration v2 with corrupt saved object documents collects corrupt saved object documents across batches
  • [job] [logs] Jest Tests #5 / Model of jobTypes
  • [job] [logs] Jest Tests #5 / PNG counts, provided count of deprecated jobs explicitly
  • [job] [logs] FTR Configs #24 / Reporting API Integration Tests with Security disabled Job Listing APIs Posted CSV job is visible in the job count
  • [job] [logs] FTR Configs #24 / Reporting API Integration Tests with Security disabled Job Listing APIs Posted CSV job is visible in the job count
  • [job] [logs] FTR Configs #27 / Reporting APIs BWC report generation into existing indexes existing 6_2 index single job posted can complete in an index created with an older version
  • [job] [logs] FTR Configs #27 / Reporting APIs BWC report generation into existing indexes existing 6_2 index single job posted can complete in an index created with an older version
  • [job] [logs] FTR Configs #4 / Reporting Functional Tests with Deprecated Security configuration enabled Security with reporting_user built-in role Discover: Generate CSV does allow user with reporting_user role
  • [job] [logs] FTR Configs #4 / Reporting Functional Tests with Deprecated Security configuration enabled Security with reporting_user built-in role Discover: Generate CSV does allow user with reporting_user role
  • [job] [logs] FTR Configs #58 / Reporting Functional Tests with Security enabled Security with reporting_user built-in role Discover: Generate CSV does allow user with reporting privileges
  • [job] [logs] FTR Configs #58 / Reporting Functional Tests with Security enabled Security with reporting_user built-in role Discover: Generate CSV does allow user with reporting privileges
  • [job] [logs] Jest Integration Tests #2 / split .kibana index into multiple system indices when migrating from a legacy version performs v1 migration and then relocates saved objects into different indices, depending on their types

Metrics [docs]

✅ unchanged

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @rshen91

@tsullivan tsullivan changed the title Export Type Registry for Reporting Export Type Registry for Reporting [POC/exploration] May 17, 2023
@rshen91 rshen91 closed this May 25, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
(Deprecated) Feature:Reporting Use Reporting:Screenshot, Reporting:CSV, or Reporting:Framework instead release_note:skip Skip the PR/issue when compiling release notes WIP Work in progress
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants