Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ML] Add Anomaly Explorer charts embeddable #94396

Merged
merged 84 commits into from
Mar 26, 2021
Merged
Show file tree
Hide file tree
Changes from 18 commits
Commits
Show all changes
84 commits
Select commit Hold shift + click to select a range
68a27e0
[ML] Add AnomalyExplorerEmbeddable
qn895 Mar 3, 2021
118b483
[ML] Add AnomalyExplorerService
qn895 Mar 4, 2021
09e4ea7
[ML] Add AnomalyExplorerService
qn895 Mar 4, 2021
1c0cd20
[ML] Clean up
qn895 Mar 4, 2021
2e1fbd7
[ML] Update set up flyout
qn895 Mar 8, 2021
dc6f52d
[ML] Update so queries work with anomaly explorer embeddable
qn895 Mar 8, 2021
eb86034
[ML] Remove swimlane
qn895 Mar 8, 2021
bd9728c
[ML] Rename var for clarity & typescript updates
qn895 Mar 8, 2021
cc34171
[ML] Fix loadDataForCharts cancelling out promises in embeddables
qn895 Mar 9, 2021
c5eb3db
[ML] Clean up input
qn895 Mar 9, 2021
fdc61d8
[ML] Add back editExplorerPanelAction
qn895 Mar 9, 2021
8ac33eb
[ML] clean up dependencies
qn895 Mar 9, 2021
a27052b
[ML] Add influencer filter, update types, refactor anomalyexplorer se…
qn895 Mar 11, 2021
38993cf
[ML] Use ExplorerChartsData
qn895 Mar 11, 2021
26d1f0d
[ML] Remove old explorer charts container service, rename anomaly_exp…
qn895 Mar 11, 2021
834070d
[ML] Update texts
qn895 Mar 11, 2021
36de5b3
[ML] Add back console
qn895 Mar 11, 2021
c52fa43
[ML] Add link for explorer
qn895 Mar 11, 2021
50ce97a
[ML] Correct ENTITY_FIELD_OPERATIONS typo
qn895 Mar 11, 2021
5db8eb1
[ML] Remove hardcoded index, move processFilters
qn895 Mar 11, 2021
7a815f8
[ML] Add const DEFAULT_MAX_SERIES_TO_PLOT
qn895 Mar 11, 2021
4609cf1
[ML] Remove operation, add comment
qn895 Mar 11, 2021
55d8224
[ML] Add InfluencersFilterQuery
qn895 Mar 11, 2021
dbcf42f
Merge remote-tracking branch 'upstream/master' into ml-7.13-explorer-…
qn895 Mar 11, 2021
6645dc8
Merge remote-tracking branch 'upstream/master' into ml-7.13-explorer-…
qn895 Mar 14, 2021
d8ad8dc
[ML] Rename explorer embeddable to explorer charts, Add to dashboard …
qn895 Mar 14, 2021
1e43dfe
[ML] Add +/- filter button to explorer page
qn895 Mar 14, 2021
28380e3
[ML] Fix double stroke on anomaly explorer charts
qn895 Mar 14, 2021
e7dfd27
[ML] Update notes for influencer filter buttons
qn895 Mar 14, 2021
ab011e1
[ML] Remove mlResultsService cache dependency
qn895 Mar 14, 2021
2c6a4d2
[ML] Remove anomalyTimelineService from explorer charts embeddable
qn895 Mar 14, 2021
43b074f
[ML] Fix styling for influencer filter
qn895 Mar 15, 2021
6cc91ec
[ML] Remove AnomalyTimelineService
qn895 Mar 15, 2021
1e4fab6
[ML] Fix url query
qn895 Mar 15, 2021
e0121d2
[ML] Fix so severityThreshold is retained
qn895 Mar 15, 2021
c98c22f
[ML] Fix i18n duplicates
qn895 Mar 15, 2021
bcd7358
[ML] Fix overflowY
qn895 Mar 15, 2021
be5d140
[ML] Remove AnomaliesContextMenu - to add back in a follow up
qn895 Mar 15, 2021
76dee18
[ML] Remove recentlyAccessed dependency
qn895 Mar 15, 2021
fc75226
[ML] Add initializeOutput method
qn895 Mar 15, 2021
47509f8
[ML] Replace charts service getCombinedJobs
qn895 Mar 15, 2021
956c053
[ML] Remove unused getTimefilter
qn895 Mar 15, 2021
22869a7
[ML] Move processFilter tests
qn895 Mar 15, 2021
9fa4765
Merge remote-tracking branch 'upstream/master' into ml-7.13-explorer-…
qn895 Mar 15, 2021
f5cba51
[ML] Change getTimeBounds back to private, remove redundant stylings
qn895 Mar 15, 2021
6784400
[ML] Fix missing recentlyAccessed in test
qn895 Mar 16, 2021
2aa489c
[ML] Add tests
qn895 Mar 17, 2021
88ac488
[ML] Add dts mlExplorerChartsContainer to avoid conflict
qn895 Mar 17, 2021
c772091
Merge remote-tracking branch 'upstream/master' into ml-7.13-explorer-…
qn895 Mar 17, 2021
2c29069
[ML] Fix test
qn895 Mar 18, 2021
c2fc3b8
[ML] Remove Influencer in filter label
qn895 Mar 18, 2021
4c0419c
[ML] Fix typescript
qn895 Mar 18, 2021
3c31e37
[ML] Rename Explorer to AnomalyCharts for embeddable
qn895 Mar 21, 2021
1541ea5
[ML] Update InfluencerFilter to EntityFilter
qn895 Mar 18, 2021
cb43b45
[ML] Fix aria-labels and texts for accessibility
qn895 Mar 21, 2021
3b3e468
[ML] Add fallback for suspense
qn895 Mar 21, 2021
0c25f29
[ML] Add functional tests and accessibility tests
qn895 Mar 22, 2021
6e81348
[ML] Fix explorer color
qn895 Mar 22, 2021
e2ded1b
Merge remote-tracking branch 'upstream/master' into ml-7.13-explorer-…
qn895 Mar 22, 2021
258c279
[ML] Fix test functional incorrectly renamed
qn895 Mar 22, 2021
fa75f4b
Merge remote-tracking branch 'upstream/master' into ml-7.13-explorer-…
qn895 Mar 22, 2021
9019ab6
[ML] Rename display names
qn895 Mar 22, 2021
8bdd67b
[ML] Update callout texts for dashboard, rename file
qn895 Mar 22, 2021
a690685
[ML] Rename entityFilters to entityFilter
qn895 Mar 23, 2021
14d383b
[ML] Rename to onChange, EntityField type
qn895 Mar 23, 2021
7586225
[ML] Rename to onChange, EntityField type
qn895 Mar 23, 2021
cef7fa9
[ML] Move kibanaContextMock, timeBucketsMock
qn895 Mar 23, 2021
d211a5d
[ML] Move mlResultsServiceMock
qn895 Mar 23, 2021
1e93306
[ML] Update contract mocks to its respective places
qn895 Mar 23, 2021
3940fd2
[ML] Revert change to isCombinedJobWithStats
qn895 Mar 23, 2021
3e2f4fc
[ML] Fix chart sizing issue with blank space
qn895 Mar 23, 2021
7ebe8b3
[ML] Add mapdata type
qn895 Mar 23, 2021
da4ad4a
[ML] Change forDashboard to noTimelineSelectionMsg
qn895 Mar 23, 2021
a87c426
[ML] Remove some cache dependencies in explorer page
qn895 Mar 23, 2021
4404ffc
Merge remote-tracking branch 'upstream/master' into ml-7.13-explorer-…
qn895 Mar 23, 2021
6b5f157
[ML] add type for seriesToPlot, add before, rename expect to assert
qn895 Mar 23, 2021
dbe8cee
[ML] Change signature of filter entity to be the same
qn895 Mar 24, 2021
d20c95a
[ML] Change to passing actual msg instead of boolean
qn895 Mar 24, 2021
7b31acf
Merge upstream/master into branch
qn895 Mar 24, 2021
673eafe
[ML] Update tests
qn895 Mar 25, 2021
2e485b3
Merge upstream/master into branch
qn895 Mar 25, 2021
82e02c6
[ML] Add common input resolver
qn895 Mar 25, 2021
32d70bf
[ML] Add expected error
qn895 Mar 25, 2021
1a9a7c0
Merge upstream/master into branch
qn895 Mar 26, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions x-pack/plugins/ml/common/util/anomaly_utils.ts
Original file line number Diff line number Diff line change
Expand Up @@ -27,10 +27,16 @@ export enum ENTITY_FIELD_TYPE {
PARTITON = 'partition',
}

export const ENTITFY_FIELD_OPERATIONS = {
ADD: '+',
REMOVE: '-',
} as const;

export interface EntityField {
fieldName: string;
fieldValue: string | number | undefined;
fieldType?: ENTITY_FIELD_TYPE;
operation?: typeof ENTITFY_FIELD_OPERATIONS[keyof typeof ENTITFY_FIELD_OPERATIONS];
}

// List of function descriptions for which actual values from record level results should be displayed.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ import { EuiSelect } from '@elastic/eui';
import { i18n } from '@kbn/i18n';
import { usePageUrlState } from '../../../util/url_state';

interface TableInterval {
export interface TableInterval {
display: string;
val: string;
}
Expand Down Expand Up @@ -64,6 +64,14 @@ export const useTableInterval = (): [TableInterval, (v: TableInterval) => void]
export const SelectInterval: FC = () => {
const [interval, setInterval] = useTableInterval();

return <SelectIntervalUI interval={interval} setInterval={setInterval} />;
};

interface SelectIntervalUIProps {
interval: TableInterval;
setInterval: (interval: TableInterval) => void;
}
export const SelectIntervalUI: FC<SelectIntervalUIProps> = ({ interval, setInterval }) => {
darnautov marked this conversation as resolved.
Show resolved Hide resolved
const onChange = (e: React.ChangeEvent<HTMLSelectElement>) => {
setInterval(optionValueToInterval(e.target.value));
};
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ const optionsMap = {
[criticalLabel]: ANOMALY_THRESHOLD.CRITICAL,
};

interface TableSeverity {
export interface TableSeverity {
val: number;
display: string;
color: string;
Expand Down Expand Up @@ -67,7 +67,7 @@ export const SEVERITY_OPTIONS: TableSeverity[] = [
},
];

function optionValueToThreshold(value: number) {
export function optionValueToThreshold(value: number) {
// Get corresponding threshold object with required display and val properties from the specified value.
let threshold = SEVERITY_OPTIONS.find((opt) => opt.val === value);

Expand Down Expand Up @@ -121,6 +121,14 @@ interface Props {
export const SelectSeverity: FC<Props> = ({ classNames } = { classNames: '' }) => {
const [severity, setSeverity] = useTableSeverity();

return <SelectSeverityUI severity={severity} setSeverity={setSeverity} />;
};

export const SelectSeverityUI: FC<{
classNames?: string;
severity: TableSeverity;
setSeverity: (s: TableSeverity) => void;
darnautov marked this conversation as resolved.
Show resolved Hide resolved
}> = ({ classNames = '', severity, setSeverity }) => {
const onChange = (valueDisplay: string) => {
setSeverity(optionValueToThreshold(optionsMap[valueDisplay]));
};
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,6 @@ import { forkJoin, of, Observable, Subject } from 'rxjs';
import { mergeMap, switchMap, tap } from 'rxjs/operators';

import { useCallback, useMemo } from 'react';
import { anomalyDataChange } from '../explorer_charts/explorer_charts_container_service';
import { explorerService } from '../explorer_dashboard_service';
import {
getDateFormatTz,
Expand All @@ -35,6 +34,9 @@ import { mlResultsServiceProvider } from '../../services/results_service';
import { isViewBySwimLaneData } from '../swimlane_container';
import { ANOMALY_SWIM_LANE_HARD_LIMIT } from '../explorer_constants';
import { TimefilterContract } from '../../../../../../../src/plugins/data/public';
import { AnomalyExplorerChartsService } from '../../services/anomaly_explorer_charts_service';
import { CombinedJob } from '../../../../common/types/anomaly_detection_jobs';
import { mlJobService } from '../../services/job_service';

// Memoize the data fetching methods.
// wrapWithLastRefreshArg() wraps any given function and preprends a `lastRefresh` argument
Expand All @@ -52,7 +54,6 @@ const memoize = <T extends (...a: any[]) => any>(func: T, context?: any) => {
return memoizeOne(wrapWithLastRefreshArg<T>(func, context) as any, memoizeIsEqual);
};

const memoizedAnomalyDataChange = memoize<typeof anomalyDataChange>(anomalyDataChange);
const memoizedLoadAnnotationsTableData = memoize<typeof loadAnnotationsTableData>(
loadAnnotationsTableData
);
Expand Down Expand Up @@ -93,6 +94,7 @@ export const isLoadExplorerDataConfig = (arg: any): arg is LoadExplorerDataConfi
*/
const loadExplorerDataProvider = (
anomalyTimelineService: AnomalyTimelineService,
anomalyExplorerService: AnomalyExplorerChartsService,
timefilter: TimefilterContract
) => {
const memoizedLoadOverallData = memoize(
Expand All @@ -103,6 +105,11 @@ const loadExplorerDataProvider = (
anomalyTimelineService.loadViewBySwimlane,
anomalyTimelineService
);
const memoizedAnomalyDataChange = memoize(
anomalyExplorerService.getAnomalyData,
anomalyExplorerService
);

return (config: LoadExplorerDataConfig): Observable<Partial<ExplorerState>> => {
if (!isLoadExplorerDataConfig(config)) {
return of({});
Expand All @@ -124,6 +131,10 @@ const loadExplorerDataProvider = (
viewByPerPage,
} = config;

const combinedJobRecords: Record<string, CombinedJob> = selectedJobs.reduce((acc, job) => {
return { ...acc, [job.id]: mlJobService.getJob(job.id) };
}, {});
Comment on lines +136 to +138
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should not use this mlJobService because it works by fetching all jobs upfront. You have data because on some page (not related to this code) it has fetched the list of jobs. As it's a new code, please replace it with the API service that fetches AD jobs instead.


const selectionInfluencers = getSelectionInfluencers(selectedCells, viewBySwimlaneFieldName);
const jobIds = getSelectionJobIds(selectedCells, selectedJobs);

Expand Down Expand Up @@ -200,23 +211,29 @@ const loadExplorerDataProvider = (
// and pass on the data we already fetched.
tap(explorerService.setViewBySwimlaneLoading),
// Trigger a side-effect to update the charts.
tap(({ anomalyChartRecords }) => {
tap(({ anomalyChartRecords, topFieldValues }) => {
if (selectedCells !== undefined && Array.isArray(anomalyChartRecords)) {
memoizedAnomalyDataChange(
lastRefresh,
explorerService,
combinedJobRecords,
swimlaneContainerWidth,
anomalyChartRecords,
timerange.earliestMs,
timerange.latestMs,
timefilter,
tableSeverity
);
} else {
memoizedAnomalyDataChange(
lastRefresh,
explorerService,
combinedJobRecords,
swimlaneContainerWidth,
[],
timerange.earliestMs,
timerange.latestMs,
timefilter,
tableSeverity
);
}
Expand Down Expand Up @@ -291,12 +308,17 @@ export const useExplorerData = (): [Partial<ExplorerState> | undefined, (d: any)
} = useMlKibana();

const loadExplorerData = useMemo(() => {
const mlResultsServices = mlResultsServiceProvider(mlApiServices);
const anomalyTimelineService = new AnomalyTimelineService(
timefilter,
uiSettings,
mlResultsServiceProvider(mlApiServices)
mlResultsServices
);
const anomalyExplorerService = new AnomalyExplorerChartsService(
mlApiServices,
mlResultsServices
);
return loadExplorerDataProvider(anomalyTimelineService, timefilter);
return loadExplorerDataProvider(anomalyTimelineService, anomalyExplorerService, timefilter);
}, []);

const loadExplorerData$ = useMemo(() => new Subject<LoadExplorerDataConfig>(), []);
Expand Down
23 changes: 20 additions & 3 deletions x-pack/plugins/ml/public/application/explorer/explorer.js
Original file line number Diff line number Diff line change
Expand Up @@ -63,13 +63,16 @@ import { AnomalyTimeline } from './anomaly_timeline';
import { FILTER_ACTION } from './explorer_constants';

// Explorer Charts
import { ExplorerChartsContainer } from './explorer_charts/explorer_charts_container';
import { ExplorerChartsContainer } from './explorer_charts/embeddable_explorer_charts_container';

// Anomalies Table
import { AnomaliesTable } from '../components/anomalies_table/anomalies_table';

import { getTimefilter, getToastNotifications } from '../util/dependency_cache';
import { ANOMALY_DETECTION_DEFAULT_TIME_RANGE } from '../../../common/constants/settings';
import { withKibana } from '../../../../../../src/plugins/kibana_react/public';
import { ML_APP_URL_GENERATOR } from '../../../common/constants/ml_url_generator';
import { getTimeBucketsFromCache } from '../util/time_buckets';

const ExplorerPage = ({
children,
Expand Down Expand Up @@ -136,7 +139,7 @@ const ExplorerPage = ({
</div>
);

export class Explorer extends React.Component {
export class ExplorerUI extends React.Component {
static propTypes = {
explorerState: PropTypes.object.isRequired,
setSelectedCells: PropTypes.func.isRequired,
Expand Down Expand Up @@ -224,6 +227,13 @@ export class Explorer extends React.Component {
updateLanguage = (language) => this.setState({ language });

render() {
const {
share: {
urlGenerators: { getUrlGenerator },
},
} = this.props.kibana.services;
const mlUrlGenerator = getUrlGenerator(ML_APP_URL_GENERATOR);

const { showCharts, severity, stoppedPartitions, selectedJobsRunning } = this.props;

const {
Expand Down Expand Up @@ -276,6 +286,7 @@ export class Explorer extends React.Component {

const timefilter = getTimefilter();
const bounds = timefilter.getActiveBounds();
const timeBuckets = getTimeBucketsFromCache();
darnautov marked this conversation as resolved.
Show resolved Hide resolved
const selectedJobIds = Array.isArray(selectedJobs) ? selectedJobs.map((job) => job.id) : [];
return (
<ExplorerPage
Expand Down Expand Up @@ -460,7 +471,11 @@ export class Explorer extends React.Component {
<EuiSpacer size="m" />

<div className="euiText explorer-charts">
{showCharts && <ExplorerChartsContainer {...{ ...chartsData, severity }} />}
{showCharts && (
<ExplorerChartsContainer
{...{ ...chartsData, severity, timefilter, mlUrlGenerator, timeBuckets }}
/>
)}
</div>

<AnomaliesTable
Expand All @@ -476,3 +491,5 @@ export class Explorer extends React.Component {
);
}
}

export const Explorer = withKibana(ExplorerUI);
Loading