[new-platform] Differentiate between Public and Internal APIs

[joshdover]

APIs registered by plugins could benefit from having an explicit difference between public and internal APIs.

• Public APIs are stable across minor releases, provide an OpenAPI spec, are documented publicly, and are exposed to other plugins via a JS client.
• Internal APIs are specific to a plugin and do not provide any stability guarantees.

Additionally, these two types of APIs could have different URL prefixes. For example:

• Public API, registered by pluginA: /api/pluginA/myRoute
• Internal API, registered by pluginA: /internal/pluginA/myRoute

Benefits:

• The explicitness of this difference would help both Kibana devs and 3rd party devs understand the guarantees of a given API.
• Accidental breakage of a public API would be more obvious.
• We can require OpenAPI specs for public APIs to help improve documentation and to generate API clients for various languages.

Related to #12464

[elasticmachine]

Pinging @elastic/kibana-platform

[mshustov]

Also, there is a request to distinguish API URLs from other resources. It can be covered by this proposal as well.

@elastic/kibana-security what does system api that you heavily use stand for?

[kobelb]

@elastic/kibana-security what does system api that you heavily use stand for?

We use this to differentiate between requests that were initiated by the end user versus requests that are initiated by the system. When it's an "end user initiated request" we want to extend the user's session to prevent them from being logged out for being idle. Where-as when it's a "system initiated request", most commonly background requests which don't signal true end-user activity like polling for new reporting jobs, we don't extend the user's session.

[joshdover]

We use this to differentiate between requests that were initiated by the end user versus requests that are initiated by the system.

I see, that is a different use case. Could we leverage route tags for this?

[kobelb]

Could we leverage route tags for this?

It's possible, but it'd potentially require use to duplicate routes for us to do this entirely properly. For Reporting, we used to use the same routes for polling in the background as we did in the foreground for listing reporting jobs. It looks like there have been some recent changes so all of these requests have that system-api header even though they technically shouldn't.

[azasypkin]

Yeah, using tags to cover "system api" requests use case can be cumbersome and may not solve the problem completely anyway: there will definitely be client side abstractions at some point that will be hitting one route or another under the hood and apps/plugins may use these abstractions (e.g. SavedObjectClient.get or something like this) to serve both system and user initiated requests without understanding the consequences.

But yeah, it's a completely different topic and seems to be outside of internal vs public APIs scope, we'll need to discuss it separately.

[mshustov]

related #21424

[mshustov]

closed in favour of #21424