Contiamo poll

docs/contiamo-long-poll.md

@@ -0,0 +1,78 @@
+# Motivation
+
+Streaming events from servers to browsers is a big messy non-standardized problem. This spec will define _how_ Contiamo sends events from backend services to web-browsers. This can be used for streaming new logs from long running process or notify that the object being viewed has been changed.
+
+In short, servers will implement long-polling [[1](https://en.wikipedia.org/wiki/Push_technology#Long_polling), [2](https://realtimeapi.io/hub/http-long-polling/)]. Clients will indicate a long-poll request using the [`Prefer` header](https://tools.ietf.org/html/rfc7240#section-4.3), servers will indicate the polling timeout with a [304 status code](https://httpstatuses.com/304), and payloads will be JSON.
+
+Alternative implementations that a service and can support _in addition_ to long-polling are grpc streams [[1](https://grpc.io/docs/guides/concepts.html#server-streaming-rpc), [2](https://grpc.io/docs/tutorials/basic/node.html#streaming-rpcs)] or http streaming [[1](https://realtimeapi.io/hub/http-streaming/), [2](https://tools.ietf.org/id/draft-loreto-http-bidirectional-07.html#streaming)] with [new-line-delimited-json](http://ndjson.org/).
+
+<!-- START doctoc generated TOC please keep comment here to allow auto update -->
+<!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->
+
+
+- [Implementation](#implementation)
+  - [Endpoints](#endpoints)
+  - [Flow](#flow)
+  - [Request Headers](#request-headers)
+  - [Response Headers](#response-headers)
+    - [Response](#response)
+  - [Status Codes](#status-codes)
+- [Research Examples](#research-examples)
+- [Specifications, Blogs, and other Documents](#specifications-blogs-and-other-documents)
+
+<!-- END doctoc generated TOC please keep comment here to allow auto update -->
+
+# Implementation
+
+## Endpoints
+
+Long polling will be implemented as an enhancement of existing REST/RPC endpoints, not as new polling specific endpoints. The API documentation must specify if the endpoint supports long-polling.
+
+## Flow
+
+HTTP long polling is a variation on the standard polling but with the distinction that the polling request are "long-lived". At Contiamo, the flow looks like this:
+
+![Long Poll Flow](long-poll-flow.png)
+
+## Request Headers
+
+To start a long-polling request, Restful React makes an `HTTP GET` request and set the `Prefer` header with a wait value and an index value, e.g. `wait=60;index=abad123`. This is a number of seconds and a query "position". This value can be set at a maximum of 60 seconds, the minimum can be defined by the backend server, but is typically >5 seconds. The index will be a server defined and supplied value.
+
+The `index` value is optional, if omitted, the request is processed exactly the same as a standard web request (meaning based on any `GET` parameters supplied). This `index` value should be read from the `X-Polling-Index` header in a previous response. When it is set, the server will use the value to wait for any changes subsequent to that index.
+
+## Response Headers
+
+On a successful `GET`, the server must set a head `X-Polling-Index`, this value is a unique identifier representing the current state of the resource. It is not required to have any specific structure or meaning for the client. Meaning that the client should not inspect the value for any specific information or structure. For example, this value could be datetime string of the last update, an int64 of the last object, or it could be a base64 encoded datetime string like `MjAxOC0wMS0wMVQwMDowMDowMFo=`. Whatever it is, the server is responsible for encoding and decoding this value to filter the query for changes from that index point.
+
+### Response
+
+In the absence of the `Prefer` header, the request will behave as normal, the backend service will immediately process and return a response as soon as it can.
+
+When the `Prefer` header is set, the server will parse (and potentially normalize the value). It will process the request. The server will wait until a maximum of the `wait` value has elapsed _or_ it can fulfill the request. If the `wait` time elapses, it will send a `304` status code indicating that the request did not fail, but contains no data. If the server decides that it can fulfill the request, it response with a `200` status code and a JSON payload, as defined by the API docs for the endpoint.
+
+Once the request has finished the client can then open a new request for more data.
+
+## Status Codes
+
+- `200` success
+- `304` long poll timeout
+- `4xx` request error
+- `5xx` server error
+
+This list is provided to highlight the distinction between the polling timeout response and other timeout responses like
+
+- `[408 Request Timeout](https://httpstatuses.com/408)`
+- `[504 Gateway Timeout](https://httpstatuses.com/504)`
+- `[599 Network Connect Timeout Error](https://httpstatuses.com/599)`
+
+These other statuses should be treated as errors.
+
+# Research Examples
+
+- [Console blocking queries](https://www.consul.io/api/index.html#blocking-queries)
+- [Dropbox longpoll endpoints](https://www.dropbox.com/developers/documentation/http/documentation#files-list_folder-longpoll)
+
+# Specifications, Blogs, and other Documents
+
+- [Prefer header RFC](https://tools.ietf.org/html/rfc7240#section-4.3)
+- [Realtime API hub docs](https://realtimeapi.io/hub/http-long-polling/)

package.json

@@ -1,7 +1,7 @@
 {
   "name": "restful-react",
   "description": "A declarative client from RESTful React Apps",
-  "version": "4.0.0-4",
+  "version": "4.0.0-5",
   "main": "dist/index.js",
   "license": "MIT",
   "files": [
@@ -62,7 +62,7 @@
     "lint-staged": "^7.2.0",
     "prettier": "^1.13.5",
     "rollup": "^0.61.2",
-    "rollup-plugin-typescript2": "^0.15.0",
+    "rollup-plugin-typescript2": "^0.16.1",
     "ts-jest": "^22.4.6",
     "tslint": "^5.10.0",
     "tslint-config-prettier": "^1.13.0",

src/Poll.tsx

@@ -60,11 +60,19 @@ interface PollProps<T = {}> {
    */
   children: (data: T | null, states: States<T>, actions: Actions, meta: Meta) => React.ReactNode;
   /**
-   * How long do we wait between requests?
+   * How long do we wait between repeating a request?
    * Value in milliseconds.
+   *
    * Defaults to 1000.
    */
   interval?: number;
+  /**
+   * How long should a request stay open?
+   * Value in seconds.
+   *
+   * Defaults to 60.
+   */
+  wait?: number;
   /**
    * A stop condition for the poll that expects
    * a boolean.
@@ -122,6 +130,10 @@ interface PollState<T> {
    * Do we currently have an error?
    */
   error?: GetComponentState<T>["error"];
+  /**
+   * Index of the last polled response.
+   */
+  lastPollIndex?: string;
 }
 
 /**
@@ -132,18 +144,13 @@ class ContextlessPoll<T> extends React.Component<PollProps<T>, Readonly<PollStat
     data: null,
     loading: !this.props.lazy,
     lastResponse: null,
-    polling: false,
+    polling: !this.props.lazy,
     finished: false,
   };
 
-  public static getDerivedStateFromProps(props: Pick<PollProps, "lazy">) {
-    return {
-      polling: !props.lazy,
-    };
-  }
-
   public static defaultProps = {
     interval: 1000,
+    wait: 60,
     resolve: (data: any) => data,
   };
 
@@ -159,6 +166,12 @@ class ContextlessPoll<T> extends React.Component<PollProps<T>, Readonly<PollStat
     return true;
   };
 
+  private getRequestOptions = () =>
+    typeof this.props.requestOptions === "function" ? this.props.requestOptions() : this.props.requestOptions || {};
+
+  // 304 is not a OK status code but is green in Chrome 🤦🏾‍♂️
+  private isResponseOk = (response: Response) => response.ok || response.status === 304;
+
   /**
    * This thing does the actual poll.
    */
@@ -175,17 +188,25 @@ class ContextlessPoll<T> extends React.Component<PollProps<T>, Readonly<PollStat
     }
 
     // If we should keep going,
-    const { base, path, requestOptions, resolve, interval } = this.props;
-    const request = new Request(
-      `${base}${path}`,
-      typeof requestOptions === "function" ? requestOptions() : requestOptions,
-    );
+    const { base, path, resolve, interval, wait } = this.props;
+    const { lastPollIndex } = this.state;
+    const requestOptions = this.getRequestOptions();
+
+    const request = new Request(`${base}${path}`, {
+      ...requestOptions,
+
+      headers: {
+        Prefer: `wait=${wait};${lastPollIndex ? `index=${lastPollIndex}` : ""}`,
+
+        ...requestOptions.headers,
+      },
+    });
     const response = await fetch(request);
 
     const responseBody =
       response.headers.get("content-type") === "application/json" ? await response.json() : await response.text();
 
-    if (!response.ok) {
+    if (!this.isResponseOk(response)) {
       const error = `${response.status} ${response.statusText}`;
       this.setState({ loading: false, lastResponse: response, data: responseBody, error });
       throw new Error(`Failed to Poll: ${error}`);
@@ -196,6 +217,7 @@ class ContextlessPoll<T> extends React.Component<PollProps<T>, Readonly<PollStat
         loading: false,
         lastResponse: response,
         data: resolve ? resolve(responseBody) : responseBody,
+        lastPollIndex: response.headers.get("x-polling-index") || undefined,
       }));
     }
 

yarn.lock

@@ -1275,17 +1275,17 @@ fragment-cache@^0.2.1:
   dependencies:
     map-cache "^0.2.2"
 
-fs-extra@6.0.0:
-  version "6.0.0"
-  resolved "https://registry.yarnpkg.com/fs-extra/-/fs-extra-6.0.0.tgz#0f0afb290bb3deb87978da816fcd3c7797f3a817"
+fs-extra@5.0.0:
+  version "5.0.0"
+  resolved "https://registry.yarnpkg.com/fs-extra/-/fs-extra-5.0.0.tgz#414d0110cdd06705734d055652c5411260c31abd"
   dependencies:
     graceful-fs "^4.1.2"
     jsonfile "^4.0.0"
     universalify "^0.1.0"
 
-fs-extra@^5.0.0:
-  version "5.0.0"
-  resolved "https://registry.yarnpkg.com/fs-extra/-/fs-extra-5.0.0.tgz#414d0110cdd06705734d055652c5411260c31abd"
+fs-extra@6.0.0:
+  version "6.0.0"
+  resolved "https://registry.yarnpkg.com/fs-extra/-/fs-extra-6.0.0.tgz#0f0afb290bb3deb87978da816fcd3c7797f3a817"
   dependencies:
     graceful-fs "^4.1.2"
     jsonfile "^4.0.0"
@@ -3391,7 +3391,7 @@ resolve@1.1.7:
   version "1.1.7"
   resolved "https://registry.yarnpkg.com/resolve/-/resolve-1.1.7.tgz#203114d82ad2c5ed9e8e0411b3932875e889e97b"
 
-resolve@^1.1.7, resolve@^1.3.2, resolve@^1.7.1:
+resolve@1.8.1, resolve@^1.1.7, resolve@^1.3.2:
   version "1.8.1"
   resolved "https://registry.yarnpkg.com/resolve/-/resolve-1.8.1.tgz#82f1ec19a423ac1fbd080b0bab06ba36e84a7a26"
   dependencies:
@@ -3420,16 +3420,16 @@ rimraf@^2.5.4, rimraf@^2.6.1:
   dependencies:
     glob "^7.0.5"
 
-rollup-plugin-typescript2@^0.15.0:
-  version "0.15.1"
-  resolved "https://registry.yarnpkg.com/rollup-plugin-typescript2/-/rollup-plugin-typescript2-0.15.1.tgz#7b35d0eaa6ad5a54a253ed158a565b99d8f15372"
+rollup-plugin-typescript2@^0.16.1:
+  version "0.16.1"
+  resolved "https://registry.yarnpkg.com/rollup-plugin-typescript2/-/rollup-plugin-typescript2-0.16.1.tgz#72e1f8a2e450550eabdc3d474e735feae322b474"
   dependencies:
-    fs-extra "^5.0.0"
-    resolve "^1.7.1"
-    rollup-pluginutils "^2.0.1"
-    tslib "1.9.2"
+    fs-extra "5.0.0"
+    resolve "1.8.1"
+    rollup-pluginutils "2.3.0"
+    tslib "1.9.3"
 
-rollup-pluginutils@^2.0.1:
+rollup-pluginutils@2.3.0:
   version "2.3.0"
   resolved "https://registry.yarnpkg.com/rollup-pluginutils/-/rollup-pluginutils-2.3.0.tgz#478ace04bd7f6da2e724356ca798214884738fc4"
   dependencies:
@@ -3927,11 +3927,7 @@ ts-jest@^22.4.6:
     source-map-support "^0.5.5"
     yargs "^11.0.0"
 
-tslib@1.9.2:
-  version "1.9.2"
-  resolved "https://registry.yarnpkg.com/tslib/-/tslib-1.9.2.tgz#8be0cc9a1f6dc7727c38deb16c2ebd1a2892988e"
-
-tslib@^1.8.0, tslib@^1.8.1, tslib@^1.9.0:
+tslib@1.9.3, tslib@^1.8.0, tslib@^1.8.1, tslib@^1.9.0:
   version "1.9.3"
   resolved "https://registry.yarnpkg.com/tslib/-/tslib-1.9.3.tgz#d7e4dd79245d85428c4d7e4822a79917954ca286"