Skip to content

Commit

Permalink
Merge branch '8.x' into bp-113749
Browse files Browse the repository at this point in the history
  • Loading branch information
@elasticmachine
elasticmachine authored Oct 2, 2024
2 parents 4989984 + e5bebfe commit 211ad0f
Show file tree
Hide file tree
Showing 6 changed files with 211 additions and 212 deletions.
100 changes: 40 additions & 60 deletions README.asciidoc
View file
Original file line number Diff line number Diff line change
Expand Up @@ -33,9 +33,11 @@ https://www.elastic.co/downloads/elasticsearch[elastic.co/downloads/elasticsearc
=== Run Elasticsearch locally

////
IMPORTANT: This content is replicated in the Elasticsearch guide. See `run-elasticsearch-locally.asciidoc`.
Both will soon be replaced by a quickstart script.
////
IMPORTANT: This content is replicated in `run-elasticsearch-locally.asciidoc`.
Ensure both files are in sync.
https://github.com/elastic/start-local is the source of truth.
////

[WARNING]
====
Expand All @@ -44,89 +46,67 @@ DO NOT USE THESE INSTRUCTIONS FOR PRODUCTION DEPLOYMENTS.
This setup is intended for local development and testing only.
====

The following commands help you very quickly spin up a single-node Elasticsearch cluster, together with Kibana in Docker.
Use this setup for local development or testing.
Quickly set up Elasticsearch and Kibana in Docker for local development or testing, using the https://github.com/elastic/start-local?tab=readme-ov-file#-try-elasticsearch-and-kibana-locally[`start-local` script].

==== Prerequisites
ℹ️ For more detailed information about the `start-local` setup, refer to the https://github.com/elastic/start-local[README on GitHub].

If you don't have Docker installed, https://www.docker.com/products/docker-desktop[download and install Docker Desktop] for your operating system.
==== Prerequisites

==== Set environment variables
- If you don't have Docker installed, https://www.docker.com/products/docker-desktop[download and install Docker Desktop] for your operating system.
- If you're using Microsoft Windows, then install https://learn.microsoft.com/en-us/windows/wsl/install[Windows Subsystem for Linux (WSL)].

Configure the following environment variables.
==== Trial license

[source,sh]
----
export ELASTIC_PASSWORD="<ES_PASSWORD>" # password for "elastic" username
export KIBANA_PASSWORD="<KIB_PASSWORD>" # Used internally by Kibana, must be at least 6 characters long
----
This setup comes with a one-month trial of the Elastic *Platinum* license.
After the trial period, the license reverts to *Free and open - Basic*.
Refer to https://www.elastic.co/subscriptions[Elastic subscriptions] for more information.

==== Create a Docker network
==== Run `start-local`

To run both Elasticsearch and Kibana, you'll need to create a Docker network:
To set up Elasticsearch and Kibana locally, run the `start-local` script:

[source,sh]
----
docker network create elastic-net
curl -fsSL https://elastic.co/start-local | sh
----
// NOTCONSOLE

==== Run Elasticsearch
This script creates an `elastic-start-local` folder containing configuration files and starts both Elasticsearch and Kibana using Docker.

Start the Elasticsearch container with the following command:
After running the script, you can access Elastic services at the following endpoints:

[source,sh]
----
docker run -p 127.0.0.1:9200:9200 -d --name elasticsearch --network elastic-net \
-e ELASTIC_PASSWORD=$ELASTIC_PASSWORD \
-e "discovery.type=single-node" \
-e "xpack.security.http.ssl.enabled=false" \
-e "xpack.license.self_generated.type=trial" \
docker.elastic.co/elasticsearch/elasticsearch:{version}
----
* *Elasticsearch*: http://localhost:9200
* *Kibana*: http://localhost:5601

==== Run Kibana (optional)
The script generates a random password for the `elastic` user, which is displayed at the end of the installation and stored in the `.env` file.

To run Kibana, you must first set the `kibana_system` password in the Elasticsearch container.
[CAUTION]
====
This setup is for local testing only. HTTPS is disabled, and Basic authentication is used for Elasticsearch. For security, Elasticsearch and Kibana are accessible only through `localhost`.
====

[source,sh]
----
# configure the Kibana password in the ES container
curl -u elastic:$ELASTIC_PASSWORD \
-X POST \
http://localhost:9200/_security/user/kibana_system/_password \
-d '{"password":"'"$KIBANA_PASSWORD"'"}' \
-H 'Content-Type: application/json'
----
// NOTCONSOLE
==== API access

Start the Kibana container with the following command:
An API key for Elasticsearch is generated and stored in the `.env` file as `ES_LOCAL_API_KEY`.
Use this key to connect to Elasticsearch with a https://www.elastic.co/guide/en/elasticsearch/client/index.html[programming language client] or the https://www.elastic.co/guide/en/elasticsearch/reference/current/rest-apis.html[REST API].

From the `elastic-start-local` folder, check the connection to Elasticsearch using `curl`:

[source,sh]
----
source .env
curl $ES_LOCAL_URL -H "Authorization: ApiKey ${ES_LOCAL_API_KEY}"
----
docker run -p 127.0.0.1:5601:5601 -d --name kibana --network elastic-net \
-e ELASTICSEARCH_URL=http://elasticsearch:9200 \
-e ELASTICSEARCH_HOSTS=http://elasticsearch:9200 \
-e ELASTICSEARCH_USERNAME=kibana_system \
-e ELASTICSEARCH_PASSWORD=$KIBANA_PASSWORD \
-e "xpack.security.enabled=false" \
-e "xpack.license.self_generated.type=trial" \
docker.elastic.co/kibana/kibana:{version}
----

.Trial license
[%collapsible]
====
The service is started with a trial license. The trial license enables all features of Elasticsearch for a trial period of 30 days. After the trial period expires, the license is downgraded to a basic license, which is free forever. If you prefer to skip the trial and use the basic license, set the value of the `xpack.license.self_generated.type` variable to basic instead. For a detailed feature comparison between the different licenses, refer to our https://www.elastic.co/subscriptions[subscriptions page].
====
// NOTCONSOLE

==== Send requests to Elasticsearch
=== Send requests to Elasticsearch

You send data and other requests to Elasticsearch through REST APIs.
You can interact with Elasticsearch using any client that sends HTTP requests,
such as the https://www.elastic.co/guide/en/elasticsearch/client/index.html[Elasticsearch
language clients] and https://curl.se[curl].

===== Using curl
==== Using curl

Here's an example curl command to create a new Elasticsearch index, using basic auth:

Expand All @@ -139,7 +119,7 @@ curl -u elastic:$ELASTIC_PASSWORD \
----
// NOTCONSOLE

===== Using a language client
==== Using a language client

To connect to your local dev Elasticsearch cluster with a language client, you can use basic authentication with the `elastic` username and the password you set in the environment variable.

Expand Down Expand Up @@ -167,7 +147,7 @@ client = Elasticsearch(
print(client.info())
----

===== Using the Dev Tools Console
==== Using the Dev Tools Console

Kibana's developer console provides an easy way to experiment and test requests.
To access the console, open Kibana, then go to **Management** > **Dev Tools**.
Expand Down
168 changes: 42 additions & 126 deletions docs/reference/run-elasticsearch-locally.asciidoc
View file
Original file line number Diff line number Diff line change
@@ -1,5 +1,11 @@
////
IMPORTANT: This content is replicated in the Elasticsearch repo root readme. Ensure both files are in sync.

https://github.com/elastic/start-local is the source of truth.
////

[[run-elasticsearch-locally]]
== Run {es} locally in Docker
== Run {es} locally
++++
<titleabbrev>Run {es} locally</titleabbrev>
++++
Expand All @@ -8,164 +14,74 @@
====
*DO NOT USE THESE INSTRUCTIONS FOR PRODUCTION DEPLOYMENTS*
The instructions on this page are for *local development only*. Do not use these instructions for production deployments, because they are not secure.
While this approach is convenient for experimenting and learning, you should never run Elasticsearch in this way in a production environment.
The instructions on this page are for *local development only*. Do not use this configuration for production deployments, because it is not secure.
Refer to <<elasticsearch-intro-deploy, deployment options>> for a list of production deployment options.
====

Follow this tutorial if you want to quickly set up {es} in Docker for local development or testing.
Quickly set up {es} and {kib} in Docker for local development or testing, using the https://github.com/elastic/start-local?tab=readme-ov-file#-try-elasticsearch-and-kibana-locally[`start-local` script].

This tutorial also includes instructions for installing {kib}.
If you don't need access to the {kib} UI, then you can skip those instructions.
This setup comes with a one-month trial of the Elastic *Platinum* license.
After the trial period, the license reverts to *Free and open - Basic*.
Refer to https://www.elastic.co/subscriptions[Elastic subscriptions] for more information.

[discrete]
[[local-dev-prerequisites]]
=== Prerequisites

If you don't have Docker installed, https://www.docker.com/products/docker-desktop[download and install Docker Desktop] for your operating system.
- If you don't have Docker installed, https://www.docker.com/products/docker-desktop[download and install Docker Desktop] for your operating system.
- If you're using Microsoft Windows, then install https://learn.microsoft.com/en-us/windows/wsl/install[Windows Subsystem for Linux (WSL)].

[discrete]
[[local-dev-env-vars]]
=== Set environment variables
[[local-dev-quick-start]]
=== Run `start-local`

Configure the following environment variables.
To set up {es} and {kib} locally, run the `start-local` script:

[source,sh]
----
export ELASTIC_PASSWORD="<ES_PASSWORD>" # password for "elastic" username
export KIBANA_PASSWORD="<KIB_PASSWORD>" # Used _internally_ by Kibana, must be at least 6 characters long
----

[discrete]
[[local-dev-create-docker-network]]
=== Create a Docker network

To run both {es} and {kib}, you'll need to create a Docker network:

[source,sh]
----
docker network create elastic-net
----

[discrete]
[[local-dev-run-es]]
=== Run {es}

Start the {es} container with the following command:

ifeval::["{release-state}"=="unreleased"]
WARNING: Version {version} has not yet been released.
No Docker image is currently available for {es} {version}.
endif::[]

[source,sh,subs="attributes"]
----
docker run -p 127.0.0.1:9200:9200 -d --name elasticsearch --network elastic-net \
-e ELASTIC_PASSWORD=$ELASTIC_PASSWORD \
-e "discovery.type=single-node" \
-e "xpack.security.http.ssl.enabled=false" \
-e "xpack.license.self_generated.type=trial" \
{docker-image}
----

[discrete]
[[local-dev-run-kib]]
=== Run {kib} (optional)

To run {kib}, you must first set the `kibana_system` password in the {es} container.

[source,sh,subs="attributes"]
----
# configure the Kibana password in the ES container
curl -u elastic:$ELASTIC_PASSWORD \
-X POST \
http://localhost:9200/_security/user/kibana_system/_password \
-d '{"password":"'"$KIBANA_PASSWORD"'"}' \
-H 'Content-Type: application/json'
curl -fsSL https://elastic.co/start-local | sh
----
// NOTCONSOLE

Start the {kib} container with the following command:
This script creates an `elastic-start-local` folder containing configuration files and starts both {es} and {kib} using Docker.

ifeval::["{release-state}"=="unreleased"]
WARNING: Version {version} has not yet been released.
No Docker image is currently available for {es} {version}.
endif::[]
After running the script, you can access Elastic services at the following endpoints:

[source,sh,subs="attributes"]
----
docker run -p 127.0.0.1:5601:5601 -d --name kibana --network elastic-net \
-e ELASTICSEARCH_URL=http://elasticsearch:9200 \
-e ELASTICSEARCH_HOSTS=http://elasticsearch:9200 \
-e ELASTICSEARCH_USERNAME=kibana_system \
-e ELASTICSEARCH_PASSWORD=$KIBANA_PASSWORD \
-e "xpack.security.enabled=false" \
-e "xpack.license.self_generated.type=trial" \
{kib-docker-image}
----
* *{es}*: http://localhost:9200
* *{kib}*: http://localhost:5601

When you access {kib}, use `elastic` as the username and the password you set earlier for the `ELASTIC_PASSWORD` environment variable.
The script generates a random password for the `elastic` user, which is displayed at the end of the installation and stored in the `.env` file.

[NOTE]
[CAUTION]
====
The service is started with a trial license. The trial license enables all features of Elasticsearch for a trial period of 30 days. After the trial period expires, the license is downgraded to a basic license, which is free forever.
This setup is for local testing only. HTTPS is disabled, and Basic authentication is used for {es}. For security, {es} and {kib} are accessible only through `localhost`.
====

[discrete]
[[local-dev-connecting-clients]]
=== Connect to {es} with language clients

To connect to the {es} cluster from a language client, you can use basic authentication with the `elastic` username and the password you set in the environment variable.

.*Expand* for details
[%collapsible]
==============
You'll use the following connection details:
* **{es} endpoint**: `http://localhost:9200`
* **Username**: `elastic`
* **Password**: `$ELASTIC_PASSWORD` (Value you set in the environment variable)
For example, to connect with the Python `elasticsearch` client:
[source,python]
----
import os
from elasticsearch import Elasticsearch
[[api-access]]
=== API access

username = 'elastic'
password = os.getenv('ELASTIC_PASSWORD') # Value you set in the environment variable
An API key for {es} is generated and stored in the `.env` file as `ES_LOCAL_API_KEY`.
Use this key to connect to {es} with a https://www.elastic.co/guide/en/elasticsearch/client/index.html[programming language client] or the <<rest-apis,REST API>>.

client = Elasticsearch(
"http://localhost:9200",
basic_auth=(username, password)
)
From the `elastic-start-local` folder, check the connection to Elasticsearch using `curl`:

print(client.info())
----
Here's an example curl command using basic authentication:
[source,sh,subs="attributes"]
----
curl -u elastic:$ELASTIC_PASSWORD \
-X PUT \
http://localhost:9200/my-new-index \
-H 'Content-Type: application/json'
[source,sh]
----
source .env
curl $ES_LOCAL_URL -H "Authorization: ApiKey ${ES_LOCAL_API_KEY}"
----
// NOTCONSOLE

==============

[discrete]
[[local-dev-next-steps]]
=== Next steps
[[local-dev-additional-info]]
=== Learn more

Use our <<quickstart,quick start guides>> to learn the basics of {es}.
For more detailed information about the `start-local` setup, refer to the https://github.com/elastic/start-local[README on GitHub].
Learn about customizing the setup, logging, and more.

[discrete]
[[local-dev-production]]
=== Moving to production
[[local-dev-next-steps]]
=== Next steps

This setup is not suitable for production use.
Refer to <<elasticsearch-intro-deploy, deployment options>> for more information.
Use our <<quickstart,quick start guides>> to learn the basics of {es}.
4 changes: 2 additions & 2 deletions ...xtras/src/test/java/org/elasticsearch/index/mapper/extras/TokenCountFieldMapperTests.java
View file
Original file line number Diff line number Diff line change
Expand Up @@ -220,7 +220,7 @@ public SyntheticSourceExample example(int maxValues) {

var textArray = values.stream().map(Value::text).toList();

var blockExpectedList = values.stream().map(Value::tokenCount).filter(Objects::nonNull).toList();
var blockExpectedList = values.stream().map(Value::tokenCount).filter(Objects::nonNull).sorted().toList();
var blockExpected = blockExpectedList.size() == 1 ? blockExpectedList.get(0) : blockExpectedList;

return new SyntheticSourceExample(textArray, textArray, blockExpected, this::mapping);
Expand All @@ -230,7 +230,7 @@ private record Value(String text, Integer tokenCount) {}

private Value generateValue() {
if (rarely()) {
return new Value(null, null);
return new Value(null, nullValue);
}

var text = randomList(0, 10, () -> randomAlphaOfLengthBetween(0, 10)).stream().collect(Collectors.joining(" "));
Expand Down
Loading

0 comments on commit 211ad0f

Please sign in to comment.