[ About | Licence | Contacts ]
Written by Oleksandr Gavenko (AKA gavenkoa), compiled on 2024-04-01 from rev 052223c22317.

elasticsearch

Elasticsearch documentation

https://amsterdam.luminis.eu/2016/10/18/elasticsearch-5-is-coming-what-is-new-and-improved/
New features of ES 5.

Releases

https://github.com/elastic/elasticsearch/releases
Git releases & tags.
https://www.elastic.co/support/eol
Elastic Product End of Life Dates.

Installing & configuring

https://www.elastic.co/guide/en/elasticsearch/reference/current/deb.html
Install Elasticsearch with Debian Package.

Basic config

Common cluster name inside given intranet:

cluster.name: mycluster

Unique node name within claster:

node.name: "node1"

Node types:

node.master: true
node.ingest: true
node.data: true

Network interfaces to bind to:

network.host: [_local_, node1.example.com]

Port definitions:

http.port : 9200
tcp.port : 9300

Override default locations:

path.data: /path/to/data1,/path/to/data2
path.logs: /path/to/logs
path.plugins: /path/to/plugins

Explicit list of seed nodes in cluster:

discovery.zen.ping.unicast.hosts: ["master1.example.com", "master2.example.com:9300"]

Dump applied configs:

GET /_cluster/settings?include_defaults=true
https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-node.html
Node types.
https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-discovery-zen.html
Discovery settings.
https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-transport.html
Transport definitions.
https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-http.html
Configuring Elasticsearch » HTTP
https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-network.html
Configuring Elasticsearch » Network settings

REST syntax conventions

To get data in table form use /_cat endpoint:

GET /_cat/nodes

To pretty print output append query:

GET /_cat/nodes?pretty=1

Get base information

Cluster health:

GET /_cat/health?v
GET /_cluster/health?pretty
GET /_cluster/health?pretty&level=cluster

List of nodes in cluster (ip, RAM, CPU):

GET /_cat/nodes?v
GET /_cat/master?v

watch -d curl -s 'localhost:9200/_cat/nodes?v'

List cluster state:

GET /_cluster/state?pretty
GET /_cluster/allocation/explain

List of tasks executed in cluster:

GET /_cat/tasks?v
GET /_cat/tasks?detailed
GET _tasks

List of indexes (status, health, size):

GET /_cat/indices
GET /_cat/indices?v
GET /_cat/indices?v&s=index
GET /_cluster/health?pretty&level=indices

List of shards:

GET /_cat/shards?v

List of mappings across all indexes:

GET /_mapping
GET /_all/_mapping
GET /twitter,kimchy/_mapping
https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-get-mapping.html
Get Mapping.

List of shards:

GET /_cluster/health?pretty&level=shards
https://www.elastic.co/guide/en/elasticsearch/reference/current/_cluster_health.html
Cluster Health.
https://www.elastic.co/guide/en/elasticsearch/reference/current/cluster-allocation-explain.html
Cluster Allocation Explain API
https://www.elastic.co/guide/en/elasticsearch/reference/current/_list_all_indices.html
List All Indices.
https://www.elastic.co/guide/en/elasticsearch/reference/current/cat.html
cat APIs.
https://www.elastic.co/guide/en/elasticsearch/reference/current/tasks.html
Task Management API.

Managing indexes

Assign number of shards and replicas:

curl -XPUT -d '{settings: {index: "number_of_shards": 3, "number_of_replicas": 1}}'
https://www.elastic.co/guide/en/elasticsearch/reference/current/_delete_an_index.html
Delete an Index.
https://github.com/elastic/curator
Manage/archive indices.
https://www.elastic.co/guide/en/elasticsearch/client/curator/current/about.html
Elasticsearch Curator helps you curate, or manage, your Elasticsearch indices and snapshots.
https://www.elastic.co/guide/en/elasticsearch/client/curator/current/about-features.html
Curator allows for many different operations to be performed to both indices and snapshots.
https://www.elastic.co/guide/en/elasticsearch/guide/current/retiring-data.html
Retiring Data.

Lucene search syntax

TERM1 TERM2 is same as TERM1 OR TERM2.

"TERM1 TERM2" is for phrase.

"TERM1 TERM2"~5 is for proximity.

TERM~0.8 is for fuzzy.

* is for wildcard.

Boosting is done with ^N syntax, like TERM1^10 TERM2.

Range with [2017-01-01 TO 2017-02-29].

Logical AND, OR, NOT and parentheses for grouping.

Fields are set before colon, like FIELD:TERM.

https://www.elastic.co/guide/en/elasticsearch/reference/master/query-dsl-query-string-query.html
Query String Query
https://www.elastic.co/guide/en/elasticsearch/reference/master/query-dsl-simple-query-string-query.html
Simple Query String Query

ES Query DSL

_exists_:<field> for testing field existence.

Note

_missing_:<field> was removed from Kibana 5.x, use NOT _exists_:<field>.

-<field>:<val> or -<field>:"<val>" for excluding field value.

+<field>:<val> or +<field>:"<val>" for including field value.

https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-exists-query.html
Exists Query

Performance

https://www.elastic.co/guide/en/elasticsearch/reference/5.5/search-profile.html
Profile API.
https://www.elastic.co/guide/en/elasticsearch/reference/current/_explain_analyze.html
Explain Analyze.
https://www.elastic.co/guide/en/elasticsearch/reference/current/search-explain.html
Explain API.
https://www.elastic.co/guide/en/elasticsearch/reference/current/tune-for-disk-usage.html
Tune for disk usage.
https://www.elastic.co/guide/en/elasticsearch/reference/current/tune-for-indexing-speed.html
Tune for indexing speed.
https://www.elastic.co/guide/en/elasticsearch/reference/current/tune-for-search-speed.html
Tune for search speed.
https://www.elastic.co/blog/how-many-shards-should-i-have-in-my-elasticsearch-cluster
How many shards should I have in my Elasticsearch cluster?
https://www.elastic.co/guide/en/elasticsearch/reference/master/heap-size.html
Setting the heap size.

Storage requirements

https://www.elastic.co/blog/elasticsearch-storage-the-true-story
The true story behind Elasticsearch storage requirements (2015).
https://www.elastic.co/blog/elasticsearch-storage-the-true-story-2.0
Part 2.0: The true story behind Elasticsearch storage requirements (2015).
https://www.elastic.co/blog/minimize-index-storage-size-elasticsearch-6-0
Space Saving Improvements in Elasticsearch 6.0 (2017).
https://www.elastic.co/blog/filebeat-modiles-access-logs-and-elasticsearch-storage-requirements
Filebeat modules, access logs and Elasticsearch storage requirements.

JSON search syntax

https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-from-size.html
Search results pagination.
https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-query-string-query.html
Query String Query.
https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations.html
Aggregation.
https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-sort.html
Sort.

Alerting

https://github.com/Yelp/elastalert
Easy & Flexible Alerting With Elasticsearch.
http://elastalert.readthedocs.io/en/latest/elastalert.html
Easy & Flexible Alerting With Elasticsearch.
https://github.com/sirensolutions/sentinl/wiki/SENTINL-Alerts-in-Dashboard
SENTINL Alerts in Dashboard.
https://github.com/sirensolutions/sentinl/wiki/SENTINL-Config-Example
SENTINL Config Example
https://github.com/sirensolutions/sentinl/wiki/SENTINL-Tutorial
SENTINL Tutorial
https://github.com/sirensolutions/sentinl/wiki/SENTINL-Watcher-Anatomy
SENTINL Watcher Anatomy
https://github.com/sirensolutions/sentinl/wiki/SENTINL-Watcher-Examples
SENTINL Watcher Examples