Elasticsearch release notes

Details
This change removes TLS_RSA ciphers from the list of default supported ciphers, for Elasticsearch deployments running on JDK 24.

Impact
The dropped ciphers are TLS_RSA_WITH_AES_256_GCM_SHA384, TLS_RSA_WITH_AES_128_GCM_SHA256, TLS_RSA_WITH_AES_256_CBC_SHA256, TLS_RSA_WITH_AES_128_CBC_SHA256, TLS_RSA_WITH_AES_256_CBC_SHA, and TLS_RSA_WITH_AES_128_CBC_SHA. TLS connections to Elasticsearch using these ciphers will no longer work. Please configure your clients to use one of supported cipher suites.

Details
The anomaly detection job flush API is deprecated since it is only required for the post data API, which was deprecated since 7.11.0.

Impact
This should have a minimal impact on users as the flush API is only required for the post data API, which was deprecated since 7.11.0.

Details
Behavioral Analytics has been deprecated as of 9.0.0 and will be removed in a future release. The APIs will still work for now, but will emit warning headers that the API has been deprecated.

Impact
Behavioral Analytics has been deprecated as of 9.0.0 and will be removed in a future release.

Details
Elasticsearch no longer allows running Date Histogram aggregations over boolean fields. Instead, use Terms aggregation for boolean fields.

Impact
We expect the impact to be minimal, as this never produced good results, and has been deprecated for years.

Details
Lucene 10 ships with an upgrade of its Snowball stemmers. For details see https://github.com/apache/lucene/issues/13209. Users using Snowball stemmers that are experiencing changes in search behaviour on existing data are advised to reindex.

Impact
The upgrade should generally provide improved stemming results. Small changes in token analysis can lead to mismatches with previously index data, so existing indices using Snowball stemmers as part of their analysis chain should be reindexed.

Details
Lucene 10 has merged the improved "german2" snowball language stemmer with the "german" stemmer. For Elasticsearch, "german2" is now a deprecated alias for "german". This may results in slightly different tokens being generated for terms with umlaut substitution (like "ue" for "ü" etc…​)

Impact
Replace usages of "german2" with "german" in analysis configuration. Old indices that use the "german" stemmer should be reindexed if possible.

Details
Lucene 10 has added a final stemming step to its PersianAnalyzer that Elasticsearch exposes as persian analyzer. Existing indices will keep the old non-stemming behaviour while new indices will see the updated behaviour with added stemming. Users that wish to maintain the non-stemming behaviour need to define their own analyzer as outlined in Persion analyzer. Users that wish to use the new stemming behaviour for existing indices will have to reindex their data.

Impact
Indexing with the persian analyzer will produce slightly different tokens. Users should check if this impacts their search results. If they wish to maintain the legacy non-stemming behaviour they can define their own analyzer equivalent as explained in Persian analyzer.

Details
Lucene 10 ships with an updated Korean dictionary (mecab-ko-dic-2.1.1). For details see https://github.com/apache/lucene/issues/11452. Users experiencing changes in search behaviour on existing data are advised to reindex.

Impact
The change is small and should generally provide better analysis results. Existing indices for full-text use cases should be reindexed though.

Details
For LDAP or AD authentication realms, setting a bind DN (via the xpack.security.authc.realms.ldap.*.bind_dn or xpack.security.authc.realms.active_directory.*.bind_dn realm settings) without a bind password is a misconfiguration that may prevent successful authentication to the node. Nodes will fail to start if a bind DN is specified without a password.

Impact
If you have a bind DN configured for an LDAP or AD authentication realm, set a bind password for LDAP or Active Directory. Configuring a bind DN without a password prevents the misconfigured node from starting.

Details
Deprecated tracing.apm.* settings got removed, use respective telemetry.* / telemetry.tracing.* settings instead.

Impact
9.x nodes will refuse to start if any such setting (including secret settings) is still present.

Details
In the past, byte values like 1.25 mb were allowed but deprecated. Now, values with up to two decimal places are allowed, unless the unit is bytes, in which case no decimals are allowed. Values with too many decimal places result in an error.

Impact
Values with more than two decimal places, like 0.123 mb will be rejected as an error, where in the past, they’d be accepted with a deprecation warning.

Details
Earlier versions of Elasticsearch accepted any non-negative value for cluster.routing.allocation.balance.threshold, but values smaller than 1.0 do not make sense and have been ignored since version 8.6.1. From 9.0.0 these nonsensical values are now forbidden.

Impact
Do not set cluster.routing.allocation.balance.threshold to a value less than 1.0.

Details
TLSv1.1 is no longer enabled by default. Prior to version 9.0, Elasticsearch would attempt to enable TLSv1.1 if the JDK supported it. In most cases, including all cases where Elasticsearch 8 was running with the bundled JDK, the JDK would not support TLSv1.1, so that protocol would not be available in Elasticsearch. However, if Elasticsearch was running on an old JDK or a JDK that have been reconfigured to support TLSv1.1, then the protocol would automatically be available within Elasticsearch. As of Elasticsearch 9.0, this is no longer true. If you wish to enable TLSv1.1 then you must enable it within the JDK and also enable it within Elasticsearch by using the ssl.supported_protocols setting.

Impact
Most users will not be impacted. If your Elastisearch 8 cluster was using a custom JDK and you relied on TLSv1.1, then you will need to explicitly enable TLSv1.1 within Elasticsearch (as well as enabling it within your JDK)

Details
The node setting client.type has been ignored since the node client was removed in 8.0. The setting is now removed.

Impact
Remove the client.type setting from elasticsearch.yml

Details
Prior to 7.8, whenever a cluster had only a single data node, the watermarks would not be respected. In order to change this in 7.8+ in a backwards compatible way, we introduced the cluster.routing.allocation.disk.watermark.enable_for_single_data_node node setting. The setting was deprecated in 7.14 and was made to accept only true in 8.0

Impact
No known end user impact

Details
The xpack.searchable.snapshot.allocate_on_rolling_restart setting was created as an escape-hatch just in case relying on the cluster.routing.allocation.enable=primaries setting for allocating searchable snapshots during rolling restarts had some unintended side-effects. It has been deprecated since 8.2.0.

Impact
Remove xpack.searchable.snapshot.allocate_on_rolling_restart from your settings if present.

Details
Earlier versions of Elasticsearch had a discovery.type setting which permitted values that referred to legacy discovery types. From v9.0.0 onwards, the only supported values for this setting are multi-node (the default) and single-node.

Impact
Remove any value for discovery.type from your elasticsearch.yml configuration file.

Details
The ability to read frozen indices has been removed. (Frozen indices are no longer useful due to improvements in heap memory usage. The ability to freeze indices was removed in 8.0.)

Impact
Users must unfreeze any frozen indices before upgrading.

Details
The user_agent ingest processor no longer accepts the ecs option. (It was previously deprecated and ignored.)

Impact
Users should stop using the ecs option when creating instances of the user_agent ingest processor. The option will be removed from existing processors stored in the cluster state on upgrade.

Details
The option fallback_to_default_databases on the geoip ingest processor has been removed. (It was deprecated and ignored since 8.0.0.)

Impact
Customers should stop remove the noop fallback_to_default_databases option on any geoip ingest processors.

Details
This change modifies the "data_stream.dataset" and "event.dataset" value for deprecation logging to use the value elasticsearch.deprecation instead of deprecation.elasticsearch. This is now consistent with other values where the name of the service is the first part of the key.

Impact
If you are directly consuming deprecation logs for "data_stream.dataset" and "event.dataset" and filtering on this value, you will need to update your filters to use elasticsearch.deprecation instead of deprecation.elasticsearch.

Details
Logsdb will be enabled by default for data streams matching with logs-- pattern. If upgrading from 8.x to 9.x and data streams matching with log-- do exist, then Logsdb will not be enabled by default.

Impact
Logsdb reduce storage footprint in Elasticsearch for logs, but there are side effects to be taken into account that are described in the Logsdb docs.

Details
The type, fields, copy_to and boost parameters are no longer supported in metadata field definition starting with version 9.

Impact
Users providing type, fields, copy_to or boost as part of metadata field definition should remove them from their mappings.

Details
The mode mapping attribute of _source metadata field mapper has been turned into a no-op. Instead the index.mapping.source.mode index setting should be used to configure source mode.

Impact
Configuring the mode attribute for the _source meta field mapper will have no effect on indices created with Elasticsearch 9.0.0 or later. Note that _source.mode configured on indices before upgrading to 9.0.0 or later will remain efficive after upgrading.

Details
The machine learning plugin is permanently disabled on macOS x86_64. For the last three years Apple has been selling hardware based on the arm64 architecture, and support will increasingly focus on this architecture in the future. Changes to upstream dependencies of Elastic’s machine learning functionality have made it unviable for Elastic to continue to build machine learning on macOS x86_64.

Impact
To continue to use machine learning functionality on macOS please switch to an arm64 machine (Apple silicon). Alternatively, it will still be possible to run Elasticsearch with machine learning enabled in a Docker container on macOS x86_64.

Details
Previously, the following classes of malformed input were deprecated but not rejected in the action lines of the a bulk request: missing closing brace; additional keys after the action (which were ignored); additional data after the closing brace (which was ignored). They will now be considered errors and rejected.

Impact
Users must provide well-formed input when using the bulk API. (They can request REST API compatibility with v8 to get the previous behaviour back as an interim measure.)

Details
When a timeout occurs in most REST requests, whether via a per-request timeout, or a system default, the request would return a 5xx response code. The response code from those APIs when a timeout occurs is now 429.

Impact
Adjust any code relying on retrying on 5xx responses for timeouts to look for a 429 response code and inspect the response to determine whether a timeout occured.

Details
This change modifies the JSON format of error messages returned to REST clients when detailed messages are turned off. Previously, JSON returned when an exception occurred, and http.detailed_errors.enabled: false was set, just consisted of a single "error" text field with some basic information. Setting http.detailed_errors.enabled: true (the default) changed this field to an object with more detailed information. With this change, non-detailed errors now have the same structure as detailed errors. "error" will now always be an object with, at a minimum, a "type" and "reason" field. Additional fields are included when detailed errors are enabled. To use the previous structure for non-detailed errors, use the v8 REST API.

Impact
If you have set http.detailed_errors.enabled: false (the default is true) the structure of JSON when any exceptions occur now matches the structure when detailed errors are enabled. To use the previous structure for non-detailed errors, use the v8 REST API.

Details
This PR removes all references to V_7 in the Rest API. V7 features marked for deprecation have been removed.

Impact
This change is breaking for any external plugins/clients that rely on the V_7 enum or deprecated version 7 functionality

Details
The POST /_cluster/reroute API no longer returns the cluster state in its response. The ?metric query parameter to this API now has no effect and its use will be forbidden in a future version.

Impact
Cease usage of the ?metric query parameter when calling the POST /_cluster/reroute API.

Details
The following APIs no longer accept the ?local query parameter: GET /_alias, GET /_aliases, GET /_alias/{name}, HEAD /_alias/{name}, GET /{index}/_alias, HEAD /{index}/_alias, GET /{index}/_alias/{name}, HEAD /{index}/_alias/{name}, GET /_cat/aliases, and GET /_cat/aliases/{alias}. This parameter has been deprecated and ignored since version 8.12.

Impact
Cease usage of the ?local query parameter when calling the listed APIs.

Details
The deprecated range query parameters to, from, include_lower, and include_upper are no longer supported.

Impact
Users should use lt, lte, gt, and gte query parameters instead.

Details
The original, tech-preview api for vector search, _knn_search, has been removed in v9. For all vector search operations, you should utilize the _search endpoint.

Impact
The _knn_search API is now inaccessible without providing a compatible-with flag for v8.

Details
The deprecated highlighting force_source parameter is no longer supported.

Impact
Users should remove usages of the force_source parameter from their search requests.

Details
The /{index}/_unfreeze REST endpoint is no longer supported. This API was deprecated, and the corresponding /{index}/_freeze endpoint was removed in 8.0.

Impact
None, since it is not possible to have a frozen index in a version which is readable by Elasticsearch 9.0

Details
Previously, setting the input.search.request.types field in the payload when creating a watcher to an empty array was allowed, although it resulted in a deprecation warning and had no effect (and any value other than an empty array would result in an error). Now, support for this field is entirely removed, and the empty array will also result in an error.

Impact
Users should stop setting this field (which did not have any effect anyway).

Details
Connector APIs now enforce the manage_connector and monitor_connector privileges (introduced in 8.15), replacing the previous reliance on index-level permissions for .elastic-connectors and .elastic-connectors-sync-jobs in API calls.

Impact
Connector APIs now require manage_connector and monitor_connector privileges

Details
Before this change, in case of shard failures, EQL queries always returned an error. With this change, they will keep running and will return partial results.

Impact
EQL queries that would previously fail due to shard failures, will now succeed and return partial results. The previous defaults can be restored by setting xpack.eql.default_allow_partial_results cluster setting to false or setting with allow_partial_search_results to false in the query request.

Details
When providing a seed parameter to a random_score function in the function_score query but NOT providing a field, the default field is switched from _id to _seq_no.

Impact
The random scoring and ordering may change when providing a seed and not providing a field to a random_score function.

Details
The previous semantic_text format used a complex subfield structure in _source to store the embeddings. This complicated interactions/integrations with semantic_text fields and _source in general. This new semantic_text format treats it as a normal text field, where the field’s value in _source is the value assigned by the user.

Impact
Users who parsed the subfield structure of the previous semantic_text format in _source will need to update their parsing logic. The new format does not directly expose the chunks and embeddings generated from the input text. The new format will be applied to all new indices, any existing indices will continue to use the previous format.

Details
data_frame_transforms_admin and data_frame_transforms_user were deprecated in Elasticsearch 7 and are being removed in Elasticsearch 9. data_frame_transforms_admin is now transform_admin. data_frame_transforms_user is now transform_user. Users must call the _update API to replace the permissions on the Transform before the Transform can be started.

Impact
Transforms created with either the data_frame_transforms_admin or the data_frame_transforms_user role will fail to start. The Transform will remain in a stopped state, and its health will be red while displaying permission failures.

Details
-| Certificate-based remote cluster security model is deprecated and will be removed in a future major version. Users are encouraged to migrate remote clusters from certificate to API key authentication. The API key-based security model is preferred way to configure remote clusters, as it allows to follow security best practices when setting up remote cluster connections and defining fine-grained access control.

Impact
-| If you have configured remote clusters with certificate-based security model, you should migrate remote clusters from certificate to API key authentication. Configuring a remote cluster using certificate authentication, generates a warning in the deprecation logs.

Details
Please describe the details of this change for the release notes. You can use asciidoc.

Impact
Please describe the impact of this change to users

Details
Passing a document with a _type property is deprecated in the /_ingest/pipeline/{id}/_simulate and /_ingest/pipeline/_simulate APIs.

Impact
Users should already have stopped using mapping types, which were deprecated in Elasticsearch 7. This deprecation warning will fire if they specify mapping types on documents pass to the simulate pipeline API.

Details
The elser service of the inference API will be removed in an upcoming release. Please use the elasticsearch service instead.

Impact
In the current version there is no impact. In a future version, users of the elser service will no longer be able to use it, and will be required to use the elasticsearch service to access elser through the inference API.

Details
Rollup is already deprecated since 8.11.0 via documentation and since 8.15.0 it is no longer possible to create new rollup jobs in clusters without rollup usage. This change updates the rollup APIs to emit a deprecation warning.

Impact
Returning a deprecation warning when using one of the rollup APIs.

Details
As part of the migration from 7.x to 8.x, the .data-frame-notifications-1 index was deprecated and replaced with the .transform-notifications-000002 index. The index is no longer created by default, all writes are directed to the new index, and any clusters with the deprecated index will have an alias created to ensure that reads are still retrieving data that was written to the index before the migration to 8.x. This change removes the alias from the deprecated index in 9.x. Any clusters with the alias present will retain it, but it will not be created on new clusters.

Impact
No known end user impact.