Adjust exception thrown when unable to load hunspell dict #123743
Non existing synonyms sets do not fail shard recovery for indices #125659 (issue: #125603)

Authentication

Fix NPE for missing Content Type header in OIDC Authenticator #126191

CAT APIs

Fix cat_component_templates documentation #120487

Data streams

Avoid updating settings version in MetadataMigrateToDataStreamService when settings have not changed #118704
Ensure removal of index blocks does not leave key with null value #122246
Match dot prefix of migrated DS backing index with the source index #120042
Refresh source index before reindexing data stream index #120752 (issue: #120314)
Updating TransportRolloverAction.checkBlock so that non-write-index blocks do not prevent data stream rollover #122905
ReindexDataStreamIndex bug in assertion caused by reference equality #121325

Downsampling

Copy metrics and default_metric properties when downsampling aggregate_metric_double #121727 (issues: #119696, #96076)
Improve downsample performance by avoiding to read unnecessary dimension values when downsampling. #124451

EQL

Fix EQL double invoking listener #124918

ES|QL

Avoid over collecting in Limit or Lucene Operator #123296
Correct line and column numbers of missing named parameters #120852
Drop null columns in text formats #117643 (issue: #116848)
ES|QL - date nanos range bug? #125345 (issue: #125439)
ES|QL - Fix lucene push down behavior when a range contains nanos and millis #125595
Fix ROUND() with unsigned longs throwing in some edge cases #119536
Fix TopN row size estimate #119476 (issue: #106956)
Fix AbstractShapeGeometryFieldMapperTests #119265 (issue: #119201)
Fix ReplaceMissingFieldsWithNull #125764 (issues: #126036, #121754, #126030)
Fix a bug in TOP #121552
Fix async stop sometimes not properly collecting result #121843 (issue: #121249)
Fix attribute set equals #118823
Fix double lookup failure on ES|QL #115616 (issue: #111398)
Fix queries with document level security on lookup indexes #120617 (issue: #120509)
Fix usage of already released null block in ValueSourceReaderOperator #126411 (issue: #125850)
Fix writing for LOOKUP status #119296 (issue: #119086)
Implicit numeric casting for CASE/GREATEST/LEAST #122601 (issue: #121890)
Lazy collection copying during node transform #124424
Limit memory usage of fold #118602
Limit size of query #117898
Make numberOfChannels consistent with layout map by removing duplicated ChannelSet #125636
Reduce iteration complexity for plan traversal #123427
Remove redundant sorts from execution plan #121156
Use a must boolean statement when pushing down to Lucene when scoring is also needed #124001 (issue: #123967)

Health

Do not recommend increasing max_shards_per_node #120458

ILM+SLM

Add missing timeouts to rest-api-spec SLM APIs #119447

Indices APIs

Fix NPE in rolling over unknown target and return 404 #125352
Include hidden indices in DeprecationInfoAction #118035 (issue: #118020)
Preventing ConcurrentModificationException when updating settings for more than one index #126077
Updates the deprecation info API to not warn about system indices and data streams #122951
Avoid hoarding cluster state references during rollover #124266

Inference

[Inference API] Put back legacy EIS URL setting #121207

Infra/Core

Epoch Millis Rounding Down and Not Up 2 #118353
Fix system data streams to be restorable from a snapshot #124651 (issue: #89261)
Have create index return a bad request on poor formatting #123761
Include data streams when converting an existing resource to a system resource #121392
System Index Migration Failure Results in a Non-Recoverable State #122326
System data streams are not being upgraded in the feature migration API #123926
Wrap jackson exception on malformed json string #114445 (issue: #114142)

Infra/Logging

Move SlowLogFieldProvider instantiation to node construction #117949

Infra/Plugins

Remove unnecessary entitlement #120959
Restrict agent entitlements to the system classloader unnamed module #120546

Ingest Node

Fix geoip databases index access after system feature migration #121196
Fix geoip databases index access after system feature migration (again) #122938
Fix geoip databases index access after system feature migration (take 3) #124604

Machine Learning

Add ElasticInferenceServiceCompletionServiceSettings #123155
Add enterprise license check to inference action for semantic text fields #122293
Avoid potentially throwing calls to Task#getDescription in model download #124527
Change format for Unified Chat #121396
Fix AlibabaCloudSearchCompletionAction not accepting ChatCompletionInputs #125023
Fix get all inference endponts not returning multiple endpoints sharing model deployment #121821
Fix serialising the inference update request #122278
Fixing bedrock event executor terminated cache issue #118177 (issue: #117916)
Fixing bug setting index when parsing Google Vertex AI results #117287
Retry on streaming errors #123076
Set Connect Timeout to 5s #123272
Set default similarity for Cohere model to cosine #125370 (issue: #122878)
Updating Inference Update API documentation to have the correct PUT method #121048
[Inference API] Fix output stream ordering in InferenceActionProxy #124225

Mapping

Avoid serializing empty _source fields in mappings #122606
Fix realtime get of nested fields with synthetic source #119575 (issue: #119553)
Merge field mappers when updating mappings with [subobjects:false] #120370 (issue: #120216)
Merge template mappings properly during validation #124784 (issue: #123372)
Tweak copy_to handling in synthetic _source to account for nested objects #120974 (issue: #120831)

Ranking

Fix LTR query feature with phrases (and two-phase) queries #125103

Search

Catch and handle disconnect exceptions in search #115836
Fix leak in DfsQueryPhase and introduce search disconnect stress test #116060 (issue: #115056)
Handle long overflow in dates #124048 (issue: #112483)
Handle search timeout in SuggestPhase #122357 (issue: #122186)
In this pr, a 400 error is returned when _source / _seq_no / _feature / _nested_path / _field_names is requested, rather a 5xx #117229
Load FieldInfos from store if not yet initialised through a refresh on IndexShard #125650 (issue: #125483)
Log stack traces on data nodes before they are cleared for transport #125732
Re-enable parallel collection for field sorted top hits #125916
Skip fetching _inference_fields field in legacy semantic_text format #121720
Support indices created in ESv6 and updated in ESV7 using different LuceneCodecs as archive in current version. #125389
Test/107515 RestoreTemplateWithMatchOnlyTextMapperIT #120898
Fix/SearchStatesIt_failures #117729
CrossClusterIT testCancel failure #117750 (issue: #108061)

Snapshot/Restore

Fork post-snapshot-delete cleanup off master thread #122731
This PR fixes a bug whereby partial snapshots of system datastreams could be used to restore system features. #124931
Use the system index descriptor in the snapshot blob cache cleanup task #120937 (issue: #120518)

Suggesters

Return an empty suggestion when suggest phase times out #122575 (issue: #122548)

TLS

Set keyUsage for generated HTTP certificates and self-signed CA #126376 (issue: #117769)

Transform

If the Transform is configured to write to an alias as its destination index, when the delete_dest_index parameter is set to true, then the Delete API will now delete the write index backing the alias #122074 (issue: #121913)

Vector Search

Apply default k for knn query eagerly #118774
Fix bbq_hnsw merge file cleanup on random IO exceptions #119691 (issue: #119392)
Knn vector rescoring to sort score docs #122653 (issue: #119711)
Return appropriate error on null dims update instead of npe #125716

Watcher

Watcher history index has too many indexed fields - #117701 (issue: #71479)

Deprecations

edit

Indices APIs

Increase the frozen indices deprecation level to CRITICAL #119879

Infra/Core

Add deprecation warning to TransportHandshaker #123188

Infra/REST API

Enhancement/v7 critical deprecation logging #118298

Enhancements

edit

Authentication

Allow SSHA-256 for API key credential hash #120997

Authorization

Allow kibana_system user to manage .reindexed-v8-internal.alerts indices #118959
Do not fetch reserved roles from native store when Get Role API is called #121971
Make reserved built-in roles queryable #117581
[Security Solution] allows kibana_system user to manage .reindexed-v8-* Security Solution indices #119054

CCS

Resolve/cluster allows querying for cluster info only (no index expression required) #119898

Data streams

Add action to create index from a source index #118890
Add index and reindex request settings to speed up reindex #119780
Add rest endpoint for create_from_source_index #119250
Add sanity check to ReindexDatastreamIndexAction #120231
Adding a migration reindex cancel API #118291
Adding get migration reindex status #118267
Consistent mapping for OTel log and event bodies #120547
Filter deprecated settings when making dest index #120163
Ignore closed indices for reindex #120244
Improve how reindex data stream index action handles api blocks #120084
Initial work on ReindexDatastreamIndexAction #116996
Make requests_per_second configurable to throttle reindexing #120207
Optimized index sorting for OTel logs #119504
Reindex data stream indices on different nodes #125171
Report Deprecated Indices That Are Flagged To Ignore Migration Reindex As A Warning #120629
Retry ILM async action after reindexing data stream #124149
Update data stream deprecations warnings to new format and filter sea… #119097

Distributed

Metrics for incremental bulk splits #116765

Downsampling

Improve downsample performance by buffering docids and do bulk processing #124477
Improve rolling up metrics #124739

EQL

Add support for partial shard results #116388
Optional named arguments for function in map #118619

ES|QL

Add ES|QL cross-cluster query telemetry collection #119474
Add a LicenseAware interface for licensed Nodes #118931 (issue: #117405)
Add a PostAnalysisAware, distribute verification #119798
Add a standard deviation aggregating function: STD_DEV #116531
Add cluster level reduction #117731
Add nulls support to Categorize #117655
Async search responses have CCS metadata while searches are running #117265
Backport Term query for ES|QL to 8.x #118135
Backport scoring support in ES|QL to 8.x branch #117747
Check for early termination in Driver #118188
Do not serialize EsIndex in plan #119580
ES|QL - Remove restrictions for disjunctions in full text functions #118544
ES|QL - enabling scoring with METADATA _score #113120
ES|QL Add ES|QL hash function #117989
ES|QL Support IN operator for Date nanos #119772 (issue: #118578)
ES|QL: CATEGORIZE as a BlockHash #114317
ES|QL: Enterprise license enforcement for CCS #118102
ES|QL: Partial result on demand for async queries #118122
Enable KQL function as a tech preview #119730
Enable LOOKUP JOIN in non-snapshot builds #121193 (issue: #121185)
Enable node-level reduction by default #119621
Enable physical plan verification #118114
ES|QL - Support date nanos in date extract function #120727 (issue: #110000)
ES|QL - support date nanos in date format function #120143 (issue: #109994)
ES|QL Support date nanos on date diff function #120645 (issue: #109999)
ES|QL bucket function for date nanos #118474 (issue: #118031)
ES|QL compare nanos and millis #118027 (issue: #116281)
ES|QL implicit casting for date nanos #118697 (issue: #118476)
Extend TranslationAware to all pushable expressions #120192
Hash functions #118938
Implement a MetricsAware interface #121074
LOOKUP JOIN using field-caps for field mapping #117246
Lookup join on multiple join fields not yet supported #118858
Move scoring in ES|QL out of snapshot #120354
Optimize ST_EXTENT_AGG for geo_shape and cartesian_shape #119889
Push down StartsWith and EndsWith functions to Lucene #123381 (issue: #123067)
Push down filter passed lookup join #118410
Resume Driver on cancelled or early finished #120020
Reuse child outputSet inside the plan where possible #124611
Rewrite TO_UPPER/TO_LOWER comparisons #118870 (issue: #118304)
ST_EXTENT_AGG optimize envelope extraction from doc-values for cartesian_shape #118802
Smarter field caps with subscribable listener #116755
Support some stats on aggregate_metric_double #120343 (issue: #110649)
Take named parameters for identifier and pattern out of snapshot #121850
Term query for ES|QL #117359
Update grammar to rely on indexPattern instead of identifier in join target #120494
_score should not be a reserved attribute in ES|QL #118435 (issue: #118460)

Engine

Enhance add-block API to flush and add verified metadata #119743

Experiences

Integrate IBM watsonx to Inference API for re-ranking task #117176

Geo

Optimize indexing points with index and doc values set to true #120271

ILM+SLM

Add a replicate_for option to the ILM searchable_snapshot action #119003

Indices APIs

Add remove_index_block arg to _create_from api #120548
Remove index blocks by default in create_from #120643
introduce new categories for deprecated resources in deprecation API #120505

Inference

[Inference API] Rename model_id prop to model in EIS sparse inference request body #122398
Add version prefix to Inference Service API path #117696
Update sparse text embeddings API route for Inference Service #118369
[Elastic Inference Service] Add ElasticInferenceService Unified ChatCompletions Integration #118871

Infra/CLI

Ignore _JAVA_OPTIONS #124843
Strengthen encryption for elasticsearch-keystore tool to AES 256 #119749

Infra/Core

Improve size limiting string message #122427
Return unique deprecation for old indices with incompatible date formats #124597

Infra/REST API

A new query parameter ?include_source_on_error was added for create / index, update and bulk REST APIs to control if to include the document source in the error response in case of parsing errors. The default value is true. #120725

Ingest Node

Optimize IngestCtxMap construction #120833
Optimize IngestDocMetadata isAvailable #120753
Optimize IngestDocument FieldPath allocation #120573
Optimize some per-document hot paths in the geoip processor #120824
Returning ignored fields in the simulate ingest API #117214

Logs

Add LogsDB option to route on sort fields #116687
Add a new index setting to skip recovery source when synthetic source is enabled #114618
Configure index sorting through index settings for logsdb #118968 (issue: #118686)
Optimize loading mappings when determining synthetic source usage and whether host.name can be sorted on. #120055

Machine Learning

Add Inference Unified API for chat completions for OpenAI #117589
Add Jina AI API to do inference for Embedding and Rerank models #118652
Add enterprise license check for Inference API actions #119893
Adding chunking settings to IbmWatsonxService #114914
Adding default endpoint for Elastic Rerank #117939
Adding endpoint creation validation for all task types to remaining services #115020
Check for presence of error object when validating streaming responses from integrations in the inference API #118375
Ignore failures from renormalizing buckets in read-only index #118674
Inference duration and error metrics #115876
Migrate stream to core error parsing #120722
Remove all mentions of eis and gateway and deprecate flags that do #116692
Remove deprecated sort from reindex operation within dataframe analytics procedure #117606
Retry on ClusterBlockException on transform destination index #118194

Mapping

Add Optional Source Filtering to Source Loaders #113827

Network

Allow http unsafe buffers by default #116115
Http stream activity tracker and exceptions handling #119564
Remove HTTP content copies #117303
ConnectTransportException returns retryable BAD_GATEWAY #118681 (issue: #118320)

Ranking

Set default reranker for text similarity reranker to Elastic reranker #120551

Search

Add match support for semantic_text fields #117839
Add support for sparse_vector queries against semantic_text fields #118617
Add support for knn vector queries on semantic_text fields #119011
Adding linear retriever to support weighted sums of sub-retrievers #120222
Feat: add a user-configurable timeout parameter to the _resolve/cluster API #120542
Make semantic text part of the text family #119792
Only aggregations require at least one shard request #115314
Prevent data nodes from sending stack traces to coordinator when error_trace=false #118266
Propagate status codes from shard failures appropriately #118016 (issue: #118482)

Snapshot/Restore

Add IMDSv2 support to repository-s3 #117748 (issue: #105135)

Store

Abort pending deletion on IndicesService close #123569

TSDB

Increase field limit for OTel metrics to 10 000 #120591

Transform

Add support for extended_stats #120340
Auto-migrate max_page_search_size #119348
Create upgrade mode #117858
Wait while index is blocked #119542
[Deprecation] Add transform_ids to outdated index #120821

Vector Search

Even better(er) binary quantization #117994
Speed up bit compared with floats or bytes script operations #117199

New features

edit

CRUD

Metrics for indexing failures due to version conflicts #119067

ES|QL

ES|QL - Add Match function options #120360
ES|QL - Allow full text functions disjunctions for non-full text functions #120291
ES|QL: Enable async get to support formatting #111104 (issue: #110926)
Expand type compatibility for match function and operator #117555
ST_EXTENT aggregation #117451 (issue: #104659)
Support ST_ENVELOPE and related (ST_XMIN, ST_XMAX, ST_YMIN, ST_YMAX) functions #116964 (issue: #104875)

Highlighting

Add Highlighter for Semantic Text Fields #118064

Infra/Core

Infrastructure for assuming cluster features in the next major version #118143

Machine Learning

ES|QL categorize with multiple groupings #118173
Support mTLS for the Elastic Inference Service integration inside the inference API #119679
[Inference API] Add node-local rate limiting for the inference API #120400

Mapping

Add option to store sparse_vector outside _source #117917
Release semantic_text as a GA feature #124670

Ranking

Add a generic rescorer retriever based on the search request’s rescore functionality #118585 (issue: #118327)

Relevance

Add Multi-Field Support for Semantic Text Fields #120128

Vector Search

Add new experimental rank_vectors mapping for late-interaction second order ranking #118804
KNN vector rescoring for quantized vectors #116663
Mark bbq indices as GA and add rolling upgrade integration tests #121105
Add new experimental rank_vectors mapping for late-interaction second order ranking #119601

Upgrades

edit

Infra/Core

Bump major version for feature migration system indices #117243
Permanently switch from Java SecurityManager to Entitlements. The Java SecurityManager has been deprecated since Java 17, and it is now completely disabled in Java 24. In order to retain a similar level of protection, Elasticsearch implemented its own protection mechanism, Entitlements. Starting with this version, Entitlements will permanently replace the Java SecurityManager. #125073
Update ASM 9.7 → 9.7.1 to support JDK 24 #118094

Machine Learning

Automatically rollover legacy .ml-anomalies indices #120885
Automatically rollover legacy ml indices #120405
Change the auditor to write via an alias #120064
Check if the anomaly results index has been rolled over #125404
Update minimum supported snapshot version for Machine Learning jobs to 8.3.0 #118166

Packaging

Update bundled JDK to Java 24 #125159

Search

Upgrade to Lucene 9.12.1 #118300

Watcher

Script for migrating .watches and .triggered_watches indices #120371

Known issues

edit

Infra/Core

Elasticsearch on Windows might fail to start, or might forbid some file-related operations, when referencing paths with a case different from the one stored by the filesystem. Windows treats paths as case-insensitive, but the filesystem stores them with case. Entitlements, the new security system used by Elasticsearch, treat all paths as case-sensitive, and can therefore prevent access to a path that should be accessible.

For example: If Elasticsearch is installed in C:\ELK\elasticsearch, and you try to launch it as c:\elk\elasticsearch\bin\elasticsearch.bat, you will get a NotEntitledException while booting. This is because Elasticsearch blocks access to c:\elk\elasticsearch, because does not match C:\ELK\elasticsearch. This issue will be fixed in a future patch release (see #126990).

As a workaround, make sure that all paths you specify have the same casing as the paths stored in the filesystem. Files and directory names should be entered as they appear in Windows Explorer or in a command prompt. This applies to paths specified in the command line, config files, environment variables and secure settings.
Active Directory authentication is blocked by default. Entitlements, the new security system used by Elasticsearch, has a policy for the x-pack-core module that is too restrictive, and does not allow the LDAP library used for AD authentication to perform outbound network connections. This issue will be fixed in a future patch release (see #126992).

As a workaround, you can temporarily patch the policy using a JVM option:
1. Create a file called ${ES_CONF_PATH}/jvm_options/workaround-127061.options.
2. Add the following line to the new file:
```
-Des.entitlements.policy.x-pack-core=dmVyc2lvbnM6CiAgLSA4LjE4LjAKICAtIDkuMC4wCnBvbGljeToKICB1bmJvdW5kaWQubGRhcHNkazoKICAgIC0gc2V0X2h0dHBzX2Nvbm5lY3Rpb25fcHJvcGVydGllcwogICAgLSBvdXRib3VuZF9uZXR3b3Jr
```
  For information about editing your JVM settings, refer to Set JVM options.

« Release notes Elasticsearch version 8.17.5 »