Update getmeili/meilisearch Docker tag to v1.41.0 #208
jansauer
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "renovate/getmeili-meilisearch-1.x"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
This PR contains the following updates:
v1.29.0→v1.41.0Release Notes
meilisearch/meilisearch (getmeili/meilisearch)
v1.41.0: 🏈Compare Source
✨ Enhancement
Support dynamic search rules with pinning by @YoEight in #6182
Introduce the Dynamic Search Rules (DSR) experimental feature, making it easy to promote the right content for the right searches with flexible, condition-based pinning. In this first version, rules can be triggered by query-based conditions such as empty queries or literal substring matches, as well as time windows. Multiple documents can be pinned in a defined order, and pins still work with filtering, pagination, facet distribution, hybrid search, and federated search.
Rules can be created or updated with
PATCH /dynamic-search-rules/{uid}and removed withDELETE /dynamic-search-rules/{uid}. In this first version, a rule can define query- or time-based conditions and pin specific documents at fixed positions in the results list.Example of rule creation/update
Default to
useNetwork: truein sharded instances by @dureuill in #6278Search requests now use the network by default when (replicated) sharding is enabled
§ Breaking changes
When
network.leaderis set in the instance,useNetworknow defaults totruein search requests when omitted.So, when a network is configured, the following:
Search request without an explicit useNetwork
is now equivalent to a federated search over multiples remotes so that all shards in the network are covered exactly once.
Equivalent federated search request
Responding with documents from all the shards:
Response
This change allows to naturally query all the documents in a sharded context.
When
network.leaderis not present (in particular, when nonetworkwas defined), then the behavior is identical to previous versions.useNetworkpicks local instance if it owns the shard by @dureuill in #6287To prevent unnecessary network activity, when deciding which remote to ask for a shard in a search over the network, Meilisearch will now always pick the local instance if it owns the shard.
More efficient FST building by @Kerollmops in #6279
It improves the construction of the word FST (word dictionary) by removing the need for a full scan of the word docids database. This should drastically improve the database's performance when inserting a large number of documents, even when inserting only a few.
🪲 Bug fixes
🔩 Miscellaneous
New Contributors
v1.40.0: 🦈Compare Source
This release introduced support for the
distinctattribute in federated search, enabling cross-index distinct attributes with facet distribution support. Additionally, significant performance improvements were delivered, including faster federated search (approximately 100ms faster), optimized JSON document generation for better handling of large documents and a much better memory usage for large workloads.✨ Enhancement
Support
distinctin federated search by @dureuill in #6214The
distinctattribute can now be passed to thefederationobject in federated search to apply a global, cross-index and cross-remote distinct computation to the results.Example of a federated search request with distinct
Sample response to a federated search request with distinct
Note the following to apply the distinct attribute at the federation level:
distinctat the query level at the same time as the federation level is disallowed and will return a HTTP 400 error.Improve performance of federated search by @dureuill in #6229
Improves performance of federated search: about 100ms faster for all requests. Improves reliability of the HTTP server: the server will no longer be blocked when too many federated search requests are being processed.
Optimize the generation of JSON documents by @Kerollmops in #6257
Addresses performance issues that users might encounter when requesting large documents. Additionally, performance is enhanced when users request only a small subset of fields from large documents.
Use the latest version of mimalloc to improve memory usage by @Kerollmops in #6201
Updates mimalloc from v2 to v3, improving memory sharing between threads and significantly reducing memory usage on large workloads. It also overrides the allocator to use mimalloc at linking time, allowing LMDB, Meilisearch, and other C libraries to share their allocations for better overall memory efficiency. @Kerollmops wrote a blog post about the story behind this improvement.
Add
POST /tasks/compactfor task queue compaction by @YoEight in #6193Compacts the task queue database and reclaim space so new tasks can keep being enqueued, without deleting existing tasks. This feature is behind the
taskQueueCompactionRouteexperimental feature flag.🔐 Security
🪲 Bug fixes
🔩 Miscellaneous
X-Accel-Buffering: nofor/logs/streamroute by @YoEight in #6268--experimental-allowed-ip-networksparameter by @paulden in #62630.38.4by @GuillaumeGomez in #6211New Contributors
v1.39.0: 🐳Compare Source
What's Changed
✨ Enhancement
🔬 Cross-index document hydration
Add a new
ForeignKeyssetting allowing to hydrate documents with documents coming from other indexes.foreignKeysexperimental featureA new experimental feature,
foreignKeys, has been added to the/experimental-featureroute that must be set to true to activate the hydration.foreignKeysindex settingA new index setting,
foreignKeys, has been added to the/indexes/{index_uid}/settings:Example
With this new feature, a document shaped as follows:
Will be hydrated as follows in a search response:
By @ManyTheFish in #6047
Disable proxy response buffering on Server-Sent Events (SSE)
Add
X-Accel-Buffering: noonPOST /chats/{workspace_uid}/chat/completionswhen the streaming mode is activated.By @YoEight in #6228
🪲 Bug fixes
Fix a memory leak in the indexation pipeline by @Kerollmops in #6212
We fixed an important memory leak caused by an invalid use of bumpalo. If you've seen Meilisearch using more and more memory over time, this issue is no longer visible in the latest version. If you want to read more about the bumpalo-trap we felt in, here is a good summary. It looks like this leak was introduced in v1.12, so approximately a year ago...
Avoid losing tasks by applying deltas on the wtxn view by @Kerollmops in #6222
We addressed a small performance regression introduced in v1.38.1 that affected sending updates to the engine during task deletion. We've restored the task deletion performance to match v1.38.0 levels and ensured no race conditions occur while still allowing updates during deletion.
🔩 Miscellaneous
Full Changelog: https://github.com/meilisearch/meilisearch/compare/v1.38.0...v1.39.0
v1.38.2: 🐍Compare Source
Meilisearch v1.38.2 fixes a regression introduced in Meilisearch v1.38.0 where Meilisearch could sometimes stop processing tasks after an automatic task queue cleanup performed while more tasks where being added.
We recommend that all users in v1.38.0 and v1.38.1 migrate to v1.38.2.
Note that the dumpless upgrade process can take up to one minute before the HTTP server becomes available when migrating from a previous v1.38 version.
Full Changelog: https://github.com/meilisearch/meilisearch/compare/v1.38.1...v1.38.2
v1.38.1: 🐍Compare Source
Meilisearch v1.38.1 fixes a regression we have seen in task processing, which caused the engine to stop processing tasks.
🪲 Bug fixes
Full Changelog: https://github.com/meilisearch/meilisearch/compare/v1.38.0...v1.38.1
v1.38.0: 🐍Compare Source
Meilisearch v1.38 drastically improves indexing performance for embeddings, cleans up the task queue of orphan tasks and batches, ensures they are correctly deleted now, fixes issues users may encounter with OpenAI or VoyageAI when generating embeddings, and improves the tooling for generating routes in the documentation.
✨ Enhancement
We drastically reduced embedding indexing time by eliminating full database scans. It is more efficient to add a couple of embeddings to a large database now.
🪲 Bug fixes
connection reset by peererrors when embedding documents or search queries using a remote embedder. By @dureuill in #6191🔩 Miscellaneous
The reference API is now generated from the source code, but it used to miss some actix-declared routes. It is now mandatory to attach a route when using the newly-provided
routes::routesandroutes::pathmacros, which ensures the route is added to the reference. Please refer to the documentation of the individual macros and the many examples throughout the codebase for more details.New Contributors
v1.37.0Compare Source
Meilisearch v1.37 introduces replicated sharding, removes the
vectorStoreSettingexperimental feature, stabilizes our new vector store for best performance, adds a security fix and miscellaneous improvements.✨ Improvements
§ Replicated sharding
§ Breaking changes
networkobjects sent to thePATCH /networkroute must now contain at least oneshardobject containing at least one remote whenleaderis notnull.Existing databases will be migrated automatically when upgraded with
--experimental-dumpless-upgradewhenleaderis notnull, such that for each remote:remoteslist: the remote with the same name as the shard.This change will not cause any document to be resharded.
To be able to upgrade without resharding, the migration uses the same name for remotes and for shards. However, in new configurations, we recommend using different names for shards and remotes.
Example of migration
For instance, the following network object:
is converted to:
Addition of
network.shardsThe
networkobject for routesPATCH /networkandGET /networknow contains the new fieldshards, which is an object whose values areshardobjects, and keys the name of each shard.Each
shardobject contains a single fieldremotes, which is an array of strings, each string representing the name of an existing remote.Convenience fields
The
shardobjects inPATCH /networkcontain the additional fieldsaddRemotesandremoveRemotesmeant for convenience:shard.addRemotesto add these remotes to the list of remotes of a shard.shard.removeRemotesto remove these remotes from the list of remotes of a shard.null,shard.remoteswill completely override the existing list of remotes for a shard.null, then the order of application isshard.remotes, thenshard.addRemotes, thenshard.removeShards.Adding a new shard with some remotes
Remotes
ms-0andms-1own the new shards-a.Fully overriding the list of remotes owning a shard
ms-2is now the sole owner ofs-a, replacingms-0andms-1.Adding a remote without overriding the list of remotes owning a shard
ms-0andms-2are now the owners ofs-a.Removing a remote without overriding the list of remotes owning a shard
ms-0is now the sole owner ofs-a.Entirely removing a shard from the list of shards
Set the shard to
null:Or set its
remoteslist to the empty list:network.shardsvalidityWhen
network.leaderis notnull, eachshardobject innetwork.shardsmust:remotesthat exist in the list ofremotes.Additionally,
network.shardsmust contain at least one shard.Failure to meet any of these conditions will cause the
PATCH /networkroute to respond with400 invalid_network_shards.Change in sharding logic
Documents are now sharded according to the list of shards declared in the network rather than the list of remotes. All remotes owning a shard will process the documents that belong to this shard, allowing for replication.
Example of replication
The following configuration defines 3 remotes
0,1and2, and 3 shardsA,B,C, such that each remote owns two shards, achieving replication (losing one remote does not lose any document).useNetworktakesnetwork.shardsinto accountWhen
useNetwork: trueis passed to a search query, it is expanded to multiple queries such that each shard declared innetwork.shardsappears exactly once, associated with a remote that owns that shard.This ensures that there is no missing or duplicate documents in the results.
_shardfiltersWhen the
networkexperimental feature is enabled, then it becomes possible to filter documents depending on the shard they belong to.Given
s-aands-bthe names of two shards declared innetwork.shards, then:_shard = "s-a"in afilterparameter to the search or documents fetch will return the documents that belong tos-a._shard != "s-a"will return the documents that do not belong tos-a_shard IN ["s-a", "s-b"]will return the documents that belong tos-aor tos-b.You can use these new filters in manual remote federated search to create a partitioning over all shards in the network.
Update instructions
When updating your Meilisearch network using dumpless upgrade, please observe the following guidelines:
PATCH /networkroute until all remotes of the network are finished updatinguseNetwork: true, call them on un-updated remotes. Calling it on already updated remotes will cause un-updated remotes to fail the search as they don't know about the_shardfilters.By @dureuill in #6128
§ Remove
vectorStoreSettingexperimental featureThe new HNSW vector store (hannoy) has been stabilized and is now the only supported vector store in Meilisearch.
As a result, updating to v1.37.0 will migrate all remaining legacy vector store indexes (using arroy) to hannoy, and the
vectorStoreSettingexperimental feature is no longer available.By @Kerollmops in #6176
Improve indexing performance for embeddings
We removed a computationally expensive step from vector indexing.
On a DB with 20M documents, this removes 300s per indexing batch of 1100s.
By @Kerollmops in #6175
§ 🔒 Security
localStorageBy @Strift and @curquiza in #6186 and #6172
§ 🔩 Miscellaneous
Full Changelog: https://github.com/meilisearch/meilisearch/compare/v1.36.0...v1.37.0
v1.36.0: 🐙Compare Source
Version v1.36.0 introduces an exciting update to the ranking rules to improve the engine's relevance. It's actually the first time we've made such a change since v1.0, and we're really happy about this improvement!
✨ Enhancement
Introduce the
attributeRankandwordPositioncriteria by @Kerollmops in #6154, #6155, and #6164We released two new ranking rules that Meilisearch had already been using internally for the
attributeone, which is basically both ranking rules applied one after the other:attributeRank: A document is considered better if the query words match in a higher searchable attribute. It ignores the position of the query words in this attribute.wordPosition: A document is considered better if the query words match closer to the beginning of an attribute. The attribute rank is ignored by this rule.We continue our policy of migrating everyone to use a homemade HNSW by introducing a new dumpless upgrade step that migrates index uses the old annoy vector store to the new Hannoy one. Changing the vector store backend affects the ranking score. This step can take a couple of minutes when the number of embeddings is high, and we recommend changing the vector store backend beforehand to gain more control if needed. To do so, you must enable the
vectorStoreSettingexperimental feature and set thevectorStoreroot setting toexperimental.🪲 Bug fixes
attributesToSearchOnon an empty index by @zen-zap in #5944🔒 Security
🔩 Miscellaneous
❤️ Thanks to @zen-zap for contributing to this release!
v1.35.1Compare Source
🦋 Bug fixes
Meilisearch v1.35.1 fixes a possible task database corruption issue that would occur when using the S3 streaming snapshot method.
Details
While snapshotting the task database, sending a task to Meilisearch could corrupt it. Index DBs were not affected, so manually rebuilding the task database fixes the corruption.
By @dureuill in #6160
Full Changelog: https://github.com/meilisearch/meilisearch/compare/v1.35.0...v1.35.1
v1.35.0Compare Source
Meilisearch v1.35.0 contains improvements to the observability of the search performance, a breaking change to the recently introduced
POST /indexes/<index_uid>/fieldsroute, and other bugfixes and improvements.🌈 Improvements
Observe the search performance
The search routes accept a new field named
showPerformanceDetails. When set totrue, the search response contains a performance trace, allowing the user to betterunderstand what takes time during a search query.
impacted routes:
POST /indexes/<index_uid>/searchGET /indexes/<index_uid>/searchPOST /multi-searchPOST /indexes/<index_uid>/similarGET /indexes/<index_uid>/similarSearch
routes:
POST /indexes/<index_uid>/searchandGET /indexes/<index_uid>/searchRequest
new request parameters:
showPerformanceDetails:true/false(boolean)example
Response
new response field:
performanceDetails:{"<span>": "<human_duration>", .. }(map)example
Multi-search
route:
POST /multi-searchRequest
new request parameters:
queries.showPerformanceDetails:true/false(boolean)example
Response
new response field:
results.performanceDetails:{"<span>": "<human_duration>", .. }(map)example
Federated Search
route:
POST /multi-searchRequest
new request parameters:
federation.showPerformanceDetails:true/false(boolean)example
Response
new response field:
performanceDetails:{"<span>": "<human_duration>", .. }(map)example
Similar
routes:
POST /indexes/<index_uid>/similarandGET /indexes/<index_uid>/similarRequest
new request parameters:
showPerformanceDetails:true/false(boolean)example
Response
new response field:
performanceDetails:{"<span>": "<human_duration>", .. }(map)example
By @ManyTheFish in #6132
Stabilize multithreaded post-processing of facets and prefixes
Multithreaded post-processing of facets and prefixes speeds-up indexing on multi-core machines.
Before v1.35.0, it could be disabled with an experimental feature.
Meilisearch v1.35.0 removes that experimental feature, resulting in the multithreaded post-processing being always enabled.
Cloud users upgrading to v1.35.0 will benefit from the speed-up.
By @ManyTheFish in #6130
🦋 Fixes
Breaking change: return pagination fields in
POST /indexes/<index_uid>/fieldsrouteExample: before/after
Before:
After:
By @ManyTheFish in #6126
POST /indexes/<index_uid>/fields: Fix pattern filtering when a parent field was matching a child pattern:titlewas matching the patterntitle.to*.Example: before/after
For the following request:
Before Response,
titlefield is returned:After Response, nothing returned:
By @ManyTheFish, also in #6126
🐛 Other fixes
🔩 Miscellaneous
New Contributors
Full Changelog: https://github.com/meilisearch/meilisearch/compare/v1.34.3...v1.35.0
v1.34.3Compare Source
🐛 Bug Fixes
Full Changelog: https://github.com/meilisearch/meilisearch/compare/v1.34.2...v1.34.3
v1.34.2Compare Source
This patch fixes an accidental breaking change in v1.34.1 where Meilisearch would not start with a configuration file if
experimental_allowed_ip_networkswas not defined.Full Changelog: https://github.com/meilisearch/meilisearch/compare/v1.34.1...v1.34.2
v1.34.1Compare Source
In Meilisearch v1.34.1, we released a security fix that affect web queries initiated by Meilisearch.
🔐 Security
In Meilisearch versions v1.8 to v1.34.0, a user with an API key with write permissions to the configuration of the Meilisearch instance could set up Meilisearch to send POST or GET requests to domains and IPs local to the private network of the Meilisearch instance, effectively bypassing the firewall. The vulnerability has not yet been assigned a CVE number; it has CVE Request
1975471for CVE ID.Meilisearch v1.34.1 fixes the vulnerability by forbidding Meilisearch from making any requests to a host resolving to a non-global IP, in the sense of the IANA IPv4 Special-Purpose Address Registry or the IANA IPv6 Special-Purpose Address Registry. If you need this functionality in your Meilisearch instance, you can set private IP networks as allowed with the
--experimental-allowed-ip-networksparameter after reviewing the security implications.This is a breaking change, which is allowed for security reasons.
Allowing non-global IP networks
The
--experimental-allowed-ip-networksCLI flag and theMEILI_EXPERIMENTAL_ALLOWED_IP_NETWORKSenvironment variable control the behavior of Meilisearch with regards to non-global IP networks, with the CLI flag taking precedence over the environment variable when both are specified.192.168.0.0/16,10.0.0.0), then web requests to the networks from the list will be permitted. Web requests to the non-global IPs not part of the list will still be rejected.any, all web requests will be permitted regardless of the target IP, similar to the behavior of Meilisearch v1.34.0 and lower. Use this option when you control both the machine and the configuration of the deployed Meilisearch instance.Acknowledgments
Thanks to Gabriel Rodrigues (aka Texugo), for reporting this vulnerability and for helping us improve the security of Meilisearch.
v1.34.0Compare Source
🌈 Improvements
Easy search over your network of machines
useNetworkfield inPOST /indexes/{:indexUid}/searchThe search query object passed in the body of
POST /indexes/{:indexUid}/searchnow accepts an optional booleanuseNetwork. When present and set totrue, the search is executed "as-if" it was a remote federated search over all remotes in the network.That is, the following:
Search request
Is executed by Meilisearch as if it was the following, assuming a network of 3 Meilisearch instances with names
"0", "1" and "2":Equivalent multi-search request
Resulting in:
Search Response
useNetworkrequires thenetworkexperimental feature to be enabled.useNetworkquery parameter inGET /indexes/{:indexUid}/searchPassing
useNetwork=trueas a query parameter toGET /indexes/{:indexUid}/searchhas the same effect as passinguseNetwork: trueas a field parameter toPOST /indexes/{:indexUid}/search.queries[].useNetworkfield inPOST /multi-searchuseNetworkcan also be passed as a field of the individual queries inside of amulti-searchrequest.POST /indexes/{:indexUid}/searchfor that queryFederated search example:
Multi-search request
Multi-search response
Limitations
useNetworkare not supporteduseNetworkat the moment: doing so is not trivial implementation-wise, because chat route expects to be able to open the index (to fetch chat configs), but federated search only opens the indexes once during a short critical section.By @dureuill in #6101
Federated search supports
pageandhitsPerPagePass
federation.pageandfederation.hitsPerPagewith the same meaning as in a regular search request to use exhaustive pagination in the federated searchBy @dureuill in #6101
Speed up settings changes when removing searchable
The settings indexer is more efficient when users are removing searchable attributes from the searchable fields.
By @VedantMadane in #6109
🔒 Security
Solves a low-severity timing attack vulnerability on key comparison by using constant-time comparison
By @curquiza in #6077
🔩 Maintenance
Remove some unwanted dependencies
New Contributors
❤️ Thanks again @VedantMadane for the contribution to this release!
Full Changelog: https://github.com/meilisearch/meilisearch/compare/v1.33.1...v1.34.0
v1.33.1: 🐞Compare Source
In Meilisearch v1.33.1, we released a security fix that affected the dump import, improvements to vector store relevance in massive databases, and a fix related to ranking scores.
🔐 Security Fix
All versions of Meilisearch before v1.33.0 are vulnerable to a path traversal vulnerability involving the dump import functionality.
Importing a specially crafted dump could result in giving access to the Meilisearch instance to arbitrary, specifically formatted files, present on the file system of the Meilisearch instance.
✨ Enhancements
🐛 Bug Fixes
Full Changelog: https://github.com/meilisearch/meilisearch/compare/v1.33.0...v1.33.1
v1.33.0: 🐞Compare Source
✨ Enhancement
Add
/fieldsroute to get all the fields of an index by @YoEight in #6082Adds a new POST
/indexes/{indexUid}/fieldsendpoint that returns detailed metadata about all fields in an index. This endpoint provides comprehensive information about each field's configuration, including display, search, filtering, and localization settings.Implement parallel cleanup of old field IDs by @Kerollmops in #6100
We reduce the time required to perform the dumpless upgrade for instances before v1.32.0 by multi-threading database fetches. By doing that, we noticed improvements from 2 hours and 50 minutes to a bit less than 7 minutes.
Bump hannoy to 0.1.4-nested-rtxns by @Kerollmops in #6103
We updated our internal vector store to speed up the dumpless upgrade by improving the graph rebuilding and improving the speed and relevance of the search by using the explore factor as a limit to stop document searches rather than the query limit.
🪲 Bug fixes
🔒 Security
🔩 Miscellaneous
Thank you @Vipul-045 for your first contribution ❤️
v1.32.2: 🐟Compare Source
🐛 Bug fixes
Full Changelog: https://github.com/meilisearch/meilisearch/compare/v1.32.1...v1.32.2
v1.32.1: 🐟Compare Source
🌈 Improvements
Skip cleaning up the field-ID-based databases
Introduce a
MEILI_EXPERIMENTAL_DISABLE_FID_BASED_DATABASES_CLEANUPenv var to opt out of the field ID-based database cleanup when upgrading a Meilisearch from versions inferior to 1.32.0.Full Changelog: https://github.com/meilisearch/meilisearch/compare/v1.32.0...v1.32.1
v1.32.0: 🐟Compare Source
🌈 Improvements
Log Search performance trace
Introduces comprehensive progress tracking and logging for search operations in Meilisearch. It adds detailed timing information for each step of the search process, enabling better observability and performance analysis.
Extract document operations from payloads in parallel
We accelerate document indexing by processing a large number of tasks in batches or a large number of records in parallel. We expedited the preparation of the payloads by extracting the various changes and assigning internal IDs in parallel. We achieved a 7x speedup on a four-million-document insertion using four CPUs, and the performance scales with the number of CPUs.
The
indexedDocumentsfield in tasks using skipCreation no longer precisely reflects the number of document operations performed, specifically forPOSTandPUToperations. This count may be higher than the actual number of operations, but it doesn't affect the computation; only the reported count is impacted. We prioritize speed over perfect accuracy here, and the documents are still correctly indexed as before.🐛 Bug fixes
Vector sort: Bucket documents with same similarity
Fixed vector sort bucketing so documents with identical similarity scores are grouped together, ensuring subsequent ranking rules are applied correctly.
Properly Delete Documents from FID-Based Databases
Fixes a bug where changing searchableAttributes from ["*"] to a subset of fields left orphaned data in fid-based databases, causing corruption and warnings during search.
Rebuild the graph links when dumpless-upgrading
Bumps hannoy to v0.1.3-nested-rtxns, which fixes graph-related recall issues and adds a method to rebuild graph links to recover previously malformed graphs. Also fixes a minor issue in the dumpless upgrade flow where the upgrade description was not displayed correctly and related operations were not properly associated with the upgrade.
🛠️ Maintenance and Misc.
Update JS SDKs tests to use
pnpminstead ofyarnUpdated the JavaScript SDK tests to use pnpm instead of yarn in CI workflows, switching the package manager across test configurations to ensure the SDK test suite runs correctly and consistently with the current tooling.
Adapt JS tests in SDK tests CI
Updated the SDK tests CI workflow for the JavaScript SDKs
Bump lru from 0.16.2 to 0.16.3
Fix Stacked Borrows violation in
IterMut.Full Changelog: https://github.com/meilisearch/meilisearch/compare/v1.31.0...v1.32.0
v1.31.0: 🦃Compare Source
🌈 Improvements
🗄️ [Enterprise Edition] Make the S3-streaming snapshots an Enterprise Edition feature
The recently introduced S3-streaming snapshots feature is now available as an Enterprise Edition feature. From now on, you'll need a license to use this feature when using Meilisearch as a self-hosted solution. Note that this version introduces a breaking change regarding the S3-streaming snapshot feature, which is no longer available in the Community Edition; however, on-disk snapshots remain accessible. You can read our BUSL license for contact information, if you like. Note that if you are using the Community Edition of Meilisearch between version 1.25 and this release, you can freely use the S3 Streaming feature without an Enterprise Edition license.
🔏 [Enterprise Edition] Support AWS IRSA to authenticate to S3 for snapshotting
We just introduced support for IRSA authentication to do snapshots on AWS. IRSA allows the use of short-lived access and secret keys to upload snapshots through S3. This feature is available under the Enterprise Edition and can be accessed through two experimental CLI parameters.
✍️ Allow strict document update without creating missing documents
Adds an optional
skipCreationboolean query parameter toPOSTandPUTon/indexes/{index}/documents. When set totrue, documents that don't exist in the index are silently ignored rather than created. Default isfalse, preserving existing behavior.🐛 Bug fixes
🛠️ Maintenance and Misc.
v1.30.1: 🐸Compare Source
What's Changed
🐛 Bug Fixes
Fix task attribution during index swap to prevent cross-index task loss by @YoEight in #6059
This change fixes the parenting of the tasks when doing an index swap.
Hotfix: log a warning when FieldidsWeightsMap is missing entry by @ManyTheFish in #6064
This hotfix avoids the encounter of following error during a search request:
🫂 New Contributors
v1.30.0: 🐸Compare Source
🌈 Improvements
Up and down network scaling
Since v1.19.0, Meilisearch Enterprise Edition allows the automatic sharding of documents over multiple Meilisearch instances, enabling scaling to more documents than a single instance would accommodate.
Meilisearch v1.30.0 adds the ability to modify the number of participants in sharding, without having to start over sending documents to a new cluster containing the number of desired machines.
To make this possible, Meilisearch v1.30.0 introduces breaking changes. These breaking changes only affect the users of the experimental
networkfeature who enabled automatic sharding (network.sharding = true). Users of the stable features are not affected.Usage
Setting up the initial network
PATCH /network:Upgrading from v1.29 or earlier
networkinstance withshardingset totrue, then thenetworkobject will be modified so that the leader is the first remote in alphabetic order. For instance, if you network contains remotes:A,B,C, the leader will be set toA.Adding a new remote
PATCH /networkon the leader with the information about the new remote:ms2).networkTopologyChangetask that will "rebalance" the documents between the existing remotes and the new remote, that is,ms0andms1will send parts of their documents toms2Removing a remote
PATCH /networkon the leader by setting any removed remote tonull:ms2that will then no longer participate in the network).networkTopologyChangetask that will "rebalance" the documents between the remaining remotes, that is,ms2will send its documents betweenms0andms1List of changes
List of changes, some of which are breaking
Networkobject returned or edited by the/networkroute is modified in the following way:shardingboolean is removedleaderfield is added as an optional string. When it is notnull, it has the same effect (and more) than havingsharding: truein the previous iteration of theNetworkobject. The leader is used as a check when receiving task creation requests.versionfield is added as a UUID, defaulting to the null UUID. The version is used when processing tasks.network.leaderis set, calling one of the following routes will fail withnot_a_leadererror if the target'snetwork.selfis not the same as itsnetwork.leader:POST /indexesPATCH/DELETE /indexes/{:indexUid}POST/PUT/DELETE /indexes/{:indexUid}/documentsPOST /indexes/{:indexUid}/documents/deletePOST /indexes/{:indexUid}/documents/delete-batchPOST /indexes/{:indexUid}/documents/editPATCH/DELETE /indexes/{:indexUid}/settingsand settings sub-routesPATCH /networkif the target is the new leaderPOST /swap-indexesleaderis set,PATCH /networkno longer returns aNetworkobject. Rather, it spawns a newNetworkTopologyChangetask, and returns the summarized task view.POST /indexes(new to this PR)PATCH/DELETE /indexes/{:indexUid}(new to this PR)POST/PUT/DELETE /indexes/{:indexUid}/documents(was already the case before this PR)POST /indexes/{:indexUid}/documents/delete(was already the case before this PR)POST /indexes/{:indexUid}/documents/delete-batch(was already the case before this PR)POST /indexes/{:indexUid}/documents/edit(was already the case before this PR)PATCH/DELETE /indexes/{:indexUid}/settingsand settings sub-routes (new to this PR)PATCH /networkif the target is the new leader (new to this PR)POST /swap-indexes(new to this PR)NetworkTopologyChangetasks that perform the following:network.versionlower than the network task's versionnetworkroute with specific headers containing the expected metadataselfandleaderfrom the networkby @dureuill in #6000
🛠️ Maintenance
Asset availability note
🌈 The Meilisearch binary is available again for
meilisearch-enterprise-macos-amd64andmeilisearch-macos-amd64. It was not available for Meilisearch v1.29.Full Changelog: https://github.com/meilisearch/meilisearch/compare/v1.29.0...v1.30.0
Configuration
📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).
🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.
♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
🔕 Ignore: Close this PR and you won't be reminded about this update again.
This PR has been generated by Renovate Bot.
1b06c419cftod4d4e33e97Update getmeili/meilisearch Docker tag to v1.40.0to Update getmeili/meilisearch Docker tag to v1.41.0View command line instructions
Checkout
From your project repository, check out a new branch and test the changes.Merge
Merge the changes and update on Forgejo.Warning: The "Autodetect manual merge" setting is not enabled for this repository, you will have to mark this pull request as manually merged afterwards.