mimir

Description:

mimir

A batteries-included NoSQL database for Dart & Flutter based on an embedded
Meilisearch instance.

Features #

🔎 Typo tolerant and relevant full-text search with no extra configuration needed
🔥 Blazingly fast search and reads (written in Rust)
🤝 Flutter friendly with a super easy-to-use API (see demo below!)
🔱 Powerful, declarative, and reactive queries
🔌 Cross-platform support (web hopefully coming soon!)
🉑️ Diverse language support, including CJK, Hebrew, and more!

Getting Started #

With Flutter, run flutter pub add mimir flutter_mimir
For Dart-only, run dart pub add mimir

For macOS, disable "App Sandbox".
Also read the caveats below.
Demo #
With Flutter, you can get started with as little as:
// Get an "index" to store our movies
final instance = await Mimir.defaultInstance;
final index = instance.getIndex('movies');

// Add movies to our index
await index.addDocuments(myMovies);

// Perform a search!
final results = await index.search(query: 'jarrassic par'); // returns Jurassic Park!
copied to clipboard

Sponsors #
You can become a sponsor of my work here!



Reference Documentation #
A collection of commonly used APIs ready for copy-paste into your application.
Note: unless otherwise stated, all asynchronous methods exposed in Mimir are fallible
and synchronous methods are infallible.
The methods are fail-fast, so you should be aware of any issues early on during development.
Getting & Creating an Index
// With Flutter (flutter_mimir)
final instance = await Mimir.defaultInstance;

// Dart-only (just mimir)
final instance = await Mimir.getInstance(
path: instanceDirectory,
// Following line will change based on your platform
library: DynamicLibrary.open('libembedded_milli.so'),
);

// Get an index (creates one lazily if not already created)
final index = instance.getIndex('movies');

// Or, specify some default settings and open the index eagerly
// If you have some settings you want to specify in advance, use openIndex!
final index = await instance.openIndex('movies', primaryKey: 'CustomIdField');
copied to clipboard
Configuring an Index
await index.updateSettings(...); // see setSettings below for arguments
final currSettings = await index.getSettings();
await index.setSettings(currSettings.copyWith(
// The primary key (PK) is the "ID field" of documents added to mimir.
// When null, it is automatically inferred for you, but sometimes you may
// need to specify it manually. See the Important Caveats section for more.
primaryKey: null,
// Fields in documents that are included in full-text search.
// Use null, the default, to search all fields
searchableFields: <String>[],
// Fields in documents that can be queried/filtered by.
// You probably don't need to change this; it is automatically
// updated for you.
filterableFields: <String>[],
// Fields in documents that can be sorted by in searches/queries.
// You probably don't need to change this; it is automatically
// updated for you.
sortableFields: <String>[],
// The ranking rules of this index, see:
// https://docs.meilisearch.com/reference/api/settings.html#ranking-rules
rankingRules: <String>[],
// The stop words of this index, see:
// https://docs.meilisearch.com/reference/api/settings.html#stop-words
stopWords: <String>[],
// A list of synonyms to link words with the same meaning together.
// Note: in most cases, you probably want to add synonyms both ways, like below:
synonyms: <Synonyms>[
Synonyms(
word: 'automobile',
synonyms: ['vehicle'],
),
Synonyms(
word: 'vehicle',
synonyms: ['automobile'],
),
],
// Whether to enable typo tolerance in searches.
typosEnabled: true,
// The minimum size of a word that can have 1 typo.
// See minWordSizeForTypos.oneTypo here:
// https://docs.meilisearch.com/reference/api/settings.html#typo-tolerance-object
minWordSizeForOneTypo: 5,
// The minimum size of a word that can have 2 typos.
// See minWordSizeForTypos.twoTypos here:
// https://docs.meilisearch.com/reference/api/settings.html#typo-tolerance-object
minWordSizeForTwoTypos: 9,
// Words that disallow typos. See disableOnWords here:
// https://docs.meilisearch.com/reference/api/settings.html#typo-tolerance-object
disallowTyposOnWords: <String>[],
// Fields that disallow typos. See disableOnAttributes here:
// https://docs.meilisearch.com/reference/api/settings.html#typo-tolerance-object
disallowTyposOnFields: <String>[],
));
copied to clipboard
Manipulating Documents
// Adding documents (replaces any documents with the same id)
await index.addDocument(document);
await index.addDocuments(documents);

// Replacing all documents
await index.setDocuments(documents);

// Deleting documents
await index.deleteDocument(id);
await index.deleteDocuments(ids);
await index.deleteAllDocuments();

// Getting documents (not querying--see next section!)
final docOrNull = await index.getDocument(someId);
final allDocs = await index.getAllDocuments();
final allDocsStream = index.documents;
copied to clipboard
Searching/Querying
// Getting a stream of results (very useful in Flutter!)
// Same arguments as index.search; see below
final documentsStream = index.searchStream(...);

// Performing a search/query (using movies here)!
final movies = index.search(
// The string to use for full-text search. Can be user-supplied.
// To do a regular database query without full-text search, leave this null.
query: 'some wordz with typoes to saerch for',
// The filter used to filter results in a full-text search or query.
// See the next section; this is a very handy feature in mimir.
// Set to null to not filter out any documents.
filter: Mimir.where('director', isEqualTo: 'Alfred Hitchcock'),
// The fields to sort by (in ascending or descending order).
// Can be left as null to sort by relevance (to the query text)!
sortBy: [
// Sort by year, newest to oldest
SortBy.desc('year'),
// In case 2+ documents share the same year, sort by increasing profit next
SortBy.asc('profit'),
],
// If you want to limit the number of results you get, use the resultsLimit.
// Defaults to null, which means return all matches.
resultsLimit: null,
// Defaults to null, see https://docs.meilisearch.com/reference/api/search.html#matching-strategy
matchingStrategy: null,
);
copied to clipboard
Filtering Search/Query Results
There is a "raw" filtering API in mimir provided by Filter,
but it is recommended to use the following API that creates Filters for you instead.
Here are the methods you need to be aware of:

Mimir.or(subFilters) creates an "or" filter (like ||) of the sub-filters
Mimir.and(subFilters) creates an "and" filter (like &&) of the sub-filters
Mimir.not(subFilter) creates a "not" filter (like !someCondition) of the sub-filter
Mimir.where(condition) creates a single filter from a given condition.
The above can be composed together to create powerful, declarative queries!

Heres an example that shows these methods in practice.
Say our Dart boolean logic is (formatted to show intent):
(
(
(movie['fruit'] == 'apple')
&&
(movie['year'] >= 2000 && movie['year'] <= 2009)
)
||
movie['colors'].any((color) => {'red', 'green'}.contains(color))
)
copied to clipboard
Then our "raw" filter API logic would be:
final filter = Filter.or([
Filter.and([
Filter.equal(field: 'fruit', value: 'apple'),
Filter.between(field: 'year', from: '2000', to: '2009'),
]),
Filter.inValues(field: 'colors', values: ['red', 'green']),
])
copied to clipboard
Which is somewhat hard to read.
Here's what the recommended approach would look like:
final filter = Mimir.or([
Mimir.and([
Mimir.where('fruit', isEqualTo: 'apple'),
Mimir.where('year', isBetween: ('2000', '2009')),
]),
Mimir.where('colors', containsAtLeastOneOf: ['red', 'green']),
])
copied to clipboard
I think most would agree that this is the easiest of the three to understand,
as it almost reads as pure English, even for complex conditions.
Important Caveats #
Please read these caveats before adding mimir to your project.

Every document in mimir needs to have a "primary key"

The PK is automatically inferred via a field ending in id (or simply just id)
If you have multiple fields ending in id, use instance.openIndex('indexName', primaryKey: 'theActualId')
The contents of the PK field can be a number, or a string matching the regex ^[a-zA-Z0-9-_]*$

In English: PKs can be alphanumeric and contain - and _




Unfortunately, you can only open 1 index on iOS devices at the moment; see here for more details and a workaround.
macOS App Sandbox is not supported at the moment, meaning you will not be able to submit apps to the Mac App Store

You will still be able to distribute macOS applications on your own
See more details here


Resource usage

While modern devices run mimir just fine, several thousand detailed documents can easily consume several MB of disk space and RAM
This is due to Milli, a heavy-weight core component of Meilisearch, which gives mimir a lot of its power
If you do not need all the features provided by mimir, also consider a more lightweight alternative!

Hive for simple key-value storage
Isar for more sophisticated use-cases

Note: while Isar does have full-text search, it is neither typo-tolerant nor relevant!


If you need easy, typo-tolerant, relevant full-text search, you will want mimir!

I am unaware of any other databases that currently provide typo-tolerant full-text search in Flutter, which is why I made mimir in the first place!




Mimir can add a couple hundred MB to your app bundle size and <100 MB to the on-device size

These numbers will hopefully be reduced in the future once Dart gets "Native Assets"