lite_embeddings_dart_server

Description:

lite embeddings dart server

Lite Embeddings Dart Server #
English · 中文
LLM Embedding tool HTTP service
Feature #

Support Vector Database: Chroma
Support file type: pure text, include Markdown, TXT
HTTP API Wrapper of Dart List Embeddings
Base on Lite Embeddings Dart的EmbeddingsService(DTO included), add Controller、Router, wrapper to HTTP/WS API.

Usage #
1. Prepare #

Docs file, according to /example/docs/*.md
Separator in the file

If markdown file, recommend to use <!--SEPARATOR--> as separator, for NOT show it in markdown after rendering


Add .env file in the example folder, and add below content in the .env file:
baseUrl = https://xxx.xxx.com # LLM API BaseURL
apiKey = sk-xxxxxxxxxxxxxxxxxxxx # LLM API ApiKey
copied to clipboard


2. Develop run server #

debug or run mode run /bin/server.dart file main()

3. HTTP API #

HTTP API

3.1 HTTP API

Docs API, include:

/version: get version number, to confirm server running
/docs/create-by-text: Create docs embeddings, post whole text and separator, service will split and write to vector database
/docs/create: Create docs embeddings, post the split docs, service will write to vector database
/docs/delete: Delete docs, post docsId
/docs/list: List all docs, return docsId and docsName Array
/docs/rename: Rename docsName
/docs/query: Text query, return N segment array with distance sort
/docs/batch-query: Text array query, query multi text at once, return N segment array in array
/docs/multi-query: Docs array query, query multi docs with one text, return N segment with docsId array
/segment/list: List all segments in the docs
/segment/insert: Insert segment by index. If not index, new segment will be inserted at last
/segment/update: Update segment
/segment/delete: Delete segment



BaseURL

http://127.0.0.1:9537/api

[GET] /version


Feature: get version number, to confirm server running


Request params: null


Response body sample:
{
"version": "0.1.0"
}
copied to clipboard


[POST] /docs/create-by-text

Feature: Create docs embeddings, post whole text and separator, service will split and write to vector database
Request params:

Docs info: docsName, text, separator, metadata, LLM Config
About metadata: Optional, same metadata for each segment. Default metadata include vdb and embeddings_model.
Sample

{
"docsName": "<Docs name, e.g. Moore's Law for Everything.md>",
"text": "<Docs full text, with separetor>",
"separator": "<Separator text>",
"metadata": "<Optional, in each segment, json map, value only int, float, string, bool, NOT support object and array. Each segment with same metadata>",
"llmConfig": {
"baseUrl": "<LLM API baseUrl, e.g. https://api.openai.com/v1>",
"apiKey": "<LLM API apiKey, e.g. sk-xxxxxxxxxx>",
"model": "<LLM API embeddings model name, e.g. text-embedding-ada-002>"
}
}
copied to clipboard

Response body:

Create successful docs info: docsId, docsName, Token Usage
Response body sample

{
"docsId": "<Docs Id>",
"docsName": "<Docs Name>",
"tokenUsage": {
"promptToken": "",
"totalToken": ""
}
}
copied to clipboard


[POST] /docs/create

Feature: Create docs embeddings, post the split docs, service will write to vector database
Request params:

Docs info: docsName, segment and metadata array, LLM Config
About metadata: Optional, default metadata include vdb and embeddings_model.
Sample

{
"docsName": "<Docs name, e.g. Moore's Law for Everything.md>",
"segmentList": [
{
"text": "<Segment text>",
"metadata": "<Optional, json map, value only int, float, string, bool, NOT support object and array>"
}
],
"llmConfig": {
"baseUrl": "<LLM API baseUrl, e.g. https://api.openai.com/v1>",
"apiKey": "<LLM API apiKey, e.g. sk-xxxxxxxxxx>",
"model": "<LLM API embeddings model name, e.g. text-embedding-ada-002>"
}
}
copied to clipboard

Response body:

Create successful docs info: docsId, docsName, Token Usage
Response body sample

{
"docsId": "<Docs Id>",
"docsName": "<Docs Name>",
"tokenUsage": {
"promptToken": "",
"totalToken": ""
}
}
copied to clipboard


[POST] /docs/delete

Feature: Delete docs, post docsId
Request params:

DocsId
Sample

{
"docsId": "xxxxxxxx"
}
copied to clipboard

Response body:

Docs be deleted info: docsId
Response body sample

{
"docsId": "xxxxxxxx"
}
copied to clipboard


[GET] /docs/list

Feature: List all docs, return docsId and docsName Array
Request params: null
Response body:

Docs info list: docsId and docsName
Response body sample

[
{
"docsId": "<Docs Id>",
"docsName": "<Docs Name>"
}
]
copied to clipboard


[POST] /docs/rename

Feature: Rename docsName
Request params:

docsId, new docsName
Sample

{
"docsId": "<Docs Id>",
"docsName": "<Docs Name>"
}
copied to clipboard

Response body:

Docs be modified info: docsId and docsName
Response body sample

{
"docsId": "<Docs Id>",
"docsName": "<Docs Name>"
}
copied to clipboard


[POST] /docs/query

Feature: Text query, return N segment array with distance sort
Request params:

docsId, queryText, return query result count, LLM Config
Sample

{
"docsId": "xxxxxxxx",
"queryText": "<query text string>",
"nResults": "<UInt, return query result by sort number>",
"llmConfig": {
"baseUrl": "<LLM API baseUrl, e.g. https://api.openai.com/v1>",
"apiKey": "<LLM API apiKey, e.g. sk-xxxxxxxxxx>",
"model": "<LLM API embeddings model name, e.g. text-embedding-ada-002>"
}
}
copied to clipboard

Response body:

docsId, segmentResultList, Token Usage
Response body sample

{
"docsId": "xxxxxxxx",
"segmentResultList": [
{
"segmentId": "<Segment Id>",
"text": "<Segment text>",
"metadata": "<json map, segment with>",
"distance": "<0.x float, segment match distance, smaller means closer>"
}
],
"tokenUsage": {
"promptToken": "",
"totalToken": ""
}
}
copied to clipboard


[POST] /docs/batch-query


Feature: Text array query, query multi text at once, return N segment array in array


Request params:

docsId, queryText array, return query result count, LLM Config
Sample

{
"docsId": "xxxxxxxx",
"queryTextList": [
"<Query Text 1>",
"<Query Text 2>"
],
"nResults": "<UInt, return query result by sort number>",
"llmConfig": {
"baseUrl": "<LLM API baseUrl, e.g. https://api.openai.com/v1>",
"apiKey": "<LLM API apiKey, e.g. sk-xxxxxxxxxx>",
"model": "<LLM API embeddings model name, e.g. text-embedding-ada-002>"
}
}
copied to clipboard


Response body:

Query result: docsId, segmentResultList array, Token Usage
Response body sample

[
{
"docsId": "xxxxxxxx",
"segmentResultList": [
{
"segmentId": "<Segment Id>",
"text": "<Segment text>",
"metadata": "<json map, segment with>",
"distance": "<0.x float, segment match distance, smaller means closer>"
}
],
"tokenUsage": {
"promptToken": "",
"totalToken": ""
}
}
]
copied to clipboard


/docs/multi-query:


[POST] /docs/multi-query

Feature: Docs array query, query multi docs with one text, return N segment with docsId array
Request params:

docsId array, queryText, return query result count
Sample

{
"docsIdList": ["xxxxxxxx", "yyyyyyyy"],
"queryText": "<Query Text 1>",
"nResults": "<UInt, return query result by sort number>",
"removeDuplicates": "<(Optional)boolean, default:true, return segments will be removed if same text>",
"llmConfig": {
"baseUrl": "<LLM API baseUrl, e.g. https://api.openai.com/v1>",
"apiKey": "<LLM API apiKey, e.g. sk-xxxxxxxxxx>",
"model": "<LLM API embeddings model name, e.g. text-embedding-ada-002>"
}
}
copied to clipboard

Response body:

Query result: segment info array, include docsId, segmentId, segment text, metadata, distance
Response body sample

{
"segmentResultList": [
{
"docsId": "xxxxxxxx",
"segmentId": "<Segment Id>",
"text": "<Segment text>",
"metadata": "<json map, segment with>",
"distance": "<0.x float, segment match distance, smaller means closer>"
}
],
"tokenUsage": {
"promptToken": "",
"totalToken": ""
}
}
copied to clipboard


[POST] /segment/list

Feature: List all segments in the docs
Request params:

docsId
Sample

{
"docsId": "xxxxxxxx"
}
copied to clipboard

Response body:

docsId, docsName, segment info list
Response body sample

{
"docsId": "xxxxxxxx",
"docsName": "<Docs Name>",
"segmentInfoList": [
{
"SegmentId": "<Segment Id>",
"text": "<Segment text>",
"metadata": "<json map, segment with>"
}
]
}
copied to clipboard


[POST] /segment/insert

Feature: Insert segment by index. If not index, new segment will be inserted at last
Request params:

docsId, new segment, index, LLM Config
About metadata: Optional, default metadata include vdb and embeddings_model.
Sample

{
"docsId": "xxxxxxxx",
"segment": {
"text": "<Segment text>",
"metadata": "<Optional, json map, segment with>"
},
"index": "(Optional) UInt, if null or large than length, be inserted at last",
"llmConfig": {
"baseUrl": "<LLM API baseUrl, e.g. https://api.openai.com/v1>",
"apiKey": "<LLM API apiKey, e.g. sk-xxxxxxxxxx>",
"model": "<LLM API embeddings model name, e.g. text-embedding-ada-002>"
}
}
copied to clipboard

Response body:

new Segment ID, Token Usage
Response body sample

{
"segmentId": "xxxxxxxx",
"tokenUsage": {
"promptToken": "",
"totalToken": ""
}
}
copied to clipboard


[POST] /segment/update

Feature: Update segment
Request params:

docsId, segment, LLM Config
About metadata, optional:

null: NOT update current metadata
{}: clear metadata, but remain default metadata include vdb and embeddings_model
values: add or update current metadata


Sample

{
"docsId": "xxxxxxxx",
"segment": {
"segmentId": "Segment Id",
"text": "<Segment text>",
"metadata": "<Optional, json map, segment with>"
},
"llmConfig": {
"baseUrl": "<LLM API baseUrl, e.g. https://api.openai.com/v1>",
"apiKey": "<LLM API apiKey, e.g. sk-xxxxxxxxxx>",
"model": "<LLM API embeddings model name, e.g. text-embedding-ada-002>"
}
}
copied to clipboard

Response body:

Segment Id, Token Usage
Response body sample

{
"segmentId": "xxxxxxxx",
"tokenUsage": {
"promptToken": "",
"totalToken": ""
}
}
copied to clipboard


[POST] /segment/delete

Feature: Delete segment
Request params:

Segment be deleted docsId and segmentId
Sample

{
"docsId": "xxxxxxxx",
"segmentId": "xxxxxxxx"
}
copied to clipboard

Response body:

Segment Id
Response body sample

{
"segmentId": "xxxxxxxx"
}
copied to clipboard


Build and Run #

Build in shell script:
dart compile exe bin/server.dart -o build/lite_embeddings_dart_server
copied to clipboard

Then the lite_embeddings_dart_server file will be in build folder
Copy config.json file to lite_embeddings_dart_server same folder
Run in shell script:
./lite_embeddings_dart_server
copied to clipboard

Terminal will show:
INFO: 2024-06-24 14:48:05.862057: PID 34567: [HTTP] Start Server - http://0.0.0.0:9537/api
copied to clipboard

After server running, will create log folder and embeddings.log file in the folder, to record server running logs.