embed

Description:

embed

embed.dart


Code generation for embedding arbitrary file content into Dart code


Explore the docs »


Pub.dev
·
Report Bug
·
Request Feature




Motivation #
Occasionally there are situations where we want to read non-Dart files for some reason, such as reading configuration values, or reading a test HTTP response for unit testing. A common way to do this is to load the file at runtime using File. However, since such files are usually bundled in the package, it would be nice to be able to read their contents directly from within the dart code without worrying about runtime errors and async I/O processing.
There are several ways to embed structured data into dart code. We can use multi-line string literal to embed a long text content, or Map literal to embed a structured data, or further, we can use the records to create a static structured data tree in a type-safe manner. However, there are still situations where reading a non-Dart file is required, because the file is downloaded from the Internet, or automatically generated by a script, or shared with another package written in a programming language other than Dart, etc. This is where embed comes in. The package solves this problem by generating code that allows to embed the contents of non-Dart files directly into the source file as literals.
Some of the other languages have a similar feature to this package, such as include_str macro from Rust, embed package from Go, and Javascript/Typescript's ability to directly import static JSON files as typed objects. Also, the C language has a similar feature: the #include delective. What the #include "header.h" actually means is that it tells the C preprocessor that "please replace me with the entire content of header.h", and interestingly, the #include delective can literally include any file other than *.h files as text. In fact, the following code works fine (might not be an intended use, but actually works fine):
// text.txt
"Hello world\n"

// main.c
#include <stdio.h>
int main(void) {
const message =
#include "text.txt"
; // ^^^^^^^^^^^^^ This line will be replaced with "Hello world\n"
printf(message); // Displays "Hello world"
}
copied to clipboard

Index #

Motivation
Index
Installation
Quickstart
Examples
How to use

Embed a text content as a string literal
Embed contents as binary
Embed a structured data as a Dart object

Preprocessing
How is the data type determined?
How to restrict the structure of data to be embedded?




Troubleshooting Guide

I edited my json file to embed, but the generated code doesn't update even when I run build_runner again


Roadmap
Contributing
Support
Thanks
Links


Installation #
Run the following command:
flutter pub add embed_annotation dev:embed dev:build_runner
copied to clipboard
For a Dart project:
dart pub add embed_annotation dev:embed dev:build_runner
copied to clipboard
This command installs three packages:

embed : the code generator
embed_annotation : a package exposing annotations for embed
build_runner : a tool to run code generators, published by the Dart team


Quickstart #
Here's an example of embedding the content of the pubspec.yaml in the Dart code as an object:
// This file is 'main.dart'

// Import annotations
import 'package:embed_annotation/embed_annotation.dart';

// Like other code generation packages, you need to add this line
part 'main.g.dart';

// Annotate a top-level variable specifing the location of a content file to embed
@EmbedLiteral("../pubspec.yaml")
const pubspec = _$pubspec;
copied to clipboard
Then, run the code generator:
dart run build_runner build
copied to clipboard
If your are working in a Flutter project, you can also run the generator by:
flutter pub build_runner build
copied to clipboard
Finally, you should see the main.g.dart is generated in the same directory as main.dart.
// This is 'main.g.dart'

// GENERATED CODE - DO NOT MODIFY BY HAND

part of 'main.dart';

// **************************************************************************
// EmbedGenerator
// **************************************************************************

const _$pubspec = (
name: "example_app",
publishTo: "none",
environment: (sdk: "^3.0.5"),
dependencies: (embedAnnotation: (path: "../embed_annotation")),
devDependencies: (
buildRunner: "^2.4.6",
lints: "^2.0.0",
embed: (path: "../embed")
),
dependencyOverrides: (embedAnnotation: (path: "../embed_annotation"))
);
copied to clipboard
You can see the content of the pubspec.yaml is embedded as a record object in the generated file. Let's print your package name to the console using this embedding:
print(pubspec.name); // This should display "example_app"
copied to clipboard
After modifying the original file, the pubspec.yaml in this case, you need to run the code generator again to update the embedded content. It is recommended to clear the cache before running the build_runner to avoid a code generation problem (see the troubleshooting guide for more information), as follows:
flutter pub run build_runner clean
copied to clipboard
Examples #
You can find many more examples in the following resources:

embed/test/literal/literal_embedding_generator_test_src.dart
embed/test/str/str_embedding_generator_test_src.dart
embed/test/binary/binary_embedding_generator_test_src.dart
example/lib/example.dart


How to use #
Currently, there are 3 types of embedding methods:

Embed a text content as a String literal
Embed contents as binary
Embed a structured data as a Dart object

What content to embed and how to embed it can be described using predefined annotations. For example, you can use the EmbedStr annotation to embed text content as a string literal. Note that only top-level variables can be annotated, as shown below:
@EmbedStr(...) // This is OK
const topLevelVariable = ...;

class SomeClass {
@EmbedStr(...) // This is invalid!
static const classVariable = ...;
}
copied to clipboard
Each annotation needs at least one parameter, the location of the content file. There are 2 ways to specify the file location, a relative path and an absolute path. If you specify a relative path, it will be treated as relative to the parent directory of the source file where the annotated variable is defined. For example, suppose we have a simple Flutter project, typically structured as:
project_root
|- lib
| |- main.dart
|- pubspec.yaml
copied to clipboard
In this scenario, we can refer the pubspec.yaml from the lib/main.dart using a relative path like ../pubspec.yaml :
@EmbedStr("../pubspec.yaml")
const pubspec = _$pubspec;
copied to clipboard
Depending on the structure of your project, it may be more intuitive to use an absolute path rather than a relative path:
@EmbedStr("/pubspec.yaml")
const pubspec = _$pubspec;
copied to clipboard
If you specify the content file path as an absolute path, as in the snippet above, it is treated as relative to the project root directory. In this example, the absolute path /pubspec.yam is interpreted by the code generator as /path/to/project/root/pubspec.yaml. Both methods can be used with all annotations, so choose one that suits your project structure.

Embed a text content as a string literal #
Use EmbedStr to embed an arbitary file content in a source file as a string literal, as-is.
// main.dart
@EmbedStr("useful_text.txt")
const usefulText = _$usefulText;
copied to clipboard
By default, it embeds the text content as a raw string:
// main.g.dart
const _$usefulText = r'''
This is a useful text for you.
''';
copied to clipboard
If this doesn't work well with your text content, you can disable this behavior by setting EmbedStr.raw to false:
@EmbedStr("useful_text.txt", raw: false)
copied to clipboard
This will generates a regular string literal:
const _$usefulText = '''
This is a useful text for you.
''';
copied to clipboard

Embed contents as binary #
Use EmbedBinary to embed a content file as a binary data.
@EmbedBinary("/assets/avator.png")
const avator = _$avator;
copied to clipboard
By default, the content is embedded as a List<int> literal.
const _$avator = [137, 88, 234, 85, ..., 13];
copied to clipboard
If you want to embed the content as a Base64 string literal, set EmbedBinary.bse64 to true.
@EmbedBinary("avator.png", base64: true)
copied to clipboard
The code generator will then have the following output:
const _$avator = 'iVBORw0KGgoAA...Sp8AAAAASUVORK5CYII=';
copied to clipboard

Embed a structured data as a Dart object #
Use EmbedLiteral to convert a structured data file such as JSON to a dart object and embed it in a source file. This is useful when you want to read a non-Dart file bundled into your package in a type-safe way, without worrying about runtime errors and asynchronous I/O operations. Currently EmbedLiteral supports JSON, TOML and YAML files.
// main.dart
@EmbedLiteral("config.json")
const config = _$config;
copied to clipboard
If the config.json is like:
// This is just an example, don't care about the meaning of the content :)
{
"url": "https://api.example.com",
"api_key": "AJFKEl04i9jlsLJFXS9w09",
"default": 2,
}
copied to clipboard
Then, the code generator will dump the following code:
// main.g.dart
const _$config = (
url: "https://api.example.com",
apiKey: "AJFKEl04i9jlsLJFXS9w09",
$default: 2,
);
copied to clipboard
You can see that the given JSON data is converted as a record object. And if you take a closer look at the output, you may notice that some JSON keys are converted to camelCase. This is because it is the recommended style for record type field names.
One more thing, when a reserved keyword like if is used as a JSON key, the code generator automatically adds a $ sign at the beginning of the key; for example, in the above example, a JSON key default is converted to $default in the dart code.
Preprocessing
In the previous example, all JSON keys are converted to camelCase, and if any reserved Dart keywords are used as JSON keys, they are prefixed with a $ sign to avoid syntax errors. This processing is done by Preprocessors. You can specify preprocessors to be applied to the content in the constructor of EmbedLiteral.
@EmbedLiteral(
"config.json",
preprocessors = [
Preprocessor.recase, // e.g. converts 'snake_case' to 'snakeCase'
Preprocessor.escapeReservedKeywords, // e.g. converts 'if' to '$if'
Preprocessor.replace("#", "0x"), // e.g. converts "#fff" to "0xfff"
],
)
const config = _$config;
copied to clipboard
These preprocessors are applied recursively to all elements in the content, in the order specified. By default, Recase and EscapeReservedKeywords are applied, but you can disable this behavior by explicitly specifying an empty list to the preprocessors parameter:
@EmbedLiteral("config.json", preprocessors = const [])
const config = _$config;
copied to clipboard
How is the data type determined?
The code generator tries to represent map-like data as records rather than Maps whenever possible. For example, the following JSON file is converted to a record because the all the keys have a valid format as record field names:
{
"snake_case": 0,
"camelCase": "text",
"PascalCase": true,
}
copied to clipboard
On the other hand, the next JSON will be converted as a Map<String, Object> because at least one of the keys has an invalid format as a record field name:
{
"snake_case": 0, // This is fine
"0_starts_with_number": "text", // BAD
"contians *invalid* characters!": true, // BAD
}
copied to clipboard
In this case, the output code will be a Map literal:
// main.g.dart
const _$config = {
"snake_case": 0,
"0_starts_with_number": "text",
"contians *invalid* characters!": true,
};
copied to clipboard
This rule is applied recursively if the input file contains nestd data structure, from root to leaf objects each time a map-like structure is converted to a literal representation.
How to restrict the structure of data to be embedded?
You can restrict the types of generated dart objects by specifying concrete types to annotated top level variables.
// Suppose you are only interested in the 'name' and 'publish_to' fields in the pubspec.yaml
typedef Pubspec = ({ String name, String publishTo });

@EmbedLiteral("/pubspec.yaml")
const Pubspec pubspec = _$pubspec; // Expects `_$pubspec` to be of type `Pubspec`

// Or if you prefer a Map to a Record
@EmbedLiteral("/pubspec.yaml")
const Map pubspecMap = _$pubspecMap;
copied to clipboard
Then, the build_runner will generates the following:
const _$pubspec = (name: "ExampleApp", publishTo: "none");
const _$pubspecMap = {"name": "ExampleApp", "publishTo": "none", "version": ... };
copied to clipboard

Troubleshooting Guide #
I edited my json file to embed, but the generated code doesn't update even when I run build_runner again #
It seems that the build_runner caches the previous output and if a source file has not changed from the previous one, it will not regenerate the code for that file. Since the source file does not change before and after modifinyg the json file, the updates are not reflected.
To avoid this problem, try removing the cache before running the build_runner as follows (replace flutter with dart if you are working in a Dart project):
flutter pub run build_runner clean && flutter pub run build_runner build
copied to clipboard
If you are still having the problem, also try this:
flutter clean && flutter pub run build_runner build
copied to clipboard

Roadmap #

✅ Restrict the type of dart object to embed by giving the corresponding variable a concrete type ➡️ Available from v1.1.0


Contributors #
Thanks to all the contributors!

@NicolaVerbeeck: Support embedding binary files (#13)
@bramp: Bumped the toml and lint dependencies to their latest(#17)


Contributing #
Contributions are what make the open source community such an amazing place to learn, inspire, and create. Any contributions you make are greatly appreciated.
If you have a suggestion that would make this better, please fork the repo and create a pull request. You can also simply open an issue with the tag "enhancement".
Don't forget to give the project a star! Thanks again!

Fork the Project
Create your Feature Branch (git checkout -b feature/AmazingFeature)
Commit your Changes (git commit -m 'Add some AmazingFeature')
Push to the Branch (git push origin feature/AmazingFeature)
Open a Pull Request


Support #
Please give me a star on GitHub if you like this package. It will motivate me!

Thanks #

Best-README-Template by @othneildrew


Links #

API Documentation
pub.dev (embed, embed_annotation)
GitHub repository