0 purchases
pydantickedro 0.8.0
pydantic-kedro
Advanced serialization for Pydantic models
via Kedro and
fsspec.
This package implements custom Kedro "datasets" for both "pure" and "arbitrary"
Pydantic models. You can also use it stand-alone, using Kedro just for
serializing other object types.
Please see the documentation for a tutorial
and more examples.
Usage with Kedro
You can use the [PydanticAutoDataset][pydantic_kedro.PydanticAutoDataset]
or any other dataset from pydantic-kedro within your
Kedro catalog
to save your Pydantic models:
# conf/base/catalog.yml
my_pydantic_model:
type: pydantic_kedro.PydanticAutoDataset
filepath: folder/my_model
Direct Dataset Usage
This example works for "pure", JSON-safe Pydantic models via
PydanticJsonDataset:
from pydantic import BaseModel
# from pydantic.v1 import BaseModel # Pydantic V2
from pydantic_kedro import PydanticJsonDataset
class MyPureModel(BaseModel):
"""Your custom Pydantic model with JSON-safe fields."""
x: int
y: str
obj = MyPureModel(x=1, y="why?")
# Create an in-memory (temporary) file via `fsspec` and save it
ds = PydanticJsonDataset("memory://temporary-file.json")
ds.save(obj)
# We can re-load it from the same file
read_obj = ds.load()
assert read_obj.x == 1
Standalone Usage
You can also use pydantic-kedro as a generic saving and loading engine for
Pydantic models:
from tempfile import TemporaryDirectory
from pydantic.v1 import BaseModel
from pydantic_kedro import load_model, save_model
class MyModel(BaseModel):
"""My custom model."""
name: str
# We can use any fsspec URL, so we'll make a temporary folder
with TemporaryDirectory() as tmpdir:
save_model(MyModel(name="foo"), f"{tmpdir}/my_model")
obj = load_model(f"{tmpdir}/my_model")
assert obj.name == "foo"
For personal and professional use. You cannot resell or redistribute these repositories in their original state.
There are no reviews.