Wanting

Wanting is a library for creating, and working with models that can represent incomplete information.

Motivation

Instances of domain models don’t always spring into existence fully formed. They may be partially constructed intially, then filled in over time. Making a model field optional that is not intially available, but eventually required is inaccurate because an optional field may always be optional, so it never has to be filled in. It would be better to make the field a required union of the type it wants, and a placholder type. The wanting types are such placeholders. They can include metadata, such as the source of the update with missing data, and even partial data from that source.

Usage

There are two wanting types that may be unioned with the type of a field. When a field is Unavailable, no information about that field is known. When a field is Unmapped, there is information about that field, but we are unable to map that information to a value that the model will accept.

A domain model may look like this:

from typing import Literal

import pydantic
import wanting


class User(pydantic.BaseModel):
    """A model that can have incomplete information."""

    name: str
    employee_id: str | wanting.Unavailable
    department_code: Literal["TECH", "FO", "BO", "HR"] | wanting.Unmapped

Then there is an onboarding system that creates a User. However, the employee_id is unavailable at this time because it will be generated later. The onboarding system sources the department code from some other system, which uses different values than those in the User model. The onboarding system knows how to map some of the codes from the other system to the User department codes, but not all of them. However, because employee_id, and department_code are unioned with wanting fields, the onboarding system can still create a fully valid model, with the information it knows:

user = User(
    name="Charlotte",
    employee_id=wanting.Unavailable(source="onboarding"),
    department_code=wanting.Unmapped(source="onboarding", value="art"),
)

The model validates, and all the wanting fields serialize to valid JSON:

assert user.model_dump() == {
    "name": "Charlotte",
    "employee_id": {
        "kind": "unavailable",
        "source": "onboarding",
        "value": {"serialized": b"null"},
    },
    "department_code": {
        "kind": "unmapped",
        "source": "onboarding",
        "value": {"serialized": b'"art"'},
    },
}

This user can now be persisted, then queried, and updated later by other systems.

A model class can be queried for its potentially wanting fields:

class Child(pydantic.BaseModel):
    """A model that can have incomplete information."""

    regular: int
    wanting: int | wanting.Unavailable


class Parent(pydantic.BaseModel):
    """A model that can have top-level, and nested incomplete information."""

    regular: int
    wanting: int | wanting.Unavailable
    nested: Child


def reduce_path(path: list[wanting.FieldInfoEx]) -> str:
    """Reduce the FieldInfoEx objects that comprise a path to a readable string."""
    return "->".join(f"{fi.cls.__name__}.{fi.name}" for fi in path)


paths = wanting.fields(Parent)
summary = [reduce_path(path) for path in paths]
assert summary == ["Parent.wanting", "Parent.nested->Child.wanting"]

A model instance can be queried for its wanting values:

p = Parent(
    regular=1,
    wanting=2,
    nested=Child(regular=3, wanting=wanting.Unavailable(source="doc")),
)
assert wanting.values(p) == {
    "nested": {"wanting": wanting.Unavailable(source="doc")}
}

A model instance can also be serialized, either including or excluding its wanting values:

inc = wanting.incex(p)
assert p.model_dump(include=inc) == {
    "nested": {
        "wanting": {
            "kind": "unavailable",
            "source": "doc",
            "value": {"serialized": b"null"},
        }
    }
}
assert p.model_dump(exclude=inc) == {
    "regular": 1,
    "wanting": 2,
    "nested": {"regular": 3},
}

Model serialization with respect to wanting fields is invertible. A model can be serialized, then the result can be deserialized back into an equivalent model.

p2 = Parent.model_validate(p.model_dump())
assert p == p2

API

The Wanting module.

class wanting.wanting.Json(*, serialized: bytes)

Bases: BaseModel

Wrapper for a JSON value.

serialized: bytes

class wanting.wanting.Wanting(*, kind: str, source: str, value: Annotated[Json, BeforeValidator(func=_to_json, json_schema_input_type=PydanticUndefined)])

Bases: BaseModel

Abstract class that represents an incomplete field value.

When serializing a model that contains wanting fields with exclude_unset, optional fields in the wanting models that have not been explicitly set would normally be omitted. However, any model that subclasses Wanting will not have their kind, and value fields omitted due to exclude_unset. This behavior can be overridden when serializing by passing a context dict with a wanting key whose value is a dict that contains an enable_exclude_unset key whose value is True.

Examples

Serialization ignores exclude_unset.

class M(pydantic.BaseModel):
    a: str = "unset"
    b: str | Unavailable


m = M(b=Unavailable(source="docs"))

assert m.model_dump(exclude_unset=True) == {
    # a is excluded by exclude_unset
    "b": {  # kind and value are not excluded by exclude_unset
        "kind": "unavailable",
        "source": "docs",
        "value": {"serialized": b"null"},
    }
}

However this behavior can be overridden by passing a context.

assert m.model_dump(
    exclude_unset=True, context={"wanting": {"enable_exclude_unset": True}}
) == {
    # a is excluded by exclude_unset
    "b": {  # kind and value are also excluded by exclude_unset
        "source": "docs"
    }
}

kind: str

source: str

value: Annotated[Json, BeforeValidator(func=_to_json, json_schema_input_type=PydanticUndefined)]

serializer(handler: SerializerFunctionWrapHandler, info: SerializationInfo) → dict[str, Any]

Ignore exclude_unset for certain fields.

class wanting.wanting.Unavailable(*, kind: Literal['unavailable'] = 'unavailable', source: str, value: Json = <factory>)

Bases: Wanting

Represents an unavailable field value.

kind: Literal['unavailable']

value: Json

class wanting.wanting.Unmapped(*, kind: Literal['unmapped'] = 'unmapped', source: str, value: Annotated[Json, BeforeValidator(func=_to_json, json_schema_input_type=PydanticUndefined)])

Bases: Wanting

Represents an unmapped field value.

kind: Literal['unmapped']

class wanting.wanting.FieldInfoEx(cls: type[BaseModel], name: str, info: FieldInfo)

Bases: NamedTuple

Extended information about a field.

wanting.wanting.fields(cls: type[BaseModel], *targets: type[Wanting], depth: int = -1) → Iterator[list[FieldInfoEx]]

Get the fields in a model class that could be target Wanting types.

Parameters:

• cls – The model class to inspect for wanting fields.

• targets – The Wanting types of fields to get. Defaults to all Wanting types.

• depth – How deeply to check nested models for wanting fields. The depth is zero-based, so 0 for only top-level fields. -1 for no limit.

Returns:

An iterator of FieldInfoEx lists, where each list describes the path to a top-level, or nested wanting field.

type wanting.wanting.WantingValues = Mapping[str, Wanting | WantingValues]

wanting.wanting.values(model: BaseModel | RootModel[TypeVar], *targets: type[Wanting], collapse_root: bool = True) → WantingValues

Get the values in a model instance that are target Wanting types.

Parameters:

• model – The model instance to inspect for wanting values.

• targets – The Wanting types of values to get. Defaults to all Wanting types.

• collapse_root – If True, the wanting values under the root field of RootModels will be moved up one level in the result, as if they were top-level fields in the model.

Returns:

A dict that mirrors the structure of the model, but only contains the fields that have wanting values.

Examples

Get all the wanting values from a model instance.

class Child(pydantic.BaseModel):
    a: str | Unmapped


class Parent(pydantic.BaseModel):
    a: str | Unavailable
    b: str | Unmapped
    c: Child


m = Parent(
    a=Unavailable(source="docs"),
    b="foo",
    c=Child(a=Unmapped(source="docs", value="bar")),
)

assert values(m) == {
    "a": Unavailable(source="docs"),
    # b is not included because it doesn't have a Wanting value
    "c": {"a": Unmapped(source="docs", value="bar")},
}

Get only the unmapped values.

assert values(m, Unmapped) == {
    # a is not included because it doesn't have an Unmapped value
    # b is not included because it doesn't have a Wanting value
    "c": {"a": Unmapped(source="docs", value="bar")}
}

type wanting.wanting.IncExMapping = Mapping[str, bool | IncExMapping]

A mapping that describes which fields to include, or exclude during serialization.

Values of this type may be used as the include or exclude parameter to pydantic.BaseModel.model_dump().

wanting.wanting.incex(model: BaseModel, *targets: type[Wanting]) → IncExMapping

Get a mapping to include or exclude Wanting fields in a model instance.

Parameters:

• model – The model instance to inspect for wanting values.

• targets – The Wanting types to include, or exclude. Defaults to all Wanting types.

Returns:

An IncExMapping that can be used as the include or exclude argument to Pydantic serialization methods.

Examples

Dump all Wanting values in a model instance.

class Child(pydantic.BaseModel):
    a: str | Unmapped


class Parent(pydantic.BaseModel):
    a: str | Unavailable
    b: str | Unmapped
    c: Child


m = Parent(
    a=Unavailable(source="docs"),
    b="foo",
    c=Child(a=Unmapped(source="docs", value="bar")),
)

inc = incex(m)
assert inc == {"a": True, "c": {"a": True}}
assert m.model_dump(include=inc) == {
    "a": {
        "kind": "unavailable",
        "source": "docs",
        "value": {"serialized": b"null"},
    },
    # b is not included because it doesn't have a Wanting value
    "c": {
        "a": {
            "kind": "unmapped",
            "source": "docs",
            "value": {"serialized": b'"bar"'},
        }
    },
}

Dump only the Unmapped values in a model instance.

inc = incex(m, Unmapped)
assert inc == {"c": {"a": True}}
assert m.model_dump(include=inc) == {
    # a is not included because it doesn't have an Unmapped value
    # b is not included because it doesn't have a Wanting value
    "c": {
        "a": {
            "kind": "unmapped",
            "source": "docs",
            "value": {"serialized": b'"bar"'},
        }
    }
}