Advanced Pydantic Features: Custom Validators and Complex Data Structures

Pydantic v2 provides powerful tools for handling data validation and management in Python. By leveraging type hints and runtime validation, Pydantic helps developers build reliable, maintainable applications. This article explores key advanced features, such as custom validators, nested models, and handling complex data structures, focusing on the improvements introduced in Pydantic v2.

Installing Pydantic v2

To get started, install Pydantic v2 using the following command:

pip install --upgrade pydantic

Custom Validators with @field_validator

In Pydantic v2, custom field-level validation is done using the @field_validator decorator. This replaces the @validator decorator from v1. The new approach improves clarity and performance.

Example: Field Validation

from pydantic import BaseModel, field_validator

class Product(BaseModel):  
    name: str  
    price: float  
    stock: int

    # Validate the price field  
    @field_validator("price")  
    def validate_price(cls, value):  
        if value <= 0:  
            raise ValueError("Price must be greater than zero.")  
        return value

    # Validate the stock field  
    @field_validator("stock")  
    def validate_stock(cls, value):  
        if value < 0:  
            raise ValueError("Stock cannot be negative.")  
        return value

Using the Model

# Valid product  
product = Product(name="Laptop", price=999.99, stock=10)  
print(product)

# Invalid product  
try:  
    invalid_product = Product(name="Laptop", price=-100.0, stock=10)  
except ValueError as e:  
    print(e)

This ensures validation is applied only to specific fields and provides clear error messages for invalid data.

Nested Models

Nested models allow you to define hierarchical relationships between data entities. This functionality remains unchanged from Pydantic v1, but Pydantic v2 enhances the performance and clarity of nested validation.

Example: Nested Models

from pydantic import BaseModel

class Address(BaseModel):  
    street: str  
    city: str  
    zipcode: str

class User(BaseModel):  
    name: str  
    email: str  
    address: Address

Using the Nested Models

data = {  
    "name": "Alice",  
    "email": "[email protected]",  
    "address": {  
        "street": "123 Elm Street",  
        "city": "Wonderland",  
        "zipcode": "12345"  
    }  
}  

user = User(**data)  
print(user)

This validates both the User model and its nested Address model, ensuring data integrity across the entire hierarchy.

Model-Level Validation with @model_validator

Pydantic v2 introduces the @model_validator decorator, allowing validation of the entire model. This replaces the need for manually overriding the __post_init__ method in v1.

Example: Model Validation

from pydantic import BaseModel, model_validator

class Order(BaseModel):  
    product_name: str  
    quantity: int  
    total_price: float

    @model_validator  
    def validate_order(cls, model):  
        if model.quantity * 10 != model.total_price:  
            raise ValueError("Total price does not match the quantity.")  
        return model

Using the Model

# Valid order  
order = Order(product_name="Notebook", quantity=5, total_price=50.0)  
print(order)

# Invalid order  
try:  
    invalid_order = Order(product_name="Notebook", quantity=5, total_price=30.0)  
except ValueError as e:  
    print(e)

This approach ensures cross-field validation logic is centralized and clear.

Working with Complex Data Structures

Pydantic v2 retains support for parsing and validating complex data structures like lists, dictionaries, and sets. It also introduces better error messaging for composite types.

Example: Validating Lists and Dictionaries

from pydantic import BaseModel

class Inventory(BaseModel):  
    items: list[str]  
    item_counts: dict[str, int]

inventory_data = {  
    "items": ["Laptop", "Phone", "Tablet"],  
    "item_counts": {"Laptop": 5, "Phone": 10, "Tablet": 3}  
}  

inventory = Inventory(**inventory_data)  
print(inventory)

Validation is applied recursively to elements within these data structures, ensuring all items conform to the expected types.

Differences Between v1 and v2

Here are some key differences between Pydantic v1 and v2 for the features discussed:

1. Custom Validators:
   • v1: Used @validator.
   • v2: Uses @field_validator, which is more explicit and performant.
2. Model Validation:
   • v1: Overrode __post_init__ or used workarounds.
   • v2: Introduces @model_validator for built-in model-level validation.
3. Performance:
   • v2: Improved runtime validation and error messaging for nested and composite data structures.

Summary

Pydantic v2 provides a robust set of tools for handling advanced data validation and management. Key takeaways include:

1. Custom Field Validators: Use @field_validator to validate individual fields efficiently.
2. Nested Models: Easily define and validate hierarchical data structures.
3. Model Validators: Leverage @model_validator for cross-field validation logic.
4. Complex Data Structures: Seamlessly validate lists, dictionaries, and other composite types.

By addressing common challenges in data validation and introducing new capabilities, Pydantic v2 makes it easier to build reliable, maintainable Python applications. Start exploring its advanced features today!