bimals.net

Using Pydantic to validate and serialize remote data

Use case

FPL’s bootstrap API’s response consists of lots of keys that I didn’t want to use for my project. And the data has to be type-checked and validated before saving in my Postgres database.

Overview

Firstly, I created a model that inherited Pydantic’s BaseModel for my data with the appropriate types.

class PlayerModel(BaseModel):
    model_config = ConfigDict(from_attributes=True)

    player_id: int = Field(..., alias='id')
    first_name: str
    second_name: str
    now_cost: int
    form: float
    selected_by_percent: float
    total_points: int
    # And so on....
    @computed_field
    @property
    def hash_value(self) -> str:

        player_dict = self.model_dump(exclude={"hash_value"})
        return calculate_hash(player_dict)

I have computed a hash_value field dynamically using the decorators. More on it on pitfalls section at the end.

model_config is defined at the start of the model to make sure it can serialize Python objects directly when fetched from APIs.

    player_id: int = Field(..., alias='id')

By defining the attribute as Field, I’ve converted the id attribute from response to player_id in my project.

After calling the bootstrap api to get the data, the result is then passed to a validation function.

def get_players():
    
    # Gets players data from remote
    players = call_bootstrap_api()
    
    # Passes the players data to validate with the pydantic model.
    filtered_players_data = validate_players(players)

The validation function then passes each individual player data to the PlayerModel and returns all in a list.

def validate_players(players_data):

    filtered_players = [
        PlayerCompleteSchema(**player).model_dump() for player in players_data
    ]
    return filtered_players

Note: model_dump() converts the resulting PlayerModel object back to a Python dictionary.

class PlayerModel(BaseModel):
    model_config = ConfigDict(from_attributes=True)
    player_id: int = Field(..., alias='id')
    first_name: str
    second_name: str
    now_cost: int
def get_players():
	# Gets players data from remote
  players = call_bootstrap_api()
	# Passes the players data to validate with the pydantic model.
	filtered_players_data = validate_required_attributes(players)
	return filtered_players_data[:2]
    
if __name__ == "__main__":
	print(get_players())

Output:

[{'player_id': 1, 'first_name': 'Fábio', 'second_name': 'Ferreira Vieira', 'now_cost': 54}, 
{'player_id': 2, 'first_name': 'Gabriel', 'second_name': 'Fernando de Jesus', 'now_cost': 68}]

Pitfalls

If exclude={”hash_value”} is not included in the hash_value property below, it will throw: maximum recursion depth exceeded error. That is because model_dump() tries to include hash_value, which calls model_dump() and so on causing infinite recursion.

@computed_field
@property
def hash_value(self) -> str:
player_dict = self.model_dump(exclude={"hash_value"})
return calculate_hash(player_dict)

(This post is a small part of my FPL Dashboard project.)

Last updated: 10/17/2024
Tags:PythonPydanticFastAPI