bimals.net
Using Pydantic to validate and serialize remote data
Use case
FPL’s bootstrap API’s response consists of lots of keys that I didn’t want to use for my project. And the data has to be type-checked and validated before saving in my Postgres database.
Overview
Firstly, I created a model that inherited Pydantic’s BaseModel for my data with the appropriate types.
class PlayerModel(BaseModel):
model_config = ConfigDict(from_attributes=True)
player_id: int = Field(..., alias='id')
first_name: str
second_name: str
now_cost: int
form: float
selected_by_percent: float
total_points: int
# And so on....
@computed_field
@property
def hash_value(self) -> str:
player_dict = self.model_dump(exclude={"hash_value"})
return calculate_hash(player_dict)
I have computed a hash_value field dynamically using the decorators. More on it on pitfalls section at the end.
model_config is defined at the start of the model to make sure it can serialize Python objects directly when fetched from APIs.
player_id: int = Field(..., alias='id')
By defining the attribute as Field, I’ve converted the id attribute from response to player_id in my project.
After calling the bootstrap api to get the data, the result is then passed to a validation function.
def get_players():
# Gets players data from remote
players = call_bootstrap_api()
# Passes the players data to validate with the pydantic model.
filtered_players_data = validate_players(players)
The validation function then passes each individual player data to the PlayerModel and returns all in a list.
def validate_players(players_data):
filtered_players = [
PlayerCompleteSchema(**player).model_dump() for player in players_data
]
return filtered_players
Note: model_dump() converts the resulting PlayerModel object back to a Python dictionary.
class PlayerModel(BaseModel):
model_config = ConfigDict(from_attributes=True)
player_id: int = Field(..., alias='id')
first_name: str
second_name: str
now_cost: int
def get_players():
# Gets players data from remote
players = call_bootstrap_api()
# Passes the players data to validate with the pydantic model.
filtered_players_data = validate_required_attributes(players)
return filtered_players_data[:2]
if __name__ == "__main__":
print(get_players())
Output:
[{'player_id': 1, 'first_name': 'Fábio', 'second_name': 'Ferreira Vieira', 'now_cost': 54},
{'player_id': 2, 'first_name': 'Gabriel', 'second_name': 'Fernando de Jesus', 'now_cost': 68}]
Pitfalls
If exclude={”hash_value”} is not included in the hash_value property below, it will throw: maximum recursion depth exceeded error. That is because model_dump() tries to include hash_value, which calls model_dump() and so on causing infinite recursion.
@computed_field
@property
def hash_value(self) -> str:
player_dict = self.model_dump(exclude={"hash_value"})
return calculate_hash(player_dict)
(This post is a small part of my FPL Dashboard project.)