[IDEA] refactor of Model onto pydantic BaseModel

[jesusch]

TLDR: Pydantic BaseModel refactor for pydynox

Problem. Two ways to declare a DynamoDB model today, each missing what the other has:

Subclass Model → full DynamoDB features, but no Pydantic (Field(description=...), validators, JSON schema).
@dynamodb_model decorator → real BaseModel, but loses aliases, templates, indexes, hooks, encryption, TTL, version, discriminator, atomic ops, conditions, ModelConfig.
Solution. Make Model itself a pydantic.BaseModel subclass. Keep all existing Attribute classes as the internal runtime representation. Users declare fields Pydantic-style with Annotated[T, Dynamo.*] markers.

Two things to know up front:

Pydantic reserves model_config → pydynox config moves to dynamodb_config: ClassVar[DynamoConfig].
Only breaking change: User.pk == "x" → User.F.pk == "x" (one-release shim provided).
Everything else survives unchanged because GSI/LSI, transactions, batch, hooks, metrics, create_table, etc. all read class dunders (_attributes, _py_to_dynamo, _indexes), not the descriptor protocol.

Where to read more
Decision: ADR/021-pydantic-base-model.md
Deep dive (diagrams, feature matrix, risks): docs/refactor/pydantic-model/overview.md
Per-PR specs: docs/refactor/pydantic-model/ (01- through 05-)

[github-actions]

👋 Automated Triage

Type: Idea / Enhancement
Area: Python layer (major refactor of Model base class)
Related files:

• python/pydynox/model.py — current Model implementation
• python/pydynox/_internal/_model/_base.py — ModelBase and ModelMeta
• python/pydynox/integrations/pydantic.py — existing @dynamodb_model decorator
• python/pydynox/attributes.py — current Attribute types

Summary:
This proposal suggests making Model inherit from pydantic.BaseModel to unify the two current approaches (native Model vs @dynamodb_model decorator). The issue references detailed design docs on a separate branch (docs/pydantic-basemodel-refactor-plan), including ADR-021 and implementation specs.

This would be a significant architectural change affecting the core Model API. The issue mentions one breaking change (condition syntax moving from User.pk == "x" to User.F.pk == "x").

---

Automated triage by the pydynox agent. Maintainer has final say on priority and acceptance.

[leandrodamascena]

Hey @jesusch, thanks for the detailed proposal and the time you put into the docs. I appreciate it.

I want to share my perspective on the direction before we invest time in PRs.

Where I stand

Pydynox's core value is speed and minimal dependencies. The Rust core handles serialization, compression, encryption, and all AWS SDK calls. Pydantic is a great library, but making it a required dependency goes against that principle. Today it's optional (pip install pydynox[pydantic]), and I want to keep it that way.

I understand the value of data validation, JSON schema generation, and native integration with API frameworks. These are real use cases. But not every pydynox user needs them - many just want a fast, lightweight DynamoDB ORM.

What I'd accept instead

PRs 1-3: yes, I like them. The renames and new syntax benefit everyone regardless of Pydantic:

• DynamoConfig / dynamodb_config - cleaner name, avoids future collision. Good.
• Annotated[T, Dynamo.*] markers - idiomatic Python, aligns with the ecosystem. Good.
• User.F.pk == "x" namespace - I can see the value even without the Pydantic motivation.

These are additive, non-breaking, and useful on their own. Happy to review PRs for these.

PR 4: not as proposed. Making Model inherit from BaseModel forces Pydantic on everyone. I don't want that.

Meet in the middle

Instead of changing Model's base class, what about a separate PydanticModel?

from pydynox import Model          # no Pydantic, works as today
from pydynox import PydanticModel  # Model + BaseModel, opt-in

PydanticModel inherits everything from Model - save, get, query, batch, transactions, hooks, indexes, encryption, the full feature set. Zero duplication. The only new code is the metaclass wiring to make Model and BaseModel cooperate, and a __setattr__ adjustment for dirty tracking under validate_assignment.

Users who want validators, model_dump_json(), JSON schema, and FastAPI integration use PydanticModel. Everyone else uses Model with zero extra dependencies.

This also gives us a clean deprecation path for @dynamodb_model - point users to PydanticModel instead.

If you're open to the PydanticModel direction, I'd be happy to discuss the details of the metaclass wiring. Let me know what you think.

[jesusch]

Olha @leandrodamascena

Thanks for taking the time to push back on this — you're right, and I've reworked the whole plan around your PydanticModel proposal. Pushed to docs/pydantic-basemodel-refactor-plan (commit ce8e91f).

What changed

Short version: Model doesn't change. Pydantic stays optional. All five PRs ship on minor versions.

• ADR 021 rewritten. Decision is now "add PydanticModel(Model, BaseModel) sibling" with "make Model a BaseModel — forces Pydantic on everyone" recorded as the rejected alternative.
• Overview has a new target-architecture diagram with Model and PydanticModel as siblings sharing the inherited DynamoDB feature set. The rollout gantt lost the "Breaking (major)" band entirely.
• PR 1 (DynamoConfig): unchanged scope, motivation retargeted. PydanticModel needs model_config free for ConfigDict; Model users can keep model_config = ModelConfig(...) indefinitely.
• PR 2 (Annotated markers): clarified as the only declaration surface on PydanticModel (Pydantic owns class-body assignments), optional alongside class-body descriptors on Model.
• PR 3 (cls.F): dropped the deprecation warning on User.pk == "x". Descriptor-style operators stay supported on Model with no nag. PydanticModel requires cls.F.* only because Pydantic owns class-level field access there.
• PR 4 renamed 04-basemodel-cutover.md → 04-pydantic-model.md and rewritten. Covers:
  • new module python/pydynox/pydantic_model.py, lazy-imports pydantic with a clear ImportError pointing at pip install pydynox[pydantic];
  • combined metaclass _PydanticModelMeta(ModelMeta, ModelMetaclass) that lets pydantic populate model_fields first, then runs _collect_dynamodb_schema(cls) to fill the same class dunders ModelMeta produces today (so CRUD, GSI/LSI, transactions, batch, hooks all work unchanged);
  • __setattr__ that delegates to BaseModel.__setattr__ first (so validate_assignment can raise before dirty-tracking moves) and then updates _original / _changed_fields;
  • to_dict / from_dict shims over model_dump(by_alias=True, mode="python") / model_validate with Attribute.serialize / deserialize applied for custom-transform fields (encrypted, compressed, S3, templates, aliases).
• PR 5 (deprecate @dynamodb_model): migration target flipped to PydanticModel, and the migration guide now shows two "after" examples — one PydanticModel path for validation/schema users, one plain Model path for users who want to drop pydynox[pydantic] entirely.

Things I'd appreciate your steer on

These are the open questions I flagged in the overview so each PR review doesn't re-litigate them:

1. Dynamo.* markers: plain @dataclass(frozen=True, slots=True) or pydantic BaseModel? I'm leaning dataclasses so Model users can use them without pulling pydantic in.
2. cls.F["pk"] string lookup: worth shipping for dynamic filter builders? I have it in the PR 3 doc; easy to drop.
3. to_dict / from_dict on PydanticModel: keep as thin wrappers around model_dump / model_validate so call sites stay identical to Model, or push users to model_dump_json directly?
4. Shall I start with PR 1, 2, 3?

[jesusch]

I openend the first PR #365
while I was not able to make sonar 100% happy :/

[leandrodamascena]

Hey @jesusch thanks for opening this! I'll take a look :D

[jesusch]

Oi @leandrodamascena
did you already had the time to look at this?

[jesusch]

Oi @leandrodamascena
tem alguma novidades?

[leandrodamascena]

Ei @jesusch, I was quite busy at work and will take a look tomorrow.

[leandrodamascena]

Hey @jesusch, I owe you an update here, and an honest correction.

In my earlier reply I said PRs 1-3 were accepted in principle. After talking to a few people who are using pydynox in production, I realized I was too hasty. Let me explain.

On PR #365 (DynamoConfig rename)

The rename only exists to free model_config for Pydantic's ConfigDict. But since we agreed that Model will NOT inherit from BaseModel - and PydanticModel will be a separate class - there's no collision for regular Model users. Ever.

Renaming ModelConfig to DynamoConfig and deprecating model_config means every current user gets a warning for a problem that doesn't affect them. That's churn and noise with no upside.

If PydanticModel needs a different config attribute name, that's something we handle inside PydanticModel itself and not by pushing a rename onto the entire codebase.

I'm going to close this PR. Sorry for the back and forth.

On PR #366 (Annotated markers)

The idea is good but the timing is wrong. Right now it:

• Depends on the rename from feat(config): rename ModelConfig to DynamoConfig with back-compat alias #365 (which I'm not merging)
• Only covers String/Number/Boolean/Binary - misses all the complex types (Map, JSON, List, Enum, Datetime, S3) where Annotated would actually shine
• Adds ~840 lines for a feature that's only strictly needed by PydanticModel (where Pydantic owns the class body)

For regular Model, the descriptor style (pk = StringAttribute(partition_key=True)) works great and is explicit. I don't think adding a second way to declare the same thing helps anyone right now.

I'd rather see this as part of the PydanticModel PR itself, where it's actually required and the full type coverage makes sense.

What I'd love to see next

The thing that actually matters is PydanticModel as a sibling class. Once that exists:

1. The Annotated markers make total sense (and should support all attribute types from day one)
2. The config name question resolves itself naturally inside that class
3. We deprecate @dynamodb_model and point people to PydanticModel

Would you be interested in working on that directly? I'm happy to pair on the metaclass wiring because that's the tricky part. Everything else (dirty tracking, to_dict/from_dict shims) should be straightforward once the metaclass cooperates.

Again, really sorry for the confusion. I should've thought this through more before saying "yes send the PR". I appreciate your patience and the quality of your work here.

Muito obrigado e desculpe de novo 💟

[jesusch]

Oi Leandro, All good. I will realign with the new information Yet this week I'm kinda busy Will let you know once I got a new PR ready B

…

On Sun, 10 May 2026, 18:15 Leandro Damascena, ***@***.***> wrote: *leandrodamascena* left a comment (ferrumio/pydynox#364) <#364 (comment)> Hey @jesusch <https://github.com/jesusch>, I owe you an update here, and an honest correction. In my earlier reply I said PRs 1-3 were accepted in principle. After talking to a few people who are using pydynox in production, I realized I was too hasty. Let me explain. On PR #365 <#365> (DynamoConfig rename) The rename only exists to free model_config for Pydantic's ConfigDict. But since we agreed that Model will NOT inherit from BaseModel - and PydanticModel will be a separate class - there's no collision for regular Model users. Ever. Renaming ModelConfig to DynamoConfig and deprecating model_config means every current user gets a warning for a problem that doesn't affect them. That's churn and noise with no upside. If PydanticModel needs a different config attribute name, that's something we handle inside PydanticModel itself and not by pushing a rename onto the entire codebase. I'm going to close this PR. Sorry for the back and forth. On PR #366 <#366> (Annotated markers) The idea is good but the timing is wrong. Right now it: - Depends on the rename from #365 <#365> (which I'm not merging) - Only covers String/Number/Boolean/Binary - misses all the complex types (Map, JSON, List, Enum, Datetime, S3) where Annotated would actually shine - Adds ~840 lines for a feature that's only strictly needed by PydanticModel (where Pydantic owns the class body) For regular Model, the descriptor style (pk = StringAttribute(partition_key=True)) works great and is explicit. I don't think adding a second way to declare the same thing helps anyone right now. I'd rather see this as part of the PydanticModel PR itself, where it's actually required and the full type coverage makes sense. What I'd love to see next The thing that actually matters is PydanticModel as a sibling class. Once that exists: 1. The Annotated markers make total sense (and should support all attribute types from day one) 2. The config name question resolves itself naturally inside that class 3. We deprecate @dynamodb_model and point people to PydanticModel Would you be interested in working on that directly? I'm happy to pair on the metaclass wiring because that's the tricky part. Everything else (dirty tracking, to_dict/from_dict shims) should be straightforward once the metaclass cooperates. Again, really sorry for the confusion. I should've thought this through more before saying "yes send the PR". I appreciate your patience and the quality of your work here. Muito obrigado e desculpe de novo 💟 — Reply to this email directly, view it on GitHub <#364 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AASA3LXKAG5ZZGV2AVLNIEL42DWP3AVCNFSM6AAAAACYBD4HB2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHM2DIMJWGM3DONZWGU> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you were mentioned.Message ID: ***@***.***>