1

I am doing some inheritance in Pydantic.

Here is my code:

from typing import Dict, Any, List, Optional, Mapping
from pydantic import BaseModel, Field, ValidationError, validator
from enum import Enum



class PartitionCountType(int, Enum):
    one = 1
    two = 2
    three = 3
    four = 4
    five = 5
    six = 6

class TopicConfigType(BaseModel):
    env: Optional[str] = Field(None, description="The environment that the topic is in.")
    cleanup_policy: Optional[str] = Field(None, description="The cleanup policy of the topic.") 
    delete_retention_ms: Optional[str] = Field(None, description="The delete retention in milliseconds of the topic.") 
    max_compaction_lag_ms: Optional[str] = Field(None, description="The maximum compaction lag in milliseconds of the topic.") 
    max_message_bytes: Optional[str] = Field(None, description="The maximum message bytes of the topic.") 
    message_timestamp_difference_max_ms: Optional[str] = Field(None, description="The maximum message timestamp difference in milliseconds of the topic.") 
    message_timestamp_type: Optional[str] = Field(None, description="The message timestamp type of the topic.")
    min_compaction_lag_ms: Optional[str] = Field(None, description="The minimum compaction lag in milliseconds of the topic.")
    retention_bytes: Optional[str] = Field(None, description="The retention bytes of the topic.") 
    retention_ms: Optional[str] = Field(None, description="The retention in milliseconds of the topic.") 
    segment_bytes: Optional[str] = Field(None, description="The segment bytes of the topic.") 
    segment_ms: Optional[str] = Field(None, description="The segment in milliseconds of the topic.")  



class Topic(BaseModel):
    """
    Topic variables that are used throughout the program.
    """
    env: Optional[str] = Field(None, description="The environment that the topic is in.")
    topic_name: str = Field(..., description="The name of the topic.")
    partitions_count: PartitionCountType = Field(None, description="The number of partitions that the topic will have.")
    config: TopicConfigType = Field (description="The configuration of the topic.")

I am looking to do validation on the config mapping with the validator function in Pydantic. When I run this code cdktf synth im getting the following error

 TypeError: type of argument config must be one of (Mapping[str, str], NoneType); got custom_types.confluent.topic.TopicConfigType instead

Not sure why. The accepted type for config is Mapping[str,str] but I am passing that with value TopicConfigType. Seems off. Any insight would be helpful

Above is what Ive tried

Here is my construct for KafkaTopic

from imports.confluent.kafka_topic import KafkaTopic
from custom_types.confluent.topic import PartitionCountType
from custom_types.confluent.topic import TopicConfigType


class DTTopic(Construct):

    topic_name = KafkaTopic

    def __init__(self, scope: Construct, id:str, topic_name: str, partitions_count: PartitionCountType, topic_config: TopicConfigType, ):
        super().__init__(scope, id)
        
        self.topic_name = KafkaTopic(
            self, "topic_name", topic_name=topic_name, partitions_count=partitions_count, topic_config=topic_config
        )```


Here is me passing the data to the Confluent Provider
```        # Load the Confluent topics
        topics = {}

        if type(confluent_generated_config_env.providers.confluent) != type(None):

            for (k,v) in confluent_generated_config_env.providers.confluent.topics.items():
                topic_id = f"topic_{k}"  # Use a unique identifier for each topic

                topics[k] = DTTopic(
                    self,
                    topic_id,
                    topic_name=v.topic_name, 
                    partitions_count=v.partitions_count, 
                    config=v.topic_config
                )```
404Everything
  • 23
  • 4
  • I can't immediately find documentation that supports this, but I think `config` is reserved for configuring the `BaseModel` class itself. Try renaming your variable to `topic_config`. – SimonUnderwood May 26 '23 at 00:26
  • Im testing. One thing thats causing headache. the correct property on confluent for creating topics is "config" and this is messing with things docs: https://registry.terraform.io/providers/confluentinc/confluent/latest/docs/resources/confluent_kafka_topic – 404Everything May 26 '23 at 02:54
  • if I change the variable to topic_config and pass that into the cdk for confluent I get this error. ```TypeError: KafkaTopic.__init__() got an unexpected keyword argument 'topic_config'``` Because the KafkaTopic provider is expecting the "config" property not "topic_config" – 404Everything May 26 '23 at 02:56
  • I actually remember getting around something similar to this by setting an alias on the field. Try `topic_config: ... = Field(..., alias="config")` (replacing `...` with your code). – SimonUnderwood May 26 '23 at 03:04
  • 1
    @SimonUnderwood thanks for your input. I ended up just adding the fields from TopicConfigType to the config field in the Topic(BaseModel). This allowed me to use the validator class in Pydantic to do data validation on the fields in config. – 404Everything May 31 '23 at 17:41

0 Answers0