TypeError when writing data to Influx with None values using Python's aioinflux user defined schema

Question

So I have a schema that I defined and use with aioinflux library to write data from Python to InfluxDB:

from datetime import datetime
from aioinflux import lineprotocol, TIMEDT, TAG, FLOAT, MEASUREMENT, INT
from dataclasses import dataclass
from typing import Optional


@lineprotocol(
    schema=dict(
        timestamp=TIMEDT,
        measurement=MEASUREMENT,
        object_id=INT,
        dt=TAG,
        weight=FLOAT,
        width=FLOAT,
        size=FLOAT,
        risk=TAG,
        confidence=FLOAT,
    )
)
@dataclass
class CbdrDebugPoint:
    timestamp: datetime
    object_id: int
    dt: str
    weight: float
    size: Optional[float]
    width: Optional[float]
    risk: Optional[str]
    confidence: Optional[float]
    measurement: str = "my_table_name"

but if any of my Optional fields get value equal to None the writing to influx fails with the following error:

TypeError("Invalid variable type: value should be str, int or float, got None")

but for sure it is possible to write null values to influx, so any idea why does this error appear?

You can't pass null via http api, you will have to skip the null fields some how if the library you are using doesn't support this. — mfahadi, May 18 '22 at 09:33

score 0 · Answer 1 · answered May 19 '22 at 08:29

So after digging in the aioinflux package, it seems that is has this werid behavior:

If you try to write the data using the dictionary format it does accept None in the dictionary and it simply omits them from writing them to influx so you get the desired behavior -If you use the use defined schema with the dataclass it does not have the same behavior, isntead it tries to actually write None to Influx and it fails

So a way around it was for me to keep the user defined schema as short of documentation where the dataclass easily describes the model and it can be easily maintained but then I just added a to_dict method to make the dictionary that aioinflux expects as shown below:

from datetime import datetime
from aioinflux import lineprotocol, TIMEDT, TAG, FLOAT, MEASUREMENT, INT
from dataclasses import dataclass
from typing import Optional


@lineprotocol(
    schema=dict(
        timestamp=TIMEDT,
        measurement=MEASUREMENT,
        object_id=INT,
        dt=TAG,
        weight=FLOAT,
        width=FLOAT,
        size=FLOAT,
        risk=TAG,
        confidence=FLOAT,
    )
)
@dataclass
class CbdrDebugPoint:
    timestamp: datetime
    object_id: int
    dt: str
    weight: float
    size: Optional[float]
    width: Optional[float]
    risk: Optional[str]
    confidence: Optional[float]
    measurement: str = "my_table_name"

    def to_dict(self):
        return {
            "measurement": "my_table_name",
            "fields": {
                "object_id": self.object_id,
                "weight": self.weight,
                "size": self.size,
                "width": self.width,
                "confidence": self.confidence,
            },
            "tags": {
                "dt": self.dt,
                "risk": self.risk,
            },
            "time": self.timestamp,
        }

and then before writing to Influx I produce this dictionary. It is a hack but given that the library does not support writing None for user defined schema it is an acceptable solution.

TypeError when writing data to Influx with None values using Python's aioinflux user defined schema

1 Answers1