Serializing many to many relationship with attributes does not show attribute in the relationship

Question

My conceptual model is that there are DemanderFeature objects which have LoadCurve objects linked to them in a many-to-many relationship, along with a single attribute indicating "how many times" the two are associated, using an attribute in the many-to-many relationship called number.

I have been struggling for quite a while now, reading many answers on stackoverflow but I just cannot get it to work in exactly the way that I want. This is my desired output, when looking at the detail view of a DemanderFeature:

HTTP 200 OK
Allow: GET, POST, HEAD, OPTIONS
Content-Type: application/json
Vary: Accept

[
    {
        "name": "testdemander",
        "loadcurves": [
            {"name": "lc", "number": 5},
            {"name": "lc2", "number": 10}
        ],
        // Other DemanderFeature fields removed for brevity...
    }
]

The closest I have been able to get to this is with this setup:

Models

class LoadCurve(models.Model):

    created = models.DateTimeField(auto_now_add=True)
    finalized = models.BooleanField(default=False)
    owner = models.ForeignKey('auth.User', on_delete=models.CASCADE)
    name = models.CharField(max_length=100)
    length = models.IntegerField(default=0)
    deleted = models.BooleanField(default=False)
    demanderfeatures = models.ManyToManyField("DemanderFeature", through="DemanderFeatureToLoadCurveAssociation")

    class Meta:
        ordering = ['name']
        constraints = [
            models.UniqueConstraint(fields=["owner", "name"], condition=models.Q(deleted=False), name="loadcurve_unique_owner_name")
        ]

class DemanderFeature(models.Model):
    created = models.DateTimeField(auto_now_add=True)
    name = models.CharField(max_length=100)
    owner = models.ForeignKey('auth.User', on_delete=models.CASCADE)
    demanderfeaturecollection = models.ForeignKey(DemanderFeatureCollection, on_delete=models.CASCADE)
    loadcurves = models.ManyToManyField("LoadCurve", through="DemanderFeatureToLoadCurveAssociation")
    deleted = models.BooleanField(default=False)
    geom = gis_models.PointField(default=None)

    class Meta:
        ordering = ['name']
        constraints = [
            models.UniqueConstraint(fields=["owner", "demanderfeaturecollection", "name"], condition=models.Q(deleted=False),
                                    name="demanderfeature_unique_owner_demanderfeaturecollection_name")
        ]

class DemanderFeatureToLoadCurveAssociation(models.Model):

    loadcurve = models.ForeignKey(LoadCurve, on_delete=models.CASCADE)
    demanderfeature = models.ForeignKey(DemanderFeature, on_delete=models.CASCADE)
    number = models.IntegerField()

Serializers

(I am using __all__ for the sake of debugging, so that I can see everything that is being serialized and available)

class LoadCurveSerializer(serializers.ModelSerializer):

    class Meta:
        model = LoadCurve
        fields = "__all__"


class DemanderFeatureToLoadCurveAssociationSerializer(serializers.ModelSerializer):

    class Meta:
        model = DemanderFeatureToLoadCurveAssociation
        fields = "__all__"


class DemanderFeatureSerializer(serializers.ModelSerializer):
    demanderfeaturecollection = serializers.SlugRelatedField(slug_field="name", queryset=DemanderFeatureCollection.objects.all())
    loadcurves = LoadCurveSerializer(read_only=True, many=True)
    # loadcurves = DemanderFeatureToLoadCurveAssociationSerializer(read_only=True, many=True)

    class Meta:
        model = DemanderFeature
        fields = "__all__"
        lookup_field = "name"

There is a commented line in the previous code block which I was trying to use to get the DemanderFeatureToLoadCurveAssociationSerializer because I thought this would be the proper way to get the number field which its related model defines, but when I uncomment this line (and comment the line just below it) I only get this error:

AttributeError at /demanderfeatures/

Got AttributeError when attempting to get a value for field `number` on serializer `DemanderFeatureToLoadCurveAssociationSerializer`.
The serializer field might be named incorrectly and not match any attribute or key on the `LoadCurve` instance.
Original exception text was: 'LoadCurve' object has no attribute 'number'.

If I do not swap those lines, however, I get this as a result:

HTTP 200 OK
Allow: GET, POST, HEAD, OPTIONS
Content-Type: application/json
Vary: Accept

[
    {
        "name": "testdemander",
        "loadcurves": [
            {
                "id": 1,
                "created": "2020-12-29T11:29:11.585034Z",
                "finalized": true,
                "name": "lc",
                "length": 0,
                "deleted": false,
                "owner": 1,
                "demanderfeatures": [
                    1
                ]
            },
            {
                "id": 2,
                "created": "2020-12-29T12:46:31.044624Z",
                "finalized": true,
                "name": "lc2",
                "length": 0,
                "deleted": false,
                "owner": 1,
                "demanderfeatures": [
                    1
                ]
            }
        ],
        // Other DemanderFeature fields removed for brevity...
    }
]

Which does not contain that critical number field which is defined in the DemanderFeatureToLoadCurveAssociation model.

I feel like I am just missing something quite obvious but I have not been able to find it.

Answer 1

I'm putting my specific implementation here in case it is useful to anyone. It is adapted from @JPG's own Q&A which was made as a response to this question (probably because it would be a pain to try to re-create my entire ORM and environment). I adapted their code to my own implementation.

It is worth noting however that this implementation only works one way. I originally tried implementing it with such a conceptual mapping:

• DemanderFeature -> Person
• LoadCurve -> Group
• Membership -> DemanderFeatureToLoadCurveAssociation

But since I wanted to see all the associated LoadCurve objects when retrieving a DemanderFeature object, I actually needed to flip this mapping around so that DemanderFeature objects are equivalent to Group objects and LoadCurve objects are equivalent to Person objects. It is obvious to me now why this is (I am conceptualizing the DemanderFeature as a GROUP of LoadCurves), but at the time I didn't see the technical difference.

Anyway here is the updated serializers.py. Thanks again @JPG for taking the time!

class LoadCurveSerializer(serializers.ModelSerializer):

    class Meta:
        model = LoadCurve
        fields = ["name"]

    def serialize_demanderfeaturetoloadcurveassociation(self, loadcurve_instance):
        # simple method to serialize the through model fields
        demanderfeaturetoloadcurveassociation_instance = loadcurve_instance \
            .demanderfeaturetoloadcurveassociation_set \
            .filter(demanderfeature=self.context["demanderfeature_instance"]) \
            .first()

        if demanderfeaturetoloadcurveassociation_instance:
            return DemanderFeatureToLoadCurveAssociationSerializer(demanderfeaturetoloadcurveassociation_instance).data
        return {}

    def to_representation(self, instance):
        rep = super().to_representation(instance)
        return {**rep, **self.serialize_demanderfeaturetoloadcurveassociation(instance)}


class DemanderFeatureToLoadCurveAssociationSerializer(serializers.ModelSerializer): # create new serializer to serialize the through model fields
    class Meta:
        model = DemanderFeatureToLoadCurveAssociation
        fields = ["number"]


class DemanderFeatureSerializer(serializers.ModelSerializer):
    demanderfeaturecollection = serializers.SlugRelatedField(slug_field="name", queryset=DemanderFeatureCollection.objects.all())
    loadcurves = serializers.SerializerMethodField()

    class Meta:
        model = DemanderFeature
        fields = ["name", "demanderfeaturecollection", "loadcurves", "geom"]
        # fields = "__all__"
        lookup_field = "name"

    def get_loadcurves(self, demanderfeature):
        return LoadCurveSerializer(
            demanderfeature.loadcurve_set.all(),
            many=True,
            context={"demanderfeature_instance": demanderfeature}  # should pass this `group` instance as context variable for filtering
        ).data