Django get all related objects from model with 'through' relationship tables

Question

In our app we have several relationships and several models, I'm trying to achieve a generic way to get all related objects of an object, even reverse ones.

If I print ._meta.get_fields() from my model Pessoa, i get these relationship fields(I omitted the 'normal' ones):

<ManyToManyRel: cadastroimoveis.pessoa>
<ManyToOneRel: cadastroimoveis.pessoa_pessoa>
<ManyToOneRel: cadastroimoveis.pessoa_pessoa>
<ManyToOneRel: cadastroimoveis.pessoa_itr>
<ManyToManyRel: cadastroimoveis.doc>
<ManyToOneRel: cadastroimoveis.doc_pessoa>
cadastroimoveis.Pessoa.relacoes
cadastroimoveis.Pessoa.itrs

This specific model only has M2M relationships, and all of them contain a 'through' model as specified Here.

As you can see, it repeats them, one for the model, and one for the 'through' intermediate table(also a model I guess). And in the case of the recursive relationship it repeats twice.

My question is, is there a way to get these non repeated?

A way to know which repeated fields 'point' to the same relationship in the end(even though it spams two tables)? Because if the through table has fields, I want to display them in a different manner.

And according to the Model _meta API documentation, you would use this to get all related objects :

[
    f for f in MyModel._meta.get_fields()
    if (f.one_to_many or f.one_to_one)
    and f.auto_created and not f.concrete
]

But the 'through' tables are not considered auto_created and are concrete.

Example :

<ManyToManyRel: cadastroimoveis.ccir>
<ManyToOneRel: cadastroimoveis.ccir_pessoa>

These two fields 'point' the same relationship, one is the intermediate table and the other is the model, is there a (automatic) way to know that these two are correlated? I couldn't find any attribute that they share.

The reason for this is because when the through table has fields, I need to edit on it instead of the M2M field on the model itself

Models.py : http://pastebin.com/szDfhHQ3 I cleaned the best I could

Answer 1

For Django 1.10, the following code has been inspired by the BaseModelForm code (Django original).

If you have the following relations:

class Group(Model):
    field = ....

class Person(Model):
    groups = ManyToManyField(Group, through='Membership')

class Membership(Model):
    person = ForeignKey(Person)
    group = ForeignKey(Group)
    position = TextField(...)

Then the related fields and attributes can be queried like this:

opts = Person._meta
for f in chain(opts.many_to_many, opts.virtual_fields):
    if f.rel.through:
        # this would return "group"
        attr_field = f.m2m_reverse_field_name()

        # this is the Membership class (a class instance)
        m2m_model = f.rel.through

        # this would return "person"
        join_field = field.m2m_field_name()

        # to get all "Membership" objects for "person" personXY
        qs_filter = {join_field: personXY}
        qs = m2m_model.objects.filter(**qs_filter)

        # get the PKs of all groups where personXY is a member of
        lookup_by_pk = '{}__pk'.format(attr_field)
        current_pks = qs.values_list(lookup_by_pk, flat=True) if qs.exists() else []

Answer 2

For instance we have this set of models. I've picked it up from this django example.

class Person(models.Model):
    name = models.CharField(max_length=50)

class Group(models.Model):
    name = models.CharField(max_length=128)
    members = models.ManyToManyField(
        Person,
        through='Membership',
        through_fields=('group', 'person'),
    )

class Membership(models.Model):
    group = models.ForeignKey(Group, on_delete=models.CASCADE)
    person = models.ForeignKey(Person, on_delete=models.CASCADE)
    inviter = models.ForeignKey(
        Person,
        on_delete=models.CASCADE,
        related_name="membership_invites",
    )
    invite_reason = models.CharField(max_length=64)

The solution looks a bit ugly, but can be optimized depending on your needs.

def get_through_field(f):                                                                                              
    opts = f.through._meta                                                                                             
    if f.through_fields:
        return opts.get_field(f.through_fields[1])                                                                     
    for field in opts.fields:                                                                                          
        rel = getattr(field, 'remote_field', None)                                                                     
        if rel and rel.model == f.model:                                                                               
            return field

model = models.Person

rels = dict(
    (f.field, f) for f in model._meta.get_fields()
    if f.is_relation
)

excludes = set()
for f in model._meta.get_fields():
    if f.many_to_many:
        through = get_through_field(f)
        excludes.add(rels[through])

for f in model._meta.get_fields():
    if f not in excludes:
        print f.name, f

Output:

group <ManyToManyRel: m.group>
membership_invites <ManyToOneRel: m.membership>
id m.Person.id
name m.Person.name

As you can see, there is no membership field.

Answer 3

The other answers definitely helped me figure this out, specifically in my case all my relationships are M2M and have a through table, also everything is done in AJAX/Javascript so I made the answer very JSON-y.

For now it only gets all through tables of the m2m models, because you have to create objects in them to create the relationship, but it can easily be expanded into getting all other relationships

def get_relationships(model):
    fields = list(model._meta.get_fields())

    m2m_fields = {}
    #Getting m2m relationships first
    for i, field in enumerate(fields):
        print(field)
        if field.is_relation:
            if field.many_to_many:
                fields.pop(i)
                try:
                    #If its a forward field, we want the relationship instead
                    if not hasattr(field,'field'):
                        field = field.remote_field
                except AttributeError:
                    pass
                if hasattr(field,'through'):
                    through = field.through
                    #In case of recursive relationships, there will be duplicates so we don't need to do it again
                    if m2m_fields.get(through._meta.model.__name__):
                        continue
                    m2m_fields[through._meta.model.__name__] = {}
                    m2m = m2m_fields[through._meta.model.__name__]
                    #Finding the models which participate in the through table and the direction
                    m2m['owner'] = {'model' : field.model.__name__}
                    m2m['related'] = {'model' : field.related_model.__name__}
                    recursive = False
                    #Checking recursivity, will use this later
                    #Finding field names for the foreignkeys of the through table
                    for through_field in through._meta.get_fields():
                        if not (through_field.related_model is None):
                            if m2m['owner']['model'] == through_field.related_model.__name__ and not m2m['owner'].get('field'):
                                m2m['owner']['field'] = through_field.name
                            elif m2m['related']['model'] == through_field.related_model.__name__ and not m2m['related'].get('field'):
                                m2m['related']['field'] = through_field.name
                        elif not through_field.primary_key:
                            if not m2m.get('rel_fields'):
                                m2m['rel_fields'] = []
                            m2m['rel_fields'].append(through_field.name)
    #Now removing the through tables from the fields list, because they appear as a regular ManyToOne relationship otherwise
    for through_table in m2m_fields.keys():
        name = through_table
        for i, field in enumerate(fields):
            if field.many_to_one:
                if field.__name__ and field.related_model:
                    if field.related_model.__name__ == name:
                        fields.pop(i)
    #Todo : OneToOne and ManyToOne relationships

    return m2m_fields


for key,value in get_relationships(Pessoa).items():
    print(key, " = ", value)

It is one hell of an ugly code, but I'm not very good at Python and just trying to learn things, but I guarantee it worked like a charm for my question