Featuretools dfs runtime error

Question

Working through the featuretools "predict_next_purchase" demo against my own data. I've created the entity set, and have also created a new pandas.dataframe comprised of the labels and times. I'm to the point of using ft.dfs for deep feature synthesis, and am getting a RuntimeError: maximum recursion depth exceeded. Below is the stack trace:

feature_matrix, features = ft.dfs(target_entity='projects', 
                              cutoff_time=labels.reset_index().loc[:,['jobnumber','time']],
                              training_window=inst_defn['training_window'],
                              entityset=es,
                              verbose=True)

Stack Trace:

Building features: 0it [00:00, ?it/s]
RuntimeError: maximum recursion depth exceeded
RuntimeErrorTraceback (most recent call last)
<ipython-input-743-f05fc567dd1b> in <module>()
      3                                   training_window=inst_defn['training_window'],
      4                                   entityset=es,
----> 5                                   verbose=True)
/Users/nbernini/OneDrive/PSC/venv/ml20/lib/python2.7/site-packages/featuretools/synthesis/dfs.pyc in dfs(entities, relationships, entityset, target_entity, cutoff_time, instance_ids, agg_primitives, trans_primitives, allowed_paths, max_depth, ignore_entities, ignore_variables, seed_features, drop_contains, drop_exact, where_primitives, max_features, cutoff_time_in_index, save_progress, features_only, training_window, approximate, verbose)
    164                                       seed_features=seed_features)
    165 
--> 166     features = dfs_object.build_features(verbose=verbose)
    167 
    168     if features_only:
/Users/nbernini/OneDrive/PSC/venv/ml20/lib/python2.7/site-packages/featuretools/synthesis/deep_feature_synthesis.pyc in build_features(self, variable_types, verbose)
    227         self.where_clauses = defaultdict(set)
    228         self._run_dfs(self.es[self.target_entity_id], [],
--> 229                       all_features, max_depth=self.max_depth)
    230 
    231         new_features = list(all_features[self.target_entity_id].values())
/Users/nbernini/OneDrive/PSC/venv/ml20/lib/python2.7/site-packages/featuretools/synthesis/deep_feature_synthesis.pyc in _run_dfs(self, entity, entity_path, all_features, max_depth)
    353                           entity_path=list(entity_path),
    354                           all_features=all_features,
--> 355                           max_depth=new_max_depth)
    356 
    357         """
/Users/nbernini/OneDrive/PSC/venv/ml20/lib/python2.7/site-packages/featuretools/synthesis/deep_feature_synthesis.pyc in _run_dfs(self, entity, entity_path, all_features, max_depth)
    338                              if self._apply_traversal_filters(entity, self.es[b_id],
    339                                                               entity_path,
--> 340                                                               forward=False) and
    341                              b_id not in self.ignore_entities]
    342         for b_entity_id in backward_entities:
/Users/nbernini/OneDrive/PSC/venv/ml20/lib/python2.7/site-packages/featuretools/synthesis/deep_feature_synthesis.pyc in _apply_traversal_filters(self, parent_entity, child_entity, entity_path, forward)
    429                               child_entity=child_entity,
    430                               target_entity_id=self.target_entity_id,
--> 431                               entity_path=entity_path, forward=forward):
    432                 return False
    433 
/Users/nbernini/OneDrive/PSC/venv/ml20/lib/python2.7/site-packages/featuretools/synthesis/dfs_filters.pyc in is_valid(self, feature, entity, target_entity_id, child_feature, child_entity, entity_path, forward, where)
     53 
     54         if type(feature) != list:
---> 55             return func(*args)
     56 
     57         else:
/Users/nbernini/OneDrive/PSC/venv/ml20/lib/python2.7/site-packages/featuretools/synthesis/dfs_filters.pyc in apply_filter(self, parent_entity, child_entity, target_entity_id, entity_path, forward)
     76             if (parent_entity.id == target_entity_id or
     77                     es.find_backward_path(parent_entity.id,
---> 78                                           target_entity_id) is None):
     79                 return True
     80             path = es.find_backward_path(parent_entity.id, child_entity.id)
/Users/nbernini/OneDrive/PSC/venv/ml20/lib/python2.7/site-packages/featuretools/entityset/base_entityset.pyc in find_backward_path(self, start_entity_id, goal_entity_id)
    308             is returned if no path exists.
    309         """
--> 310         forward_path = self.find_forward_path(goal_entity_id, start_entity_id)
    311         if forward_path is not None:
    312             return forward_path[::-1]
/Users/nbernini/OneDrive/PSC/venv/ml20/lib/python2.7/site-packages/featuretools/entityset/base_entityset.pyc in find_forward_path(self, start_entity_id, goal_entity_id)
    287 
    288         for r in self.get_forward_relationships(start_entity_id):
--> 289             new_path = self.find_forward_path(r.parent_entity.id, goal_entity_id)
    290             if new_path is not None:
    291                 return [r] + new_path
... last 1 frames repeated, from the frame below ...
/Users/nbernini/OneDrive/PSC/venv/ml20/lib/python2.7/site-packages/featuretools/entityset/base_entityset.pyc in find_forward_path(self, start_entity_id, goal_entity_id)
    287 
    288         for r in self.get_forward_relationships(start_entity_id):
--> 289             new_path = self.find_forward_path(r.parent_entity.id, goal_entity_id)
    290             if new_path is not None:
    291                 return [r] + new_path
RuntimeError: maximum recursion depth exceeded

Can you share the structure of EntitySet you're working with? — Gaurav, Mar 21 '18 at 17:32
Solved my own issue. I had an entity w/ a relationship to itself. Entityset: modeldata Entities: hours (shape = [43, 6]) inspections (shape = [1736, 4]) incidents (shape = [9, 3]) observations (shape = [56206, 19]) projects (shape = [2, 1]) Relationships: observations.inspectionid -> inspections.inspectionid inspections.jobnumber -> projects.jobnumber hours.jobnumber -> projects.jobnumber projects.jobnumber -> projects.jobnumber — Nick Bernini, Mar 21 '18 at 19:37

score 3 · Answer 1 · answered Mar 21 '18 at 20:26

The issue here is cyclical relationships in your entity set. Currently, Deep Feature Synthesis can only create features when there is one unique path between two entities. If you have an entity with a relationship to itself, you would also get this error.

A future release of Featuretools will offer better support for this use case.

Featuretools dfs runtime error

1 Answers1