tfx.components.StatisticsGen display train and eval in two different figures, is it possible to have them in a single figure as tfdv does?

Question

a superimposed display for train/val splits using StatisticsGen

Hi,

I'm currently using tfx pipeline inside kubeflow. I struggle to have StatisticsGen showing a single graph with train and validation splits curves superimposed, allowing better comparaison distributions. this is exactly how tfdv.visualize_statistics(lhs_statistics=train_stats, rhs_statistics=eval_stats, lhs_name='train', rhs_name='eval') behaves (see illustration 1), and I would like StatisticsGen to also provide a superimposed splits graph.

Thanks for any reference or help so that i can move forward.

Regards

score 0 · Answer 1 · answered Feb 05 '23 at 17:28

You can use something like

# docs-infra: no-execute
# Compare evaluation data with training data
tfdv.visualize_statistics(lhs_statistics=eval_stats, rhs_statistics=train_stats,
                          lhs_name='EVAL_DATASET', rhs_name='TRAIN_DATASET')

From the tensorflow data validation tutorial

tfx.components.StatisticsGen display train and eval in two different figures, is it possible to have them in a single figure as tfdv does?

1 Answers1