Feeding time data to tf.estimator: `input_fn` must be callable

Question

Goal

I want to mod this tutorial to consume this time series dataset instead of the run-of-the-mill image data.

Approach

I have identified several approaches to get the data into the tf.estimator API. The most convenient (because using .from_generator is guesswork...) was to use tf.data.Dataset.from_tensor_slices(training_data_ndarray) as follows:

trnX, trnY, tstX, tstY = load_dataset()
trnXl = trnX.tolist()
tstXl = tstX.tolist()
tstYl = tstY.tolist()
trnYl = trnY.tolist()
trndataset = tf.data.Dataset.from_tensor_slices((trnXl, trnYl))
tstdataset = tf.data.Dataset.from_tensor_slices((tstXl, tstYl))
...
def _input_fn(partition):
    if partition == "train":
        dst = trndataset
    elif partition == "predict":
        dst = tstdataset
    else:
        dst = tstdataset

    return dst

Error/Issue

TypeError: input_fn must be callable, given: DatasetV1Adapter shapes: ((128, 9), (6,)), types: (tf.float32, tf.float32)>

Reproduction

I use paperspace instance. If you have an account you can take a look here.

If not, get full code in this gist:.
Dataset from 2.

Setup:

Tensorflow 1.15
Python 3.6.8
Eager execution: off (adanet library won't handle it).

Error appears when calling:

tf.estimator.train_and_evaluate(
estimator,
train_spec=tf.estimator.TrainSpec(
    input_fn=_input_fn("train"),
    max_steps=TRAIN_STEPS),
eval_spec=tf.estimator.EvalSpec(
    input_fn=_input_fn("test"),
    steps=None,
    start_delay_secs=1,
    throttle_secs=1,  
))

What now?

I see no path forward using the approach of creating a tf.data.Dataset since it seems the input function ("create an input function that returns a dataset") is incorrect.

score 1 · Answer 1 · answered Feb 27 '20 at 19:10

This may help someone in the future but it doesn't solve my deeper issue.

In order to make the input_fn callable but still accept arguments, there are two options: wrap the function call in a lambda operator OR wrap the actual input_fn inside another input_fn that returns the inner function.

classifier.train(
input_fn=lambda: input_fn(train, train_y, training=True),
steps=5000)

Note that you wrap up your input_fn call in a lambda to capture the arguments while providing an input function that takes no arguments, as expected by the Estimator

Source: https://www.tensorflow.org/tutorials/estimator/premade