"You must feed a value for placeholder tensor ‘Placeholder’" when using BeholderHook

[EliaCereda]

• TensorBoard version: 1.8.0
• OS Platform and version: Linux Ubuntu 16.04 and macOS 10.13.4
• Python version: 3.5 and 3.6

Trying to train a model with a custom Estimator and the tensorboard.plugins.beholder.BeholderHook fails with the following error:

Traceback (most recent call last):
  File "/Users/elia/Applications/tf/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1322, in _do_call
    return fn(*args)
  File "/Users/elia/Applications/tf/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1307, in _run_fn
    options, feed_dict, fetch_list, target_list, run_metadata)
  File "/Users/elia/Applications/tf/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1409, in _call_tf_sessionrun
    run_metadata)
tensorflow.python.framework.errors_impl.InvalidArgumentError: You must feed a value for placeholder tensor 'Placeholder' with dtype uint8 and shape [?,?,?]
	 [[Node: Placeholder = Placeholder[dtype=DT_UINT8, shape=[?,?,?], _device="/job:localhost/replica:0/task:0/device:CPU:0"]()]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File “main.py", line 76, in <module>
    tf.app.run(main)
  File "/Users/elia/Applications/tf/lib/python3.6/site-packages/tensorflow/python/platform/app.py", line 126, in run
    _sys.exit(main(argv))
  File "main.py", line 57, in main
    hooks=hooks
  File "/Users/elia/Applications/tf/lib/python3.6/site-packages/tensorflow/python/estimator/estimator.py", line 363, in train
    loss = self._train_model(input_fn, hooks, saving_listeners)
  File "/Users/elia/Applications/tf/lib/python3.6/site-packages/tensorflow/python/estimator/estimator.py", line 843, in _train_model
    return self._train_model_default(input_fn, hooks, saving_listeners)
  File "/Users/elia/Applications/tf/lib/python3.6/site-packages/tensorflow/python/estimator/estimator.py", line 859, in _train_model_default
    saving_listeners)
  File "/Users/elia/Applications/tf/lib/python3.6/site-packages/tensorflow/python/estimator/estimator.py", line 1059, in _train_with_estimator_spec
    _, loss = mon_sess.run([estimator_spec.train_op, estimator_spec.loss])
  File "/Users/elia/Applications/tf/lib/python3.6/site-packages/tensorflow/python/training/monitored_session.py", line 567, in run
    run_metadata=run_metadata)
  File "/Users/elia/Applications/tf/lib/python3.6/site-packages/tensorflow/python/training/monitored_session.py", line 1043, in run
    run_metadata=run_metadata)
  File "/Users/elia/Applications/tf/lib/python3.6/site-packages/tensorflow/python/training/monitored_session.py", line 1134, in run
    raise six.reraise(*original_exc_info)
  File "/Users/elia/Applications/tf/lib/python3.6/site-packages/six.py", line 693, in reraise
    raise value
  File "/Users/elia/Applications/tf/lib/python3.6/site-packages/tensorflow/python/training/monitored_session.py", line 1119, in run
    return self._sess.run(*args, **kwargs)
  File "/Users/elia/Applications/tf/lib/python3.6/site-packages/tensorflow/python/training/monitored_session.py", line 1191, in run
    run_metadata=run_metadata)
  File "/Users/elia/Applications/tf/lib/python3.6/site-packages/tensorflow/python/training/monitored_session.py", line 971, in run
    return self._sess.run(*args, **kwargs)
  File "/Users/elia/Applications/tf/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 900, in run
    run_metadata_ptr)
  File "/Users/elia/Applications/tf/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1135, in _run
    feed_dict_tensor, options, run_metadata)
  File "/Users/elia/Applications/tf/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1316, in _do_run
    run_metadata)
  File "/Users/elia/Applications/tf/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1335, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: You must feed a value for placeholder tensor 'Placeholder' with dtype uint8 and shape [?,?,?]
	 [[Node: Placeholder = Placeholder[dtype=DT_UINT8, shape=[?,?,?], _device="/job:localhost/replica:0/task:0/device:CPU:0"]()]]

Caused by op 'Placeholder', defined at:
  File "main.py", line 76, in <module>
    tf.app.run(main)
  File "/Users/elia/Applications/tf/lib/python3.6/site-packages/tensorflow/python/platform/app.py", line 126, in run
    _sys.exit(main(argv))
  File "main.py", line 57, in main
    hooks=hooks
  File "/Users/elia/Applications/tf/lib/python3.6/site-packages/tensorflow/python/estimator/estimator.py", line 363, in train
    loss = self._train_model(input_fn, hooks, saving_listeners)
  File "/Users/elia/Applications/tf/lib/python3.6/site-packages/tensorflow/python/estimator/estimator.py", line 843, in _train_model
    return self._train_model_default(input_fn, hooks, saving_listeners)
  File "/Users/elia/Applications/tf/lib/python3.6/site-packages/tensorflow/python/estimator/estimator.py", line 859, in _train_model_default
    saving_listeners)
  File "/Users/elia/Applications/tf/lib/python3.6/site-packages/tensorflow/python/estimator/estimator.py", line 1056, in _train_with_estimator_spec
    log_step_count_steps=self._config.log_step_count_steps) as mon_sess:
  File "/Users/elia/Applications/tf/lib/python3.6/site-packages/tensorflow/python/training/monitored_session.py", line 405, in MonitoredTrainingSession
    stop_grace_period_secs=stop_grace_period_secs)
  File "/Users/elia/Applications/tf/lib/python3.6/site-packages/tensorflow/python/training/monitored_session.py", line 816, in __init__
    stop_grace_period_secs=stop_grace_period_secs)
  File "/Users/elia/Applications/tf/lib/python3.6/site-packages/tensorflow/python/training/monitored_session.py", line 532, in __init__
    h.begin()
  File "/Users/elia/Applications/tf/lib/python3.6/site-packages/tensorboard/plugins/beholder/beholder.py", line 212, in begin
    self.beholder = Beholder(self._logdir)
  File "/Users/elia/Applications/tf/lib/python3.6/site-packages/tensorboard/plugins/beholder/beholder.py", line 46, in __init__
    self.frame_placeholder = tf.placeholder(tf.uint8, [None, None, None])
  File "/Users/elia/Applications/tf/lib/python3.6/site-packages/tensorflow/python/ops/array_ops.py", line 1808, in placeholder
    return gen_array_ops.placeholder(dtype=dtype, shape=shape, name=name)
  File "/Users/elia/Applications/tf/lib/python3.6/site-packages/tensorflow/python/ops/gen_array_ops.py", line 4848, in placeholder
    "Placeholder", dtype=dtype, shape=shape, name=name)
  File "/Users/elia/Applications/tf/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
    op_def=op_def)
  File "/Users/elia/Applications/tf/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3392, in create_op
    op_def=op_def)
  File "/Users/elia/Applications/tf/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1718, in __init__
    self._traceback = self._graph._extract_stack()  # pylint: disable=protected-access

InvalidArgumentError (see above for traceback): You must feed a value for placeholder tensor 'Placeholder' with dtype uint8 and shape [?,?,?]
	 [[Node: Placeholder = Placeholder[dtype=DT_UINT8, shape=[?,?,?], _device="/job:localhost/replica:0/task:0/device:CPU:0"]()]]

I have found that replacing the definition of the placeholder in "tensorboard/plugins/beholder/beholder.py", line 46:

self.frame_placeholder = tf.placeholder(tf.uint8, [None, None, None])

with a placeholder with default values seems to work correctly:

self.frame_placeholder = tf.placeholder_with_default(tf.constant([[[]]], dtype=tf.uint8), [None, None, None])

[nfelt]

Thanks for the report, this was actually fixed in #1148 which should be in tb-nightly now and in the next release.

[EliaCereda]

For some reason GitHub search didn’t pull that up when I was looking for it. I’ll wait for the next release then, thanks for the fast response!