TensorFlow - Importing data from a TensorBoard TFEvent file?
Following works as of tensorflow version 2.0.0-beta1
:
import os
import tensorflow as tf
from tensorflow.python.framework import tensor_util
summary_dir = 'tmp/summaries'
summary_writer = tf.summary.create_file_writer('tmp/summaries')
with summary_writer.as_default():
tf.summary.scalar('loss', 0.1, step=42)
tf.summary.scalar('loss', 0.2, step=43)
tf.summary.scalar('loss', 0.3, step=44)
tf.summary.scalar('loss', 0.4, step=45)
from tensorflow.core.util import event_pb2
from tensorflow.python.lib.io import tf_record
def my_summary_iterator(path):
for r in tf_record.tf_record_iterator(path):
yield event_pb2.Event.FromString(r)
for filename in os.listdir(summary_dir):
path = os.path.join(summary_dir, filename)
for event in my_summary_iterator(path):
for value in event.summary.value:
t = tensor_util.MakeNdarray(value.tensor)
print(value.tag, event.step, t, type(t))
the code for my_summary_iterator
is copied from tensorflow.python.summary.summary_iterator.py
- there was no way to import it at runtime.
You can simply use:
tensorboard --inspect --event_file=myevents.out
or if you want to filter a specific subset of events of the graph:
tensorboard --inspect --event_file=myevents.out --tag=loss
If you want to create something more custom you can dig into the
/tensorflow/python/summary/event_file_inspector.py
to understand how to parse the event files.
As Fabrizio says, TensorBoard is a great tool for visualizing the contents of your summary logs. However, if you want to perform a custom analysis, you can use tf.train.summary_iterator()
function to loop over all of the tf.Event
and tf.Summary
protocol buffers in the log:
for summary in tf.train.summary_iterator("/path/to/log/file"):
# Perform custom processing in here.
UPDATE for tf2:
from tensorflow.python.summary.summary_iterator import summary_iterator
You need to import it, that module level is not currently imported by default. On 2.0.0-rc2
To read a TFEvent you can get a Python iterator that yields Event protocol buffers.
# This example supposes that the events file contains summaries with a
# summary value tag 'loss'. These could have been added by calling
# `add_summary()`, passing the output of a scalar summary op created with
# with: `tf.scalar_summary(['loss'], loss_tensor)`.
for e in tf.train.summary_iterator(path_to_events_file):
for v in e.summary.value:
if v.tag == 'loss' or v.tag == 'accuracy':
print(v.simple_value)
more info: summary_iterator