Parsing without tbparse

To the best of our knowledge, there are no official documentation on parsing Tensorboard event files. It can be done with Tensorboard in two ways, which require inspecting the TensorFlow source code.

If you have some event logs that are difficult to parse with the current tbparse, you can ask us by opening an issue. Alternatively, you may parse the event logs by the methods described below.

Similar to other pages, we need to first generate a event file. We use PyTorch event writer to log scalars for simplicity. Since tensors and other type of events may require parsing Protocol Buffers, which needs more parsing code.

>>> import os
>>> import tempfile
>>> from torch.utils.tensorboard import SummaryWriter
>>> # Prepare temp dirs for storing event files
>>> tmpdir = tempfile.TemporaryDirectory()
>>> log_dir = tmpdir.name
>>> writer = SummaryWriter(log_dir)
>>> for i in range(5):
...   writer.add_scalar('y=2x', i * 2, i)
>>> writer.close()
>>> event_file = os.path.join(log_dir, os.listdir(log_dir)[0])

Event Accumulator

>>> from tensorboard.backend.event_processing.event_accumulator import EventAccumulator
>>> event_acc = EventAccumulator(event_file)
>>> event_acc.Reload() 
<tensorboard.backend.event_processing.event_accumulator.EventAccumulator object at ...>
>>> print(event_acc.Tags())
{'images': [], 'audio': [], 'histograms': [], 'scalars': ['y=2x'], 'distributions': [], 'tensors': [], 'graph': False, 'meta_graph': False, 'run_metadata': []}
>>> for e in event_acc.Scalars('y=2x'):
...   print(e.step, e.value)
0 0.0
1 2.0
2 4.0
3 6.0
4 8.0

Summary Iterator

>>> import tensorflow as tf
>>> from tensorflow.python.summary.summary_iterator import summary_iterator
>>> for e in summary_iterator(event_file):
...   for v in e.summary.value:
...     if v.tag == 'y=2x':
...       print(e.step, v.simple_value)
0 0.0
1 2.0
2 4.0
3 6.0
4 8.0

Experiment Data Access API for tensorboard.dev

The offical tensorboard data access API is only available if you are using tensorboard.dev, and only supports parsing scalar logs, which makes it not useful for general cases. The source code of the data acess API parses everything on the tensorboard.dev server, so it’s not possible to modify it for offline use.