When running the logger, four main output files are (or can be) generated. Three of these share the same prefix but with different file extensions. These are the main data file (ending with .dat), a channel mapping file (ending with .var) that contains information about the sources and channels recorded, and an event log file (ending with .log). The fourth file is a summary of the system state, written to the file name set as stateFile in the configuration file.

As an example, if the configuration file contains:

stateFile = "data/example.state"
saveState = True
prefix = "data/Log-"
rotate = True

Then the output files would be:

data/Log-2023030200.dat
data/Log-2023030200.var
data/Log-2023030200.log
data/example.state

File types

Main data file

As the name suggests, this file contains the data recorded from the configured sources. Any valid messages received from external sources, together with the messages generated by the logging software itself are stored in this file as a continuous stream of msgpack encoded data.

Each message is encoded as a four element array, containing a constant marker (0x55), source ID, channel ID, and finally the data for that message as illustrated in the technical details page.

Channel mapping / variable information file

This file is used to reduce the time required to process recorded data, and includes copies of source name and channel map messages that are also stored in the main data file.

This means that in the event that this channel mapping file is lost or damaged, the information can still be processed with only the main data file at a slower rate.

Data is encoded identically to the main data file.

Text log file

This file contains any information, warning, or error messages generated during recording. It is a plain text file, and should be readable using a standard text editor.

The contents of this file are never read by the software - it exists purely for information and troubleshooting purposes.

State file

The state file is a text based file containing a snapshot of the system state. There are three parts to the file:

Current timestamp
- The current value of the main time source (Source 2, Channel 2).
- This can be used to convert timestamps in the file back to an approximate date/time based on the file modification time
Path to channel mapping file
- This path is only valid on the computer running the logging software
- It allows tools to convert the source and channel IDs in this file back to friendly names
CSV data snapshot
- Each line of the snapshot contains the source ID, channel ID, timestamp of the last received message on that channel, and the last received value.

Software

In addition to the general programs and utilities, the python library includes support for reading and processing these files. See python/SELKIELogger/SLFiles.py for details.

Table of Contents