API Reference

All of the following can be imported from datalogs.

Logger

class Logger(root_directory: str, param_db: ParamDB[Any] | None = None)

class Logger(*, parent: Logger, description: str, timestamp: bool = True)

Logger corresponding to a directory that generates log files and sub-Logger objects corresponding to subdirectories.

If root_directory is given, that will be used as the directory, and this Logger will function as a root. Optionally, param_db can be given to enable commit tagging.

Otherwise, parent and description must be given, and this will be a sub-Logger object that corresponds to a subdirectory within its parent’s directory (and uses its parent’s ParamDB, if given). See Logger.sub_logger() for an explanation of the timestamp option.

sub_logger(description: str, timestamp: bool = True) → Logger

Create a new sub-Logger with the given description corresponding to a subdirectory within the parent Logger.

By default, timestamp is True, meaning that the directory name will include a timestamp corresponding to when it was created. (Note that the directory will be created when first needed so that the timestamp more accurately reflects when its content was created.)

If timestamp is False, the directory name will not include a timestamp. If there is an existing directory, it will be used. If not, a new directory will be created immediately.

property directory: str

Directory where this logger saves subdirectories or files.

If the directory does not yet exist (i.e. if this is a sub-Logger with a timestamp), it is created.

file_path(filename: str) → str

Generate a path to a file or directory with the given name within the directory of this Logger.

Note that this simply generates the path, with no checks for whether a file or directory with that path exists.

log_data(description: str, coords: Coord | Sequence[Coord], data_vars: DataVar | Sequence[DataVar], *, commit_id: int | None = None) → DataLog

Construct an Xarray from the given data and corresponding metadata, save it in a NetCDF file, and return a DataLog with this data and metadata.

The log will be tagged with the given commit ID, or the latest commit ID if none is given (and if this Logger has a corresponding ParamDB).

classmethod convert_to_json(obj: Any, convert: Callable[[Any], Any] | None = None) → Any

Return a JSON-serializable version of the given object. This function is used to convert objects to JSON for Logger.log_dict() and Logger.log_props().

1. If provided, convert() will be used to convert the object.

2. Numpy scalars will be unpacked and Pandas DataFrames will be converted to dictionaries.

3. Mapping and Collection objects will be converted to dictionaries and lists, with keys converted to strings and values converted according to these rules.

4. Other non-JSON-serializable values will be converted to repr() strings.

log_dict(description: str, dict_data: dict[str, Any], *, commit_id: int | None = None, convert: Callable[[Any], Any] | None = None) → DictLog

Save the given dictionary data and corresponding metadata in a JSON file, and return a DictLog with this data and metadata.

Objects will be converted according to Logger.convert_to_json(), with convert() passed to that function.

The log will be tagged with the given commit ID, or the latest commit ID if none is given (and if this Logger has a corresponding ParamDB).

log_props(description: str, obj: Any, *, commit_id: int | None = None, convert: Callable[[Any], Any] | None = None) → DictLog

Save a dictionary of the given object’s properties and corresponding metadata in a JSON file, and return a DictLog with this data and metadata.

Only properties that have been marked with a LoggedProp type hint at the top of the class definition will be saved. For example:

class Example:
    value: LoggedProp
    number: LoggedProp[float]

Objects will be converted according to Logger.convert_to_json(), with convert() passed to that function.

The log will be tagged with the given commit ID, or the latest commit ID if none is given (and if this Logger has a corresponding ParamDB).

LoggedProp

Used as a type hint to indicate that properties of a class should be logged by Logger.log_props().

Load Log

load_log(path: str) → DataLog | DictLog

Load the log specified by the given file path.

If the extension is “.nc” (NetCDF), a DataLog containing an Xarray Dataset will be loaded.

If the extension is “.json” (JSON), a DictLog containing a dictionary will be loaded.

Log Objects

class DataLog(metadata: LogMetadata, dataset: xarray.Dataset, path: str | None = None)

Log containing an Xarray Dataset which can be saved to a NetCDF (“.nc”) file.

See https://docs.xarray.dev/en/stable/generated/xarray.Dataset.html.

classmethod from_variables(metadata: LogMetadata, coords: Coord | Sequence[Coord], data_vars: DataVar | Sequence[DataVar]) → DataLog

Create a data log containing an Xarray Dataset constructed from the given coordinate and data variables.

property data: xarray.Dataset

Data stored in this log.

classmethod load(path: str) → DataLog

Load from the log file specified by the given path.

property metadata: LogMetadata

Metadata associated with this log.

property path: str

Path to the log file.

save() → None

Save log to a file.

class DictLog(metadata: LogMetadata, data_dict: dict[str, Any], path: str | None = None)

Log containing a dictionary which can be saved to a JSON (“.json”) file.

property metadata: LogMetadata

Metadata associated with this log.

property path: str

Path to the log file.

save() → None

Save log to a file.

property data: dict[str, Any]

Data stored in this log.

classmethod load(path: str) → DictLog

Load from the log file specified by the given path.

class LogMetadata(directory: str, timestamp: datetime.datetime, description: str, commit_id: int | None, param_db_path: str | None)

Metadata for a log, including the directory path, commit ID, description, created timestamp, and ParamDB path.

directory: str

Directory this log was created in.

timestamp: datetime

When the log was created.

description: str

Log description.

commit_id: int | None

Commit ID in the ParamDB.

param_db_path: str | None

Path to the ParamDB.

Data Log Variables

class Coord(name: str, data: numpy.typing.ArrayLike, long_name: str | None = None, units: str | None = None)

Wrapper for an Xarray Variable representing a dimensional coordinate.

• name is used as the coordinate and dimension name.

• data should be a 1D array labeling points along the dimension.

• long_name and units are stored in the underlying variable’s attribute dictionary.

See https://docs.xarray.dev/en/stable/user-guide/data-structures.html#coordinates for more information on coordinates in Xarray.

property name: str

Name of this coordinate or data.

property variable: xarray.Variable

Underlying Xarray Variable object.

See https://docs.xarray.dev/en/stable/generated/xarray.Variable.html.

class DataVar(name: str, dims: str | Sequence[str], data: numpy.typing.ArrayLike, long_name: str | None = None, units: str | None = None)

Wrapper for an Xarray Variable representing a data variable along particular dimensions.

• data should be an array that aligns with the named dimensions.

• long_name and units are stored in the underlying variable’s attribute dictionary.

property name: str

Name of this coordinate or data.

property variable: xarray.Variable

Underlying Xarray Variable object.

See https://docs.xarray.dev/en/stable/generated/xarray.Variable.html.