vis4d.data.io¶

Init io module.

class DataBackend[source]¶

Abstract class of storage backends.

All backends need to implement three functions: get(), set() and exists(). get() reads the file as a byte stream and set() writes a byte stream to a file. exists() checks if a certain filepath exists.

abstract close()[source]¶

Close all opened files in the backend.

Return type:: None

abstract exists(filepath)[source]¶

Check if filepath exists.

Parameters:: filepath (str) – The filepath to check.
Returns:: True if the filepath exists, False otherwise.
Return type:: bool

abstract get(filepath)[source]¶

Get the file content at the given filepath as bytes.

Parameters:: filepath (str) – The filepath to retrieve the data from.”
Returns:: The content of the file as bytes.
Return type:: bytes

abstract isfile(filepath)[source]¶

Check if filepath is a file.

Parameters:: filepath (str) – The filepath to check.
Returns:: True if the filepath is a file, False otherwise.
Return type:: bool

abstract listdir(filepath)[source]¶

List all files in a directory.

Parameters:: filepath (str) – The directory to list.
Returns:: A list of all files in the directory.
Return type:: list[str]

abstract set(filepath, content, mode)[source]¶

Set the file content at the given filepath.

Parameters:

filepath (str) – The filepath to store the data at.
content (bytes) – The content to store as bytes.
mode (str) – The mode to open the file in.

Return type:

None

class FileBackend[source]¶

Raw file from hard disk data backend.

close()[source]¶

No need to close manually.

Return type:: None

exists(filepath)[source]¶

Check if filepath exists.

Parameters:: filepath (str) – Path to file.
Returns:: True if file exists, False otherwise.
Return type:: bool

get(filepath)[source]¶

Get file content as bytes.

Parameters:: filepath (str) – Path to file.
Raises:: FileNotFoundError – If filepath does not exist.
Returns:: File content as bytes.
Return type:: bytes

isfile(filepath)[source]¶

Check if filepath is a file.

Parameters:: filepath (str) – Path to file.
Returns:: True if file exists, False otherwise.
Return type:: bool

listdir(filepath)[source]¶

List all files in the directory.

Parameters:: filepath (str) – Path to file.
Returns:: List of all files in the directory.
Return type:: list[str]

set(filepath, content, mode='w')[source]¶

Write the file content to disk.

Parameters:

filepath (str) – Path to file.
content (bytes) – Content to write in bytes.
mode (Literal["w", "a"], optional) – Overwrite or append mode. Defaults to “w”.

Return type:

None

class HDF5Backend[source]¶

Backend for loading data from HDF5 files.

This backend works with filepaths pointing to valid HDF5 files. We assume that the given HDF5 file contains the whole dataset associated to this backend.

You can use the provided script at vis4d/data/datasets/to_hdf5.py to convert your dataset to the expected hdf5 format before using this backend.

Creates an instance of the class.

close()[source]¶

Close all opened HDF5 files.

Return type:: None

exists(filepath)[source]¶

Check if filepath exists.

Parameters:: filepath (str) – Path to file.
Returns:: True if file exists, False otherwise.
Return type:: bool

get(filepath)[source]¶

Get values according to the filepath as bytes.

Parameters:

filepath (str) – The path to the file. It consists of an HDF5 path together with the relative path inside it, e.g.: “/path/to/ file.hdf5/key/subkey/data”. If no .hdf5 given inside filepath, the function will search for the first .hdf5 file present in the path, i.e. “/path/to/file/key/subkey/data” will also /key/ subkey/data from /path/to/file.hdf5.

Raises:

FileNotFoundError – If no suitable file exists.
ValueError – If key not found inside hdf5 file.

Returns:

The file content in bytes

Return type:

bytes

isfile(filepath)[source]¶

Check if filepath is a file.

Parameters:

filepath (str) – Path to file.

Raises:

FileNotFoundError – If no suitable file exists.
ValueError – If key not found inside hdf5 file.

Returns:

True if file exists, False otherwise.

Return type:

bool

listdir(filepath)[source]¶

List all files in the given directory.

Parameters:

filepath (str) – Path to directory.

Raises:

FileNotFoundError – If no suitable file exists.
ValueError – If key not found inside hdf5 file.

Returns:

List of files in the given directory.

Return type:

list[str]

set(filepath, content, mode='a')[source]¶

Set the file content.

Parameters:

filepath (str) – path/to/file.hdf5/key1/key2/key3
content (bytes) – Bytes to be written to entry key3 within group key2 within another group key1, for example.
mode (Literal['w', 'a']) – “w” to overwrite the file, “a” to append to it.

Raises:

ValueError – If filepath is not a valid .hdf5 file

Return type:

None

class ZipBackend[source]¶

Backend for loading data from Zip files.

This backend works with filepaths pointing to valid Zip files. We assume that the given Zip file contains the whole dataset associated to this backend.

Creates an instance of the class.

close()[source]¶

Close all opened Zip files.

Return type:: None

exists(filepath)[source]¶

Check if filepath exists.

Parameters:: filepath (str) – Path to file.
Returns:: True if file exists, False otherwise.
Return type:: bool

get(filepath)[source]¶

Get values according to the filepath as bytes.

Parameters:

filepath (str) – The path to the file. It consists of an Zip path together with the relative path inside it, e.g.: “/path/to/ file.zip/key/subkey/data”. If no .zip given inside filepath, the function will search for the first .zip file present in the path, i.e. “/path/to/file/key/subkey/data” will also /key/ subkey/data from /path/to/file.zip.

Raises:

ZipFileNotFoundError – If no suitable file exists.
OSError – If the file cannot be opened.
ValueError – If key not found inside zip file.

Returns:

The file content in bytes

Return type:

bytes

isfile(filepath)[source]¶

Check if filepath is a file.

Parameters:: filepath (str) – Path to file.
Returns:: True if file exists, False otherwise.
Return type:: bool

listdir(filepath)[source]¶

List all files in the given directory.

Parameters:: filepath (str) – The path to the directory.
Returns:: List of all files in the given directory.
Return type:: list[str]

set(filepath, content, mode='w')[source]¶

Write the file content to the zip file.

Parameters:

filepath (str) – path/to/file.zip/key1/key2/key3
content (bytes) – Bytes to be written to entry key3 within group key2 within another group key1, for example.
mode (Literal['w', 'a']) – Mode to open the file in. “w” for writing a file, “a” for appending to existing file.

Raises:

ValueError – If filepath is not a valid .zip file
NotImplementedError – If the method is not implemented.

Return type:

None

Modules

`vis4d.data.io.base`	Backends for the data types a dataset of interest is saved in.
`vis4d.data.io.file`	Standard backend for local files on a hard drive.
`vis4d.data.io.hdf5`	Hdf5 data backend.
`vis4d.data.io.util`	Data I/O Utilities.
`vis4d.data.io.zip`	Zip data backend.