gobbli.model.base module

class gobbli.model.base.BaseModel(data_dir=None, load_existing=False, use_gpu=False, nvidia_visible_devices='all', logger=None, **kwargs)[source]

Bases: abc.ABC

Abstract base class for all models.

Derived classes should be careful to call super().__init__(…) with the appropriate arguments if they override __init__() to preserve all the functionality.

Functionality to facilitate making GPU(s) available to derived classes is available.

Create a model.

Parameters
  • data_dir (Optional[Path]) – Optional path to a directory used to store model data. If not given, a unique directory under GOBBLI_DIR will be created and used.

  • load_existing (bool) – If True, data_dir should be a directory that was previously used to create a model. Parameters will be loaded to match the original model, and user-specified model parameters will be ignored. If False, the data_dir must be empty if it already exists.

  • use_gpu (bool) – If True, use the nvidia-docker runtime (https://github.com/NVIDIA/nvidia-docker) to expose NVIDIA GPU(s) to the container. Will cause an error if the computer you’re running on doesn’t have an NVIDIA GPU and/or doesn’t have the nvidia-docker runtime installed.

  • nvidia_visible_devices (str) – Which GPUs to make available to the container; ignored if use_gpu is False. If not ‘all’, should be a comma-separated string: ex. 1,2.

  • logger (Optional[Logger]) – If passed, use this logger for logging instead of the default module-level logger.

  • **kwargs – Additional model-specific parameters to be passed to the model’s init() method.

build()[source]

Perform any pre-setup that needs to be done before running the model (building Docker images, etc).

property class_weights_dir

The root directory used to store initial model weights (before fine-tuning). These should generally be some pretrained weights made available by model developers. This directory will NOT be created by default; models should download their weights and remove the weights directory if the download doesn’t finish properly.

Most models making use of this directory will have multiple sets of weights and will need to store those in subdirectories under this directory.

Return type

Path

Returns

The path to the class-wide weights directory.

data_dir()[source]
Return type

Path

Returns

The main data directory unique to this instance of the model.

property info_path
Return type

Path

Returns

The path to the model’s info file, containing information about the model including the type of model, gobbli version it was trained using, etc.

abstract init(params)[source]

Initialize a derived model using parameters specific to that model.

Parameters

params (Dict[str, Any]) – A dictionary where keys are parameter names and values are parameter values.

property logger
Return type

Logger

Returns

A logger for derived models to use.

property metadata_path
Return type

Path

Returns

The path to the model’s metadata file containing model-specific parameters.

classmethod model_class_dir()[source]
Return type

Path

Returns

A directory shared among all classes of the model.

property weights_dir

The directory containing weights for a specific instance of the model. This is the class weights directory by default, but subclasses might define this property to return a subdirectory based on a set of pretrained model weights.

Return type

Path

Returns

The instance-specific weights directory.

gobbli.model.base.LOGGER = <Logger gobbli.model.base (WARNING)>