ersilia.hub.pull package

Submodules

ersilia.hub.pull.pull module

class ersilia.hub.pull.pull.ModelPuller(model_id: str, overwrite: bool = None, config_json: dict = None, docker_tag: str = None)

Bases: ErsiliaBase

ModelPuller is responsible for pulling models from DockerHub.

Parameters:

• model_id (str) – The ID of the model to be pulled.

• overwrite (bool, optional) – Whether to overwrite existing files.

• config_json (dict, optional) – Configuration settings for the puller.

Examples

puller = ModelPuller(
    model_id="eosxxxx", config_json=config
)
await puller.async_pull()

async async_pull()

Asynchronously pull the Docker image.

is_available_in_dockerhub() → bool

Check if the Docker image is available in DockerHub.

Returns:

True if the image is available in DockerHub, False otherwise.

Return type:

bool

is_available_locally() → bool

Check if the Docker image is available locally.

Returns:

True if the image is available locally, False otherwise.

Return type:

bool

pull()

This method pulls the Docker image non-asynchronously.

class ersilia.hub.pull.pull.PullProgress(expected_bytes=None, layer_sizes=None)

Bases: object

Progress of a Docker image pull, from the Docker Engine API’s pull events.

The Engine API reports progress as structured events (layer id, status, and byte counts), the same across Docker versions, instead of the text that docker pull prints.

Parameters:

• expected_bytes (int, optional) – Compressed size of the image, if known beforehand. Used as the total until every layer has reported its own size.

• layer_sizes (dict, optional) – Compressed size of each layer, keyed by its short digest (the 12 characters Docker uses as layer id), if known beforehand (e.g. from Docker Hub). Docker only reports a layer’s size once it starts downloading, so this keeps the total exact from the start.

DONE = ('Pull complete', 'Already exists')

bar()

Completed and total units for a progress bar.

Returns:

Downloading and extracting count as two halves when byte counts are known; otherwise finished layers out of all layers. The total is None until anything is known.

Return type:

tuple of (float, float or None)

describe()

Text shown next to the bar, e.g. “95/182 MB 3/14 layers”.

Returns:

Megabytes (decimal, as Docker reports them) and layers.

Return type:

str

property downloaded_bytes

Bytes downloaded so far.

property extracted_bytes

Bytes extracted so far.

property extracting

Whether everything is downloaded and layers are being extracted.

property layers_done

Number of layers pulled or already present.

property layers_total

Number of layers seen so far.

property total_bytes

Bytes to download (compressed), as far as known.

update(event)

Record one pull event.

Parameters:

event (dict) – A decoded event, e.g. {"id": "55d2dadd4bbc", "status": "Downloading", "progressDetail": {"current": 1, "total": 2}}.

ersilia.hub.pull.pull.pull_error(model_id, tag, error)

Turn a pull failure into an error that says why and what to do.

Parameters:

• model_id (str) – The model identifier.

• tag (str) – The image tag that was pulled.

• error (Exception) – The failure reported by Docker.

Returns:

The error to raise.

Return type:

ImagePullError

ersilia.hub.pull.pull.pull_with_progress(repository, tag, callback, platform=None, expected_bytes=None, layer_sizes=None, stop=None)

Pull a Docker image through the Docker Engine API, reporting progress.

Parameters:

• repository (str) – Image repository, e.g. “ersiliaos/eos3b5e”.

• tag (str) – Image tag.

• callback (callable) – Called with a PullProgress after every event.

• platform (str, optional) – Platform to pull, e.g. “linux/amd64”.

• expected_bytes (int, optional) – Compressed image size, if known.

• layer_sizes (dict, optional) – Compressed size of each layer by short digest, if known.

• stop (threading.Event, optional) – When set, the pull stops (e.g. after Ctrl+C).

Raises:

RuntimeError – If the pull fails.

Module contents