ersilia.store package

Submodules

ersilia.store.api module

class ersilia.store.api.InferenceStoreApi(model_id: str, output: str | None = None, n_samples: int = -1, output_source: str = 'cloud-cache-only', click_iface: ClickInterface | None = None, api_client: ApiClient | None = None, file_manager: FileManager | None = None)

Bases: ErsiliaBase

Synchronous client for the Inference Store API orchestrating end-to-end batch cloud precalculation fetcher.

Parameters:

• model_id (str) – Identifier of the model to query.

• output (str, optional) – Path to save the merged CSV (default is “<model_id>_precalc.csv”).

• n_samples (int, optional) – Number of samples to retrieve (default is -1 for all available).

• output_source (OutputSource, optional) – Source of input data (default is OutputSource.CLOUD_ONLY).

• click_iface (ClickInterface, optional) – CLI helper instance for styled output (default creates new one).

• api_client (ApiClient, optional) – HTTP client for API interactions (default creates new one).

• file_manager (FileManager, optional) – File operations helper (default creates new one).

get_precalculations(inputs: list) → str

Execute full inference workflow: upload inputs, submit job, poll status, retrieve result shards, and merge them into a single CSV.

Parameters:

inputs (list) – List of input records to process.

Returns:

File path to the merged CSV containing inference results.

Return type:

str

Raises:

RuntimeError – If the job fails, no shards are returned, or polling times out.

ersilia.store.dump module

class ersilia.store.dump.DumpLocalCache

Bases: object

cache_missing_results(model_id, missing_inputs, computed_results)

conn_redis()

fetch_all_cached(model_id, dtype)

fetch_cached_results(model_id, data)

fetch_or_cache_header(model_id, computed_headers=None)

generate_redis_key(raw_string)

get_cached(model_id, data, dtype, computed_headers=None)

init_redis()

list_cached_inputs(model_id)

make_hashable(obj)

orient_to_json(values, columns, index, orient, output_type)

ersilia.store.utils module

class ersilia.store.utils.ApiClient(base_url: str | None = None, timeout: int = 3600)

Bases: object

HTTP client wrapper for JSON-based and streaming API requests.

Parameters:

• base_url (str, optional) – Base URL for all HTTP requests.

• timeout (int, optional) – Default timeout in seconds for HTTP requests.

download_stream(url: str, chunk_size: int = 65536)

Stream binary data from a URL in configurable chunk sizes.

Parameters:

• url (str) – URL to download data from.

• chunk_size (int, optional) – Size of each chunk in bytes (default is 65536).

Yields:

bytes – Chunks of data from the response.

get_json(endpoint: str, params: dict | None = None)

Perform a GET request and return parsed JSON.

Parameters:

• endpoint (str) – API endpoint path (appended to base_url).

• params (dict, optional) – Query parameters for the request.

Returns:

Parsed JSON response.

Return type:

dict

head(url: str)

Perform a HEAD request and return response headers.

Parameters:

url (str) – Full URL to send HEAD request to.

Returns:

Response headers.

Return type:

requests.structures.CaseInsensitiveDict

post_json(endpoint: str, json: dict | None = None)

Perform a POST request with JSON payload and return parsed JSON.

Parameters:

• endpoint (str) – API endpoint path (appended to base_url).

• json (dict, optional) – JSON payload to send.

Returns:

Parsed JSON response.

Return type:

dict

class ersilia.store.utils.ClickInterface(colourize: bool = True)

Bases: object

Interactive CLI helper for styled console output and progress bars.

Parameters:

colourize (bool, optional) – Whether to enable ANSI styling in output (default is True).

echo(message: str, fg: str | None = None, bg: str | None = None, bold: bool = False, blink: bool = False)

Print a styled message with optional colors and text effects.

Parameters:

• message (str) – The text to display.

• fg (str, optional) – Foreground color name or code.

• bg (str, optional) – Background color name or code.

• bold (bool, optional) – Whether to use bold text.

• blink (bool, optional) – Whether to use blinking text.

progress_bar(total: int, **kwargs)

Create and return a progress bar instance.

Parameters:

• total (int) – Total count for the progress bar.

• **kwargs – Additional keyword arguments passed to tqdm.

Returns:

Configured progress bar.

Return type:

tqdm.tqdm

class ersilia.store.utils.FileManager

Bases: object

Utility class for temporary CSV creation, S3 uploads, and merging result shards.

static create_temp_csv(records, input_adapter) → str

Write records to a temporary CSV file via a GenericInputAdapter.

Parameters:

• records (list) – Iterable of input records to write.

• input_adapter (GenericInputAdapter) – Adapter that yields dicts with ‘input’ key.

Returns:

Path of the created temporary CSV file.

Return type:

str

static merge_shards(shards: list, header: list, output_path: Path, downloader, click_iface: ClickInterface, local_cache_csv_path: str)

Download, decompress, and concatenate multiple CSV shards into a single file.

Parameters:

• shards (list) – List of dicts with keys ‘url’, ‘key’, and optional ‘size’.

• header (list) – CSV header row.

• output_path (Path) – Path to write the merged CSV.

• downloader (ApiClient) – Used for HEAD and streaming GET requests.

• click_iface (ClickInterface) – CLI interface for printing progress and warnings.

static upload_to_s3(presigned_info: dict, file_path: str)

Upload a local file to S3 using provided presigned URL and form fields.

Parameters:

• presigned_info (dict) – Contains ‘url’ and optional ‘fields’ for multipart upload.

• file_path (str) – Local path of the file to upload.

class ersilia.store.utils.InferenceStoreMessage(model_id)

Bases: object

Base class for inference store messages.

Parameters:

model_id (str) – The ID of the model for which the message is being generated.

class ersilia.store.utils.JobStatus

Bases: object

Class to define job completetion status.

FAILED = 'FAILED'

PENDING = 'PENDING'

RUNNING = 'RUNNING'

SUCCEEDED = 'SUCCEEDED'

class ersilia.store.utils.ModelNotInStore(model_id)

Bases: InferenceStoreMessage

Message class for models not found in the inference store.

Parameters:

model_id (str) – The ID of the model that is not found.

echo()

Echo the message for model not found in inference store.

class ersilia.store.utils.OutputSource

Bases: object

Class to define output source options.

ALL = ['local-cache-only', 'cloud-cache-only']

CACHE_ONLY = 'cache-only'

CLOUD = 'cloud-cache'

CLOUD_ONLY = 'cloud-cache-only'

LOCAL = 'local-cache'

LOCAL_ONLY = 'local-cache-only'

classmethod is_cloud(option)

Check if the option is cloud.

Parameters:

option (str) – The option to check.

Returns:

True if the option is cloud, False otherwise.

Return type:

bool

classmethod is_local(option)

Check if the option is local.

Parameters:

option (str) – The option to check.

Returns:

True if the option is local, False otherwise.

Return type:

bool

classmethod is_precalculation_enabled(option)

Check if the option is cloud.

Parameters:

option (str) – The option to check.

Returns:

True if the option is cloud, False otherwise.

Return type:

bool

class ersilia.store.utils.PrecalculationsInStore(model_id, output_url)

Bases: InferenceStoreMessage

Message class for precalculations found in the inference store.

Parameters:

• model_id (str) – The ID of the model for which precalculations are found.

• output_url (str) – The URL where the precalculations can be downloaded.

echo()

Echo the message for precalculations available for download.

Parameters:

output_url (str) – The URL for downloading the precalculations.

class ersilia.store.utils.PrecalculationsNotInStore(model_id)

Bases: InferenceStoreMessage

Message class for precalculations not found in the inference store.

Parameters:

model_id (str) – The ID of the model for which precalculations are not found.

echo()

Echo the message for precalculations not found in inference store.

ersilia.store.utils.delete_file_upon_upload(response_code: int, file_path: str)

Delete the file upon successful upload.

Parameters:

• response_code (int) – The HTTP response code from the upload request.

• file_path (str) – The path of the file to delete.

ersilia.store.utils.echo_found_shards(click_iface, count: int)

ersilia.store.utils.echo_intro(click_iface)

ersilia.store.utils.echo_job_submitted(click_iface, job_id: str)

ersilia.store.utils.echo_job_succeeded(click_iface)

ersilia.store.utils.echo_local_sample_warning(click_iface, n: int, cache_size: int)

ersilia.store.utils.echo_merged_saved(click_iface, output_path: Path)

ersilia.store.utils.echo_redis_fetched_missed(click_iface, size_res, size_missing)

ersilia.store.utils.echo_redis_job_submitted(click_iface)

ersilia.store.utils.echo_redis_local_completed(click_iface)

ersilia.store.utils.echo_small_sample_warning(click_iface, n: int)

ersilia.store.utils.echo_status(click_iface, status: str)

ersilia.store.utils.echo_submitting_job(click_iface, model_id)

ersilia.store.utils.echo_upload_complete(click_iface)

ersilia.store.utils.echo_uploading_inputs(click_iface)

ersilia.store.utils.log_prefix()

ersilia.store.utils.merge_csvs_stdlib(files, output)

ersilia.store.utils.store_has_model(model_id: str) → bool

Check if the model exists in the inference store.

Parameters:

model_id (str) – The ID of the model to check.

Returns:

True if the model exists in the store, False otherwise.

Return type:

bool

Module contents