ersilia.publish.test.services package

Submodules

ersilia.publish.test.services.checks module

class ersilia.publish.test.services.checks.CheckService(logger: Any, model_id: str, dir: str, from_github: bool, from_dockerhub: bool, ios: IOService)

Bases: object

Service for performing various checks on the model.

Parameters:

• logger (logging.Logger) – Logger for logging messages.

• model_id (str) – Identifier of the model.

• dir (str) – Directory where the model repository is located.

• from_github (bool) – Flag indicating whether to fetch the repository from GitHub.

• from_dockerhub (bool) – Flag indicating whether to fetch the repository from DockerHub.

• ios (IOService) – Instance of IOService for handling input/output operations.

Examples

check_service = CheckService(
    logger=logger,
    model_id="model_id",
    dir="/path/to/dir",
    from_github=True,
    from_dockerhub=False,
    ios=ios,
)
check_service.check_files()

check_consistent_output(**kwargs)

check_files()

Check the existence of required files for the model.

check_information(**kwargs)

check_model_output_content(**kwargs)

check_simple_model_output(run_model)

compare_csv_columns(column_csv, csv_file)

get_inputs(types)

trim_string(text, max_length=100)

validate_file_content(file_path, input_type)

Validate output file content and check input SMILES match.

ersilia.publish.test.services.constants module

class ersilia.publish.test.services.constants.Checks(value)

Bases: Enum

Enum for different check types.

COLUMN_CHECK_FAILURE = 'Columns not coincide with run_columns'

COLUMN_CHECK_SUCCESS = 'Columns coincides with run_columns'

COLUMN_NAME_VALIDITY = 'Columns'

CONSISTENCY = 'Model Output Was Consistent'

CSV_CONTENT_CHECK = 'CSV File Content Check'

DEPENDENCY_PINNED = 'Checking package versions and file structure'

DIR_SIZE = 'Directory Size Mb'

ENV_SIZE = 'Environment Size Mb'

IMAGE_SIZE = 'Image Size Mb'

INCONSISTENCY = 'Inconsistent Output Detected'

MODEL_CONSISTENCY = 'Check Consistency of Model Output'

PREDEFINED_EXAMPLE = 'Check Predefined Example Input'

RUN_BASH = 'RMSE-MEAN'

SIMPLE_MODEL_RUN = 'Simple Model Run'

SIMPLE_MODEL_RUN_COLUMNS = 'Simple Model Run Columns'

SIZE_CACL_FAILED = 'Size Calculation Failed'

SIZE_CACL_SUCCESS = 'Size Successfully Calculated'

TOTAL_DIR_SIZE = 'Total directory size'

class ersilia.publish.test.services.constants.Options(value)

Bases: Enum

Enum for different options.

BASE = 'base'

DEEP_INPUT = '/root/eos/temp/files/deep_input.csv'

EXAMPLE_CSV = '/root/eos/temp/files/example.csv'

INPUT_CSV = '/root/eos/temp/files/input.csv'

INPUT_TYPES = ['str', 'list', 'csv']

NUM_SAMPLES = 5

OUTPUT1_CSV = '/root/eos/temp/files/output1.csv'

OUTPUT2_CSV = '/root/eos/temp/files/output2.csv'

OUTPUT_CSV = '/root/eos/temp/files/result.csv'

OUTPUT_FILES = ['/root/eos/temp/files/file.csv', '/root/eos/temp/files/file.h5', '/root/eos/temp/files/file.json']

class ersilia.publish.test.services.constants.STATUS_CONFIGS(value)

Bases: Enum

Enum for status configurations.

FAILED = ('FAILED', 'red', '✘')

NA = ('N/A', 'dim', '~')

NOT_PRESENT = ('NOT PRESENT', 'yellow', '✘')

PASSED = ('PASSED', 'green', '✔')

SUCCESS = ('SUCCESS', 'green', '★')

WARNING = ('WARNING', 'yellow', '⚠')

class ersilia.publish.test.services.constants.TableConfig(title: str, headers: List[str])

Bases: object

Configuration for a table.

headers: List[str]

title: str

class ersilia.publish.test.services.constants.TableType(value)

Bases: Enum

Enum for different table types.

COMPUTATIONAL_PERFORMANCE = 'Computational Performance Check'

CONSISTENCY_BASH = 'Consistency Summary Between Ersilia and Bash Execution Outputs'

DEPENDECY_COLUMN_CHECK = 'Dependency and Column Value Checks'

FINAL_RUN_SUMMARY = 'Test Run Summary'

INSPECT_SUMMARY = 'Inspect Summary'

MODEL_DIRECTORY_SIZES = 'Directory Size Check'

MODEL_FILE_CHECKS = 'Model File Checks'

MODEL_INFORMATION_CHECKS = 'Model Metadata Checks'

MODEL_OUTPUT = 'Input Output Check'

MODEL_RUN_CHECK = 'Model Run Check'

MODEL_SIZES = 'Model Size Check'

RUNNER_CHECKUP_STATUS = 'Runner Checkup Status'

SHALLOW_CHECK_SUMMARY = 'Model Output Consistency Check'

ersilia.publish.test.services.inspect module

class ersilia.publish.test.services.inspect.CheckStrategy(check_function, success_key, details_key)

Bases: object

Execuetd a strategy for checking inspect commands.

Parameters:

• check_function (callable) – The function to check.

• success_key (str) – The key for success.

• details_key (str) – The key for details.

execute()

Execute the check strategy.

Returns:

The results of the check.

Return type:

dict

class ersilia.publish.test.services.inspect.InspectService(dir: str, model: str, remote: bool = False, config_json: str | None = None, credentials_json: str | None = None)

Bases: ErsiliaBase

Service for inspecting models and their configurations.

Parameters:

• dir (str, optional) – Directory where the model is located.

• model (str, optional) – Model identifier.

• remote (bool, optional) – Flag indicating whether the model is remote.

• config_json (str, optional) – Path to the configuration JSON file.

• credentials_json (str, optional) – Path to the credentials JSON file.

Examples

inspector = InspectService(
    dir="/path/to/model", model="model_id"
)
results = inspector.run()

run(check_keys: list | None = None) → dict

Run the inspection checks on the specified model.

Parameters:

check_keys (list, optional) – A list of check keys to execute. If None, all checks will be executed.

Returns:

A dictionary containing the results of the inspection checks.

Return type:

dict

Raises:

• ValueError – If the model is not specified.

• KeyError – If any of the specified keys do not exist.

class ersilia.publish.test.services.inspect.ModelInspector(model: str, dir: str, config_json=None)

Bases: object

Class for inspecting model repositories.

Parameters:

• model (str) – The ID of the model to be inspected.

• dir (str) – The directory where the model repository is located.

• config_json (str, optional) – Path to the configuration JSON file.

Examples

inspector = ModelInspector(
    model="model_id", dir="path/to/repo"
)
result = inspector.check_repo_exists()
result = inspector.check_complete_metadata()

BENTOML_FILES = ['model/framework/run.sh', 'README.md', 'LICENSE', 'Dockerfile', 'metadata.json', 'model/framework/run.sh', 'src/service.py', 'pack.py', 'README.md', 'LICENSE']

ERSILIAPACK_FILES = ['model/framework/run.sh', 'README.md', 'LICENSE', 'install.yml', 'metadata.yml', 'model/framework/examples/run_input.csv', 'model/framework/examples/run_output.csv', 'model/framework/columns/run_columns.csv', 'model/framework/run.sh', 'README.md', 'LICENSE']

REQUIRED_FIELDS = ['Publication', 'Source Code', 'S3', 'DockerHub']

check_complete_folder_structure()

Check if the folder structure of the repository is complete.

Returns:

A namedtuple containing the success status and details of the check.

Return type:

Result

check_complete_metadata()

Check if the metadata file is complete.

Returns:

A namedtuple containing the success status and details of the check.

Return type:

Result

check_computational_performance()

Check the computational performance of the model.

Returns:

A namedtuple containing the success status and details of the check.

Return type:

Result

check_dependencies_are_valid()

Check if the dependencies in the Dockerfile or install.yml are valid.

For PACK_METHOD_BENTOML, the Dockerfile is validated. For PACK_METHOD_FASTAPI, either file is acceptable. If one exists, it is validated; if both exist, both are validated. If neither is present, an error is returned.

Returns:

A namedtuple containing the success status and details of the check.

Return type:

Result

check_no_extra_files()

Check if there are no extra files in the repository.

Returns:

A namedtuple containing the success status and details of the check.

Return type:

Result

check_repo_exists()

Check if the model repository exists.

Returns:

A namedtuple containing the success status and details of the check.

Return type:

Result

validate_repo_structure()

Validate the repository structure.

Return type:

List of missing items.

class ersilia.publish.test.services.inspect.Result(success, details)

Bases: tuple

details

Alias for field number 1

success

Alias for field number 0

ersilia.publish.test.services.io module

class ersilia.publish.test.services.io.IOService(logger, model_id: str, dir: str)

Bases: object

Service for handling input/output operations related to model testing.

Parameters:

• logger (logging.Logger) – Logger for logging messages.

• model_id (str) – Identifier of the model.

• dir (str) – Directory where the model repository is located.

Examples

ios = IOService(
    logger=logger,
    model_id="model_id",
    dir="/path/to/dir",
)
ios.read_metadata()

calculate_directory_size(path: str) → int

Calculate the size of a directory.

Parameters:

path (str) – The path to the directory.

Returns:

The size of the directory.

Return type:

int

calculate_image_size(tag='latest')

Calculate the size of a Docker image.

Parameters:

tag (str, optional) – The tag of the Docker image (default is ‘latest’).

Returns:

The size of the Docker image.

Return type:

str

collect_and_save_json(results, output_file)

Helper function to collect JSON results and save them to a file.

get_conda_env_size()

Get the size of the Conda environment for the model.

Returns:

The size of the Conda environment in megabytes.

Return type:

int

Raises:

Exception – If there is an error calculating the size.

get_directories_sizes(**kwargs)

get_env_sizes(**kwargs)

get_file_requirements()

Get the list of required files for the model.

Returns:

List of required files.

Return type:

List[str]

Raises:

ValueError – If the model type is unsupported.

static get_model_type(model_id: str, repo_path: str) → str

Get the type of the model based on the repository contents.

Parameters:

• model_id (str) – Identifier of the model.

• repo_path (str) – Path to the model repository.

Returns:

The type of the model (e.g., PACK_METHOD_BENTOML, PACK_METHOD_FASTAPI).

Return type:

str

get_output_consistency()

static read_csv_values(file_path)

Reads a CSV file and returns its values excluding the header row.

Parameters:

file_path (str) – The path to the CSV file.

Returns:

A list of rows (each row is a list of values) from the CSV file without the header.

Return type:

list

class ersilia.publish.test.services.io.PackageInstaller(dir, model_id)

Bases: object

ersilia.publish.test.services.runner module

class ersilia.publish.test.services.runner.RunnerService(model_id: str, logger, ios_service: IOService, checkup_service: CheckService, setup_service: SetupService, dir: str, from_dir: str, model_path: Callable, from_github: bool, from_s3: bool, from_dockerhub: bool, version: str, shallow: bool, deep: bool, report_path: str, inspector: InspectService, surface: bool)

Bases: object

Service for running model tests and checks.

Parameters:

• model_id (str) – Identifier of the model.

• logger (logging.Logger) – Logger for logging messages.

• ios_service (IOService) – Instance of IOService for handling input/output operations.

• checkup_service (CheckService) – Instance of CheckService for performing various checks on the model.

• setup_service (SetupService) – Instance of SetupService for setting up the environment and fetching the model repository.

• dir (str) – Directory where the model repository is located.

• from_dir (str) – Directory where the model repository is located.

• model_path (Callable) – Callable to get the model path.

• from_github (bool) – Flag indicating whether to fetch the repository from GitHub.

• from_s3 (bool) – Flag indicating whether to fetch the repository from S3.

• from_dockerhub (bool) – Flag indicating whether to fetch the repository from DockerHub.

• version (str) – Version of the model.

• shallow (bool) – Flag indicating whether to perform shallow checks.

• deep (bool) – Flag indicating whether to perform deep checks.

• report_path (bool) – Flag to specify the path for output as json.

• inspector (InspectService) – Instance of InspectService for inspecting models and their configurations.

• surface (bool) – Flag indicating whether to perform surface checks.

delete(**kwargs)

fetch(**kwargs)

run()

Run the model tests and checks.

run_bash(**kwargs)

run_example(n_samples: int, file_name: str | None = None, simple: bool = True, try_predefined: bool = False)

Generate example input samples for the model.

Parameters:

• n_samples (int) – Number of samples to generate.

• file_name (str, optional) – Name of the file to save the samples.

• simple (bool, optional) – Flag indicating whether to generate simple samples.

• try_predefined (bool, optional) – Flag indicating whether to try predefined samples.

Returns:

List of generated input samples.

Return type:

list

run_model(**kwargs)

ersilia.publish.test.services.setup module

class ersilia.publish.test.services.setup.SetupService(model_id: str, dir: str, from_github: bool, from_s3: bool, logger: Any)

Bases: object

Service for setting up the environment and fetching the model repository.

Parameters:

• model_id (str) – Identifier of the model.

• dir (str) – Directory where the model repository will be cloned.

• from_github (bool) – Flag indicating whether to fetch the repository from GitHub.

• from_s3 (bool) – Flag indicating whether to fetch the repository from S3.

• logger (Any) – Logger for logging messages.

BASE_URL = 'https://github.com/ersilia-os/'

static get_conda_env_location(model_id: str, logger) → str

Get the location of the Conda environment for the model.

Parameters:

• model_id (str) – Identifier of the model.

• logger (logging.Logger) – Logger for logging messages.

Returns:

The location of the Conda environment.

Return type:

str

Raises:

subprocess.CalledProcessError – If the command to list Conda environments returns a non-zero exit code.

get_model()

static run_command(command: str, logger, capture_output: bool = False, shell: bool = True) → str

Run a shell command.

Parameters:

• command (str) – The command to run.

• logger (logging.Logger) – Logger for logging messages.

• capture_output (bool, optional) – Flag indicating whether to capture the command output.

• shell (bool, optional) – Flag indicating whether to run the command in the shell.

Returns:

The output of the command.

Return type:

str

Raises:

subprocess.CalledProcessError – If the command returns a non-zero exit code.

Module contents