ersilia.io package

Subpackages

• ersilia.io.readers package
  • Submodules
  • ersilia.io.readers.file module
    • BaseJsonFile
      • BaseJsonFile.is_single_input()
      • BaseJsonFile.read_input_json()
    • BaseTabularFile
      • BaseTabularFile.get_delimiter()
      • BaseTabularFile.get_string_delimiter()
      • BaseTabularFile.has_header()
      • BaseTabularFile.is_flattened()
      • BaseTabularFile.is_input()
      • BaseTabularFile.is_key()
      • BaseTabularFile.is_single_input()
      • BaseTabularFile.read_input_columns()
      • BaseTabularFile.resolve_columns()
    • BatchCacher
      • BatchCacher.get_cached_files()
      • BatchCacher.get_cached_input_files()
      • BatchCacher.get_cached_output_files()
      • BatchCacher.name_cached_output_files()
    • FileTyper
      • FileTyper.get_extension()
      • FileTyper.is_csv()
      • FileTyper.is_hdf5()
      • FileTyper.is_json()
      • FileTyper.is_tabular()
      • FileTyper.is_tsv()
      • FileTyper.is_valid_input_file()
      • FileTyper.is_valid_output_file()
    • JsonFileReader
      • JsonFileReader.read()
    • JsonFileShapeStandardizer
      • JsonFileShapeStandardizer.standardize()
    • StandardJsonFileReader
      • StandardJsonFileReader.is_worth_splitting()
      • StandardJsonFileReader.read()
      • StandardJsonFileReader.split_in_cache()
    • StandardTabularFileReader
      • StandardTabularFileReader.get_delimiter()
      • StandardTabularFileReader.is_worth_splitting()
      • StandardTabularFileReader.read()
      • StandardTabularFileReader.read_header()
      • StandardTabularFileReader.split_in_cache()
    • TabularFileReader
      • TabularFileReader.read()
    • TabularFileShapeStandardizer
      • TabularFileShapeStandardizer.standardize()
  • ersilia.io.readers.pyinput module
    • PyInputReader
      • PyInputReader.is_single_input()
      • PyInputReader.read()
  • Module contents
• ersilia.io.types package
  • Subpackages
    • ersilia.io.types.examples package
      • Submodules
      • ersilia.io.types.examples.compound module
      • ersilia.io.types.examples.text module
      • Module contents
  • Submodules
  • ersilia.io.types.compound module
    • IO
      • IO.column_delimiter()
      • IO.example()
      • IO.example_fixed()
      • IO.is_input()
      • IO.is_key()
      • IO.parse()
      • IO.setup()
      • IO.string_delimiter()
      • IO.test()
  • ersilia.io.types.naive module
    • IO
      • IO.parse()
  • ersilia.io.types.protein module
    • IO
      • IO.example()
      • IO.parse()
      • IO.string_delimiter()
  • ersilia.io.types.text module
    • IO
      • IO.column_delimiter()
      • IO.example()
      • IO.is_input()
      • IO.is_key()
      • IO.parse()
      • IO.setup()
      • IO.string_delimiter()
      • IO.test()
  • Module contents

Submodules

ersilia.io.annotated module

class ersilia.io.annotated.AnnotatedDataTyper(data, annotated_type, annotated_shape)

Bases: object

Class to determine the type of annotated data.

Parameters:

• data (any) – The annotated data.

• annotated_type (str) – The type of the annotated data (e.g., “String”, “Float”, “Integer”).

• annotated_shape (str) – The shape of the annotated data (e.g., “Single”, “List”, “Flexible List”).

get_type()

Get the type of the annotated data.

Returns:

Dictionary containing the type and shape of the annotated data, or None if the type is not recognized.

Return type:

dict or None

ersilia.io.dataframe module

class ersilia.io.dataframe.Dataframe(keys=None, inputs=None, texts=None, values=None, features=None)

Bases: object

Class to handle data in a dataframe-like structure.

Parameters:

• keys (list, optional) – List of keys.

• inputs (list, optional) – List of inputs.

• texts (list, optional) – List of texts.

• values (list, optional) – List of values.

• features (list, optional) – List of features.

from_csv(filename)

Load data from a CSV file.

Parameters:

filename (str) – Path to the CSV file.

Return type:

None

iterrows()

Iterate over the rows of the dataframe.

Yields:

dict – Dictionary representing a row in the dataframe.

ersilia.io.input module

class ersilia.io.input.BaseIOGetter(config_json=None)

Bases: ErsiliaBase

Base class to get IO handlers based on model or specifications.

Parameters:

config_json (dict, optional) – Configuration JSON.

get(model_id=None, input_type=None, input_shape=None)

Get the IO handler based on model or specifications.

Parameters:

• model_id (str, optional) – Model identifier.

• input_type (str, optional) – Input type.

• input_shape (str, optional) – Input shape.

Returns:

IO handler object.

Return type:

IO

shape(model_id)

Get the input shape from the model card.

Parameters:

model_id (str) – Model identifier.

Returns:

Input shape object.

Return type:

InputShape

class ersilia.io.input.ExampleGenerator(model_id, config_json=None)

Bases: ErsiliaBase

Class to generate examples for a model.

Parameters:

• model_id (str) – Model identifier.

• config_json (dict, optional) – Configuration JSON.

check_model_id(model_id)

Check if the model ID is valid.

Parameters:

model_id (str) – Model identifier.

Return type:

None

Raises:

NullModelIdentifierError – If the model ID is None.

example(n_samples, file_name, mode)

Generate example data.

Parameters:

• n_samples (int) – Number of samples to generate.

• file_name (str) – File name to save the examples.

• mode (str) – Mode for generating examples: “random”, “deterministic”, or “curated” (the model’s own example inputs; “predefined” is accepted as an alias).

Returns:

List of example data or file content if saved to file.

Return type:

list or str

fixed_example(n_samples, file_name)

Generate deterministic example data. Samples same types of examples for each any sampling.

Parameters:

• n_samples (int) – Number of samples to generate.

• file_name (str) – File name to save the examples.

Return type:

None

predefined_example(file_name)

Get predefined example data.

Parameters:

file_name (str) – File name to save the examples.

Returns:

True if predefined examples are available, False otherwise.

Return type:

bool

random_example(n_samples, file_name)

Generate random example data.

Parameters:

• n_samples (int) – Number of samples to generate.

• file_name (str) – File name to save the examples.

Return type:

None

test()

Get test examples.

Returns:

List of test examples.

Return type:

list

class ersilia.io.input.GenericInputAdapter(model_id=None, input_type=None, input_shape=None, config_json=None)

Bases: object

Class to adapt generic inputs to a standard format.

This class uses the _GenericAdapter to handle different types of inputs and convert them into a standard format that can be processed in batches or one by one.

Parameters:

• model_id (str, optional) – Model identifier.

• input_type (str, optional) – Input type.

• input_shape (str, optional) – Input shape.

• config_json (dict, optional) – Configuration JSON.

adapt(inp, batch_size)

Adapt the input data in batches.

Parameters:

• inp (any) – The input data.

• batch_size (int) – Size of each batch.

Yields:

list – List of adapted data in batches.

adapt_one_by_one(inp)

Adapt the input data one by one.

Parameters:

inp (any) – The input data.

Yields:

dict – Adapted data.

batch_iter(data, batch_size)

Yield batches of data.

Parameters:

• data (iterable) – The data to be batched.

• batch_size (int) – The size of each batch.

Yields:

iterable – A batch of data.

ersilia.io.output module

class ersilia.io.output.DataFrame(data: list, columns: list, dtype: any, dim: int)

Bases: object

A class used to represent a DataFrame.

data

The data of the DataFrame.

Type:

list

columns

The column names of the DataFrame.

Type:

list

dtype

Data type for the output part of the data

Type:

any

dim

The number of dimensions of the data.

Type:

int

decompose()

Decomposes the DataFrame into its components.

write_hdf5(file_name)

Writes the DataFrame to an HDF5 file.

write_text(file_name, delimiter=None)

Writes the DataFrame to a text file.

write(file_name, delimiter=None)

Writes the DataFrame to a file, determining the format based on the file extension.

decompose() → dict

Decomposes the DataFrame into its components.

Returns:

A dictionary containing keys, inputs, features, and values.

Return type:

dict

write(file_name: str, delimiter: str = None, append: bool = False, check_unprocessable: bool = True)

Writes the DataFrame to a file, determining the format based on the file extension.

Parameters:

• file_name (str) – The name of the file to write to.

• delimiter (str, optional) – The delimiter to use in the text file (default is None).

• append (bool, optional) – If True, append rows to a file previously written by this method.

• check_unprocessable (bool, optional) – If True, raise when the data is a single unprocessable input. Disable when writing one chunk of a larger output.

write_hdf5(file_name: str, append: bool = False)

Writes the DataFrame to an HDF5 file.

Parameters:

• file_name (str) – The name of the file to write to.

• append (bool, optional) – If True, append rows to an existing file written by this method.

write_text(file_name: str, delimiter: str = None, chunksize: int = 50000, append: bool = False)

Writes the DataFrame to a text file, wrapping any string-valued field in quotes, except for the first and second columns, which are never quoted. If append is True, rows are appended to an existing file without a header.

class ersilia.io.output.DictlistDataframeConverter(config_json: dict)

Bases: GenericOutputAdapter

A class used to convert between dictlist and DataFrame.

config_json

Configuration settings in JSON format.

Type:

dict

dictlist2dataframe(dl, model_id, api_name)

Converts a dictlist to a DataFrame.

dataframe2dictlist(df, model_id, api_name)

Converts a DataFrame to a dictlist.

dataframe2dictlist(df: DataFrame, model_id: str, api_name: str) → list

Converts a DataFrame to a dictlist.

Parameters:

• df (DataFrame) – The DataFrame to convert.

• model_id (str) – The model identifier.

• api_name (str) – The API name.

Returns:

The converted dictlist.

Return type:

list

dictlist2dataframe(dl: dict, model_id: str, api_name: str) → Dataframe

Converts a dictlist to a DataFrame.

Parameters:

• dl (dict) – The dictlist to convert.

• model_id (str) – The model identifier.

• api_name (str) – The API name.

Returns:

The converted DataFrame.

Return type:

Dataframe

class ersilia.io.output.GenericOutputAdapter(model_id: str = None, columns_info: dict = None, config_json: dict = None)

Bases: ResponseRefactor

A class used to adapt generic outputs.

model_id

The model identifier.

Type:

str

config_json

Configuration settings in JSON format.

Type:

dict

adapt(result, output, model_id=None, api_name=None)

Adapts the output based on the result and model.

adapt(result: str, output: str, model_id: str = None, api_name: str = None) → dict

Adapts the output based on the result and model.

Parameters:

• result (str) – The result to adapt, stringyfied JSON.

• output (str) – The output file name.

• model_id (str, optional) – The model identifier (default is None).

• api_name (str, optional) – The API name (default is None).

Returns:

The adapted result.

Return type:

dict

merge(subfiles: list, output_file: str)

Merges the subfiles into a single output file.

Parameters:

• subfiles (list) – The list of subfiles to merge.

• output_file (str) – The name of the output file.

write_batch(inputs: list, results: list, output: str, append: bool)

Writes one batch of merged API results to a CSV, TSV or HDF5 file.

Produces the same file as standardizing the batch and calling write_chunk, but builds the rows in one pass and converts float outputs as a single matrix instead of cell by cell.

Parameters:

• inputs (list) – Input records ({“key”: …, “input”: …}), one per result.

• results (list) – One dict per input: the input first, then the output values in column order.

• output (str) – The output file name.

• append (bool) – If True, append to the file written by a previous batch.

write_chunk(result: list, output: str, append: bool)

Writes one chunk of standardized results to a CSV, TSV or HDF5 file.

Used to stream large runs to disk batch by batch instead of holding the full output in memory.

Parameters:

• result (list) – Standardized results ({“input”: …, “output”: …} dicts).

• output (str) – The output file name.

• append (bool) – If True, append to the file written by a previous chunk.

class ersilia.io.output.ResponseRefactor(config_json: dict)

Bases: ErsiliaBase

A class used to refactor API responses.

config_json

Configuration settings in JSON format.

Type:

dict

refactor_response(result)

Refactors the API response.

refactor_response(result: dict) → dict

Refactors the API response.

Parameters:

result (dict) – The API response to refactor.

Returns:

The refactored API response.

Return type:

dict

class ersilia.io.output.TabularOutputStacker(file_names: list)

Bases: object

A class used to stack tabular outputs.

file_names

The list of file names to stack.

Type:

list

stack(output)

Stacks the files into a single output file.

stack(output: str)

Stacks the files into a single output file.

Parameters:

output (str) – The name of the output file.

stack_hdf5(output: str)

Stacks HDF5 files into a single output file.

Parameters:

output (str) – The name of the output file.

stack_text(output: str)

Stacks text files into a single output file.

Parameters:

output (str) – The name of the output file.

ersilia.io.output_logger module

class ersilia.io.output_logger.TabularResultLogger

Bases: object

Class to log tabular results.

This class handles logging of results from tabular files such as CSV and TSV.

Parameters:

None

summary(output)

Summarize results statistics This class produces a summary of the results, counting inputs, output dimensions, etc. :param output: Path to the output file. :type output: str

Returns:

data – Summary of the results.

Return type:

dict

ersilia.io.pure module

class ersilia.io.pure.PureDataTyper(data: any, model_id: str = None, config_json: str = None)

Bases: ErsiliaBase

A class used to determine the type of data provided.

Parameters:

• data (any) – The data to be typed.

• model_id (str, optional) – The model identifier, by default None.

• config_json (str, optional) – Path to the configuration JSON file, by default None.

Examples

>>> data_typer = PureDataTyper(data=[1, 2, 3])
>>> data_typer.get_type()
{'type': 'numeric_array', 'shape': (3,)}

get_type() → dict

Determine the type of the data.

Returns:

A dictionary containing the type and shape of the data.

Return type:

dict

get_type_from_metadata() → dict

Get the type of data from the model metadata.

Returns:

A dictionary containing the type and shape of the data if available, otherwise None.

Return type:

dict

ersilia.io.shape module

class ersilia.io.shape.InputShape(input_shape: str = None)

Bases: object

A class used to determine the input shape.

Parameters:

input_shape (str or None) – The input shape type. Can only be ‘single’. If None, defaults to ‘Single’.

Examples

>>> shape = InputShape("list")
>>> shape.get().name
'List'

get()

Get the input shape object.

Returns:

The input shape object.

Return type:

object

class ersilia.io.shape.InputShapeSingle

Bases: object

A class representing a single input shape.

Module contents