CLI and YAML configuration¶

Mira's command-line interface turns a YAML recipe into a sequence of Python function calls. It is useful when we want to generate similar versions of a dataset, e.g. clinical reports from a particular combination of providers, clinical reports with different levels of metadata,...

This guide starts with one ClinicalTrials.gov study from AACT. It then explains how the same configuration model supports the full Open Targets workflow and user-owned recipes.

Inspect the CLI¶

From a source checkout, run Mira through uv:

uv run mira --help

An installed package also provides the mira command directly. The help output lists the packaged recipes:

• aact_clinical_report creates reports and indications for one AACT study;
• clinical_report_generation runs the complete Open Targets workflow;
• aact_llm_extractor is an experimental extraction workflow and is outside the current documentation scope.

Inspect a recipe's fully composed configuration without loading data or running functions:

uv run mira +recipe=aact_clinical_report --cfg job --resolve

Run one AACT study¶

The introductory recipe connects to an AACT PostgreSQL database and selects NCT05188521 from the ctgov schema. It needs the studies, interventions, and conditions tables.

For the default local connection:

uv run mira +recipe=aact_clinical_report

Set database credentials when the local server requires them:

export AACT_USER="your-user"
export AACT_PASSWORD="your-password"
uv run mira +recipe=aact_clinical_report

Choose another study with its NCT identifier:

MIRA_AACT_STUDY_ID=NCT00144209 \
  uv run mira +recipe=aact_clinical_report

The recipe queries each table with the study filter, rather than loading the complete AACT database and filtering afterwards.

Find the outputs¶

Mira writes steps whose names begin with output_ beneath a dated directory:

data/
└── outputs/
    └── YYYY-MM-DD/
        ├── clinical_report.parquet
        └── clinical_indication.parquet

Inspect the generated records with Polars:

from datetime import date
from pathlib import Path

import polars as pl


run_dir = Path("data/outputs") / date.today().isoformat()
reports = pl.read_parquet(run_dir / "clinical_report.parquet")
indications = pl.read_parquet(run_dir / "clinical_indication.parquet")

print(reports.select("id", "clinicalStage", "drugs", "diseases"))
print(
    indications.select(
        "drugName",
        "diseaseName",
        "mappingStatus",
        "clinicalReportIds",
    )
)

With the current AACT record, the default example produces one Phase II Clinical Report. Its two dose-specific baricitinib intervention labels produce two unmapped Clinical Indications for cutaneous lichen planus. AACT is a live database, so source corrections can change these details over time.

uv run mira +recipe=aact_clinical_report \
  datasets.output_path=/path/to/run-outputs

Relative paths are resolved from the directory in which the command starts.

How configuration is composed¶

Four layers determine a run, from general defaults to the most specific value:

1. config.yaml supplies shared database and path defaults.
2. A named file under recipe/ adds inputs and workflow steps.
3. Environment interpolation supplies deployment-specific values such as credentials and the data root.
4. Command-line overrides replace individual values for one run.

Recipes begin with:

# @package _global_

This Hydra directive merges the recipe's inputs and workflow keys into the root configuration beside the shared defaults.

The following command changes the data root only for that invocation:

uv run mira +recipe=clinical_report_generation \
  datasets.data_root=/path/to/mira-data

Use --cfg job --resolve with the same overrides to see the final values without executing the workflow.

Three kinds of references¶

Recipe files contain three similar-looking expressions with different purposes.

Refer to another configuration value¶

Hydra resolves ${...} while composing the configuration:

path: ${datasets.disease_index_path}

This copies the configured value of datasets.disease_index_path.

Read an environment variable¶

The oc.env resolver reads the process environment and can provide a default:

data_root: ${oc.env:MIRA_DATA_DIR,data}

Refer to a runtime result¶

A string beginning with $ refers to an object already held in the current run:

parameters:
  report: $output_clinical_report

Anatomy of a recipe¶

A recipe declares inputs and a workflow:

# @package _global_

inputs:
  studies:
    format: db_table
    db: aact
    select_cols: [nct_id, phase, study_type]
    where_clause: "nct_id = 'NCT05188521'"

workflow:
  transform:
    generate:
      output_clinical_report:
        function: mira.provider.aact.extract_clinical_report
        parameters:
          studies: $studies
          interventions: $interventions
          conditions: $conditions

Inputs¶

Inputs are loaded before any workflow step. Their keys become names in the runtime data store, so $studies refers to the loaded studies input.

Mira supports these input formats:

For a database input:

• the input key is the table name unless table_name is supplied;
• db selects a connection under db_properties and defaults to aact;
• schema overrides the schema from that database connection;
• select_cols can be a list of columns or "*";
• where_clause is SQL text without the WHERE keyword;
• limit adds a database-side row limit.

Use an explicit engine for Spark data:

disease_index:
  format: parquet
  engine: spark
  path: ${datasets.disease_index_path}

Spark starts lazily when a Spark input or $spark_session parameter is encountered.

Steps¶

The transform workflow has three sections, executed in this order:

1. setup
2. generate
3. post_process

Steps run in their YAML order within each section. Every step declares an importable Python function or class method and the keyword arguments passed to it. The step name becomes the runtime key used by later $... references.

When a function returns a Mira dataset object such as ClinicalReport, the runner stores its .df Polars DataFrame. This lets a later step pass $aact_report directly to a function expecting a DataFrame.

Outputs¶

A step beginning with output_ is persisted after all transform sections finish:

The output_ prefix is removed from the filename. Other runtime values are intermediate results and are not written automatically. Rename an intermediate step with the prefix when you want to inspect it as a file.

Base configuration reference¶

All local dataset defaults live beneath datasets.data_root. Setting MIRA_DATA_DIR relocates the entire file layout without editing packaged files.

Use your own recipe¶

Keep deployment-specific recipes outside the installed package. Create a configuration directory containing a recipe subdirectory:

mira-config/
└── recipe/
    └── my_pipeline.yaml

Run it by adding that directory to Hydra's configuration search path:

uv run mira \
  --config-dir /absolute/path/to/mira-config \
  +recipe=my_pipeline

The user-owned recipe still inherits Mira's packaged config.yaml. This approach works after installation and avoids editing site-packages.

Run the complete Open Targets workflow¶

The full recipe is clinical_report_generation.yaml. It expects both databases and the following layout beneath MIRA_DATA_DIR:

<MIRA_DATA_DIR>/
├── inputs/
│   ├── disease/
│   ├── drug_molecule/
│   ├── chembl_mapping.parquet
│   ├── P1-05-Drug_disease.txt
│   ├── medicines-output-medicines-report_en.xlsx
│   ├── pmda_approvals.pdf
│   └── aact_extraction_batch_results/
└── outputs/

After placing the files and setting the AACT and ChEMBL connection variables, inspect the resolved plan:

MIRA_DATA_DIR=/path/to/mira-data \
  uv run mira +recipe=clinical_report_generation --cfg job --resolve

Then run it:

MIRA_DATA_DIR=/path/to/mira-data \
  uv run mira +recipe=clinical_report_generation

This workflow loads AACT and ChEMBL tables, creates reports for all configured providers, maps drugs and diseases, and writes clinical_report.parquet and clinical_indication.parquet.

See Data inputs for authoritative source links, release alignment, required database tables, file contracts, and the planned acquisition boundary.

Common configuration changes¶

Disable drug NER¶

uv run mira +recipe=clinical_report_generation \
  workflow.transform.post_process.output_clinical_report.parameters.ner_extract_drug=false

Change the NER cache¶

uv run mira +recipe=clinical_report_generation \
  workflow.transform.post_process.output_clinical_report.parameters.ner_cache_path=/path/to/ner-cache.parquet

Run without ChEMBL clinical-trial curation¶

Remove the input and pass a null value to the mapping function:

uv run mira +recipe=clinical_report_generation \
  '~inputs.chembl_curation' \
  workflow.transform.post_process.output_clinical_report.parameters.chembl_curation=null

The mapper will continue with OnToma label mapping and the optional drug NER fallback. See Entity mapping for the effect on mapping precedence.

Save an intermediate step¶

Outputs are selected by step name. In a copied recipe, rename a step such as all_reports to output_all_reports, then update later references from $all_reports to $output_all_reports. Mira will persist it as all_reports.parquet and continue passing the same DataFrame to later steps.

Relevant implementation¶

• mira.cli loads inputs, runs sections, and writes outputs.
• mira.utils.pipeline imports callables and resolves runtime references.
• mira.utils.db constructs database queries and returns Polars DataFrames.
• Packaged recipes define the supported workflows.