BioMaster logoBioMaster

Benchmark

Workflow coverage and benchmark results.

BioMaster supports diverse omics workflows and was evaluated on 49 tasks spanning 102 bioinformatics tools.

Supported workflows

A broad tool orchestration surface for bioinformatics analysis.

RNA-seq ChIP-seq single-cell RNA-seq spatial transcriptomics Hi-C WGS/WES metagenomics nanopore sequencing DNA methylation Ribo-seq microRNA proteomics

Transcriptomics

Differential expression, expression quantification, splicing analysis, APA, RNA editing, functional enrichment, and single-cell workflows.

Genome and epigenome

WGS/WES variant analysis, ChIP-seq peak calling, motif discovery, DNA methylation analysis, and DNase-seq related workflows.

Spatial and 3D genome

Spatial clustering, cell type annotation, ligand-receptor analysis, Hi-C mapping, pair processing, and contact matrix generation.

Task coverage

BioMaster completed 47 of 49 benchmark tasks end to end.

47

BioMaster

Completed 47 of 49 tasks, corresponding to a 95.9% execution success rate.

24

SingleAgent

Completed 24 of 49 tasks using a single merged knowledge base.

13

AutoBA

Completed 13 of 49 tasks under the same tool-level Execute KB setting.

12

ChatGPT

Completed 12 of 49 tasks without an external knowledge base.

System Completed tasks Success rate Key distinction
BioMaster 47 / 49 95.9% Multi-agent loop with Plan RAG, Execute RAG, Debug Agent, and Check Agent.
SingleAgent 24 / 49 49.0% Single-agent baseline using merged planning and execution knowledge.
AutoBA 13 / 49 26.5% Single-agent pipeline executor with tool-level knowledge.
ChatGPT 12 / 49 24.5% General-purpose LLM baseline without external BioMaster knowledge bases.
Task coverage comparison across systems
Task coverage of BioMaster, AutoBA, and ChatGPT across bioinformatics workflows.

Multi-step workflows

Complex workflows expose the value of recovery and validation.

The benchmark included representative long workflows such as Hi-C, single-cell RNA-seq, spatial transcriptomics, and WGS/WES. BioMaster showed stronger stepwise completion across full workflows.

Execution success across multi-step workflows
Execution success rates across representative multi-step bioinformatics workflows.

Case studies

Representative outputs are comparable to manually implemented pipelines in selected analyses.

Hi-C analysis

BioMaster generated contact matrices visually consistent with manual workflows, with SCC values exceeding 0.99 across chromosomes in the reported evaluation.

Single-cell RNA-seq

BioMaster recovered 12 transcriptionally distinct clusters in PBMC data with UMAP structures consistent with manual analysis.

Metagenomic profiling

Taxonomic profiles and sample separation were highly concordant with manual pipelines in the representative Arabidopsis root microbiome analysis.

Spatial transcriptomics

Domain segmentation aligned with known mouse brain anatomy in the selected Visium case study.

Comparison of BioMaster and manually curated workflow outputs
Comparison of results produced by BioMaster and manually curated pipelines across representative multi-omics workflows.
These comparisons are illustrative evaluations of selected analyses. BioMaster focuses on stabilizing workflow execution and does not replace biological interpretation.

Open models

BioMaster can operate with open-source LLM backbones.

The manuscript evaluates GPT-oss-120B, Qwen3-235B, Qwen3-30B, DeepSeek-R1, and Kimi-K2 across 26 representative tasks.

BioMaster performance across open-source LLMs
Performance across diverse bioinformatics tasks using different open-source large language models.