Making Custom Databases

EsViritu ships with a curated database of human, animal and plant virus genomes, but you can build and use a custom database if you need to target a different set of viruses (e.g. environmental viruses, phages, or a custom subset of NCBI sequences). One use case is improving consensus genome assembly by using lots of closely related genomes for a species of interest. See this example for RSV.

Two helper commands are provided:

• esv_create_taxonomy — Takes a table of accessions and generates a fully annotated EsViritu-style metadata TSV using NCBI taxonomy.
• esv_combine_tax — Merges two or more metadata TSVs into a single file, deduplicating records.

A custom database requires three files, both pointing to the same set of accessions:

1. A FASTA file of virus genome sequences (e.g. downloaded from NCBI).
2. virus_pathogen_database.fna
3. a minimap2 index (.mmi) with short-read settings.
4. virus_pathogen_database.mmi
5. via: minimap2 -x sr -d virus_pathogen_database.mmi virus_pathogen_database.fna
6. A metadata TSV produced by esv_create_taxonomy (and optionally combined with esv_combine_tax).
7. virus_pathogen_database.all_metadata.tsv

The directory with the three files are then passed to EsViritu via --db, respectively.

Making an EsViritu-Style Database with NCBI Records

Install pytaxonkit

esv_create_taxonomy requires pytaxonkit, which is available on Bioconda:

conda install bioconda::pytaxonkit">=0.10"

Download the NCBI accession-to-TaxID mapping

wget https://ftp.ncbi.nih.gov/pub/taxonomy/accession2taxid/nucl_gb.accession2taxid.gz

zcat nucl_gb.accession2taxid.gz | cut -f 2,3 | gzip -c > acc2taxid.tsv.gz

Estimated size: ~900 MB (compressed).

Download the NCBI taxonomy dump

mkdir taxdump && cd taxdump

wget https://ftp.ncbi.nlm.nih.gov/pub/taxonomy/taxdump.tar.gz

tar -xvf taxdump.tar.gz

Estimated size: ~3 GB (uncompressed).

Step 1: Prepare your accession table

Create a tab-delimited input TSV with at minimum these required columns:

This table is typically exported from the NCBI Virus web interface, but could be generated from NCBI entrez.

Step 2: Run esv_create_taxonomy

esv_create_taxonomy \
    -i my_accessions.tsv \
    -o my_virus_metadata.tsv \
    -a /path/to/acc2taxid.tsv.gz \
    -t /path/to/taxdump

Arguments

--subspecies-label options

By default (-s not specified), esv_create_taxonomy uses pytaxonkit to assign the subspecies/strain rank from NCBI taxonomy. This is the recommended option.

Alternatively, you can override the subspecies field using an existing column from your input:

Output columns

The output TSV contains the following columns (plus any extra columns from the input):

NOTE: There will be several extra columns from pytaxonkit that are not removed by default as they do not effect EsViritu processing.

Step 3 (optional): Combine multiple metadata tables with esv_combine_tax

If you have built metadata tables from multiple input TSVs (e.g. different virus groups or NCBI datasets exports), combine them into a single file:

esv_combine_tax standard_DB/virus_pathogen_database.all_metadata.tsv table1.tsv table2.tsv -o custom_DB/virus_pathogen_database.all_metadata.tsv

• Duplicate rows (identical across all columns) are removed.
• Tables must all have the required columns (Accession, description, Name, Segment, kingdom, phylum, tclass, order, family, genus, species, subspecies, Length, TaxID, Assembly, Asm_length). Tables missing any of these will cause an error.
• Extra columns beyond the required set are preserved.

Arguments

Step 4: Combine FASTA sequences and Index for minimap2

Combine all the FASTA files that correspond to records in your metadata table.

cat standard_DB/virus_pathogen_database.fna custom_seqs.fna | seqkit rmdup | seqkit seq > custom_DB/virus_pathogen_database.fna

minimap2 -x sr -d custom_DB/virus_pathogen_database.mmi custom_DB/virus_pathogen_database.fna

Step 5: Run EsViritu with the custom database

Pass your custom FASTA and metadata files via --db:

EsViritu \
    -r /path/to/reads1.fastq /path/to/reads2.fastq \
    -s my_sample \
    -o my_output_dir \
    --db /path/to/custom_DB