What can we do?
KMCP adopts a novol metagenomic profiling strategy,
by splitting reference genomes into 10 chunks and mappings reads to these
chunks via fast k-mer matching.
KMCP performs well on both prokaryotic and viral organisms, with higher
sensitivity and specificity than other k-mer-based tools
(check the benchmark).
2. Fast sequence search against large scales of genomic datasets
KMCP can be used for fast sequence search against large scales of genomic dataset
as BIGSI and COBS do.
We reimplemented and modified the Compact Bit-Sliced Signature index (COBS) algorithm,
bringing a smaller index size and much faster searching speed (4x-10x faster than COBS)
(check the tutorial and benchmark).
3. Fast genome similarity estimation
KMCP can be used for fast similarity estimation of assemblies/genomes against known reference genomes.
Genome sketching is a method of utilizing small and approximate summaries of
genomic data for fast searching and comparison.
Mash and Sourmash
provide fast genome distance estimation using MinHash (Mash) or FracMinHash (Scaled MinHash) (Sourmash).
KMCP utilizes multiple k-mer sketches
(Minimizer,
FracMinHash
(previously named Scaled MinHash) and
Closed Syncmers) for genome similarity estimation.
KMCP is 4x-7x faster than Mash/Sourmash
(check the tutorial and benchmark).
Features
- Easy to install
- Easy to use
- Building database is easy and fast
- Fast searching speed
- The index structure is modified from COBS, while KMCP is 4x-10x faster.
- Automatically scales to exploit all available CPU cores.
- Searching time is linearly related to the number of reference genomes.
- Scalable searching. Searching results against multiple databases can be fast merged.
This brings many benefits:
- There's no need to re-built the database with newly added reference genomes.
- HPC cluster could linearly accelerate searching with each computation node hosting a database built with a part of reference genomes.
- Computers with limited main memory would also support searching by building small databases.
- Accurate taxonomic profiling
Documents
Installation
Download executable binaries,
or install using conda:
conda install -c bioconda kmcp
SIMD extensions including AVX512
, AVX2
, SSE2
are sequentially detected and used
in two packages for better searching performance.
- pand,
for accelerating searching on databases constructed with multiple hash functions.
- pospop,
for batch counting matched k-mers in bloom filters.
Commands
subcommand |
function |
compute |
Generate k-mers (sketches) from FASTA/Q sequences |
index |
Construct database from k-mer files |
search |
Search sequences against a database |
merge |
Merge search results from multiple databases |
profile |
Generate taxonomic profile from search results |
utils filter |
Filter search results and find species/assembly-specific queries |
utils merge-regions |
Merge species/assembly-specific regions |
utils unik-info |
Print information of .unik file |
utils index-info |
Print information of index file |
Quickstart
# compute k-mers
kmcp compute -k 21 --split-number 10 --split-overlap 100 \
--in-dir genomes/ --out-dir genomes-k21-n10
# index k-mers
kmcp index --in-dir genomes-k21-n10/ --out-dir genomes.kmcp
# delete temporary files
# rm -rf genomes-k21-n10/
# search
kmcp search --db-dir genomes.kmcp/ test.fa.gz --out-file search.tsv.gz
# profile and binning
kmcp profile search.tsv.gz \
--taxid-map taxid.map \
--taxdump taxdump/ \
--out-prefix search.tsv.gz.k.profile \
--metaphlan-report search.tsv.gz.m.profile \
--cami-report search.tsv.gz.c.profile \
--binning-result search.tsv.gz.binning.gz
Support
Please open an issue to report bugs,
propose new functions or ask for help.
License
MIT License
Acknowledgments
- Zhi-Luo Deng (Helmholtz Centre for Infection Research, Germany)
gave a lot of valuable advice on metagenomic profiling and benchmarking.
- Robert Clausecker (Zuse Institute Berlin, Germany)
wrote the high-performance vectorized positional popcount package
(pospop)
during my development of KMCP,
which greatly accelerated the bit-matrix searching.