• Images
• Videos
• Documents
• Books
• Articles

We don't have any images related to Milvus (vector database) yet.

You can add one yourself here.

We don't have any YouTube videos related to Milvus (vector database) yet.

You can add one yourself here.

We don't have any PDF documents related to Milvus (vector database) yet.

You can add one yourself here.

We don't have any Books related to Milvus (vector database) yet.

You can add one yourself here.

Archived Web Articles Add Article

Zilliz raises $60M, relocates to SF

Aug 24 2022

History

Milvus has been developed by Zilliz since 2017.²

Milvus joined Linux Foundation as an incubation project in January 2020 and became a graduate in June 2021.³ The details about its architecture and possible applications were presented on ACM SIGMOD Conference in 2021⁴

Milvus 2.0, a major redesign of the whole product with a new architecture,⁵ was released in January 2022.

Features

Similarity search

Major similarity search related features that are available in the active 2.4.x Milvus branch:⁶

• In-memory, on-disk and GPU indices,
• Single query, batch query and range query search,
• Support of sparse vectors, binary vectors, JSON and arrays,
• FP32, FP16 and BF16 data types,
• Euclidean distance, inner product distance and cosine distance support for floating-point data,
• Hamming distance and jaccard distance for binary data,
• Support of graph indices (including HNSW), Inverted-lists based indices and a brute-force search.
• Support of vector quantization for lossy input data compression, including product quantization (PQ) and scalar quantization (SQ), that trades stored data size for accuracy,
• Re-ranking.

Milvus similarity search engine relies on heavily-modified forks of third-party open-source similarity search libraries, such as Faiss,⁷⁸ DiskANN⁹¹⁰ and hnswlib.¹¹

Milvus includes optimizations for I/O data layout, specific to graph search indices.¹²

Database

As a database, Milvus provides the following features:¹³

• Column-oriented database
• Four supported data consistency levels, including strong consistency and eventual consistency.¹⁴
• Data sharding
• Streaming data ingestion, which allows to process and ingest data in real-time as it arrives
• A dynamic schema, which allows inserting the data without a predefined schema
• Independent storage and compute layers
• Multi-tenancy scenarios (database-oriented, collection-oriented, partition-oriented)¹⁵
• Memory-mapped data storage
• Role-based access control
• Multi-vector and hybrid search¹⁶

Deployment options

Milvus can be deployed as an embedded database, standalone server, or distributed cluster. Zillis Cloud offers a fully managed version.¹⁷

GPU support

Milvus provides GPU accelerated index building and search using Nvidia CUDA technology¹⁸¹⁹ via Nvidia RAFT library,²⁰ including a recent GPU-based graph indexing algorithm Nvidia CAGRA²¹

Integration

Milvus provides official SDK clients for Java, NodeJS, Python and Go.²² An additional C# SDK client was contributed by Microsoft.²³²⁴ The database can integrate with Prometheus and Grafana for monitoring and alerts, frameworks Haystack²⁵ and LangChain,²⁶ IBM Watsonx,²⁷ and OpenAI models.²⁸²⁹

See also

• Free and open-source software portal

• Nearest neighbor search
• Similarity search
• Vector database
• Vector embedding
• Vector quantization

References

1. "LF AI & Data Foundation Announces Graduation of Milvus Project". June 23, 2021. https://lfaidata.foundation/blog/2021/06/23/lf-ai-data-foundation-announces-graduation-of-milvus-project/ ↩

2. Liao, Ingrid Lunden and Rita (2022-08-24). "Zilliz raises $60M, relocates to SF". TechCrunch. Retrieved 2024-10-21. https://techcrunch.com/2022/08/24/zilliz-the-startup-behind-the-milvus-open-source-vector-database-for-ai-applications-raises-60m-and-relocates-to-sf/ ↩

3. "LF AI & Data Foundation Announces Graduation of Milvus Project". June 23, 2021. https://lfaidata.foundation/blog/2021/06/23/lf-ai-data-foundation-announces-graduation-of-milvus-project/ ↩

4. "Milvus: A Purpose-Built Vector Data Management System". SIGMOD '21: Proceedings of the 2021 International Conference on Management of Data. June 18, 2021. pp. 2614–2627. doi:10.1145/3448016.3457550. ISBN 978-1-4503-8343-1. 978-1-4503-8343-1 ↩

5. Guo, Rentong; Luan, Xiaofan; Xiang, Long; Yan, Xiao; Yi, Xiaomeng; Luo, Jigao; Cheng, Qianya; Xu, Weizhi; Luo, Jiarui; Liu, Frank; Cao, Zhenshan; Qiao, Yanliang; Wang, Ting; Tang, Bo; Xie, Charles (2022). "Manu: A Cloud Native Vector Database Management System". arXiv:2206.13843 [cs.DB]. /wiki/ArXiv_(identifier) ↩

6. "Milvus overview". Retrieved September 23, 2024. https://milvus.io/docs/overview.md ↩

7. "Faiss". GitHub. Retrieved September 23, 2024. https://github.com/facebookresearch/faiss ↩

8. Douze, Matthijs; Guzhva, Alexandr; Deng, Chengqi; Johnson, Jeff; Szilvasy, Gergely; Mazaré, Pierre-Emmanuel; Lomeli, Maria; Hosseini, Lucas; Jégou, Hervé (2024). "The Faiss library". arXiv:2401.08281 [cs.LG]. /wiki/ArXiv_(identifier) ↩

9. "DiskANN library". GitHub. Retrieved September 23, 2024. https://github.com/microsoft/DiskANN ↩

10. Subramanya, Suhas Jayaram; Kadekodi, Rohan; Krishaswamy, Ravishankar; Simhadri, Harsha Vardhan (8 December 2019). "DiskANN: fast accurate billion-point nearest neighbor search on a single node". Proceedings of the 33rd International Conference on Neural Information Processing Systems. Curran Associates Inc.: 13766–13776. https://dl.acm.org/doi/abs/10.5555/3454287.3455520 ↩

11. "Hnswlib - fast approximate nearest neighbor search". GitHub. Retrieved September 23, 2024. https://github.com/nmslib/hnswlib ↩

12. Wang, Mengzhao; Xu, Weizhi; Yi, Xiaomeng; Wu, Songlin; Peng, Zhangyang; Ke, Xiangyu; Gao, Yunjun; Xu, Xiaoliang; Guo, Rentong; Xie, Charles (2024). "Starling: An I/O-Efficient Disk-Resident Graph Index Framework for High-Dimensional Vector Similarity Search on Data Segment". Proceedings of the ACM on Management of Data. 2: 1–27. arXiv:2401.02116. doi:10.1145/3639269. /wiki/ArXiv_(identifier) ↩

13. "Milvus overview". Retrieved September 23, 2024. https://milvus.io/docs/overview.md ↩

14. "Consistency levels in Milvus". Retrieved September 29, 2024. https://milvus.io/docs/consistency.md ↩

15. "Multi-tenancy strategies". Retrieved September 29, 2024. https://milvus.io/docs/multi_tenancy.md ↩

16. "Hybrid Search". Retrieved September 23, 2024. https://milvus.io/docs/multi-vector-search.md ↩

17. "Zilliz cloud". Retrieved October 10, 2024. https://zilliz.com/cloud ↩

18. "What's New In Milvus 2.3 Beta - 10X faster with GPUs". Retrieved September 29, 2024. https://zilliz.com/blog/milvus-2-3-beta-new-features-and-updates ↩

19. "Milvus 2.3 Launches with Support for Nvidia GPUs". 23 March 2023. Retrieved September 29, 2024. https://www.datanami.com/2023/03/23/milvus-2-3-launches-with-support-for-nvidia-gpus/ ↩

20. "NVIDIA RAFT library". GitHub. https://github.com/rapidsai/raft ↩

21. Ootomo, Hiroyuki; Naruse, Akira; Nolet, Corey; Wang, Ray; Feher, Tamas; Wang, Yong (August 2023). "CAGRA: Highly Parallel Graph Construction and Approximate Nearest Neighbor Search for GPUs". arXiv:2308.15136 [cs.DS]. /wiki/ArXiv_(identifier) ↩

22. "Install Milvus Go SDK". Retrieved September 29, 2024. https://milvus.io/docs/install-go.md ↩

23. "Milvus overview". Retrieved September 23, 2024. https://milvus.io/docs/overview.md ↩

24. "Get Started with Milvus Vector DB in .NET". March 6, 2024. Retrieved September 29, 2024. https://devblogs.microsoft.com/dotnet/get-started-milvus-vector-db-dotnet/ ↩

25. "Integration HayStack + Milvus". Retrieved September 23, 2024. https://haystack.deepset.ai/integrations/milvus-document-store ↩

26. "Milvus connector for LangChain". Retrieved September 23, 2024. https://python.langchain.com/docs/integrations/vectorstores/milvus/ ↩

27. "IBM watsonx.data's integrated vector database: unify, prepare, and deliver your data for AI". IBM. April 9, 2024. Retrieved September 29, 2024. https://www.ibm.com/blog/announcement/ibm-watsonx-data-vector-database-ai-ready-data-management/ ↩

28. "Getting started with Milvus and OpenAI". Mar 28, 2023. Retrieved September 23, 2024. https://cookbook.openai.com/examples/vector_databases/milvus/getting_started_with_milvus_and_openai ↩

29. "OpenAI and Milvus simple app". GitHub. Retrieved September 23, 2024. https://github.com/Snaiel/OpenAI-Milvus-QA-Over-Docs ↩