README
¶
Weaviate 
Overview
Weaviate is a cloud-native, open source vector database that is robust, fast, and scalable.
To get started quickly, have a look at one of these pages:
- Quickstart tutorial To see Weaviate in action
- Contributor guide To contribute to this project
For more details, read through the summary on this page or see the system documentation.
[!NOTE] Help us improve your experience by sharing your feedback, ideas and thoughts: Fill out our Community Experience Survey, preferably by June 14th, 2024.
Why Weaviate?
Weaviate uses state-of-the-art machine learning (ML) models to turn your data - text, images, and more - into a searchable vector database.
Here are some highlights.
Speed
Weaviate is fast. The core engine can run a 10-NN nearest neighbor search on millions of objects in milliseconds. See benchmarks.
Flexibility
Weaviate can vectorize your data at import time. Or, if you have already vectorized your data, you can upload your own vectors instead.
Modules give you the flexibility to tune Weaviate for your needs. More than two dozen modules connect you to popular services and model hubs such as OpenAI, Cohere, VoyageAI and HuggingFace. Use custom modules to work with your own models or third party services.
Production-readiness
Weaviate is built with scaling, replication, and security in mind so you can go smoothly from rapid prototyping to production at scale.
Beyond search
Weaviate doesn't just power lightning-fast vector searches. Other superpowers include recommendation, summarization, and integration with neural search frameworks.
Who uses Weaviate?
-
Software Engineers
- Weaviate is an ML-first database engine
- Out-of-the-box modules for AI-powered searches, automatic classification, and LLM integration
- Full CRUD support
- Cloud-native, distributed system that runs well on Kubernetes
- Scales with your workloads
-
Data Engineers
- Weaviate is a fast, flexible vector database
- Use your own ML model or third party models
- Run locally or with an inference service
-
Data Scientists
- Seamless handover of Machine Learning models to engineers and MLOps
- Deploy and maintain your ML models in production reliably and efficiently
- Easily package custom trained models
What can you build with Weaviate?
A Weaviate vector database can search text, images, or a combination of both. Fast vector search provides a foundation for chatbots, recommendation systems, summarizers, and classification systems.
Here are some examples that show how Weaviate integrates with other AI and ML tools:
Use Weaviate with third party embeddings
Use Weaviate as a document store
Use Weaviate as a memory backend
Demos
These demos are working applications that highlight some of Weaviate's capabilities. Their source code is available on GitHub.
How can you connect to Weaviate?
Weaviate exposes a GraphQL API and a REST API. Starting in v1.23, a new gRPC API provides even faster access to your data.
Weaviate provides client libraries for several popular languages:
There are also community supported libraries for additional languages.
Where can You learn more?
Free, self-paced courses in Weaviate Academy teach you how to use Weaviate. The Tutorials repo has code for example projects. The Recipes repo has even more project code to get you started.
The Weaviate blog and podcast regularly post stories on Weaviate and AI.
Here are some popular posts:
Blogs
- What to expect from Weaviate in 2023
- Why is vector search so fast?
- Cohere Multilingual ML Models with Weaviate
- Vamana vs. HNSW - Exploring ANN algorithms Part 1
- HNSW+PQ - Exploring ANN algorithms Part 2.1
- The Tile Encoder - Exploring ANN algorithms Part 2.2
- How GPT4.0 and other Large Language Models Work
- Monitoring Weaviate in Production
- The ChatGPT Retrieval Plugin - Weaviate as a Long-term Memory Store for Generative AI
- Combining LangChain and Weaviate
- How to build an Image Search Application with Weaviate
- Cohere Multilingual ML Models with Weaviate
- Building Multimodal AI in TypeScript
- Giving Auto-GPT Long-Term Memory with Weaviate
Podcasts
Other reading
- Weaviate is an open-source search engine powered by ML, vectors, graphs, and GraphQL (ZDNet)
- Weaviate, an ANN Database with CRUD support (DB-Engines.com)
- A sub-50ms neural search with DistilBERT and Weaviate (Towards Datascience)
- Getting Started with Weaviate Python Library (Towards Datascience)
Join our community!
At Weaviate, we love to connect with our community. We love helping amazing people build cool things. And, we love to talk with you about you passion for vector databases and AI.
Please reach out, and join our community:
To keep up to date with new releases, meetup news, and more, subscribe to our newsletter
Directories
¶
| Path | Synopsis |
|---|---|
|
adapters
|
|
|
handlers/graphql
Package graphql provides the graphql endpoint for Weaviate
|
Package graphql provides the graphql endpoint for Weaviate |
|
handlers/graphql/descriptions
Package descriptions provides the descriptions as used by the graphql endpoint for Weaviate
|
Package descriptions provides the descriptions as used by the graphql endpoint for Weaviate |
|
handlers/graphql/graphiql
Based on `graphiql.go` from https://github.com/graphql-go/handler only made RenderGraphiQL a public function.
|
Based on `graphiql.go` from https://github.com/graphql-go/handler only made RenderGraphiQL a public function. |
|
handlers/graphql/local/aggregate
Package aggregate provides the local aggregate graphql endpoint for Weaviate
|
Package aggregate provides the local aggregate graphql endpoint for Weaviate |
|
handlers/graphql/local/common_filters
Package common_filters provides the filters for the graphql endpoint for Weaviate
|
Package common_filters provides the filters for the graphql endpoint for Weaviate |
|
handlers/graphql/utils
Package utils provides utility methods and classes to support the graphql endpoint for Weaviate
|
Package utils provides utility methods and classes to support the graphql endpoint for Weaviate |
|
handlers/rest
Package rest with all rest API functions.
|
Package rest with all rest API functions. |
|
repos/db
Some standard accessors for the shard struct.
|
Some standard accessors for the shard struct. |
|
repos/db/clusterintegrationtest
clusterintegrationtest acts as a test package to provide a component test spanning multiple parts of the application, including everything that's required for a distributed setup.
|
clusterintegrationtest acts as a test package to provide a component test spanning multiple parts of the application, including everything that's required for a distributed setup. |
|
repos/db/lsmkv/entities
ent contains common types used throughout various lsmkv (sub-)packages
|
ent contains common types used throughout various lsmkv (sub-)packages |
|
repos/db/roaringset
Package roaringset contains all the LSM business logic that is unique to the "RoaringSet" strategy
|
Package roaringset contains all the LSM business logic that is unique to the "RoaringSet" strategy |
|
repos/db/vector/hnsw/distancer/asm
asm only has amd64 specific implementations at the moment
|
asm only has amd64 specific implementations at the moment |
|
cmd
|
|
|
Code generated by go generate; DO NOT EDIT.
|
Code generated by go generate; DO NOT EDIT. |
|
entities
|
|
|
moduletools
moduletools contains helpers that are passed to modules as part of their capability methods
|
moduletools contains helpers that are passed to modules as part of their capability methods |
|
grpc
|
|
|
modules
|
|
|
text2vec-contextionary
modcontextionary concentrates some of the code that relates to the contextionary module, this must be extracted when Weaviate becomes modular.
|
modcontextionary concentrates some of the code that relates to the contextionary module, this must be extracted when Weaviate becomes modular. |
|
test
|
|
|
benchmark_bm25
Module
|
|
|
tools
|
|
|
usecases
|
|
|
byteops
Package byteops provides helper functions to (un-) marshal objects from or into a buffer
|
Package byteops provides helper functions to (un-) marshal objects from or into a buffer |
|
objects
package objects provides managers for all kind-related items, such as objects.
|
package objects provides managers for all kind-related items, such as objects. |
|
schema
Package migrate provides a simple composer tool, which implements the Migrator interface and can take in any number of migrators which themselves have to implement the interface
|
Package migrate provides a simple composer tool, which implements the Migrator interface and can take in any number of migrators which themselves have to implement the interface |