Weaviate is completely modularized. The Core of Weaviate, without any modules attached, is a pure vector-native database and search engine. Data is stored as vectors, and these vectors are searchable by the provide vector index algorithm. Without any modules attached, Weaviate does not know how to vectorize data, i.e. how to calculate the vectors from a data item. Depending on the type of data you want to store and search (text, images, etc), and depending on the use case (like search, question answering, etc, depending on language, classification, ML model, training set, etc), you can choose and attach a module that best fits your use case.
Unless you specify a default vectorization module in Weaviate’s configuration, you’ll need to specify which vectorization module is used per class you add to the data schema (or you need to enter a vector for each data point you add manually). Set the default with the environment variable
DEFAULT_VECTORIZER_MODULE in the docker-compose configuration file, for example:
services: weaviate: environment: DEFAULT_VECTORIZER_MODULE: text2vec-contextionary
Text vectorizer Contextionary
One vectorizer that is provided is
text2vec-contextionary is a text vectorizer that gives context to the textual data using a language model trained using fasttext on Wiki data and CommonCrawl. More information can be found here. The contextionary is a Weighted Mean of Word Embeddings (WMOWE) vectorizer module which works with popular models such as fastText and GloVe.
Text vectorizer Transformers
Another type of text vectorization is possible with the
Note: at the moment, text vectorization modules cannot be combined in a single setup. This means that you can either enable the
text2vec-transformers or no text vectorization module.
Check here how you can create and use your own modules.
If you can’t find the answer to your question here, please look at the: