Table of documentation contents

/v1/batch

With batch you can upload a lot of data objects in bulk. This saves time compared to a lot of single request.

Batch data objects

For sending data objects to Weaviate in bulk.

Method and URL

POST /v1/batch/objects

Parameters

The body requires the following field:

nametyperequireddescription
objectslist of data objectsyesa list of data objects, which correspond to the data object body

Example request

  import weaviate

client = weaviate.Client("http://localhost:8080")

first_object_props = {
    "name": "Jane Doe",
    "writesFor": [{
        "beacon": "weaviate://localhost/f81bfe5e-16ba-4615-a516-46c2ae2e5a80"
    }]
}
second_object_props = {
    "name": "John Doe",
    "writesFor": [{
        "beacon": "weaviate://localhost/f81bfe5e-16ba-4615-a516-46c2ae2e5a80"
    }]
}

batch = weaviate.ObjectsBatchRequest()
batch.add(first_object_props, "Author", "36ddd591-2dee-4e7e-a3cc-eb86d30a4303")
batch.add(second_object_props, "Author", "36ddd591-2dee-4e7e-a3cc-eb86d30a4304")
client.batch.create(batch)

Batch references

For batching cross-references between data objects in bulk.

Method and URL

POST /v1/batch/references

Parameters

The body of the data object for a new object is a list of objects containing:

nametyperequireddescription
frombeaconyesThe beacon, in the form of weaviate://{host}/{Classname}/{id}/{cref_property_name}
tobeaconyesThe beacon, in the form of weaviate://{host}/{id}

Example request

  import weaviate

client = weaviate.Client("http://localhost:8080")

batch = weaviate.ReferenceBatchRequest()

# Format:
# batch.add(<from_entity_uuid>, <from_class_name>, <from_property_name>, <to_entity_uuid>)

batch.add("36ddd591-2dee-4e7e-a3cc-eb86d30a4303", "Author", "wroteArticles", "6bb06a43-e7f0-393e-9ecf-3c0f4e129064")
batch.add("36ddd591-2dee-4e7e-a3cc-eb86d30a4303", "Author", "wroteArticles", "b72912b9-e5d7-304e-a654-66dc63c55b32")
batch.add("36ddd591-2dee-4e7e-a3cc-eb86d30a4304", "Author", "wroteArticles", "b72912b9-e5d7-304e-a654-66dc63c55b32")

client.batch.create(batch)

Error handling

When sending a batch request to your Weaviate instance, it could be the case that an error occurs. This can be caused by several reasons, for example that the connection to Weaviate is lost or that there is a mistake in a single data object that you are trying to add.

Only errors that are caused by sending the whole batch are shown when sending a batch. Errors on individual batch items will not be shown on creating and sending a batch request. Thus, sending a batch and getting no errors does not guarantee that each batch item is added/created. Sending a batch can lead to a successful batch creation but unsuccessful per batch item creation. An example of an error on an individual data object that might be unnoticed by sending a batch request without checking the individual results is: Adding an object to the batch that is in conflict with the schema (for example a non existing class name).

The following Python code can be used to handle errors on individual data objects in the batch.

import weaviate

client = weaviate.Client("http://localhost:8080")

object_to_add = {
    "name": "Jane Doe",
    "writesFor": [{
        "beacon": "weaviate://localhost/f81bfe5e-16ba-4615-a516-46c2ae2e5a80"
    }]
}

batch = weaviate.ObjectsBatchRequest()
batch.add(object_to_add, "Author", "36ddd591-2dee-4e7e-a3cc-eb86d30a4303")
results = client.batch.create(batch)

if results is not None:
    for result in results:
        if 'result' in result and 'errors' in result['result'] and  'error' in result['result']['errors']:
            for message in result['result']['errors']['error']:
                print(message['message'])

This can also be applied to adding references in batch. Note that sending batches, especially references, skips some validation on object and reference level. Adding this validation on single data objects like above makes it less likely for errors to pass without discovering.

More Resources

If you can’t find the answer to your question here, please look at the:

  1. Frequently Asked Questions. Or,
  2. Knowledge base of old issues. Or,
  3. For questions: Stackoverflow. Or,
  4. For issues: Github. Or,
  5. Ask your question in the Slack channel: Slack.
Tags
  • RESTful API
  • references
  • batching