SQLite#
One can use SQLite as the document store for DocumentArray. It is useful when you want to access a large number Document which can not fit into memory.
Usage#
from docarray import DocumentArray
da = DocumentArray(storage='sqlite') # with default config
da1 = DocumentArray(
storage='sqlite', config={'connection': 'example.db'}
) # with customize config
To reconnect a formerly persisted database, one can need to specify both connection
and table_name
in config
:
from docarray import DocumentArray
da = DocumentArray(
storage='sqlite', config={'connection': 'example.db', 'table_name': 'mine'}
)
da.summary()
Other functions behave the same as in-memory DocumentArray.
Config#
The following configs can be set:
Name | Description | Default |
---|---|---|
connection |
SQLite database filename | a random temp file |
table_name |
SQLite table name | a random name |
serialize_config |
Serialization config of each Document | None |
conn_config |
Connection config pass to sqlite3.connect |
None |
journal_mode |
SQLite Pragma: journal mode | 'DELETE' |
synchronous |
SQLite Pragma: synchronous | 'OFF' |