Record managers
Record Managers Overview
Record Managers are essential components in vector database operations, designed to track and manage document writes, updates, and deletions. They play a crucial role in maintaining the consistency and efficiency of vector stores, especially when dealing with large document collections that change over time.
Available Components
MySQL Record Manager
Record manager implementation for MySQL databases
Postgres Record Manager
Record manager implementation for PostgreSQL databases
SQLite Record Manager
Record manager implementation for SQLite databases
Use Cases and Benefits
Record Managers are beneficial in the following scenarios:- Incremental Updates: They allow for efficient updating of vector stores by identifying which documents have changed since the last update, reducing processing time and resource usage.
- Version Control: Keep track of document versions and when they were last updated or inserted into the vector database.
- Cleanup Operations: Facilitate the removal of outdated or deleted documents from vector stores, maintaining data relevance and optimizing storage.
- Consistency Management: Ensure consistency between source documents and their representations in vector databases.
- Large-Scale Document Processing: Support batch operations for improved performance when dealing with large document collections.
- Namespace Organization: Allow for the organization of records into namespaces, useful for managing multiple projects or datasets within the same database.
- Time-Based Queries: Enable retrieval of documents based on specific time ranges, useful for time-sensitive applications or auditing purposes.
Previous
MySQL Record ManagerThe MySQL Record Manager node is used to keep track of document writes into vector databases. It utilizes MySQL to store and manage records of document insertions, updates, and deletions. This node is particularly useful for maintaining the state of documents across different runs or sessions, enabling incremental updates and efficient management of large document collections.
Next