Record managers
MySQL Record Manager
The MySQL Record Manager node is used to keep track of document writes into vector databases. It utilizes MySQL to store and manage records of document insertions, updates, and deletions. This node is particularly useful for maintaining the state of documents across different runs or sessions, enabling incremental updates and efficient management of large document collections.
Node Information
- Name: MySQLRecordManager
- Type: MySQL RecordManager
- Version: 1.0
- Category: Record Manager
Parameters
Input Parameters
-
Host (string, required)
- The hostname of the MySQL server.
-
Database (string, required)
- The name of the MySQL database to use.
-
Port (number, optional)
- The port number for the MySQL server. Default is 3306.
-
Additional Connection Configuration (JSON, optional)
- Additional MySQL connection options in JSON format.
-
Table Name (string, optional)
- The name of the table to use for storing records. Default is ‘upsertion_records’.
-
Namespace (string, optional)
- A namespace for the records. If not specified, the chatflow ID will be used.
-
Cleanup (options, optional)
-
Specifies the cleanup method for old content:
- None: No cleanup of old content.
- Incremental: Delete previous versions if content has changed.
- Full: Same as incremental, but also deletes records if source documents are removed.
-
Specifies the cleanup method for old content:
-
SourceId Key (string, optional)
- The key used to identify the true source of a document. Default is ‘source’.
Credential Parameters
-
Connect Credential (credential)
- Requires a MySQL API credential with ‘user’ and ‘password’ fields.
Functionality
- Record Management: Tracks document insertions, updates, and deletions in vector databases.
- Incremental Updates: Supports efficient updating of changed documents.
- Namespace Support: Allows organization of records into namespaces.
- Cleanup Options: Provides different modes for managing old or deleted content.
- Time Tracking: Uses server time for accurate record timestamps.
- Key Existence Checking: Efficiently checks for the existence of multiple keys.
- Key Listing: Retrieves keys based on various criteria like time range and group IDs.
- Batch Operations: Supports batch inserts, updates, and deletions for improved performance.
Usage
This node is typically used in workflows where documents are being inserted or updated in vector databases. It helps in:- Tracking which documents have been processed.
- Managing incremental updates to avoid reprocessing unchanged documents.
- Maintaining consistency between source documents and their representations in vector databases.
- Efficiently managing large document collections by supporting batch operations and cleanup strategies.
Internal Implementation
- Uses TypeORM for database operations.
- Implements the RecordManagerInterface for standardized record management operations.
- Creates and manages a MySQL table for storing record information.
- Provides methods for CRUD operations on records, including batch operations.
- Supports indexing for improved query performance.
Note
Proper MySQL credentials and database access are required for this node to function correctly. Ensure that the MySQL server is accessible and that the specified database and table can be created and modified by the provided user credentials.Previous
Postgres Record ManagerThe Postgres Record Manager node is used to keep track of document writes into vector databases. It utilizes PostgreSQL to store and manage records of document insertions, updates, and deletions. This node is particularly useful for maintaining the state of documents across different runs or sessions, enabling incremental updates and efficient management of large document collections.
Next