API Loader

​

Node Information

• Name: apiLoader
• Type: Document
• Category: Document Loaders
• Version: 1.0

​

Parameters

1. Text Splitter (optional)

   • Type: TextSplitter
   • Description: A text splitter to break down large API responses into smaller chunks.

2. Method

   • Type: Options
   • Options: GET, POST
   • Description: The HTTP method to use for the API request.

3. URL

   • Type: string
   • Description: The URL of the API endpoint.

4. Headers (optional)

   • Type: JSON
   • Description: HTTP headers to be sent with the request.

5. Body (optional)

   • Type: JSON
   • Description: JSON body for POST requests.

6. Additional Metadata (optional)

   • Type: JSON
   • Description: Extra metadata to be added to the extracted documents.

7. Omit Metadata Keys (optional)

   • Type: string
   • Description: Comma-separated list of metadata keys to omit from the final document. Use ’*’ to omit all default metadata.

​

Input/Output

​

Input

• Configuration parameters as described above.

​

Output

• An array of IDocument objects, each containing:
  • pageContent: The API response data as a JSON string.
  • metadata: An object containing at least the url of the API endpoint, and any additional metadata specified.

​

Functionality

1. The node sends an HTTP request (GET or POST) to the specified API endpoint.
2. The response is converted into a JSON string.
3. The JSON string is wrapped in a Document object with metadata.
4. If a text splitter is provided, the document is split into smaller chunks.
5. Additional metadata is added, and specified metadata keys are omitted if requested.

​

Use Cases

• Fetching data from RESTful APIs for processing in language models.
• Integrating external data sources into document-based workflows.
• Preprocessing API responses for further analysis or summarization.

​

Notes

• The node uses Axios for making HTTP requests.
• Error handling is implemented for failed requests.
• The node supports both GET and POST methods, with customizable headers and body for POST requests.
• Metadata handling is flexible, allowing for addition of custom metadata and omission of default or specified keys.