Skip to main content

localAIHandler

Proxy built-in AI APIs to a local LLM implementation

Process: Utility

This module is intended to be used by a script registered to a session via ses.registerLocalAIHandler(handler)

Methods​

The localAIHandler module has the following methods:

localAIHandler.setPromptAPIHandler(promptAPIHandler) Experimental​

  • promptAPIHandler Function<typeof LanguageModelUtility | null>
    • details Object
      • webContentsId Integer - The unique id of the WebContents calling the Prompt API.
      • securityOrigin string - Origin of the page calling the Prompt API.
      • frameToken string - The frame token of the frame calling the Prompt API.
      • renderProcessId Integer - The process id of the renderer process hosting the frame calling the Prompt API.

Sets the handler for new Prompt API binding requests from the renderer process. This happens once per pair of webContentsId and securityOrigin. Returning null from the handler will reject the creation of a new Prompt API session in the renderer. If you want to invalidate existing Prompt API sessions, clear the local AI handler for the session using ses.registerLocalAIHandler(null).

[!NOTE] If a renderer calls the Prompt API before setPromptAPIHandler() has been called, the request is queued. Once the handler is set, all queued requests are flushed. If too many requests are queued, the oldest pending request is dropped and pending promises in the renderer will be rejected. To avoid this, be sure to call setPromptAPIHandler() as early as possible.