Function Details

Documents of Functions caller in packages

Download Models

• getDownloaderProvider()

Get the model download site, return "ModelScope" "HuggingFace" or "Modelers"

• setDownloadProvider(String repo)

Set the model download site, repo can be "ModelScope" "HuggingFace" or "Modelers"

• download(String modelId, {ignore = false})

Download model from the Internet, parameters modelId refering to Model to be downloaded, and ignore is used to ignore modeld check before downloading.

• downloadWithListen(String modelId, {listen = null, ignore = false, interval = 500})

Download model with downloading progress listening.

Initialize Models

• getSessionId(String? sessionId)

Generate chat session id, if sessionId is null, a new session id will be returned accroding to timestramp.

• getConfigPath(String modelID)

Return model config file path, if model not be downloaded, it will return null.

• getModelName(String? modelId)

Return model Name according to modelId. If modelId is null, return string data ""

• initModel(String modelId, String modelName, String configFile, String sessionId, {bool keepHistory = false})

Load model to device, if keepHistory is set, it can support multi-turn chats.

• modelInitState()

If model initialize successfully it will return 0, else 1.

Generating Tokens

• setThinking(String modelName, bool think)

Set Thinking or Non-thinking mode for LLM, only support reasoning models.

• setSystemPrompt(String systemPrompt)

Set system prompt of LLM.

• setMaxTokens(int maxTokens)

Set the max number of tokens can be generated.

• getConfig()

Get the detail of configure content, such as Temperature TopP TopK ...

• setConfig(String config)

Set the detail of configure content, config must be json format.

• stopGenerate()

Stop Generating Tokens.

• generateStreamAnswer(String input, onData, onError)

Generate answer in stream, onData refer to method call when data received, onError called when exception happens.

• generateFinalAnswer(String input, {refresh = 500})

Obtain the final generated data, refresh indicates the time interval of polling.

• generateAnswer(String input, {bool stream = false})

Create a new thread for LLM generation of tokens

• onListenGeneration(onData, onError)

An inner method called in generateStreamAnswe(String input, onData, onError).

Destory LLM

• resetLLM()

Re-set LLM, clear all the chat historys.

• destroyLLM()

Destroy the LLM when Flutter dispose a stateful widget.