Model Inference and Testing
After fine-tuning is complete, you can interact with your digital avatar in the following ways.
Simple Inference with a Browser Demo
weclone-cli webchat-demo
The script will start a local web service (usually at http://127.0.0.1:7860
), which you can open in your browser to have a conversation. The optimal inference parameters (such as temperature
, top_p
) found here can be updated back to the infer_args
section of settings.jsonc
for future use.
Inference using the API Interface
WeClone provides an API service that can be called by other applications, such as chatbot platforms.
Start the API service
bashweclone-cli server
After the service starts, it will usually listen on
http://127.0.0.1:8005/v1
.Call via API
You can use any HTTP client to send requests to this API.
The API is compatible with the OpenAI format.
Testing with Common Chat Questions
The project also provides a script to test the model with a preset list of questions. You can modify the test question file by changing the test_model_args
parameter.
Make sure the API service (
weclone-cli server
) is running.Open a new command line window (and activate the virtual environment), then run:
bashweclone-cli test-model
The test results will be output to
test_result-my.txt
.