Current real session replay PRODUCT EVIDENCE / MODEL EVALUATION

Local Chat · Real conversation evaluation across three models

Fixed character cards, first scene and original seven rounds of input, and round-by-round real session playback of DeepSeek, GLM 5.3 Flash and Qwen 3.8 Flash to provide a basis for formal models and supplier routing.

3candidate models
1Fixed character card
7round raw input
6class check dimensions

belong Local Chat Model Evaluation Module · Badcase Enter Attribution ledger · ← Return to project page

01 / Playback settings

The same character, the same first scene, the same set of seven rounds of input

All three sets of tests used existing character cards, a fixed opening act, and the same original seven rounds of user input. Each model enters the real dialogue chain of Local Chat in turn, retaining the above context, role constraints and supplier interface performance.

The tested candidates are DeepSeek, GLM 5.3 Flash and Qwen 3.8 Flash. The evaluation object is the actual performance of the model after being connected to the current product, including text quality and interface behavior.

02 / Check dimensions

Separately examine language performance, state drift, and vendor stability

Arole consistency

Check that titles, relationships, tone, and actions continue to match the character card and previous text.

Blanguage quality

Compare naturalness, information progression, repetition, templated expression, and response to specific situations.

Cfact drift

Record the addition of new people, places, events, or relationships that do not exist, as well as the rewriting of previous facts.

Drejection and interruption

Distinguish between reasonable boundaries, false rejections, truncation, and responses that do not complete the current round of tasks.

Edelay

Record the waiting time round by round and observe whether the response fluctuates significantly after the long context grows.

FInterface stability

Log timeouts, empty responses, format exceptions, and vendor-side errors, and attribute them separately from content quality.

03 / Product Decision

Hengping results directly enter model selection, routing and regression samples

Round-by-round results are broken down by role, language, facts, rejections, and interface issues. Content Badcase enters prompt words, role cards or contextual policy modifications; interface Badcase enters supplier routing, retries and error prompts.

The same seven rounds of input can be played back repeatedly after adjustments to the model, prompt words, or routing to compare whether the problem disappears, shifts, or creates new degradations. The page only announces the current methods and test boundaries; the specific model conclusions are subject to the playback record of the completed review.

04 / Historical Experiment

Five rounds of testing of the old seven models are retained as method evolution records

A seven-model × fixed five-run experiment in 2026-07 was used to validate the structural inspection, cost and delay recording methods. The current product has switched to three-model real playback of existing character cards, fixed opening scenes, and original seven-round input, and the old data no longer bears formal model selection conclusions.

Sentiment Data/Series

1 / 4 Real conversation review Three models · Original seven round playback 2 / 4 Badcase ledger Attribution records · In progress 3 / 4 State modeling Emotional Runtime · Prototype 4 / 4 Character card structure Excerpts from methods and real cards