Distilled Qwen3.5-4B shopping agent nears its GLM-5.3 teacher on ShoppingBench

The method uses GLM-5.3 on Amazon Bedrock as the teacher, sampled twice on each of 426 training problems from ShoppingBench. The benchmark's own scorer kept only the trajectories that solved the task, and those were split into per-step training rows, about 2,600 for training and 130 for validation.

The student, Qwen3.5-4B, was fine-tuned with LoRA at rank 64 for three epochs using serverless supervised fine-tuning in SageMaker AI model customization. Across 324 held-out problems, the student reached an average Absolute Success Rate of 0.541, against 0.297 for the base model and 0.511 and 0.540 for the teacher sampled and greedy. Training on the teacher's reasoning along with its tool calls added about 13 points over tool-calls-only training.

For serving, the base model is deployed once on a real-time endpoint and the student is attached as a LoRA adapter, so both run as inference components on shared weights. The deployment uses the DJL LMI container with the vLLM engine, pinned to a specific version, and a small application harness executes the tool calls and feeds results back to the model.

Developers can distill a multi-step agent using only a closed teacher's text outputs, without teacher weights or logits, and filter the training data with a programmatic grader.

Keeping the teacher's reasoning in the training targets mattered more than the tool calls alone, so it is worth preserving in datasets.

The results come from one benchmark and a single run with a few points of expected variation, so teams should test on their own tasks before replacing a frontier model.