Behavioral Transfer via Automated Prompt Optimization and LLM-as-a-Judge Evaluation Loops for Prompt-Based Knowledge Distillation

Large language models (LLMs) are efficient and expensive to implement. We present APO-KD, a framework that uses zero-fine-tuning to replicate the observable behavior of a teacher LLM using a cheaper student by maximising discrete packages of prompts rather than weights. APO-KD (i) produces teacher reference outputs, (ii) executes the student with candidate prompts, (iii) assesses alignment with an LLM-as-a-Judge rubric in terms of answer quality, format fidelity, constraint adherence, and consistency, and (iv) rewrites prompts based on judge feedback. On limited bullet-point summarization and code generation (function and unit tests only), distilled prompts are much more effective in getting students to comply and lowering the behavior gap to the teacher compared to zero-shot and manual prompts, and can be deployed quickly and with control using a small budget when fine-tuning is not feasible.