From natural language to RL formulation: a digital twin–centered optimization framework for manufacturing systems.

Digital twins (DTs) have become a key enabler for simulation-driven analysis and optimization in manufacturing systems. As optimization tasks increasingly involve complex dynamics, operational constraints, and uncertainty, reinforcement learning (RL) offers a flexible and generalizable solution paradigm. However, the practical adoption of RL in manufacturing remains limited, primarily due to the difficulty of formulating well-defined, executable learning problems from high-level process objectives.

This work introduces a DT-centric optimization framework that bridges this gap by allowing practitioners to specify optimization intent directly at the process level, while automatically constructing the corresponding RL formulation, including state, action space, and reward structure. Central to the framework is an evolving playbook that accumulates and refines formulation knowledge across iterative optimization cycles, improving robustness and efficiency over time.

Experimental results on representative manufacturing scenarios demonstrate that the proposed approach achieves a 28% improvement in final return and reduces convergence time by 27%, while maintaining a stable constraint-satisfying success rate of 0.87.

Recommended citation: Fan, Haolin, et al. "From natural language to RL formulation: a digital twin–centered optimization framework for manufacturing systems." CIRP Annals (2026).