AR-based human-in-the-loop manual assembly process evaluation with high-fidelity assembly simulation method

Assembly simulation and evaluation have become an effective means to analyze product assembly design and assembly plan. Compared with the widely used digital assembly simulation method, the AR-based human-in-the-loop assembly simulation and evaluation method can discover and solve the problems related to the human factors, which is very important for manual assembly. This paper proposes an AR-based human-in-the-loop manual assembly evaluation with high-fidelity assembly simulation method. By proposing the natural human assembly gesture intention recognition method and the virtual assembling parts motion simulation method based on the dual geometric and physical constraints, the AR-based human-in-the-loop manual assembly simulation is realized to provided natural assembly simulation experience and generate realistic assembly simulation process data. In addition, the assembly evaluation methods are integrated to objectively evaluate the performance and operation difficulty of manual assembly process, which comprehensively capture and utilize the human-in-the-loop assembly simulation data including the assembly hand actions, assembly visibility, and assembly time. Finally, the prototype system was developed based on the proposed method and verified in an assembly case by a set of experiments. The results illustrate that the assembly simulation is high fidelity and the assembly evaluation more comprehensively by using the proposed method, realizing a high consistence between virtual and real assembly process validation. This study provides a practical methodology for the manual assembly simulation and evaluation through human-in-the-loop in the AR environment.