LLM2R: From Large Language Models to 3D Reassemblers

LLM2, a novel approach to 3D part assembly, uses a pretrained decoder-only large language model (LLM) with a custom architecture to process parts as a sequence. The method compresses parts into geometric tokens using a Point-BERT encoder and Q-Former, then uses a causal LLM with LoRA adaptation to predict part poses in a single non-autoregressive pass. A key innovation is the use of learnable summary tokens that aggregate the entire assembly, enabling efficient and accurate part assembly. LLM2R achieves the highest part accuracy and success rate on all three PartNet categories, the best rotation and translation error on PartNeXt, and the lowest translation error and fastest inference on Breaking Bad.