| Model | Type | Overall | Reasoning Depth | MLLM Dimensions | Construction Eng. Domain Aspects | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| R1 | R2 | R3 | VP | K | R | OCR | Adm. | EI | LU | DR | SU | QTO | Comp. | |||
| Random | - | 27.2 | 25.8 | 17.7 | 40.7 | 23.5 | 37.5 | 27.9 | 29.0 | 37.5 | 18.9 | 29.2 | 29.0 | 23.0 | 16.6 | 42.8 |
| Human | - | 68.4 | 75.6 | 72.6 | 59.9 | 75.8 | 81.4 | 84.9 | 75.0 | 46.8 | 71.9 | 82.7 | 82.1 | 99.2 | 83.3 | 66.2 |
| * Undergraduate | - | 62.8 | 71.8 | 70.2 | 53.8 | 64.3 | 66.7 | 74.9 | 53.0 | 43.6 | 49.7 | 70.7 | 75.4 | 97.4 | 69.4 | 46.2 |
| * Graduate & Young Professionals | - | 78.2 | 85.4 | 76.6 | 68.8 | 73.9 | 81.9 | 83.8 | 76.0 | 50.0 | 70.0 | 81.9 | 75.0 | 100.0 | 83.7 | 62.5 |
| * Professionals | - | 94.9 | 90.0 | 85.0 | 93.3 | 89.1 | 95.6 | 96.0 | 96.0 | 46.7 | 96.0 | 95.6 | 96.0 | 100.0 | 96.6 | 90.0 |
| GPT-4o | Large | 48.9 | 61.3 | 44.1 | 40.7 | 47.1 | 59.4 | 44.3 | 48.7 | 56.3 | 32.4 | 70.8 | 41.9 | 50.0 | 25.0 | 42.9 |
| o3 | Reasoning | 58.7 | 64.5 | 61.8 | 48.2 | 58.8 | 65.6 | 54.1 | 59.2 | 62.5 | 56.8 | 54.2 | 48.4 | 61.5 | 25.0 | 42.9 |
| Gemini-2.5-flash | Reasoning | 66.3 | 77.4 | 73.5 | 44.4 | 64.7 | 81.2 | 57.4 | 63.2 | 87.5 | 51.4 | 79.2 | 51.6 | 67.3 | 33.3 | 71.4 |
| Gemini-2.5-pro | Reasoning | 71.7 | 80.6 | 76.5 | 55.6 | 67.6 | 78.1 | 67.2 | 72.4 | 87.5 | 56.8 | 79.2 | 67.7 | 75.0 | 41.7 | 57.1 |
| Claude-4.5-Haiku | Reasoning | 54.3 | 51.6 | 73.5 | 33.3 | 52.9 | 65.6 | 52.5 | 54.0 | 62.5 | 43.2 | 50.0 | 48.4 | 51.9 | 33.3 | 42.9 |
| Claude-4.5-Sonnet | Reasoning | 57.6 | 64.5 | 64.7 | 40.7 | 58.8 | 59.4 | 57.9 | 54.1 | 68.8 | 48.7 | 66.7 | 41.9 | 57.7 | 41.7 | 57.1 |
| Gemini-3-pro-preview | Reasoning | 77.2 | 80.6 | 79.4 | 70.4 | 76.5 | 81.3 | 72.1 | 80.3 | 93.8 | 75.7 | 83.3 | 77.4 | 78.8 | 33.3 | 85.7 |
| LLaVA-v1.6-Mistral-7B | Open | 28.3 | 25.8 | 29.4 | 29.6 | 29.4 | 37.5 | 27.9 | 23.7 | 31.2 | 24.3 | 16.7 | 32.3 | 30.8 | 25.0 | 14.3 |
| Llama-3.2-11B-Vision-Instruct | Open | 39.1 | 45.2 | 47.1 | 22.2 | 41.2 | 40.6 | 34.4 | 38.2 | 50.0 | 29.7 | 45.8 | 32.3 | 42.3 | 8.3 | 28.6 |
| LLaVA-OneVision-1.5-8B-Instruct | Open | 40.2 | 61.3 | 41.2 | 14.8 | 38.2 | 46.9 | 31.1 | 39.5 | 43.8 | 24.3 | 62.5 | 25.8 | 42.3 | 0.0 | 0.0 |
| Phi-4-multimodal-instruct | Open | 40.2 | 41.9 | 47.1 | 29.6 | 29.4 | 40.6 | 42.6 | 40.8 | 37.5 | 40.5 | 45.8 | 35.5 | 40.4 | 25.0 | 28.6 |
| LLaVA-v1.6-34B | Open | 42.4 | 38.7 | 44.1 | 44.4 | 50.0 | 53.1 | 42.6 | 40.8 | 43.8 | 37.8 | 41.7 | 35.5 | 40.4 | 33.3 | 42.9 |
| Qwen3-VL-30B-A3B-Instruct | Open | 47.8 | 51.6 | 58.8 | 29.6 | 38.2 | 53.1 | 41.0 | 48.7 | 62.5 | 29.7 | 62.5 | 45.2 | 48.1 | 16.7 | 28.6 |
| InternVL3.5-30B-A3B | Open | 41.3 | 51.6 | 35.3 | 37.0 | 26.5 | 43.8 | 42.6 | 42.1 | 50.0 | 40.5 | 45.8 | 45.1 | 46.1 | 8.3 | 14.3 |
| InternVL3.5-8B | Open | 50.0 | 58.1 | 58.8 | 29.6 | 44.1 | 53.1 | 49.2 | 50.0 | 56.3 | 37.8 | 58.3 | 41.9 | 38.5 | 58.3 | 28.6 |
| InternVL3.5-38B | Open | 50.0 | 48.4 | 64.7 | 33.3 | 41.2 | 59.4 | 47.5 | 54.0 | 62.5 | 35.1 | 62.5 | 54.8 | 44.2 | 16.7 | 14.3 |
| Llama-4-Scout-17B-16E-Instruct | Open | 53.3 | 58.1 | 58.8 | 40.7 | 52.9 | 62.5 | 52.5 | 55.3 | 62.5 | 37.8 | 58.3 | 41.9 | 48.1 | 50.0 | 42.9 |
| Qwen3-VL-8B-Instruct | Open | 53.3 | 51.6 | 61.8 | 44.4 | 50.0 | 53.1 | 45.9 | 55.3 | 81.2 | 40.5 | 66.7 | 45.2 | 46.2 | 33.3 | 42.9 |
| Qwen3-VL-32B-Instruct | Open | 58.7 | 48.4 | 79.4 | 44.4 | 55.9 | 56.3 | 55.7 | 63.2 | 75.0 | 48.7 | 66.7 | 67.7 | 57.7 | 33.3 | 42.9 |
Color legend: Reasoning (API) Large (API) Open-source Baseline | Reasoning Depth: R1 = Recall, R2 = Comprehension, R3 = Inference | MLLM Dims: VP = Visual Perception, K = Knowledge, R = Reasoning, OCR | CE Aspects: Adm. = Administration, EI = Element Identification, LU = Language Understanding, DR = Dimensional Recognition, SU = Design Semantic Understanding, QTO = Quantity Take Off, Comp. = Compliance