DrawingVQA: A Real-World Benchmark for Multi-Depth Visual-Textual Reasoning on Construction Drawings

CVPR Findings 2026

*¹, *², ³

¹Louisiana State University, ²Independent Researcher, ³University of Illinois Urbana-Champaign

*Equal contribution

arXiv Benchmark Project Page Leaderboard

Civil and construction engineering heavily depend on construction drawings—dense, multi-layered visual artifacts. We introduce DrawingVQA, the first benchmark designed to evaluate multimodal large language models (MLLMs) on real-world "Issued for Construction" (IFC) drawings.

92

Expert QA Pairs

33

Unique IFC Drawings

3

Reasoning Depths

7

CE Domain Dimensions

4

MLLM Capability Axes

DrawingVQA Deconstructing IFC Drawing Complexity.

Contributions:

First Authentic Domain Data: The first VQA benchmark built on complex, real-world "Issued for Construction" (IFC) structural drawings, packed with abstract geometry, symbolic notation, and tabular data.
Practical Questioning by Domain Experts: The first benchmark to capture how engineers reason across visual, textual, and spatial modalities in professional workflows. The question-answer (QA) pairs are crafted by three construction engineers to resemble day-to-day queries, moving beyond surface-level academic questions.
Dual Categorization Framework: We explicitly link professional engineering workflows to core AI reasoning competencies, enabling multidimensional analysis of model strengths and failures.
Expert-Level Baseline: We provide a comprehensive analysis highlighting a critical gap: while frontier models approach entry-level competency, they still fall substantially short of true professional engineering expertise.

The Complexity of IFC Drawings

Unlike schematic floor plans or standard natural images, an IFC drawing spans the entire project lifecycle as a "source of truth." It integrates precise geometry, engineering annotations, material specifications, and regulatory compliance notes. DrawingVQA tests MLLMs on these unique multimodal challenges:

Complex Geometries: Precise 2D representations (plans, sections, elevations) of structural elements.
Symbolic Language: Weld symbols, material hatches, and discipline-specific acronyms.
Dense Annotations: Textual and symbolic callouts with arrows requiring high-fidelity OCR.
Data-Rich Tables: Complex tabular schedules for columns, beams, and footings.
Cross-References: Pointers requiring a model to align information across different dimensional views and details.

Three Progressive Reasoning Depths

Every QA pair in DrawingVQA is classified into one of three reasoning levels that mirror how engineers progressively reason from perception to professional judgment:

★

R1 — Perceptual Understanding

Simple, direct questions requiring visual search or basic OCR. Example: "What is the sheet title?"

37% of questions

★★

R2 — Contextual Interpretation

Connecting and reasoning across multiple pieces of information on a single sheet. Example: "What is the top of the footing elevation for F7x13.83 along grid line 24?"

34% of questions

★★★

R3 — Domain-Expert Reasoning

Compositional reasoning, implicit domain knowledge, and cross-drawing alignment. Example: "What type of vertical reinforcements would I need for the walls shown? Use the other drawing if needed."

29% of questions

R3 is the clearest bottleneck: scores collapse dramatically for all models at this level. In contrast, experienced human Professionals actually score highest at R3 (93.3), revealing where AI and true domain expertise diverge most sharply.

Dual Categorization Framework

To analyze model performance with high granularity, we mapped every QA pair across two orthogonal axes. This allows us to move beyond a single accuracy score and precisely diagnose why a model fails for engineering workflows.

Dual categorization mapping MLLM Capabilities to Construction-Engineering Dimensions

Axis	Dimensions	Primary Dependencies
MLLM Capability ("The AI Task")	Visual Perception · Knowledge · Reasoning (Visual / Alignment / Spatial) · OCR	Maps to foundational AI competencies required to solve each question
Construction Engineering ("The Practical Task")	General/Admin · Element Identification · Language Understanding · Dimensional Recognition · Design Semantic Understanding · QTO · Compliance	Maps to real-world engineering workflows performed on job sites daily

Task Heterogeneity: A single optimization strategy is insufficient. Distinct construction tasks require targeted multimodal improvements:

Administrative Tasks: Rely heavily on OCR and Visual Perception to read explicit information.
Technical Tasks (Element Identification & Cross-Referencing): Shift towards complex spatial, visual, and alignment Reasoning.
Quantity Take-Off (QTO): Highly sensitive to OCR and visual alignment errors, struggling to ground text within complex geometries.
Compliance: Requires external domain knowledge beyond what is explicitly drawn.

Leaderboard: SOTA vs. Human Experts

We benchmarked leading proprietary and open-source models against a robust human baseline comprising Undergraduates, Graduate/Young Professionals, and Experienced Professionals.

The Expertise Gap: Only Gemini-3-pro-preview (77.2%) and Gemini-2.5-pro (71.7%) surpassed the Undergraduate baseline (62.8%). However, all models fall substantially short of Experienced Professionals (94.9%).
Expert Reasoning (R3) Bottleneck: While models score well on basic perception (R1), their performance collapses on R3 tasks requiring deep domain-specific inference, unlike human professionals who peak on these tasks.
Quantity Take-Off (QTO) Failures: QTO remains a major challenge for all MLLMs due to limitations in precise spatial grounding and counting in dense engineering drawings.
Vision is Indispensable: Our text-only ablation study caused a significant performance collapse, confirming the benchmark relies heavily on grounded visual understanding, not just linguistic priors.

Where do MLLMs go wrong?

Understanding Error Modes. We extracted the Chain-of-Thought (CoT) responses to investigate how models derail during complex engineering workflows.

Visual Perception and Visual Grounding Error

Reasoning and perception error on reading sub-drawings

Visual Grounding Failures: A common failure occurs when the model's reasoning plan is logically sound, but its visual perception derails (e.g., misidentifying a grid line while tracing).
Reasoning Errors: Models struggle with the abstract nature of engineering sub-drawings and frequently misunderstand the type of view presented (e.g., mistaking a detail plan view for a section or elevation view)[cite: 1038, 1039, 1097]. This lack of spatial and contextual comprehension can also cause models to hallucinate physical connections between adjacent but entirely separate structural details[cite: 1040, 1041, 1042].
Cross-Referencing Breakdowns: When searching across multiple details for a governing reference, models often anchor to the first plausible link and fail to perform a complete visual scan.
Parametric Knowledge Bias: Models often over-rely on training priors. For example, if a standard weld symbol is intentionally removed from an adversarial drawing, the model still hallucinates the standard procedure rather than reading the explicit visual syntax.
Attention Misalignment: Visual attention maps reveal that models often look at irrelevant regions (like the wrong grid labels) while generating text for specific structural elements, indicating spatial reasoning hallucinations.

Explore the Benchmark

Data Privacy Notice: Due to privacy constraints regarding real-world "Issued for Construction" documents, the original drawing images are not fully open to the public. However, the complete question set (text) is publicly available under CC BY-NC-SA 4.0. Researchers can still perform benchmark evaluations using our HuggingFace leaderboard repository (Coming Soon).