SmartUI: Human-in-the-Loop Editable Interface Generation through Semantic Structures

Key findings from this study

• The study found that SmartUI achieves 0.93 structure consistency between semantic specifications and generated interface structures through a domain-grounded component taxonomy derived from 7,381 webpages.
• The researchers demonstrate that human-in-the-loop editing capability significantly reduces modification operations and improves component reuse rates compared to manual design workflows.
• The authors report that a Structured UI Language establishing hierarchical logic and component relationships enables both pre-generation refinement and post-generation interface modification.

Overview

SmartUI is a Figma-integrated plugin implementing a human-in-the-loop approach to generative UI design that addresses structural editability constraints in existing generative systems. The system operationalizes a Structured UI Language to establish high-fidelity mappings between semantic inputs and interface structures, enabling iterative refinement and component reuse throughout design workflows. The work grounds layout generation in empirical analysis of 7,381 webpages to establish a high-frequency taxonomy of UI components.

Methods and approach

The study derives a component taxonomy through statistical analysis of 7,381 webpages to establish high-frequency patterns of container and non-container components. SmartUI incorporates a Structured UI Language to represent design intent, hierarchical logic, and component relationships. The system supports two interaction phases: pre-generation requirement refinement through semantic specification and post-generation interface modification through fine-grained editing. The approach integrates generative AI capabilities within an existing professional design tool to enable incremental design refinement rather than one-shot synthesis.

Results

SmartUI achieves a structure consistency metric of 0.93, indicating high fidelity between specified semantic intent and generated interface structure. Experimental results demonstrate significant improvements in generation speed, component reuse rate, and reduction in modification operations compared to manual design workflows. The structured representation enables both improved efficiency metrics and preservation of design semantic integrity across generation and editing cycles.

Implications

The work establishes that editability and iterative refinement capabilities are critical requirements for generative UI systems operating in professional design contexts. By grounding generative processes in empirically derived component taxonomies and structured semantic representations, the system achieves measurable improvements in both efficiency and design quality preservation. The human-in-the-loop architecture positions generative capabilities as complementary to designer expertise rather than replacement mechanisms.

SmartUI's integration within professional design platforms and its emphasis on structural editability suggest a scalable pathway for generative AI adoption in design workflows. The demonstrated improvements in component reuse and modification efficiency indicate that structured semantic representations enable downstream design operations beyond initial generation. The framework establishes patterns for achieving both creative flexibility and reproducible design quality in generative systems.

The empirical foundation derived from webpage analysis provides a stable semantic basis for layout generation across heterogeneous design contexts. The methodology suggests that domain-specific component taxonomies may be prerequisite for effective generative design systems in specialized application areas. The results indicate potential for structured semantic approaches to enable generative systems that preserve designer control and intent throughout extended design processes.

Scope and limitations

This summary is based on the study abstract and available metadata. It does not include a full analysis of the complete paper, supplementary materials, or underlying datasets unless explicitly stated. Findings should be interpreted in the context of the original publication.