What the study found
The study found that foundation models often generate domain-specific language (DSL) workflows with defects, and that static analysis feedback can help detect and repair some of those defects. The authors also report an initial taxonomy of 20 defect types and note that 89.23% of the studied workflows contained at least one defect.
Why the authors say this matters
The authors conclude that systematically detecting and repairing defects is a step toward reliable and automated generation of executable workflows from natural language requirements. They present this as important because manually creating coherent workflows requires considerable effort and specialized domain knowledge.
What the researchers tested
The researchers studied foundation-model-generated workflows written in DSLs, which are structured languages used to describe processes. They built Timon, a static analyzer for these workflows, and used its feedback to guide Pumbaa, an FM-based tool, in repairing detected defects.
What worked and what didn't
The study found 20 distinct defect types in FM-generated DSL workflows. It reports that nine of those defect types can be identified through static analysis, and that Timon’s feedback can guide Pumbaa to repair detected defect incidences. The abstract does not say which defect types were not detectable by static analysis or how effective the repairs were overall.
What to keep in mind
The abstract only describes the study at a high level, so detailed limits of the approach are not described in the available summary. It also does not provide comparative performance numbers for Timon or Pumbaa beyond the defect-detection and repair claims stated here.
Key points
- 89.23% of the studied FM-generated DSL workflows contained at least one defect.
- The authors identified 20 distinct types of defects in these workflows.
- Nine defect types were said to be detectable through static analysis.
- Timon is described as the first static analyzer designed specifically for FM-generated DSL workflows.
- Feedback from Timon was used to guide Pumbaa in repairing detected defects.
Disclosure
- Research title:
- Static analysis helped detect and repair workflow defects
- Authors:
- Sogol Masoumzadeh, Keheliya Gallaba, Dayi Lin, Ahmed E. Hassan
- Institutions:
- Huawei Technologies (Canada), Huawei Technologies (Canada), McGill University, Queen's University
- Publication date:
- 2026-04-21
- DOI:
- 10.1145/3809500
- OpenAlex record:
- View
Get the weekly research newsletter
Stay current with peer-reviewed research without reading academic papers — one filtered digest, every Friday.