A groundbreaking study published in the prestigious INFORMS journal Decision Analysis on November 11, 2025, has shed new light on the capabilities and limitations of generative artificial intelligence (GenAI) in the realm of organizational and policy decision-making. The research offers a nuanced perspective on how AI tools like ChatGPT and its contemporaries are transforming the foundational step of defining objectives—an essential task before any thorough decision analysis can unfold.
In decision science, establishing clear, prioritized objectives is paramount. These objectives define the criteria by which potential solutions, resource allocations, and policy designs are evaluated. Without well-structured objectives, decision processes risk becoming unfocused or ineffective. This study critically examines whether cutting-edge GenAI models can autonomously generate objective sets of the quality necessary to replace or substantially aid human experts in this foundational role.
The research team, led by Jay Simon of American University and Johannes Ulrich Siebert of Management Center Innsbruck, conducted a rigorous empirical comparison of objectives produced by leading GenAI models, including GPT-4o, Claude 3.7, Gemini 2.5, and Grok-2. These AI-generated objectives were benchmarked against those crafted by seasoned decision analysts through six benchmark studies previously published in the same journal. To ensure a robust evaluation, the researchers applied nine crucial criteria rooted in value-focused thinking (VFT), such as completeness, decomposability, and redundancy, to gauge the quality and structural integrity of the objective sets.
Findings revealed a mixed yet insightful picture: while generative AI models consistently produced individually plausible objectives, the overall coherence and completeness of the resulting objective sets were frequently compromised. A common issue was inclusion of “means objectives”—objectives that describe how to achieve ends rather than defining the ends themselves—despite explicit prompts instructing their exclusion. Moreover, AI-generated lists were often redundant or lacked necessary decomposition, hindering their utility in rigorous decision analysis.
An authoritative voice in the field, Ralph Keeney, a pioneer of value-focused thinking, weighed in on the findings. Keeney emphasized that even though both AI-generated and expert-created lists outperform typical layperson efforts, neither should be considered sufficient in isolation for high-quality decision analysis. He reminded practitioners that the fundamental objectives—those core values that truly guide decision alternatives—must be explicitly identified and validated for effective evaluation.
The study did not stop at identifying limitations; it also offered promising insights on how to enhance GenAI’s performance. By experimenting with advanced prompting techniques, including chain-of-thought reasoning that encourages transparent AI deliberation and expert critique-and-revise workflows where human reviewers iteratively refine AI outputs, the researchers significantly improved the quality of generated objective sets. Combining these methodologies yielded more concise, logically consistent, and focused objective frameworks that better align with best practices in decision science.
Simon elaborated, noting that while current generative AI systems excel along multiple dimensions—such as breadth of coverage and idea generation—they still falter in constructing coherent, nonredundant sets of objectives that match expert craftsmanship. This underscores the continuing indispensability of human decision analysts, whose expertise is critical not only in validating but also in refining AI-generated content to reach practical applicability.
Siebert reinforced this synergy viewpoint, advocating a hybrid human-AI approach where generative models assist as creative brainstorming agents, augmenting rather than supplanting expert judgment. He argued that such collaboration harnesses the complementary strengths of machines’ rapid processing and humans’ nuanced understanding, thereby enhancing decision-making quality across complex organizational and policy contexts.
The study culminates with the proposal of a four-step hybrid model for integrating GenAI into decision analysis workflows. This model initiates with AI-driven brainstorming to generate a wide array of potential objectives, followed by expert-led filtering and structuring to ensure the objectives are essential, nonredundant, and decomposable. Subsequent iterative refinement leverages combined human-AI feedback loops to polish the final objective set used in rigorous evaluation and decision-making.
This research marks a seminal contribution to rapidly evolving discussions about the roles artificial intelligence can play in expert domains—highlighting both its transformative potential and inherent constraints. As AI technologies continue to advance, the study’s findings offer a timely reminder that human cognition and judgment remain irreplaceable in discerning what truly matters when making critical organizational and policy decisions.
INFORMS, the world’s leading association for operations research, AI, analytics, and data science professionals, publishes Decision Analysis to communicate such impactful research. The journal continues to champion interdisciplinary innovation and practical advancements that improve decision-making under uncertainty, empowering practitioners globally.
For decision analysts, policymakers, and AI researchers alike, this study provides invaluable empirical evidence and a strategic framework to guide the ethical and effective integration of generative AI into decision science—a field foundational to navigating complexity and uncertainty in the 21st century.
Readers interested in exploring the full study, “ChatGPT vs. Experts: Can GenAI Develop High-Quality Organizational and Policy Objectives?” are encouraged to access it directly via the INFORMS website for an in-depth understanding of the methodology, results, and implications.
Subject of Research: People
Article Title: ChatGPT vs. Experts: Can GenAI Develop High-Quality Organizational and Policy Objectives?
News Publication Date: 11-Nov-2025
Web References: https://pubsonline.informs.org/doi/full/10.1287/deca.2025.0387?af=R
References: DOI 10.1287/deca.2025.0387
Keywords: Artificial intelligence
