The research team, led by Yann Thorens of FINMA alongside Martin Angerer, Pejman Ebrahimi, and Johannes Schneider of the University of Liechtenstein, approached the problem through Design Science Research, a methodology that builds and evaluates practical artifacts rather than merely observing phenomena. They constructed a framework and a prototype tool that applies LLM-based evaluation to token-offering whitepapers, drawing on the Token Offerings Research Database, a hand-collected repository containing more than 6,400 ICOs, STOs, and IEOs. From this database they extracted 29 token projects whose whitepapers remained accessible, 27 of which carried aggregated expert ratings on a scale from 0 to 5 collected before the token launches. The design was fully crossed: every project was evaluated under all three summarization approaches and all four prompting strategies, at two temperature settings, generating 1,139 evaluation runs in total. After excluding runs without expert benchmarks or parseable ratings, 1,008 complete expert–LLM rating pairs covering 26 unique projects formed the analyzed sample.

The technical architecture tested three distinct ways of processing whitepaper content. The first approach, partial document rating, fed as much raw whitepaper text as possible into the model, truncated at 4,000 tokens to accommodate context window limitations. The second, structured summary rating, adopted a two-step process in which the whitepaper was first summarized and then rated. The third, aspect-focused summary rating, went further by extracting targeted information organized into predefined due diligence dimensions: management personality and experience, product characteristics, market conditions, and potential red flags such as insufficient founder experience, lack of financial participation, or ill-conceived project structures. This aspect-focused pipeline produces an explicit intermediate evidence representation that a human reviewer can inspect, exposing which document passages were treated as relevant to individual criteria.

Alongside these content-processing approaches, the team implemented four prompting strategies within a standardized scaffold that assigned the model an evaluator role, fixed the rating scale from 0 to 5, and bounded response length to roughly 100 to 150 words. Zero-Shot prompting provided direct task instructions. Few-Shot added explicit assessment dimensions covering management, product, and market characteristics. Chain-of-Thought requested a factor-by-factor rationale, and Chain-of-Thought with Context and Rubric incorporated domain-specific considerations such as token-locking mechanisms and project originality. The calibration process proved revealing in itself: when prompted without focus on critical aspects, the model produced overly positive feedback, but introducing specific risk criteria shifted a rating from a positive 4 to a critical 2, demonstrating extreme sensitivity to prompt structure.

The quantitative results expose a sobering gap between machine and human judgment. The pooled correlation between human and LLM ratings was essentially zero, with a mean absolute error of 1.14 on the 0-to-5 scale. Approaches 1 and 2 consistently overestimated investment quality relative to human ratings, with mean signed differences of −1.01 and −0.92 respectively. Yet the configuration-level analysis uncovered a clear winner: structured summarization combined with Chain-of-Thought prompting achieved the lowest observed error, with a mean absolute error of 0.706 and a root-mean-squared error of 0.967. This advantage proved robust under project-clustered bootstrap comparisons, significantly outperforming Zero-Shot and aspect-focused alternatives. However, differences in rank correlation between configurations were not statistically resolvable given the sample size, and the researchers were careful to account for the fact that evaluation runs are nested within repeatedly measured projects rather than independent observations.

Perhaps the most striking finding emerged from the stratified error analysis, which revealed that model error is far from homogeneous across cases. Projects rated low by experts exhibited the highest prediction error, with a mean absolute error of 1.549 and a strongly negative mean signed error of −1.353, indicating systematic overestimation: the model consistently assigned higher scores than experts to weak projects. Medium-tier projects showed moderate error, while highly rated projects exhibited the lowest absolute error but a positive signed error of +0.435, suggesting mild underestimation of the strongest ventures. This asymmetric pattern, observed across all three summarization approaches, implies that the model’s optimism bias concentrates precisely in genuinely weaker projects where critical signals are sparse or ambiguous. The mechanism is plausible: generic and lightly structured summaries tend to preserve the promotional tone of whitepapers, overestimating weak projects when red flags are implicit rather than explicit.

To address the possibility that these results were specific to a single closed model, the team added a lightweight multi-model robustness check using two compact open-source models, Qwen2.5-0.5B and Qwen2.5-1.5B, on a matched subset of 47 evaluation runs spanning 25 projects. The additional models reproduced the same directional pattern of overestimation, suggesting that the observed optimism bias is not specific to the original closed LLM, although the comparison remains exploratory given its limited scope. The researchers also analyzed the sentiment of generated text, finding a moderate correlation between sentiment scores and LLM ratings but only a weak correlation with human ratings, indicating that the tone expressed in LLM-generated assessments aligned more closely with the model’s own judgments than with expert opinion.

The authors emphasize that these findings position LLMs as decision-support tools for screening and explanation rather than substitutes for expert judgment. The aspect-focused approach structures criterion-specific evidence and candidate red flags explicitly, producing an inspectable intermediate representation, but its rating error is higher and the interpretability of this representation was not independently evaluated. The researchers recommend a hybrid human–AI workflow in which the LLM structures evidence, surfaces possible red flags, and highlights uncertainty, while the final due diligence judgment remains with domain experts. Cases where the model’s score, extracted evidence, and narrative rationale diverge should be escalated to human reviewers, since these borderline and low-quality projects exhibit the largest systematic overestimation.

The study also acknowledges important limitations. The sensitivity of LLM outputs to small variations in prompt design highlights a critical challenge for scalability and robustness, and the lack of access to individual judge-level annotations means the human benchmark should be interpreted as a reference score rather than a noise-free ground truth. Some of the observed model–human disagreement may reflect genuine human variability in early-stage venture assessment, a well-documented feature of angel and venture-capital screening decisions, rather than model error alone. The effective sample for between-project inference is 26 projects, not 1,008 runs, and replication on a substantially larger and more heterogeneous set of projects is required before the configuration ranking can be treated as general.

Looking forward, the researchers outline several concrete avenues for further work, including testing external validity on additional sources beyond whitepapers, such as governance proposals, tokenomics documents, and smart-contract audit reports, and extending benchmarks to more recent post-cutoff material. They envision the integration of judge models, which provide more consistent ratings as evaluators rather than generative agents, and agentic models that combine reasoning, planning, and tool-use to autonomously retrieve documents and synthesize due diligence information across multiple protocols. In the medium term, a hybrid approach combining the constrained scoring role of judge models with the contextual processing capabilities of agentic LLMs appears promising. For now, the message for investors navigating the fragmented digital asset landscape is clear: AI can help you read the whitepaper, but it cannot yet be trusted to spot the scam.

Subject of Research: Application of large language models to digital asset investment due diligence using mixed-methods evaluation of whitepaper analysis

Article Title: LLM application in digital asset assessments: mixed methods research

Article References: Thorens, Y., Angerer, M., Ebrahimi, P., & Schneider, J. (2026). LLM application in digital asset assessments: mixed methods research. Knowledge and Information Systems, 68(1), Article 289. https://doi.org/10.1007/s10115-026-02910-2

Image Credits: AI Generated

DOI: 10.1007/s10115-026-02910-2

Keywords: large language models, digital assets, investment due diligence, token offerings, prompt engineering, natural language processing, design science research, machine learning, blockchain, investor protection, application, digital