The distinction the authors draw is between assistants and agentic systems. Assistants, such as code-completion helpers, boost productivity while keeping the human in full control. Agentic systems, exemplified by tools like Claude Code, OpenAI’s Codex and Devin, can plan and execute multi-step tasks autonomously, writing code, running tests and updating documentation with limited supervision. Cursor, GitHub Copilot and Amazon Q Developer began life as assistants and have since added agentic modes, reflecting an industry-wide drift toward automation. Caret deliberately swims against that current, positioning itself as a copilot whose every suggestion requires explicit developer approval before a single line of code changes.

What sets Caret apart technically is its architecture of so-called AI-extension points, built on Eclipse’s native plugin mechanism. Three such points structure the system. AgentTechnology lets developers connect new large language model back-ends, whether commercial services such as GPT, Gemini and DeepSeek or self-hosted models like Llama, keeping sensitive source code in-house. TaskGroup allows the definition of entirely new assistive tasks, from refactoring to test generation, each bound to IDE commands. ValidatorTask supports pre- and post-validation workflows that automatically check and improve the quality of generated suggestions before and after they reach the codebase.

Those validation pipelines are among Caret’s most consequential features. When a developer requests assistance, the system builds a task-specific prompt enriched with precisely configured context, retrieved through static analysis of the project. An LLM produces a candidate suggestion, which then passes through an iterative improvement cycle: if the code fails to compile, the error and the output are returned to the model for correction, repeating up to a configured limit. Pre-validation runs before a suggestion is displayed, while post-validation, typically long-running checks such as executing the project’s JUnit test suite, occurs after the developer has applied the change. Anything applied can be reverted with Eclipse’s standard undo command.

The numbers from the team’s evaluation make a compelling case for these loops. Working on a JavaBench project requiring the completion of 66 methods across 14 classes, validated by a suite of 222 JUnit tests, the researchers measured first-pass success rates of 61.4 percent, 70 percent and 82.8 percent for three different LLMs. After improvement cycles, those figures rose to 90 percent, 87.1 percent and 92.8 percent respectively. The cost was real: each compilation iteration increased latency by roughly 79 to 93 percent, and each JUnit iteration by 92 to 169 percent. Notably, the model with the highest success rate was also the slowest, while the fastest model delivered the second-best results, underscoring the value of Caret’s ability to configure and rank multiple back-ends per task.

Context management receives equally careful treatment. Many AI assistants rely on retrieval-augmented generation to stuff prompts with project knowledge, but overly broad context introduces noise, degrades response quality and inflates token costs. Caret’s extension mechanism instead permits fine-grained selection of exactly which elements accompany a request: supertypes, enclosing types, implemented interfaces, attributes, method parameters and local variables, in any combination. A refactoring task might declare only its method parameters, enclosing class and supertypes as context, keeping prompts lean and relevant. Developers can also specify task parameters the assistant should extract from a natural-language request or ask for explicitly, along with IDE actions to perform once a suggestion is accepted.

Accountability is the second pillar of the design. Every interaction between developer and assistant is recorded in a rich traceability model stored in a shared MongoDB database, and assistant-generated code is marked with custom Java annotations directly in the source. A Statistics View presents this data across five tabs, organised by project, user, agent, task and coverage, reporting metrics such as requested, accepted and rejected suggestions alongside acceptance, survival and preservation rates. Filtering distinguishes suggestions that were merely accepted from those that survived in the project or remained unmodified. Developers can even query the interaction history in natural language, asking which model performed best for code completion or which tasks colleagues consistently revert.

That shared database also powers Caret’s approach to collaboration, an area the authors argue most commercial tools handle only coarsely. Where GitHub issues and pull requests reveal agent contributions at the repository level, Caret records where, by whom and what kind of assistance was used, down to individual methods. Teams can inspect how colleagues use the assistant, navigate to the exact locations where tasks were applied, and install assistive tasks published by others through a p2 repository. A developer who notices a teammate using a task called Convert code using lambdas can install it locally with a single click, spreading project-specific conventions and automation knowledge through the team.

Usability evidence supports the design. In a study of 14 participants ranging from near novices to Eclipse experts, the researchers asked participants to define and use an AI-based refactoring task that modified a method body to comply with given guidelines. Caret scored 85.89 on the standard System Usability Scale, above the 85.5 threshold for excellent usability, with participants praising its ease of use, IDE integration and extensibility. An earlier evaluation on JavaBench method completion found that combining task context with validation loops improved effectiveness by up to 147 percent depending on the completion strategy, compared with a bare configuration lacking both.

Caret arrives at a moment when questions of transparency, vendor lock-in and code provenance are becoming urgent for the software industry. Most competing systems are not fully open source, limit which LLMs they support, and offer little native support for tracking the contributions of humans versus machines; only Cursor Blame and Claude’s team dashboard provide partial attribution. Caret, released under the Eclipse Public License 2.0 with source code, documentation and a reproducible capsule publicly available, runs on Windows, Linux and macOS and requires Java SDK 21 and Eclipse SDK 4.33. The authors plan to extend the system with an extension point for repair tasks, restrictions on task executability, configurable usage caps for LLM consumption and a proactive mode that suggests improvements, and are weighing a hybrid assistive-agentic architecture. For now, the message is clear: AI-assisted programming does not have to mean surrendering the controls.

Subject of Research: An open-source conversational AI coding assistant for Java development in Eclipse with extensible validation pipelines, traceability and team collaboration features

Article Title: Caret : A conversational assistant for Java development within Eclipse

Article References: Contreras, A., Guerra, E., & de Lara, J. (2026). Caret: A conversational assistant for Java development within Eclipse. SoftwareX, 36, Article 103076. https://doi.org/10.1016/j.softx.2026.103076

Image Credits: AI Generated

DOI: 10.1016/j.softx.2026.103076

Keywords: Caret, Eclipse IDE, large language models, AI-assisted programming, software engineering, code validation, traceability, open source, Java, conversational assistants, developer collaboration, code accountability