In modern software development, using large language models to initiate tasks and refine requirements often reveals two pain points: unclear requirements and recursive, unhelpful follow-up questions. In recent engineering experiments using Codex, powered by GPT-5.4-medium, in conjunction with Atlassian’s Rovo integration, I tested a lightweight, specification-driven workflow designed to evaluate whether file-based state can address these issues.
In my previous article on AI-powered workflow development, I examined the differences between having an agent pre-read a detailed specification file and conducting interactive launch sessions using chat. During this analysis, I noticed that processing static specification files often puts the model in a passive, rigid context mode, whereas interactive chat allows for dynamic context creation. Based on these findings, this experiment assessed whether storing state exclusively on local file systems and forcing the creation of new context windows for each run could further improve the launch phase.
Architectural design of an isolated project launch workflow.
To test this approach, I created a custom skill in Codex called ec-kick-off. The skill’s core contract was focused: it accepted a single Jira history key as an explicit source of truth, fetching the issue context directly from Jira and rejecting unstructured input.
At launch, the workflow ran in a subagent’s isolated environment rather than inheriting conversation context. It began by fetching the issue summary, description, and status directly through the Atlassian Rovo Jira plugin, storing the raw data in agents_utils/raw_tickets/<KEY>.json. The workflow then normalized this JSON data into a clean Markdown document in agents_utils/raw_tickets/<KEY>.md, removing redundant API metadata and retaining only human-relevant content. In the final security step, the model read the normalized snapshot, merged existing draft context from previous passes, if any, and created an insert-ready technical ticket stored in agents_utils/hardened_tickets/<KEY>.md.
During the execution, I noticed an interesting implementation detail regarding how Codex handled updates. To process open issues and overwrite existing protected tickets, the agent generated temporary system scripts that created intermediate prompt files, such as /tmp/<KEY>_kickoff_prompt_v2.txt, before writing the output back to the target directory. This pattern of cloning and modifying demonstrates how file-based agent workflows leverage the generation of temporary local scripts to manage state transitions across isolated boundaries.
System conclusions and engineering trade-offs
Managing Question Loops with Skill Rules
The primary purpose of storing context in local files was to prevent the model from generating redundant clarifying questions when initiating application processing. Results showed that the storage medium does not affect the model’s behavior when providing prompts. Passing context through files, rather than through prompt strings, does not alter the model’s underlying decision-making logic.
File storage serves solely as a persistent storage layer. Preventing infinite loops requires explicit constraints built into the skill request configuration, rather than changes to the way context is provided.
To eliminate repeated feedback loops, constraints should be defined directly in the skill request configuration. If requirements are incomplete, the skill should instruct the model to apply minimal, reasonable assumptions, record any remaining uncertainties in a designated open questions section, and terminate without further prompting the user.
The Fallacy of Stateless Context Isolation
The second hypothesis was that creating a new context window for each iteration would eliminate hallucinations and noise in the output. Isolated execution was also expected to prevent contamination of the generation space by accumulated conversation history.
The experiment refuted this assumption. When the model reads state from a local file in a completely new context window, it must completely reconstruct its conceptual understanding of the problem domain. Since the model has no history of previous conversations, its baseline probability of generating hallucinations remains unchanged on each pass.
The new context windows don’t eliminate the hallucination space; they reset it. Multi-stage refinement in chat allows the context buffer to strengthen over successive iterations, resulting in higher-quality technical characteristics.
In practice, multi-step dialogue within a single context flow consistently yielded better results than isolated single-pass executions. Interactive chat allows engineers to gradually guide the model, validating assumptions and creating a shared context that strengthens over time.
Changing context in local documentation snapshots
To provide static context such as architectural rules and testing guidelines, I tested an approach where Codex downloaded a central Confluence page and created a local copy in Markdown format that served as a local rules file.
When inspecting the resulting local file, its contents were found to be identical to the source code. However, during subsequent implementation phases, the model generated end-to-end test cases containing extensive negative test cases. This conflicted with our team testing strategy, which explicitly limited end-to-end tests to core success scenarios, while negative testing was included in unit and integration tests.
While reviewing the local Markdown file generated by Codex, I discovered a minor change. The original Confluence code stated that end-to-end tests were limited to testing only the primary success scenarios. In the model-generated copy, this statement had been changed, and now, in addition to the primary success scenarios, critical business processes were included.
Allowing LLM to generate intermediate local copies of static documentation results in hidden context modification. Retrieving rules directly from the source of truth using read-only tools eliminates the possibility of model rewriting and ensures policy integrity.
This minor wording change expanded the scope of our testing and altered our engineering strategy. While this addition was overlooked during the initial review, it revealed a key risk: whenever a model generates local copies of reference documentation, it retains the ability to be modified. Reading documentation directly from the source with read-only tools eliminates the possibility of rewriting the model and ensures policy integrity.
Strategic Conclusion
The transition to specification-driven design requires a balance between automatic isolation and interactive refinement. While file persistence provides auditability and state history, relying on fresh contextual walkthroughs does not prevent hallucinations or looping questions. Successful implementation requires the use of direct read-only links for static documentation, guiding model behavior through strict skill hint definitions, and the use of multi-step chats to gradually tighten complex software specifications.