The Original Vision and the One-Shot Approach
When I first conceived WriteGeniuses, my goal was to build an automated writing tool that could generate entire articles with minimal user input. The initial AI content generation workflow was straightforward: a user would provide a topic and some keywords, and the system would send a single, comprehensive prompt to a large language model, expecting a complete article in return. This one-shot generation approach seemed intuitive at first glance, promising simplicity and speed. However, as I detailed in my , this naive strategy quickly revealed significant limitations that necessitated a complete architectural overhaul.
The Failure Modes of One-Shot Generation
My initial approach for WriteGeniuses was to generate an entire article in a single, monolithic request to the LLM. The idea was simple: provide a comprehensive brief and expect a complete, 8,000-word article in return. This strategy quickly revealed severe limitations, primarily due to context degradation within the language model. As the requested output length grew, the model struggled to maintain coherence and consistency across the entire document.
Concrete failure modes emerged consistently. I observed frequent repetitive phrasing, where the LLM would re-state ideas or use similar sentence structures across different sections. More critically, outputs were often truncated, cutting off mid-sentence or mid-paragraph, leaving incomplete thoughts and requiring manual intervention. Furthermore, achieving granular, section-level control was nearly impossible; the model would often blend topics or fail to adhere to specific structural requirements for individual sections, leading to a weak overall article structure. These large, monolithic LLM responses proved inherently fragile, making reliable parsing and post-processing a constant battle. Any minor deviation in the output format, such as an unexpected heading level or a missing paragraph, could break the entire downstream processing pipeline, leading to a frustratingly unreliable AI content generation workflow.
Pivoting to Briefs, Plans, and Independent Units
Recognizing the inherent limitations and unreliability of attempting to generate an entire article in a single, monolithic request, I made a fundamental architectural shift in WriteGeniuses. My solution was to break down the complex task of article generation into distinct, manageable phases: briefing, planning, and execution. The process now begins with a detailed brief, which then informs the creation of an executable plan. This plan is a structured blueprint, systematically dividing the article into a series of independent content units, each with its own specific generation instructions and context. This modular approach was a critical engineering decision because it enabled unit-level retries, allowing me to regenerate only problematic sections without re-processing the entire article, and provided far greater structural and quality control over the final output.
The Convex Timeout Problem with External Requests
My initial architecture for WriteGeniuses relied heavily on Convex actions to orchestrate the AI content generation workflow. When a user requested an article, a Convex action would call the LLM API directly, waiting synchronously for the entire article to be generated. This approach quickly revealed a critical bottleneck: Convex workers, designed for fast, transactional operations, remained occupied for the entire duration of the LLM's response time. Since generating a full article could take several minutes, these long-running external requests frequently led to timeouts, causing the generation process to fail. This highlighted a fundamental limitation of synchronous external calls in serverless environments, where functions have strict execution limits and are not designed to idle while waiting for extended I/O operations. As the explains, actions are best suited for short-lived tasks, and long-running external calls can easily exceed their timeout thresholds, leading to an unreliable AI content generation workflow.
Migrating to Asynchronous Gemini Batch Jobs
The initial synchronous approach to content generation, where a Convex action would call the Gemini API and wait for a response, quickly hit a wall due to worker timeouts. Generating an entire article, or even several independent content units, often exceeded the execution limits of a single function call. To overcome this, I transitioned WriteGeniuses to an asynchronous architecture leveraging the . This migration allowed me to submit multiple generation requests for independent content units as a single batch job to Google's infrastructure. The key advantage here is that the Batch API processes these requests asynchronously without holding open a connection to my Convex worker. Instead, upon job completion, the Gemini Batch API sends a webhook notification to a designated Convex HTTP action, which then processes the results and updates the database. This shift enabled true concurrent generation of content units, significantly improving throughput and eliminating the timeout issues that plagued the synchronous model.
State Management and Request Mapping via Webhooks
Matching asynchronous webhook payloads back to the correct content unit in the database presented a significant architectural challenge. Instead of relying on the AI model to return application-owned IDs, which introduces a dependency on the model's output format and potential for errors, I implemented a more robust system using a requestKey. Each outbound request to the Gemini Batch API is associated with a unique requestKey stored in a Convex request record, and this key is embedded within the webhook URL provided to Gemini.
When the Gemini Batch API completes a job, it sends a POST request to the specified webhook URL. This request is handled by a on the WriteGeniuses backend. The requestKey extracted from the incoming webhook URL allows the HTTP action to precisely map the response back to the original request record and, subsequently, to the specific content unit that initiated the generation. This mechanism ensures reliable state updates and decouples the application's internal IDs from the AI model's output, enhancing system resilience and maintainability.
Building Separate Planning Systems for Different Formats
A one-size-fits-all planning system quickly proved inadequate for diverse content formats. Generating an entire article in one go was problematic, but even a segmented approach struggled when the underlying structure for a blog post was applied to something like an X (formerly Twitter) thread. The core issue is that the goals, constraints, and audience expectations for a long-form blog article differ fundamentally from a series of short, punchy tweets. To address this, I implemented separate planning systems. For long-form blog articles, the system generates a detailed ArticlePlan comprising an introduction, multiple sections with subheadings, and a conclusion, where each section is broken down into individual paragraph units. In contrast, for X threads, a distinct XThreadPlan is created, which focuses on generating a sequence of individual tweets, each designed as a self-contained unit with specific character limits and a clear narrative progression across the thread. This approach ensures that the unit structure adapts precisely to the format, with paragraphs for articles and individual tweets for X threads, optimizing for the unique demands of each medium.
Structured Output Failures, Zod, and Mermaid Edge Cases
Even with robust prompt engineering and explicit instructions for structured output, large language models occasionally return malformed JSON. I've encountered scenarios where an expected array was a string, a required field was missing, or the entire output was an unparseable blob of text. To safeguard against these inconsistencies, I implemented Zod as the final validation boundary for all LLM-generated structured data. Before any content unit or plan step is written to the Convex database, it must conform to its predefined Zod schema, ensuring data integrity and preventing downstream errors.
This validation layer is particularly critical for specialized outputs like Mermaid syntax. Generating Mermaid diagrams often presents unique edge cases, such as unquoted node labels containing special characters, missing semicolons between statements, or incorrect arrow syntax. While the LLM is instructed to produce valid Mermaid, the Zod schema, often combined with a custom parser or regex checks within the validation pipeline, catches these subtle errors. If a generated Mermaid block fails validation, the system can either trigger a retry with a refined prompt or flag the unit for manual correction, ensuring that only executable and correctly formatted diagrams are stored and displayed.
The Current WriteGeniuses Architecture
My current WriteGeniuses architecture is designed for a resilient and efficient AI content generation workflow, integrating several key technologies. The frontend, built with , provides a dynamic and responsive user interface for managing content plans and units. Convex serves as the real-time database and powers all backend logic through its functions and actions, ensuring data consistency and efficient processing of user requests. For the core AI capabilities, I rely on Google's Gemini API, specifically its Batch API for asynchronous content generation, which is crucial for handling long-running tasks. Research capabilities are integrated via Serper, which fetches up-to-date information to enrich the AI's output. This combination allows for a highly integrated system where user requests flow from the TanStack Start frontend, trigger Convex actions, which then orchestrate calls to Gemini and Serper, with results seamlessly updated back to the client via Convex's real-time capabilities.
Tradeoffs and the Next Evolution of the Workflow
The architectural shift to a unit-based, asynchronous workflow with batch jobs and webhooks introduced a necessary increase in complexity. While the system now manages more states and database records, this tradeoff is justified by the significant gains in observability, retryability, and the independent testability of each content unit. This robust foundation allows us to focus on the next evolution: integrating firsthand experience, original data, strong opinions, and deeper customer context directly into the content generation workflow, moving beyond mere summarization to truly unique output. I invite fellow developers and SaaS founders to explore WriteGeniuses and share their insights as we continue to refine this AI content generation workflow.
Q: What is an AI content generation workflow?
A: An AI content generation workflow is the structured sequence of automated steps—such as research, briefing, planning, drafting, and validation—used to produce high-quality content reliably, moving beyond simple single-prompt generation.
Q: Why is one-shot AI article generation unreliable?
A: I found that one-shot generation often leads to context degradation, repetitive phrasing, and truncated outputs because the model struggles to maintain structural coherence and depth over a massive, monolithic response. Attempting to generate an entire article in a single request frequently results in a loss of quality and consistency as the AI tries to manage a vast amount of information within a single context window.
Q: How does the Gemini Batch API improve content generation?
A: The Gemini Batch API significantly improves content generation by allowing my system to process multiple independent content units asynchronously. This prevents backend serverless functions from timing out while waiting for long-running generation tasks to complete. By integrating webhooks, I can initiate a batch job and then receive a callback when the results are ready, transforming the AI content generation workflow into a more robust, event-driven process. You can learn more about it in the Gemini Batch API documentation.
Q: Why use Zod for validating AI structured outputs?
A: I use Zod because it acts as a strict validation boundary that ensures the JSON returned by the AI matches the expected schema. If the model hallucinates keys or returns malformed data, Zod catches the error before it corrupts the database.