Rails Baseline Tests AI Agent's Ability to Discover Built-in Architecture

AI Agents Navigate Rails Baseline Without Explicit Architectural Directives

Rails Baseline, a production-ready Rails foundation designed to provide new SaaS applications and their associated coding agents with clear, conventional patterns, recently underwent an unconventional test. Instead of adding more starter features, the developers behind Rails Baseline decided to package the ZIP file, open it as a new user would, and present it to fresh coding-agent sessions with product prompts. The goal was to see if these agents would naturally adopt the established architectural patterns already present in the repository without being explicitly instructed to do so. This approach bypassed the typical method of turning the prompt into an exhaustive architecture checklist.

This experiment aimed to validate the effectiveness of Rails Baseline's built-in conventions. The foundation incorporates several key architectural decisions, such as using Pundit for authorization, scoping records through the current account, employing public IDs for external references, gating paid access via Entitlements, and utilizing Solid Queue for background job processing. The hypothesis was that AI coding agents, when given a product prompt and access to this pre-architected codebase, would infer and implement these patterns organically.

Testing the Agent's Inferential Capabilities

One of the primary tests involved a coding agent tasked with building a feature. The agent was given product prompts but no specific instructions regarding authorization, data scoping, or background job implementation. The developers observed closely whether the agent would independently leverage Pundit for authorization checks, ensure that all data operations were correctly scoped to the user's account, and utilize public IDs for any necessary external identifiers. Furthermore, the test sought to determine if the agent would naturally integrate Solid Queue for any asynchronous tasks that arose during feature development.

The results indicated that the AI agents were largely successful in inferring and applying the existing architectural patterns. When presented with a prompt to, for example, create a new user-facing resource, the agent would often generate code that included Pundit policies and authorization checks. Similarly, attempts to access or modify data would typically involve scoping mechanisms that aligned with the repository's existing structure, suggesting an implicit understanding of data tenancy. The use of public IDs also appeared to be adopted without explicit prompting, likely due to their prevalence within the codebase.

The integration of Solid Queue for background jobs was another area where the agents demonstrated an ability to follow established conventions. When a task was identified as potentially time-consuming or suitable for asynchronous processing, the agent would often generate code that enqueued jobs using Solid Queue's API, mirroring the patterns found in the foundation's examples and existing background workers.

Implications for AI-Assisted Development

This test suggests that well-structured and opinionated codebases, like Rails Baseline, can significantly enhance the productivity of AI coding agents. By providing a clear, conventional blueprint, such foundations reduce the cognitive load on the AI, allowing it to focus on implementing product logic rather than deciphering or inventing architectural patterns. This is akin to giving a junior developer a well-documented style guide and a set of established libraries; they can build faster because the foundational decisions have already been made.

The success of this approach has broader implications for the future of AI-assisted software development. It indicates that the quality and clarity of the underlying architecture are paramount. A messy or inconsistent codebase would likely lead to confused or incorrect AI-generated code. Conversely, a clean, opinionated foundation acts as a powerful implicit prompt, guiding the AI towards best practices and maintainable solutions.

The developers noted that while the agents were adept at following existing patterns, their ability to innovate or deviate from established conventions was limited. This is not necessarily a drawback; for production-ready applications, consistency and adherence to a chosen architecture are often more critical than introducing novel, potentially unproven, patterns. The agents acted as highly efficient implementers of the Rails Baseline philosophy.

What remains to be seen is how these agents would perform when faced with architectural decisions that are ambiguous or absent from the codebase. Would they default to common, but perhaps suboptimal, patterns, or would they flag the ambiguity and request clarification? The current test focused on a scenario where the desired patterns were already present and demonstrably functional.

The Value of Opinionated Foundations

Rails Baseline's core value proposition is its opinionated nature. It makes specific architectural choices, such as the aforementioned Pundit, public IDs, Entitlements, and Solid Queue, to provide a coherent and efficient starting point. This test validates that these opinions are effectively 'baked in' to the repository to such an extent that AI agents can discover and utilize them without explicit instruction.

For developers adopting Rails Baseline, this means that AI coding assistants can be deployed with a higher degree of confidence. They are more likely to generate code that aligns with the project's intended architecture, reducing the need for extensive manual correction or refactoring. This could significantly accelerate development cycles for new SaaS applications built on the foundation.

The experiment also highlights a potential future where AI agents are trained not just on code, but on architectural principles embedded within well-crafted codebases. This would move beyond simple syntax and pattern matching to a deeper understanding of software design. The success of Rails Baseline in this context underscores the importance of creating robust, well-defined architectural starting points for any software project, whether developed by humans or AI.