Software Templates: Redesigning Component Creation to Earn Developer Trust in AI
Developers across Capital One build new components, the services and APIs that power the bank's software, through DevNav Hub, an internal developer platform. To start one, they can use a scaffolded template (a pre-built, pre-approved starting point) or begin from a blank slate and build it themselves. The business wants scaffolding, since it keeps every component secure and compliant from day one, but engineers were splitting close to 70/30 in favor of blank: they trusted their own control more than a template they couldn't fully see into. I designed and tested two different ways to create a new component to find out what it would take to earn that trust.
Highlights
Executive Summary
DevNav Hub's scaffolded templates exist to give every new component a secure, compliant foundation, but adoption was stuck close to 70/30 in favor of starting from blank. With a designer, a content strategist, and a researcher, I designed two structurally different ways to create a new component: a predictable guided form and a conversational AI agent. We prototyped both in Claude and Figma Make, tested them head-to-head with 11 engineers, and I synthesized the results into a recommendation that went to engineering for build.
The Problem
Product leadership had set a target of 60% scaffolded-template adoption, the foundation for automated compliance, drift detection, and CI/CD at scale. But the existing "Create a new component" page buried curated templates inside a flat, undifferentiated grid, sorted only by category, with no signal for which templates were trustworthy, current, or worth the time. Faced with that, most engineers just started from blank.
User Insights
This wasn't a cold start. An earlier internal study had already named a trust deficit behind that 70/30 split and identified a correctly formatted Bogiefile (the file that defines a component's build, compliance, and networking) as the single artifact developers cared about most. That prior work gave me three hypotheses to design against and test directly.
- H1, Remediation Debt: developers abandon templates if cleaning up unnecessary, bloated code takes longer than building from scratch.
- H2, Social Proof: surfacing metadata like "Last Updated" and usage stats will increase template trust and adoption.
- H3, The Black Box: users will only accept automated or agentic creation if they have deep visibility into the generated infrastructure before deploying.
Ideation
Rather than iterate on one direction, we built two prototypes that answered the trust question in opposite ways: one leaning into predictability, the other into flexibility. Getting both in front of real engineers quickly mattered more than getting either one right on the first try.
Divergent concepts. Used Claude and Figma Make to build both a form-based flow and a chat-based flow to a reviewable state in days, not weeks, so partners could react to real screens instead of a written brief.
Early user feedback. Shared both directions informally with internal engineers for early reactions before either one was fully built out.
Stakeholder review. Brought refined prototypes to product and engineering leadership to confirm both were worth testing head-to-head.
Formal usability testing. Ran a moderated head-to-head study with both finished prototypes across 11 engineers.
Claude and Figma Make let the team prototype and test two competing bets in parallel, instead of committing to one direction and hoping it was right. That speed was what made a real head-to-head test possible in one quarter.
The Prototypes
Both flows start from the same question, "what would you like to build," and both end at the same shared execution panel. Everything in between is different.
Prototype 1: Guided Form
Search or browse Golden Path templates, each with a usage count and a deployment-success signal. Choosing one opens a split screen: a structured form on the left, a live Code Viewer, Arch Preview, and File Tree on the right.
Prototype 2: AI Chat Agent
A conversational landing point: describe what you want to build, attach a spec or Bogiefile, and set an autonomy level before the agent proposes an architecture, asks for approval, then drops the same form fields inline once you confirm.
Usability Testing
I ran both prototypes head-to-head with 11 engineers spanning a range of experience levels, working with a researcher on our team to synthesize findings.
Participants split cleanly into two behavioral groups. High-volume creators prioritized strict control and actively avoided anything that felt like bloat. Low-volume creators prioritized guardrails and wanted to understand the "why" behind each choice before committing to it.
Empathy sessions told us engineers had gotten used to chat-based AI coding tools like Windsurf and Claude. A conversational agent felt like meeting developers where they already were.
Experienced engineers already knew what they wanted and didn't want to talk their way through it. Less experienced engineers were wary of a flexible interface that could let them make a costly mistake without realizing it. A few participants put it plainly: if they wanted a chat-based AI to write code, they'd rather just use the tools they already trusted.
The Results
Participants preferred Prototype 1 (the guided form) for actually executing a build. Only 2 of 11 preferred the AI chat agent.
Average usability score out of 5, Prototype 1 versus Prototype 2.
Prototype 1: Guided Form
Predictable and clear. Every configurable option was laid out up front, and users knew exactly what was required and felt in control.
Forced selections, like a required language choice, broke the workflow for experienced engineers building custom or non-standard packages.
"I like the predictability of the questionnaire flow vs. agent. If there are mistakes, I know they're mine."High-volume creator
Prototype 2: AI Chat Agent
Strong for discovery. The conversational interface shone when engineers didn't yet know what they needed and wanted to explore architectural patterns.
A disconnect between the chat and the execution panel, plus an autonomy selector sitting right on the landing page, created immediate hesitation and a feeling of lost control.
"If I'm chatting with an agent I feel like I'm already on a path to failure."Study participant
Bogiefile auto-fill was the standout request
A correctly formatted Bogiefile was already known to be the artifact developers cared about most. Testing confirmed why: participants wanted to drop one in and have DevNav Hub auto-replace account details, owner info, and security groups.
"If it could take that Bogiefile and replace the name, the ASVs, the security groups, that would be fundamentally amazing."High-volume creator
Both prototypes shared the same trust gap
Across both flows, participants explicitly asked to click into the File Tree and view actual generated code before deploying. It was the single most-requested feature for building trust in either direction.
PRDs and specs met real skepticism
Engineers didn't trust an AI to parse a PRD directly into working code, and expected uploads to produce high-level recommendations, not an executed build. For specs, participants warned that a junior engineer executing one without understanding the underlying architecture creates real downstream risk.
"100% Deployment Success" backfired
The Golden Path badge was meant to build confidence, but several participants misread it as a guarantee that deployments always succeed. They asked for an exact "Last Updated on [date]" stamp and explicit language and framework tags instead.
Security Groups need to stay optional
Auto-assigning a security group was praised as a workflow win, but senior engineers warned that AWS caps how many a component can carry. Forcing a new one every time risks silently breaking a deployment.
"What I don't want to do is introduce unnecessary Security Groups."High-volume creator
Saved Drafts split the room
High-volume creators found drafts unnecessary since they build in a single sitting. Engineers navigating complex compliance details wanted them, specifically to pause and gather missing information without losing progress.
Hypothesis Validation
| Hypothesis | Status | Primary finding |
|---|---|---|
| H1, Remediation Debt | ✓ Validated | Engineers actively avoid templates that generate bloated, unnecessary code, and will choose to start from blank rather than manually strip out what they didn't ask for. |
| H2, Social Proof | ~ Mixed | Metadata builds trust, but vague phrasing like "up to date" reads as marketing speak. Engineers wanted hard data, like an exact last-updated date. |
| H3, The Black Box | ✓ Highly validated | Developers demand deep visibility into generated infrastructure before deploying. Being told exactly what's about to happen was the single most-requested feature. |
Next Steps
The form flow won, but the recommendation wasn't to throw the AI work away. I synthesized testing into a phased plan, walked it through with stakeholders, and handed it off to the designers on my team to finalize and move into development.
Ship the transparent form
Move forward with the guided form as the foundation. Remove the redundant Exec Plan tab and make the File Tree fully interactive, so every generated file is inspectable before a single line deploys.
Layer AI on top, not in front
Use AI for smart template selection and document parsing on the existing form, rather than a separate chat surface. Clarify or remove autonomy options like "Full Provision & Deploy" so users never feel they've relinquished control.
AI as a guide, not a builder
Longer term, point the AI at surfacing components other teams already built, rather than generating net-new infrastructure, and let it explain the "why" behind architectural trade-offs to mentor less experienced engineers.