AI in Product Development: Process, Uses and Evaluation
AI can assist product research, prototyping, coding and release analysis. Use it for a defined task with a visible review step. The relevant question is whether the whole workflow produces a better result at an acceptable cost, including checking and correcting the output.
AI-generated customer opinions are not customer evidence. A convincing prototype is not a production system. Keeping these boundaries clear prevents apparent speed from concealing a weaker development decision.
Assign AI a bounded role at each stage
| Stage | Useful assistance to evaluate | Human responsibility |
|---|---|---|
| Research | Organize authorised notes and propose themes | Verify themes against source observations |
| Concept | Generate alternative flows and edge cases | Choose the customer problem and constraints |
| Prototype | Draft interface copy or disposable code | Test comprehension and inspect behavior |
| Implementation | Suggest code, tests or documentation | Review security, correctness and maintainability |
| Release | Group incidents and summarize feedback | Confirm severity, decide action and communicate |
Start with an internal task where a mistake is easy to detect and reverse. An assistant that drafts release notes has a different consequence profile from a model that decides customer access or changes financial records.
Preserve the evidence trail
Give research summaries stable source identifiers. Ask for supporting excerpts and separate observed behavior from proposed explanations. Check every material finding against the original notes, including minority experiences that a summary may flatten.
Before sending customer data to a model, establish whether the intended use is permitted, what information is actually necessary and what the selected service does with it. Redact unnecessary identifiers. Review the applicable provider agreement and configuration rather than assuming all AI services handle data identically.
NIST’s Generative AI Profile provides a useful risk reference, including concerns around unreliable output and data privacy. It is not a certificate that an implementation is safe. Translate the relevant risks into specific checks for your workflow.
Build evaluation before broad use
Collect a small, representative set of cases with expected outcomes or a grading rubric. Include ordinary tasks, ambiguous inputs, missing information and known failure cases. Keep some cases aside when refining the prompt so improvements are not judged only on examples the team has repeatedly optimized against.
For a research assistant, grade factual traceability, missed findings, invented claims and handling of uncertainty. For generated code, run the normal software checks and inspect the actual changes. For an AI feature embedded in the product, add misuse attempts, unexpected instructions inside retrieved material and permission boundaries.
Compare against a real baseline, such as the team’s current manual method. Include review time, corrections, model charges and operational overhead. Record the model or service version, prompt, source material and evaluation date so a later change can be investigated.
Hypothetical example: an interview-note assistant
A fictional product team wants help grouping interview notes about appointment scheduling. It prepares authorised, redacted notes with observation IDs. A reviewer creates a reference list of important findings, including an uncommon accessibility problem.
The assistant produces themes with linked observations. In the first evaluation it merges “cannot see available slots” with “does not trust availability,” despite those requiring different fixes. It also omits the accessibility issue. The team changes the output structure to require contradictory observations and unmet needs, then evaluates again on separate notes.
The team approves the tool for draft organization only. A researcher checks the findings before they enter planning. The result is a bounded workflow decision, not proof that AI can replace user research or reduce development time by a particular percentage.
Release with a way to detect regression
Start with a limited group, retain the previous workflow and define who can disable the feature. Review a sample of real outputs as well as complaints. Customers may silently accept an incorrect answer, so absence of complaints is weak quality evidence.
Re-run relevant evaluations when models, prompts, retrieval sources or product permissions change. Review whether users can identify uncertainty and reach a person where necessary. Keep the underlying product development process focused on customer outcomes, and use the AI output evaluation guide to structure the quality checks.
Distinguish product-development assistance from an AI product feature
Using AI internally and shipping an AI feature create different responsibilities. An internal assistant may draft a prototype that a developer inspects. A customer-facing feature exposes customers to outputs the team cannot review individually. Decide which situation you are evaluating before copying an internal workflow into production.
For an internal task, record the input allowed, expected output and reviewer. For a shipped feature, also define latency expectations, access boundaries, unsupported requests, failure messages and the alternative route. Review the whole service, including retrieval and actions, rather than judging the language model in isolation.
Avoid a claim of development “in record time” unless you have a defined baseline and measured the complete process. Generating a screen quickly can move work into review, integration or maintenance. The relevant unit is a verified product outcome, not the number of generated files or lines of code.
Make research, design and coding checks concrete
For research organization, preserve observation identifiers and ask for contradictory evidence. A reviewer should be able to trace a theme without searching an entire transcript. If the assistant suggests a new research question, label it as a proposal rather than a finding.
For interface concepts, specify the user, task, existing conventions and accessibility constraints. Generate alternatives that differ in a meaningful interaction choice. Test them with users; model-generated preferences cannot substitute for observing comprehension. Use synthetic content only when participants can understand that it is illustrative.
For code, limit the change, inspect dependencies and run the checks appropriate to the touched behavior. Pay particular attention to permissions, error handling and destructive operations. A plausible test generated from the same mistaken assumption as the implementation may pass without detecting the problem. Ask a reviewer to examine the requirement independently.
For release analysis, require links to actual incidents and separate classification from severity decisions. A summarizer can incorrectly merge an outage with a minor interface complaint because both contain similar words. Keep an incident owner responsible for the operational response.
Hypothetical cost comparison
Consider a fictional weekly documentation task. The current method takes four staff hours to prepare and review. A proposed AI workflow takes one hour of preparation, two hours of review and half an hour of corrections. Under those planning assumptions, it saves half an hour before service charges and maintenance.
If the assistant produces polished drafts but requires four hours of checking, the arithmetic changes. If a later source update causes repeated errors, maintenance also belongs in the assessment. Use the automation cost and time-saving calculator to explore declared assumptions, then replace them with observed measurements during a pilot. The calculator cannot measure output quality for you.
Keep an evaluation casebook
Store each case with its purpose, authorised input, expected behavior, severity of failure and reviewer notes. Include a case where the right answer is to ask a question, abstain or escalate. Add discoveries from real use without continually tuning to the same small set.
For subjective tasks, write a grading rubric with examples of acceptable and unacceptable output. Review disagreement between graders. If people cannot agree on what a good output looks like, changing the model may not resolve the underlying product requirement.
Version the casebook and record what changed between evaluations. Keep a separate set of fresh cases for release judgment. Repeatedly improving a prompt until it passes familiar examples establishes performance on those examples, not broad reliability.
Decide what to automate next
Expand the narrowest workflow whose measured results justify it. If quality is adequate but review remains expensive, improve the input structure or task boundary before adding more autonomy. If errors are hard for reviewers to detect, choose a lower-consequence use case or retain manual delivery. Use the experiment plan to state the next comparison and the evidence required for expansion.