AI Image Editing Prompts: How to Change One Detail Without Breaking the Whole Image

If you have ever typed "change the shirt to red" and gotten back a woman with a different face, a new haircut, and a kitchen instead of a park, you have hit the single most common failure mode in AI image editing.

The model is not broken. The prompt is underspecified. When you say "change the shirt to red," the model has to decide what else to keep, what to re-render, and what "shirt" even means in this image. Without explicit constraints, it re-rolls the whole image and you lose everything you were trying to preserve.

The fix is the preserve list + the change list — a two-part edit prompt that names what is changing and what must not change. Below is the structure, the 7 patterns that work, the model-specific gotchas, and the iteration loop that turns "change one thing" into a reliable workflow.

Why "change the shirt to red" silently breaks

When the model receives an image and a text prompt, it does not have a concept of "the shirt as a separate object." It has a probability distribution over the entire image. The text prompt is one of many conditioning signals. Without explicit preservation rules, the model re-derives the image from the prompt + a noisy sample, and the rest of the image drifts.

Three things cause the drift:

No preserve list. The model does not know what to keep, so it re-decides everything.
Vague change description. "Shirt" can mean a long-sleeve button-down, a t-shirt, a blouse, or a tank top. "Red" can mean a thousand shades.
No anti-drift language. Without "keep everything else exactly the same," the model re-samples lighting, color grading, framing, and identity.

The fix for all three is a structured edit prompt.

The preserve-list + change-list structure

Every reliable edit prompt has two parts:

Change list: what is changing. Be specific. Name the object, the new state, the new property.
Preserve list: what is staying the same. Name every element that should not move. Be explicit.

Format:

Using the provided image of [subject/scene], change only [specific change] to [specific new state].
Keep the rest unchanged: [list of every element that must not change].

The "keep the rest unchanged" line is the most important sentence. Without it, the model has permission to re-roll.

7 edit patterns that hold up

These work across the four primary models (Nano Banana 2, GPT Image 2, Seedream 5.0 Lite, Midjourney v7) and Midjourney's inpaint / vary-region tools. The exact syntax differs by model, but the structure is the same.

1. Single-object color or material change

Change one thing about one object. The simplest edit, the one that breaks most often.

Vague (breaks): "Change the shirt to red."

Structured (works):

Using the provided image, change only the man's shirt from blue to a deep crimson red (#8B0000).
Keep the rest unchanged: same man, same face, same hair, same pose, same expression,
same jeans, same sneakers, same background, same lighting direction, same camera angle,
same color grading. No other changes.

Why it works: the change is specific (color named, hex if you have it), the preserve list names every other element. The model has no room to re-roll.

2. Single-object swap (shirt → jacket, coffee → tea)

Replace one object with a different object, keep everything else.

Using the provided image, replace the man's blue button-down shirt with a tan
suede bomber jacket, zip halfway, soft worn leather. Keep the rest unchanged:
same man, same face, same hair, same pose, same expression, same dark jeans,
same white sneakers, same street background, same golden hour lighting,
same camera angle. No other clothing changes.

The "no other clothing changes" line is critical. Without it, the model may decide the bomber jacket needs different pants.

3. Background change (subject preserved, scene replaced)

Replace the background. The subject is the preserve list. The new scene is the change list.

Using the provided image of the woman, change only the background from the
beach to a quiet Tokyo street at night, neon reflections on wet pavement.
Keep the subject unchanged: same woman, same face, same hair, same outfit
(white linen dress), same pose, same expression, same scale in frame.
Add natural light spill from the neon signs onto her face.

Add a lighting interaction line ("light spill from the neon signs onto her face") so the model does not composite a daylight subject onto a night scene.

4. Lighting or weather change (everything else preserved)

Change the lighting setup while keeping identity, pose, and framing.

Using the provided image of the office scene, change only the lighting
from midday fluorescent to dramatic golden hour sun streaming through
the window from the right, warm 3200K color temperature, long soft shadows.
Keep everything else unchanged: same people, same clothing, same poses,
same expressions, same furniture, same camera angle, same composition.

Lighting changes that touch the whole image are tricky because the model wants to re-render everything. The explicit "keep everything else unchanged" plus naming the specific new lighting geometry is the way to lock it.

5. Style transfer (photoreal → illustration, day → night)

Apply a stylistic transform while preserving content.

Transform the provided photograph into a Studio Ghibli-style illustration.
Preserve: same composition, same subject positions, same scene elements,
same color palette intent, same focal point. The illustration should
have watercolor textures, soft outlines, and warm natural lighting.
Do not change which elements are in the frame or where they sit.

Style transfer is the highest-risk edit. The preserve list must be very explicit about content (what is in the frame) while allowing style to change.

6. Text replacement (preserving everything else)

This is the edit where the model most often invents new text or shifts layout. The preserve list must be exhaustive.

Using the provided poster image, change only the headline text from
"SUMMER SALE 50% OFF" to "WINTER SALE 30% OFF". Preserve: same font
(foundational sans-serif), same color (deep navy), same position
(top third, left-aligned), same size, same kerning, same letter spacing,
same hierarchy with the subhead below, same background image, same
logo position, same overall layout. No new text, no removed text.

For text edits, name the typography rules (font, color, size, position) explicitly. Models drift on these silently otherwise.

7. Multi-element targeted edit (change 2-3 things, keep the rest)

When you need to change more than one thing, the preserve list gets longer and the change list gets split into numbered items.

Using the provided image, make these changes only:
1. Change the wall color from white to sage green.
2. Change the sofa from gray velvet to cognac leather.
3. Add a single dried eucalyptus stem in the ceramic vase on the side table.

Keep everything else unchanged: same room layout, same hardwood floor,
same window with the same view, same natural daylight from the window,
same camera angle, same focal length, same color temperature, same
overall composition. No new furniture, no removed items, no shifted
objects, no re-arranged decor.

When you change 2-3 things, the model gets more "permission" to drift on the third change. The "no new furniture, no removed items" line cuts that off.

The model-specific syntax

The structure above is model-agnostic. The way you deliver the prompt differs:

Nano Banana 2 (gemini-3.1-flash-image)

Pass the image as inline_data alongside the text prompt
Multi-turn: same conversation, follow-up with "Using the same image, change only..."
The model holds context across turns — you can iterate without re-uploading
Best at semantic masking: "the man in the red shirt" → model identifies and edits that object without explicit coords

GPT Image 2 (gpt-image-2)

Use the edits endpoint with the source image and the text prompt
Explicit preserve language works best: "Keep everything else exactly the same"
Strong identity preservation when you name the subject explicitly
For region edits, use a mask in the API call, then describe only the masked region

Seedream 5.0 Lite

Pass reference image + edit prompt in the same call
Reasoning model — be explicit about what is changing vs staying
"Only modify X, preserve Y" structure works
Multi-image edit supported (combine two references into one)

Midjourney v7

Use the inpaint / vary-region tool with a mask
Select the region in the UI, then describe the change
Or use --cref / --sref for character/style reference, then describe the new state
For targeted edits, the mask-based UI is more reliable than text-only

The iteration loop when the first attempt drifts

Even with a perfect prompt, the first attempt often drifts. The fix is a structured follow-up, not a re-roll.

Step 1: Identify what drifted. Look at the failed output and name what changed that should not have. "The face is different. The lighting is cooler. The background has a new building."

Step 2: Name the drift in the next prompt. "In the previous output, the face changed slightly. The new prompt must preserve the face exactly."

Step 3: Add to the preserve list. "Keep the face identical to the original image. Keep the lighting at 5500K daylight, not the cooler 4200K you used in the last pass."

Step 4: Iterate. Pass 1, 2, 3, 4. The model converges. By pass 3-4 you usually have a clean edit.

Do not re-roll from scratch. The convergence loop is faster and more reliable.

Common mistakes that break edits

"Change the shirt to red" with no preserve list. The model re-rolls the whole image.
"Make it look better" with no specifics. "Better" is not a constraint, it is a vibe. Specify the change.
"Subtle" or "slight" as the only magnitude word. The model does not have a calibrated sense of "slight." Name the new value instead ("lighter by one stop," "10% warmer").
No negative constraints. "Do not change the face, do not change the background, do not add new elements." Without these, the model grants itself permission to drift on each one.
Asking for too many changes in one prompt. Two or three well-specified changes per pass. Beyond that, the model loses the ability to preserve the rest.
Re-rolling instead of iterating. Each new pass should be a refinement of the previous output, not a fresh start. The model's convergence is faster with continuity.

The summary

"Change one thing" in AI image editing is not a single prompt — it is a structured edit prompt with a change list and a preserve list.

Name the change specifically (object, new state, new value).
Name the preserve list explicitly (subject, lighting, camera, scene, color, composition).
Add the "keep everything else unchanged" line — the single most important sentence.
Use model-specific syntax (inline_data for Nano Banana 2, edits endpoint for GPT Image 2, reference image for Seedream, mask tool for Midjourney).
Iterate, do not re-roll when the first pass drifts. Name the drift, add to the preserve list, converge in 3-4 passes.

The model is not the problem. The prompt is underspecified. Add the structure, and "change one thing" becomes a reliable workflow.