Living dossier

Image Models

Back to topics•Blog
Mehran Mozaffari·
9 resources0 related posts

The licence page decides more image-model adoptions than the leaderboard

The honest definition of this field: systems that turn text, images, and structural hints into finished, editable raster images at production quality — plus the runtimes, control layers, and interfaces that make the outputs repeatable. What gets miscategorised: raw model quality. A benchmark win is a footnote; the decisions that actually bind are which weights you may legally deploy, whether downloads work in your CI, and whether the fine-tune ecosystem can build on your base. Those are licence and distribution facts, and this field's licence map is genuinely strange.

The split, from fetched model metadata. Qwen's Qwen-Image ships under Apache-2.0 — a true open-source licence, no gate, 289,743 downloads and 2,601 likes on the Hub, 20.4 billion BF16 parameters, bilingual English and Chinese. Stability AI's Stable Diffusion 3.5 Large is gated: licence listed as "other" (the stabilityai-ai-community licence), and the download gate asks for your name, email, country, organization, and intended use — Research, Personal, Creative Professional, Startup, or Enterprise — before you can pull weights. Black Forest Labs' FLUX.2-dev is likewise gated with licence "other", 609,000 downloads and 2,100 likes, tagged image-to-image with a diffusers Flux2Pipeline class. Same technical territory, three different contracts: one you can fork into a product tomorrow, two you agree to terms before the first byte arrives.

Why this matters more than benchmark deltas: the ecosystem compounds only on permissive weights. Qwen-Image's Hub page links a fan-out of community Spaces — InstantX ControlNet and inpainting adapters, LoRA collections, 8-step fast distillations — all built because anyone can download, modify, and redistribute. The gated models get ecosystems too, but each derivative inherits its own compliance question. My position: for anything commercial, start from the Apache-2.0 backbone and treat the gated frontier models as quality references you evaluate through hosted APIs rather than weight downloads, unless you have read their licence documents line by line.

Every serious image stack is the same four layers wearing different badges

Strip the marketing and the production stack decomposes identically whether you run it on one GPU in a garage or across a render farm. A backbone (the diffusion or flow transformer that makes pixels: Qwen-Image, FLUX.2, SD3.5) sits behind a runtime (diffusers, the 34,400-star PyTorch library for state-of-the-art image, video, and audio generation, or ComfyUI's own execution engine). Control enters at a conditioning layer — ControlNet and its descendants, inpainting masks, reference images — and everything is surfaced through an interface that is either a node graph, a web UI, or an API your code calls. The layer that gets forgotten until launch week is governance: which prompts are allowed, who approves outputs, where generated assets are stored and versioned.

flowchart TD
    B[Backbone weights<br/>Qwen-Image or FLUX.2 or SD3.5] --> R[Runtime<br/>diffusers pipelines or ComfyUI engine]
    C[Conditioning<br/>ControlNet or masks or reference images] --> R
    P[Prompt and text encoders<br/>CLIP plus T5 stacks] --> R
    R --> I{Interface}
    I --> G[Node graph<br/>ComfyUI workflows]
    I --> W[Web UI<br/>A1111 and derivatives]
    I --> A[API<br/>ComfyUI api_server or hosted]
    I --> V[Review and approval<br/>brand and policy gates]
    V --> ASSET[Versioned asset store]

The sequence that actually runs in production is not "prompt, image, done" — it is a controlled generation loop where the conditioning layer and a human gate convert a stochastic model into a repeatable content system:

sequenceDiagram
    participant C as Content brief
    participant G as Generation graph
    participant CN as ControlNet conditioning
    participant R as Reviewer
    C->>G: brief with style and layout contract
    G->>CN: request pose or depth or edge guidance
    CN-->>G: conditioned latents
    G-->>R: candidate batch
    alt rejected
        R->>G: edit notes and fixed seed
        G-->>R: revised batch
    else approved
        R->>G: sign off seed and workflow
        G->>G: freeze workflow for reuse
    end

The fixed-seed sign-off is the whole trick. Image models are deterministic only when you pin the seed, the workflow, and every intermediate; teams that treat the approved workflow JSON as a versioned artifact get reproducibility for free, and teams that do not rediscover their own prompts forever.

ComfyUI won the interface war, and the AGPL incumbent proves the stakes

ComfyUI — now under the Comfy-Org organization after originating with a solo author — has 131,000 stars and describes itself as the most powerful and modular diffusion GUI, API, and backend with a graph/nodes interface. The repository structure tells you it stopped being a toy: a formal api_server for programmatic submission, a comfy_execution engine, a custom_nodes extension system that has become its own economy, and blueprints for shareable templates. Its decisive property is that the graph is the program: a workflow file encodes the model, the conditioning, the samplers, and the post-processing as an inspectable, shareable, re-runnable artifact. Stability's own SD3.5 Large repository ships a SD3.5L_example_workflow.json — the model vendor distributing a ComfyUI graph alongside the weights is the clearest possible endorsement of where the interface battle ended.

The incumbent it displaced, AUTOMATIC1111's Stable Diffusion WebUI, still holds 165,000 stars with an extension system that defined the first generation of image-model tooling. My position: A1111 remains the best single-user studio for interactive experimentation, and its AGPL-3.0 licence makes it a poor embedding choice for hosted products — copyleft terms propagate into anything you ship that links it. ComfyUI is the correct production substrate: graphs version in git, the API accepts workflow JSON from any client, and the custom-node ecosystem means capability arrives without waiting for a core release. The cost shape is real either way — a GPU you own or rent, hours of node-graph literacy, and the operational burden of updating models without breaking workflows that hardcode their file paths.

Control moved from prompt craft to architecture, and that changed who can use these systems

The field's most important architectural idea of the last three years is that generation quality is not the binding constraint — control is. ControlNet (34,100 stars, tagline "Let us control diffusion models!") established the pattern: freeze a pretrained backbone, train a conditioned copy that injects structural evidence — edges, depth maps, poses — so the model renders within a layout instead of hallucinating one. The repository still demonstrates the idea with gradio scripts named exactly for their contracts: canny-to-image, depth-to-image, backed by an annotator module that extracts the conditioning evidence from arbitrary input images, and a cldm module holding the control architecture itself. The Qwen-Image ecosystem's community Spaces repeat the pattern a generation later — ControlNet and ControlNet-inpainting adapters appear as first community derivatives of the Apache-licensed base, which is the compounding argument for permissive weights made concrete.

Editing followed the same path from prompt-hack to architecture. Qwen's own product line now positions editing as a unified-model capability: Qwen VLo markets modify, style-transfer, generate-from-scratch, and multi-element combination in one model, and the July 2026 release note for Qwen-Image-3.0 frames the generation's theme as a single word — "Real" (实) — across authenticity dimensions. My position: instruction-driven editing models have quietly killed the "inpaint with a hand-drawn mask" workflow for most commercial retouching, and the remaining reasons to keep mask-based control are precision work where a wrong pixel is a client escalation. Teams still anchoring their pipelines on pure text prompting are leaving the entire controllability layer unused.

Six named options on the axes that actually gate adoption

Option Role Licence Access shape Ecosystem proof
Qwen-Image Open backbone, bilingual text-to-image Apache-2.0 Ungated download; 20.4B BF16 params; about 58 GB on disk Hundreds of community Spaces: ControlNet, inpainting, LoRA packs, 8-step distills
FLUX.2-dev Frontier image-to-image backbone Custom ("other") Gated download behind an agreement; 609K downloads diffusers Flux2Pipeline integration
SD3.5 Large Established backbone with three-encoder stack Community licence Gated with an intent questionnaire (Research through Enterprise) Ships its own ComfyUI workflow JSON in the repo
ComfyUI Node-graph interface and execution backend Open source (Comfy-Org) Self-host; 131K stars; API server and blueprints Custom-node economy; vendors ship workflow JSON for it
A1111 WebUI Interactive studio AGPL-3.0 Self-host; 165K stars Extension and embedding ecosystem; textual-inversion tooling
diffusers Python runtime Apache-2.0 pip install; 34.4K stars First-class pipeline classes for each major backbone

Read down the licence column and the market's real structure appears: exactly one backbone in the current frontier tier ships under a licence with no gate and no conditions, and it is the one whose derivative ecosystem grew fastest. Read the access column as an operations forecast — every gated row is a CI credential to manage and an agreement that can change; every ungated row is a download URL that behaves like software.

The gigabytes have opinions: encoder stacks and precision tiers are architecture now

The fetched file listings of these model repositories show how much of the engineering has moved into distribution decisions. Stable Diffusion 3.5 Large's repository carries three separate text-encoder stacks — two CLIP variants plus a T5-XXL — with the T5 offered in both fp16 and fp8_e4m3fn quantizations, because the text encoder alone is a meaningful fraction of a consumer GPU. Qwen-Image's transformer ships as nine safetensors shards totalling roughly 58 GB of storage for one model. These are not incidental packaging facts; they are the reason ComfyUI's model-directory conventions, fp8 support, and offloading options are production features, and they explain why the community fast-distillations and fp8 re-exports proliferate on the Hub within weeks of a base release. The precision tier you run is the real cost dial: full precision for quality benchmarks, fp8 for the single-GPU shop, distilled few-step variants for interactive products — with quality falling off at the low end in ways that are visible in fine typography and small hands, which is exactly where clients look.

The ecosystem compound interest shows up in Spaces and pipelines, not in model cards

The strongest predictor of whether a backbone will still matter in a year is not its benchmark row — it is whether other people are building on it without being asked. The Qwen-Image model page lists dozens of community Spaces within months: fast 8-step distillations, LoRA trainers and collections, ControlNet and inpainting variants, image-to-LoRA experiments. diffusers' pipeline catalog adopts each significant backbone as a first-class class — Flux2Pipeline and QwenImagePipeline both appear as library integrations in fetched model metadata — which means the model becomes loadable in five lines by an entire generation of developers. This is the compound-interest layer of the field, and it only pays out on weights that are downloadable without a negotiation.

Download counters are the field's most honest popularity contest

The Hub exposes numbers that cut through benchmark disputes, and the pattern across the three backbones is worth reading carefully. FLUX.2-dev: 609,000 downloads, gated. Qwen-Image: 289,743 downloads, ungated, Apache-2.0. SD3.5 Large: 77,496 downloads, gated with an intent questionnaire, last modified October 2024. The likes column tells a subtler story — SD3.5 Large holds 3,742 likes against Qwen-Image's 2,601 and FLUX.2-dev's 2,100 — which suggests the older model's reputation is anchored in a long accumulation window while the newer models are still in their growth phase. The downloads column, though, measures completed access agreements, and FLUX.2-dev's leading count despite its gate says the demand side will fill in a questionnaire for frontier quality. The interesting number is Qwen-Image's: nearly 290,000 downloads in roughly a year with zero friction is the fastest uncomplicated distribution in the fetched set, and the Space derivative count is the downstream consequence. Watch these counters quarterly; a plateau in downloads with rising Space counts means a model has shifted from evaluation to production embed, which is the transition where licence questions start arriving from legal rather than from engineers.

One more surface from the Qwen announcement deserves note for anyone building product: the Qwen API speaks an OpenAI-compatible format, which means the same client code path that already talks to hosted chat models can carry image generation without a bespoke SDK — and Qwen Studio packages the consumer-facing version of the same capabilities. The unification trend runs in one direction: fewer client integrations, more interchangeable backends behind them, which is the same consolidation the orchestration layer of the agent world went through a year earlier.

Cost shape, stated plainly. Self-hosting is free software plus a GPU bill: a 20-billion-parameter transformer in BF16 wants datacenter-class VRAM or aggressive offloading, and the fast distillations and fp8 quantizations in the community Spaces exist precisely because the full-precision path prices out individuals. Hosted inference converts that capital cost to per-image tokens, with zero ops burden and a different ceiling — rate limits, queue times, and licence terms you did not negotiate. The gated-weight models push you toward their own hosted APIs by construction, which is a rational business strategy and should be priced as one. The only genuinely free lunch in the field is the open-licence backbone on hardware you already own, and even there the electricity is real.

What changed in this window, dated

Three shifts altered decisions. Qwen-Image-3.0 shipped July 21, 2026 as the third generation of the series, with the release note organizing the whole model around authenticity rather than fidelity metrics — a positioning move that says the typography-and-adherence race has been won well enough to compete on realism. FLUX.2-dev's arrival on the Hub as a gated, licence-"other" image-to-image backbone with a diffusers pipeline confirmed the frontier open-weights pattern: weights visible, terms attached, download behind an agreement. And ComfyUI's migration to the Comfy-Org organization — with blueprints, an API server, and unit tests in the tree — marked its transition from hacker tool to maintained platform, which is exactly the point at which production teams should re-evaluate it as infrastructure rather than a community experiment.

Where it breaks

Citation and link rot in the model-news pipeline. Trigger: trusting a news aggregator's paper link instead of the primary source. In preparing this dossier, an arXiv identifier circulating as the FLUX.2 paper resolved to a convex-optimization paper in math.OC — nothing to do with image generation. Every downstream claim built on that link would have been fabricated. Verify model claims against the model card, not the announcement.

The gated-download break in CI. Trigger: automated builds or fine-tune pipelines pulling gated weights by token. HF gates are per-account agreements; a token that works interactively can fail in CI when the licence re-acceptance lands, when the account email changes, or when a mirror is used. Cache weights as artifacts and pin revisions, or your Tuesday training run becomes a licence-prompt autopsy.

The VRAM wall on frontier backbones. Trigger: running a 20B-parameter transformer at full BF16 precision on a single consumer GPU. Offloading works until it doesn't; the practical paths are community fast-distillations, fp8 quantizations, or hosted inference. Plan the precision tier before the art direction meeting, not after. Interface monoculture risk. Trigger: encoding your entire pipeline as ComfyUI graphs with community custom nodes. The graph is opaque to version-control diffs, custom nodes update out from under workflows, and the team's real asset becomes tribal node-graph knowledge that lives in one artist's head. Export the workflow JSON, commit it beside the assets, keep a diffusers script path for the critical few generations, and document every custom node with its pinned version — the repository's own move toward unit tests and database migrations shows the platform is professionalizing, which is exactly when sloppy workflow hygiene stops being forgivable.

Workflow-path fragility. Trigger: ComfyUI graphs that hardcode absolute model file paths. A rename or a reorganization breaks dozens of saved workflows silently — the graph loads, the model node errors, and the failure is discovered at render time. Standardize a models directory convention and pin it.

Fine-tune licence drift. Trigger: redistributing LoRAs or merges trained on gated bases. The derivative inherits the base's restrictions, and hub re-uploads routinely strip the licence metadata. An Apache-2.0 base makes this entire class of problem disappear; a gated base makes every share a compliance review.

Typography and text still punish over-trust. Trigger: long strings, non-Latin scripts, small point sizes in generated signage. The field converged on text rendering as a headline capability, and it is genuinely good — at the sizes and languages the marketing shows. production assets with dense text still need a render-and-composite pass with real fonts for anything a customer will read at print scale.

Interface monoculture risk. Trigger: encoding your entire pipeline as ComfyUI graphs with community custom nodes. The graph is opaque to your version control diffs, custom nodes update out from under workflows, and the team's real asset is tribal node-graph knowledge. Export the workflow JSON, commit it, and keep a diffusers script path for the critical few generations.

Open questions

Whether the gated-weights-plus-API model is stable or a transitional phase — if the frontier keeps shipping behind agreements while permissive backbones close the quality gap, the gated tier's ecosystem moat erodes and the Apache tier wins on gravity alone. Whether node graphs survive as the interface once agents compose workflows in code — the api_server direction suggests ComfyUI is betting both ways. Whether unified edit-and-generate models fully absorb the specialized control stack, or ControlNet-style conditioning remains necessary for precision work indefinitely. And whether text rendering at production scale — dense, multilingual, brand-font-exact — arrives in the base models or stays a compositing pass forever. The economics are already settled in one respect: weights are converging toward free, GPUs are the rent, and the durable costs are governance and review — the layers this field keeps discovering last.

Resources

Model cards and licences:

Qwen announcements and products:

Runtimes, interfaces, and control:

Checked and rejected as a FLUX.2 source: