Published on Aug 20, 2026

Project Duration: July 28 2026 – Aug 20 2026

Diffusion-Model-Agnostic ComfyUI 3D Pipeline

Model generated with no artistic input.

Diffusion-Model-Agnostic ComfyUI 3D Pipeline is a ComfyUI workflow for turning a Stable Diffusion-generated image into a textured 3D model.

The workflow combines 2D image generation, pose control, image-to-3D reconstruction, multi-view rendering, diffusion-based texture generation, facial refinement, and final 3D texturing into a single pipeline.

The key design goal is to keep the image-generation stage interchangeable. While the example uses SDXL, the downstream workflow is structured around conditioning techniques such as ControlNet and IP-Adapter rather than being tied to a single diffusion checkpoint.

GitHub Repository → · Download model.glb →

Pipeline Overview

The complete workflow is divided into five stages:

T-Pose Generation → 3D Model Generation → Multi-View Images → Face Refinement → Texture Generation

Each stage produces information that is passed into the next part of the workflow.

The result is a textured .glb model that can be viewed interactively above and downloaded directly from the repository.

1. T-Pose Generation

The pipeline begins with a T-pose image generated with Stable Diffusion.

OpenPose Studio and ControlNet are used to control the pose and positioning of the character, creating a consistent source image for the subsequent 3D reconstruction stages.

The example uses:

sd_xl_base_1.0.safetensors

The purpose of this stage is to establish the visual appearance and pose that the rest of the pipeline will attempt to preserve.

T Pose

2. 3D Model Generation

The T-pose image is passed into Hunyuan3D to generate an initial untextured 3D model.

The example uses:

Hunyuan3d-dit-v2-0-fp16.safetensors

through the comfyui-hunyuan3dwrapper extension.

At this stage, the focus is on reconstructing the underlying geometry rather than producing the final appearance of the asset.

Model Generation

3. Model to Multi-View Images

After the mesh has been generated, it is rendered from multiple angles using Hy3D Render MultiViewDepth, provided by comfyui-hunyuan3dwrapper.

The renders provide normal and depth information describing the generated mesh from different viewpoints.

These views are then passed back through Stable Diffusion so that the model can be textured from multiple perspectives while maintaining consistency.

The texture-generation stage uses:

controlnet-union-sdxl-1.0.safetensors

An IP-Adapter is also used to preserve the visual style and appearance established by the original T-pose image:

ip-adapter-plus_sdxl_vit-h.safetensors

using the IPAdapter_plus nodes.

This creates a bridge between the original 2D reference and the newly generated 3D geometry.

Model to Multi View Images

The generated face is processed separately to improve facial detail.

The face-refinement workflow uses:

comfyui-impact-subpack
UltralyticsDetectorProvider
face_yolov8m.pt
ZenfaceDetailer

The dedicated refinement stage allows facial features to be treated independently from the broader multi-view texture-generation process.

Face Refiner

5. Final Texture Generation

The refined multi-view results are passed back into the Hunyuan3D texture-generation workflow.

This final stage combines the generated texture information with the reconstructed geometry to produce the completed textured model.

The resulting asset is exported as a GLB file.

Texture Generation

Final 3D Asset

The final result of the pipeline is the included:

model.glb

The model can be inspected interactively at the top of this page or downloaded directly from the GitHub repository.

Download model.glb →

Diffusion-Model-Agnostic Design

The main idea behind the project is to separate image generation from 3D reconstruction and texturing.

The example pipeline uses:

sd_xl_base_1.0.safetensors

for its Stable Diffusion stages, but the overall structure is not intended to depend on that checkpoint specifically.

The T-pose is generated and controlled first, while the later stages use geometry-aware conditioning, multi-view information, and reference-image conditioning.

This makes the workflow suitable for experimenting with different diffusion checkpoints while retaining the same general 2D-to-3D pipeline.

Why Multi-View Conditioning?

A single image does not provide complete information about a 3D object’s appearance.

The workflow therefore generates several views of the reconstructed mesh and uses those views as inputs to the texture-generation process.

The inclusion of depth and normal information provides additional information about the geometry, while ControlNet helps constrain the generated imagery to the rendered structure.

The IP-Adapter then provides an additional reference for preserving the visual identity of the original image.

The combined approach is intended to reduce the amount of arbitrary information that the diffusion model has to invent independently for each viewpoint.

ComfyUI Workflow Structure

Each stage is represented by its own ComfyUI workflow.

The repository follows the same naming convention as the example images, with the workflow definitions stored separately from the rendered image assets:

assets/
├── 1) T Pose.png
├── 2) model generation.png
├── 3) Model to multi view images.png
├── 4) Face Refiner for Model.png
├── 5) texture generation.png
├── 1) T Pose.txt
├── 2) model generation.txt
├── 3) Model to multi view images.txt
├── 4) Face Refiner for Model.txt
├── 5) texture generation.txt
└── model.glb

This makes the individual stages easier to inspect, modify, and reuse independently.

End-to-End Pipeline

The complete process can be represented as:

Stable Diffusion
      │
      ▼
   T-Pose Image
      │
      ▼
   Hunyuan3D
      │
      ▼
 Untextured 3D Mesh
      │
      ▼
Multi-View Rendering
 + Depth / Normals
      │
      ▼
 Stable Diffusion
 + ControlNet
 + IP-Adapter
      │
      ▼
Textured Multi-View Images
      │
      ▼
 Face Refinement
      │
      ▼
 Hunyuan3D Texture Stage
      │
      ▼
   Textured GLB

Each stage has a specific responsibility, allowing the workflow to decompose an otherwise difficult image-to-3D problem into several more controllable operations.

Technical Components

ComfyUI

ComfyUI provides the node-based environment in which the complete pipeline is assembled.

This makes the different generative and reconstruction stages visible as an explicit graph and allows individual components to be swapped or adjusted without rewriting the complete system.

Stable Diffusion

Stable Diffusion is used for both the initial reference-image generation and the later multi-view texture-generation stages.

The example uses SDXL, but the pipeline is designed to avoid depending on a particular image-generation model.

Hunyuan3D

Hunyuan3D provides the main 3D reconstruction and texturing stages.

It is first used to generate the untextured mesh and later used again to produce the final textured asset.

ControlNet

ControlNet provides structural conditioning for the Stable Diffusion stages.

In the multi-view stage, it helps constrain the generated appearance around the geometry-derived information produced from the reconstructed mesh.

IP-Adapter

The IP-Adapter provides a reference-image conditioning mechanism.

In this workflow, it helps preserve the appearance established by the original T-pose image while generating the additional views required for texturing.

The face-refinement stage uses dedicated detection and detailing nodes to improve facial information independently from the rest of the generated texture.

Example Model

The included example uses SDXL throughout the Stable Diffusion stages:

sd_xl_base_1.0.safetensors

The 3D generation stage uses:

Hunyuan3d-dit-v2-0-fp16.safetensors

while the texture-generation stage uses:

controlnet-union-sdxl-1.0.safetensors
ip-adapter-plus_sdxl_vit-h.safetensors

This provides a concrete example of the complete pipeline while leaving the overall image-generation stage open to substitution.

Project Outcome

The final result is a reusable ComfyUI-based image-to-3D pipeline that demonstrates how several generative AI techniques can be chained together to produce a textured 3D asset.

Rather than relying on a single model to solve the complete problem, the workflow separates:

visual generation → pose control → geometry reconstruction → multi-view rendering → texture synthesis → facial refinement → final asset generation

This makes the individual stages easier to understand, modify, and experiment with while providing a complete path from a 2D generated character to a textured 3D model.

Repository

The full workflow and example assets are available in the public GitHub repository:

View the GitHub Repository →

Download the Generated 3D Model →

Development Period

August 20, 2026 – Present

Ongoing ComfyUI / generative AI tooling project