DemoFusion: Democratising High-Resolution Image Generation With No $$$

Ruoyi Du; Dongliang Chang; Timothy Hospedales; Yi-Zhe Song; Zhanyu  Ma

DemoFusion: Democratising High-Resolution Image Generation With No $$$

Ruoyi Du, Dongliang Chang, Timothy Hospedales, Yi-Zhe Song, Zhanyu Ma

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

High-resolution image generation with Generative Artificial Intelligence (GenAI) has immense potential but, due to the enormous capital investment required for training, it is increasingly centralised to a few large corporations, and hidden behind paywalls. This paper aims to democratise high-resolution GenAI by advancing the frontier of high-resolution generation while remaining accessible to a broad audience. We demonstrate that existing Latent Diffusion Models (LDMs) possess untapped potential for higher-resolution image generation. Our novel DemoFusion framework seamlessly extends open-source GenAI models, employing Progressive Upscaling, Skip Residual, and Dilated Sampling mechanisms to achieve higher-resolution image generation. The progressive nature of DemoFusion requires more passes, but the intermediate results can serve as "previews", facilitating rapid prompt iteration.

Original language	English
Title of host publication	2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Publisher	Institute of Electrical and Electronics Engineers
Number of pages	18
Publication status	Accepted/In press - 27 Feb 2024
Event	The IEEE/CVF Conference on Computer Vision and Pattern Recognition 2024 - Seattle, United States Duration: 17 Jun 2024 → 21 Jun 2024 https://cvpr.thecvf.com/

Publication series

Name	Proceedings (IEEE Computer Society Conference on Computer Vision and Pattern Recognition.
Publisher	IEEE
ISSN (Print)	1063-6919
ISSN (Electronic)	2575-7075

Conference

Conference	The IEEE/CVF Conference on Computer Vision and Pattern Recognition 2024
Abbreviated title	CVPR 2024
Country/Territory	United States
City	Seattle
Period	17/06/24 → 21/06/24
Internet address	https://cvpr.thecvf.com/

Access to Document

DemoFusion_DU_DOA27022024_AFV_CC BYAccepted author manuscript, 42.7 MBLicence: Creative Commons: Attribution (CC-BY)

Cite this

Du, R., Chang, D., Hospedales, T., Song, Y.-Z., & Ma, Z. (Accepted/In press). DemoFusion: Democratising High-Resolution Image Generation With No $$$. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (Proceedings (IEEE Computer Society Conference on Computer Vision and Pattern Recognition.). Institute of Electrical and Electronics Engineers.

@inproceedings{a9bdbc7c63124f78890ddd87c2575bb1,

title = "DemoFusion: Democratising High-Resolution Image Generation With No $$$",

abstract = "High-resolution image generation with Generative Artificial Intelligence (GenAI) has immense potential but, due to the enormous capital investment required for training, it is increasingly centralised to a few large corporations, and hidden behind paywalls. This paper aims to democratise high-resolution GenAI by advancing the frontier of high-resolution generation while remaining accessible to a broad audience. We demonstrate that existing Latent Diffusion Models (LDMs) possess untapped potential for higher-resolution image generation. Our novel DemoFusion framework seamlessly extends open-source GenAI models, employing Progressive Upscaling, Skip Residual, and Dilated Sampling mechanisms to achieve higher-resolution image generation. The progressive nature of DemoFusion requires more passes, but the intermediate results can serve as {"}previews{"}, facilitating rapid prompt iteration.",

author = "Ruoyi Du and Dongliang Chang and Timothy Hospedales and Yi-Zhe Song and Zhanyu Ma",

year = "2024",

month = feb,

day = "27",

language = "English",

series = "Proceedings (IEEE Computer Society Conference on Computer Vision and Pattern Recognition.",

publisher = "Institute of Electrical and Electronics Engineers",

booktitle = "2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)",

address = "United States",

note = "The IEEE/CVF Conference on Computer Vision and Pattern Recognition 2024, CVPR 2024 ; Conference date: 17-06-2024 Through 21-06-2024",

url = "https://cvpr.thecvf.com/",

}

Du, R, Chang, D, Hospedales, T, Song, Y-Z & Ma, Z 2024, DemoFusion: Democratising High-Resolution Image Generation With No $$$. in 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Proceedings (IEEE Computer Society Conference on Computer Vision and Pattern Recognition., Institute of Electrical and Electronics Engineers, The IEEE/CVF Conference on Computer Vision and Pattern Recognition 2024, Seattle, Washington, United States, 17/06/24.

DemoFusion: Democratising High-Resolution Image Generation With No $$$. / Du, Ruoyi; Chang, Dongliang; Hospedales, Timothy et al.
2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Institute of Electrical and Electronics Engineers, 2024. (Proceedings (IEEE Computer Society Conference on Computer Vision and Pattern Recognition.).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - DemoFusion: Democratising High-Resolution Image Generation With No $$$

AU - Du, Ruoyi

AU - Chang, Dongliang

AU - Hospedales, Timothy

AU - Song, Yi-Zhe

AU - Ma, Zhanyu

PY - 2024/2/27

Y1 - 2024/2/27

N2 - High-resolution image generation with Generative Artificial Intelligence (GenAI) has immense potential but, due to the enormous capital investment required for training, it is increasingly centralised to a few large corporations, and hidden behind paywalls. This paper aims to democratise high-resolution GenAI by advancing the frontier of high-resolution generation while remaining accessible to a broad audience. We demonstrate that existing Latent Diffusion Models (LDMs) possess untapped potential for higher-resolution image generation. Our novel DemoFusion framework seamlessly extends open-source GenAI models, employing Progressive Upscaling, Skip Residual, and Dilated Sampling mechanisms to achieve higher-resolution image generation. The progressive nature of DemoFusion requires more passes, but the intermediate results can serve as "previews", facilitating rapid prompt iteration.

AB - High-resolution image generation with Generative Artificial Intelligence (GenAI) has immense potential but, due to the enormous capital investment required for training, it is increasingly centralised to a few large corporations, and hidden behind paywalls. This paper aims to democratise high-resolution GenAI by advancing the frontier of high-resolution generation while remaining accessible to a broad audience. We demonstrate that existing Latent Diffusion Models (LDMs) possess untapped potential for higher-resolution image generation. Our novel DemoFusion framework seamlessly extends open-source GenAI models, employing Progressive Upscaling, Skip Residual, and Dilated Sampling mechanisms to achieve higher-resolution image generation. The progressive nature of DemoFusion requires more passes, but the intermediate results can serve as "previews", facilitating rapid prompt iteration.

M3 - Conference contribution

T3 - Proceedings (IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

BT - 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

PB - Institute of Electrical and Electronics Engineers

T2 - The IEEE/CVF Conference on Computer Vision and Pattern Recognition 2024

Y2 - 17 June 2024 through 21 June 2024

ER -

DemoFusion: Democratising High-Resolution Image Generation With No $$$

Abstract

Publication series

Conference

Access to Document

Fingerprint

Cite this