К literall mind окау SO can literally геаа our minds now. isual i data using stabl diffusi first is the ted to the test subject, second row 15 the reconstructed image from mri dat Gata. wild. High-res

К literall mind
окау SO can literally геаа our minds now.
isual i data using stabl
diffusi
first is the ted to the test
subject, second row 15 the reconstructed image
from mri dat
Gata.
wild.
High-resolution image reconstruction with latent diffusion models from human
brain activity
Shinji
School of Frontier Biosciences, Osaka University, Japan
“CiNet, МСТ, Japan
Я
Figure 1. images (red box, top row) and images reconstructed from {MRI signals (gray box, bottom row) for one subject
Abstract forward fashion, without the need for any additional train-
ing and fine-tuning of complex deep-learning models. We
Reconstructing visual experiences from human brain also provide a quantitative interpretation of different LDM
tivity offers a unique way to understand how the brain rep- components from a neuroscientific perspective, Overall, our
resents the world, and to interpret the connection between study proposes a promising method for reconstructing
computer vision models and our visual system. While deep ages from human brain activity, and provides a new frame
generative models have recently been employed for this work for understanding DMs. Please check out our web-
task, reconstructing realistic images with high semantic page at
delity is still a challenging problem. Here, we propose a
new method based on a diffusion model to recon
struct images from human brain activity obtained via func-
tional magnetic resonance imaging ({MRI). More specifi- 1. Introduction
cally, we rely on a latent diffusion model (LDM) termed
A fundamental goal of computer vision is to construct
Stable Diffusion. This model reduces the computational
artificial systems that see and recognize the world as
cost of DMs, while preserving their high generative perfor-
man visual systems Recent developments in the mea-
mance. We also characterize the inner mechanisms of the
surement of population brain activity, combined with
LDM studying how its different components (such as the
vances in the implementation and design of deep neu-
latent vector of image conditioning inputs and differ-
ral network models, have allowed direct comparisons be-
ent elements of the denoising U-Net) relate to distinct brain
5 tween latent representations in biological brains and
functions, We show that our proposed can recon-
chitectural characteristics of artificial networks. providing