If I'm not wrong SSGs GPUs from AMD already do something like that.
"The extended framebuffer uses NAND Flash (we explain this here) storage. It's effectively an SSD bolted onto a GPU, and allows production users to locally store large files that are being actively crunched by the GPU. Whenever the GPU needs a large texture, big data, or composite objects, it'll talk to the CPU (via PCIe bus), which then talks to the system storage device or memory, depending on if that data happens to already be loaded in volatile memory. The SSG bypasses this process by storing files for entire projects locally. Those files remain resident until cleared by the application or user, and only need to be migrated to the SSG once (unless cleared regularly) when working on a project. In this way, it acts as proper solid-state storage; this is not your standard volatile VRAM."
Whenever the GPU needs a large texture, big data, or composite objects, it'll talk to the CPU (via PCIe bus), which then talks to the system storage device or memory, depending on if that data happens to already be loaded in volatile memory. -
www.gamersnexus.net
The purpose of SSGs is other, it is not run gaming, but I think Sony is using this tech in PS5, otherwise, 4.5 GBps rate can become a problem to its CPU.