Sony May Implement Gaze Tracking Technology in Non-VR Games


NeoGAFs Kent Brockman

Sony is exploring a technology that would allow it to track player gaze even without a virtual reality headset. According to its latest patent filings, the company has recently been experimenting with a heads-up display rendering solution which places HUD elements around the screen in a dynamic fashion, based on where the player is looking at.

The idea of using gaze tracking to inform HUD placement is something Sony has been researching since at least the PlayStation VR2 development cycle. The solution was initially explored as an extension of foveated rendering, an imaging technique powered by machine learning that allows VR headsets to conserve resources by only rendering high-definition textures in specific areas of the screen which players are focusing on. According to another recently surfaced patent filing, the PSVR 2 could feature this particular image enhancement.

Earlier this year, Sony filed to secure a patent for automatic HUD positioning based on gaze tracking. Instead of a VR headset, the experimental solution would use a single camera to track what the player is looking at. It would then analyze that information with a specialized machine learning model to render a custom view of a virtual environment which would only contain HUD elements in areas that the player is actively inspecting. The filing doesn't delve into any technical characteristics of the camera used for the gaze tracking solution it describes, though it does mention that the implementation would rely on both head movement and biometrics such as eye detection for accurate measurements. For context, the PS5 HD Camera is capable of both.

This is the second patent for a HUD positioning mechanism based on gaze tracking that Sony filed within the last year. The initial filing described a VR-only solution, though both technologies primarily seek to deliver more responsive and personalized gaming experiences.



I do wonder what they had in mind when they designed the PS5 HD camera. Why it had two lenses spaced roughly IPD-width apart never made sense to me. Felt like they had some kind of 3D application in mind, but there's been nothing to show for it.

Hard to believe that this was the plan for the camera. The patent seems a bit late for that.
Last edited:
Top Bottom