Support NeoGAF

[Eddie-Griffin]

https://www.patentlyapple.com/2023/...-pencil-as-a-vr-game-controller-and-more.html

Today the US Patent & Trademark Office published a patent application from Apple that relates to interacting with and manipulating a user interface of a Mixed Reality Headset, iPad or Mac display using a future Apple Pencil that could be used as a VR gaming controller to control a virtual agent (gaming character) or managing interactions with physical objects. The patent also covers controlling the movement of objects using gaze controls.

In Apple's patent FIGS. 1 and 6B below we see a user holding an Apple Pencil, which in this context, is used as a gaming controller to move objects or a gaming character which Apple refers to as a "Virtual Agent" #606. The functions of the Apple Pencil will include inputs involving tapping, rolling, twirling, flicking and swiping on a touch surface of the Pencil.

The Apple Pencil/control device #130 includes a sensor or set of sensors that detect inputs from the user based on haptic and/or tactile contact with the touch-sensitive surface #175. In some implementations, the Apple Pencil/control device includes any of a plurality of touch sensing technologies including but not limited to capacitive, resistive, infrared, and surface acoustic wave technologies, as well as other proximity sensor arrays or other elements for determining one or more points of contact with the touch-sensitive surface.https://patentlyapple.typepad.com/.a/6a0120a5580826970c02b7517010ed200b-pi

In the other half of the patent, Apple goes into detail about using eye/gaze tracking as a means of moving objects in an extended reality (XR) environment or game.

Gaze controls are on the bottom half of the last image, just in case there's confusion.

Between the finger rings helping with precision controls, the thread on gesture and Kinect movements making you the controller, and now the pencil and Eye gazing controls, Apple seems to be considering every method for controlling regarding VR gaming.

EXCEPT having actual controllers.

But based on the recent consumer interest report, many people on the fence of VR don't want controllers, so Apple is giving them what they want.

It is unique and does differentiate them from other VR headsets. Gaze controls could be really useful if implemented right, and not just in games but in general. The pencil idea also seems innovative at first glance, though for gaming I'm mixed on it.

 

[midnightAI]

More evidence Apple just aren't aiming this device at gamers at all.

 

[CobraAB]

More evidence Apple just aren't aiming this device at gamers at all.

The RUMORED 3K price tag kind of gives that away.