Recently, we saw that the U.S. Patent and Trademark Office published an authorized patent from Apple, which relates to a user interface for interacting with their future mixed reality head-mounted display (HMD). Users only need to use their eyes to gaze. (Technology) can be used for interaction. Is it using eye tracking technology? Maturity remains to be seen.

Apple’s authorized patent describes technology that uses eye gaze to interact with an HMD. According to some embodiments, a user may use their eyes to select a text input field displayed on the HMD display.

This technology provides a more natural and efficient interface, and in some exemplary embodiments, allows the user to identify text to be entered using primarily eye gaze.

These technologies are beneficial for virtual reality, augmented reality and mixed reality devices and applications. These technologies can also be applied to traditional user interfaces on devices such as desktop computers, laptops, tablets, and smartphones.

Apple's patent Figure 2 depicts a top view of a user (#200) whose gaze is focused on an object (#210). The user's line of sight is defined by the visual axis of each eye of the user (as shown by rays 201A and 201B). The direction of the visual axis defines the direction of the user's gaze, and the distance at which the axes converge defines the depth of gaze.

Gaze direction may also be called gaze vector or line of sight. In Figure 2, the gaze direction is in the direction of the object, and the gaze depth is the distance D relative to the user. Gaze direction and/or gaze depth are features used to determine gaze location.

In some embodiments, the center of the user's cornea, the center of the pupil, and/or the center of rotation of the eyeball are used to determine the position of the visual axis of the user's eye. Therefore, this data can also be used to determine the user's gaze direction and/or gaze depth.

In some embodiments, gaze depth is based on the point of convergence of the visual axes of the user's eyes (or the location of the minimum distance between the visual axes of the user's eyes) or some other measurement of the focus of the user's eyes. Definitely. Optionally, gaze depth is used to estimate the distance at which the user's eyes are focused.

Figure 4 of Apple’s patent illustrates a head-mounted display device (HMD) with a built-in gaze sensor. Users will be able to look at a form in a VR world or a real-world environment and direct their text input to specific areas of the form as the gaze sensor detects the user's focus. The technology is so precise that tiny gaps in gaze from the "Name" input slot to the next box "Last Name" are accurately detected, allowing users to fill in the boxes without using a mouse.

Apple states that the gaze sensor (#410) faces the user and, during operation, captures characteristics of the user's eye gaze, such as image data of the user's eyes.

In some embodiments, the gaze sensor includes an event camera that detects event data from the user (e.g., the user's eyes) based on detected changes in light intensity over time and uses the event Data to determine gaze direction and/or gaze depth.

The HMD uses both image data and event data to determine gaze direction and/or gaze depth. Optionally, the HMD uses ray casting and/or cone casting to determine gaze direction and/or gaze depth. In some embodiments, multiple gaze sensors are used.

AR also has other patents from large companies.

Last weekend, a new smart glasses patent from Google was exposed. Google noted in their patent filing that smart glasses could add information alongside what the wearer sees through the glasses. Information (such as digital images) can be overlaid into the user's field of view through smart optical devices, such as optical head-mounted displays (OHMD), or transparent head-up displays (HUD) embedded in wireless glasses, or augmented reality (AR) devices. Modern smart glasses are actually wearable computers that can run standalone mobile applications. Some may be hands-free and communicate with the internet via natural language voice commands, while others may use touch buttons.

In addition to Google, the World Intellectual Property Organization also announced a Samsung AR smart glasses patent last Thursday.

The patent focuses on how their glasses will incorporate communication antennas.

In general, ar companies are now making efforts, looking forward to more breakthroughs and surprises, and can enter the general c-end as soon as possible.