MixedReality for Low vision
~Navigation for Low vision useing Microsoft HoloLens ~
* In this article, low vision is used to mean anyone with visual difficulties.
●Prologue
What do you draw in MixedReality Worlds?
Before get into the main topic, will describe myself and how this happened.
My friend who has visual handicap said to me “I can feel something” when she put on HoloLens.
Me and she are high school classmates. She’s one eye is blind and other is low eyesight.
She has very active and one of 2020 olympic volunteer group reader .
When tell about my HoloLens story, she immediately replied, “want to experience it.”
Then about a year I learned a lot from her as a developer,as a xRcreator, as a adult.
The other day, in her connection, I’ve been exploring the potential of HoloLens with two Low vision people.
●What’s xR technology can do for Low vision?
The definition of visual impairment is reduced vision not corrected by glasses or contact lenses.
It is said that more than 61milion people live in world.
There are many useful technology they can use and help themself. A typical example is a smartphone. Many applications have been released that can be used with or without handicap.
My friend I mentioned at the prologue uses Seeing AI.
●Seeing AI
Seeing AI can recognize images on a smartphone and read out the surrounding conditions, colors and light intensity by sound.
If you’ve registered your loved one, you’ll can hear what kind of look she’s emotion at and how many meters away by your smartphone.
●Demonstration experiments at actual stations using HoloLens
●Date:2020/2/7
● Place:Kitasenjyu Sta. in Tokyo (Japan)
● Number:2 person (including people who has diagnosed as blind)
●DemonstrationApps
I was prepare very simple Apps used SpatialSounds.
prepared five spheres, each of which have a separate sound source.
These can be freely arranged in space using gesture. Also, AoudiOclusion is set for SpatialMesh.
In the experiment, I first attached a sphere to a suitable wall in the station yard.
Also, one sphere was placed on the pillar as a virtual obstacle.
In this state, I asked the cooperating person with low vision to equip HoloLens and check if I can walk to the target wall avoiding obstacles.
●Spatial Sound
SpatialSound is a sound feature that allows the user to recognize the position of the hologram.
It can be used simply by setting SpatialBlend of Unity AudioSource to 3D.
SpatialSounds lets the user perceive the position of the object by the change in sound with the left and right ears of the user, but this time adjusts the value of MaxDistance and sets the difference between the change in the left and right sound violently.
This makes it easier for the user to perceive the location of the object. (We set it this way because it was navigation, but over-setting could hurt the dialogue with reality.)
●Result
Both of them avoided the obstacles, and reaching the goal immediately. Without auxiliary tools such as white canes.
●Discussion
Collaborators who had experienced this time have Lowvision.
However, they said “feel something” when approaching the Holohram object.
They tried to touch it by hand as if were looking at the hologram object.
Listening to them story, I realized one stereotype of my own.
~Lowvison people can’t see~
That may be true. but, It’s not necessarily true.
They can feel The light also. of course it depends on the person.
I found the following article the day before the experiment.
In this article, it is described that HoloLens without lens etc. is used.
Also, it looks like that even when looking at the pictures. (Though this may be a reference image).
The most important thing, of course, is that it helps their lives.
But I feel it can should be a universal device as much as possible, not just HoloLens for low vision.
I asked a collaborator. “Does Custom HoloLens for low vision people need a lens?”
Their answer was, “What can do with and without a lens is very different.
In my case, I can feel the light, so I can feel the hologram thanks to the display.”
This is a very valuable word to connect xR and HoloLens with Low Vision people.
Perhaps not necessary for some people. It is heavy.
In that case, I think that it would be good if it could be customized according to the individual.
●Two types of blindness
After the experiment,I talked with my friend who made an opportunity by me this time.
She said to me “There are Two types of visual handicap”
One is medical blindness.
This is a person who can not feel the brightness medically, that is, a person who has completely lost vision
Another is social blindness ie low vision.
She says most people with visual handicap seem to be social blindness.
I asked her. “What’s the biggest problem right now?”
Hers answer is “Moving in an environment that I cannot imagine”.
I’ll also consider from the results of this experiment.
What if could accurately recognize traffic signals and signs in combination with AI, image processing, etc. and overlay them with HoloLens to make them glow?
I asked her.
“What I can do alone grows infinitely.
Even if it is not a signboard, it is enough that the entire display is shining”
I did not use any special HoloLens or any special technology.
However, I realized one possibility of HoloLens.
HoloLens still has endless possibilities.
I am excited to be able to meet this discovery as convey the appeal of HoloLens to various people.
A year ago, when I first had her experience HoloLens, I had a complicated mind.
She can feel and find HoloLens attractive?
She experienced some of the popular apps on HoloLens, but unfortunately didn’t enjoy it.
However, she enjoyed one app very much.
It is an app developed by me.
What’s the difference? It’s not a matter of technical excellence.
The difference was the near clip.
HoloLens has a function called near clip to prevent users from getting buried in the hologram.
Normally it is set to 0.3, objects within 30cm will be clipped.But my app happened to have the near clip set to zero.
This allowed her to enjoy the application by getting close to the objects.
Without this happening, we couldn’t do this kind of experiment.
This can make a huge difference in the quality of the experience when considering different users.
●Conclusion
・ HoloLens has endless possibilities that extend the possibilities of various people.
・Even low vision users do not need to prepare anything special, and there are many things that can be done with HoloLens now.
・By applying HoloLens to people in various positions, various possibilities are created.
By the way, lastly, it is a question to the wonderful you who read this far.
“What kind of world do you draw with HoloLens?”
Thank you!
This experiment was done as an individual.
My past projects are next.
If you have any comments or questions, you can connect me.
