VR helmet that is designed to have private and secure conversations face-to-face in a virtual environment.

Emmanuele Leggio has a dream. During his economics studies, he asked himself how to validate the knowledge gathered through online courses as they were gathered in a brick University. He has the vision to combine the advantages of remote learning with exams face to face with a professor. His solution: a virtual reality communication device with a focus on privacy and integrity.

When Emmanuele presented us his idea we were not sure if it is possible to build such a complex system in just a few days. In contrast to most existing virtual reality headsets, it has to record the face of the user and portray it in the virtual reality environment, while the other virtual reality headset only shows the virtual reality environment. We need to create a virtual environment for a professor and a student where privacy and integrity are very important since the student shouldn’t be able to communicate with anyone else than the professor.

We directly started with breaking down his idea into pieces and tried to figure out which of them are worth validating and which have already been validated by others. Luckily for us, we were able to reduce the core functionality: We need to figure out how we can record the user’s face in an encapsulated environment while the user at the same time sees the virtual world. As we finish set the goal for the development of this MVP, we can start on the development process.

The goal for the breeding and therefore the hypothesis was clear: The device should make sure the integrity is assured and that the two persons in the conversation are the two they claim to be.

Approach

During the design thinking workshop, different ideas came up to enhance privacy, integrity and comfort of the solution. Finally, we ended up with solutions either containing a helmet with a display and cameras for recording facial expressions or a phone booth like box with displays and cameras. The latter one would have introduced some additional problems when going to further iterations in order to use lenses for making a stereoscopic image. So, we went for the helmet with display and cameras or how we called it: “The head mounted selfie stick”

Saturday afternoon we already had a very first version of our MVP. As a base we needed something to attach a display and cameras to the head. We bought a motorcycle helmet for less than CHF 50. Those helmets are very stable. We removed the front shield and attached 3D printed parts to have a base structure for our system. On this structure, we mounted multiple webcams which will be used for recording the facial expression. This was the base setup we needed to get started on the software side. We had to figure out which is the minimal distance between cameras and head and how many cameras to use in total.

We were finally able to use laser cutters for one of our breedings. They allow us to go much more into detail since their resolutions are much higher than the one of a 3D printer. Additionally, they also work much faster than 3D printers. And for our work, time is the most limiting factor.

There isn’t an embedded computer for powering the cameras, the 5-inch display or doing the processing. For the sake of simplicity, we just used a 2017 gaming notebook, so we didn’t have to care too much about limited computing power.

On the software side, we hacked together a Python environment using OpenCV for taking images with multiple webcams. We experimented with different ways of improving the field of view, reducing the distortion by calibrating the cameras. Additionally, we were looking into ways to create a 3D image of the user’s face.

We calibrated the cameras for stereo vision which gives us different matrices for the rotation, translation and distortion of the individual cameras. Before we were able to do that, we had to disassemble the cameras to change the focus. We quickly verified our software results. On the software, the horizontal distance of the cameras was set to 3.7cm and we measured almost the same value with a ruler.

Our final MVP used a lot of laser cut parts, especially to mount the cameras and the display. On the display, you see a virtual office room with a big oval shape showing the processed image of the webcams.

On Sunday our focus was on testing different setups and lighting conditions. Especially since the final product should be encapsulated.

Results

This is the final MVP we built just after one weekend. For those of you interested in how the very first Oculus prototype looked like here is the link.

We wish the best to the project and to succeed.

If the project raised your interest, at the current stage the project needs a CTO and developers to growth. You will find more information on the current project homepage and here.

‍