AI frontdesk – improve office security and working conditions

Imagine that someone in your office serves as doorkeeper, takes care of visitors and even cares about your working conditions, 24-7? One of our missions at Ailabs.tw is to explore AI solutions to address society’s problems and improve the quality of life of people and, we have developed one AI-powered front-desk to do all of the tasks mentioned above.

Based on 2016 annual report from Taiwan MOL (Ministry of Labor), the average work hours per year of Taiwanese employee is 2106 hours. Compared with OECD stats, this number ranked No.3 in the world which is just below Mexico and Costa Rica.

In May 2017, an employee Luo Yufen from Pxmart, one of Taiwan’s major supermarket chain, died from a long time of overwork after 7 days of being in the state of coma. However, the OSHA(Occupational Safety and Health Administration) initially find no evidence of overwork after reviewing the clocking report provided by Pxmart which looks ‘normal’. It wasn’t until August, when Luo’s case are requested for further investigation, that the Luo’s real working hours before her death proves her overwork condition.

The biggest problem here is OSHA need data to determines the working hours of the employee. This data usually comes from the reports at employer’s hand. Common ways the employer can do for the clocking reports is either making up the report or having employee clock in after they start working.

In Ailabs office, we built a “Face-In” system that not only stays at the front desk to manage completely secure access control, but also provides easier and more reliable time records for payroll.

It has a built-in facial recognition software to identify the person from the camera by matching with the employee database. Once confirmed with your identity, voice command kicks off. The employee interacts with the system by speaking to it such as saying “Open the door” and depends on your roles and authorization, you will either allowed to access the door or one of our employee would be notified about your visit.

The system recognize employee and grant access to the person on a voice command request to ‘open the door’.

Apart from security, it also integrates with time and attendance system that records each employee’s working hour. Attendance data was stored without manually “punching card” into the clock-in machine or the need for cards and badges. Moreover, our system provides these additional services:

• Employee working hour dashboard: Displays employee’s daily/monthly working hours reports and alerts an employee if one is over-worked.
• Visitor notification system: If the system recognizes a guest, it sends an instant message with guest’s information to the employee’s phone and transfer access control to the employee.

This “Face-In” system is expected to solve an essential part of lots of labor dispute — a reliable clocking report to prove employee’s actual time and attendance. Enabling camera to have the intelligence to recognize an employee and a touch-free clocking mechanism, an employee is free from the risk of having clocking reports being intentionally made up.

If the employee is over-worked, our system will gently reminds him/her to care more for the health. The above figure shows the daily and monthly working hour of the employer.

We set up an USB camera and Android box at the front-door to record and detect the appearance of a human face. Image and voice signals are transmitted through RTMP(Real-Time Messaging Protocol) to Ailab’s backend server and processors for further analysis. Calculated results and control signals are sent to front-end display as well as automatic office door lock through bluetooth and wifi-network .

How the system is setup

Facial recognition

Our system uses camera to perform facial recognition and identification. It captures and segments facial features from sequences of images and performs database matching and verification with template extraction. In a real-life scenario, external factors such as pose change or fast motion can corrupt the image quality for face recognition.

The algorithms and data analysis behind the system are powered by our core facial recognition engine and face ID database. For more detail about this facial recognition engine, please visit this blog post.

Recognizing an unknown visitor

From an application’s perspective, it is essential to recognizes a person as someone who is not an employee or someone unknown to the current database. However, a traditional face classification problem always output a prediction from a list of candidates in the database even if the confidence is low. This will be undesirable since the system will always open the door for anyone even if the person is actually a complete stranger.

For this purpose, we define an impurity matrix which takes classification prediction’s confidence scores across candidates into account. It is based on the theory that if the person to be recognized is in our database, the prediction distribution tends to converge to a single candidate. However, if prediction distribution is scattered across multiple candidates, we consider it as the uncertain case and classify it as ‘others’.

Noise resistent model

The voice input quality are often degraded by the physical structure of the surrounding. Echoes, noises and random sounds that happen at the door front can confuse the system to make accurate detection.

We train our ASR (automatic speech recognition) system to performance under a certain level of interferences or noises by simulating a degraded voice signals in the phrase of training to produce a noise resistant ASR model.

Learning to recognize celebrities

Lately, we learned thousands of celebrities’ faces from Google Search, IMDB and Wikipedia that are built into AILab’s face ID database. These celebrities can be instantly recognized by our system and shows his or her public information, which Youtube videos he/she had appeared on and even tells the accurate timing in those video clips.

Next time, when you visit Ailab don’t forget to say hi to our newly-hired AI frontdesk!

Featured image by Michael Semensohn / CC BY