,

AI frontdesk – improve office security and working conditions

Imagine that someone in your office serves as doorkeeper, takes care of visitors and even cares about your working conditions, 24-7? One of our missions at Ailabs.tw is to explore AI solutions to address society’s problems and improve the quality of life of people and, we have developed one AI-powered front-desk to do all of the tasks mentioned above.

Based on 2016 annual report from Taiwan MOL (Ministry of Labor), the average work hours per year of Taiwanese employee is 2106 hours. Compared with OECD stats, this number ranked No.3 in the world which is just below Mexico and Costa Rica.

Recently on 4th, December, 2017,  the first review of the Labor Standards Act revision was passed. The new version of the law will allow flexible work-time arrangements and expand monthly maximum work hours up to 300. Other major changes of the amendment includes conditionally allowing employees to work 12 days in a row and reduction of a minimum 11 hour break between shifts down to 8 hours. The ruling party plans to finish second and third-reading procedure of this revision early next year (2018), and it will put 9-million Taiwanese labors in worse working environment.To get rid off the bad reputation of “Taiwan – The Island of Overwork “, a system which will notify both employee and employer that one has been extremely over-working, and the attendance report can not easily be manipulated is needed.

In May 2017, an employee Luo Yufen from Pxmart, one of Taiwan’s major supermarket chain, died from a long time of overwork after 7 days of being in the state of coma. However, the OSHA(Occupational Safety and Health Administration) initially find no evidence of overwork after reviewing the clocking report provided by Pxmart which looks ‘normal’. It wasn’t until August, when Luo’s case are requested for further investigation, that the Luo’s real working hours before her death proves her overwork condition.

Read more

The Road to Understand Aerial Cinematography

How does AI picks out the best views and shoot an aerial documentary? At Ailabs, we’ve built a system, equipped a drone and a 360 camera, to have an eye of a videographer.

We want to enable filmmaking to achieve camera movement and tracking shot by artificial intelligence itself. We design our system to be able to pick interesting object and features such as lighthouse or coastline from a 360-degree video and create a flat, standard documentary without manual control of camera movement and angle.

 

The drone carries a 360-degree 4K camera and flies alongside Tainan city, Taiwan, to collect videos that are later used for post editing done by our system.

 

This project is inspired by Chi Po-lin‘s documentary film “Beyond Beauty: Taiwan from Above (看見台灣) which captures Taiwan completely in aerial cinematography and broke the Taiwan box office records for the largest opening weekend and the highest total gross of a locally produced documentary. Unfortunately, Chi died in a helicopter crash in a mountainous area in Hualien County’s when the group was shooting footage for the sequel.

Chi’s pointed out it is due to the poor image quality and the monotone of the camera movement that he did not consider to make film by drones. Besides, flying on helicopters for aerial cinematography put photographer in tremendous danger. For this reason, we started the “Chi Po-Lin project” using a drone, a 360 camera and our AI-powered post-editing software algorithm.

Automatic Cinematography for 360

When using a helicopter for videography, the pilot flies the route, and the videographer operates the video camera respectively. In the case of drone,  the videographer is replaced by a 360 camera and an algorithm in post-editing to determines where to focus on in the 360 images.

 

 

As in the scenario above, we installed a 360 camera on the drone to let the AI control the perspective in the 360 image and render a portion of the 360 image to create virtual camera movements such as pan, tilt, and zoom.

 

Algorithmically controlled perspectives from the 360 images 

 

A reason for the monotone of the aerial videos is that it is very difficult for a person to manipulate the controls of the drone and consider the composition details at the same time. Moreover, the conventional methods of automatic cinematography simplify the problem to aligning the camera to the center of the object of interest. So, we give the control of the camera to the AI which, unlike the existing method, takes into account the composition and the semantic flow of the scene.

360 cameras record every point of view. After the videos are collected, the AI recognizes the scenes and objects of interest encountered during the flight, selects the best angles of each moment, and automatically plans multiple sets of suitable trajectories to assist the user in editing the movie.

Automatic Color Enhancement

In order to enhance image quality, we start from color enhancement. We want our video looks as appealing and clearly as professional flim producer took. We leveraged a model which can learn color enhancement by the original input photo and high quality HDR image. The model is learned in an unsupervised way with GAN. In order to reduce model complexity and speed up training, the model is trained with low-resolution image and thus can only output low-resolution image. We extend the model to support high-resolution image by patch-based method. We divide high-resolution image to several overlapping patches and use alpha blending to stitch them. Although it is a image enhancement model, the model is stable enough that we can directly apply it on frames from video without temporal problem. The result looks more appealing than original video in color aspect. According to detail enhancement, result video have more detail than original. For example, the sky looks more clearly than original input video.

Comparison video

Final result with automatic color enhancement

Partners:

– AILabs (台灣人工智慧實驗室)
– Southern Taiwan Science Park Bureau (科技部南科管理局)
– Tainan City Government (台南市政府)
– Department of Aeronautics and Astronautics, NCKU (成大航太)
– GEOSAT Aerospace & Technology Inc (經緯航太)

Sponsored:

– Microsoft
– Nvidia
– Garmin

 

Featured image by YELLOW Mao. 黃毛, Photographer / CC BY