Facebook's Lumos AI Tool Lets Users Search for Images by Their Content

Last year Facebook introduced a feature to its mobile app that made use of artificial intelligence to help visually challenged people identify the contents of an image. Now the social media networking giant has figured out that this technology can have more utility, as it now allows the use of AI to help users search for images based on their content.

Facebook is not the first company to provide this sort of search functionality through its app, Google Photos has done it for years, but this can be seen as a valuable feature addition from the company. With the help of Facebook's Lumos AI technology, users will be able to search through images by their content instead of the usual text-based process that requires the images to be sufficiently tagged or captioned in first place.

Facebook Open Sources Its Object Recognition Machine Vision Software

"We've pushed computer vision to the next stage with the goal of understanding images at the pixel level. This helps our systems do things like recognise what's in an image, what type of scene it is, if it's a well-known landmark, and so on. This, in turn, helps us better describe photos for the visually impaired and provide better search results for posts with images and videos," Facebook's Director of Applied Machine Learning Joaquin Quinonero Candela said in a blog post on Thursday.

In order to make sure that the engineers building machine learning pipelines didn't have to worry about provisioning machines or deal with scaling their service for real-time traffic, Facebook designed a system called FBLearner Flow. Candala says that the company is now conducting 1.2 million AI experiments on this particular system now.

"As this platform has become more widely used, we've continued to build on top of it. From tools to automate the process of machine learning to dedicated content understanding engines, we've built an active ecosystem that allows engineers to write training pipelines that parallelise over many machines so it can be reused by any engineer at the company," he added.

Lumos, which was built on top of FBLearner Flow, was developed for image and video understanding. "The Lumos platform keeps improving all the time, both through all the newly labeled data we feed it, and through the annotated data from the applications our teams build" he said. Candala even says that the company is looking into what AI can do with video format as well.

The image recognition should be available to users in US but there has been no word for other regions as of now, The Next Web points out in its report. It will be interesting to see how much the machine learning improves going ahead and whether Facebook is able to find more ways of implementing its machine learning tools to make its application more user-friendly.