WHAT IS COMPUTER VISION?

What Is computer vision

Computer vision is an interdisciplinary scientific field that deals with how computers can gain high-level understanding from digital images or videos. From the perspective of engineering, it seeks to understand and automate tasks that the human visual system can do. 

Computer vision is a field of artificial intelligence (AI) that enables computers and systems to derive meaningful information from digital images, videos and other visual inputs — and take actions or make recommendations based on that information. If AI enables computers to think, computer vision enables them to see, observe and understand.

Computer vision works much the same as human vision, except humans have a head start. Human sight has the advantage of lifetimes of context to train how to tell objects apart, how far away they are, whether they are moving and whether there is something wrong in an image.

Computer vision trains machines to perform these functions, but it has to do it in much less time with cameras, data and algorithms rather than retinas, optic nerves and a visual cortex. Because a system trained to inspect products or watch a production asset can analyze thousands of products or processes a minute, noticing imperceptible defects or issues, it can quickly surpass human capabilities.

Computer vision is used in industries ranging from energy and utilities to manufacturing and automotive – and the market is continuing to grow. It is expected to reach USD 48.6 billion by 2022.

WHAT IS COMPUTER VISION ENGINEERING

Computer vision engineering lies at the intersection of artificial intelligence and machine learning. A computer vision engineer’s purpose is to help computers “see” – through the use of deep/machine learning and mathematical architectures in code.

A computer vision engineer, or CV engineer, is also a computer science professional who often uses software to handle the processing and analysis of large data populations in an effort to support the automation of predictive decision-making through visuals.

A computer vision engineer, or CV engineer, is also a computer science professional who often uses software to handle the processing and analysis of large data populations in an effort to support the automation of predictive decision-making through visuals

The history of computer vision

Scientists and engineers have been trying to develop ways for machines to see and understand visual data for about 60 years. Experimentation began in 1959 when neurophysiologists showed a cat an array of images, attempting to correlate a response in its brain. They discovered that it responded first to hard edges or lines, and scientifically, this meant that image processing starts with simple shapes like straight edges.

At about the same time, the first computer image scanning technology was developed, enabling computers to digitize and acquire images. Another milestone was reached in 1963 when computers were able to transform two-dimensional images into three-dimensional forms. In the 1960s, AI emerged as an academic field of study, and it also marked the beginning of the AI quest to solve the human vision problem.

1974 saw the introduction of optical character recognition (OCR) technology, which could recognize text printed in any font or typeface.Similarly, intelligent character recognition (ICR) could decipher hand-written text using neural networks. Since then, OCR and ICR have found their way into document and invoice processing, vehicle plate recognition, mobile payments, machine translation and other common applications.

In 1982, neuroscientist David Marr established that vision works hierarchically and introduced algorithms for machines to detect edges, corners, curves and similar basic shapes. Concurrently, computer scientist Kunihiko Fukushima developed a network of cells that could recognize patterns. The network, called the Neocognitron, included convolutional layers in a neural network.

By 2000, the focus of study was on object recognition, and by 2001, the first real-time face recognition applications appeared. Standardization of how visual data sets are tagged and annotated emerged through the 2000s. In 2010, the ImageNet data set became available. It contained millions of tagged images across a thousand object classes and provides a foundation for CNNs and deep learning models used today. In 2012, a team from the University of Toronto entered a CNN into an image recognition contest. The model, called AlexNet, significantly reduced the error rate for image recognition. After this breakthrough, error rates have fallen to just a few percent.

Why is Computer Vision important?

Computer vision is a rapidly popularized field of artificial intelligence that is becoming increasingly used in technology industries and startups. Computer vision technology relates to computers not only being able to visualize images but also extracting the message or purpose of that image, such as determining distances and movements of incoming objects.

Computer vision companies and startups , introduced AI vision successfully in medicine, defense, manufacturing, and various types of monitoring. Common use cases of computer vision include biotechnology, where it is used for skin cancer detection, gene editing, and more. In addition, AI vision has broken invaluable ground in the medical industry, for example, where it is an extremely useful technology in diagnosis implementations.

Computer vision is also useful for numerous other applications, such as in sports, retail, agriculture, transportation, manufacturing, and more.

Computer vision examples

Many organizations don’t have the resources to fund computer vision labs and create deep learning models and neural networks. They may also lack the computing power required to process huge sets of visual data. Companies such as IBM are helping by offering computer vision software development services. These services deliver pre-built learning models available from the cloud — and also ease demand on computing resources. Users connect to the services through an application programming interface (API) and use them to develop computer vision applications.

IBM has also introduced a computer vision platform that addresses both developmental and computing resource concerns. IBM Maximo Visual Inspection includes tools that enable subject matter experts to label, train and deploy deep learning vision models — without coding or deep learning expertise. The vision models can be deployed in local data centers, the cloud and edge devices.

While it’s getting easier to obtain resources to develop computer vision applications, an important question to answer early on is: What exactly will these applications do? Understanding and defining specific computer vision tasks can focus and validate projects and applications and make it easier to get started.

Here are a few examples of established computer vision tasks:

• Image classification sees an image and can classify it (a dog, an apple, a person’s face). More precisely, it is able to accurately predict that a given image belongs to a certain class. For example, a social media company might want to use it to automatically identify and segregate objectionable images uploaded by users.
• Object detection can use image classification to identify a certain class of image and then detect and tabulate their appearance in an image or video. Examples include detecting damages on an assembly line or identifying machinery that requires maintenance.
• Object tracking follows or tracks an object once it is detected. This task is often executed with images captured in sequence or real-time video feeds. Autonomous vehicles, for example, need to not only classify and detect objects such as pedestrians, other cars and road infrastructure, they need to track them in motion to avoid collisions and obey traffic laws.
• Content-based image retrieval uses computer vision to browse, search and retrieve images from large data stores, based on the content of the images rather than metadata tags associated with them. This task can incorporate automatic image annotation that replaces manual image tagging. These tasks can be used for digital asset management systems and can increase the accuracy of search and retrieval.

Examples of Computer Vision in Mobile Apps

As we’ve already stated, technology has finally progressed to the point where mass usage of computer vision has become viable. Mobile apps have become powerful and complex enough to facilitate computer vision applications in ways that are already affecting your life. Real time reaction and learning have provided a means for a variety of new user experiences, both for fun and practicality. For example:

1. Snapchat filters

Snapchat is one of the most notable appliers of computer vision for entertaining their user base. From dog ears to flower crowns to rainbow waterfalls, Snapchat provides a variety of ways to alter your face. This is, of course, possible through the relatively recent advent of a computer vision application that can manipulate images in real time.

The filters work by Snapchat analyzing your face and, in a matter of seconds, recognizing and quantifying your features and structures. The human face has a few landmarks that provide excellent jumping off points for this process, including your nose, mouth, eyes, and eyebrows. Once your face has been mapped out, Snapchat draws on its deep learning to equate your features to an “average face”.

The “average face” is the most important part for the real time filters, as the computer vision creates a mesh that overlaps with your facial structure. From there, the algorithm can react and manipulate its selection of filters to correspond to how your face changes.

2. Amazon Go

Imagine a convenience store where the shopping process has been perfectly streamlined. You walk in, grab what you need, and walk out without ever bothering with a cashier. While that may at one time have been more of science fiction than reality, Amazon has delivered, through the power of computer vision and machine learning, exactly that.

Amazon Go is a collaboration of app and store, you’ll need one to get into the other. It uses computer vision to keep track of stock, maintenance, and every customer in the store to ensure security and effectiveness. Their cameras and sensors, located around the store, detect and connect everyone in the store to their Amazon account, while simultaneously keeping stock of every item that each customer is currently carrying.

In a nutshell, it’s impressive and only attainable through this specific AI technology. As soon as you’ve finished shopping, you can walk straight out the door and Amazon will automatically charge your account for everything you’ve taken with you.

3. Pinterest Lens

Rather than focusing on real time movement like Amazon and Snapchat have gone for, Pinterest focuses on what it does best: connecting you with your interests. All it takes is snapping a photo of something you like in the world, such as a car, plant, or artwork, and Pinterest Lens immediately routes you towards anything inspired by that interest.

As always, AI technology is the critical component to making this work, and does so through a comprehensive deep learning computer vision backlog. Pinterest is nothing but images, an enormous catalogue of information that feeds and informs their algorithm. Said algorithm deconstructs, analyzes, and then compares the image you took with thousands of others on Pinterest and across the web.

4. Amazon Echo Look

Rather than focusing on music and audio functions like others in Amazon’s Echo line of products, the Amazon Echo Look is dedicated to fashion. This includes voice-activated camerawork, requested styling advice for your outfits, and detailed cinematography for capturing the best picture.

Chances are you can see where computer vision comes into this. Not only does the Echo Look analyze your outfits while affecting your surroundings to create a photogenic likeness, its AI components even help you accentuate your look. It also keeps track of what’s in your wardrobe, categorizes your clothing, and suggests what you can buy from Amazon to complete your look.

The Echo Look algorithm derives its deep learning knowledge to leverage the experiences and feedback gathered from its consumers to build a stronger network dedicated to fashion design and stylization. It needs to take numerous factors into account to get this right: size, skin tone, color, what’s available, so on so forth. Computer vision and machine learning are what make it all possible.

Who is a Computer Vision Engineer?

Computer vision engineers apply computer vision and machine learning research to solve real-world problems. Their work uses large sums of data and statistics in order to complete complex tasks and supervised learning as part of computer vision tasks. Also, CV engineers are tasked with spending much of their time researching and implementing machine learning and computer vision systems for their client companies and overarching corporations.

The engineers work closely with other personnel, often in fields outside of computer science, to facilitate the implementation of novel embedded architectures in existing programs and devices. Computer vision engineers generally have a significant amount of experience with a variety of systems, such as image recognition, machine learning, networking and communication, deep learning, artificial intelligence, computations, data science, and image/video segmentation.

What does a Computer Vision Engineer do?

Computer vision engineers are able to automate various functions using programming that the human visual system can do to fulfill a task, like creating the adaptive cruise control features on a car.

The tasks required of computer vision engineers often involve skills dependent on linear algebra math libraries and a foundational understanding of algorithms and mathematical processes.

Furthermore, prosperous CV engineers will need to have various software skills in the areas of database management, development environment, and component or object-oriented software and programming languages.

Computer vision engineers are often asked to multitask and focus on more than one objective at once. The job requires working efficiently in a collaborative setting. Like many other careers in computer science, computer vision engineering requires high levels of self-motivation and the ability to coordinate with other teammates.

Analytical and critical-thinking skills are important because these engineers work on complex problems and must be able to analyze results for making accurate conclusions. Logical thinking, clear reasoning, and being detail-oriented are critical skills in a computer vision engineer position because of the short deadlines and amount of research and programming-related work required of computer vision engineers.

How does computer vision work?

Computer vision needs lots of data. It runs analyses of data over and over until it discerns distinctions and ultimately recognize images. For example, to train a computer to recognize automobile tires, it needs to be fed vast quantities of tire images and tire-related items to learn the differences and recognize a tire, especially one with no defects.

Two essential technologies are used to accomplish this: a type of machine learning called deep learning and a convolutional neural network (CNN).

Machine learning uses algorithmic models that enable a computer to teach itself about the context of visual data. If enough data is fed through the model, the computer will “look” at the data and teach itself to tell one image from another. Algorithms enable the machine to learn by itself, rather than someone programming it to recognize an image.

A CNN helps a machine learning or deep learning model “look” by breaking images down into pixels that are given tags or labels. It uses the labels to perform convolutions (a mathematical operation on two functions to produce a third function) and makes predictions about what it is “seeing.” The neural network runs convolutions and checks the accuracy of its predictions in a series of iterations until the predictions start to come true. It is then recognizing or seeing images in a way similar to humans.

Much like a human making out an image at a distance, a CNN first discerns hard edges and simple shapes, then fills in information as it runs iterations of its predictions. A CNN is used to understand single images. A recurrent neural network (RNN) is used in a similar way for video applications to help computers understand how pictures in a series of frames are related to one another.

Advantages of Computer Vision

The computer vision advantages that come with the territory fall under a fascinating amount of headers. Nearly every sector, both private and public, can benefit from using computers to track, analyze, and interpret the world around them. As more powerful organizations come to realize what computer vision and machine learning can bring to the table, the more we’ll see this AI technology affecting our lives.

Improved Online Merchandising

Online merchandising has traditionally relied on tagging to find what the customer is working for. A product, such as a backpack, may come attached with various keywords like “bag,” “blue,” “polyester,” or “cotton” to name a few to help narrow down the search to the right one.

It’s not the most efficient system, but it’s what we’ve been working with for years. However, computer vision helps loosen up that process, making it easier and more accessible for customers to find exactly what they’re looking for.

Rather than rely on tags to rotate between different styles of product, computer vision instead compares the actual physical characteristics in each image. This application means customers will be able to find search via images to find similar styles to what they’re looking for.

Unique Customer Experiences

Services like Snapchat and Animoji are aimed to provide an experience that can only be considered “unique.” The goal is provide an appealing, entertaining, intuitive product for consumers to return to. Computer vision, especially in facial mapping, augmentation, and manipulation, was unheard of in the mainstream market up until recently.

Real-world Product and Content Discovery

As Pinterest Lens exemplifies, concepts across the entire internet and even the real world can become connected through the power of computer vision. A single photograph of anything you’d like opens up a search that brings your interests directly to your doorstep.

Whether you’re looking to buy a similar product or discover new ideas similar to what you’re looking for, services like Pinterest Lens and Facebook can bring that experience to you.

Seamless Store Experiences

Amazon has already demonstrated this concept to full effect. No more waiting in long lines, dealing with cashiers, or worrying about handling your wallet when it comes time to pay. The store experience, amplified with computer vision, creates a seamless, efficient environment to do you shopping in. The keyword here is convenience, both for the customer and the company.

Augmented Reality

When Google Glass came out, it was hallmarked as being the next big innovation in how technology impacts our daily lives. Granted, Google Glass wasn’t the greatest success story, and it wasn’t off the mark. Augmented reality is the concept of overlaying our daily lives with information provided by the internet and our phones.

For instance, let’s say you wanted to buy a new bike. Rather than going through the time consuming task of searching for information on that bike, computer vision can use augmented reality to provide reviews, facts, and stats about the product immediately.

Services like Google Translate are already making use of this function, providing a means of translating language in real time on your phone. Other companies, like Apple, are diving into the possibilities as well, researching the potential that augmented reality can bring.

Disadvantages of Computer Vision

While plenty of laurels rest on the head of the future for computer vision, every new innovation has its drawbacks. The computer vision disadvantages regard a hefty issue in the modern age: privacy.

The driving force that makes computer vision as effective as it is is the same issue that lead consumers to doubt whether it should be pursued. By gathering and learning from thousands and thousands of photos, videos, and other pieces of information, everything you do is stored online somewhere, owned by corporations or freely visible to everyone.

With the ability to recognize people’s faces, as well as track their whereabouts and habits, computer vision has changed the future of privacy. As this AI technology becomes more prevalent, users will need to become more aware of what sort of data they put out into the world. Computer vision searches and analyzes countless images and videos, and chances are that means you’re going to be in some of them.