IT Strategy

Deep Learning Trends to Expect in 2016

The internet is rapidly moving forward to the next level where data will rule everything imaginable. As a result, deep learning (a new field in machine learning) has also started to gain momentum as it’s one of the key aspects of data science.

Tech giants like Google and Facebook have already been investing heavily in deep learning (as it’s going to play a crucial role in future technologies), so expect to hear a lot more about it in the months to come.

Although the idea has been around for decades, it’s only recently that it has started to flourish. Since the 1970s, researchers having been trying to teach algorithms, but by the 1980s computational technology and data just weren’t good enough to make any significant progress. As a result, exploration into artificial intelligence (AI) gradually slowed down.

However, technological advances in recent years have made it much easier to engage in deep learning. It’s a key characteristic of the Internet of Things (IoT) and basically what will drive this phenomenon.

You have probably already used deep learning tools in your everyday life and now known about it. Useful applications like machine translation, speech recognition, and facial recognition are all products of this field. Although the apps that we have today aren’t perfect, we are starting to see that we are not far away from achieving our goals.

So what can we expect from deep learning in 2016?

Rapid Progress

Deep learning has been evolving quite rapidly over the past year and the momentum is expected to continue. A year ago, designing vision networks cost 5-10 times more and required more parameters (15 times more), but now things are cheaper and more accessible.

The major contributors to this transformation are better training methodologies and enhanced network architectures.

Further, experts believe that deep learning will be highly efficient and run on cheap mobile devices, without any extra hardware, within the next decade. It’s a good indication as to what you can expect in the future.

Bigger Architecture, Higher Levels of Complexity

If you take an in-depth look at deep learning, you will notice that architectures are becoming more complex and significantly large. It makes sense as we are working towards developing large neural systems. In these complex systems, we can now pre-train parts of the network on various data sets, switch neural components, tweak it, and add more modules.


According to 5th year Stanford University PhD student, Andrej Karpathy, “Convolutional Networks were once among the largest/deepest neural network architectures, but today they are abstracted away as a small box in the diagrams of most newer architectures.” As a result, you can expect a lot of the present architecture to go the same way as the others with this year’s innovations.

Unsupervised Learning

Over the last couple of years, deep learning has had high levels of success in a wide area of application. Although no one really expects to have human level of cognition by 2020, experts predict a significant improvement over the next few years.

Voice recognition and computer vision basically confirmed that machine learning at higher levels of complexity was possible. As a result, you will probably see more attention placed on semi-supervised or unsupervised training algorithms to handle the large influx of data.

Further, reinforcement-learning approaches will also become more prominent. As only having a supervised approach comes with significant limitations, deep reinforcement learning will play a key role in making unsupervised machine learning more efficient.

Increased Adoption

2016 will be the year where deep learning algorithms will be used for tasks that require solutions for problems. This in turn will accelerate the rate of adoption of deep learning across various industries. A sign of things to come is the recent collaboration of  Google and Movidius who have teamed up to increase adoption (of deep learning technology) within mobile devices.

Further, you can also take an educated guess from observing the number of deep learning projects on GitHub. With all this activity, it is safe to say that there is real promise of achieving a high level of AI in the near future.

Sharing Data & Sharing Tools

Industry leaders are highly motivated to break tradition and share data and tools with each other to accelerate the evolution of deep learning. An example of this is a free deep learning course on Udacity that’s taught by one of Google’s leading scientists, Vincent Vanhoucke. According to Professor Vanhoucke, the “our overall goal in designing this course was to provide the machine learning enthusiast a rapid and direct path to solving real and interesting problems with deep learning techniques, and we’re now very excited to share what we’ve built!

So not only will various enterprises share data and engage in developing the technology, they’re also quickly educating people to obtain the necessary knowledge to help take this technology to the next level.

Although progress was slow for the first few decades, deep learning has really taken off over the past two years. In 2016, expect deep learning to encourage more innovation and take significant leaps forward in the months to come.

What are your views on future deep learning trends? Do you have anything else to add to this list? Please share your thoughts in the comment sections below.

IT Storyteller and Copywriter
Andrew's current undertaking is big data analytics and AI as well as digital design and branding. He is a contributor to various publications with the focus on emerging technology and digital marketing.