Group items tagged

love this one ... it seems to take physicist to explain to the AI crowd what they are actually doing ... Deep learning is a broad set of techniques that uses multiple layers of representation to automatically learn relevant features directly from structured data. Recently, such techniques have yielded record-breaking results on a diverse set of difficult machine learning tasks in computer vision, speech recognition, and natural language processing. Despite the enormous success of deep learning, relatively little is understood theoretically about why these techniques are so successful at feature learning and compression. Here, we show that deep learning is intimately related to one of the most important and successful techniques in theoretical physics, the renormalization group (RG). RG is an iterative coarse-graining scheme that allows for the extraction of relevant features (i.e. operators) as a physical system is examined at different length scales. We construct an exact mapping from the variational renormalization group, first introduced by Kadanoff, and deep learning architectures based on Restricted Boltzmann Machines (RBMs). We illustrate these ideas using the nearest-neighbor Ising Model in one and two-dimensions. Our results suggests that deep learning algorithms may be employing a generalized RG-like scheme to learn relevant features from data.

A good historical overview of the Deep Learning revolution. If you think the quote above is an exageration, here are some fresh news from Microsoft: Internal email: Microsoft forms new 5,000-person AI division

Implementation of the deep learning-based image classifier (online). Try making a picture with your phone and upload it there. Pretty impressive results. EDIT: Okay, it works the best with well exposed simple objects (pen, mug).

New optics based computer architecture

This has been around for over a year. The current trend in deep learning is "deeper is better". But a consequence of this is that for a given network depth, we can only feasibly evaluate a tiny fraction of the "search space" of NN architectures. The current approach to choosing a network architecture is to iteratively add more layers/units and keeping the architecture which gives an increase in the accuracy on some held-out data set i.e. we have the following information: {NN, accuracy}. Clearly, this process can be automated by using the accuracy as a 'signal' to a learning algorithm. The novelty in this work is they use reinforcement learning with a recurrent neural network controller which is trained by a policy gradient - a gradient-based method. Previously, evolutionary algorithms would typically be used. In summary, yes, the results are impressive - BUT this was only possible because they had access to Google's resources. An evolutionary approach would probably end up with the same architecture - it would just take longer. This is part of a broader research area in deep learning called 'meta-learning' which seeks to automate all aspects of neural network training.

Btw that techxplore article was cringing to read - if interested read this article instead: https://research.googleblog.com/2017/05/using-machine-learning-to-explore.html

Winter is coming... and deep learning, too!

"Sergey Brin has said he wants to build a benign version of HAL in 2001: A Space Odyssey" ... didn't they try to do that in that movie called "2001: A Space Odyssey" ?

Physicists from MIT explain why deep learning is that much successful using physics.

An intelligent (that is, self-adaptive) cloak driven by deep learning. Quite cool, eh? Something we can get inspired by?

In a world first, an artificial intelligence machine plays chess by evaluating the board rather than using brute force to work out every possible move. It's been almost 20 years since IBM's Deep Blue supercomputer beat the reigning world chess champion, Gary Kasparov, for the first time under standard tournament rules.

The disadvantage in this kind of engine lies exactly in its inability to extrapolate. You might actually be able to beat it if you play like an idiot.

Genetic algorithms are a competitive alternative for training deep neural networks for reinforcement learning. For paper see: https://arxiv.org/pdf/1712.06567.pdf

Interesting pointers in this one! I would like to explore neuroevolution as well, although it seems extremely resource-demanding?

Not necessarily, I think it can be made to be much faster hybridizing it with backprop and Taylor maps. Its one ideas in the closet we still have not explored (Differential Intelligence: accelerating neuroevolution).

A new idea called the "information bottleneck" is helping to explain the puzzling success of today's artificial-intelligence algorithms - and might also explain how human brains learn.

Software that roughly mimics the way the brain works could give smartphones new smarts-leading to more accurate and sophisticated apps for tracking everything from workouts to emotions. The software exploits an artificial-intelligence technique known as deep learning, which uses simulated neurons and synapses to process data.

We humans take for granted our remarkable ability to predict things that happen around us. Here, a deep learning model trained from real-world videos and a 3D graphics engine was able to infer physical properties of objects against humans.

There we go: Deep Learning predicts disruptions in plasmas. The paper related to this article is here: https://arxiv.org/abs/1802.02242

Scientific use cases show promise, but challenges remain for complex data analytics.

Neural Networks to analyse sleeplessness.

Dataset automatically constructed from public images uploaded by users on FB/Instagram with hashtags used as labels! They refer to this as 'weakly supervised learning'. Then neural net fine-tuned for ImageNet and achieved record 85.4% accuracy.

Random branches are collected, scanned to 3D, and connected with servos. Then a neural network is trained to control this "robot".

Also a repo here: https://github.com/thunil/Physics-Based-Deep-Learning definitely looks interesting