Group items tagged

Deep neural networks "dreaming" psychedelic images

Google released the code behind this, and applications of it are now popping up: - http://googleresearch.blogspot.nl/2015/07/deepdream-code-example-for-visualizing.html - https://imgur.com/a/nVlLe - https://twitter.com/hashtag/deepdream

Funny how representation errors in NNs suddenly become art. God.... neo-post-modernism.

"Hinton's new approach, known as capsule networks, is a twist on neural networks intended to make machines better able to understand the world through images or video. In one of the papers posted last week, Hinton's capsule networks matched the accuracy of the best previous techniques on a standard test of how well software can learn to recognize handwritten digits." Links to papers: https://arxiv.org/abs/1710.09829 https://openreview.net/forum?id=HJWLfGWRb&noteId=HJWLfGWRb

impressive!

seems a very impressive guy :"Hinton formed his intuition that vision systems need such an inbuilt sense of geometry in 1979, when he was trying to figure out how humans use mental imagery. He first laid out a preliminary design for capsule networks in 2011. The fuller picture released last week was long anticipated by researchers in the field. "Everyone has been waiting for it and looking for the next great leap from Geoff," says Kyunghyun Cho, a professor"

Siemens is using neural networks to improve operation of wind turbines, reducing maintenaince needs and improving output by one precent. It seems even that Siemens has quite a large neural network study group, probably linked to german universities, with various examples in practice (see websie)

Explanation of artificial neural networks (explains very nicely the basics of the programming behind Christos' PhD thesis)

Faced with a navigational challenge, neural networks spontaneously evolved units resembling the grid cells that help living animals find their way.

This has been around for over a year. The current trend in deep learning is "deeper is better". But a consequence of this is that for a given network depth, we can only feasibly evaluate a tiny fraction of the "search space" of NN architectures. The current approach to choosing a network architecture is to iteratively add more layers/units and keeping the architecture which gives an increase in the accuracy on some held-out data set i.e. we have the following information: {NN, accuracy}. Clearly, this process can be automated by using the accuracy as a 'signal' to a learning algorithm. The novelty in this work is they use reinforcement learning with a recurrent neural network controller which is trained by a policy gradient - a gradient-based method. Previously, evolutionary algorithms would typically be used. In summary, yes, the results are impressive - BUT this was only possible because they had access to Google's resources. An evolutionary approach would probably end up with the same architecture - it would just take longer. This is part of a broader research area in deep learning called 'meta-learning' which seeks to automate all aspects of neural network training.

Btw that techxplore article was cringing to read - if interested read this article instead: https://research.googleblog.com/2017/05/using-machine-learning-to-explore.html

Its already "old" news but kinda nice...

On Android speech recognition but also with a very nice video: direct translation of English voice input to Chinese audio. Looks like it might be really useful eventually.

Cute info-graphics on different machine learning architectures

Not easily digestible piece of esa document, but to prove Paul's point. And yes, they have already planned to train neural networks on a database of different water types, so that the satellite figures out from the combined retrieval of backscattering and absorption = f(λ) which type of water it is looking at. Type of water relates to οptical clarity of the water, a variable called turbidity. We could do this as well for mapping iron fertilization locations if we find its spectral signature. Lab time?????

Almost in time for the workshop last week! This new Nature paper (e-mail me for full paper) claims training and usage of neural network implemented with metal-oxide memristors, without selector CMOS. They used it to implement a delta-rule algorithm for classification of 3x3 pixel black and white letters. Very impressive work!!!!

For those not that much into the topic, see the Nature's News and View section www.nature.com/nature/journal/v521/n7550/full/521037a.html?WT.ec_id=NATURE-20150507 where they feature this article.

A key technical barrier to furthering our understanding of complex neural networks has been the lack of tools for the simultaneous spatiotemporal control and detection of activity in a large number of neurons.

Other possible study: get a textbook example of an image of a pen, evolve it just enough so NN can't recognize it anymore, while minimizing the distance between the original and evolved images. EDIT: Its been done already: http://cs.nyu.edu/~zaremba/docs/understanding.pdf

Of course, you can't really use them to extrapolate. The unknown unknown is always the trickiest :P They should just make another class "random bullshit", really and dump all of this stuff in there. I think there's a potential paper right there

A personal interest of mine that I want to explore a bit more in the future. I just bought a ZigBee electricity monitor and I am wondering whether from the signal of the mains one could detect (reliably) the oven turning on, lights, etc. Probably requires Neural Network training. The idea would be to make a simple device which basically saves you money by telling you how much electricity you are wasting. Then again, its probably already done by Google...

Randomly I found this: http://spectrum.ieee.org/view-from-the-valley/consumer-electronics/gadgets/curb-startup-home-electricity-tracking Isn't it a similar kind of stuff Thijs?

haha, yep, that's it... but we can do better than that right! :)

There you go, the latest-and-greatest chip is there. Now the only remaining tiny detail - program it.

Let's buy it first and we'll figure the rest out later :P

A hybrid image classification method was employed for this specific purpose by incorporating the advantages of both pixel-based and object-based image classification approaches. This was performed in two steps: firstly, a pixel-based image classification method, i.e., artificial neural network was applied to classify potential targets with similar spectral reflectance at pixel level; and then an object-based image classification method was used to further differentiate animal targets from the surrounding landscapes through the applications of expert knowledge. As a result, the large animals in two pilot study areas were successfully detected with an average count error of 8.2%, omission error of 6.6% and commission error of 13.7%. The results of the study show for the first time that it is feasible to perform automated detection and counting of large wild animals in open savannahs from space

And Paul, it includes neural networks!

I kept telling you guys but you just laughed and laughed :))

Neural Networks to analyse sleeplessness.