This week, Yann LeCun came to speak at M.I.T. His talk was great and, like LeCun, I am a big fan of both deep learning and the vector based embeddings that have become popular in ML. The ability to process math, instead of logic, is indeed powerful. But one thing disturbed me about the talk.
During the Q&A session, LeCun was asked several questions about other technologies, including the Hierarchical Temporal Memory Approach used by companies like Numenta and Cortical.io, and spiking neuron approaches like those used by IBM's Truenorth chip. LeCun's criticism was that neither approach had done well on standard data sets used by the machine learning community, and had not been able to beat any existing benchmarks. This would be a good point were it not coming from a guy who admittedly toiled away on Convolutional Neural Networks for two decades, while they (the CNNs) were unable to beat any existing machine learning benchmarks, and people thought he was wasting his time.
It bothers me because I've seen too many technology waves come and go, and deep learning, while powerful, feels like a fad. People seem to think its the solution to every problem, but it reminds me of AJAX, user generated content, noSQL, containers, and other technologies that were indeed interesting, but not panaceas. What surprises me is the lemming like nature of machine learning research - with both academia and industry taking a bandwagon approach, and criticizing those who don't jump on the bandwagon.
One of the things I admire about LeCun is his conviction in his approach, and his long term view. I hope his attitude doesn't discourage other researchers who share those rare traits.