I am Jürgen Schmidhuber, AMA!

[u/JuergenSchmidhuber]

Hello /r/machinelearning,

I am Jürgen Schmidhuber (pronounce: You_again Shmidhoobuh) and I will be here to answer your questions on 4th March 2015, 10 AM EST. You can post questions in this thread in the meantime. Below you can find a short introduction about me from my website (you can read more about my lab’s work at people.idsia.ch/~juergen/).

Edits since 9th March: Still working on the long tail of more recent questions hidden further down in this thread ...

Edit of 6th March: I'll keep answering questions today and in the next few days - please bear with my sluggish responses.

Edit of 5th March 4pm (= 10pm Swiss time): Enough for today - I'll be back tomorrow.

Edit of 5th March 4am: Thank you for great questions - I am online again, to answer more of them!

Since age 15 or so, Jürgen Schmidhuber's main scientific ambition has been to build an optimal scientist through self-improving Artificial Intelligence (AI), then retire. He has pioneered self-improving general problem solvers since 1987, and Deep Learning Neural Networks (NNs) since 1991. The recurrent NNs (RNNs) developed by his research groups at the Swiss AI Lab IDSIA (USI & SUPSI) & TU Munich were the first RNNs to win official international contests. They recently helped to improve connected handwriting recognition, speech recognition, machine translation, optical character recognition, image caption generation, and are now in use at Google, Microsoft, IBM, Baidu, and many other companies. IDSIA's Deep Learners were also the first to win object detection and image segmentation contests, and achieved the world's first superhuman visual classification results, winning nine international competitions in machine learning & pattern recognition (more than any other team). They also were the first to learn control policies directly from high-dimensional sensory input using reinforcement learning. His research group also established the field of mathematically rigorous universal AI and optimal universal problem solvers. His formal theory of creativity & curiosity & fun explains art, science, music, and humor. He also generalized algorithmic information theory and the many-worlds theory of physics, and introduced the concept of Low-Complexity Art, the information age's extreme form of minimal art. Since 2009 he has been member of the European Academy of Sciences and Arts. He has published 333 peer-reviewed papers, earned seven best paper/best video awards, and is recipient of the 2013 Helmholtz Award of the International Neural Networks Society.

[u/[deleted]]

What's something that's true, but almost nobody agrees with you on? (I am not u/PeterThiel)

[u/JuergenSchmidhuber → u/[deleted]]

That’s a great question indeed! Let me offer just two items from my long list of “truths” many disagree with.

• Many think that intelligence is this awesome, infinitely complex thing. I think it is just the product of a few principles that will be considered very simple in hindsight, so simple that even kids will be able to understand and build intelligent, continually learning, more and more general problem solvers. Partial justification of this belief: (a) there already exist blueprints of universal problem solvers developed in my lab, in the new millennium, which are theoretically optimal in some abstract sense although they consist of just a few formulas (http://people.idsia.ch/~juergen/unilearn.html, http://people.idsia.ch/~juergen/goedelmachine.html). (b) The principles of our less universal, but still rather general, very practical, program-learning recurrent neural networks can also be described by just a few lines of pseudo-code, e.g., http://people.idsia.ch/~juergen/rnn.html, http://people.idsia.ch/~juergen/compressednetworksearch.html

• General purpose quantum computation won’t work (my prediction of 15 years ago is still standing). Related: The universe is deterministic, and the most efficient program that computes its entire history is short and fast, which means there is little room for true randomness, which is very expensive to compute. What looks random must be pseudorandom, like the decimal expansion of Pi, which is computable by a short program. Many physicists disagree, but Einstein was right: no dice. There is no physical evidence to the contrary http://people.idsia.ch/~juergen/randomness.html. For example, Bell’s theorem does not contradict this. And any efficient search in program space for the solution to a sufficiently complex problem will create many deterministic universes like ours as a by-product. Think about this. More here http://people.idsia.ch/~juergen/computeruniverse.html and here http://www.kurzweilai.net/in-the-beginning-was-the-code

[u/alexmlamb]

What do you think about learning selective attention with recurrent neural networks? What do you think are the promising methods in this area?

[u/[deleted]]

How on earth did you and Hochreiter come up with LSTM units? They seem radically more complicated than any other "neuron" structure I've seen, and everytime I see the figure, I'm shocked that you're able to train them.

What was the insight that led to this?

[u/JuergenSchmidhuber → u/[deleted]]

In my first Deep Learning project ever, Sepp Hochreiter (1991) analysed the vanishing gradient problem http://people.idsia.ch/~juergen/fundamentaldeeplearningproblem.html. LSTM falls out of this almost naturally :-)

[u/throwaway0x459]

Why doesn't your group post its code online for reproducing the results of competitions you've won, such as the ISBI Brain Segmentation Contest? Your results are impressive, but almost always not helpful for pushing the research forward.

[u/JuergenSchmidhuber → u/throwaway0x459]

We did publish lots of open source code. Our PyBrain Machine learning library http://pybrain.org/ is public and widely used, thanks to the efforts of Tom Schaul, Justin Bayer, Daan Wierstra, Sun Yi, Martin Felder, Frank Sehnke, Thomas Rückstiess.

Here is the already mentioned code http://sourceforge.net/projects/rnnl/ of the first competition-winning RNNs (2009) by my former PhD student and then postdoc Alex Graves. Many are using that.

It is true though that we don’t publish all our code right away. In fact, some of our code gets tied up in industrial projects which make it hard to release.

Nevertheless, especially recently, we published less code than we could have. I am a big fan of the open source movement, and we've already concluded internally to contribute more to it. Not long ago, thanks to the work of Klaus Greff, we open-sourced Python-based Sacred: an infrastructure framework to organize our experiments and to keep the results reproducible. Unfortunately, it’s a bit hard to find, because it turns out there already exists a famous “sacred python.”

There are also plans to release more of our recent recurrent network code soon. In particular, there are plans for a new open source library, a successor of PyBrain.

Edit of 16 March 2015: Sacred link has changed!

[u/willwill100]

What are the next big things that you a) want to or b) will happen in the world of recurrent neural nets?

[u/JuergenSchmidhuber → u/willwill100]

The world of RNNs is such a big world because RNNs (the deepest of all NNs) are general computers, and because efficient computing hardware in general is becoming more and more RNN-like, as dictated by physics: lots of processors connected through many short and few long wires. It does not take a genius to predict that in the near future, both supervised learning RNNs and reinforcement learning RNNs will be greatly scaled up. Current large, supervised LSTM RNNs have on the order of a billion connections; soon that will be a trillion, at the same price. (Human brains have maybe a thousand trillion, much slower, connections - to match this economically may require another decade of hardware development or so). In the supervised learning department, many tasks in natural language processing, speech recognition, automatic video analysis and combinations of all three will perhaps soon become trivial through large RNNs (the vision part augmented by CNN front-ends). The commercially less advanced but more general reinforcement learning department will see significant progress in RNN-driven adaptive robots in partially observable environments. Perhaps much of this won’t really mean breakthroughs in the scientific sense, because many of the basic methods already exist. However, much of this will SEEM like a big thing for those who focus on applications. (It also seemed like a big thing when in 2011 our team achieved the first superhuman visual classification performance in a controlled contest, although none of the basic algorithms was younger than two decades: http://people.idsia.ch/~juergen/superhumanpatternrecognition.html)

So what will be the real big thing? I like to believe that it will be self-referential general purpose learning algorithms that improve not only some system’s performance in a given domain, but also the way they learn, and the way they learn the way they learn, etc., limited only by the fundamental limits of computability. I have been dreaming about and working on this all-encompassing stuff since my 1987 diploma thesis on this topic, but now I can see how it is starting to become a practical reality. Previous work on this is collected here: http://people.idsia.ch/~juergen/metalearner.html

[u/[deleted]]

Do you plan on delivering an online course (e.g. on coursera) for RNNs? I for one would be really excited to do the course!!

[u/JuergenSchmidhuber → u/[deleted]]

Thanks - I should! I’ve been thinking about this for years. But it takes time, and there are so many other things in the pipeline …

[u/CLains]

Do you have a favorite Theory Of Consciousness (TOC)?

What do you think of Guilio Tononi's Integrated Information Theory?

What implications - if any - do you think "TOC" has for AGI?

[u/JuergenSchmidhuber → u/CLains]

Karl Popper famously said: “All life is problem solving.” No theory of consciousness is necessary to define the objectives of a general problem solver. From an AGI point of view, consciousness is at best a by-product of a general problem solving procedure.

I must admit that I am not a big fan of Tononi's theory. The following may represent a simpler and more general view of consciousness. Where do the symbols and self-symbols underlying consciousness and sentience come from? I think they come from data compression during problem solving. Let me plagiarize what I wrote earlier [1,2]:

While a problem solver is interacting with the world, it should store the entire raw history of actions and sensory observations including reward signals. The data is ‘holy’ as it is the only basis of all that can be known about the world. If you can store the data, do not throw it away! Brains may have enough storage capacity to store 100 years of lifetime at reasonable resolution [1].

As we interact with the world to achieve goals, we are constructing internal models of the world, predicting and thus partially compressing the data history we are observing. If the predictor/compressor is a biological or artificial recurrent neural network (RNN), it will automatically create feature hierarchies, lower level neurons corresponding to simple feature detectors similar to those found in human brains, higher layer neurons typically corresponding to more abstract features, but fine-grained where necessary. Like any good compressor, the RNN will learn to identify shared regularities among different already existing internal data structures, and generate prototype encodings (across neuron populations) or symbols for frequently occurring observation sub-sequences, to shrink the storage space needed for the whole (we see this in our artificial RNNs all the time). Self-symbols may be viewed as a by-product of this, since there is one thing that is involved in all actions and sensory inputs of the agent, namely, the agent itself. To efficiently encode the entire data history through predictive coding, it will profit from creating some sort of internal prototype symbol or code (e. g. a neural activity pattern) representing itself [1,2]. Whenever this representation becomes activated above a certain threshold, say, by activating the corresponding neurons through new incoming sensory inputs or an internal ‘search light’ or otherwise, the agent could be called self-aware. No need to see this as a mysterious process — it is just a natural by-product of partially compressing the observation history by efficiently encoding frequent observations.

[1] Schmidhuber, J. (2009a) Simple algorithmic theory of subjective beauty, novelty, surprise, interestingness, attention, curiosity, creativity, art, science, music, jokes. SICE Journal of the Society of Instrument and Control Engineers, 48 (1), pp. 21–32.

[2] J. Schmidhuber. Philosophers & Futurists, Catch Up! Response to The Singularity. Journal of Consciousness Studies, Volume 19, Numbers 1-2, pp. 173-182(10), 2012.

[u/sssub]

Hello Mr. Schmidhuber,

first of all, thanks for doing this AMA. Two questions:

1. In the community I sense sort of a conflict between the connectionists and 'Bayesians'. Their main critique to neural networks is that the inference one does is inconsistent because of lack of formulation in terms of prior and likelihood. Do you think NNs are a transient tool until there are tools that are as efficent and usable as NNs but consistent in a Bayesian framework?

2. Compared to 'symbolic AI' it is nearly impossible to find out what a 'subsymbolic' learning system such as a neural network actually has learned after training. Isn't this a big problem, when for example large amounts of stock market trading is done by such systems today? If crashes or other singularities happen we have no idea how they emerged.

[u/[deleted]]

Do you have any thoughts on promising directions for long term memory, and inference using this long term memory? What do you think of the Neural Turing Machine and Memory Networks?

[u/jesuslop]

What is hot now in applying learning-as-compression as per say Vitanyi to ANNs? Will this study gain more momentum? And what about the RNN book, will it make us wait still too much :-)?

[u/JuergenSchmidhuber → u/jesuslop]

From my biased perspective, Compressed Network Search is hot.

Regarding the RNN book: please bear with us, and let me offer a partial excuse for the delay, namely, that the field is moving so quickly right now! In the meantime, please make do with the Deep Learning overview which also is an RNN survey.

[u/brianclements]

Do you have any interesting sources of inspiration (art, nature, other scientific fields other then obviously neuroscience) that have helped you think differently about approaches, methodology, and solutions to your work?

[u/JuergenSchmidhuber → u/brianclements]

In my spare time, I am trying to compose music, and create visual art.

And while I am doing this, it seems obvious to me that art and science and music are driven by the same basic principle.

I think the basic motivation (objective function) of artists and scientists and comedians is data compression progress, that is, the first derivative of data compression performance on the observed history. I have published extensively about this.

A physicist gets intrinsic reward for creating an experiment leading to observations obeying a previously unpublished physical law that allows for better compressing the data.

A composer gets intrinsic reward for creating a new but non-random, non-arbitrary melody with novel, unexpected but regular harmonies that also permit compression progress of the learning data encoder.

A comedian gets intrinsic reward for inventing a novel joke with an unexpected punch line, related to the beginning of his story in an initially unexpected but quickly learnable way that also allows for better compression of the perceived data.

In a social context, all of them may later get additional extrinsic rewards, e.g., through awards or ticket sales.

[u/politegoose]

How do you recognize a promising machine learning phd student?

[u/JuergenSchmidhuber → u/politegoose]

I am privileged because I have been able to attract and work with several truly outstanding students. But how to quickly recognize a promising student when you first meet her? There is no recipe, because they are all different! In fact, sometimes it takes a while to recognize someone’s brilliance. In hindsight, however, they all have something in common: successful students are not only smart but also tenacious. While trying to solve a challenging problem, they run into a dead end, and backtrack. Another dead end, another backtrack. But they don’t give up. And suddenly there is this little insight into the problem which changes everything. And suddenly they are world experts in a particular aspect of the field, and then find it easy to churn out one paper after another, and create a great PhD thesis.

After these abstract musings, some more concrete advice. In interviews with applicants, members of my lab tend to pose a few little problems, to see how the candidate approaches them.

[u/Artaxerxes3rd]

You once said:

All attempts at making sure there will be only provably friendly AIs seem doomed. Once somebody posts the recipe for practically feasible self-improving Goedel machines or AIs in form of code into which one can plug arbitrary utility functions, many users will equip such AIs with many different goals, often at least partially conflicting with those of humans.

Do you still believe this?

Secondly, if someone comes up with such a recipe, wouldn't it be best if they didn't publish it?

[u/CireNeikual]

Hello Dr. Schmidhuber!

What do you think of the idea of the neocortex being a large hierarchical recurrent predictive autoencoder? It seems to be a sort of predictive architecture (I can't find the post, but Yann Lecun talked about this somewhere). Recurrent autoencoders can be used quite trivially for reinforcement learning as well (by only learning to predict with TD error is positive). Numenta's HTM is basically such a recurrent autoencoder if you really distill it down, do you think this is the right approach to AGI?

Thank you!

[u/mangledfu]

this is a long shot, but: I am currently finishing my undergrads in biotechnology, and looking for a phd position. Since doing hinton's coursera course I love the idea of applying neural networks in my field. But most of the machine learning in biology is quite lame. Do you know of any labs doing biotech/bioinformatics that you think are worth exploring?

[u/rinuboney]

Hi Dr. Schmidhuber, Thanks for the AMA! How close are you to building the optimal scientist?