The general process in an artificial neural network when it is trained is called deep learning. The adjective deep in this case refers to neural networks consisting of several layers with a large number of weights that are calibrated with input and output examples. But what does learning here mean? The question I want to discuss here is whether the learning in a deep learning network is learning in the genuine sense: does a deep learning algorithm learn in the way human beings do? And if a network learns it must learn something, then: what is it that a deep learning network learns?

Suppose you have an artificial neural network with a suitable architecture for image detection and you want to train it to detect cats in pictures. You have a training set of labelled pictures. You can train the network by back propagating through the network and change the weights between the nodes to minimize the output error. During the training process the parameters of each node in the network are changed in a similar way as would be done in fitting a (linear) regression model: the output of an error function is minimized by changing weights, and this is done for each picture in the training set, for each node in the network, and in a number of iterations. If you have done that then, with a sufficiently large training set, the network would be able to detect cats in pictures.

The question is: would it be correct to say that the structure of the network with all its hyper parameters somehow learned the basic properties of cats such that it can detect them in pictures? And to take this one step further, would it be correct to say that the network with trained weights and other parameters somehow knows what cats are?

It depends, of course, on what we understand by learning. If we would define the result of learning as the ability to recognize cats in specific pictures then it would qualify as learning. But if the result of learning is the ability to explain or to know what a cat is then I don’t think what the network does comes anywhere near this definition. A trained neural network calculates a complex function. There is no knowledge in or above the network about what cats are, at least not in a form that we can easily understand. Nowhere during the learning phase there occurs a magical jump from data to knowledge; nowhere becomes the pile of sand a heap. And yet it is able to recognize cats.

What kind of learning it this? Perhaps a better word for the way in which artificial neural networks learn is that they are being conditioned in a certain manner. The basic mode of learning of artificial neural networks, I would say, is conditioning and not learning in a genuine sense (where it would be able to explain what a cat is). In conditioning an association is formed between action (or output) and reward (or desired output); this is similar to the way animals learn.

This mode of learning results is an ability without knowledge. And there has yet to be found a general way to reconstruct knowledge that is tacitly present in the network. If an artificial network learns then it learns in such a way that what is learned remains concealed for us (until somehow we are able to extract from the network the knowledge it learned, but that would be our learning and not the networks learning).

By calling deep learning a form of learning we project a certain idea of learning onto the training process of neural networks. I think the reason for this is the apparent analogy with human brains, and natural neural networks in general, and artificial ones. For this analogy the reasoning is: because a human brain is able to learn and because an artificial neural network mimics the human brain processes, it is only logical to conclude that artificial neural networks are also able to learn. But there is a difference between human learning and the biological process during this learning. I myself am able to learn in the sense that I can explain what I learned (I hope); my brain is conditioned during this learning.

From this perspective a neural network is similar to any non human brain. During a learning phase it is conditioned for an ability. It does not acquire any knowledge during this process about what it has learned.