How do you explain Hebbian Learning in an intuitive way?

Question

In these lecture slides, it's written

The neuropsychologist Donald Hebb postulated in 1949 how biological neurons learn:

"When an axon of cell A is near enough to excite a cell B and repeatedly or persistently takes part in firing it, some growth process or metabolic change takes place on one or both cells such that A's efficiency as one of the cells firing B, is increased."

In more familiar terminology, that can be stated as the Hebbian Learning rule:

1. If two neurons on either side of a synapse (connection) are activated simultaneously (i.e. synchronously), then the strength of that synapse is selectively increased.

Mathematically, we can describe Hebbian learning as:

$$w_{i j}[n+1]=w_{i j}[n]+\eta x_{i}[n] x_{j}[n]$$

Here, $\eta$ is a learning rate coefficient, and $x$ are the outputs of the ith and jth elements.

---

So, my main question is: what do all these descriptions mean? Here are a few sub-questions.

1. Is Hebbian learning applicable for single-neuron networks?
2. What does "two neurons on either side of a synapse" mean?
3. Why/when would two neurons activate "simultaneously"?
4. What are these "elements"?

Answer 1

1. Hebbian learning is trying to answer "How the strength of the synapse between 2 neurons evolve over period of time based on the activity of the 2 neurons involved.". You can call it learning if you think learning is just strengthening of synapses.

2. The connection between 2 neurons are called synapse. A synapse is the point where the axons of a neuron meets with the dendrites of another neuron.

3. The 2 neurons are connected to each other but they are also connected to other neurons as well. So it may happen that both the neurons gets activated by their other connected neurons at the same time.

4. The elements refer the 2 neurons. The equation is basically saying that the synapse strength between i and j neuron at time (n+1) depends on its strength at time n plus activations of the i and j neurons at time n.