In neural Turing machine (NTM), reading memory is represented as
\begin{align} r_t \leftarrow \sum\limits_i^R w_t(i) \mathcal{M}_t(i) \tag{2} \end{align}
and writing to memory is represented as
Step1: Erase
\begin{align} \mathcal{M}_t^{erased}(i) \leftarrow \mathcal{M}_{t-1}(i)[\mathbf{1} - w_t(i) e_t ] \tag{3} \end{align}
Step2: Add
\begin{align} \mathcal{M}_t(i) \leftarrow \mathcal{M}_t^{erased}(i) + w_t(i) a_t \tag{4} \end{align}
In the reading mechanism, if we take this example values and applied to the above formula, instead of a vector, we get a scalar of value 2.
M_t =[[1,0,1,0],
[0,1,0,0],
[1,1,1,0]]
w_t = [1,1,1]
The same thing happens in writing as well; here we take the dot product of two vectors, $w_t(i) e_t$, with a scalar value as output. According to paper, unless $w_t$ or $e_t$ are zeros, it will erase all values in the memory matrix.
My own idea about NTM memory was that it uses the weights to find the indices or rows inside the memory matrix corresponding to a certain task.
How does the memory in NTM work?
How a memory for a particular task is stored, that is, is it stored row-wise or it's stored in the whole matrix?
