What is the status of the research on regional specialization of the artificial neural networks? Biology knows that such specialization exists in the brain and it is very important for the functioning of the brain. My thinking is that specialization can solve the transfer learning/catastrophic forgetting by creating centers of sophisticated skills if such sophistication is necessary. Actually - there is no much alternative to specialization? If specialization exists then there can be small decision centers that routes the request to the actual part and such decision centers can be efficient. But if specialization does not exist then routing happens in the total soup/pudding of neuronal see and such all-to-all routing should be very inefficient. Of course there should be some mixing of specialization vs pudding, because there is always mixing between rationality and emotions, between execution and exploration, but nevertheless - specialization should happen, at least partially.
The problem is - that I can not find any focused article about such specialization and I can not find how such specialization can be trained? There are research on hierarchical reinforcement learning - but that is about imposing the external fixed structure on the set of neural networks, but it is not how the nature works - nature implements such hiearchy within neural network and not by imposing rigorous, symbolic structures.
Are there some notions, terms, keywords, research trends, important articles (and researchers) devoted to such specialization (including the machine learning of such specialization).
Of course, my topic is very large, but the actual work on this is small ir nonexistant and that is why it is focused.
There is work on convolution neural networks but maybe there is another approach for language processing where the parts can be specialized in - parsing, understanding, anaphora resolution, translation, etc? And is the convolution the kind of specialization I am seeking?
Maybe the notion of attention somehow is connected with my question. But usually the attention is connected with the single neurons and not with the regions? Maybe there is notions about hierarchy of attentions - one level of attention values refers to high level tasks/skills, but another level of attention values refers to subskills, etc.
