From Neural Architecture Search: A Survey, first published in 2018:
Moreover, common search spaces are also based on predefined building blocks, such as different kinds of convolutions and pooling, but do not allow identifying novel building blocks on this level; going beyond this limitation might substantially increase the power of NAS.
Has anyone tried that? If not, do you have any thoughts about the feasibility of this idea?
