Artificial Intelligence Stack Exchange
Artificial Intelligence Stack Exchange

Questions tagged [neural-architecture-search]

For questions related to the concept of neural (network) architecture search (NAS), which is a way of automating the design (that is, the hyper-parameters) of a neural network. NAS is related to neuroevolution, given that neuroevolution can be used to perform NAS, but neuroevolution is not the only way of performing NAS. For example, reinforcement learning can also be used to perform NAS.

See e.g. https://en.wikipedia.org/wiki/Neural_architecture_search for more info.

13 questions
5
votes
3 answers

How is neural architecture search performed?

I have come across something that IBM offers called neural architecture search. You feed it a data set and it outputs an initial neural architecture that you can train. How is neural architecture search (NAS) performed? Do they use heuristics, or is…
Adam Geringer
  • 51
  • 2
4
votes
2 answers

Is it possible to train a neural network with 3 inputs and 12 outputs?

The selection of experimental data includes a set of vectors of different dimensions. The input is a 3-dimensional vector, and the output is a 12-dimensional vector. The sample size is 120 pairs of input 3-dimensional and output 12-dimensional…
dtn
  • 227
  • 1
  • 6
4
votes
1 answer

When using Neural Architecture Search, how are the hyper-parameters chosen?

I have read a lot about NAS, but I still do not understand one concept: When setting up a neural network, hyperparameters (such as the learning rate, dropout rate, batch size, filter size, etc.) need to be set up. In NAS, only the best architecture…
cocojambo
  • 73
  • 3
4
votes
1 answer

How does RL based neural architecture search work?

I have read through many of the papers and articles linked in this thread but I haven't been able to find an answer to my question. I have built some small RL networks and I understand how REINFORCE works. I don't quite understand how they are…
jaaq
  • 153
  • 1
  • 5
3
votes
0 answers

Which hyper-parameters are considered in neural architecture search?

I want to understand automatic Neural Architecture Search (NAS). I read already multiple papers, but I cannot figure out what the actual search space of NAS is / how are classical hyper-parameters considered in NAS? My understanding: NAS aims to…
cocojambo
  • 73
  • 3
2
votes
3 answers

Is it possible to learn the number of layers?

Is it possible, in a transformer or other deep architecture, to include the number of layers as a parameter of the model so it could be learned? In fact, I have a keras layer that I use to change the final layer without rebuilding the model, so I…
arivero
  • 51
  • 7
2
votes
0 answers

Can operations like convolution and pooling be discovered with a neural architecture search approach?

From Neural Architecture Search: A Survey, first published in 2018: Moreover, common search spaces are also based on predefined building blocks, such as different kinds of convolutions and pooling, but do not allow identifying novel building…
SubstantialRange
  • 21
  • 1
2
votes
0 answers

How do I determine the best neural network architecture for a problem with 3 inputs and 12 outputs?

This post continues the topic in the following post: Is it possible to train a neural network with 3 inputs and 12 outputs?. I conducted several experiments in MATLAB and selected those neural networks that best approximate the data. Here is a…
dtn
  • 227
  • 1
  • 6
1
vote
1 answer

Why are the non-linear activations in deep nets not learned?

Why can we not parametrize and learn the non-linear activations? For example, if we look at leaky ReLu which equals to $f(y)=y$ for $y>0$ and $f(y)=\alpha y$ for $y<0$, it seems that we can differentiate the parameter $\alpha$ with respect to the…
Gilad Deutsch
  • 629
  • 5
  • 12
1
vote
0 answers

How can an "architectural motif" be extracted from a trained MLP?

I am trying to reproduce the paper Synthetic Petri Dish: A novel surrogate model for Rapid Architecture Search. In the paper, the authors try to reduce the architecture of an MLP model trained on MNIST (2 layers - 100 neurons) by initializing a…
Perl Del Rey
  • 131
  • 5
1
vote
0 answers

Regional specialization in neural networks (especially for language processing)?

What is the status of the research on regional specialization of the artificial neural networks? Biology knows that such specialization exists in the brain and it is very important for the functioning of the brain. My thinking is that specialization…
TomR
  • 823
  • 5
  • 15
0
votes
1 answer

Are there metrics for image complexity for informing neural network design?

BACKGROUND: I am trying to think of rational approaches to designing deep learning models for image classification. One thought is to quantify the complexity of image datasets and use that to inform model design. By the way, I know that rational…
Snehal Patel
  • 912
  • 1
  • 1
  • 25
0
votes
1 answer

A neural network to learn the connection between two totally different type of images

I have a dataset of two different type of images. Say, I have images of a person and his all 10 fingerprints. I want to create a relation between them to predict one from another. How I can do that and which architecture is suitable for this problem…
Prapon
  • 1
  • 1