How can I automate the choice of the architecture of a neural network for an arbitrary problem?

Question

Assume that I want to solve an issue with a neural network that either I can't fit to existing architectures (perceptron, Konohen, etc) or I'm simply not aware of the existence of those or I'm unable to understand their mechanics and I rely on my own instead.

How can I automate the choice of the architecture/topology (that is, the number of layers, the type of activations, the type and direction of the connections, etc.) of a neural network for an arbitrary problem?

I'm a beginner, yet I realized that in some architectures (or, at least, in perceptrons) it is very hard if not impossible to understand the inner mechanics, as the neurons of the hidden layers don't express any mathematically meaningful context.

Answer 1

I think in this case, you'll probably want to use a genetic algorithm to generate a topology rather than working on your own. I personally like NEAT (NeuroEvolution of Augmenting Topologies).

The original NEAT paper involves evolving weights for connections, but if you only want a topology, you can use a weighting algorithm instead. You can also mix activation functions if you aren't sure which to use. Here is an example of using backpropagation and multiple neuron types.

Answer 2

The other answer mentions NEAT to generate network weights or topologies. The paper NeuroEvolution: The Importance of Transfer Function Evolution and Heterogeneous Networks, which also gives a short summary of neuroevolution techniques, provides an alternative approach to NEAT. It uses Cartesian Genetic Programming to evolve multiple activation functions.