What would be theoretically the best performing optimization algorithm(s) in this case?
- Very high dimensional problem: 250-500 parameters
- Goal is to obtain very high quality solutions, not just "good" solutions
- Parameters form multiple lower-dimensional structures
- Inexpensive evaluation: 10k-50k iterations allowed
I am aware that each problem is unique and there is no free lunch. Currently I am testing the following algos:
FST-PSO (PSO variant with enhanced ~isolated particle reflexion)
Paper: https://www.sciencedirect.com/science/article/abs/pii/S2210650216303534 Code: https://github.com/aresio/fst-pso
TuRBO (bayesian optimization with targeted sub-spaces via size-adaptable "trust regions")
Paper: https://arxiv.org/pdf/1910.01739.pdf Code: https://github.com/uber-research/TuRBO
ALEBO (bayesian optimization with lower-dimensional structures characterization via linear embeddings)
Paper: https://arxiv.org/pdf/2001.11659.pdf Code: https://github.com/facebookresearch/alebo
I discarded CMA-ES and equivalents, genetic algos and some other BO-based algos (REMBO, SAASBO) because I concluded the previously cited ones should perform better in principle.
Am I forgetting about another algo that should perform well in the case described above?
Bonus: Do you have knowledge of an existing set of benchmark functions that would be well-adapted for studying this type of problems, and benchmarking algos on?
