1

I have not found any neural network training methods that recommend manually intervening in the training process while it is happening. However, some experiments I've done seem to show this can be an effective way to speed up training.

For example, once the network has converged on a suboptimal solution, I can have the network focus on particular sections of the training data for awhile, and then switch back to the entire training data to get the network out of the local optima. Gradient descent alone would not be able to do this because while I have the network focus on a subset of the training data the error becomes very large, and then after I release the focus the error drops below the previous local optima value. Gradient descent would not choose to explore an area of the solution space with a significantly higher error than the current region.

Has there been any research into whether these kinds of manual intervention methods can improve over purely automated training?

yters
  • 387
  • 2
  • 10
  • 1
    Do you have reasons to believe that your intervention methods cannot be automated? Generally, when someone discovers a method by which they can improve a learning algorithm, they automate it. Certainly vanilla gradient descent barely scratches the surface of the breadth of available automated optimization schemes. – Him Apr 23 '23 at 05:43
  • @Him if they were, then we'd already have AGI. – yters Apr 26 '23 at 22:43
  • Something tells me that the manual intervention method you have found is not the missing link to AGI. My point here is that you may simply be googling the wrong thing: You are looking for "how to use manual intervention to improve nn performance", but really what you've found is not that manual intervention helps, but that actively focused training helps. There is almost certainly an abundance of research on this topic, and I would wager than approximately 0% of it suggests doing it manually. – Him Apr 27 '23 at 17:26
  • @Him what I mean is that humans have a pattern identification ability that has yet to be fully captured algorithmically. So, makes sense to let humans manually intervene when they spot patterns that haven't been captured algorithmically yet. Also, a lot more accessible than understanding the sophisticated optimization algorithms. – yters Apr 27 '23 at 17:49

1 Answers1

3

There are two machine learning research fields that are related to your problem: one is curriculum learning, the other is active learning.

Curriculum Learning

The basic idea is to manually design increasingly difficult training tasks that aim to guide the learning of the network towards a better local minima. Say you have a very hard problem, from that you build a curriculum: initially it contains easy and simplified versions of your initial problem, and as the training progresses you switch to a more difficult curriculum, and keep doing this until the difficulty of the training tasks match or even surpasses the one of the original problem.

Such curriculum learning approach has been shown to be beneficial both in deep learning and reinforcement learning problems.

Actually there is much interest in automatically building the curriculum: there are some example of this in the literature, allowing to automatically generate a curriculum according to the network's training performance. This would, at least in principle, solve the issue about how to properly design an effective curriculum which is not trivial in some cases.

References:

Active Learning

The basic idea is to let the training algorithm be helped by a user from time to time.

A use case of this is training data labeling, in which in an online fashion a user labels or even scores data points that are poorly predicted (or classified) by the model. In this regard, a working approach is called DAgger (dataset aggregation) that is used in imitation learning: basically a human manually label data with the best action to take (as the goal of the problem is to learn a policy - so learn best actions that are unknown.)

References:

The answer to your question is: in principle yes, you can speed up training by either manual or automatic guidance of the training, or by manually providing better labels or training data.

Luca Anzalone
  • 2,120
  • 2
  • 13
  • ISTR that curriculum learning is used informally all the time. Autoencoders can be built one layer at a time. The P in GPT stands for the fact it was first trained on some existing fill-in-the-missing-word task IIRC. CLIP/VQGAN is CLIP plus VQGAN. – user253751 Apr 24 '23 at 21:36
  • @user253751 Well, there is also InstructGPT that is trained by reinforcement learning from human feedback of instruction following tasks – Luca Anzalone Apr 25 '23 at 08:20