For questions related to the concept of meta-learning (or learning-to-learn).
Questions tagged [meta-learning]
38 questions
12
votes
4 answers
Why LLMs and RNNs learn so fast during inference but, ironically, are so slow during training?
Why LLMs learn so fast during inference, but, ironically, are so slow during training? That is, if you teach an AI a new concept in a prompt, it will learn and use the concept perfectly and flawless, through the whole prompt, after just one shot.…
MaiaVictor
- 355
- 1
- 9
8
votes
4 answers
What are the differences between transfer learning and meta learning?
What are the differences between meta-learning and transfer learning?
I have read 2 articles on Quora and TowardDataScience.
Meta learning is a part of machine learning theory in which some
algorithms are applied on meta data about the case to…
Long
- 145
- 1
- 8
7
votes
2 answers
How does rotating an image and adding new 'rotated classes' prevent overfitting?
From Meta-Learning with Memory-Augmented Neural Networks in section 4.1:
To reduce the risk of overfitting, we performed data augmentation by randomly translating and rotating character images. We also created new classes through 90◦, 180◦ and 270◦…
AAC
- 171
- 1
- 2
6
votes
2 answers
What are the state-of-the-art meta-reinforcement learning methods?
This question can seem a little bit too broad, but I am wondering what are the current state-of-the-art works on meta reinforcement learning. Can you provide me with the current state-of-the-art in this field?
Sara El
- 63
- 4
5
votes
2 answers
Do genetic algorithms also evolve?
After witnessing the rise of deep learning as automatic feature/pattern recognition over classic machine learning techniques, I had an insight that the more you automate at each level, the better the results, and I, therefore, turned my focus to…
Kayonga Arnauld
- 59
- 1
4
votes
1 answer
How important is learning to learn for the development of AGI?
Some people say that abstract thinking, intuition, common sense, and understanding cause and effect are important to make AGI.
How important is learning to learn for the development of AGI?
loopy_fun
- 41
- 1
4
votes
3 answers
Can we optimize an optimization algorithm?
In this answer to the question Is an optimization algorithm equivalent to a neural network?, the author stated that, in theory, there is some recurrent neural network that implements a given optimization algorithm.
If so, then can we optimize the…
Dimer
- 331
- 1
- 10
4
votes
1 answer
Why not go another layer deeper with Auto-AutoML?
So I'm finding AutoML to be pretty interesting but I'm still learning how it all works. I've played with the incredibly broken AutoKeras and got some decent results.
The question is, if you are using a NN to optimize the architecture of another…
Josiah Swaim
- 171
- 1
- 5
3
votes
0 answers
How many tasks are needed for meta-learning?
This is an empirical question, essentially how many tasks do you need data for, to make a useful meta learning model (e.g. using MAML)? I'm looking for ranges based on personal experience or if anyone has done research on the topic and you know of…
profPlum
- 360
- 1
- 9
3
votes
1 answer
What exactly does meta-learning in reinforcement learning setting mean?
We can use DDPG to train agents to stack objects. And stacking objects can be viewed as first grasping followed by pick and place. In this context, how does meta-reinforcement learning fit? Does it mean I can use grasp, pick and place as training…
Sofi
- 31
- 1
3
votes
0 answers
What is the difference between "out-of-distribution (generalisation)" and "(meta)-transfer learning"?
I'm trying to develop a better understanding of the concept of "out-of-distribution" (generalization) in the context of Bengio's "Moving from System 1 DL to System 2 DL" and the concept of "(meta)-transfer learning" in general.
These concepts seem…
maxcompression
- 31
- 2
3
votes
1 answer
What does "episodic training" mean?
I'm reading the book Hands-On Meta Learning with Python, and in Prototypical networks said:
So, we use episodic training—for each episode, we randomly sample a
few data points from each class in our dataset and we call that a
support set and train…
VansFannel
- 493
- 2
- 15
2
votes
3 answers
Is it possible to learn the number of layers?
Is it possible, in a transformer or other deep architecture, to include the number of layers as a parameter of the model so it could be learned?
In fact, I have a keras layer that I use to change the final layer without rebuilding the model, so I…
arivero
- 51
- 7
2
votes
1 answer
What is $ \nabla_{\theta_{k-1}} \theta_{k}$ in the context of MAML?
I am attempting to fully understand the explicit derivation and computation of the Hessian and how it is used in MAML. I came across this blog: https://lilianweng.github.io/lil-log/2018/11/30/meta-learning.html.
Specifically, could someone help to…
Blake Camp
- 23
- 2
2
votes
0 answers
In few-shot classification, should I use my custom dataset as the validation dataset and mini-ImageNet as the training dataset?
I am new to few-shot learning, and I wanted to get a hands-on understanding of it, using Reptile algorithm, applied to my custom dataset.
My custom dataset has 30 categories, with 5 images per category, so this would be a 30 way 5 shot.
Given a new…
Geneveve08
- 61
- 1
- 3