4

I am trying to understand the concept of fine-tuning and few-shot learning.

I understand the need for fine-tuning. It is essentially tuning a pre-trained model to a specific downstream task.

However, recently I have seen a plethora of blog posts stating zero-shot learning, one-shot learning and few-shot learning.

How are they different from fine-tuning? It appears to me that few-shot learning is a specialization of fine-tuning. What am I missing here?

Can anyone please help me?

Exploring
  • 223
  • 6
  • 16
  • @nbro no. They don't discuss about few shot learning and their difference. – Exploring Jun 14 '22 at 10:47
  • Ok, then edit your post to explain that you're just interested in few-shot learning and why that other post doesn't answer your question. – nbro Jun 14 '22 at 14:27

1 Answers1

3

I believe the standard meanings are as follows, but not everyone uses words in the same way, so you might see examples that differ.

Fine tuning refers to slightly changing the weights of a pre-trained model (i.e. using a low learning rate) to adapt to a specific task.

Few shot learning refers to using a very small dataset to adapt to a specific task.

Someone might do both at the same time (fine-tuning with a small dataset), just fine-tuning (with a big dataset), or just few shot learning (e.g. giving GPT a few examples to prompt a particular kind of answer).

Lee Reeves
  • 491
  • 1
  • 5