Artificial Intelligence Stack Exchange
Artificial Intelligence Stack Exchange

Questions tagged [multi-task-learning]

For questions related to multi-task learning (MTL), which is a subfield of machine learning in which multiple learning tasks are solved at the same time, while exploiting commonalities and differences across tasks. MTL can result in improved learning efficiency and prediction accuracy for the task-specific models, when compared to training the models separately.

13 questions
5
votes
1 answer

How to deal with losses on different scales in multi-task learning?

Say I'm training a model for multiple tasks by trying to minimize sum of losses $L_1 + L_2$ via gradient descent. If these losses are on a different scale, the one whose range is greater will dominate the optimization. I'm currently trying to fix…
SpiderRico
  • 960
  • 8
  • 18
3
votes
1 answer

Does ChatGPT use different transformers for different downstream tasks?

What I find hard to figure out is whether ChatGPT guesses from the prompt the downstream NLP task to be performed - text summary, text generation, question-answering, doing logic or arithmetic, translation, sentiment or style analysis - and then…
Hans-Peter Stricker
  • 811
  • 1
  • 8
  • 20
2
votes
1 answer

Multi-objective training involving maximization of one loss function and minimization of another

I need my model to predict $s$ from my data $x$. Additionally, I need the model to not use signals in $x$ that are predictive of a separate target $a$. My approach is to transform $x$ into a representation $\Psi(x)$ such that it's good at predicting…
ChargeShivers
  • 121
  • 1
2
votes
0 answers

Do the terms multi-task and multi-output refer to the same thing in the context of deep learning?

Do the terms multi-task and multi-output refer to the same thing in the context of deep learning (with neural networks)? For example, do neural networks for multi-task learning use multiple outputs? If not, what is the difference between them? It…
user366312
  • 351
  • 1
  • 12
2
votes
0 answers

Is optimizing weighted sum multi objective tasks considered a multi-task learning?

I have two sequence prediction tasks, finding $\vec{\pi} \in \Pi$ and $\vec{\psi} \in \Psi$. Each sequence has its own objective function, i.e. $f_1(\vec{\pi})$ and $f_2(\vec{\psi})$. The input for the two sequence prediction tasks are also of…
Sanyou
  • 165
  • 2
  • 10
1
vote
1 answer

Instead of accumulating the gradient, can we accumulate loss values?

I have read and used Gradient Accumulation as a method to handle large batch size on smaller memory restrictions. It is described as following: for step, eachBatch in enumerate(dataloader): ... loss = loss_func(ytrue, ypred) …
LSM
  • 11
  • 2
1
vote
1 answer

What is the difference between multi-label and multi-task classification?

I am working on a data-set that has multiple labels associated with it (not necessarily independent of each other). During my development, I am confused if I should consider it as a multi-class multilabel data or a multi-class MTL kind of an…
Payal Mohapatra
  • 11
  • 2
1
vote
0 answers

How do I format task features with a one-hot task identification vector to ensure separate weight matrices for each task in multi-task RL?

I am on Lecture 2 of Stanford CS330 Multi-Task and Meta-learning, and on slide 10, the professor describes using a one-hot input vector to represent the task, and she also explained that there would be independent weight matrices for each task How…
iamPres
  • 116
  • 7
0
votes
0 answers

How to generate original training videos based on existing videoset?

I am a software engineer who is quickly ramping up on AI tech, but am nevertheless very new to the sector. A collegue has an extensive collection of training videos, the vertical is wheelchair seating and mobility and the training content are the…
lukabloomrox
  • 1
  • 3
0
votes
0 answers

What is the meaning of task distribution in the context of meta-learning?

I am working on meta-reinforcement learning and after reading the literature, I cannot clear myself on the meaning of task distribution. Please describe task distribution if you can rigorously!
Engr. Moiz Ahmad
  • 3
  • 1
0
votes
0 answers

References for the theory of pretraining and unsupervised learning to improve subsequent supervised learning

I am not sure if the title of this post uses the correct terminology, so suggestions are welcome. I have been following a lot of the ideas of using Pre-training methods on neural networks, to improve the accurancy of those networks on subsequent…
krishnab
  • 197
  • 7
0
votes
0 answers

Multi-task learning using single encoder + single decoder like structure?

It seems that a lot of researchers predominantly use single encoder + multiple decoders like structure to achieve multi-task learning in computer vision. Would it be reasonable to achieve the multi-task learning using single decoder to deal with…
HOJUN LEE
  • 1
  • 1
0
votes
1 answer

How should I incorporate numerical and categorical data as part of the inputs to the U-net for semantic segmentation?

I am using a U-Net to segment cancer cells in images of patients' arms. I would like to add patient data to it in order to see if it is possible to enhance the segmentation (patient data comes in the form of a table containing features such as…
Skyris
  • 115
  • 3