Artificial Intelligence Stack Exchange
Artificial Intelligence Stack Exchange

Questions tagged [online-learning]

For questions related to online learning algorithms, that is, algorithms that learn while e.g. the associated agent interacts with an environment.

27 questions
15
votes
2 answers

What is the difference between active learning and online learning?

The definitions for these two appear to be very similar, and frankly, I've been only using the term "active learning" the past couple of years. What is the actual difference between the two? Is one a subset of the other?
David
  • 293
  • 2
  • 5
14
votes
1 answer

What is the relation between online (or offline) learning and on-policy (or off-policy) algorithms?

In the context of RL, there is the notion of on-policy and off-policy algorithms. I understand the difference between on-policy and off-policy algorithms. Moreover, in RL, there's also the notion of online and offline learning. What is the relation…
nbro
  • 39,006
  • 12
  • 98
  • 176
9
votes
2 answers

Are there dynamic neural networks?

Are there neural networks that can decide to add/delete neurons (or change the neuron models/activation functions or change the assigned meaning for neurons), links or even complete layers during execution time? I guess that such neural networks…
TomR
  • 823
  • 5
  • 15
8
votes
0 answers

Normalizing Normal Distributions in Thompson Sampling for online Reinforcement Learning

In my implementation of Thompson Sampling (TS) for online Reinforcement Learning, my distribution for selecting $a$ is $\mathcal{N}(Q(s, a), \frac{1}{C(s,a)+1})$, where $C(s,a)$ is the number of times $a$ has been picked in $s$. However, I found…
Kevin
  • 81
  • 2
5
votes
3 answers

What are the state-of-the-art approaches for continual learning with neural networks?

There seems to be a lot of literature and research on the problems of stochastic gradient descent and catastrophic forgetting, but I can't find much on solutions to perform continual learning with neural network architectures. By continual learning,…
gcorso
  • 356
  • 1
  • 7
4
votes
1 answer

Is continuous learning possible with a deep convolutional neural network, without changing its topology?

In general, is continuous learning possible with a deep convolutional neural network, without changing its topology? In my case, I want to use a convolutional neural network as a classifier of heartbeat types. The ECG signal is split, and a color…
Dominiksr
  • 65
  • 8
4
votes
1 answer

Can a CNN be trained incrementally?

Like our human brain, we can first learn (train) the handwriting 0 and 1. After the traing (and test) accuray is good enough, we only need to study (traing) the hardwriting 2, Instead of cleaning all of learned memory, and relearn handwriting data…
Sunson29
  • 41
  • 2
4
votes
0 answers

What are stable ways of doing online machine learning?

I am trying to deploy a machine learning solution online into an application for a client. One thing they requested is that the solution must be able to learn online because the problem may be non-stationary and they want the solution to track the…
Rui Nian
  • 423
  • 3
  • 13
3
votes
0 answers

How to fix high variance of the returns on a 2d env?

I'm trying to train an agent on a self-written 2d env, and it just doesn't converge to the solution. It is basically a 2d game where you have to move a small circle around the screen and try to avoid collisions with randomly moving "enemy" circles…
debrises
  • 31
  • 3
3
votes
1 answer

How do I keep my system (online) learning if I can get ground truth labels only for examples flagged positive?

I have a binary classifier (think of it as a content moderation system) that is deployed after having being trained via batch learning. Once deployed, humans review and check for correctness only items predicted positive by the algorithm. In other…
Davide Fiocco
  • 83
  • 5
3
votes
2 answers

Which online machine learning technique to use for multi-class classification problem with multiple inputs?

I have the following problem. We have $4$ separate discrete inputs, which can take any integer value between $-63$ and $63$. The output is also supposed to be a discrete value between $-63$ and $63$. Another constraint is that the solution should…
John Mayer
  • 31
  • 3
2
votes
0 answers

Why isn't RL considered a continual learning strategy itself?

I have read about methods that apply continual learning strategies to reinforcement learning. Since reinforcement learning also learns step by step (i.e., task by task, in a sense) during the training phase, why isn't it itself considered a…
convaldo
  • 121
  • 3
2
votes
1 answer

How does a neural network that has been trained keep learning while in a real world scenario

Say I trained a Neural Network (not RNN or CNN) to classify a particular data set. So I train using a specific data set & then I test using another and get an accuracy of 95% which is good enough. I then deploy this model in a production level…
user3536523
  • 131
  • 4
2
votes
0 answers

Deriving hyperparameter updates in Online Interactive Collaborative Filtering

I've been going through "Online Interactive Collaborative Filtering Using Multi-Armed Bandit with Dependent Arms" by Wang et al. and am unable to understand how the update equations for the hyperparameters (section 4.3, equation set (23)) were…
Shashank Gupta
  • 21
  • 2
2
votes
0 answers

Is there an online RL algorithm that receives as input a camera frame and produces an action as output?

I want to build a reinforcement learning model, which takes a camera picture as input, that learns online (in terms of machine learning). Based on the position of an object on the camera, I want the model to output an action. That action would be a…
Wuuhi
  • 21
  • 1
1
2