For questions related to gradient boosting, which is a machine learning technique that can be used for regression and classification problems and which produces a prediction model in the form of an ensemble of other smaller prediction models (typically decision trees).
Questions tagged [gradient-boosting]
13 questions
5
votes
3 answers
How do weak learners become strong in boosting?
Boosting refers to a family of algorithms which converts weak learners to strong learners. How does it happen?
Legend
- 103
- 3
4
votes
2 answers
When do the ensemble methods beat neural networks?
In many applications and domains, computer vision, natural language processing, image segmentation, and many other tasks, neural networks (with a certain architecture) are considered to be by far the most powerful machine learning…
spiridon_the_sun_rotator
- 2,454
- 8
- 16
3
votes
1 answer
Why is the exponential loss used in this case?
I am reading the paper Tracking-by-Segmentation With Online Gradient Boosting Decision Tree. In Section 2.1, the paper says
Given training examples, $\left\{\left(\mathbf{x}_{i}, y_{i}\right) \mid \mathbf{x}_{i} \in \mathbb{R}^{n}\right.$ and…
Zhang Liao
- 33
- 2
3
votes
0 answers
Can XGBoost solve XOR problem?
I've read that decision trees are able to solve XOR operation so I conclude that XGBoost algorithm can solve it as well.
But my tests on the datasets (datasets that should be highly "xor-ish") do not produce good results, so I wanted to ask whether…
GKozinski
- 1,240
- 8
- 19
3
votes
1 answer
Are there tabular datasets where deep neural networks outperform traditional methods?
Are there (complex) tabular datasets where deep neural networks (e.g. more than 3 layers) outperform traditional methods such as XGBoost by a large margin?
I'd prefer tabular datasets rather than image datasets, since most image dataset are either…
Clara
- 31
- 1
2
votes
1 answer
House price inflation modelling
I have a data set of house prices and their corresponding features (rooms, meter squared, etc). An additional feature is the sold date of the house. The aim is to create a model that can estimate the price of a house as if it was sold today. For…
Melly Donald
- 35
- 4
2
votes
1 answer
What are some applications where tree models perform better than neural networks?
Neural networks are known to be generally better modeling techniques as compared to tree-based models (such as decision trees). Are there any exceptions to this?
jaiswati_b
- 39
- 2
1
vote
0 answers
React on train-validation curve after trening
I have a regression task that I tray to solve with AI.
I have around 6M rows with about 30 columns. (originally there was 100, but I reduce it with drop feature importance)
I understand basic principle: Look if model overfit or underfit - according…
Marko Zadravec
- 169
- 7
1
vote
1 answer
Has "deep vs. wide" been resolved?
All else being equal, including total neuron count, I give the following definitions:
wide is a parallel ensemble, where good chunks of the neurons have the same inputs because the inputs are shared and they have different outputs.
deep is a series…
EngrStudent
- 361
- 3
- 12
1
vote
0 answers
How would the "best function" been constructed if there are no computationally limitations?
I am reading the Wikipedia article on gradient boosting. There is written:
Unfortunately, choosing the best function $h$ at each step for an arbitrary loss function $L$ is a computationally infeasible optimization problem in general. Therefore, we …
jennifer ruurs
- 579
- 2
- 8
1
vote
0 answers
How can I use gradient boosting with multiple features?
I'm trying to use gradient boosting and I'm using sklearn's GradientBoostingClassifier class.
My problem is that I'm having a data frame with 5 columns and I want to use these columns as features. I want to use them continuously. I mean I want each…
Kamran Hosseini
- 111
- 3
0
votes
1 answer
Can i train xgboost on multiple time series csv files at the same time?
I built an xgboost model to predict stock it now trains on 1 stock at a time its a csv file I use pandas to load it.
Is there a way to train the model on multiple stocks at the same time? What would be the best approach?
I don't need code just…
AJB
- 1
- 1
0
votes
1 answer
Multi-Variate Time-Series forecasting with XGBoost
I have trained an XGBoost model on a time-series dataset for predicting a value. The time series has 5 features and one label (the target value). The trained model works fine on both training and testing data, so far so good. As I said, this dataset…
Arashsyh
- 181
- 5