Artificial Intelligence Stack Exchange
Artificial Intelligence Stack Exchange

Questions tagged [linear-algebra]

For questions about the use/aspects/implementation/intuition/mathematical proofs of various Linear Algebra methods used in Machine Learning and AI algorithms.

Linear algebra is the study of linear sets of equations and their transformation properties. Linear algebra allows the analysis of rotations in space, least squares fitting, solution of coupled differential equations, determination of a circle passing through three given points, as well as many other problems in mathematics, physics, and engineering. Confusingly, linear algebra is not actually an algebra in the technical sense of the word "algebra" (i.e., a vector space V over a field F, and so on).

Linear Algebra - WolframMathWorld

Linear Algebra - Wikipedia

39 questions
6
votes
1 answer

What does it mean to do multi-dimensional processing with tensors in tensor cores?

In some tweets about NeurIPS 2018, this video from NVIDIA appeared. At around 0.37, she says: If you think about the current computations in our deep learning systems, they are all based on Linear Algebra. Can we come up with better paradigms to do…
wrong_path
  • 161
  • 6
6
votes
1 answer

How to express a fully connected neural network succintly using linear algebra?

I'm currently reading the paper Federated Learning with Matched Averaging (2020), where the authors claim: A basic fully connected (FC) NN can be formulated as: $\hat{y} = \sigma(xW_1)W_2$ [...] Expanding the preceding expression $\hat{y} =…
user1360448
  • 73
  • 6
5
votes
3 answers

How does neural network classifier classify from just drawing a decision plane?

I understand that a neural network basically distorts(non-linear transformation) and changes the perspective(linear transformations) of input space to draw a plane to classify data. How does the network deduce if an input is one side of a plane and…
Daniel
  • 326
  • 2
  • 9
5
votes
2 answers

Which linear algebra book should I read to understand vectorized operations?

I am reading Goodfellow's book about neural networks, but I am stuck in the mathematical calculus of the back-propagation algorithm. I understood the principle, and some Youtube videos explaining this algorithm shown step-by-step, but now I would…
lolveley
  • 151
  • 3
4
votes
1 answer

Why MLP cannot approximate a closed shape function?

[TL;DR] I generated two classes Red and Blue on a 2D space. Red are points on Unit Circle and Blue are points on a Circle Ring with radius limits (3,4). I tried to train a Multi Layer Perceptron with different number of hidden layers, BUT all the…
entropyfeverone
  • 143
  • 4
3
votes
2 answers

Is there any difference between affine transformation and linear transformation?

Consider the following statements from A Simple Custom Module of PyTorch's documentation To get started, let’s look at a simpler, custom version of PyTorch’s Linear module. This module applies an affine transformation to its input. Since the…
hanugm
  • 3,571
  • 3
  • 18
  • 50
3
votes
2 answers

How can we find the value function by solving a system of linear equations?

I am following the book "Reinforcement Learning: An Introduction" by Richard Sutton and Andrew Barto, and they give an example of a problem for which the value function can be computed explicitly by solving a system of $\lvert S \rvert $ equations…
phil
  • 143
  • 4
3
votes
1 answer

How do you find the homography matrix given 4 points in both images?

I want to understand the process of finding a homography matrix given 4 points in both images. I am able to do that in python OpenCV, but I wonder how it works behind the scenes. Suppose I have points $p_1, p_2, p_3, p_4$ in the first image and…
Hissaan Ali
  • 175
  • 1
  • 6
2
votes
1 answer

Does k consistency always imply (k - 1) consistency?

From Russell-Norvig: A CSP is strongly k-consistent if it is k-consistent and is also (k − 1)-consistent, (k − 2)-consistent, . . . all the way down to 1-consistent. How can a CSP be k-consistent without being (k - 1)-consistent? I can't think of…
amad-person
  • 131
  • 4
2
votes
0 answers

Is orthogonal initialization still useful when hidden layer sizes vary?

Pytorch's orthogonal initialization cites "Exact solutions to the nonlinear dynamics of learning in deep linear neural networks ", Saxe, A. et al. (2013), which gives as reason for the usefulness of orthogonal initialization the fact that for a…
Gabi
  • 121
  • 2
2
votes
2 answers

Do solving system of linear equations required anywhere in contemporarty deep learning?

Consider the following from Numerical Computation chapter of Deep Learning book Machine learning algorithms usually require a high amount of numerical computation. This typically refers to algorithms that solve mathematical problems by methods that…
hanugm
  • 3,571
  • 3
  • 18
  • 50
2
votes
0 answers

How to interpret the variance calculation in a Guassian process

I answered another question here about the mean prediction of a GP, but I have a hard time coming up with an intuitive explanation of the variance prediction of a GP. Thew specific equation that I am speaking of is equation 2.26 in the Gaussian…
Joff
  • 139
  • 4
2
votes
1 answer

What exactly is the eigenspace of a graph (in spectral clustering)?

When we find the eigenvectors of a graph (say in the context of spectral clustering), what exactly is the vector space involved here? Of what vector space (or eigenspace) are we finding the eigenvalues of?
Manish Kausik Hari Baskar
  • 85
  • 4
2
votes
2 answers

How does PCA work when we reduce the original space to 2 or higher-dimensional space?

How does PCA work when we reduce the original space to a 2 or higher-dimensional space? I understand the case when we reduce the dimensionality to $1$, but not this case. $$\begin{array}{ll} \text{maximize} & \mathrm{Tr}\left(…
VN Pikachu
  • 21
  • 3
2
votes
1 answer

What do we mean by 'principal angle between subspaces'?

I came across the term 'principal angle between subspaces' as a tool for comparing objects in images. All material that I found on the internet seems to deal with this idea in a highly mathematical way and I couldn't understand the real physical…
Manish Kausik Hari Baskar
  • 85
  • 4
1
2 3