1

I made a simple feedforward neural network (FFNN) to predict $x$ from $\sin(x)$. It failed. Does it mean the model has overfitted? Why doesn't it work?

enter image description here

set.seed(1234567890)
Var3 <- runif(500, 0, 20)
mydata3 <- data.frame(Sin=sin(Var3),Var=Var3)
set.seed(1234567890)
winit <- runif(5500, -1, 1)
#hidUnit <- c(9,1)
set.seed(1234567890)
nn3 <-neuralnet(formula = Var~Sin,data = mydata3,
                hidden =c(4,2,1),startweights =winit,
              learningrate = 0.01,act.fct = "tanh")

plot(mydata3, cex=2,main='Predicting x from Sin(x)',
     pch = 21,bg="darkgrey",
     ylab="X",xlab="Sin(X)")
points(mydata3[,1],predict(nn3,mydata3), col="darkred", 
       cex=1,pch=21,bg="red")

legend("bottomleft", legend=c("true","predicted"), pch=c(21,21),
       col = c("darkgrey","red"),cex = 0.65,bty = "n")
homa taha
  • 13
  • 2
  • Im not familiar with r code. you are just feeding the x value into the network and adding tanh function to the hidden layers in the network am i right ? Christopher Bishop's machine learning and pattern recognition textbook has an example of sin(x) function I believe. However, he tries to fit sin(x) with a polynomial function. – calveeen Dec 13 '20 at 16:01
  • it's the other way around! feeding the network sin x and try to predict x. – homa taha Dec 13 '20 at 23:49
  • 1
    Any loss function metrics that you can report ? Eg. training loss – calveeen Dec 14 '20 at 00:54

1 Answers1

1

The labels are not unique for the input domain [0,20]. Think about sin(x)=0, x could be 0, pi, 2*pi, 3*pi, ..., n*pi, all are correct from a mathematical point of view, but this is not reflected in your MSE loss. At this point your NN has to guess the correct label from your input data. Predicting the mean of your input data is the safest bet for the network.

In essence you're trying to build a arcsin function with a NN. If only consider x values in [-0.5*pi,0.5*pi], the labels are unique and your network should work.

Tinu
  • 618
  • 1
  • 4
  • 12