I am a software engineer that meddled with machine learning (classifiers) during my thesis.
After being out of it for a while I decided I want to try and do a neural network project to learn from, specifically reinforcement learning. We'll see how smart is was to try and make a chess AI for my first project but here we go:
I did some research and came to a few conclusions:
I want to make an AI that teaches itself by trial and error, rather than going over lots of previous games. Hence the
reinforcement-learningrequirement.It seems far better to provide some basic rules, like possible moves (policies?) for a given position (state, right?), because illegal moves far outnumber legal moves. So it seems like a waste of time and processing power to make it work on lots of useless scenarios (all the illegal move options). Because of this I think I understand that I am looking for an algorithm that learns
model-based.If I understand it correctly my problem has sparse rewards, since a chess game can last for 60 moves easily without a game result. I have pondered using reward shaping to try and deal with this, for example a stockfish (open source chess engine) position evaluation or captured piece values. However it occurs to me that that would result in some pretty nasty drawbacks:
- take a lot of time in the case of stockfish (stockfish takes seconds to evaluate a position).
- discourage sacrificing pieces in case of using captured piece values. (or would this be overcome by tuning the exploitation vs exploration ratio?)
So it seems like reward shaping is not the way to go. However if I understand it correctly I can use a
curiosity-drivenapproach to get around this?
So to summarize: I think I am looking for an algorithm that does well with a model-based curiosity-driven approach.
Do you agree with my conclusions and if so, can you recommend an algorithm for me to check out? I have searched for algorithms already but it seems to me that this question is a bit specific to just find an answer for on google.
