1

What will happen if to train an LLM on taking integrals and solving equations? The process of mathematical education can be absolutely automated by a computer algebra system because the verification is easy.

Is it possible that LLM will gan the ability to take integrals and simplify expressions better than the computer algebra system itself?

Anixx
  • 301
  • 8
  • 2
    they struggle with multiplications... – Alberto Aug 10 '23 at 19:50
  • _"if i train a LLM on pairs of prime numbers and their product will it learn to factorize faster than any other algorithm?"_ – Alberto Aug 10 '23 at 19:51
  • @AlbertoSinigaglia I think, this needs to be tested. – Anixx Aug 10 '23 at 20:06
  • nope, you cannot learn to do something by strictly imitating.. at most you can generalize, but you cannot extrapolate... if you want to beat human you first need to extrapolate, and then maybe some RL – Alberto Aug 10 '23 at 22:41
  • @AlbertoSinigaglia the best models can compose poems better than most humans can. If they can compose poems, why not tking integrals? – Anixx Aug 10 '23 at 23:06
  • _best models can compose poems better than most humans_ I don't see how a subjective task can have anything to do with math – Alberto Aug 10 '23 at 23:20
  • @AlbertoSinigaglia finding rhymes is like solving equations – Anixx Aug 10 '23 at 23:23
  • 1
    Please see this related question: https://ai.stackexchange.com/questions/25210/can-we-use-ml-to-do-anything-else-other-than-predicting-in-the-case-of-mathemat/25240#25240 – Robin van Hoorn Aug 11 '23 at 08:42
  • @RobinvanHoorn given all conversations in the 2 threads, I'd say OP wants to be proven right, not seeking answer nor objection. – lpounng Aug 14 '23 at 02:05

3 Answers3

3

If you're interested in research on using LLMs for arithmetic tasks, Goat: Fine-tuned LLaMA Outperforms GPT-4 on Arithmetic Tasks is a nice recent (May 2023) article.

If you're interested in research on using LLMs for more advanced mathematics, MiniF2F is a common benchmark. Check out the papers linked from the leaderboard.

If you're looking for LLM models or datasets for mathematics, https://huggingface.co/hoskinson-center and https://leandojo.org/ are good starting points.

Daniel Darabos
  • 131
  • 3
2

Depends what you mean by 'better' (in what sense?).

Nevertheless, though it is possible that an LLM can solve algebra, in general it shall not offer any benefit than a computer algebra system (again depends on what is 'better', and assuming a fair comparison).

This is because in doing so an LLM needs to deal with both the language and the math, while computer algebra system only needs to work on the math. It is like a top human runner and a vehicle - in general vehicle moves faster because this is what they are crafted solely for.

And a few words on the comments - finding rhymes is NOT like solving equations, because no rhymes is wrong (even if it sounds terrible, but that's subjective); but there is straight wrong in solving equations.

On the other hand, there are efforts to discover and improve algorithms, most notably Google's AlphaTensor. However it has nothing to do with LLM, but a lot to do with RL.

lpounng
  • 313
  • 1
  • 8
  • One can define good and incorrect rhymes, this is not subjective, this is measurable. English poetry rarely has perfect rhymes, but in Russian poetry it is quite common for poems to only have absolutely perfect rhymes. LLMs cannot make Russian poetry yet, though. When they will be able, it would be a sensation. – Anixx Aug 11 '23 at 07:13
  • Well, you could have specified it. And make sure this is true for all existing languages. – lpounng Aug 11 '23 at 07:17
  • Besides, as a Go player myself, I am not arguing about whether a computer can perform better than human; we are talking about whether LLM can solve algebra better than a machine specially crafted for solving algebra, under fair assumptions. – lpounng Aug 11 '23 at 07:25
  • But people take integrals better than CAS can. – Anixx Aug 11 '23 at 11:05
  • 1. Maybe, but we are comparing LLM and CAS, not CAS and human. 2. Once again, please formally define what 'better' is. 3. Anyway my statement still holds: given fair assumptions (same computing resources, same level of development effort etc.) a machine specially crafted for 1 task usually performs better than multitasking ones. – lpounng Aug 14 '23 at 01:57
1

Is it possible that LLM will gan the ability to take integrals and simplify. expressions better than the computer algebra system itself?

Possibly, but not in one run - i.e., if you instruct it and train it to analyse and simplify step by step, while writing this out, and it fits into the context - it is an experiment.

You may also want to STILL use a GAN approach with multiple LLM working on the problem - one writing, others correcting. Which requires a multi-step approach.

Otherwise: no. Math has specific elements that do not really work well with a "forward only" approach of a neural network.

TomTom
  • 166
  • 1