New · Public release · From scratch · Maths word problems
Lilavati
Pocket Mathematician
A 95M-parameter model, trained from scratch, that solves maths word problems by writing a short Python program and running it. Every answer comes with the program behind it, so you can check the working.
- Parameters
- 95M
- Licence
- Apache-2.0
- Runs on
- PyTorch and transformers, on a laptop CPU
“A chai stall sells 45 cups of tea in the morning and twice as many in the evening. Each cup costs 12 rupees. How many rupees does the stall earn in a day?”
The program it wrote
morning_sales = 45
evening_sales = morning_sales * 2
cup_price = 12
total_sales = (morning_sales + evening_sales) * cup_price
result = total_sales
return resultRunning it gives
1,620
- GSM8K, first try
- 54.7%
- With the checker
- 72.3%
On GSM8K, the standard test of grade-school maths word problems, it solves 54.7% on its first try with its program executed, and 72.3% when Lilavati-Checker, a second 95M model, weighs 48 tries. Good at shopping, discounts and tax, speeds, averages, interest and areas; weak at numbers that should be ignored, working backwards and unit conversions.
A small research model: answers can be wrong, so check the program. See the results and limitations
