We counted the input token of the prompts generated by LISP, as well as the output token of the LLM’s response, and calculated the amount spent.
Token Consumption: The pricing of gpt-3.5-turbo we use is “US$0.50 per 1M input tokens”and “US$1.50 per 1M output tokens”
LISP
| # Input Token | # Output Token | # Cost | |
|---|---|---|---|
| commons-lang3 | 19,877,517 | 1,726,830 | 12.529812 |
| JFreeChart | 3,020,835 | 324,240 | 1.9969238 |
| JGraphT | 2,341,596 | 251,334 | 1.5479114 |
| guava | 3,485,580 | 374,121 | 2.3041429 |
| joda-time | 3,306,834 | 354,936 | 2.1859817 |
| threeten | 4,247,010 | 435,843 | 2.7774227 |
| time4j | 2,804,631 | 287,820 | 1.834147 |
| ical4j | 8,053,296 | 826,458 | 5.2666223 |
| sis-utility | 15,889,005 | 1,255,674 | 9.828611 |
| xchart | 3,926,481 | 402,951 | 2.5678059 |
| # SUM | 66,952,785 | 6,240,207 | 42.8393807 |
LISP-CG
| # Input Token | # Output Token | # Cost | |
|---|---|---|---|
| commons-lang3 | 20,764,098 | 2,127,645 | 13.574352 |
| JFreeChart | 5,544,456 | 411,198 | 3.389278 |
| JGraphT | 4,297,776 | 318,741 | 2.627192 |
| guava | 6,397,452 | 474,462 | 3.910706 |
| joda-time | 6,069,378 | 450,129 | 3.710157 |
| threeten | 5,219,025 | 490,041 | 3.34479 |
| time4j | 3,446,526 | 323,613 | 2.208823 |
| ical4j | 9,896,454 | 929,232 | 6.3424792 |
| sis-utility | 17,849,982 | 1,751,994 | 11.553664 |
| xchart | 4,825,137 | 453,057 | 3.092353 |
| # SUM | 84,310,284 | 7,730,112 | 53.7537942 |
LLM-Baseline
| # Input Token | # Output Token | # Cost | |
|---|---|---|---|
| commons-lang3 | 2,931,549 | 540,330 | 2.2764086 |
| JFreeChart | 909,048 | 167,553 | 0.705896 |
| JGraphT | 704,646 | 129,879 | 0.547173 |
| guava | 1,048,902 | 193,329 | 0.814495 |
| joda-time | 995,112 | 183,414 | 0.772726 |
| threeten | 570,174 | 105,093 | 0.442751 |
| time4j | 376,530 | 69,399 | 0.292383 |
| ical4j | 1,964,694 | 353,664 | 1.5129235 |
| sis-utility | 4,936,167 | 888,558 | 3.801126 |
| xchart | 957,909 | 172,434 | 0.737644 |
| # SUM | 15,394,731 | 2,803,653 | 11.9035261 |