OpenAI GPT-6 Sol Scores 97.9% on COBOL Migration: What Students Must Check
OpenAI's GPT-6.1 Sol scored 97.9% on a COBOL migration test at high reasoning. That score is not a guarantee of correct real-world results, so here is what freshers need to verify before trusting translated code.
OpenAI's GPT-6.1 Sol scored 97.9% on a COBOL migration test at high reasoning, according to Interesting Engineering's report. That score comes from OpenAI's own test suite, and only one news report gives the exact 97.9% figure. OpenAI's own pages do not show it, so treat it as reported, not confirmed.
A high score can also hide a specific problem. COBOL stores money as fixed-point numbers, and it can quietly drop extra decimal digits unless the code asks for rounding (RMN Digital explains this). If the translated code rounds differently, it can print a different rupee amount and never show an error.
97.9% is a score on OpenAI's own COBOL migration test, not a guarantee that a real migration is correct. In the test, the model converts COBOL to a language like Java or Python, and the new code must match the old output byte for byte. Real systems have hidden dependencies and money rounding rules that a test suite may not cover, so a human still has to verify the result.
OpenAI GPT-6 COBOL migration test: what was announced
On Oct 19, 2026 OpenAI announced new evaluations for low-resource programming languages, covering software engineering and SQLite tasks, plus COBOL migration (Blockchain.News).
OpenAI's post was titled "Advancing Coding in Low-Resource Programming Languages". A "low-resource" language is one with little public code for a model to learn from, such as COBOL or Fortran (RSWebSols).
The announcement did not list any scores or name the languages tested, Blockchain.News reports. It introduced new tests. The numbers below come from news coverage of the announcement and from the public benchmark repo.
OpenAI changed other things around the same time. GPT-6 Pro was updated on Oct 16, 2026, and Codex Cloud arrived on Sept 29, 2026 (Blockchain.News). OpenAI's own developer community announcement covers the Sol and Luna models. This post covers two separate tests. The COBOL migration test (the 97.9% score) uses GPT-6.1 Sol. MultiLangBench is a different test and uses GPT-5.6 Sol and GPT-6 Astra. Compare a model only with the one named in its own row or sentence.
How the 97.9% score was measured
The test asks a model to convert COBOL into C++, Java, Kotlin, Python or Rust while keeping the output identical byte for byte, and 97.9% is the result at high reasoning (Interesting Engineering).
"Byte for byte" means every character must match, including spaces and leading zeros. Say the old program prints a balance with a zero in front. The new program must print that zero as well. Output that is only close counts as a fail.
| Setting | Model | Score | Note |
|---|---|---|---|
| Low reasoning | GPT-5.6 Sol (older model, setup unknown) | 5.4% | Not comparable with the rows below |
| Low reasoning | GPT-6.1 Sol | 86.6% | 39% lower API cost |
| High reasoning | GPT-6.1 Sol | 97.9% | The headline number |
Source for the table: Interesting Engineering. The 5.4% is as reported there, and the report does not say exactly what older setup it was measured on, so read the jump to 86.6% with care. The 39% lower API cost is also as reported, and I could not confirm what baseline it is lower than.
"Reasoning" here means how long the model thinks before it answers. Longer thinking costs more money and takes more time. The step from 86.6% to 97.9% is what that extra cost pays for (Interesting Engineering).
The coverage does not all say the same thing. The Decoder says the model does better at "higher reasoning" but does not give the same exact figure.
MultiLangBench scores across old languages
This section is about a different test from the 97.9% one, and it uses different models. On the open-source MultiLangBench, GPT-6 Astra averages 87.4% across 20 languages, but COBOL scores lower at 72.1% (RMN Digital).
Here the model must build a SQLite-compatible database engine from scratch. SQLite is the small database that runs inside phones and browsers. The MultiLangBench repo targets SQLite 3.40.1 and uses 272 tests converted from SQLite's TCL tests, under an Apache-2.0 licence. Each result gets two scores: API cost and test pass rate.
Test pass rates from RMN Digital.
COBOL has the lowest score of the four languages in the table. That matters, because COBOL is also the language that handles the most money.
There is also a cheaper model. GPT-6 Luna averages 40.3% at $0.77 per sample, which is 78% cheaper than GPT-5.6 Sol at $3.48 (RMN Digital). None of the sources gives a rupee price, so these figures stay in dollars.
Check which older model each article compares against. The Decoder uses GPT-5.6 Terra, at about 38%, as its starting point (The Decoder). That is a different model from GPT-5.6 Sol. If you quote how much a model improved, name the model you are comparing it with.
A high COBOL score does not mean error-free migrations
A 97.9% score means some tests still fail, and passing a test suite is not the same as being correct on your real data (Interesting Engineering).
Outside OpenAI, nobody yet knows which cases failed. In a bank's code, the failures could be the rare cases that matter most, like year-end interest or a closed account.
The biggest risk is in the money maths. COBOL keeps a fixed number of decimal places. If a result has more digits than the field can hold, COBOL can cut off the extra digits unless the programmer wrote explicit rounding (RMN Digital). A translation that rounds where the old code truncated still runs. It just produces different amounts.
| Exact result (made-up example) | Old COBOL (truncates) | New code (rounds) |
|---|---|---|
| Ten rupees and twenty-six and a half paise | Ten rupees, twenty-six paise | Ten rupees, twenty-seven paise |
| Nineteen and nine-tenths paise | Nineteen paise | Twenty paise |
| Exactly one-third of a rupee | Thirty-three paise | Thirty-three paise |
Look at the last row. The two programs agree on that amount, and a test that only checks amounts like it would pass. Neither program shows an error. They disagree only on the amounts in the first two rows.
The second risk is dependencies that nobody documented. Interesting Engineering notes that developers must still test against real workloads and undocumented dependencies. A COBOL program often relies on things outside its own file. Copybooks (shared data layouts), overnight batch jobs and files written by other programs are common examples. If nobody wrote them down, no benchmark can test them.
What the benchmark shows
- The model can translate COBOL that it has never seen in that exact form.
- Output can match byte for byte on a fixed set of tests.
- Newer models do far better than older ones.
Where the benchmark falls short
- Whether your system's rounding rules were kept.
- Whether hidden batch jobs and copybooks still work.
- How the code behaves on your real customer data.
These scores come from OpenAI and were reported by the press. Read the primary pages yourself before you repeat them: the benchmark repo and OpenAI's developer announcement.
Why COBOL mistakes cost real money
COBOL runs a large share of banking, so a silent rounding error can reach many accounts before anyone notices.
As quoted by RMN Digital and The Decoder, IBM estimates that COBOL powers over 40% of online banking, 80% of in-person credit card transactions and 95% of ATM transactions. These are global estimates, and neither outlet gives figures for Indian banks.
Here is a made-up example. Say you join a TCS or Infosys team that is moving an Indian bank's interest system to Java. An AI tool translates the code. The Java version rounds halves up, but the old COBOL truncated. Every test passes, because the test data used round amounts.
After launch, each account is off by one paisa a month. That looks tiny. But the same error repeats across millions of savings accounts, every month. Months later, an auditor finds a gap in the books, and the team has to trace it back through code that no error log ever flagged.
Verify AI-migrated code before you trust it
Treat the AI output as a first draft and prove it correct with tests built from real data.
Collect real samples
Pull real inputs and the matching outputs from the old system, with sensitive data masked.
Run both versions side by side
Feed the same input to the old and new code, then diff the outputs line by line.
Test rounding on purpose
Use amounts that end in half a paisa, negative values and very large balances.
Hunt for hidden dependencies
List every batch job, input file and copybook the program touches.
Get a senior review
Ask an engineer who knows the business rules to sign off before release.
The MultiLangBench repo is a good model for building a scored test set. It turns known behaviour into tests and records two numbers for each run: cost and pass rate. You can apply the same idea at a smaller scale to your own project.
Cost goes up with accuracy. On critical code, pay for the higher setting and still run your own tests. If you are comparing cheaper models, my post on DeepSeek V4.1 Flash looks at the same trade-off.
What this means for freshers in India
Learning to check migrated code is a more useful skill for a fresher than hoping the AI is right (The Decoder covers why legacy languages are getting this attention).
Large services firms do a lot of legacy modernisation work for banks and insurers. If you are heading into campus placements, you may be put on a project like this. My view: the AI will do most of the translation, so your value is in checking that translation and in knowing the business rules behind it. I explain how this fits the wider hiring picture in the junior developer job market in India.
Do
- Compare old and new outputs on real data.
- Read the original COBOL, even slowly.
- Ask your lead what the rounding rules are.
Don't
- Accept a benchmark number as proof your code is correct.
- Skip tests because the model scored high.
This is also why I think writing prompts is a weaker skill than engineering. Anyone can ask a model to translate code. Fewer people can prove the translation is correct. I compare the two career paths in software engineer vs prompt engineer. A diff-based check like the one above also makes a strong take-home assignment submission.
Here is a task for this week. You can run COBOL for free with GnuCOBOL. Take one small COBOL program, or any fixed-point money calculation (a number with a set count of decimal places), and ask an AI tool to convert it to Python or Java. Then diff the output against the original on twenty edge-case amounts. After that, open the MultiLangBench repo on GitHub and read how its tests are scored.
