Airbnb: migrate a test suite

Move thousands of test files to a new framework, with each file verified by its own passing tests.

By OneShot · Updated October 8, 2026

Read the solution ↓

What the workflow produces

Test migration ledger

File status: Converted, passing, merged or handed off, per file.
Checks: Test, lint and type results for the final attempt.
Attempts: Retries used and the context added each time.
Hand-off: Engineer assigned, best attempt and remaining errors.

The documented problem

Airbnb’s engineering blog describes migrating nearly 3,500 React component test files from Enzyme to React Testing Library. The team first estimated the work at 1.5 years of engineering time by hand.

Using large language models and an automated pipeline, Airbnb reports finishing in six weeks: 75% of the files migrated in four hours, 97% after four days of tuning prompts and scripts, and the remainder fixed manually. These are Airbnb’s results from its own tooling. The OneShot workflow below is a proposed design for running a migration like it.

Airbnb Tech Blog: Accelerating large-scale test migration with LLMsPrimary source · Checked 2026-10-08

The OneShot solution

Bulk test migration with tests as the check

Move every in-scope test file to the new framework with its tests passing and its original intent preserved.

Work enters the queue: An approved migration plan, a batch of files ready to convert or a file that has used up its retries.
Decision owner: Migration lead

01
Sample and set the rules
Convert a small sample first. Agree with the migration lead what passing means: the file’s tests run green, lint and type checks pass, and no test is deleted or skipped to get there.
- Source repositories and CI
02
Convert file by file
The OneShot agent converts each file and runs its tests. A file moves forward only when every check passes; the result and the checks run are recorded per file.
- Source repositories and CI
03
Retry with more context
For a failing file, retry with the error output, the component under test and neighbouring tests as context, up to the agreed attempt limit and budget. Stop when further attempts stop making progress.
- Source repositories and CI
04
Hand off the hard tail
Give engineers the files that still fail, with the best attempt and its errors attached. Send converted batches for review and track them to merge.
- Email send
- Email Inbox
- Source repositories and CI

What counts as complete

Every in-scope file is merged with passing tests, or assigned to a named engineer with the last attempt attached.

Decisions and exceptions

A conversion that deletes, skips or weakens a test is rejected even if the file passes.
Files that use up their attempts go to an engineer; the agent does not keep spending.
Merging follows the owning team’s normal review.

What to measure

Share of files passing automatically
Attempts per file
Files handed to engineers
Coverage before and after

Establish the baseline and review period before launch. Compare completed cases, unresolved work and reviewer corrections against the same scope.

The agent’s tool basket

OneShot gives its agents the tools to analyze records, communicate, research and act on authorized business data. The platform carries the workflow from the first action through to the recorded result.