Formatting a 25M-line codebase overnight
Stripe has successfully reformatted its entire 25-million-line Ruby codebase in 12 hours using a custom-built formatter called rubyfmt, marking a significant engineering effort in large-scale code maintenance. The operation relied on a distributed computing setup comprising a 100-node cluster and a novel approach combining incremental parsing with parallel processing to achieve 99.9% formatting accuracy. This effort underscores the challenges and solutions involved in maintaining consistency across massive, long-lived codebases at scale. ## Overview The Stripe codebase, written primarily in Ruby, had accumulated formatting inconsistencies over years of development. Introducing a uniform style manually or with standard tooling would have been impractical due to size and complexity. Instead, the team developed rubyfmt, a custom formatter designed specifically for Stripe’s code patterns and syntax extensions. Unlike general-purpose formatters, rubyfmt was built to handle nonstandard Ruby constructs used internally, ensuring high fidelity during reformatting. The formatter was not applied in a single-threaded manner. To accelerate processing, Stripe engineers implemented a parallel execution model across 100 nodes. Each node processed a subset of files, with workloads distributed to maximize CPU utilization and minimize idle time. The system leveraged incremental parsing to avoid reprocessing unchanged syntactic structures, reducing computational overhead and improving speed. ## What it does rubyfmt parses Ruby source files and applies a deterministic formatting style based on predefined rules. It supports Stripe-specific syntax variations that deviate from standard Ruby, which existing tools like RuboCop or standard formatters cannot handle reliably. The tool operates in two phases: first analyzing the abstract syntax tree (AST) with modifications to support Stripe’s dialect, then generating formatted output while preserving semantic equivalence. Key technical aspects include: - **Distributed execution**: The formatting job was split across 100 machines to enable concurrent processing. - **Incremental parsing**: Only changed or complex syntactic regions were deeply analyzed, reducing redundant computation. - **High accuracy**: Achieved 99.9% correctness rate, minimizing manual intervention post-format. - **Idempotency**: Repeated runs produce identical output, ensuring stability in CI pipelines. The entire process completed in 12 hours, after which the formatted code was reviewed, tested, and merged into the main branch with minimal disruption to ongoing development. ## Tradeoffs While the outcome was successful, the approach required significant upfront investment in tooling and infrastructure. Building a custom formatter is not a viable path for most organizations due to maintenance burden and engineering cost. Additionally, running a 100-node cluster for 12 hours represents substantial compute usage, though justified by long-term gains in code readability and maintainability. There is no public release
