Why Financial Data Quality Matters More Than Quantity for Analysis

When I ran product engineering and AI teams at American Express, the analytical projects that went sideways rarely had anything wrong with the math. The models were fine. The data underneath them was the problem, usually in ways nobody noticed until the conclusions had already shipped to a decision maker.

The instinct across the industry runs the other way. When an analysis feels thin, we add data. More companies, more metrics, more years of history, an alternative data feed somebody wants to try. But if the underlying numbers are wrong, inconsistent, or misaligned, more volume just produces more noise, delivered with more confidence. So this post is a practical tour of where financial data actually breaks, how those breaks flow through real analysis, and the checks I use to catch problems before they turn into decisions.

The Five Ways Financial Data Goes Bad

Most quality problems fall into one of five buckets, and it helps to know them by name because each one has a different fix.

Transcription and unit errors

The dumbest errors are the most dangerous because nobody looks for them. Financial statements are reported in stated units, usually thousands or millions, and every hop between systems is a chance for that scale to get lost. A revenue line reported in thousands gets ingested as raw dollars. A figure gets keyed as $1.2 billion when the filing says $12 billion. A currency code gets dropped and a yen-denominated number lands in a dollar comparison looking wildly off. These errors propagate silently: a screen ranks the company first or last, a ratio goes haywire, and if nothing in your process asks whether the number is even plausible, the error rides along into the final output.

Definitional drift

This one is subtler and, honestly, more common. Different providers define the same metric differently. EBITDA is the classic offender. One source starts with operating income and adds back depreciation and amortization. Another starts with net income and adds back interest, taxes, and the rest. Companies themselves publish adjusted EBITDA with their own list of add-backs, and the list can change year to year. Everything gets labeled EBITDA, and none of it has to match.

Free cash flow is just as slippery. Operating cash flow minus capital expenditures is the textbook version, but you'll find variants that exclude working capital swings, fold in acquisitions, or subtract stock-based compensation. Net debt shifts depending on whether leases and pension obligations count. When you compare ten companies on one metric, you're trusting that whoever assembled the data made the same choices ten times. Often they didn't, because the companies themselves disclose differently and the vendor had to pick something.

Temporal misalignment

Fiscal years are not calendar years. Walmart's fiscal year ends January 31, so its fiscal 2025 covers a different stretch of the economy than a calendar-year peer's 2024. Compare the two without adjustment and you're partly measuring the calendar rather than the companies. Trailing twelve month figures help, but they have to be assembled from quarterly data carefully, and quarterly data brings its own revisions.

The same problem shows up inside a single company's history. When a business changes its fiscal year end or reclassifies a segment, the historical series you pull may or may not reflect the change. A clean ten-year revenue trend can have a definitional seam right in the middle, and the chart won't show it.

Restatements and revisions

Companies restate their financials more often than casual readers expect, and when they do, history contains two versions of the numbers. Data vendors handle this differently. Some overwrite history with as-restated figures, which is what you want for understanding the business today. Others preserve as-reported figures, which is what you want for backtesting, because an investor at the time only knew the original number. If you don't know which convention your source follows, you don't entirely know what you're analyzing.

Survivorship and look-ahead bias

These two mostly matter for backtesting, and both flatter your results. Survivorship bias creeps in when your universe only includes companies that still exist, so every bankruptcy and delisting quietly vanishes from history and your strategy looks smarter than it was. Look-ahead bias uses information before the market had it, for example treating a fiscal year figure as known on January 1 when the 10-K wasn't filed until weeks later. The fix for both is point-in-time data, where every value carries the date it became public rather than only the period it describes.

How One Bad Number Moves Through an Analysis

It helps to trace the blast radius. Screening is the most vulnerable step because it's binary. Say you screen for companies trading under eight times EV/EBITDA. A company whose EBITDA got doubled by a definitional mismatch sails in looking cheap. One whose figure got halved silently drops out, and you'll never see it again to question it. Inclusion errors at least get a second look when you review the results. Exclusion errors are invisible, which makes them worse.

Valuation models compound errors rather than just carrying them. Take a deliberately simple example. Say a company reports $100 million in revenue and you believe it grows 10 percent a year. Five years out you're modeling roughly $161 million. If a data problem nudges your growth input to 12 percent, you get roughly $176 million instead. Then you apply an exit multiple to the inflated figure and discount it back, and a small upstream error has quietly become the biggest driver of your valuation. Every formula in the spreadsheet is correct. Only the input was bad, and no amount of auditing the formulas will ever surface it.

Comparative work degrades more quietly. If half your comp set reports lease-adjusted debt and half doesn't, your leverage ranking is partly a ranking of accounting presentation. Trend analysis has the same failure mode across time instead of across companies: a methodology change at the data provider shows up looking like a business inflection, and you end up writing a narrative about a company when the real story is a vendor's schema migration.

Validation Checks That Catch Most of It

The encouraging part is that a handful of cheap checks catch a large share of problems. They need no fancy tooling, just the decision to treat validation as part of the analysis itself.

• Range and sanity checks. Revenue should be positive. Margins should land somewhere defensible for the industry. An operating margin of 400 percent is a data problem, and so is a negative share count. Write the rules down and run them on everything, every time.
• Internal consistency. The statements tie to each other. Assets should equal liabilities plus equity. Net income at the top of the cash flow statement should match the income statement. Ending cash should reconcile to the balance sheet. When these don't tie, something got mangled in extraction.
• Cross-source validation. Pull the same metric from two providers, or from a provider and the filing itself, and compare. Exact matches build confidence. Systematic mismatches usually expose a definitional difference you needed to know about anyway.
• Time series continuity. Flag any period-over-period change beyond a threshold and make a human look at it. Real businesses do jump around, especially through acquisitions, but the flag forces the question, and the answer is either an interesting fact about the company or a bug. Both are worth finding.
• Source lineage. Track where every number came from, down to the filing and line item if you can manage it. When an error surfaces, lineage is the difference between a five minute fix and an afternoon of archaeology.

When in Doubt, Go to the Filing

For US public companies the primary source is free, and it gets used far less than it should. Every 10-K, 10-Q, 8-K, and proxy statement lives on EDGAR, and the full-text search works better than most people expect. When a vendor number looks strange, the fastest resolution is usually the filing itself. A few specific habits pay off.

• Check the units line. Every statement declares its reporting units near the top, usually "in thousands" or "in millions." Ten seconds of reading prevents the most embarrassing class of error.
• Read the non-GAAP reconciliation. When a company promotes adjusted EBITDA or adjusted earnings, SEC rules under Regulation G require a reconciliation back to the most directly comparable GAAP measure. That table tells you exactly what got added back, and whether you would have made the same call.
• Mine the footnotes. Segment definitions, lease obligations, pension assumptions, and revenue recognition policies all live in the notes. When a metric changes methodology, the footnotes usually say so, and the vendor feed usually doesn't.
• Use the proxy statement for anything about people. Executive compensation, board relationships, and related-party transactions sit in the DEF 14A, and they're often more revealing than anything in the 10-K.

One caveat on structured data. Filings carry XBRL tags, which makes them machine readable, but companies can define custom extension tags for their own line items and tagging quality varies. XBRL is a great starting point for automation and a bad thing to trust blindly.

If You're Building a Pipeline

I've spent most of my career building data systems, and the failures cluster at the seams, wherever data moves between formats, vendors, or teams. A pipeline that holds up tends to share five traits.

1. Anchor on the most authoritative source. For US equities that means the filing. Vendor data is a convenience layer on top, valuable for standardization, but it shouldn't be the arbiter when numbers disagree.
2. Normalize definitions explicitly. Write down how the system computes every derived metric, including the boring ones. A single sentence like "free cash flow equals operating cash flow minus purchases of property and equipment" prevents years of ambiguity.
3. Version your data. Store as-reported and as-restated values separately, keep point-in-time stamps, and make it possible to rerun last quarter's analysis on last quarter's data. Restatements and vendor corrections will happen, and you want them to be visible events instead of silent mutations.
4. Validate at ingestion. Every load runs the range checks, consistency checks, and continuity flags automatically, and failures get quarantined instead of flowing into the main tables. Waiting until an analyst notices something odd means the bad data already had an audience.
5. Track quality itself as a metric. Watch validation failure rates, cross-source match rates, and outlier counts per load. Quality tends to erode slowly and then suddenly, usually when an upstream provider changes something without telling you, and a trend line is how you catch it early.

Working With Data You Know Is Imperfect

Perfect data doesn't exist, and waiting for it is its own failure mode. What works in practice is being honest about the flaws.

Document known issues next to the analysis they affect. If segment data before a certain year is unreliable because of a reclassification, say so where the reader will actually see it, because a caveat buried in a separate wiki page might as well not exist. And triangulate anything that matters. A conclusion resting on a single data point from a single source is a guess wearing a suit.

Build sensitivity analysis around the inputs you trust least. If a valuation swings hard when one uncertain assumption moves within its plausible range, the honest output is a range, and presenting a single number would be theater. Sometimes the right move is to decline the analysis altogether. When the data underneath a question can't support a real answer, a confident wrong answer does more damage than admitting you don't know yet.

The principle I keep coming back to is that a simple analysis built on numbers you've verified beats a sophisticated model built on numbers you haven't. The model gets the attention because it's the interesting part, but the data decides the outcome. So before adding another source, another decade of history, or another thirty metrics, spend an afternoon validating what you already have. It's unglamorous work, and it pays off on nearly every analysis that follows.