Scanner Philosophy — Why Most Screeners Fail

Why Most Screeners Fail

Here is a stat that should bother you: over 90% of retail traders who use stock screeners lose money. Not because screening is a bad idea — it is, in fact, the foundation of every systematic trading approach — but because the way most people screen is fundamentally broken.

The typical retail workflow looks like this:

Think about what a basic screener does: it applies binary pass/fail filters to a universe of stocks. RSI below 30? Pass. Volume above 1 million? Pass. That is it. No ranking, no scoring, no context about whether the market is in a bull run or a crash. A stock that barely passes every filter gets treated the same as one that crushes every metric.

The result is a flat list of "candidates" with zero signal about which ones are actually worth trading. You are left making the final decision based on gut feel, chart pattern recognition (which is notoriously unreliable), or whatever the latest Reddit thread says.

The Three Fatal Flaws

Filter, Score, Rank, Validate

A scanner that actually generates alpha is built on four pillars. Each one is necessary. Skip any one and your scanner is a random number generator with a nice UI.

Notice how each stage dramatically reduces the number of stocks. The funnel is doing the heavy lifting for you — transforming thousands of names into a handful of high-conviction setups that you can actually research and trade.

Why Each Pillar Matters

What Institutional Scanners Do Differently

Hedge funds and prop desks do not use Finviz. They build custom systems that incorporate layers of intelligence retail traders never see. Here is what separates a $50B fund's scanner from a retail screener:

The good news: you can build most of this yourself. The data is available (often free or cheap). The computation is trivial for modern hardware. The only barrier is knowledge — which is exactly what this series provides.

Scanning 10,000 vs. Curating 500

The first decision you make when building a scanner is also the most consequential: what is your universe?

There are over 10,000 stocks listed on US exchanges alone. Add international markets and you are looking at 40,000+. Scanning all of them is technically possible but practically terrible. Here is why:

• Noise explosion: Most small-cap stocks have no institutional ownership, thin volume, and unreliable technical signals. Your scanner will drown in false positives.
• Data quality nightmare: Many micro-caps have incorrect splits, missing data, or stale prices. Your indicators will compute garbage.
• Execution impossible: Even if you find a great setup in a $20M market cap stock, the spread is 2% and you can not get filled without moving the price.
• Survivorship bias: If you only scan currently listed stocks, you miss the ones that went to zero — making your backtest look better than reality.

The Curated Universe Approach

Instead of scanning everything, institutional scanners work with a curated universe of 300-800 quality names that meet minimum criteria:

# Universe selection criteria
def build_universe(all_stocks):
    """Filter to quality names only."""
    universe = []
    for stock in all_stocks:
        if (
            stock.market_cap > 500_000_000      # $500M+ market cap
            and stock.avg_volume_20d > 500_000  # 500K avg daily volume
            and stock.price > 5.0              # No penny stocks
            and stock.listed_years >= 2        # At least 2 years listed
            and stock.has_options                # Options available (liquidity proxy)
        ):
            universe.append(stock)
    return universe  # Typically 400-800 stocks

This is not just about speed — though scanning 500 stocks is 20x faster than scanning 10,000. It is about signal quality. Every stock in your universe should be one you could actually trade: liquid enough to enter and exit, large enough that technical analysis works, and with clean enough data to compute reliable indicators.

The "Bar Quality" Problem

Your scanner is only as good as its data. This sounds obvious, but the number of scanners built on bad data is staggering. Here are the data problems you must solve before writing a single filter:

1. Splits and Dividends

When a stock splits 4:1, the historical price drops by 75% overnight — but that is not a real price move. If your scanner sees AAPL at $40 in 2019 and $175 in 2024, it looks like a 337% gain. But AAPL had a 4:1 split in August 2020, so the actual gain is different. You must use split-adjusted data.

# Always use adjusted close prices
import yfinance as yf

# WRONG: unadjusted prices include split artifacts
# data = yf.download("AAPL", period="2y")["Close"]

# RIGHT: adjusted prices account for splits and dividends
data = yf.download("AAPL", period="2y", auto_adjust=True)
close = data["Close"]
volume = data["Volume"]

# Verify: no single-day drops > 30% (split artifact check)
daily_returns = close.pct_change()
assert (daily_returns.abs() < 0.50).all(), "Possible unadjusted split detected"

2. Minimum History Requirement

Your indicators need history. An EMA(200) needs at least 200 bars to be meaningful. A Bollinger Band Width comparison needs at least 20 bars of BBW history, which itself needs 20 bars of price — so 40 bars minimum. A volume spike detection comparing current volume to 20-day average needs 20 bars.

Rule of thumb: require 252 trading days (1 year) minimum, prefer 504 days (2 years). Anything less and your indicators are unreliable in the early bars.

3. Delistings and Survivorship Bias

If you only scan currently listed stocks and backtest against the same list, you have survivorship bias. The stocks that went bankrupt, got delisted, or merged out of existence are invisible in your data. Your backtest will overestimate returns because it never saw the losers.

4. Volume Data Integrity

Volume is critical for confirmation signals, but volume data has its own problems:

• Dark pool volume is not included in exchange-reported volume. Real institutional buying may not show up in your data.
• Ex-dividend dates cause volume spikes that are not real demand signals.
• Options expiration (monthly OpEx, quarterly) inflates volume due to hedging.
• Index rebalancing (quarterly) causes non-organic volume in added/removed stocks.

# Data quality validation pipeline
def validate_bar_data(df, ticker):
    """Check data quality before feeding to scanner."""
    issues = []

    # 1. Minimum history
    if len(df) < 252:
        issues.append(f"Only {len(df)} bars (need 252+)")

    # 2. Missing data
    missing_pct = df.isnull().sum().max() / len(df) * 100
    if missing_pct > 2:
        issues.append(f"{missing_pct:.1f}% missing data")

    # 3. Suspicious returns (unadjusted splits?)
    max_daily = df["close"].pct_change().abs().max()
    if max_daily > 0.50:
        issues.append(f"Suspicious {max_daily:.0%} daily move")

    # 4. Volume sanity
    zero_vol_days = (df["volume"] == 0).sum()
    if zero_vol_days > 5:
        issues.append(f"{zero_vol_days} zero-volume days")

    if issues:
        print(f"[WARN] {ticker}: {', '.join(issues)}")
        return False
    return True

Scanner Architecture Overview

Now let us put it all together. Here is the complete pipeline that we will build over the next five parts of this series:

The Daily Cycle

A production scanner runs on a daily cycle. Every morning (or evening, depending on your market), the following happens:

The entire process takes under 30 minutes, most of which is data fetching. The computation itself is measured in seconds. This is achievable on a $5/month VPS — you do not need a Bloomberg terminal.

Scanner is Not a Strategy

What We Will Build in This Series