Market data preparation including OHLCV resampling, gap handling, anomaly detection, normalization, and multi-source merging

OHLCV Processing — Market Data Preparation Clean, consistent OHLCV data is the foundation of every trading analysis. Garbage in, garbage out — a single anomalous candle can trigger false signals, corrupt indicator calculations, and produce misleading backtest results. This skill covers the full data preparation pipeline: validation, cleaning, resampling, normalization, and multi source merging. Why this matters : Crypto OHLCV data is messier than traditional markets. 24/7 trading means no official close, DEX aggregators disagree on prices, low liquidity tokens produce impossible candles, and API outages create gaps. Every analysis workflow should start with this pipeline. Quick Start 1. Install Dependencies 2. Standard OHLCV DataFrame Format All processing functions expect this canonical format: 3. Full Processing Pipeline Data Validation Column Checks Structural Validation Impossible Candle Detection Gap Handling Crypto trades 24/7, but gaps still occur from API outages, low liquidity, or aggregator downtime. Detect Gaps Fill Gaps Anomaly Detection See references/data quality.md for the complete anomaly taxonomy. Price Spike Detection Zero Volume Detection Composite Anomaly Flagging Resampling See references/resampling guide.md for detailed guidance. Standard Resample Common Timeframe Ladder VWAP Calculation Normalization Multi Source Merging When combining data from multiple sources (e.g., Birdeye + DexScreener), timestamps may not align and prices may differ due to different DEX aggregation. Timezone Handling Standard : Always store and process in UTC. Convert only for display. Data Quality Report Files References references/data quality.md — Anomaly types, detection methods, correction strategies, crypto specific data issues references/resampling guide.md — Resample rules, timeframe use cases, partial bar handling, VWAP resampling, multi timeframe alignment Scripts scripts/process ohlcv.py — Full processing pipeline: validate, clean, resample, normalize with anomaly reporting (run with demo for synthetic data) scripts/merge sources.py — Multi source OHLCV merging with conflict resolution and discrepancy reporting (run with demo )

ohlcv-processing