Research Methodology
Our framework for data collection, cleaning, and analysis ensures that Chess Multiverse studies are reproducible and academically rigorous.
Data Pipeline
We do not rely on raw PGN dumps. Every game in our repository undergoes a strict four-stage cleaning process to remove noise (engine cheaters, sandbaggers, and connection drops) before it enters our analysis pipeline.
Ingestion & Filtering
We aggregate games from open sources (Lichess) and internal experiments. We filter for "rated" games only and exclude players with unstable Glicko-2 deviations (>150).
Sanitization
We detect and flag games with suspicious move times (e.g., bot-like consistency) or 100% engine correlation. These are removed to ensure human-centric data.
Annotation
Using Stockfish 16.1 (depth 20), we annotate key moments. We specifically track eval swings > 2.0 to identify critical errors and psychological turning points.
Evaluation Standards
-
Engine Depth: All final analysis is conducted at a minimum depth of 22 ply using Stockfish 16 NNUE.
-
Time Normalization: "Rapid" is defined strictly as 10+0 to 15+10. "Blitz" is 3+0 to 5+5. Bullet games are excluded unless specified.
-
Rating Brackets: We segment players into strict rating bands (e.g., 1500-1800) to ensure comparisons are skill-relevant.
Reproduce Our Work
We believe in open science. You can download the exact scripts and datasets we use for our analysis.