Repositoryjosch14/double-descentResearch
Double descent bridges classical statistics (which warns that larger models overfit) and modern deep learning practice (where overparameterised models often perform better). This project offers reproducible empirical evidence on how noise and capacity interact.
Implemented in Python 3.10 using PyTorch 2.0.1, scikit-learn, and Matplotlib, this repository serves academic researchers and students studying statistical learning theory. With a modest footprint (15 stars, 1 fork) and an academic origin, it is structured as an experimental testbed and reproducible report rather than production software.
Summary generated by AI (Gemini 3.8 Flash, 27 September 2026). It may contain errors – the original source is authoritative.
Similar in content, found by the AI search.
Link:Jared Wilber & Brent Werness
MLU-explAIn illustrates the double descent phenomenon, demonstrating why extremely complex, over-parameterised models can achieve lower test errors beyond the classical bias-variance threshold.
Article:Mikhail Belkin, Daniel Hsu, Siyuan Ma, Soumik Mandal
A research team led by Mikhail Belkin resolves the apparent contradiction between classical learning theory and modern over-parameterised models. The authors propose a unified 'double descent' performance curve explaining why model performance improves when capacity is increased beyond the interpolation threshold.
Repository:karpathy/nanoGPT
nanoGPT by Andrej Karpathy is a lean and fast codebase designed for training and fine-tuning medium-sized GPT models in PyTorch. Developed as a rewrite of minGPT, it can reproduce GPT-2 (124M). As of November 2025, the repository is officially marked as deprecated by the author, who recommends nanochat instead.
62.8kstarsPython
Repository:aipoch/open-science
Open Science by AIPOCH is an open-source, local-first, model-agnostic desktop workbench that provides scientific AI agents for reproducible research across macOS, Windows, and Linux.
3.5kstarsTypeScript
Repository:volotat/mini-AGI
mini-AGI is an experimental open-source byte-level language model designed to demonstrate continual learning from a single stream of data (batch size 1) on modest 8 GB VRAM hardware. By paging model weights from disk into GPU memory as needed, it decouples parameter capacity from VRAM limits while dynamically scaling its architecture.
793starsPython
Repository:test-time-training/e2e
This repository provides the official JAX implementation of End-to-End Test-Time Training (TTT) for long contexts. It approaches long-context language modeling as a continual learning challenge, using a standard Transformer with sliding-window attention that compresses input contexts into its weights at test time via next-token prediction.
693starsPython
Parts of this content were created with the assistance of AI.
