Maintainers
Core team
Want to help maintain ScalingDepth? Reach out on Discussions.
ScalingDepth is open and community-driven. Researchers and engineers who care about depth, residual streams and scaling are all welcome.
Ask questions, propose methods, share negative results and discuss open problems.
Open an issue with the paper link, and we or another contributor will add it to the collection.
Evaluate your residual design across width–depth ratios with open training code, or add a new benchmark.
Fork keyu-wang-2002/ScalingDepth on GitHub.
Append an entry to SD.methods (or SD.papers, SD.tutorials, SD.blogs) in assets/js/data.js. Formulas use KaTeX syntax.
Preview locally with python3 -m http.server. Once your PR is merged, the site updates on GitHub Pages.
{
id: "my-residual", // unique, kebab-case
name: "MyResidual",
family: "aggregation", // classic | norm | scaling |
// aggregation | multistream
year: 2026,
authors: "Doe, Smith",
venue: "NeurIPS 2026",
arxiv: "2610.00000",
code: "https://github.com/...", // optional
formula: "x_{l+1} = x_l + F_l(x_l)",
desc: "One or two sentences on the idea.",
tags: ["dynamic weights"]
}
A shared playground for racing residual designs. Codebase is coming soon!
Want to help maintain ScalingDepth? Reach out on Discussions.
