DiScoFormer: One transformer for density and score, across distributions
DiScoFormer: One transformer for density and score, across distributions DiScoFormer: One transformer for density and score, across distributions Enterprise Article Published June 29, 2026 Upvote 6 Kyle Wiggers Ai2Comms allenai 📄 Tech report: arxiv.org/abs/2511.05924 Many problems in machine learning and the sciences come down to the same task: you have a collection of data points and want to recover the distribution they came from—which values are common, and which are rare. Pinning down that distribution means estimating two quantities: the distribution's density and, more useful as dimensionality grows, its score.
This Research is relevant to the technology intelligence record because it involves ArtificialIntelligence activity. The source article should remain the factual reference for follow-up coverage.
- DiScoFormer: One transformer for density and score, across distributions Enterprise Article Published June 29, 2026 Upvote 6 Kyle Wiggers Ai2Comms allenai 📄 Tech report: arxiv.org/abs/2511.05924 Many problems in machine learning and the sciences come down to the same task: you have a collection of data points and want to recover the distribution they came from—which values are common, and which are rare.
- Pinning down that distribution means estimating two quantities: the distribution's density and, more useful as dimensionality grows, its score.
- The density is the smooth version of a histogram—high where points cluster and low where they're scarce.
- The score—the gradient of the log-density—points in the direction the density rises fastest: move a point along the score and it heads toward a more probable region.
- Diffusion-based generative models (the technology behind AI image generators like Stable Diffusion and DALL-E) start from random noise and repeatedly follow the score, turning that noise into a realistic image.
- The same score drives Bayesian sampling and the particle simulations used to model systems such as plasma.