End-to-end training of sparse deep neural networks with little-to-no performance loss.
-
Updated
Jan 26, 2023 - Python
#
sparse-training
Here are 18 public repositories matching this topic...
Always sparse. Never dense. But never say never. A Sparse Training repository for the Adaptive Sparse Connectivity concept and its algorithmic instantiation, i.e. Sparse Evolutionary Training, to boost Deep Learning scalability on various aspects (e.g. memory and computational time efficiency, representation and generalization power).
deep-neural-networks sparsity deep-learning scalability randomization neuroevolution deep-learning-algorithms classification evolutionary-algorithms artificial-neural-networks restricted-boltzmann-machine complex-networks deep-learning-papers multi-layer-perceptron generative-models sparse-neural-networks scalable-deep-learning sparse-training sparse-evolutionary-training adaptive-sparse-connectivity
-
Updated
Jul 21, 2021 - Python
[NeurIPS'21] "Chasing Sparsity in Vision Transformers: An End-to-End Exploration" by Tianlong Chen, Yu Cheng, Zhe Gan, Lu Yuan, Lei Zhang, Zhangyang Wang
pruning model-compression sparse-training vision-transformers efficient-transformers dynamic-sparsity token-slimming
-
Updated
Dec 1, 2023 - Python
[ICML 2021] "Do We Actually Need Dense Over-Parameterization? In-Time Over-Parameterization in Sparse Training" by Shiwei Liu, Lu Yin, Decebal Constantin Mocanu, Mykola Pechenizkiy
sparsity deep-learning generalization sparse-training over-parameterization overparameterization in-time-overparameterization dynamic-sparse-training in-time-over-parameterization
-
Updated
Nov 11, 2023 - Python
[ICML2022] Training Your Sparse Neural Network Better with Any Mask. Ajay Jaiswal, Haoyu Ma, Tianlong Chen, ying Ding, and Zhangyang Wang
-
Updated
Jul 24, 2022 - Python
[Machine Learning Journal (ECML-PKDD 2022 journal track)] Quick and Robust Feature Selection: the Strength of Energy-efficient Sparse Training for Autoencoders
sparsity deep-learning feature-selection autoencoder dimensionality-reduction denoising-autoencoders sparse-neural-networks sparse-training
-
Updated
Oct 2, 2023 - Python
[IJCAI 2022] "Dynamic Sparse Training for Deep Reinforcement Learning" by Ghada Sokar, Elena Mocanu , Decebal Constantin Mocanu, Mykola Pechenizkiy, and Peter Stone.
deep-neural-networks sparsity reinforcement-learning deep-learning deep-reinforcement-learning sparse-neural-networks sparse-training continuous-control-tasks
-
Updated
May 13, 2022 - Python
Actually-sparse dynamic training for PyTorch. CPU-native, Apple Silicon first. Pluggable routers, drop-in SparseLinear.
deep-learning pytorch pybind11 sparse-neural-networks sparse-training apple-silicon neon-intrinsics dynamic-sparse-training cpu-optimization rigl
-
Updated
May 19, 2026 - Python
Code for "Training Adversarially Robust Sparse Networks via Bayesian Connectivity Sampling" [ICML 2021]
-
Updated
Mar 14, 2022 - Python
[ICML 2021] "Selfish Sparse RNN Training" by Shiwei Liu, Decebal Constantin Mocanu, Yulong Pei, Mykola Pechenizkiy
recurrent-neural-networks awd-lstm onlstm sparse-training dynamic-sparse-training awd-mos-lstm sparse-rnn-training
-
Updated
Oct 8, 2021 - Python
Implementation for the paper "SpaceNet: Make Free Space For Continual Learning" in PyTorch.
deep-neural-networks deep-learning incremental-learning sparse-representations brain-inspired continual-learning catastrophic-forgetting sparse-neural-networks sparse-training life-long-learning
-
Updated
Feb 28, 2021 - Python
[Machine Learning Journal (ECML-PKDD 2022 journal track)] A Brain-inspired Algorithm for Training Highly Sparse Neural Networks
-
Updated
Feb 20, 2023 - Python
PyTorch Implementation of TopKAST
-
Updated
Dec 28, 2022 - Python
[TMLR] Supervised Feature Selection with Neuron Evolution in Sparse Neural Networks
python deep-learning keras efficiency high-dimensional-data feature-selection neural-networks sparse-neural-networks sparse-training dynamic-sparse-training
-
Updated
Feb 12, 2023 - Python
This is the repository for the paper on "Generalization and Memorization in Sparse Neural Networks".
-
Updated
Feb 15, 2023 - Python
Structured pruning and sparse training for PyTorch models.
-
Updated
Oct 19, 2026 - Python
A small, readable framework for reducing redundant computation in neural network training — loss-based example selection (TopK, AdaptiveK, MeanAdaptive) and low-rank gradient approximation. Includes 300-epoch ViT-B/16 ImageNet-1k results. BSc thesis code (NKUA, 2025).
machine-learning research deep-learning pytorch imagenet deeplearning cifar10 loss-functions data-selection efficient-training sparse-training vision-transformer
-
Updated
Sep 12, 2026 - Python
Pre-registered study of dynamic sparse training in 3D medical segmentation (nnU-Net, BraTS-MEN). v1 pilot void by a design error found after it ran; v2 gates proposed and not in effect, no v2 arm run yet.
reproducible-research pytorch medical-imaging image-segmentation sparse-training nnunet dynamic-sparse-training
-
Updated
Aug 2, 2026 - Python
Add this topic to your repo
To associate your repository with the sparse-training topic, visit your repo's landing page and select "manage topics."