INTRODUCTION¶
Learning PyTorch+ssnTorch > Go To Page
A fundamental differences between SNN like in the brain and modern deep learning ANN is that the brain encodes information in spikes of activity rather than a continuous activations or inhibition (Figure 1). Found a python based framework for modelling SNN > called snnTorch, which is based on the ANN framework PyTorch. GOAL :Learning PyTorch + snnTorch > unlocks SNN modelling > Neuromorphic algorithms > applications
Figure 1 – A skematic representation of a SNN neuron receiving input in the form of spikes over a period of time, and responding to that input with with a spike
PyTorch is an optimized tensor library for deep learning using GPUs and or CPUs. Key features include: "GPU acceleration" and "automatic differentiation" capabilities, which as I understand it, will save me time on tedious programming work and makes programming a neural network, Arificial or Spiking, much smoother.
- A tensor is a multi-dimensional array, that serves as the fundamental building block for computation in deep learning.
snnTorch is an open-source python library which leverages PyTorch’s tensor operations, GPU acceleration, and automatic differentiation for designing, training, and evaluating spiking neural networks (SSN). It extends PyTorch’s capabilities to include:
- Specific tools for simulating, training, and evaluating spiking neurons $\Rightarrow$ SNN Modelling
- Offers four Predefined Spiking Neuron Models = {Leaky Integrate-and-Fire (LIF), Hodgkin-Huxley, Adaptive Integrate-and-Fire}
- Supports event-driven computations $\Rightarrow$ processing temporal and sparse data efficiently, reducing computational cost and energy usage.
- Supports training SNNs using surrogate gradients, a technique that approximates gradients for non-differentiable spiking activation functions.
- At present, neuron models are represented by recursive functions $\rightarrow$ removes the need to store membrane potential traces for all neurons in a system in order to calculate the gradient.
- Allows the simulation of spatiotemporal dynamics for tasks involving time-series or event-based data.
- Compatible $\Rightarrow$ Integration with PyTorch’s ecosystem.
- designed to be intuitively used with PyTorch, as though each spiking neuron were simply another activation in a sequence of layers.
Note:
- lean requirements of snnTorch enable small and large networks to be viably trained on CPU.
- Provided the network models and tensors are loaded onto CUDA, snnTorch takes advantage of GPU acceleration in the same way as PyTorch.
- my programming skills in python are the basics. But I think I can make my way through by reading the documentation and examples, and using AI assists.
[1] ssnTorch [2] pyTorch [3] chatGPT: "What is snnTorch?"
