Heaviside function

New HTAF activation function enables stable training of binary neural networks

Researchers have developed a new activation function called Heavy Tailed Activation Function (HTAF) to address the challenges of training neural networks with binary representations. HTAF is a smooth approximation of th…