[Algorithm][Parametric] Implement the GDN layer

[jejjohnson]

A basic implementation of the GDN algorithm. It is mainly used in compression. It features a normalization which can be coupled with a convolution/linear layer.

y[i] = x[i] / sqrt(beta[i] + sum_j(gamma[j, i] * x[j]))

---

Resources

• GDN PyTorch Implementation
• Paper - Density Modeling of Images using a Generalized Normalization Transformation - Balle et al, (2016)
• Paper - Self Normalizing Flows - Keller et al (2020)

---

Potential Issues

Somehow we need to constrain a few parameters, e.g., gamma and beta. Probably a simple np.max would work.