In many scenarios, using L1 regularization drives some neural network weights to 0, leading to a sparse network. Using L2 regularization often drives all weights to small values, but few weights ...

Recall that the NeuralNetwork class has a hardcoded tanh hidden node activation function. The derivative variable holds the calculus derivative of the tanh function. So, if you change the hidden node ...