Here is a paraphrased version of an argument I have often heard from students and some instructors on this topic
You should be allowed to use a calculator during exams because you would always be allowed to use a calculator in the real world or at a real job.
The same argument is sometimes applied to access to things like WolframAlpha or for access to the internet in general during exams. I agree with the premise that you will (essentially) always have access to a calculator in the real world. However, I think this argument is fundamentally flawed and the…
How do neural networks behave when the hidden layers are very large? A new research direction in machine learning studies network behavior in the limit that the size of each hidden layer increases to infinity. Surprisingly, in this limit the behavior of the network dramatically simplifies and we can understand what the network is doing theoretically. Why does this happen and what can we say?
The fundamental reason that wide networks have a simpler behavior is that they are close to their linear approximation:
where the linear approximation to the network is defined by