Choose a hidden-neuron budget N, then vary width / depth. Does a wide shallow network or a narrow deep one fit better?
Final MSE after 5,000 steps per network
Sweep trains 9 fresh networks from scratch — one per aspect ratio, spread from deep/narrow to wide/shallow — using the current N, learning rate, and target. Each runs for 5 000 SGD steps, then its final MSE is plotted. The dashed line marks the aspect ratio currently set above. The best config is circled. Equal neuron counts do not imply equal parameter counts; rounded widths and depths may differ from the requested budget.