(made with Claude Code and ChatGPT)
MIT 6.7960 — Approximation Theory

Sinusoidal Positional Encoding

Both networks have H hidden units and are trained on the same fixed scatter of random points. Left: raw x input. Right: x mapped to [sin(π·2kx), cos(π·2kx)] for k = 0…L−1.

16
4
0.003
iter 0 plain MSE — pos-enc MSE —
data points
plain MLP fit
pos-enc MLP fit
Plain MLP — input: x  (1-dim)
Pos-enc MLP — input dim: 2L