Neural Network Fundamentals

- Feedforward networks as transformer core - Linear layers for learned projections - Nonlinear activations enable expressiveness - SwiGLU powering modern FFN blocks - MLPs refine token representations - LayerNorm stabilizes deep training - Dropout prevents co-adaptation overfitting - Skip connections preserve information flow - Positional encoding injects word order - NLL loss guides probability learning - Encoder vs decoder architectures explained - FFNN + attention form transformer blocks