Short, precise definitions.

The Laplace-domain ratio $G(s)=Y(s)/U(s)$ of a system's output to its input, assuming zero initial conditions. The poles (roots of the denominator) determine stability; the zeros shape the transient response. Nearly every classical-control tool — Bode plots, root locus, Nyquist criteria — operates on $G(s)$.

Measured at the frequency where $|G(j\omega )|=1$ (the gain crossover), the phase margin is ${\phi }_m=180^{\circ }+\angle G(j{\omega }_c)$. A rule of thumb: aim for 45°–60°. Below 30° the system rings; negative means it's already unstable.

Measured at the frequency where the open-loop phase is $-180^{\circ }$, it's the reciprocal of $|G(j{\omega }_{180})|$ in dB. Paired with phase margin, it gives a quick read on how close a controller is to the edge.

The standard classical-control chart. Reads phase margin, gain margin, bandwidth, and roll-off rate at a glance. Pairs naturally with PID tuning and loop shaping.

For a closed-loop system to be stable, the Nyquist plot of $G(s)$ must encircle the $-1$ point a number of times equal to the number of right-half-plane open-loop poles. Stronger than Bode margins because it works for unstable open-loop systems.

The workhorse controller: $u(t)=K_{p}e(t)+K_i{\int }_0^{t}e(\tau )d\tau +K_d\dot{e}(t)$. Proportional for response, integral to kill steady-state error, derivative for damping. Still runs more of the world's control loops than anything else.

A system described by its internal state $x$ rather than input/output relations. Makes multi-input multi-output systems tractable and is the foundation for LQR, Kalman filtering, and MPC.

Minimizes $J={\int }_0^{\infty }(x^{\top }Qx+u^{\top }Ru)dt$ for $\dot{x}=Ax+Bu$. The solution is a linear state feedback $u=-Kx$, with $K$ computed from the algebraic Riccati equation. Guaranteed infinite gain margin and $\ge 60^{\circ }$ phase margin — a free lunch as far as classical-control robustness goes.

The core idea behind model predictive control. At each timestep you optimize a finite-horizon trajectory, execute only the first control, and resolve the full problem at the next step with updated state. Trades a harder per-step computation for the ability to handle constraints and disturbances online.

Formulated as: ${\min }_{u_{0:N-1}}{\sum }_{k=0}^{N-1}\ell (x_k,u_k)+V_f(x_N)$ subject to $x_{k+1}=f(x_k,u_k)$ and constraints on $x,u$. Solved with quadratic programming (linear MPC) or nonlinear solvers. Dominates in HVAC, process control, autonomous vehicles, and increasingly legged robotics.

A function $V(x)>0$ (except at equilibrium) whose time derivative along trajectories is $\dot{V}(x)\le 0$. If such a function exists, the system is stable; if $\dot{V}<0$, it's asymptotically stable. Finding a valid Lyapunov function is the hard part — increasingly, SOS programming and neural networks are used to search for one.

Given noisy measurements and a linear model, maintains a belief $\mathcal{N}(\hat{x},P)$ over the state that is provably optimal in the MMSE sense. Two steps: predict ($\hat x^-, P^-$) and update (fold in the new measurement via the Kalman gain $K$). Extensions (EKF, UKF, particle filter) handle nonlinearity.

A tuple $(\mathcal{S},\mathcal{A},P,R,\gamma )$: states, actions, transition kernel $P(s^{\prime }\mid s,a)$, reward function $R(s,a)$, and discount factor $\gamma \in [0,1)$. Everything in RL — policies, value functions, Bellman equations — is defined with respect to an MDP.

For a policy $\pi$: $V^{\pi }(s)={\mathbb{E}}_{a\sim \pi }\left[R(s,a)+\gamma {\mathbb{E}}_{s^{\prime }}[V^{\pi }(s^{\prime })]\right]$. The optimal form swaps the expectation over $\pi$ for a max over actions. Every RL algorithm is some way of exploiting this recursion.

${\nabla }_{\theta }J(\theta )={\mathbb{E}}_{\tau \sim {\pi }_{\theta }}\left[{\sum }_t{\nabla }_{\theta }\log {\pi }_{\theta }(a_t\mid s_t)A(s_t,a_t)\right]$. Lets you optimize a stochastic policy by following the gradient of expected reward. Variance reduction (baselines, GAE, trust regions) is what makes it work in practice.

$A^{\pi }(s,a)=Q^{\pi }(s,a)-V^{\pi }(s)$. Subtracting the state value $V$ from the action value $Q$ removes state-dependent variance from policy-gradient estimators without adding bias — the reason every modern policy-gradient method (A2C, PPO, GAE) uses advantage rather than raw returns.

Updates an estimate of the optimal action-value function: $Q(s,a)\leftarrow Q(s,a)+\alpha [r+\gamma {\max }_{a^{\prime }}Q(s^{\prime },a^{\prime })-Q(s,a)]$. With tabular states and enough exploration, it provably converges. The deep variant (DQN) replaces the table with a neural net.

The $\gamma \in [0,1)$ in the discounted return ${\sum }_t{\gamma }^t{r}_t$. Low $\gamma$ makes the agent myopic; high $\gamma$ makes credit assignment hard over long horizons. Often treated as a hyperparameter even when the true problem has a natural horizon.

The critic estimates $V$ or $Q$; the actor uses those estimates as a baseline or target for its own updates. Modern deep-RL algorithms (A2C, PPO, SAC, TD3) are all actor-critic variants — they get the low-variance gradients of policy gradient plus the bootstrapping of value methods.

The dominant recipe for robot learning. Key tricks: domain randomization (randomize mass, friction, actuator lag, sensor noise), privileged training (give the expert extra state, then distill to a student that only sees what the real robot sees), and system identification to narrow the gap. Fails loudest when the sim omits something the real world cares about.

Train across a distribution of physics parameters (friction, mass, motor gains) and visual properties (lighting, textures) wide enough that the real world is effectively another draw from the same distribution. Cheap to implement, surprisingly effective, and the reason so many sim-to-real stories work on legged robots.

Simplest form: behavior cloning, treat it as supervised learning on $(s,a)$ pairs. Classic failure mode: compounding errors, where small deviations take the agent off the demo distribution and everything falls apart (DAgger and its successors address this). Modern variants use diffusion, transformers, or VLAs to model the demonstrator distribution.

Descendants of vision-language models, finetuned to output action tokens. Examples: RT-2, OpenVLA, ${\pi }_0$. Promise: a single policy that can follow arbitrary language commands across many embodiments. Reality as of 2026: real progress on benchmarks, still brittle outside training distributions, and expensive to run at control rates.

Predicts a short horizon of future actions by denoising random noise conditioned on recent observations. Handles multimodal demonstrations naturally — a problem where regression-based behavior cloning tends to average across modes and fail.

${\hat{s}}_{t+1}=f_{\theta }(s_t,a_t)$ trained to reconstruct observed transitions. Planning inside a world model (Dreamer-style) can be far more sample-efficient than model-free RL, at the cost of errors compounding over long rollouts.

$J_{ij}=\partial f_i/\partial x_j$. In robotics, the manipulator Jacobian maps joint velocities to end-effector velocities. In control, the Jacobian linearizes a nonlinear system around an operating point.

For $\dot{x}=f(x,u)$ at equilibrium $(x^{\ast },u^{\ast })$, the linearization is $\dot{\delta x}\approx A\delta x+B\delta u$ with $A=\partial f/\partial x$ and $B=\partial f/\partial u$. Lets you apply every tool from linear control — local to the operating point only.

For a linear system $\dot{x}=Ax+Bu$, check whether the controllability matrix $[B,AB,A^{2}B,\dots ,A^{n-1}B]$ has full row rank. A practical warning flag: if controllability only holds near the edge of numerical rank, your real system probably has modes you can barely influence.

The dual of controllability: the observability matrix $[C;CA;C{A}^2;\dots ;C{A}^{n-1}]$ must have full column rank. Failure means some state directions are invisible to your sensors — a state estimator can't recover them no matter how clever the filter.