Self contained: It has no dependencies, so it can be easily bundled.
Capable of server-side rendering: it has an option to generate HTML on the server (so, for example, one can pre-render expressions using Node.js and send them as plain HTML).
KaTeX implements a smaller subset of LaTeX's mathematical notation features than MathJax.