Deep neural networks (DNNs) with the flexibility to learn good top-layer representations have eclipsed shallow kernel methods without that flexibility. Here, we take inspiration from DNNs to develop the deep kernel machine. Optimizing the deep kernel machine objective is equivalent to exact inference in an infinitely wide Bayesian neural network or deep Gaussian process, which has been scaled carefully to retain representation learning. We conjecture that the deep kernel machine objective is unimodal, and give a proof of unimodality for linear kernels. We describe a fast optimizer that uses the Continuous Algebraic Ricatti Equations from control theory to converge in ~10 steps, and we show superior predictive performance over alternative approaches to learning the kernel.