Optical whispering gallery modes (WGMs) derive their name from a famous acoustic phenomenon of guiding a wave by a curved boundary observed nearly a century ago. This phenomenon has a rather general nature, equally applicable to sound and all other waves. It enables resonators of unique properties attractive both in science and engineering. Very high quality factors of optical WGM resonators persisting in a wide wavelength range spanning from radio frequencies to ultraviolet light, their small mode volume, and tunable in- and out- coupling make them exceptionally efficient for nonlinear optical applications. Nonlinear optics facilitates interaction of photons with each other and with other physical systems, and is of prime importance in quantum optics. In this paper we review numerous applications of WGM resonators in nonlinear and quantum optics. We outline the current areas of interest, summarize progress, highlight difficulties, and discuss possible future development trends in these areas.