The browser will attempt to display cover images if they're available. By default it looks for images cover_small.jpg, cover_med.jpg, etc. Customize emms-browser-covers to use your own covers. Note that you'll probably want to resize your existing covers to particular sizes. Suggested sizes are 100x100 for small, and 200x200 for medium.
Also, Emacs by default will jump around a lot when scrolling a buffer with images. In order to prevent that, you can set scroll-up-aggressively and scroll-down-aggressively to the number “0.0”.
To show a 'no cover' image for albums which don't have a cover, add the following code to your .emacs:
(setq emms-browser-default-covers (list "/path/to/cover_small.jpg" nil nil)
The medium and large images can be set as well.
You can download an example `no cover' image.