How to show only options with products, not all options?

If you have such issue in Improved Layered Navigation, please open the attribute in admin > catalog > attributes > manage attributes

and change filterable(no results)  with filterable (with results)