Advertising (Ad) revenue is a major revenue source for many technology and e-commerce companies; most of the revenue optimization research has been around third party display ads or Cost-Per-Click based first party ads. This paper discusses the Cost-Per-Action ad product at eBay; and the challenge of balancing ad revenue and relevance. We proposed a new measurement that uses Kullback–Leibler (KL) divergence to both optimize ad revenue and improve buyer experience in item recommendations. KL divergence is adopted in the re-ranking algorithm as a constraint for revenue optimization and it is solved by a greedy grid search algorithm. In addition, we are able to approximate KL divergence with inventory based features, and that simplified a full greedy search operation to a regression. Overall, we designed and A/B tested three different approaches, all of them showed significant improvement over the baseline. Through effective re-ranking, we showed that we can achieve significant revenue gain in a sponsored listing recommendation system, even without making any improvement on conversion estimation. We launched one of the implementations to production that yielded more than 12% revenue lift with minimum impact on user experience.