In the E-commerce platform, the sponsored search engine not only takes the role of revenue contributor, but also contributes to the long-term growth of the platform by improving user experiences and facilitating advertisers’ commercial goals. The key to the satisfactory of the platform, the user and the advertiser is the decision of the list of advertisements to show and the charging prices for the advertisers. In the sponsored search platform, these decisions are made according to a ranking function. In the E-commerce platform, advertisements showing positions under different queries from different users may be associated with advertisement candidates of different bid price distributions and click probability distributions, which requires the ranking functions to be optimized adaptively to the traffic characteristics. In this work, we proposed a generic framework to optimize the ranking functions by deep reinforcement learning methods. Experimental results on a large-scale sponsored search platform (Alibaba sponsored search engine) confirm the effectiveness of the proposed method.