In this paper, we consider cellular networks powered by both renewable energy and the Smart Grid. We study the problem of minimizing the cost of on-grid energy while maximizing the satisfaction of users with different requirements. We consider patterns of renewable energy generation, traffic variation and real-time price of grid energy. Knowing that these patterns are all time related, we use Q-learning to extract a common pattern as well as to decide the number of radio resource blocks activated to maximize the users' satisfaction and minimize the on-grid energy cost. Results show that using Q-learning achieves a good tradeoff with more than 75% reduction in energy cost and negligible degradation in users' satisfaction.