In this paper, we provide self-configuration and adaptation capabilities to Underwater sensor networks (UWSN) thanks to Q-learning. UWSN deployed for the long term over large areas for environmental monitoring are possible applications of our work. Sensor nodes deployed on the sea bottom are devoted to measure a physical quantity of interest transmitted to surface buoys considered as access points. Packet transmission are asynchronous and low overheads are desirable so as to save throughput and battery life. Prior to a transmission, the nodes choose, depending on the channel conditions, which access point maximizes the probability of successful decoding a the receiver side. Results show that Q-learning is able to perform close to an ideal "genie-aided" scheme, without the need of a detailed knowledge on the environment.