The Internet of Things (IoT) is a major driving use case of future communication systems. 5G and beyond networks should be properly designed to meet the quality of service (QoS) requirements of IoT devices in terms of massive access, data rates, latency and reliability. In this paper, we study the uncoordinated spectrum and power allocation problems in an uplink IoT network. In the considered setting, IoT devices aim at organizing their transmissions, without any coordination, on the available frequency channels and power levels. To enhance performance, we propose the deployment of full-duplex relays as well as the use of non-orthogonal multiple access and queuing at the level of the relays. A new algorithm, based on the multiplayer multi-armed bandits framework, is proposed with the aim of reducing the transmit power of IoT devices. Simulation results validate the convergence of the proposed algorithm. Results also confirm the superior performance of the proposed technique in terms of satisfying QoS requirements and reducing the needed user transmit power when compared to two baseline schemes.