Sea surface height (SSH) is a key geophysical parameter for monitoring and studying mesoscale surface ocean dynamics. For several decades, the mapping of SSH products at regional and global scales has relied on nadir satellite altimeters, which provide 1-D-only along-track satellite observations of the SSH. The surface water and ocean topography (SWOT) mission deploys a new sensor that acquires for the first time wide-swath 2-D observations of the SSH. This provides new means to observe the ocean at previously unresolved spatial scales. A critical challenge for the exploitation of SWOT data is the separation of the SSH from other signals present in the observations. In this article, we propose a novel learning-based approach for this SWOT calibration problem. It benefits from calibrated nadir altimetry products and a scale-space decomposition adapted to the structure of the different processes in play in the SWOT’s swath geometry. In a supervised setting, our method reaches the state-of-the-art residual error of ≈ 1.4 cm while proposing a correction on the entire spectrum from 10 to 1000 km and with less restrictive constraints on the modeled error signal.