Recently, the area of decision and control has been interested in studying the connectivity of large-scale networks. As networks under study are large, to have a complete knowledge of the network is impossible, whereas little but representative information is available with an efficient exploration scheme. Machine learning approaches were presented and used to tackle this difficulty to hold it up. In this regard, we present and prove the convergence of an efficient algorithm that converges to the Fielder vector when the topology is initially unknown and the only accessible information is gathered by a random walk process throughout the entire network. The Rayleigh quotient optimization problem and the notion of stochastic approximation are the foundations of our technique. We consider multiple sampling strategies that are categorized under random walks, as well as adapting another sampling approach that are considered random walk, the Gibbs sampling, and it showed better results. Finally, we demonstrate its performance on different network topologies.