Distributed control protocols routinely have to operate oblivious of dynamic network information for scalability or complexity reasons. However, more informed protocols are likely to make more intelligent decisions. We argue that protocols can leverage dynamic congestion information without suffering the mentioned penalties. In this paper we show that routers can readily exchange congestion information in a purely passive fashion using congestion markings from existing traffic. As a result, each router can locally infer a congestion map of the network. Moreover, the maps are continuously updated with near real-time information. Our solution for building the congestion maps leverages standardized and widely used congestion management protocols and does not require changes to end hosts. We find that 90% of the time, the inference accuracy is usually within 10% even for environments with multiple congestion points and sudden changes in the traffic pattern.