The millimeter scale carrier wavelength of the 60 GHz spectrum makes it feasible to pack two order of magnitudes more antennas into the same form factor compared to legacy bands (i.e. 2.4 and 5 GHz band). Prior works in 60 GHz have exploited this large antenna arrays to enhance the link budget of a single user transmission, which suffers from high path loss in 60 GHz. We are proposing a scalable multi-user scheme in 60 GHz WLANs in order to serve multiple clients with multi-Gbps data rate simultaneously in the same environment using the same frequency channel. To this end, we first propose a scalable beam training protocol, which tracks the users for directional transmissions. Then we have designed and evaluated incremental policies that add clients to a transmission sequentially until the AP's resources are exhausted or client link budgets, including interference, are exceeded. We further target polarization diversity and non-uniform antenna partitioning as mechanisms to dramatically reduce inter-stream interference enabling vastly improved aggregate rate. At lower bands, multi-user aggregation is typically achieved by zero-forcing inter-user interference via sender-side digital pre-coding using channel state information at the source. Unfortunately, such techniques do not scale to 60 GHz since (i) 60 GHz transmission is highly directional and lacks the rich scattering propagation environment assumed for most prior work; (ii) even efficient mechanisms for CSIT collection do not scale to large antenna arrays; (iii) prior techniques employ a large number of radio frequency chains (up to one per antenna) which are not feasible in our scenario. Our experiments through over-the-air testbed built over WARP platform and trace-driven simulations show that our methodology can achieve performance near to that of exhaustive search of all possible client combinations, yet with substantially less overhead.