Robot motion planning has been a key component in the race to achieve true robot autonomy. It encompasses methods to generate robot motion that meets kinematic constraints, robot dynamics and that is safe (avoids colliding with the environment). It has been particularly successful in efficiently finding motions for high degree-of-freedom robots such as manipulators, but despite tremendous advances, motion planning methods are not ready for human environments. The uncertainty, diversity and clutter of the human world challenge the assumptions of motion planning methods breaking their guarantees, rendering them useless or dramatically worsening their performance. In this thesis, we propose methods to address three important challenges in augmenting motion planning and long-horizon manipulation for human environments. First, we present a framework that enables human-guided motion planning and demonstrate how it can be used for safe planning in partially-observed environments. Second, we present two methods for safe motion planning in the presence of sensing uncertainty, one that requires the poses of segmented objects and another one that acts directly on distance information from a noisy sensor. Finally, we present a framework that dramatically improves the performance of long-horizon manipulation tasks in the presence of clutter for an important class of manipulation problems. All of our contributions have mathematical optimization as a connecting thread to synthesize high-dimensional trajectories using low-dimensional information or as a layer between high-level and low-level planners. Our results demonstrate how these formulations can be effectively used to augment motion planning and planning for manipulation in novel ways, attaining more robust, efficient and reliable methods.