Single particle tracking (SPT) methods assume predetermined motion models that affect tracking analysis. I present an unbiased SPT algorithm, Knowing Nothing Outside Tracking (KNOT), that utilizes the point clouds provided by iterative deconvolution to educate particle localization and trajectory linking between frames for 2-D and 3-D tracking. Retaining the information between point clouds fuels a machine learning paradigm that predicts the next particle position using an adaptive per-particle motion model, avoiding the motion biases present in other SPT methods. KNOT competes with or surpasses other methods presented in the 2012 International Symposium on Biomedical Imaging (ISBI) particle tracking challenge. Further, we apply KNOT to lysozyme adsorption on polymer surfaces and early endosome transport in live cells, distinguishing motion types and preferred directions of motion. KNOT can differentiate motion between transported and diffused vesicles in the cellular environment crucial to understanding transitions in cellular structure and chemistry that precedes metastasis.