Browsing by Author "Balakrishnan, Guha"

Now showing 1 - 6 of 6
  • Thumbnail Image
    Item
    Artificial intelligence uncertainty quantification in radiotherapy applications − A scoping review
    (Elsevier, 2024) Wahid, Kareem A.; Kaffey, Zaphanlene Y.; Farris, David P.; Humbert-Vidan, Laia; Moreno, Amy C.; Rasmussen, Mathis; Ren, Jintao; Naser, Mohamed A.; Netherton, Tucker J.; Korreman, Stine; Balakrishnan, Guha; Fuller, Clifton D.; Fuentes, David; Dohopolski, Michael J.
    Background/purpose The use of artificial intelligence (AI) in radiotherapy (RT) is expanding rapidly. However, there exists a notable lack of clinician trust in AI models, underscoring the need for effective uncertainty quantification (UQ) methods. The purpose of this study was to scope existing literature related to UQ in RT, identify areas of improvement, and determine future directions. Methods We followed the PRISMA-ScR scoping review reporting guidelines. We utilized the population (human cancer patients), concept (utilization of AI UQ), context (radiotherapy applications) framework to structure our search and screening process. We conducted a systematic search spanning seven databases, supplemented by manual curation, up to January 2024. Our search yielded a total of 8980 articles for initial review. Manuscript screening and data extraction was performed in Covidence. Data extraction categories included general study characteristics, RT characteristics, AI characteristics, and UQ characteristics. Results We identified 56 articles published from 2015 to 2024. 10 domains of RT applications were represented; most studies evaluated auto-contouring (50 %), followed by image-synthesis (13 %), and multiple applications simultaneously (11 %). 12 disease sites were represented, with head and neck cancer being the most common disease site independent of application space (32 %). Imaging data was used in 91 % of studies, while only 13 % incorporated RT dose information. Most studies focused on failure detection as the main application of UQ (60 %), with Monte Carlo dropout being the most commonly implemented UQ method (32 %) followed by ensembling (16 %). 55 % of studies did not share code or datasets. Conclusion Our review revealed a lack of diversity in UQ for RT applications beyond auto-contouring. Moreover, we identified a clear need to study additional UQ methods, such as conformal prediction. Our results may incentivize the development of guidelines for reporting and implementation of UQ in RT.
  • Thumbnail Image
    Item
    ISLAND: Informing Brightness and Surface Temperature Through a Land Cover-based Interpolator
    (2023-12-01) Liu, Yuhao; Veeraraghavan, Ashok; Balakrishnan, Guha
    Cloud occlusion is a common problem in the field of remote sensing, particularly for thermal infrared imaging. Clouds degrade thermal signals emitted from the Earth’s surface and interfere with the retrieved Land Surface Temperature (LST). Such cloud contamination severely reduces the set of serviceable thermal images for downstream applications. We introduce a novel method to remove cloud occlusions from Landsat 8 LST images. We call our method ISLAND, an acronym for Informing Brightness and Surface Temperature Through a Land Cover-based Interpolator. ISLAND predicts occluded LST through a set of spatio-temporal filters that perform distance-weighted spatio-temporal interpolation. A critical feature of ISLAND is that the filters are land cover-class aware, making it particularly advantageous in complex urban settings with heterogeneous land cover types and distributions. In this thesis, (1) We show ISLAND achieves robust reconstruction performance with an accuracy of around 1 Kelvin. (2) We provide a public dataset of 20 U.S. cities with pre-computed ISLAND thermal infrared and LST outputs. (3) Using several case studies, we demonstrate that ISLAND opens the door to a multitude of high-impact urban and environmental applications across the continental United States.
  • Thumbnail Image
    Item
    NeuroView: Explainable Deep Network Decision Making
    (2022-07-06) Barberan, CJ; Baraniuk, Richard G; Balakrishnan, Guha
    Deep neural networks (DNs) provide superhuman performance in numerous computer vision tasks, yet it remains unclear exactly which of a DN's units contribute to a particular decision. A deep network’s prediction cannot be explained in a formal mathematical manner such that you know how all the parameters contribute to the decision. NeuroView is a new family of DN architectures that are explainable by design. Each member of the family is derived from a standard DN architecture by concatenating all of the activations and feeding them into a global linear classifier. The resulting architecture establishes a direct, causal link between the state of each unit and the classification decision. We validate NeuroView on multiple datasets and classification tasks to show that it has on par performance to a typical DN. Also, we inspect how it’s unit/class mapping aids in understanding the decision-making process. In this thesis, we propose using NeuroView in other architectures such as convolutional and recurrent neural networks to show how it can aid in providing additional understanding in applications that need more explanation.
  • Thumbnail Image
    Item
    Recent Advances in Bayesian Copula Models for Mixed Data and Quantile Regression
    (2023-04-13) Feldman, Joseph; Kowal, Daniel; Balakrishnan, Guha
    This thesis advances novel Bayesian approaches towards joint modeling of mixed data types and quantile regression. In the first part of this work, we advance methodological and theoretical properties of the Bayesian Gaussian copula, and deploy these models in a variety of applications. Copula models link arbitrary univariate marginal distributions under a multivariate dependence structure to define a valid joint distribution for a random vector. By estimating the joint distribution of a multivariate random vector, we are granted access to a myriad of information, from marginal properties and conditional relationships, to multivariate dependence structures. The final portion of this thesis introduces a novel technique for quantile regression that is broadly compatible with any Bayesian predictive model, including copulas. We utilize posterior summarization to estimate coherent and interpretable quantile functions with the added benefit of quantile-specific variable selection. In the first chapter, we deploy the Gaussian copula towards the generation of privacy-preserving fully synthetic data. Often, the dissemination of data sets containing information on real individuals poses harmful privacy risks. However, the lack of rich, publicly available data hinders policy and decision making, as well as statistics education. Synthetic data are a promising alternative for data sharing: they are simulated from a model estimated on the confidential data, which destroys any one-to-one correspondences between synthetic and real individuals. If the synthetic data are shown to be sufficiently useful and private, they may be disseminated and studied with minimal adverse privacy implications. In this chapter, we synthesize a data set comprised of dozens of sensitive health and academic achievement measurements on nearly 20,000 children from North Carolina which precludes its public release. In addition, the data set is comprised mixed continuous, count, ordinal and nominal data types which poses substantial modeling challenges. We develop a novel Bayesian Gaussian copula model for synthesis of the North Carolina data based on the Extended Rank-Probit Likelihood (RPL), which modifies existing copula models to additionally handle nominal variables. We demonstrate state-of-the-art utility of synthetic data synthesized under the RPL copula model, and study the post-hoc privacy implications of synthetic data releases. In the second chapter, we apply copula models towards imputation of missing values, which are commonplace in modern data analysis. With abundant missing values, it is problematic to conduct a complete case analysis, which proceeds using only observations for which all variables are observed. Thus, imputation is necessary, but limited by the ability of the model to jointly predict missing values of mixed data types. Recognizing the broad compatibility of RPL copula models with mixed data types, we develop a novel Bayesian mixture copula for flexible imputation. Most uniquely, we introduce a technique for marginal distribution estimation, the margin adjustment, which enables automated and consistent estimation of marginal distribution functions in the presence missing data. Our Bayesian mixture copula demonstrates exceptional performance in simulation, and we apply the model on a subset of variables from the National Health and Nutrition Examination Survey subject to abundant missing data. Our results demonstrate the risks of a complete case analysis, and how a suitable model for imputation can correct these shortcomings. We conclude with new perspectives on Bayesian quantile regression, which provides a more robust view into how covariates affect the distribution of a response variable. Given any Bayesian predictive model, we view the quantile function as a posterior functional, which enables point estimation through decision theory. Our technique unifies estimation of quantile-specific functions under a singular, coherent model, which alleviates issues of quantile crossing. Furthermore, through careful justification of the loss function in our posited decision analysis, we develop quantile-specific variable selection techniques. Thus, this work connects the extensive literature on valid quantile function estimation (i.e. techniques to prevent quantile crossing) with variable selection in the mean regression setting. Extensive simulation highlights the vast improvements of the proposed approach over existing Bayesian and frequentist methods in terms of prediction, inference, and variable selection.
  • Thumbnail Image
    Item
    To the Fairness Frontier and Beyond: Identifying, Quantifying, and Optimizing the Fairness-Accuracy Pareto Frontier
    (2023-04-21) Little, Camille; Allen, Genevera; Balakrishnan, Guha; Sabharwal, Ashutosh
    Large-scale machine learning systems are being deployed to aid in making critical decisions in various areas of our society, including criminal justice, finance, healthcare, and education. In many cases, however, systems trained on biased data reflect or exacerbate these biases leading to unfair algorithms that disadvantage protected classes based on gender, race, sexual orientation, age, or nationality. Unfortunately, improved fairness often comes at the expense of model accuracy. Existing works addressing the fairness-accuracy tradeoff report fairness and accuracy separately at a single hyperparameter, making it impossible to compare performance between models and model families across the entire frontier. Taking inspiration from the AUC-ROC literature, we develop a method for identifying (TAF) and measuring (Fairness-AUC) the Pareto fairness-accuracy frontier. Further, we ask: Is it possible to expand the empirical Pareto frontier and thus improve the Fairness-AUC for a given collection of fitted models? We answer affirmatively by developing a novel fair model stacking framework, FairStacks, that solves a convex program to maximize the accuracy of the model ensemble subject to a relaxed bias constraint. We show that optimizing with FairStacks always expands the empirical Pareto frontier and improves the Fairness-AUC; we additionally study other theoretical properties of our proposed approach. Finally, we empirically validate TAF, Fairness-AUC, and FairStacks through studies on several real benchmark data sets, showing that FairStacks leads to major improvements in Fairness-AUC that outperform existing algorithmic fairness approaches.
  • Thumbnail Image
    Item
    Vision and Language: Information Integration and Transformation
    (2023-11-30) Yang, Ziyan; Ordonez-Roman, Vicente; Balakrishnan, Guha
    Images and text are large-scale data sources for deep learning models to mimic how humans perceive and understand multimodal information. Combining images and text can construct better feature representations by involving complementary information from different sources. For example, text descriptions may ignore the commonsense information such as ”apple is red”, but such information can be effectively learned from images with red apples. Learning independently from a single modality is not enough for complicated tasks that require an understanding of interactions and connections between multi-modalities, such as image-text retrieval, visual question answering, visual grounding, and image captioning. In particular, we explore and develop techniques combining image and text information to improve transformation between modalities. First, we exploit the multilingual image captioning and the multimodal machine translation tasks, using additional vision information to transfer information from images and one language to another language or just between two languages. Second, we focus on the visual grounding task, which aims to ground corresponding image regions from query phrases. Finally, we introduce a novel relation detection task and our solution for it. This thesis consists of three parts: In the first part, we propose a pipeline to combine image and text information from a source language to generate text for a target language during inference time. We design the feedback propagation process for image captioning models during inference time to show that information can be transferred between images and different languages to achieve benefits in text generation without an additional training process. By providing additional text information from one language, we show that this technique can construct better multimodal representations to generate text for another language. In the second part, we demonstrate that directly improving the gradient-based explanations for vision-language models produces superior visual grounding results. Visual grounding is a well-defined task that requires the model to figure out image regions corresponding to given text phrases. Most previous works study this problem by extracting image region features and measuring the similarities between image and text features. We observe that the visual explanations for text phrases can be used to solve the visual grounding task directly. Then, we propose a margin-based loss for tuning joint vision-language models so that their gradient-based visual explanations are consistent with region-level annotations provided by humans. In the last part, we introduce a new setting for relationship prediction, called Subject-Conditional Relation Detection (SCoRD), which consists of enumerating relations, objects, and boxes conditioned on an input subject in an image. We design a model that efficiently mixes up images and text inputs to solve this new task. Specifically, we explore a generation-based method to ground related objects by providing a subject and its location as the text inputs.