INTRODUCTION: In the last decade, a significant number of institutions have adopted open access (OA) policies. Many of those working with OA policies are tasked with measuring policy compliance. This article reports on a survey of Coalition of Open Access Policy Institutions (COAPI) members designed to better understand the methods currently used for measuring and communicating OA policy success. METHODS: This electronic survey was distributed to the COAPI member listserv, inviting both institutions who have passed an implemented policies and those who are still developing policies to participate. RESULTS: The results to a number of questions related to topics such as policy workflows, quantitative and qualitative measurement activities and related tools, and challenges showed a wide range of responses, which are shared here. DISCUSSION: It is clear that a number of COAPI members struggle with identifying what should be measured and what tools and methods are appropriate. The survey illustrates how each institution measures compliance differently, making it difficult to benchmark against peer institutions. CONCLUSION: As a result of this survey, we recommend that institutions working with OA policies be as transparent as possible about their data sources and methods when calculating deposit rates and other quantitative measures. It is hoped that this transparency will result in the development of a set of qualitative and quantitative best practices for assessing OA policies that standardizes assessment terminology and articulates why institutions may want to measure policies.