BLOG

Executive Imperative: Rapid Data Reporting for Federal Agencies Advancing Racial Equity and Support for Underserved Communities

Jaime De La Ree
Nisha Iyer

May 4, 2021

Blog Thought leadership

The Imperative: Assess & Address Barriers to Benefits & Opportunities

President Biden’s executive order (EO 13985) on racial equity instructs the head of each agency to review “policies and programs to assess whether underserved communities and their members face systemic barriers in accessing benefits and opportunities.” Each agency must complete this review “within 200 days [and file] a report with the Assistant to the President for Domestic Policy (APDP)."

The goal when undertaking these kinds of analyses is to ensure relevant and actionable findings; however, in Data Society’s experience with work of this nature, tackling underlying forces that sneak bias into data, programs and analyses will be challenging without expert guidance. Bias can be introduced at many points of data collection, and expert understanding of the data collection process, as well as analysis techniques, is necessary to ensure that insights are presented to decision makers accurately, free from bias or, at minimum, with appropriate context.

The goal of this white paper is to share Data Society’s experience and perspectives to help agency leaders comply with this EO faithfully, objectively, and with a mind towards improving the mission’s effectiveness.

The First Step: Reimagining Success

Given the limited time this directive allows, and the absence of additional resources provided to assist already-busy departments in performing the task, we expect many agencies will be inclined to focus on a few attainable projects and perform a review based only on readily available data. Using this approach, discovered bias will be minimal, if found at all. While this would narrowly satisfy the requirements of the EO, this is not a good-faith outcome.

At the outset of any data-driven project, the best practice would be to define, even reimagine, what success looks like. To the program manager, success can be finding no bias and returning to mission-critical activities. But to the President, department executives, and the public, success looks different. To them, success relies on accurate data collection and analysis that unequivocally demonstrates where bias does exist, and in what form. As a result, the government can implement appropriate and (often simple) corrective action to ensure fairness and equity across the board.

Considering the Chief Data Officer (CDO) of the United States is among the key stakeholders receiving this report, this objective, data-centric approach is the only credible approach.

Human Psychology Predisposes Past, Non-Data-Driven Practices to Bias.

Identifying such biases through a regimented data collection process is critical. For a report to the Office of Management and Budget (OMB) and the CDO to be credible, selection of sample projects should be made by an objective internal entity; for example, by the ombudsman or general counsel, alongside independent data science experts versed in survey methodologies, and other techniques specific to identifying bias. Thereafter, participants should be brought into the process in a way that reimagines success, defines what a good-faith outcome looks like, and communicates that truthful, accurate analysis – no matter the outcome – supports the most effective, actionable recommendations to holistically address bias and advance equity.With respect to the executive order, success lies in looking beyond the apparent symptoms of bias, and into the heart of the data story, to illuminate underlying structural issues. For example:

Historical data and legacy processes can contain stereotypes and representations of convention rather than be reflective of reality (e.g., predicting criminal recidivism based on historical demographics of incarcerated populations assumes that everyone is justly incarcerated and therefore policing reforms are not necessary).

The existence of prior policies or practices, perhaps in generations past and long overwritten by new ones, that still act as real barriers to entry and opportunity (e.g., urban housing and development trends reveal the scars of historical redlining 50 years later).

Terms, approaches, and tools that may work perfectly in one cultural context may communicate disincentives to others (e.g., assessing vaccination efforts across demographics without cultural, local or historical context will bias resulting conclusions from that data).

Bias can be Measured and Addressed Quickly, Dispassionately, and Accurately — with the Right Support.

Agencies often focus on collecting and assessing econometric and demographic data that may inherently be prone to, or may even exacerbate existing biases. At Data Society, we train and serve both government agencies and commercial enterprises in critical data-savvy solutions. Based on experience, we recommend the following be assessed to deliver actionable findings:

Participants and Beneficiaries: The difference between participants and beneficiaries can be seen in educational programs, for example: parents, teachers, administrators, and students are all participants in the programs; only the students are intended beneficiaries. An impactful assessment, in this case, would evaluate inclusivity in the selection of participants and communities in addition to whether sufficient beneficiary goals were met.

Inputs and outputs: Building on the preceding analysis of how people were touched by a selected program, we must then assess the allocation, consumption, and creation of non-human assets and factors. Did the program leverage or create funds? Labor? Physical infrastructure? Was the output increased citizen health? Improved workforce safety? Something else? A successful review will seek to recognize and evaluate how effectively inputs and outputs are structured and whether they were appropriately consumed, distributed, or allocated in the intended beneficiaries’ interest.

Processes: To root out discovered issues, a review of program execution processes should address whether efficiencies (or inefficiencies) exist disproportionality in some geographies, affect participants or beneficiaries disparately, or otherwise alter how opportunities are perceived.

Data: Most federal programs record data on how programs were delivered. This comprehensive historical record describes not only the factors above, but also outcomes, quality, efficiency, and impact. Agencies can readily advance their investigations of bias by layering commonly maintained data in support of the mission and its beneficiaries – effectively using that data to create a point of reference against which outcomes of the programs can be referenced (e.g. if 50% of people who took a vaccine were between the ages of 20 – 30, this may be a poor result among this age category if 90% of a given population fall into that group). The low-hanging opportunities emerging from such analysis allow for targeted near-term impact to begin noticeably addressing bias in the eyes of beneficiaries.

Reporting Should be Clear, Objective, and Action-oriented.

Beyond findings and facts, the required report to the White House should illuminate priorities for possible corrective actions to enhance mission effectiveness, public confidence, and budgetary efficiency—while embracing the EO’s spirit—to embark on corrective change for the greater population. With the right leadership and partnership, such reporting secures a foundation for lasting equity, cohesively ties figures to impact (or lack thereof) and spells out priorities for action.

Partner with Data Society.

Data Society is dedicated to using data science to advance and accelerate society’s progress towards key challenges and opportunities. Through our systems and solutions businesses, our community of data scientists, engineers, and instructional designers have addressed, for example:

Food scarcity emergency management through our work with the World Central Kitchen

Oversight of global financial funds at scale, monitoring real-time data flows to mitigate risk and better address development needs, for the Inter-American Development Bank

Global sustainability efforts, identifying greater efficiencies, reduced cost and waste, and reduced energy consumption for various large-scale manufacturers and federal agencies.

Data Society is unique among state and federal contractors in combining access to the world’s strongest data scientists and practitioners with training capabilities unparalleled in the industry, especially as a Washington D.C.-based Woman-Owned Small Business. This is why, and how, we continue to work with our clients to achieve extraordinary things.

Subscribe to our newsletter

[piotnetforms id=52]