Data to Decisions: The role of AI and analytics

There is a lot of excitement, buzz if you will, about artificial intelligence and the capabilities it can bring to bear.

The Biden administration and Congress recognize the potential and peril of AI. There are at least 256 bills in Congress that mention AI in some way shape or form.

President Joe Biden signed an executive order around AI at the end of October and the Office of Management and Budget followed with its draft memo a day later.

All of this excitement around AI is great but success of any initiative that plans to use these technologies depends on one thing: Data.

Data is the foundation for AI tools and capabilities. Agencies need solid data architectures, employees need to have data literacy and there are data privacy and security challenges.

Agencies must begin preparing today for a future where AI tools are a part of their daily workflow as part of how they are improving mission outcomes.

Melanie Krause, the chief data and analytics officer at the IRS, said using data to drive decisions is not only a key focus of Commissioner Danny Werfel, but quickly becoming core to how the tax agency works.

“To get that full 360 degree view of what is happening with the taxpayer, we’ve been able to build out capabilities that use, for example, large language models to mine the text of call transcripts, and then join that with information about a taxpayers account to better understand where people are getting stuck in our process. Those capabilities, then help us to identify those taxpayers that are experiencing the most burden,” Krause said on the discussion Data to Decisions: The role of AI and analytics. “We’ve brought together groups of stakeholders across the business to look at what the data tells us. That’s one thing the commissioner often asks is ‘what does the data say? What is our evidence base for where we’re focusing?’ We’ve been able to use those types of analytic capabilities to deliver new capabilities to stakeholders every two to three weeks.”

The IRS has a list of the top 10 areas that need to be fixed before the next filing driven by the data and analytics.

Extractive vs. generative AI

Krause said by understanding the root causes of what is creating taxpayer burdens and it also gives the IRS an effective approach to reducing it.

Like the IRS, the Securities and Exchange Commission is using data in their AI tools that’s more extractive than generative.

Austin Gerig, the chief data officer at the SEC, said the agency has provided data internally and externally to help drive decisions for some time, specifically through the EDGAR database.

“The main use cases and the main ways in which we are using data in the SEC to drive decision making is whenever we do a rule, we do a cost-benefit analysis. What are the costs? What are the benefits to the public, to our registrants and to others? The SEC especially spends a lot of time and we have a large group of economists who do that,” Gerig said. “Next, to the extent that we promulgate rules, we then need to examine them to make sure that those rules are being followed. We use data to help us decide who to go and examine. We can’t and we don’t have a large enough staff to examine everyone. You can use data to target examinations, either across groups or individually. Then finally, if we find evidence of wrongdoing, or in the process of trying to find evidence that our rules are being broken, we can use data to do that.”

Almost all data that comes to the SEC is used by employees whether for investigations or by data scientists, economist or attorneys.

“One example is with the consolidated audit trail. Pretty quickly, we were able to bring forward the largest front running case that the SEC has ever brought. It’s to the tune of $47 million that was discovered using the consolidated audit trail data,” he said. “That’s a specific example of where we’re using a new dataset to look for evidence of wrongdoing, and we put together a team to do that, and they were successful.”

Workforce training remains key

A continued challenge for public and private sector organizations is having the teams to use the data and the AI tools. To meet that need, companies and agencies are focused on upskilling and reskilling the workforce.

Jessica Palatka, chief human capital officer for the Commerce Department, said one of her goals is to make sure the agency’s workforce’s skillsets support the data governance as much as the data analysis the agency needs to be successful in using AI and technology tools.

“We’re looking for the creation of a data infrastructure that governs and manages data as a strategic asset that we’re utilizing, enabling more efficient data access and analysis, promoting appropriate data use and equitable access to that data,” Palatka said. “Where my particular interest comes into play, cultivating a modern data skills workforce and coordinating a collaborative data culture. Those last two are of particular importance to me being the chief human capital officer. I’m looking to focus on ensuring that our data strategy enables the identification, the execution of strategies, to close the workforce skill gaps that we have in improving the workforce, data literacy and data skills amongst our staff that is central to the advancement of this data analysis and management.  I really look to employ this across the entire lifecycle of human capital management, recruitment, retention, talent management, it touches every single thing that we do.”

One big challenge in the human capital management area is the quality and standardization of the data.

Palatka said figuring out both from a Commerce perspective and from a governmentwide perspective if she has the correct data elements and whether Commerce’s data is complete is a huge struggle.

Ensuring confidence in AI models

“A lot of the times we jumped to that ‘Oh, let’s use AI, where can we use AI’? I’m going back to this data governance board going, ‘Hey, you guys didn’t resource me for a data warehouse? What am I going to put the AI on top of this piece of paper I’m passing around?’” she said. “We absolutely are at a point of extractive analysis. But the to get to a point of having enough data to use extractive artificial intelligence, even the data is so different coming out of each one of those sets that trying to get to that point is can be a challenge. Now, obviously, we do what we can and we have pockets where we absolutely get that and a lot of the times we get it externally.”

Julie Chapman, head of legal for North America at LexisNexis, said there are things agencies can do to improve their confidence and use of data in AI models and tools.

She said walling off data in the large language models, for example, can ensure your organization’s security and privacy requirements.

“You have to make sure the content that is that the largest language model was talking to is reliable content. You don’t want to be have using the ChatGPT and government space, for example. Then, you have to keep humans in the process,” Chapman said. “We’re not trying to get rid of the humans. We want to make sure that we’re looking at and rating the responses you receive from a generative AI tool.”

She added the key for many organizations is to start by cleaning up their data to analytics and AI can be successful over time. By eliminating redundant information, Chapman said agencies can make sure their data is current to better ensure it is reliable and accurate.

Learning objectives:

• Data strategies across government agencies
• Data-driven decisions
• Using data to improve future mission outcomes