City making progress in guidelines for generative AI technology

The city has established basic guidelines for employees regarding the use of generative artificial intelligence technology for city business and documents, though it is still determining its policies regarding vendors and other outside entities’ use of the tools.

The efficiencies and broadening of skill sets made possible by AI tools could bring changes to the city’s workforce, according to a recent memo.

Released late last month by chief innovation officer Daniel Culotta, the memo follows a City Council resolution approved in February that sought to create rules and best practices for the fast-evolving technology available on platforms such as ChatGPT and Bard.

In response, the Office of Innovation convened an advisory committee from three city departments – the Communications and Technology Management Department, the Information Security Office and the Equity Office – as well as outside experts and stakeholders. The group’s goals were to encourage collaboration within and outside City Hall; create an evaluation process for new technologies; identify and monitor inherent risks; prioritize transparency, equity and accountability; and find responsible ways to scale technology usage.

Much of the city’s research and guidance on generative AI use is contained in its Generative AI Standards document, which was assembled and approved for use this past fall based largely on guidelines established by San Jose, California.

The standards document covers the many factors that public employees must take into consideration when using generative AI tools. Because any queries or prompts submitted become publicly available for AI training purposes, the standards spell out that any data used automatically becomes a record open to public information requests.

It also specifies that any outputs from AI tools should be carefully fact-checked and reviewed before final approvals. Anytime AI tools are used in the creation of a city document or other deliverable, staff are instructed to cite the usage and where it came from.

The city has also adopted three levels of risk – mid-level risk, high risk and prohibited – covering the consequences of an information breach of data used in AI tools.

Mid-risk uses include internal documents involving nonconfidential information with no personal data identification issues.

High-risk uses include personally identifiable information and nonpublic business information that could impact city processes. No high-risk material is allowed in generative AI systems.

Prohibited-risk uses include highly sensitive identifying information including credit card numbers, bank account information and health or other personal data.

The memo also makes several references to recommendations made in recent months concerning generative AI use but doesn’t specifically list those recommendations that are currently being used by city staff.

Staff is also determining how to address errors and incidents of bias involving AI tools while also “exploring the adoption of procurement review guidelines, vendor guidelines, factsheets, and other tools that will promote transparency, explainability, understanding, and auditability of the structure and function of AI applications the City is evaluating or using.”

The Innovation Office is also leading city efforts related to training and usage of AI technology, with more than 300 employees receiving instruction on responsible AI usage.

Looking ahead, the memo notes, “The Office of Innovation will work with the Deputy Chief Learning Officer to research and identify additional resources to expand AI training and education options. The AI core team is building a strategy plan with the Deputy Chief Learning Officer and the Human Resources Department Director to comprehensively address staff readiness, reskilling, and support as AI tools continue to redefine the workforce.”

The Austin Monitor’s work is made possible by donations from the community. Though our reporting covers donors from time to time, we are careful to keep business and editorial efforts separate while maintaining transparency. A complete list of donors is available here, and our code of ethics is explained here.