Artificial intelligence (AI) is a useful and powerful tool but users must also appreciate a number of challenges associated with adopting new programs.

The MACo Summer Conference session, “Accelerating Economic Development with AI and Tech Tools,” hosted a packed room of attendees. As previously covered on Conduit Street, governments have already begun exploring opportunities for incorporating programs driven by AI. Speakers on the panel discussed why it is imperative for local governments to be accessing new technology to bolster more efficient processes but also what to be cautious of in that process.

Much of the discussion focused on the positive applications of AI in government, like customer service functions, primarily. Law enforcement agencies are also beginning to explore ways the technology can help in crime control and prevention. A third application is with things like generative AI programs that can help governments produce marketing and informational materials, videos, designs, and the like.

And while the presentation focused mostly on positive applications, panelists warned of some pitfalls to be aware of. One example was using large-language models for research. The programs can produce something called “hallucinations” particularly if a prompt was not well crafted. For instance a user might ask a large-language model (LLM) program, like ChatGPT, to make a list of 10 legal cases involving a particular person or company. If this is a question that does not have 10 answers, the program might produce three accurate responses and make up the rest of them. Biased responses is another issue with the LLM programs where the answers they give are only as good as the information they are trained on and can therefore produce information that is not accurate and intentionally misleading. The same types of biases can apply to facial recognition technology used in law enforcement where a biased data base misidentifies a potential suspect.

Audience members were left with two primary takeaways. One, there are a lot of efficiencies associated with adopting artificial intelligence to government needs and processes. And two, guardrails and regulations will be necessary to make sure the systems are used in accordance with appropriate guidelines and not overly relied on without cross-checking accuracy.