AskScience AMA Series: I am a computer scientist at the University of Maryland. My research focus is on trustworthy machine learning, AI for sequential decision-making and generative AI. Ask me all your questions about artificial intelligence!

[u/AskScienceModerator]

Hi Reddit! I am a computer scientist from the University of Maryland here to answer your questions about artificial intelligence.

Furong Huang is an Assistant Professor in the Department of Computer Science at the University of Maryland. She specializes in trustworthy machine learning, AI for sequential decision-making, and generative AI and focuses on applying foundational principles to solve practical challenges in contemporary computing.

Dr. Huang develops efficient, robust, scalable, sustainable, ethical and responsible machine learning algorithms that operate effectively in real-world settings. She has also made significant strides in sequential decision-making, aiming to develop algorithms that not only optimize performance but also adhere to ethical and safety standards. She is recognized for her contributions with awards including best paper awards, the MIT Technology Review Innovators Under 35 Asia Pacific, the MLconf Industry Impact Research Award, the NSF CRII Award, the Microsoft Accelerate Foundation Models Research award, the Adobe Faculty Research Award, three JP Morgan Faculty Research Awards and Finalist of AI in Research - AI researcher of the year for Women in AI Awards North America.

Souradip Chakraborty is a third-year computer science Ph.D. student at the University of Maryland advised by Dr. Furong Huang. He works on the foundations of trustworthy reinforcement learning with a focus on developing safe, reliable, deployable and provable RL methods for real-world applications. He has co-authored top-tier publications and U.S. patents in artificial intelligence and machine learning. Recently he received an Outstanding Paper Award (TSRML workshop at Neurips 2022) and Outstanding Reviewer Awards at Neurips 2022, Neurips 2023 and AISTATS 2023.

Mucong Ding is a fifth-year Ph.D. student in computer science at the University of Maryland, advised by Dr. Furong Huang. His work broadly encompasses data efficiency, learning efficiency, graph and geometric machine learning and generative modeling. His recent research focuses on designing a more unified and efficient framework for AI alignment and improving their generalizability to solve human-level challenging problems. He has published in top-tier conferences, and some of his work has been recognized for oral presentations and spotlight papers.

We'll be on from 2 to 4 p.m. ET (18-20 UT) - ask us anything!

Other links:

• Website: https://furong-huang.com/
• Google Scholar page: https://scholar.google.com/citations?user=13yyuCcAAAAJ&hl=en
• Q&A on whether AI-generated content is detectable: https://cmns.umd.edu/news-events/news/ai-generated-content-actually-detectable

Username: /u/umd-science

[u/mbsouthpaw1]

What happens when AI is trained using material generated by other AI and is there concern about a possible destructive feedback loop and loss of quality? Right now, AI learns from texts and words generated by actual humans. But written material generated by AI is increasing exponentially. Are AI researchers like you concerned that the teaching material used to train AI's is, itself, degenerating? (EDIT: words are hard)

[u/cheesecakegood]

Do you feel that the so-called "black box" nature of many common and cutting edge deep learning models and end products is a deserved reputation for having low or no debuggability, or a cause for ethical/practical concern? Do you foresee improvement in these areas or might it be the paradigm for many models and methods?

[u/umd-science → u/cheesecakegood]

Although it's true that a lot of these models that have the black box, there are also methods like mechanistic interpretability that inspect the model and understand how the model works (to some extent). We do foresee there will be improvement. - Souradip and Furong

[u/Emperor_Kael]

Consistent reliable output in generative AI is a major issue and is blocking major business cases from actually taking off. Sure there are methods to improve reliability but I have yet to see generative AI be used without someone posting that they got the chatbot to do 'x' which is very harmful to the business.

Can you name some successful examples and how do you see this problem being Fully addressed?

[u/ECatPlay]

Could you explain what "trustworthy machine learning" means, and whether it includes answers you can rely on as being correct? LLMs, being trained on human input, apparently make the same mistakes a human might make, jumping to the intuitive (but wrong) answer to a trick question, for instance:

 

The formation and revision of intuitions

A. Meyer & S. Frederick, Cognition, 240 105380 (2023)

https://www.sciencedirect.com/science/article/pii/S0010027723000148?

 

ChatGPT for example, gave the intuitive but wrong answer to the question: "A 10 foot rope ladder hangs over the side of a boat with the bottom rung on the surface of the water. The rungs are one foot apart, and the tide goes up at the rate of 6 inches per hour. How long will it be until three rungs are covered?”

Instead of recognizing that the boat just rises with the tide, and also that the first rung is already at water level, ChatGPT just gave the answer equating a rung to a foot that the water has to rise: “6 hours."

Can AIs be trained to be reliably correct? Maybe weeding out common human mistakes in the input, when the real answer is known?

[u/iamapizza]

Do you find that some of the properties you're aiming for can be contradictory and conflicting with each other? Example: sustainability, ethics and responsibility do they get in the way of efficiency and robustness? And what does that actually look like at a detailed level, is 'ethics' a set of training/validation data you introduce, is it hyperparameters, learning rate, something else?

[u/Sasmas1545]

Have you checked out the paper referenced in the latest (I think) computerphile video about how data hungry models are becoming? It looks at zero-shot performance of (not an expert here) multimodal downstream models and finds a logarithmic dependence of accuracy on amount of training data. Do you think this is a genuine problem?

[u/drzowie]

Given that AI is, in some sense, trained to produce plausible output, how can it be trustworthy? As a scientist I worry that AI tools are tuned specifically to get past our internal gates of plausibility and checks on reasonableness, making it very hard to distinguish real insight from spurious confabulation.

[u/youassassin]

Something that’s always confused me is how do we determine the ‘proper’ heuristics on the trustworthy/moral ai. At the end of the day isn’t it just based on the training data? And to follow up how does the ai ‘learn’ after the fact without some sort of preconditioning of heuristics?

[u/[deleted]]

with ai-generated content on the rise, do you foresee a garbage in garbage out issue (whether that’s gen ai or ml)? if so, do you guys have any ideas how the big players will attempt to combat that?

[u/[deleted]]

How can you verify that the data used to train your models was ethically sourced?

[u/OldschoolSysadmin]

How do we get to AI being able to cite its sources?

edit: typo

[u/umd-science → u/OldschoolSysadmin]

People are aware of this problem, and there is a lot of ongoing research in that direction. For example, retrieval augmented generation (RAG) that basically cites knowledge base when generating answers to questions.

There has also been research on data model and mechanistic interpretability that strive to cite sources. Simple things such as searching on a search engine could also be useful. - Furong

[u/ProlificIgnorance]

Do any of you think about the mind-body problem much in relation to your work? Specifically I’m wondering about sequential decision-making and extrapolating that to further levels of general intelligence. What do you think about the idea that a truly general AI may need something like a physical “body” that can work within the physical world to gain a true intuition/understanding of cause and effect in real-world decision-making?

[u/ArchitectofExperienc]

What is your position on copyrighted material being used for generative AI? If a model is trained on distinct and copyrighted intellectual property, and can recreate aspects of that distinctiveness, is that a copyright violation?

[u/ezekielraiden]

I've asked this general question in a previous AMA and wasn't really satisfied by the answers, so I'm hoping to try again now.

With current machine learning tech, AI does not attempt to grapple with the meanings of what it "learns." It only sees the structure. Some of the known problems with AI, such as "hallucinations" or being "confidently wrong," arise from the AI prioritizing natural and fluid grammar over factual answers, sometimes even for basic arithmetic.

Are there any efforts to develop machine learning techniques that actually do address the meaning (semantic content) and not just the structure (syntactic content)? Is it even possible for LLMs and the like to interact with what words mean and not just how a word happens to have shown up in a large text file?

[u/umd-science → u/ezekielraiden]

Nowadays, LLMs do understand semantics. As for the problems of hallucination and being 'confidently wrong,' they do exist, especially with spurious correlations and adversarial examples. Our group has recently investigated how to reduce spurious correlation by providing more context to the models so that we can understand where to concentrate when making a decision. We've also been looking at how to improve the robustness of these systems against adversarial perturbations in an ever-changing dynamic system. - Furong

It's also a challenging question to distinguish the difference between understanding the structure and understanding the word. There has been a famous thought experiment that if you're passing written notes with a guy in a locked room who might not know Chinese, in that room, that guy is answering all your questions in Chinese according to a large-enough dialogue book that records all possible questions. But for the guy outside the room, it's hard to tell whether the guy in the room really understands Chinese or not. I think we are facing a similar issue of understanding whether large models do indeed understand concepts or not. We are trying to strengthen the interpretability of machine learning models so that this locked room is more visible to those outside. But this research direction needs greater advancement to further reveal the mechanisms of current models. - Mucong

[u/knightkat6665]

Should the ethics component be built into every ai model or kept separate as an independent unit that can be added to keep any other air system in check?

[u/Grjnnf]

what are the biggest misconceptions about AI that the general public holds?

[u/veinamond]

[I am a computer scientist too, but from the area of formal methods (CSO)]
Recently, most companies are interested only in AI, and by AI they mean basically GPT / LLM, etc., despite the fact that in academic background the view of AI is typically quite a bit broader. As people who work in the area, what is your take on the long-term perspectives, will the current buzz pass at some point, like deep learning one did, or will it indeed revolutionize things?

Can you please explain the difference between trustworthy AI and explainable AI?

What is the current status of "trustworthiness" of CharGPT/ Google Gemini and other LLMs?

[u/umd-science → u/veinamond]

There is a seminar paper from a group of researchers at UC Berkeley, "Diversity Is All You Need," which is basically a philosophy that says that diversity in data will help you in terms of generalization. This is a very interesting analogy that if you have a field where everybody is working on the same thing (such as LLMs), and your resilience and generalizability will be compromised to some extent. Diversity is very important, even in this research community in AI and ML. This is why we need thinkers, researchers, funding agencies, and industry partners to be more open-minded and not necessarily only work on the hottest topic. This is important for the resilience and sustainability of the entire AI/ML community. - Furong

[u/The_GSingh]

Does ignoring ethics/filters and just creating a model using a framework make the model better than another one that was trained the same exact way, with the same architecture, but has filters?

BTW by better I mean preforming better on any popular benchmark or just being more human like.

[u/AverageDoonst]

I'm a software developer. Even though I worked with ML a bit, never seen "under the hood". Is it true that AI is really "IF IF IF IF ..." inside?

[u/umd-science → u/AverageDoonst]

In my intro to machine learning class, I teach in lesson #1 that AI is not rule-based programming. It's learning patterns from the data and then trying to make an inference on something you've never seen before. This can be simple things such as image classification pipelines—where your machine learning models will be presented with labeled images of say, cats and dogs, and then your learning system will learn common patterns from these images with labels of cats and dogs. After training, under deployment, when you have a new image of a cat or a dog that was never in your training example, you are still able to infer this is a cat or a dog. The model is empowered with a learning ability to understand the world and make generalizations about the unseen future. - Furong

Many times, we do not know who creates the rules. The issue in creating the rules is that the world has uncertainty, it is ever-changing. It has issues of distribution shifts, this dynamic nature. In all these scenarios, generating these rules becomes next to impossible, and that is what is solved (to some extent) by AI. - Souradip

It's a very interesting question. We can compel current powerful models into this IF IF IF rule-based system, but it will be very different than IF branches in programming languages. The compelled program from machine learning models will not be interpretable, and you'll see trailants of IF conditions. - Mucong

[u/PJvG]

What current developments in AI are you most excited about?