Tongtong Liu (Frank) WFU

Tongtong (Frank) Liu

I am currently a master's student at University of Pennsylvania, School of Engineering and Applied Science, studying Computer and Information Science. I graduated from Wake Forest University in 2023 with a B.S. in Computer Science with honors and B.S. in Mathematical Business. I was previously an undergraduate researcher advised by Dr. Sarra Alqahtani and worked as a research intern at IBM Research mentored by Dr. Mu Qiao in the summer of 2022. My research interests lie in robustness and security of Reinforcement Learning, explainable Reinforcement Learning (XRL), and Foundation Language Model in Natural Language Processing.

Besides research, I have also interned in various technology companies like ByteDance (TikTok), IBM, DataMimo, and Mesoor AI. I have industrial experiences in Software Development Engineering and Machine Learning Engineering. I co-founded The Wakers, an information platform for international students. I am an innovative problem-solver and an inspiring team player. Please get in touch with me and I am happy to talk about any of my past experiences :)

Email / LinkedIn / GitHub

Awards

The John W. Sawyer Prize in Computer Science 2023 (for the top senior graduates in CS)
Academic Excellence in Math Business Award 2023

So, basically I am the best senior in both of my majors :) Yeaaa!

Finalist for the Computing Research Association's (CRA) Outstanding Undergraduate Researcher Award for 2023
Burke M. McConnell Management Excellence Scholarship, 2022
Wake Forest Research Fellowship, 2021

Research

My general research interests lie in different topics of Machine Learning, including data mining, foundation model based chatbot system in NLP, and security, safety, and robustness of Reinforcement Learning.

	Adversarial Behavior Exclusion for Safe Reinforcement Learning Md Asifur Rahman, Tongtong Liu , Sarra Alqahtani IJCAI, 2023 [paper] Adversarial Behavior Exclusion for Safe RL (AdvEx-RL) learns a behavioral representation of the agent’s safety violations by approximating an optimal adversary utilizing exploration and later uses this representation to learn a separate safety policy that excludes those unsafe behaviors.
	A Policy-Graph Approach to Explain Reinforcement Learning Agents: A Novel Policy-Graph Approach with Natural Language and Counterfactual Abstractions for Explaining Reinforcement Learning Agents Tongtong Liu , Joe McCalmon, Thai Le, Dongwon Lee, Sarra Alqahtani AAMAS Journal, 2023 [paper] In this work, we proposea novel approach that summarizes an agent’s policy in the form of a directed graph with natural language descriptions that help end user to understand the logic behind agent's decision. This work is submitted to JAAMAS.
	Weaponizing Actions in Multi-Agent Reinforcement Learning: Theoretical and Empirical Study on Security and Robustness Tongtong Liu, Joe McCalmon, Md Asifur Rahman, Cameron Lischke, Talal Halabi, Sarra Alqahtani PRIMA, 2022 [paper] [code] This paper investigates the robustness of c-MARL to a novel adversarial threat, where we target and weaponize one agent, termed the compromised agent, to create natural observations that are adversarial for its team. This paper shows mathematically the exploitation steps of such an adversarial policy in the centralized-learning and decentralized-execution paradigm of c-MARL. I presented this work at PRIMA 2022.
	Safe Reinforcement Learning via Observation Shielding Joe McCalmon, Tongtong Liu , Reid Goldsmith, Andrew Cyhaniuk, Talal Halabi, Sarra Alqahtani HICSS, 2023 [paper] [code] We proposed a method called observation-shielding RL (OSRL) to increase the robustness of RL against large perturbations using predictive models and threat detection. OSRL builds on the idea of model predictive shielding, where an observation predictive model is used to override the perturbed observations as needed to ensure safety.
	Empathetic Financial GPT (E-FinGPT): A Mixture-of-Expert Way of Building Customer Service Chatbot in Financial Domain Tongtong Liu , Mu Qiao, Divyesh Jadav IBM Research Internship, 2022 We proposed a novel mixture of expert method that combines few-shot learning and model fine-tuning on the SOTA foundation model decoder -- GPT -- to build a customer service chatbot that can respond to offensive customer complaints in a professional and empathetic way. We submitted a patent application and a manuscript is under preparation for ACL 2023.
	LSTM-Based Anomalous Behavior Detection in Multi-Agent Reinforcement Learning Cameron Lischke, Tongtong Liu , Joe McCalmon, Md Asifur Rahman, Talal Halabi, Sarra Alqahtani IEEE CSR, 2022 [paper] [code] We present a novel stacked-LSTM ensemble approach to detect a serious vulnerability in Multi-agent Reinforcement Learning system, compromised agent attack, which one of the agent in the team is controlled by an attacker to subsequently pushes its cooperative agents to act off-policy.
	Robustness-driven Exploration with Probabilistic Metric Temporal Logic Xiaotian Liu, Pengyi Shi, Tongtong Liu , Sarra Alqahtani, Paul Pauca, Miles Silman ICAART, 2021 [paper] The ability to perform autonomous exploration is essential for unmanned aerial vehicles (UAV) operating in unknown environments where it is difficult to describe the environment beforehand. Algorithms for autonomous exploration often focus on optimizing time and full coverage in a greedy fashion that collect irrelevant data and wastes time navigating areas with no important information. In this paper, we aim to improve the efficiency of exploration by maximizing the probability of detecting valuable information
	Multi-Agent Reinforcement Learning for Cooperative Adaptive Cruise Control Joe McCalmon, Ashley Peake, Benjamin Raiford, Tongtong Liu , Sarra Alqahtani ICTAI, 2020 [paper] A growing trend in the field of autonomous vehicles is the use of platooning. The design of control algorithms for platoons is challenging considering that coordination among vehicles is obtained through diverse communication channels. In this paper, we propose a multi-agent reinforcement learning approach for autonomous vehicles which communicate in a platoon formation.