Human Stories in AI: Rick Marks

Overview

Rick Marks, a professor at UNC Chapel Hill's School of Data Science and Society, shares his career journey from a high school interest in using technology for social augmentation to leading AI research at Google and founding the PlayStation Magic Lab. His work has focused on computer vision, ambient computing, and generative AI, with a current emphasis on combining AI with virtual reality to create dynamic, user-driven experiences, exemplified by student projects like a "magic canvas" for VR art generation.

Key takeaways

Rick Marks' career began with a high school project to create an AI for social augmentation, a concept he revisited at Google and explored with generative AI.
His PhD research pivoted from control theory to computer vision due to the challenges of underwater robot sensing, leading to work on marine snow modeling.
At PlayStation, he founded the Magic Lab, and at Google, he worked on ambient computing and real-time AI inference, including gesture recognition.
Current research at UNC focuses on integrating generative AI with VR, as seen in student projects like the 'Magic Canvas' for dynamic VR art creation.
Latency in generative AI, especially for 3D models, is a significant challenge for real-time interactive applications.
Marks advocates for a 'T-shaped' professional profile, combining broad interdisciplinary knowledge with deep specialization, to enhance collaboration and communication in tech teams.

Chapters

0:00 Early Aspirations: AI for Social Augmentation

Rick Marks' initial high school project aimed to use technology to augment social interactions, like generating the perfect joke or quote.
This early concept foreshadowed current AI capabilities like ChatGPT, which can provide relevant conversational input.
The project highlighted the challenge of real-time context awareness, a theme that would recur throughout his career.

6:10 The Daxter Project: Interactive AI Companions

Marks explored combining AI with video game concepts, leading to the 'Daxter' project.
Daxter was envisioned as an AI companion offering color commentary on user's lives, inspired by the character from the Jack and Daxter game series.
The goal was to integrate AI unobtrusively, offering insights or humor without constant distraction, potentially through screen savers or glanceable interfaces.

10:03 Transition to Computer Vision and Robotics

Marks repurposed early code for a senior year robot project featuring a computer with a customizable face and personality.
His career evolved through interests in 3D graphics, virtual reality, and computer vision.
A PhD thesis initially focused on control theory for an underwater robot at the Monterey Bay Aquarium Research Institute, but shifted to computer vision due to the complexity of underwater sensing.

15:05 Industry Experience: Sony, PlayStation, and Google

Marks worked at a computer vision startup, then at Sony using computer vision for video games.
At PlayStation R&D, he founded the Magic Lab, focusing on new interaction approaches for ambient computing.
At Google's Advanced Technology and Projects group, his team researched context understanding and real-time AI inference for applications like gesture recognition.

20:08 Generative AI and VR: The Magic Canvas Project

Marks' current research interests include combining AI and virtual reality.
A student project, 'Magic Canvas,' used generative AI to create images in VR based on user voice commands, demonstrating text-to-image generation.
Challenges in generative AI include latency, particularly for 3D model generation (1 minute vs. seconds for images), impacting real-time user interfaces.

28:54 The Future of Storytelling Conference and T-Shaped Careers

Marks highlighted the 'Future of Storytelling' conference in New York, focusing on the intersection of technology and narrative.
He advocates for a 'T-shaped' career profile: broad knowledge across many areas with deep expertise in one specific field.
This T-shape facilitates communication and collaboration within diverse teams, bridging different technical languages and perspectives.

Summary, takeaways, and chapters were generated by AI from the video's transcript and may contain errors. The video belongs to its creator, StatQuest with Josh Starmer.