Chief of Staff for the Masses: How Meta’s Joshua To Designs Wearables with AI

In this episode, Joshua To, VP of Product Design at Meta, shares how AI is reshaping how—and where—we interact with technology. He walks us through Meta’s evolving approach to AR and wearables, why notifications are still the killer use case, and how AI is becoming the “brain behind empathy.” We dig into what it means to build interfaces that understand you, why audio might be the future’s most underrated platform, and how designing for emotion changes everything—from form factor to function. Joshua also reflects on his path from launching a clothing brand to leading design at Google and Meta, and what those worlds taught him about craft, context, and human-centered systems. This one’s for anyone designing AI into the real world—where every interface choice carries weight, and intelligence starts with listening. Key takeaways: Empathy Is the Real Intelligence — Joshua flips the definition of smart tech. It’s not just about outputs—it’s about understanding you. Context, tone, emotion— that’s what great AI will sense and respond to. Design for the Moment, Not the Feed — AR’s killer use case isn’t games—it’s restraint. Joshua shares why the best AI product might just be the one that knows not to ping you. Context-aware computing is the real unlock. Audio Is the Interface to Watch — Forget screens. The most powerful interface might be your ears. From wearables to ambient signals, Joshua explains why audio design is the next big frontier for human-centered AI. AR Isn’t a Feature—It’s a System of Consideration — Joshua reframes augmented reality as quiet, ambient infrastructure. The real power of AR isn’t spectacle—it’s subtlety. It helps you move through the world with less friction, not more. LinkedIn: Joshua To | LinkedIn Website: Home - Joshua To Meta: Meta Careers 00:00 Intro: Fixing Notifications With AI 00:54 Meet Josh: VP of Product Design at Meta 02:06 From Hoodies to Hardware: Josh's Journey 03:53 The Google Experience: From Ads to Product Management 10:37 The Evolution of Google Glass and AR 19:12 The Role of AI in Josh's Career 22:25 Designing the Future: AR, VR, and Attention Management 32:49 Contextually Aware Suggestions 33:38 Leveraging Generative AI in Design 34:52 AI's Role in Concept Art and Storyboarding 41:24 AI Tools and Model Capabilities 45:54 The Future of AI and Wearables 51:58 Reflections and Takeaways 📜 Read the transcript for this episode: Transcript of Chief of Staff for the Masses: How Meta’s Joshua To Designs Wearables with AI For more prompts, tips, and AI tools. Check out our website: https://www.beyondtheprompt.ai/ or follow Jeremy or Henrik on Linkedin: Henrik: https://www.linkedin.com/in/werdelin Jeremy: https://www.linkedin.com/in/jeremyutley Show edited by Emma Cecilie Jensen.