lorraine2 (Jonathan Lorraine)

reacted to their post with ❤️ about 22 hours ago

Post

773

📽️ New NVIDIA paper: Motion Attribution for Video Generation 📽️

We propose MOTIVE, a method for taking query video clips and identifying which training data will improve or degrade performance after finetuning, enabling sophisticated data curation and beyond!

🔎 Project Page: https://research.nvidia.com/labs/sil/projects/MOTIVE/
📖 Full Paper: https://arxiv.org/abs/2601.08828

Check out more work from the NVIDIA Spatial Intelligence Lab here: https://research.nvidia.com/labs/sil/

This project was led by the great work of Xindi(Cindy) Wu, along with Despoina Paschalidou, Jun Gao, Antonio Torralba, Laura Leal-Taixé, Olga Russakovsky, and Sanja Fidler.

posted an update about 22 hours ago

Post

773

📽️ New NVIDIA paper: Motion Attribution for Video Generation 📽️

We propose MOTIVE, a method for taking query video clips and identifying which training data will improve or degrade performance after finetuning, enabling sophisticated data curation and beyond!

🔎 Project Page: https://research.nvidia.com/labs/sil/projects/MOTIVE/
📖 Full Paper: https://arxiv.org/abs/2601.08828

Check out more work from the NVIDIA Spatial Intelligence Lab here: https://research.nvidia.com/labs/sil/

This project was led by the great work of Xindi(Cindy) Wu, along with Despoina Paschalidou, Jun Gao, Antonio Torralba, Laura Leal-Taixé, Olga Russakovsky, and Sanja Fidler.

authored a paper 6 days ago

Score Distillation Sampling for Audio: Source Separation, Synthesis, and Beyond

Paper • 2505.04621 • Published May 7, 2025 • 1

upvoted a paper 7 days ago

Motion Attribution for Video Generation

Paper • 2601.08828 • Published 8 days ago • 66

authored a paper 7 days ago

Motion Attribution for Video Generation

Paper • 2601.08828 • Published 8 days ago • 66

posted an update 9 months ago

Post

534

🔊 New NVIDIA paper: Audio-SDS 🔊

We adapt Score Distillation Sampling (SDS), originally developed for text-to-3D generation, to audio diffusion models, allowing us to reuse large pretrained models for new text-guided parametric audio tasks such as source separation, physically informed impact synthesis, and more.

🔎 Project Page: https://research.nvidia.com/labs/toronto-ai/Audio-SDS/
📖 Full Paper: https://arxiv.org/abs/2505.04621

Check out more from NVIDIA’s Spatial Intelligence Lab here: https://research.nvidia.com/labs/toronto-ai/

This project was led by the great work of Jessie Richter-Powell, along with Antonio Torralba.

Notably, we find a new and exciting use case for Stable Audio Open 🚀

authored a paper about 1 year ago

Multi-student Diffusion Distillation for Better One-step Generators

Paper • 2410.23274 • Published Oct 30, 2024

posted an update about 1 year ago

Post

2035

🦙New NVIDIA paper: LLaMA-Mesh 🦙

We enable large language models to generate and understand 3D meshes by representing them as text and fine-tuning. This unifies the 3D and text modalities in a single model and preserves language abilities, unlocking conversational 3D creation with mesh understanding.

🔎 Project Page: https://research.nvidia.com/labs/toronto-ai/LLaMA-Mesh/
🕹️ Interactive Demo: Zhengyi/LLaMA-Mesh (courtesy of HuggingFace and Gradio)
📖 Full Paper: https://arxiv.org/abs/2411.09595
👨‍💻Code: https://github.com/nv-tlabs/LLaMa-Mesh
💾 Model Checkpoint: Zhengyi/LLaMA-Mesh
🧩 Blender Addon: https://github.com/huggingface/meshgen (courtesy of Dylan Ebert)
🎥 5-min Overview Video: https://youtu.be/eZNazN-1lPo?si=-idQa5aaceVw0Bbj (courtesy of AI Papers Academy)

reacted to their post with 👀 about 1 year ago

Post

1234

New NVIDIA paper: ⚡ Multi-student Diffusion Distillation for Better One-step Generators ⚡

Do you want to make your diffusion models (a) run in a single step, (b) run with a smaller model, and (c) have improved quality simultaneously? Check out our multi-student distillation (MSD) method, which is simple and applicable to most diffusion models! The only catch is now we have to distill (and store) a mixture-of-expert student generators.

Explore the MSD project page to learn more: https://research.nvidia.com/labs/toronto-ai/MSD/

Work led by Yanke Song along with Weili Nie, Karsten Kreis and James Lucas

Check out more work from the Toronto AI Lab here: https://research.nvidia.com/labs/toronto-ai/

2 replies

·

posted an update about 1 year ago

Post

1234

New NVIDIA paper: ⚡ Multi-student Diffusion Distillation for Better One-step Generators ⚡

Do you want to make your diffusion models (a) run in a single step, (b) run with a smaller model, and (c) have improved quality simultaneously? Check out our multi-student distillation (MSD) method, which is simple and applicable to most diffusion models! The only catch is now we have to distill (and store) a mixture-of-expert student generators.

Explore the MSD project page to learn more: https://research.nvidia.com/labs/toronto-ai/MSD/

Work led by Yanke Song along with Weili Nie, Karsten Kreis and James Lucas

Check out more work from the Toronto AI Lab here: https://research.nvidia.com/labs/toronto-ai/

2 replies

·

reacted to their post with 👀 about 1 year ago

Post

296

New NeurIPS paper: “Training Data Attribution via Approximate Unrolling”

Ever wondered how individual data points influence AI decisions? 🤔 We explore how specific training data pieces affect machine learning models' behavior, which can be crucial for making AI systems more transparent, trustworthy, and fair.

Our method, SOURCE, bridges the gap between implicit differentiation and unrolling approaches, combining computational efficiency with flexibility making it suitable for non-converged models and multi-stage training pipelines.

📄 Full paper: https://openreview.net/pdf?id=3NaqGg92KZ

Juhan Bae led along with Wu Lin and Roger Grosse.

Supported by the University of Toronto, Vector Institute, NVIDIA, and Anthropic

posted an update about 1 year ago

Post

296

New NeurIPS paper: “Training Data Attribution via Approximate Unrolling”

Ever wondered how individual data points influence AI decisions? 🤔 We explore how specific training data pieces affect machine learning models' behavior, which can be crucial for making AI systems more transparent, trustworthy, and fair.

Our method, SOURCE, bridges the gap between implicit differentiation and unrolling approaches, combining computational efficiency with flexibility making it suitable for non-converged models and multi-stage training pipelines.

📄 Full paper: https://openreview.net/pdf?id=3NaqGg92KZ

Juhan Bae led along with Wu Lin and Roger Grosse.

Supported by the University of Toronto, Vector Institute, NVIDIA, and Anthropic

liked a model about 1 year ago

Zhengyi/LLaMA-Mesh

Text-to-3D • 8B • Updated Dec 7, 2024 • 328 • 177

liked a Space about 1 year ago

LLaMA Mesh

👀

146

Create 3D mesh by chatting.

upvoted a paper about 1 year ago

LLaMA-Mesh: Unifying 3D Mesh Generation with Language Models

Paper • 2411.09595 • Published Nov 14, 2024 • 77

authored a paper about 1 year ago

LLaMA-Mesh: Unifying 3D Mesh Generation with Language Models

Paper • 2411.09595 • Published Nov 14, 2024 • 77

posted an update over 1 year ago

Post

567

🚨 Code now available for "Using Large Language Models for Hyperparameter Optimization" at https://github.com/michaelrzhang/LLM-HyperOpt 🚨

TLDR: You can just ask LLMs which hyperparameters to use, and it works pretty well! You can even directly optimize your model’s code as a hyperparameter with this.

Check out the paper at https://arxiv.org/abs/2312.04528 - with Michael Zhang, Nishkrit Desai, Juhan Bae, and Jimmy Ba

authored 3 papers over 1 year ago

Jonathan Lorraine

AI & ML interests

Recent Activity

Organizations

Score Distillation Sampling for Audio: Source Separation, Synthesis, and Beyond

Motion Attribution for Video Generation

Motion Attribution for Video Generation

Multi-student Diffusion Distillation for Better One-step Generators

Zhengyi/LLaMA-Mesh

LLaMA Mesh

LLaMA-Mesh: Unifying 3D Mesh Generation with Language Models

LLaMA-Mesh: Unifying 3D Mesh Generation with Language Models

Using Large Language Models for Hyperparameter Optimization

Graph Metanetworks for Processing Diverse Neural Architectures

Scalable Nested Optimization for Deep Learning

Jonathan Lorraine

AI & ML interests

Recent Activity

Organizations

lorraine2's activity

LLaMA Mesh