1st Workshop on Cooperative Intelligence for Embodied AI

The 18th European Conference on Computer Vision 2024 in Milan, Italy

September 30th, 2024

This workshop focuses on cooperative intelligence within multi-agent autonomous systems. The first session will primarily discuss “Multi-Agent Autonomous Systems Meet Foundation Models: Challenges and Futures”.

The progress in artificial intelligence has propelled the development of Embodied AI, particularly in autonomous driving and robotics. However, achieving autonomy in complex and open environments remains challenging for individual agents. This is where cooperative intelligence comes in, a paradigm where agents collaborate and interact with infrastructure equipment to handle diverse tasks effectively. In autonomous driving, the availability of datasets and breakthrough algorithms has spurred research interest in cooperative autonomous driving. Through Vehicle-to-Everything (V2X) interactions including Vehicleto-to-Vehicle (V2V) and Vehicle-to-Infrastructure (V2I), autonomous vehicles can improve their perception and safety capabilities, surpassing the limitations of single-vehicle autonomy and paving the way for widespread adoption. Similarly, in robotics, recent advancements in multi-agent systems offer a cost-effective solution for exploring unknown environments. These breakthroughs empower robots to assist humans in navigating challenging tasks within open environments. Despite notable progress, the challenges of coordinating multi-agent systems remain insufficiently explored. Issues like determining what information to transmit, how to transmit it, and how to fuse information across different task levels (perception, prediction, planning, etc.) pose practical deployment hurdles. Recent breakthroughs in large language models and foundational models offer a promising avenue for addressing these challenges. This workshop, complementing existing workshops focused on individual intelligence in autonomous driving and robotics, introduces a fresh perspective. It fosters discussions on a comprehensive, system-level approach to cooperative framework design for autonomous agents across diverse domains.

Specifically, this workshop aims to address the following issues regarding cooperative intelligence:

  • The value and prospects of cooperative intelligence for autonomous driving and robotics.
  • The challenges and recent advancements in cooperative intelligence for multi-agent autonomous systems.
  • The foundation models for multi-agent autonomous systems.

   

Call for papers

Submission Topics

We invite submissions including but not limited to the following topics:

  • Vehicle-to-Vehicle (V2V) and Vehicle-to-Infrastructure (V2I)
  • Multi-agent Robotic System / Swarm Robots
  • Communication-Efficient Cooperative Perception
  • Cooperative Motion Prediction and Decision-Making
  • End-to-End Cooperative Policy Learning
  • Efficient and Intelligent V2X Communication
  • Data Security and Privacy in Multi-agent Communication
  • Multi-Robot Exploration and Mapping
  • Intelligent Transportation System
  • Large Language Model (LLM)-assisted Cooperative System
  • Foundation models for Cooperative System
  • Datasets and Benchmarks for Cooperative Learning

Important Dates

  • Paper submission open: June 1st, 2024

  • Paper submission deadline: July 25th, 2024 (PST) August 10th, 2024 (PST)

  • Notification of acceptance: August 8th, 2024 August 15th, 2024

  • Camera ready: August 15th, 2024 (PST) August 22nd, 2024 (PST)

  • Workshop date: PM of Sep 30th, 2024 (See ECCV Workshop Lists)

Submission Guidance

  • Submission portal: ECCV 2024 Workshop Co-Intelligence (OpenReview).

  • Submission format: Submissions must follow the ECCV 2024 template (here) and will be peer-reviewed in a double-blind manner. Submission must be no more than 14 pages (excluding references). By default, accepted papers will be included in the ECCV workshop proceedings. Accepted papers will be presented in the form of posters, with several papers being selected for spotlight sessions.

Important Notes

The accepted papers will be featured in the proceedings, in a separate volume. The Best Paper Award and Best Paper Nomination will receive a cash prize of 500 US dollars and 300 US dollars, respectively.

Contact

If you have any questions, please contact us at: coop-intelligence@googlegroups.com or yuhaibao94@gmail.com.

   

Accepted papers

We accept 20 papers from submissions, including 6 orals. Paper list is as following: Title-Authors-Oral/Poster

  • Multi-agent Collaborative Perception for Robotic Fleet: A Systematic Review - Apoorv Singh, Gaurav Raut, Alka Choudhary - Poster
  • RP3D: A Roadside Perception Framework for 3D Object Detection via Multi-View Sensor Fusion - Shaowu Zheng, Ruyi Huang, Yuan Ji, Ming Ye, Weihua Li - Oral
  • StreamLTS: Query-based Temporal-Spatial LiDAR Fusion for Cooperative Object Detection - Yunshuang Yuan, Monika Sester - Poster
  • GPT4RoI: Instruction Tuning Large Language Model on Region-of-Interest - Shilong Zhang, Peize Sun, Shoufa Chen, Min Xiao, Wenqi Shao, Wenwei Zhang, Yu Liu, Kai Chen, Ping Luo - Poster
  • SC-Track: State Transition and Constrained Non-negative Matrix Factorization for Multi-Camera Multi-Target Tracking - Xiaolong Yang, Xuting Duan, Jianshan Zhou, Chunmian Lin, XuHan - Poster
  • Gen-Swarms: Adapting Deep Generative Models to Swarms of Drones - Carlos Plou, Pablo Pueyo, Ruben Martinez-Cantin, Mac Schwager, Ana C. Murillo, Eduardo Montijano - Oral
  • VICooper: A Practical Vehicle-Infrastructure Cooperative Perception Framework for Autonomous Driving - Shaowu Zheng, Ming Ye, Yuan Ji, Ruyi Huang, Weihua Li - Poster
  • MEDCO: Medical Education Copilots Based on A Multi-Agent Framework - Hao Wei, Jianing Qiu, Haibao Yu, Wu Yuan - Poster
  • V2X-Based Decentralized Singular Value Decomposition in Dynamic Vehicular Environment - Jianxin Zhao, Min-Bin Lin, Alexey Vinel - Poster
  • LLaMAPed: Multi-modal Pedestrian Crossing Intention Prediction - Je-Seok Ham, Sunghun Kim, Jia Huang, Peng Jiang, Jinyoung Moon, Srikanth Saripalli, Changick Kim - Poster
  • Optimization of Layer Skipping and Frequency Scaling for Convolutional Neural Networks under Latency Constraint - Minh David Thao Chan, Ruoyu Zhao, Yukuan Jia, Ruiqing Mao, Sheng Zhou - Poster
  • An Infrastructure-based Localization Method for Articulated Vehicles - Alberto Justo, Iker Pacho, Javier Araluce, Jesus Murgoitio, Luis M. Bergasa - Poster
  • HEAD: A Bandwidth-Efficient Cooperative Perception Approach for Heterogeneous Connected and Autonomous Vehicles - Deyuan Qu, Qi Chen, Yongqi Zhu, Yihao Zhu, Sergei S. Avedisov, Song Fu, Qing Yang - Poster
  • Rethinking the Role of Infrastructure in Collaborative Perception - Hyunchul Bae, Minhee Kang, Minwoo Song, Heejin Ahn - Oral
  • Empowering Autonomous Shuttles with Next-Generation Infrastructure - Sven Ochs, Melih Yazgan, Rupert Polley, Albert Schotschneider, Stefan Orf, Marc Uecker, Maximilian Zipfl, Julian Burger, Abhishek Vivekanandan, Jennifer Amritzer, Marc Rene Zofka, J. Marius Zöllner - Oral
  • MAPPO-PIS: A Multi-Agent Proximal Policy Optimization Method with Prior Intent Sharing for CAVs’ Cooperative Decision-Making - Yicheng Guo, Jiaqi Liu, Rongjie Yu, Peng Hang, Jian Sun - Oral
  • RoboTwin: Dual-Arm Robot Benchmark with Generative Digital Twins (early version) - Yao Mu, Tianxing Chen, Shijia Peng, Zanxin Chen, Zeyu Gao, Yude Zou, Lunkai Lin, Zhiqiang Xie - Oral
  • iIPPC-V2X: Multi-modality Fusion Perception System for Cooperative Vehicle Infrastructure System with Self-supervised Learning - Guoyu Zhang, Rongjie Yu, Jian Sun, Peng Hang - Poster
  • Non-verbal Interaction and Interface with a Quadruped Robot using Body and Hand Gestures: Design and User Experience Evaluation - Soohyun Shin, Trevor Evetts, Hunter Saylor, Hyunji Kim, Soojin Woo, Wonhwha Rhee, Seong-Woo Kim - Poster
  • Transfer Learning from Simulated to Real Scenes for Monocular 3D Object Detection - Sondos Mohamed, Walter Zimmer, Ross Greer, Ahmed Alaaeldin Ghita, Modesto Castrillón-Santana, Mohan Trivedi, Alois Knoll, Salvatore Mario Carta, Mirko Marras - Poster

   

Invited Speakers (TBD)

 

Jiaqi Ma
UCLA, USA
Alois C. Knoll
TUM, Germany
Hang Qiu
University of California, Riverside, USA
Guillaume Sartoretti
NUS, Singapore
Sheng Zhou
Tsinghua University, China
Amanda Prorok
Cambridge University, UK
Zsolt Kira
Gatech, USA
Manabu Tsukada
The University of Tokyo, Japan
Marco Pavone
Standford University, USA
Jingwei Ji
Waymo, USA
Boris Ivanovic
Nvidia, USA
Federico Tombari
Google, Switzerland

 

Organizers

 

Haibao Yu
University of Hong Kong & Tsinghua University
Jiankai Sun
Stanford University
Li Chen
Shanghai AI Lab
Walter Zimmer
Technical University of Munich
Mac Schwager
Stanford University
Ping Luo
University of Hong Kong
Ruigang Yang
Inceptio Technology
Si Liu
Beihang University
Zaiqing Nie
Tsinghua University

Program Committee

 

• Jiaru Zhong (BIT)

• Zhenwei Yang (USTB)

• Yang Liu (Tsinghua University)

• Enze Xie (HKU)

• Yifeng Shi (Baidu)

• Weitao Zhou (Tsinghua University)

• Yue Hu (SJTU)

• Xuting Duan (Beihang University)

• Zhixuan Liang (HKU)

• Chuheng Wei (University of California, Riverside)

• Churan Zhi (University of California, San Diego)

• Ruiyang Hao (Tsinghua University)

• Siqi Fan (Tsinghua University)

• Yan Wang (Tsinghua University)

• Tuopu Wen (Tsinghua University)

• Yao Mu (HKU)

• Mingyu Ding (UC Berkeley)

• Lipeng Chen (Tencent Robotics X)

• Yilun Wang (Tsinghua University)

• Deyuan Qu (UNT)

• Alberto Justo (Tecnalia Research & Innovation)

• Lin Li (KCL)

• Wenxian Yang (Tsinghua University)

• Yuexin Ma (ShanghaiTech University)

• Guangliang Cheng (UOL)

• Rui Song (TUM)

• Jirui Yuan (Tsinghua University)

• Jianping Shi (SenseTime)

• Jiachen Li (University of California, Riverside)

• Xiaopan Zhang (University of California, Riverside)

• Ghayoor Shah (University of Central Florida)

• Hyunchul Bae (KAIST)

Sponsors