Hexa-P: Physical Plausibility-aware Pre-training for Pedestrian Path Prediction

Both methods see the last two frames (0.8 s) of a pedestrian in JTA (synthetic) or JRDB (real) and predict five paths for the next 4.8 s.

Abstract

Pedestrian path prediction is a safety-critical component of autonomous vehicles and social robots that must navigate alongside humans in open urban environments. In such deployments, predictions that imply unnatural accelerations or impossible turns can directly trigger unsafe planning decisions, especially when only a few past frames of the pedestrian are observed. Because pedestrians are full-body humans whose paths are governed by articulated locomotion, we exploit two resources: 3D human body pose, which encodes locomotion-level constraints, and a humanoid physics simulator, which synthesizes virtually unlimited, physically executable pose-and-path data. We propose “Physical Plausibility-aware Pre-training for Pedestrian Path Prediction” (Hexa-P), which uses this synthesized data to jointly pre-train a path predictor and a plausibility estimator. Unlike prior work that uses a separately trained plausibility estimator, this joint pre-training improves both prediction accuracy and physical plausibility. While this simulation involves only a single agent, multiple agents mutually influence one another's future paths in real-world scenes. We therefore introduce an adapter-based fine-tuning that preserves physical knowledge while adapting to scene-specific social path interactions. We also propose Target-relative Positional Encoding (TaPE) to bridge the gap between single-agent simulation and multi-agent path prediction. Experiments on JRDB, JTA, and the Urban dataset demonstrate accurate and plausible predictions of Hexa-P. Particularly in safety-critical short-observation scenarios, Hexa-P reduces prediction errors by up to 50% on the JRDB dataset compared with a strong pose-aware baseline.

Overview of Hexa-P: physics pre-training, adapter-based fine-tuning and dual-mode inference

Large-scale pre-training in a physics simulator

In IsaacGym, a humanoid controller walks randomly generated paths. Hexa-P pre-trains on these rollouts before it sees any real pedestrian.

4,096humanoids in parallel

2,000pre-training episodes

1.19 Bsimulated frames

1A100 GPU

Green: paths the humanoid walks. Red: paths on which it falls.

1Sample a path

Random speed, acceleration and turns.

2Walk it under physics

The humanoid is rewarded for a natural gait along the path.

3Learn from the rollout

The walked path trains the Path-Head; the reward trains the LocoVal-Head.

Learning what a body can and cannot do

A rollout earns a high reward when the humanoid follows its path and a low one when it falls. The LocoVal-Head learns this reward and, at test time, rejects implausible predicted paths.

Walks its path: high reward.

Falls on sudden turns: low reward, and no path loss.

Fewer observed frames

The same pedestrian with fewer observed frames. The curve gives the test ADE of the dataset.

Safer robot navigation

A robot crosses replayed JTA crowds and avoids the predicted paths. The bars give the collision rate over 100 test episodes. In the videos, thin lines link each person to the positions predicted about 2 s ahead, and gray crosses mark hypotheses that LocoVal rejects.

More results

More walking pedestrians in JTA and JRDB, with two observed frames.

Paper

Hiromu Taketsugu, Shohei Nobuhara and Norimichi Ukita. Hexa-P: Physical Plausibility-aware Pre-training for Pedestrian Path Prediction. Pattern Analysis and Applications, 2026.

Code, configurations and instructions to reproduce the experiments are on GitHub; weights and preprocessed data are on Hugging Face.

BibTeX

@article{taketsugu2026hexap,
  title   = {Hexa-P: Physical Plausibility-aware Pre-training for Pedestrian Path Prediction},
  author  = {Taketsugu, Hiromu and Nobuhara, Shohei and Ukita, Norimichi},
  journal = {Pattern Analysis and Applications},
  year    = {2026}
}