Every attempt reveals more about the rope.
Swinging, twirling and whipping a rope all depend on how the rope responds. The same arm motion can produce very different behaviour as mass, stiffness and damping change. These properties are difficult to measure, and their effects are difficult to reproduce faithfully in simulation.
One way to handle this uncertainty is to model the rope: calibrate a simulator, estimate physical parameters, or fit a model from earlier attempts. RopeFormer explores another way to use that experience.
Can a robot use what happened in one attempt to do better in the next?
RopeFormer uses a Transformer-XL policy to carry information about robot states, actions and rope motion across repeated trials. The policy adapts through this interaction history, with fixed weights and no explicit estimate of the rope's physical parameters. We evaluate it on three tasks — single-arm Rope_Swing, bimanual Rope_Twirl and target-line Rope_Whip — in simulation and on a Unitree H1-2 using previously unseen physical ropes.
Project video
Three tasks, a shared approach to adaptation
We train a separate policy for each task, using the same architecture and multi-trial training setup. A trial is one attempt; an episode is a sequence of attempts with the same rope. Between trials, the robot and rope return to their starting state while the policy retains its context. That context is cleared at the start of each new episode.
(a) Rope_Swing
Holding one end in its right hand, the robot keeps the free tip rotating at a target speed and direction. We measure how quickly it reaches the commanded rotation.
(b) Rope_Twirl
Holding one end in each hand, the robot rotates the rope's midpoint around the line between its hands. We measure how quickly it meets a combined criterion for rotation rate, radius, direction and clearance.
(c) Rope_Whip
The robot whips the free tip along a target line segment in a single sweep. We measure the tip path's largest perpendicular deviation from that line.
Adapt through history, with fixed policy weights
At each control step, the policy receives joint states, measured rope points, the task command or target, and its previous action. An encoder turns this observation into a token. A six-layer Transformer-XL combines it with stored representations of earlier steps, then an action head produces joint targets at 30 Hz. The next observation captures how the rope responded, giving the policy new information for its next action.
Carry experience into the next trial
Resetting the robot and rope does not reset the policy's context. Information from one attempt remains available to guide the next. Attention masks keep histories from different episodes separate.
Practise across a range of ropes
Training in the Newton simulator varies rope properties and actuator behaviour between episodes, then gives the policy several attempts under the same conditions. This lets it encounter a wide range of dynamics and use experience from earlier trials within each episode.
Act on observations alone
We train with PPO and an asymmetric critic: the critic has access to simulated rope and actuator parameters, while the policy acts only on its observation history. Advantage estimation stops at trial boundaries; the policy's context carries across them.
How much does the previous attempt help?
For each task, we compare the same trained policy on a fixed set of 384 ropes, with matched starting states and noise. One condition retains context between trials; the other clears it. Both keep history within each trial, isolating the effect of experience from earlier attempts. TXL-1 and TXL-6 observe one and six rope points, respectively; an MLP with an eight-frame observation stack provides a non-recurrent reference. The videos show individual examples, while the figures summarise results across ropes.
Rope_Swing
one arm, sustained rotation · same command, two ropesPrior experience helps establish the target rotation.
The video shows the same policy and speed command with two ropes: one softer (bending stiffness 2.1), the other stiffer (21.1). Without receiving either rope's physical parameters, the policy settles into different arm motions in response to their dynamics.
Across 384 ropes and trials 2–5, retaining context reduces TXL-1's mean capped acquisition time from 7.39 s to 4.89 s and raises success from 48.2% to 79.8%. TXL-6 improves from 5.64 s to 4.34 s. The MLP results overlap because it carries no state between trials.
Rope_Twirl
two arms, sustained rotation · trial 2 with memory cleared vs. keptA second attempt, with and without prior experience.
Both panels show trial 2 with the same rope, starting state and noise. On the left, context from trial 1 is cleared; on the right, it is retained. In this example, retaining context reduces acquisition time from 3.57 s to 2.37 s. The accompanying figure shows how the effect varies across the evaluation set.
The benefit depends on rope stiffness and the available observations. With one observed rope point, the largest gains occur for low and medium stiffness (1.55 s and 1.19 s faster). With six points, the largest gain occurs for high stiffness (0.79 s faster). Which architecture performs best also changes with stiffness.
Rope_Whip
one arm, transient sweep · five attempts at the same line
Repeated attempts bring the tip closer to the target line.
This example follows five attempts with the same rope, target line and initial state, retaining context throughout. Maximum deviation falls from 7.8 cm to 4.9, 3.8, 3.1 and 2.9 cm. The first attempt exceeds the 5 cm threshold; the next four stay within it.
The figure compares trial 1 (grey) and trial 2 (orange) for the same ropes at three selected target angles. Mean deviation is lower in trial 2 at each angle shown, with the largest decrease at 60°. Target-length coverage remains above 95% in both trials.
Across all three tasks, retaining context improves aggregate performance. The size of the benefit varies with rope dynamics and the observations available to the policy; individual attempts do not always improve.
From simulation to unfamiliar physical ropes
Each task runs on a Unitree H1-2 with a frozen TXL-1 policy at 30 Hz, using physical ropes unseen during training.
One tracked point
Two ZED 2i cameras track a single marker at the rope tip for Swing and Whip, or at the midpoint for Twirl. Its triangulated position provides the rope observation.
Three consecutive attempts
Each block begins with empty context, which is retained through trials 2 and 3. Between attempts, the policy pauses while the rope is returned to rest.
Rope_Swing
Rope_Twirl
Rope_Whip
Citation details will be available with the paper.