Partner & R&D Director @ Manifold AI流形空间(Manifold AI)技术合伙人、研发总监
Explore the world as well as our inner heart探索世界,也探索我们的内心
A gentle black sheep. I hold warmth toward others while remaining inwardly independent. I'm drawn to clear facts, vivid details, well-reasoned arguments, measured expression paired with a kind heart. I hold no claim to absolute truth, and have little patience for context-free moralizing.
I am inclined toward order: knowledge should be hierarchical, data should be structured, and cognition is the continual process of organizing both data and knowledge. Only human beings are equal by birth; at the level of all living things, there is no place for hierarchy.一只羊,保持对他人的尊重和善意,内心独立有态度。喜欢清晰的事实,生动的细节,有条理的逻辑,妥善的表达和善良的心灵;知自己没有掌握真理,也厌烦被脱离场景的真理训诫。
2026-08Joined Manifold AI as Partner & R&D Director, working on world models and embodied manipulation.加入流形空间(Manifold AI),任技术合伙人、研发总监,聚焦世界模型与具身操作。
2026-06One paper accepted to ECCV 2026: GTA-VLA (Guide, Think, Act)!一篇论文被 ECCV 2026 录用:GTA-VLA(Guide, Think, Act)!
2026-04-27Keynote talk on SpatialPoint at the Guangdong Provincial AI Application Matchmaking Conference (coverage).在广东省人工智能应用对接大会作 SpatialPoint 主题演讲(报道)。
2025-07One paper accepted to ICCV 2025: UniGS (UniGS: Modeling Unitary 3D Gaussians for Novel View Synthesis from Sparse-view Images)!一篇论文被 ICCV 2025 录用:UniGS(UniGS: Modeling Unitary 3D Gaussians for Novel View Synthesis from Sparse-view Images)!
2025-06One paper accepted to IJCAI 2025: SeqPose!一篇论文被IJCAI 2025录用:SeqPose!
2025-02Three papers accepted to CVPR 2025: LeanGaussian, HandOS, HumanMM!三篇论文被CVPR 2025录用:LeanGaussian、HandOS、HumanMM!
2024-09Geo6D accepted to IEEE Transactions on Multimedia (TMM).Geo6D 被 IEEE Transactions on Multimedia (TMM) 录用。
2023-01Uni6Dv2 accepted to AISTATS 2023.Uni6Dv2 被 AISTATS 2023 录用。
How to Deliver a VLA System: A Small-Parts Picking Case
Start with delivery goals, SOP design and hardware, then close the loop across data, policy, control and real-robot feedback. Every step is illustrated with a field project in which a single arm picks small parts from a cluttered bin onto a conveyor.
00 / The case: single-arm small-parts picking
A single arm picks small boxes one by one from a cluttered bin and places them on a conveyor. The policy is a π0.5-class vision-language-action model, bootstrapped from teleoperated SOP data collected at another site, then extended with data from the target site and deployed on an on-site edge GPU. It took 26 days from the first field test to a demonstration the customer accepted. Every "small-parts picking example" below comes from this project.
Metric
Result
Peak throughput on site
680 pieces/hour
Sustained throughput
420 pieces/hour
Single-grasp success in a cluttered bin
41% → 75–94%
Server-side inference per request
165 → 42 ms (48.5 ms measured on the robot)
Internal star rating
1 → 4.3 stars
Training / field evaluation
104 models in 25 days; 30 field evaluations in 8 days
01 / Define delivery goals
VLA (Vision-Language-Action) delivery usually targets a specific operational setting. Object, destination and workstation priors can support faster iteration through post-training and reinforcement learning, with different priorities from pre-training and mid-training.
Delivery requires sustained operation on site, with agreed success rate, cycle time, accuracy and recovery behavior. Confirm these goals first, then design the standard operating procedure (SOP) and select hardware. Data collection follows those decisions.
Define task boundaries: objects, reachable locations, allowed environmental changes and cases requiring human assistance.
Evaluate success rate together with cycle time, and agree on test conditions and failure criteria. Reconcile differences across development, collection and delivery sites.
02 / Validate hardware across the workspace
Reach and gripper: cover all operating locations; check opening width, closing speed and grasp stability on irregular objects. A reading at the gripper base may differ from the actual fingertip gap.
Edges and corners: check whether the gripper or wrist camera collides with the bin while reaching boundary objects.
Vision and touch: the main camera must cover the scene without critical occlusion by the end effector. Check whether tactile signals distinguish aligned and misaligned contact.
Check mechanics and sensing early: loose grippers, motor power loss, motion blur, low camera frame rate and duplicate frames can all appear as policy failures.
03 / SOP design is the critical step
Simplify the task physically first: a ramp may assist flipping, or a two-handed operation may become a one-handed one. Simpler procedures with distinguishable states are easier to learn and debug.
Keep the robot, table and bin layout reproducible in simulation. Vary bin position within an allowed range during collection to cover deployment tolerances. A compliant layer at the bin bottom can reduce impact.
Define ideal, routine and recovery procedures separately. Reproduce typical errors through teleoperation and collect recovery actions for states such as missed grasps and slips.
Reduce accidental contact, target switching, hesitation and unnecessary object adjustments. Small demonstration habits can become policy behavior.
Check ambiguous stages: returning and approaching, or lifting after successful and failed grasps, may look alike in a single frame. An action classifier using images, gripper width and joint angles can expose confusion between stages.
The person training the policy should collect at least ten demonstrations; key collaborators should also participate to develop a shared understanding of the procedure.
04 / Put guardrails in the execution loop
Detect bottom contact using the physical fingertip geometry, such as corners or edges, rather than only the end-effector reference position.
At a depth limit, or when downward commands persist without corresponding motion, inhibit further descent while preserving permitted gripper and retreat actions. A 5 cm upward retreat is one example where clearance permits; actual thresholds depend on tooling, objects and clearance.
Six-axis force sensing requires calibration and compensation for gravity, acceleration and payload before it can reliably inform control.
05 / Collect clean demonstrations and recovery data
Start each demonstration from the agreed initial state. Remove takeover jitter and observation periods; execute decisively and keep relevant objects visible.
Distinguish task, sub-task and action/skill labels, with environmental context for special actions. Brief pauses can mark important skill boundaries and should not be removed indiscriminately.
Remove irrelevant stationary frames and anomalous spikes, with action smoothing where appropriate. For imitation learning, retain correct behavior and recovery while excluding the preceding erroneous actions.
06 / Focus training and offline evaluation on critical stages
Augment for image changes caused by robot motion so the policy does not rely solely on smooth teleoperation footage.
Use counterfactual samples to assess instruction following. Annotate special SOP branches with scene and action descriptions, and consider action and progress prediction to distinguish stages.
Weight critical frames and skills more heavily in training and evaluation, and consider end-effector auxiliary heads and losses. Inspect grasping, contact and recovery in addition to aggregate error.
07 / Turn policy actions into smooth robot control
Start with stepwise execution to assess targeting accuracy, grasp stability and recovery separately. Then inspect inference latency, trajectory interpolation and low-level control before attributing every failure to the policy.
Consider a 30 Hz action chunk feeding a 500 Hz controller. Interpolation bridges the rates; quintic polynomials can provide continuous position, velocity and acceleration. These rates are examples and must match the robot interface.
An overly stiff position controller can amplify jitter. Tune PID for the platform and evaluate hybrid force/position control when supported.
08 / Combine the policy with smooth rule-based handoffs
After a successful grasp and lift, rules can handle deterministic carrying, placement and return motions, leaving the policy to manage complex interactions.
Maintain smooth velocity and acceleration across handoff, obstacle-avoidance and placement points. Avoid abrupt stops or jumps when switching between rules and the policy. Placement can vary within the allowed region.
Validate grasp retention separately. Additional closing actions after grasping may reduce slips; force and repetition must suit the object and gripper.
09 / Close the loop with real-robot feedback
Test out-of-distribution states such as empty bins, absent targets and missing wrist-camera views, with defined stop, recovery or assistance behavior.
Keep human feedback for each real-robot evaluation. Distinguish targeting errors, unstable grasps, failed recovery, control jitter and hardware faults before choosing what to change.
Feed real-robot data back into iteration, especially jitter, blur and recovery cases. Deployment imagery differs from demonstrations and needs coverage in subsequent collection and training.
10 / Before handover
Goals agreed: task scope, success rate, cycle time, test conditions and human-assistance boundaries are defined.
Procedure validated: hardware reaches required locations, cameras provide sufficient coverage, and routine and recovery procedures have data and real-robot validation.
Operation supported: guardrails work, handoffs are smooth, exceptional states have defined handling and known hardware faults can be diagnosed.
Feedback actionable: site issues can be diagnosed, and data plus human feedback feed the next iteration. Detection, tracking and segmentation may supplement perception; check how their outputs differ under occlusion.
VLA Data Preprocessing: An Eight-Layer Pipeline (L0–L7) for Small-Parts Picking
How a continuous bin-clearing recording becomes training samples: signal checks, cycle and core-segment extraction, pose-based target admission, 3D trajectory comparison of data sources, failure classes, skill labels, training weights, pause removal and speed normalization. Each layer comes with its rules, thresholds and measured results on about twenty million frames.
01 / Why data processing sets the ceiling
The data sets the ceiling on what a policy can learn. In the small-parts picking project, problem after problem traced back to data. The head camera actually ran at 12.5 fps; in one bin-clearing collection the left and right wrist cameras were swapped at the recorder; different uses of failure-recovery data moved offline error anywhere from −61% to +30%; and training on returning and initial approaching together confused the policy. Every recording therefore passes through the same processing pipeline before it reaches training.
The three principles below come from the data-quality section of the delivery article; the eight layers turn them into executable, measurable rules:
Start each demonstration from the agreed initial state. Remove takeover jitter and observation periods; execute decisively and keep relevant objects visible.
Distinguish task, sub-task and action/skill labels, with environmental context for special actions. Brief pauses can mark important skill boundaries and should not be removed indiscriminately.
Remove irrelevant stationary frames and anomalous spikes, with action smoothing where appropriate. For imitation learning, retain correct behavior and recovery while excluding the preceding erroneous actions.
02 / Overview: from one recording to training samples
Data is collected by continuous bin clearing, with many picks per recording, mixing successes, empty grasps, touches, recoveries and pauses. The pipeline first runs the L0 signal check, then seven layers, L1–L7, cut, classify, weight and align the recording. Left and right arms are processed independently. Every layer only writes labels and never edits raw data, so any layer's rules can be changed and recomputed on their own.
Layer
Unit
What it does
What it solves
L0 signal check
Each recording and arm
Joint jumps, FK consistency, gripper reading vs command, timestamps, camera sync
Screening: faulty segments get zero training weight
L1.1 full cycle
Cycle
HOME → grasp → lift → place → HOME
Non-critical phases come only from the target scene
L1.2 core segment
Cycle
10 cm above the object → grasp → lift to 5 cm above it
Borrow only the hardest, most valuable segment
L1.3 target / auxiliary
Core segment
Compare joint poses at three moments with the target scene
Admission rule for multi-source data
L2 attempts
Grasp attempt
Every descent to grasp height counts, closed or not
Find empty grasps, touches and other failed attempts
L3 outcomes
Attempt
Success or failure classes F1–F14
Do not learn failures; do learn recoveries
L4 skills
Frame
Three stages (fall / grasp / raise) plus 12 skills
Stage prompts and fine-grained data analysis
L5 weights
Frame
Train 0/1; contact window weighted ×2
Combine everything into training weights
L6 pause removal
Frame sequence
Cut pauses, inject bridging actions at the seams
Operators' pause habits
L7 speed normalization
Segment
Align free-motion speed to the benchmark operator; contact window unchanged
Speed differences between operators
03 / L0 signal check: remove the obviously broken first
Following the state-action filtering in Qwen's robot-manipulation work, each arm of each recording first passes L0. Failing segments, or whole recordings, are marked qa_fail and get zero training weight. The thresholds are initial values, to be revised from the distribution after a full pass.
Across 9,328 files and 16,688 single-arm recordings, 2,556 contained faulty segments, mainly timestamp gaps (2,412) and gripper readings that did not follow commands (115).
Check
Rule
Initial threshold
Joint jumps
Sudden rise in frame-to-frame joint change or its second difference
> 0.2 rad in one frame, or > 3× the joint's p99.9
Extremes / limit violations
Joints beyond URDF limits or outside the recording's p0.1–p99.9
Flag on violation
FK consistency
FK from recorded joints vs the recorded end-effector pose
> 1–2 cm or > 5°
Gripper reading vs command
Measured width must drop after ≥ 1 s of a close command
04 / L1 cycles and core segments: the grasp is the hard part
A full cycle runs HOME → grasp → lift → place → HOME and is kept only for the target scene. The core segment runs from 10 cm above the object down to the grasp and up to 5 cm above the object, with the end-effector height at closing as the object reference. It is the hardest and most valuable part of the motion. When borrowing data from other SOPs or scenes, borrow only this segment; the easy phases (leaving home, carrying, placing, returning) come only from the target scene.
Before the definitions were unified, each pipeline had its own rules, and frames called focus or lift meant different things in different datasets. In one, focus began about 1.7 cm above the grasp point and lift ended near 15 cm; in another, focus began near 3.6 cm and lift ended at about 4.1 cm. Under the new definition, 95% of 11,329 valid closes in 2,582 recordings have both anchors, 10 cm above and lifted 5 cm. The rest start from the highest point since the previous opening. For side picks, the start is 10 cm from the grasp point instead.
Source
10 cm → close
Close → lifted 5 cm
Core total
Target site (benchmark operator)
2.43 s
0.53 s
3.03 s
SOP from another site
2.60 s
0.60 s
3.23 s
Stacking scene
2.92 s
1.00 s
4.03 s
Hard cases (crowded bin)
3.70 s
0.67 s
4.45 s
Failure recovery
4.33 s
1.03 s
5.43 s
05 / L1.3 target or auxiliary: decide by pose, not by source
One more split on the core segment: is this grasp the same move as in the target scene? The rule compares the seven joint angles at three moments, HOME, grasp and lifted 5 cm, with the nearest target-scene core. Each distance must be within the p95 of the target scene's own leave-one-out nearest-neighbour distance (HOME 0.092 rad, grasp 0.243 rad, lift 0.227 rad). Passing cores train under their subtype (normal grasp, recovery after failure, field RECAP data and so on); the rest are marked auxiliary. Recovery cores start where the failure happened, so HOME is not compared for them.
The target site itself has 88% within range at all three moments, consistent with radii meant to cover about 80–90% of samples.
The SOP from another site is nominally the same task, yet only 2% qualify. Its HOME joints are offset by 0.1–0.23 rad and tightly clustered (a different start pose), and wrist roll at the grasp differs by about 0.3 rad. By joint angles, it is not the same move.
Multi-item small-parts data and dual-arm data are close to 0%. They are still useful, but only as auxiliary data, not mixed in as target data.
Source
Core segments
HOME in range
Grasp in range
Lift in range
Labelled target
Target site
735
95%
95%
95%
646 (88%)
SOP from another site
802
36%
16%
8%
16 (2%)
Failure recovery
1320
50%
22%
20%
169 (13%, HOME not compared)
Hard cases (crowded bin)
475
43%
11%
7%
0 (auxiliary by source)
Stacking scene
1368
78%
30%
30%
0 (auxiliary by source)
Dual-arm recordings
11437
4%
1%
2%
0
Multi-item small parts
9243
0%
0%
0%
0
06 / 3D trajectory comparison (1): how far grasp poses are from the target
Some sources train well and some do not, and the core test is to compare 3D trajectories. Start with the moment of grasping. At the frame where the gripper starts closing, take the seven joint angles, the end-effector pose in the torso frame and the torso state, and compare each source with the target site's set of grasps. In region means the distance to the nearest target grasp is within the target's own leave-one-out p95 (0.25 rad in joints, 18 mm in position). Orientation difference is the rotation to the positionally nearest target grasp.
Source
Grasps
Joint NN (median rad)
Joints in region
Position in region
Position NN (mm)
Orientation diff
Grasp height z (mm)
Target site (reference)
633
0.16
95%
95%
9
13°
−180
SOP from another site
653
0.31
18%
18%
37
20°
−90
Failure recovery
993
0.30
27%
64%
14
19°
−133
Rotate-and-place (earlier project)
192
0.32
15%
61%
14
22°
−132
Hard cases (crowded bin)
327
0.35
17%
44%
20
22°
−115
Left arm of dual-arm recordings
4929
0.47
3%
8%
62
37°
−92
Right-arm data from another scene
643
0.76
0%
0%
219
43°
−192
Multi-item small parts
2077
0.87
0%
0%
118
57°
−321
Newly collected small-part clearing
769
0.32
33%
47%
19
30°
−178
07 / 3D trajectory comparison (2): path shape, operator differences and selection
Next, compare whole core paths. With the grasp point as the origin, draw each grasp from 10 cm above the object to the 5 cm lift and compare top, side and front views plus a rotatable 3D view. Even within one scene, operators differ a lot. A smooth grasp has a single extremum near the grasp point; others adjust back and forth on the way down, overshooting and backtracking, with several extrema in mid-air.
Speed: measured against the target site's benchmark operator, other operators' free-motion segments need a 1.1× to 3× speed-up to match.
Pauses: some operators stop visibly in mid-air, others move in one continuous stroke.
Extrema: 1.2 per grasp at the target site, with none in 38% of grasps; 1.6 for the other site's SOP; 3.0 for multi-item small parts; 4.6 for dual-arm recordings.
Selection: L1.3 pose regions decide target versus auxiliary; speed (L7) and pauses (L6) are then aligned. Multiple extrema and overshoot are only measured so far and are the next layer to add.
De-jittering: teleoperation data carries a roughly 10 Hz component during motion. Adaptive DCT smoothing on 200 SOP recordings removed the 7–13 Hz jitter, kept 0–3 Hz motion intact and cut jerk by 95%; it has not yet been validated in training.
Side view (forward vs height, mm). Up to 14 random grasps per source, drawn from 10 cm above the object to the 5 cm lift; the red dot is the grasp point. The target site's benchmark operator (A) has the cleanest paths; C and D overshoot and backtrack, with many mid-air extrema.
08 / L2–L3 grasp attempts and failure classes: F1–F14
Every descent to grasp height within a cycle counts as an attempt, whether or not the gripper closes. Each attempt ends as a success or one of 14 failure classes. Classes that can be measured reliably are handled now; the rest are marked seen but unprocessed, with zero training weight. F14 is the catch-all for highly suspicious cases that cannot be measured or classified.
All data together has 38,045 attempts: 24,712 successes, 5,255 regrasps before lift (F9), 3,632 finger touches (F2), 2,412 aborted descents (F1), 780 other suspicious (F14), 473 empty closes (F5), 342 edge grasps (F7) and 322 drops during lift (F10). Without force sensing, some F1/F2 cases may just be re-alignment without contact; the rise that follows them is trained as recovery.
Class
Name
Rule
Handling
F1
Aborted descent
Rises near grasp height without closing
Not yet handled
F2
Finger touch
Local low point near grasp height, rebound ≥ 0.5 cm, no close
Handled
F3
Push / knock
Tracked object moves > 0.35 object lengths or > 10° before closing
Handled (with object tracking)
F4
Wrong target
Aligned object ≠ the object finally picked
Not yet handled
F5
Empty close
Minimum width ≤ 20 mm while closing
Handled
F6
Partial close
Minimum width 42–48 mm
Not yet handled → F14
F7
Edge grasp
Width 6–13 mm while moving ≥ 20 cm
Handled
F8
Misplaced close
Closing away from any object (needs tracking)
Not yet handled
F9
Regrasp before lift
More than one close before lifting, including grip-drag-regrip
Handled: drag not trained, train from the last grasp
F10
Drop during lift
Width < 5 mm while lifting
Handled
F11
Drop while carrying
Held once, falls below 5 mm before the release command
Handled
F12
Carrying with an empty gripper
Never held but moves ≥ 20 cm
Handled
F13
Collision
Arm or object hits the bin or other objects (needs tracking or force)
Not yet handled
F14
Other suspicious
Highly suspicious but unmeasurable or unclassifiable
Zero weight, pending human review
09 / L4 skill labels: 12 skills and 3 stages
A pick-and-place cycle contains many kinds of motion. Each frame gets one of 12 skills by what the motion does, and the skills merge into three stages (fall, grasp, raise) used as stage prompts. Under the new definitions, focus runs from about 10 cm above the grasp point to the close command, lift ends at about 5 cm, and the rest until release is carry. Data with teleoperation gripper commands is segmented by rules; data without them uses a wrist detector's skill head per frame plus smoothing, and the two are reconciled where both exist.
How much pausing: 24–25% of frames are wait at the target site and in the other site's SOP, only 6% for multi-item small parts.
How long alignment takes: focus is 17% of frames at the target site versus 24–31% for stacking and dual-arm data, which spend much longer adjusting above the object.
Where failures and recoveries are: in failure-recovery data 2.7% of frames are touch, 1.4% lift_recovery and 12% other; at the target site these are almost zero.
How rare the key action is: actual closing (close_gripper) is only 3% of all frames, which is why the core segment is weighted rather than drowned out by other phases.
Stage
Skill
Meaning
fall
approach
Descend toward the object from above
fall
focus
Align and fine-tune above the object (from about 10 cm)
grasp
close_gripper
Close the gripper
raise
lift
Lift after grasping (to about 5 cm)
raise
lift_recovery
Lift after a failure, then start again
—
touch
Failure: fingers touch the object or bin without grasping
—
carry
Carry to the placement point
—
open_gripper
Open the gripper to release
—
return
Return home (label kept, normally not trained)
—
wait
Stop and wait
—
global_observe
Observe the scene near home
—
other
None of the above, including empty closes and other failures
10 / L5 what to train and how much
L5 combines the earlier layers into per-frame training switches and weights. Weights apply to each step of an action chunk; steps that land on failure frames get zero weight. Among public work, GiB likewise zeroes the imitation loss on bad segments, while π*0.6's RECAP keeps them and assigns low advantage labels instead, which is a possible second approach.
Segment
Train
Weight
Core segment of a successful attempt
1
1
Contact window (45 frames before to 5 after contact)
1
2
Recovery after a failure (also in truncated cycles)
Rest of the full cycle outside the core (excluding return)
1 in the target scene, 0 elsewhere
1
return
0
0 (label kept)
Unfinished cycle at the end of a recording
0
0
11 / L6 cut pauses and inject bridging actions
Measured on 2,876 recordings, pauses inside core segments last a median 8 frames (30 Hz; p90 22 frames), and the median gap between neighbouring pauses is 32 frames, with 14% of gaps at 10 frames or less, so clusters of close pauses are real. After a pause is cut, a bridging action at the seam uses the mean velocity before and after, with cubic Hermite interpolation. Injected frames serve only as action targets, never as sample starts, because they have no images.
Public practice differs. openpi, on DROID, neither splices nor interpolates; it only forbids sample starts inside idle segments and about half a chunk before them, and chunks still read the raw future. That is simpler and risk-free, but pauses remain inside chunks, so it is a good control experiment for L6.
Parameter
Value
Reason
Shortest pause to cut
≥ 6 frames (0.2 s)
Shorter pauses are hand tremor; cutting them breaks continuity
Velocity window k
5 frames
Matches the ±5-frame idle rule; with a 32-frame median gap, both sides fit
Merging
Cuts less than 10 frames apart merge, with one injection
Otherwise the velocity window reaches into another pause
Injected velocity
Mean of before and after, cubic Hermite interpolation
Keeps velocity continuous at the seam
Contact window
Never cut, never merged across
Small pauses around contact can be skill boundaries
12 / L7 normalize speed to the benchmark operator
For each operator, take the median duration of core segments (10 cm → grasp → 5 cm). The target site's benchmark operator is the fastest, yet the same person is 45% slower in another site's SOP, so the target speed is the operator's target-site core segments rather than anything tied to a name. Scene difficulty matters too; one operator takes 3.23 s in the SOP but 4.45 s on hard cases. Ratios are therefore compared within a scene, and only free-motion segments are aligned across scenes.
Public work agrees. DemoSpeedup accelerates only free motion, and uniform 2× hurts success; ESPADA accelerates only casual segments; SAIL gains a lot on real robots except in contact-rich tasks; TempoVLA conditions on speed as an input.
Split: the core segment divides into free-motion segments (10 cm → start of contact window, end of contact window → 5 cm) and the contact window.
Ratio: r = segment duration / the benchmark operator's median for the same segment.
Speed up only: free segments with r > 1 are subsampled by r, images and actions together, so no images need to be synthesized; r ≤ 1 is left alone; the contact window stays at 1×.
Order: L6 pause removal before L7, otherwise pauses inflate r.
Operator
Scene
Grasps
Core total
Relative to benchmark
A (benchmark)
Target site
766
3.03 s
1.00×
A
SOP from another site
109
4.40 s
1.45×
B
SOP from another site
745
3.23 s
1.07×
B
Stacking scene
1230
4.03 s
1.33×
B
Hard cases
528
4.45 s
1.47×
B
Failure recovery
549
5.43 s
1.79×
C
Dual-arm recordings
3465
3.73 s
1.23×
D
Dual-arm recordings
2911
4.37 s
1.44×
E
Dual-arm recordings
296
5.90 s
1.95×
13 / Results of the full pipeline on all data
L0–L7 ran on all pick-and-place data, 9,328 recordings, 16,688 active arms and about 20.49 million frames, in roughly two minutes on a single process. It only writes labels and leaves training files untouched. Recovery after failure is always trained (4,410 segments; previously 46% had been zeroed by neighbouring failure segments). L6 never cuts the contact window, so all 1.248 million contact frames are untouched. Core frames shrink by about 31% after L6 and L7, but only 10% at the target site, which was already fast.
Source
Core frames
After L6
After L7
Total core reduction
All
3136348
2823916
2171569
31%
Small parts in a new scene (multi-source)
1054743
918437
726408
31%
Target site
68306
67274
61189
10%
SOP from another site
85981
82662
69173
20%
Hard cases
68364
61036
42997
37%
Stacking scene
186971
174541
120294
36%
Dual-arm recordings
1498568
1361049
1039143
31%
Failure recovery
173415
158917
112365
35%
14 / Takeaways and what is still missing
Screen before use: signal, gripper and camera problems are caught once in L0 and checked automatically on every sync. Problems where the stream is alive but its identity is wrong are found too late by eye.
Multi-source data needs an admission rule: compare poses with the target scene (L1.3) instead of accepting or rejecting whole sources; a nominally identical task may qualify only 2% of the time.
Borrow only the core: other sources contribute only the 10 cm → grasp → 5 cm segment, with the contact window weighted, never cut and never sped up.
3D trajectory comparison is the core tool for judging data. Joints and pose at the grasp tell whether it is the same move, extrema and backtracking along the path tell how clean the demonstration is, and the comparison can expose root causes such as collection setup.
Normalization comes before mixing: recompute normalization statistics for every data mix; in time, L6 removes pauses and L7 aligns speed; in space, smoothing and alignment start with the core segment, anchored at the grasp point and the two heights.
Still missing: cycle-level quality scores and deduplication; train/validation splits by operator or session (random splits by recording put similar cycles on both sides); balanced sampling by operator or site; and a layer that handles multiple extrema and overshoot in trajectories.
@article{coca-splat2025,
title={Coca-Splat: Collaborative Optimization for Camera Parameters and 3D Gaussians},
author={Jiamin Wu, Hongyang Li, Xiaoke Jiang, Yuan Yao, Lei Zhang},
journal={arXiv 2025},
year={2025}
}
@article{leangaussian2025,
title={LeanGaussian: Breaking Pixel or Point Cloud Correspondence in Modeling 3D Gaussians},
author={Jiamin Wu, Kenkun Liu, Han Gao, Xiaoke Jiang, Yuan Yao, Lei Zhang},
journal={Conference on Computer Vision and Pattern Recognition (CVPR) 2025},
year={2025}
}
@article{handos2025,
title={HandOS: 3D Hand Reconstruction in One Stage},
author={Xingyu Chen, Zhuheng Song, Xiaoke Jiang, Yaoqing Hu, Junzhi Yu, Lei Zhang},
journal={Conference on Computer Vision and Pattern Recognition (CVPR) 2025},
year={2025}
}
@article{ssn3d2021,
title={SSN3D: Self-Separated Network to Align Parts for 3D Convolution in Video Person Re-Identification},
author={Xiaoke Jiang, Yu Qiao, Junjie Yan, Qichen Li, Wanrong Zheng, Dapeng Chen},
journal={AAAI Conference on Artificial Intelligence (AAAI) 2021},
year={2021}
}
@article{atsra2019,
title={ATSRA: An Accelerated Transmission Strategy Based on Request Aggregation in NDN},
author={Shanshan Shi, Jun Li, Haibo Wu, Yongmao Ren, Xiaoke Jiang},
journal={IEEE International Conference on Computer Communications (INFOCOM) 2019 (poster)},
year={2019}
}
@article{cacheanddeliveryofvrvideoovernameddatanetworking2018,
title={Cache and delivery of VR video over named data networking},
author={Yi Zhang, Xiaoke Jiang, Yi Wang, Kai Lei},
journal={IEEE INFOCOM Workshops 2018},
year={2018}
}
@article{whatbenefitsdoesndnhaveinsupportingmobility2014,
title={What Benefits Does NDN Have in Supporting Mobility},
author={Xiaoke Jiang, Jun Bi, You Wang},
journal={IEEE Symposium on Computers and Communications (ISCC) 2014},
year={2014}
}
@article{mcbs2014,
title={MCBS: Matrix Computation Based Simulator of NDN},
author={Xiaoke Jiang, Jun Bi, You Wang},
journal={Journal of Computers, Vol.9, No.9, 2014},
year={2014}
}
@article{interestsetmechanismtoimprovethetransportofnameddatanetworking2013,
title={Interest Set Mechanism to Improve the Transport of Named Data Networking},
author={Xiaoke Jiang, Jun Bi},
journal={ACM SIGCOMM Conference on Data Communication 2013 (poster)},
year={2013}
}
@article{msdn2013,
title={MSDN: A Mechanism for Scalable Intradomain Control Plane in SDN},
author={Pingping Lin, Jun Bi, Hongyu Hu, Xiaoke Jiang},
journal={Journal of Chinese Computer Systems, Vol.34, No.9, 2013},
year={2013}
}
@article{acontentprovidermobilitysolutionofnameddatanetworking2012,
title={A Content Provider Mobility Solution of Named Data Networking},
author={Xiaoke Jiang, Jun Bi, You Wang, Pingping Lin, Zhaogeng Li},
journal={IEEE International Conference on Network Protocols (ICNP) 2012},
year={2012}
}
@article{aneasymatrixcomputationbasedsimulatorofndn2012,
title={An Easy Matrix Computation based Simulator of NDN},
author={Xiaoke Jiang, Jun Bi, You Wang, Pingping Lin, Zhaogeng Li},
journal={IEEE International Conference on Networking and Distributed Computing (ICNDC) 2012},
year={2012}
}
@article{mobilitysupportintheinternetusingidentifiers2012,
title={Mobility Support in the Internet Using Identifiers},
author={You Wang, Jun Bi, Xiaoke Jiang},
journal={ACM International Conference on Future Internet Technologies (CFI) 2012},
year={2012}
}
@article{thecompressionofpendinginteresttablewithbloomfilterincontentcentricnetwork2012,
title={The Compression of Pending Interest Table with Bloom Filter in Content Centric Network},
author={Zhaogeng Li, Jun Bi, Sen Wang, Xiaoke Jiang},
journal={ACM International Conference on Future Internet Technologies (CFI) 2012},
year={2012}
}
@article{easytrace2011,
title={EasyTrace: Easily-Deployable Light-Weight IP Traceback on an AS-Level Overlay Network},
author={Hongcheng Tian, Jun Bi, Wei Zhang, Xiaoke Jiang},
journal={IEEE International Conference on Network Protocols (ICNP) 2011},
year={2011}
}
A Quick Survey on Selected Approaches for Preparing Programmable Networks
Pingping Lin, Jun Bi, Hongyu Hu, Tao Feng, Xiaoke Jiang
Asian Internet Engineering Conference (AINTEC) 2011
@article{aquicksurveyonselectedapproachesforpreparingprogrammablenetworks2011,
title={A Quick Survey on Selected Approaches for Preparing Programmable Networks},
author={Pingping Lin, Jun Bi, Hongyu Hu, Tao Feng, Xiaoke Jiang},
journal={Asian Internet Engineering Conference (AINTEC) 2011},
year={2011}
}
DINO-XGrasp
Open-Set Robotic Grasping powered by DINO-X
VLARobotOpen-Set DetectionPick&Place
Enabling robots to grasp any object accurately using consumer-level hardware (camera, GPU, robot arm), powered by open-set detection/segmentation, real-time tracking, and 3D ReID.
oVP uses Supervised Prompt Tuning (SPT) to generate optimized visual prompts that customize open-set detection for specialized scenarios. Unlike text prompts that rely on category names, SPT learns domain-specific prompt embeddings from as few as 7 labeled images. Only prompt embeddings are updated during training — no model fine-tuning needed. SPT+Grounding DINO 1.6 consistently outperforms YOLOv8 across 12 industry domains (agriculture, construction, manufacturing, retail, etc.), with advantages intensifying as training data decreases.
These are small utilities I built to scratch my own itches in research and daily life — fast to use, single-purpose, and open source. I keep them lean so they stay useful instead of accreting features, and I share them in case the same itch bites someone else.下面是我为科研和日常使用顺手做的几个小工具——专一、轻量、开源。 我刻意保持它们简单,避免功能膨胀,也希望对有同样需求的人有用。
TianYan (天眼)天眼 TianYan
Multi-format file & dataset previewer for the web多格式文件 / 数据集的网页预览器
WebFastAPIReactDataset PreviewerMulti-format
Why: Real research projects run on cloud GPUs, and the artifacts I actually want to look at — training datasets, model outputs, depth/mask predictions, HDF5 trajectories — live on those remote machines. SCP-ing files back is slow, Jupyter is heavyweight for "just let me see it", and OS file managers do not preview ML formats anyway. TianYan runs as a small web server next to the data: point a browser at the cloud machine and inspect any project's datasets and outputs through one unified, pluggable viewer.
Main features
Tree-style sidebar that recursively browses any local root directory
Why: Phones accumulate large datasets — photos, screen recordings, sensor logs — and every "easy" way to get them off the device has a catch: USB cabling is slow and needs a desktop client; mainstream file-transfer apps push you to a cloud upload, require an account login, or are smothered in ads. MobileEye flips the relationship: the phone itself runs a tiny LAN web server, and any laptop on the same Wi-Fi browses, streams, and bulk-uploads through a normal browser — no account, no ads, no cloud round-trip. It also speaks the same API as TianYan, so the same viewer UI works for both local disks and phones.
Main features
Runs entirely on the phone — no companion desktop app needed
TianYan-shaped API, so the same web frontend browses phone and laptop storage interchangeably
Range-aware video streaming directly from the phone's storage
Multi-GB chunked uploads that survive flaky Wi-Fi
Bundled minimal frontend ready out-of-the-box; works in any browser
Web UI (browsing phone storage from a laptop)网页端界面(在笔记本上浏览手机存储)
FolderPlayerFolderPlayer 文件夹播放器
Folder-scoped Android music player以文件夹为单位的 Android 音乐播放器
AndroidMusic PlayerMedia3ExoPlayerno-adno-login
Why: Mainstream Android music players are either ad-walled or push you into an account plus a cloud library, and they all force every track into a tag-based "library" that fights against the way I actually organize music — as folders on disk, by album / mood / workout. FolderPlayer treats a chosen folder (and its subfolders) as the playlist: pick a folder, hit play, done. No ads, no login, no library, no scan.
Main features
Folder = playlist: navigate the disk tree and tap to play a subtree
ID3 album-art rendering on the now-playing screen
Sleep timer for falling asleep to music
Shuffle within the current folder
Background playback with lock-screen / notification controls
Media3 / ExoPlayer engine: gapless playback and modern audio-focus handling
Open-set Grasping & VLA: Developing systems to enable robots to grasp any object using consumer-level hardware, powered by open-set detection/segmentation, real-time tracking, and 3D ReID