Improvement: analyzing rowing technique with AI

Post 3 of the ergy.coach build-in-public series.

I have my numbers from the last post. In rowing, technique decides the rest. You can pull hard and lose half your power to a sloppy sequence. I wanted to see my stroke, not only my meters. So one evening I built a proof of concept.

A micro web app, fully local

I load a side-view video of my session and get a technique read, all in the browser with nothing to install.

MediaPipe runs the pose estimation and tracks my skeleton frame by frame: shoulder, elbow, wrist, hip, knee, ankle. Measured angles sit on top of the video for knee, trunk, and elbow, live. The app finds the catch and the finish from the knee-angle extremes and cuts each stroke. From there it computes real metrics: drive-to-recovery ratio, layback at the finish, lean at the catch, and a seat-versus-handle check for the dreaded “shooting the slide.” It also exports the annotated video with the skeleton and markers baked in, so I can rewatch the stroke.

The code measures, the agent coaches

Hard-coded thresholds like “if ratio < 1.6 then fault” break, and they fit one case. So the code computes deterministic features, and an AI agent reads them like a coach, against the norms of the sport, anchored to the numbers. The camera measures; the agent interprets.

The validation, and a lesson in humility

To find out whether it was worth anything, I tested it against a video that real coaches had annotated. The tool landed on the same faults they did: recovery too fast, catch too upright. That surprised me.

It also caught me out. At one point the AI and I over-read a weak signal and invented a fault the numbers didn’t support. So I gave the agent a rule: anchor every verdict to a measured number, invent nothing. The human in the loop earns its place by catching confident hallucinations.

I added a camera-tilt calibration too. It came back near zero, about 1.7 degrees, so I knew my faults were real and not measurement artifacts.

What’s next

I have three pieces now: the conversational plan from post one, the session data from post two, and the technique read from this one. Next I connect them: one coach that reads my numbers, my notes, and my stroke, and pushes me toward the sub-8:00 2k faster than I’d manage alone.

Now the three pieces come together, and that’s the part I want to build most.

The running case study in this series is ergy.coach — an AI coach I’m building in public while chasing a sub-8:00 2k on my living-room erg.