Why fit prediction still stalls at the shoulder

Ask a tailor where a jacket is decided and the answer is the shoulder. The chest can be let out, the waist can be taken in, the sleeve can be shortened and the hem can be raised. The shoulder is the one line that cannot be moved far without rebuilding the garment around it, because it is the seam the whole weight of the cloth hangs from. Get it wrong and the error does not stay local: the sleeve twists, the collar stands away from the neck, and a drag line runs diagonally across the back that no amount of adjustment elsewhere will remove.

This is precisely the measurement that automated fit systems handle worst, and the reason is not a shortage of computing power. It is that the shoulder is hard to define, hard to see, and hard to write down in the format the rest of the pipeline expects.

Three measurements wearing one name

Start with the definition. A shoulder measurement can mean the distance across the back between the two points where the arm meets the torso. It can mean the length of one shoulder from the base of the neck out to that same point. It can mean the slope of that line away from horizontal, expressed as an angle. Pattern drafting uses all three, and they are not interchangeable. Two bodies can share an identical across-back figure and need different jackets because one has a square shoulder and the other a sloping one.

Most size charts record at most the first of these, and many record none at all. A chart that gives chest, waist, hip and inside leg is describing a body as a set of circumferences and one length, which is a reasonable summary of a trouser and a poor summary of a coat. When a model is trained on the numbers such charts contain, it is not that the model fails to learn shoulder slope; it is that nobody ever gave it the column.

The point the camera cannot find

The second problem is perceptual. The acromion, the bony point at the outer end of the shoulder that tailors work from, is a landmark you can feel through the skin with a fingertip. It is not reliably a landmark you can see. Under a shirt it is smoothed over by the deltoid; under a coat it is invisible; in a photograph taken from the front it is foreshortened, and in a photograph taken at three-quarters it moves relative to everything else in the frame.

Pose estimation systems, which is what most garment pipelines lean on for body structure, generally emit a shoulder keypoint. It is worth being precise about what that keypoint is. It is a learned estimate of a joint centre, trained on human annotators clicking on images. Annotators click where the shoulder looks like it is, which is usually somewhere in the mass of the deltoid rather than on the bony point a tailor would use. The keypoint is consistent, and it is useful for describing posture. It is not the tailoring landmark, and treating it as one imports a systematic offset that varies with build.

The keypoint is consistent, and it is useful for describing posture. It is not the tailoring landmark.

Error that will not stay put

The third problem is the one that turns a small inaccuracy into a visible failure. Shoulder error propagates. If the shoulder point is placed a centimetre too far out, the sleeve length measured from it is a centimetre short, the armhole is drafted larger than it should be, and the sleeve cap is set into a hole it does not match. A system that reports each of these as an independent measurement, each within tolerance on its own, can still produce a garment recommendation that is wrong in a way anyone would notice in a mirror.

This is why aggregate accuracy figures on fit tasks can be misleading. A model can score well on the average of many measurements while being unreliable on the one measurement that structures the rest. Averaging treats the shoulder as one input among a dozen. The garment does not.

What would actually help

None of this argues that the problem is unsolvable. It argues that the effort is better spent on representation than on model capacity. A pipeline that recorded shoulder slope as an angle rather than inferring it from two point positions would lose less information. A pipeline that reported its shoulder estimate with an explicit uncertainty, rather than a single number, would let the recommendation downstream be appropriately cautious. And a pipeline evaluated on the shoulder specifically, rather than on a mean across measurements, would at least make the weakness visible to the people building it.

The pattern is familiar from other applied vision work. The hard part is rarely the model. It is admitting that the quantity everyone has been recording is not quite the quantity that matters, and then going back to change what gets written down.