Robot Learns to Sew Fabric Like a Human Using Vision, Neural Networks and Force Control

Robots have conquered car assembly lines, electronics factories, and warehouse floors, but one stubbornly human skill has resisted automation for decades: sewing. The reason lies in the material itself. Fabric is flexible, lightweight, and highly deformable, changing shape under even the slightest touch, and different textiles behave in dramatically different ways depending on composition, thickness, and rigidity. Now, a team of researchers in Greece has built a robotic sewing system that combines computer vision, learning-based grasping, and force control to handle fabric with a level of autonomy that brings fully automated garment manufacturing a significant step closer to reality.

The research, published in the International Journal of Intelligent Robotics and Applications, presents an integrated framework developed by Nikolaos Anatoliotakis and Panagiotis Koustoumpardis of the University of Patras, together with Paraskevi Zacharia of the University of West Attica. What makes their approach remarkable is its economy of hardware: a single robotic arm works alongside an unmodified industrial sewing machine, performing every manipulation task from grasping and rotating the fabric to holding it under precise tension while the needle runs at high speed. Most previous systems have tackled only isolated stages of the sewing workflow or relied on dual-arm setups and custom machinery.

The process begins the moment a piece of cloth is placed on the sewing machine bench. A fixed overhead camera captures an image of the workspace, which the system converts to grayscale, preprocesses, and binarizes to separate the fabric from its background. A clever two-kernel filtering scheme then scans the image with concentric square windows of three-by-three and five-by-five pixels to distinguish interior fabric pixels from outline pixels, filtering out noise caused by loose threads that would otherwise corrupt the detected boundary. The result is a clean set of boundary points that describes the fabric’s geometry with striking robustness, even when stray objects or lighting imperfections interfere.

Extracting an accurate contour is harder than it sounds, particularly for non-convex shapes with concavities that a single convex hull computation cannot capture. The researchers devised an iterative refinement strategy: the convex hull algorithm is applied to the full boundary, each resulting line segment is tested against the actual outline, and whenever a segment fails to represent the underlying shape, the hull is recomputed on the points between that segment’s endpoints. This process repeats until every segment accurately traces the fabric edge. Sewing trajectories are then generated by offsetting the extracted outline inward by the desired selvedge distance, producing candidate paths for both straight and curved seams without any predefined geometric model of the workpiece.

Knowing where to grasp the fabric is arguably the most human part of the problem. Experienced operators intuitively pick locations that allow stable manipulation and smooth rotation, yet no deterministic rule generalizes across arbitrary shapes and sizes. The team captured this implicit expertise by recording human demonstrations: for each training instance, two images were taken, one of the fabric alone and one with the operator’s hand placed on it. Image subtraction isolated the hand region, from which a representative grasp point was extracted as ground truth. A feed-forward neural network with two hidden layers of 64 neurons and ReLU activations was then trained on the coordinates of up to 30 outline points, learning to predict grasp locations directly from fabric geometry.

The trained network’s performance is impressive for such a compact model. Across 20 unseen test samples spanning convex and non-convex contours of varying sizes and aspect ratios, the average deviation between predicted and human-selected grasp points was approximately 3.24 millimeters, with about 3 millimeters of error along the x-axis and 2.3 millimeters along the y-axis. When a practical tolerance of plus or minus 4 millimeters was applied, 85 percent of predictions fell within range, and the predicted grasp points enabled successful fabric positioning in roughly 93 percent of manipulation trials. The researchers note that accuracy depends on how visually similar new fabrics are to the training set, underscoring the need for larger and more diverse datasets as the approach scales.

With the grasp point chosen, the robot must align the fabric beneath the needle before stitching can begin. The system computes the angular deviation between the initial segment of the selected sewing path and the sewing machine’s axis, then applies a coordinated sequence of rotational and translational movements so that the starting point sits precisely under the needle with the initial seam segment parallel to the sewing direction. Throughout positioning, a force controller regulates the interaction between the gripper and the workbench, maintaining a constant normal force that prevents slippage and unintended deformation while establishing appropriate tension for the sewing phase ahead.

During sewing itself, three feedback controllers operate in parallel. A camera mounted roughly 80 millimeters above the needle captures a 50 by 40 millimeter field of view around the stitch point, allowing two vision-based controllers to work simultaneously: one compensates the lateral deviation between the needle and the fabric outline, while the other measures local orientation using two parallel virtual scan lines ahead of the needle and applies corrective rotations. Meanwhile, a six-axis force/torque sensor mounted between the gripper and the robot flange feeds a tension controller that keeps the fabric at a reference force of 2 newtons, enough to stabilize the material without causing damage, wrinkles, or excessive stretching. The lateral and orientation corrections are summed into a single control command, exploiting the natural coupling between position and orientation in planar fabric manipulation.

The experimental validation covered fabrics ranging from 180 by 120 millimeters to 300 by 220 millimeters, including medium-stiff cotton, lightweight knit, denim-like composites, and layered materials, all sewn at 45 percent of the machine’s maximum 1800 stitches-per-minute speed, corresponding to a fabric feed velocity of about 84 millimeters per second. Across all trials, seam deviation stayed within plus or minus 2.5 millimeters, with an average of approximately 1.2 millimeters, a standard deviation of 0.9 millimeters, and 79 percent of seam points falling inside the desired offset tolerance. Even layered fabrics with higher friction and deformation resistance were sewn with consistent trajectory tracking, confirming that the tension controller maintained the target force despite increased material stiffness.

The implications for the textile industry, which still relies overwhelmingly on skilled human operators, are substantial. Rather than replacing only one subtask, this framework integrates fabric detection, contour extraction, trajectory generation, grasp point estimation, alignment, tension regulation, and seam execution into a unified workflow compatible with standard industrial equipment. Limitations remain, including modest user intervention for fabric placement and sewing-path selection, and the grasp estimator’s dependence on representative training data. But the work demonstrates convincingly that combining perception, learning, and coordinated vision-and-force control can push robotic sewing toward the autonomous, flexible textile manufacturing that has long eluded the field.

Subject of Research: An integrated robotic sewing system for flexible fabric manipulation using computer vision, learning-based grasp point estimation, and vision- and force-based control

Article Title: A Robotic sewing system for flexible fabric manipulation using computer vision, learning-based grasping and force control

Article References: Anatoliotakis, N., Zacharia, P., & Koustoumpardis, P. (2026). A Robotic sewing system for flexible fabric manipulation using computer vision, learning-based grasping and force control. International Journal of Intelligent Robotics and Applications. https://doi.org/10.1007/s41315-026-00590-3

Image Credits: AI Generated

DOI: 10.1007/s41315-026-00590-3

Keywords: robotic sewing, fabric manipulation, computer vision, neural networks, grasp point estimation, force control, seam tracking, deformable objects, textile automation, SCARA robot, vision-based control, intelligent manufacturing