The core problem the researchers set out to solve is deceptively simple to state but difficult to engineer around: fields are full of things that move. People, animals, vehicles, and other machines can cross a tractor’s path unpredictably, and most existing avoidance systems react only to where an obstacle is at the current instant, not where it will be moments later. For large machinery with high inertia, long wheelbases, and limited steering agility, that lag translates into unnecessarily wide detours, abrupt steering corrections, and, in the worst cases, unsafe proximity to obstacles. Static obstacles, by contrast, can usually be handled in advance through global path planning, which is why dynamic obstacles remain the central challenge for autonomous navigation in agriculture.

To give the tractor reliable information about its surroundings, the team fused the complementary strengths of two sensors. A Velodyne HDL-32E 3D LiDAR delivers precise three-dimensional geometry—exact positions, dimensions, and distances of surrounding objects—while an RGB camera supplies rich visual features that make it possible to classify what an object actually is. The technical difficulty lies in merging these heterogeneous data streams. The researchers solved spatial alignment by computing a rigid-body transformation between the LiDAR and camera coordinate systems, consisting of a 3-by-3 rotation matrix and a translation vector, then projecting LiDAR points onto the image plane through the camera’s intrinsic parameter matrix. Temporal synchronization was handled with the ROS message_filters library, which matches point clouds and images by timestamp. The calibration accuracy was quantified as a projection error of just 4.83 pixels, evaluated with a calibration board placed at distances of 2, 5, 10, and 20 meters from the sensors.

On top of this fused perception layer sits a multi-object tracking pipeline. Three-dimensional bounding boxes are constructed from the point cloud data, and an Extended Kalman Filter (EKF) estimates each object’s state—a vector containing its bounding-box height, width, length, center coordinates, and velocity relative to the LiDAR. Because obstacle motion in the field is inherently nonlinear, the EKF linearizes the state transition model by computing the Jacobian matrix of partial derivatives, then iterates between prediction and update steps using the Kalman gain to fuse new LiDAR measurements with the predicted state. In parallel, the YOLO v8 deep network classifies targets from camera images. Data association between detections and predicted tracks is performed with the Hungarian algorithm operating on a cost matrix that blends two distance measures: the Mahalanobis distance, which captures geometric consistency in state space, and the cosine distance between image appearance feature vectors, which helps preserve target identity when objects partially occlude one another. Critically, the weighting between these two terms adapts automatically to the intersection-over-union (IoU) between predicted and detected bounding boxes, ranging from a lower bound of 0.4 to an upper bound of 0.8. When spatial overlap is high, geometry dominates; when it is low—suggesting occlusion or detection drift—appearance features carry more weight.

The tracking experiments demonstrated just how much this fusion strategy matters. Across three field scenarios of 3,500 frames each, the fused method achieved a Multiple Object Tracking Accuracy (MOTA) of 94.72 to 96.37 percent, compared with 88.32 to 91.32 percent for a conventional LiDAR-only Kalman filter approach. Higher Order Tracking Accuracy (HOTA), a comprehensive metric integrating detection, association, and identity preservation, improved by at least 4.25 percentage points, while identity switches—a measure of how often the system confuses one target with another—dropped by at least 106 per scene, from more than 120 to as few as 16. All of this ran on a Jetson Orin NX edge computing platform at 54 to 56 frames per second, comfortably meeting real-time requirements even though it was slightly slower than the lighter LiDAR-only pipeline.

With perception solved, the researchers turned to decision-making. The backbone of the avoidance planner is the Dynamic Window Approach, a local planning method that samples linear and angular velocity pairs within the machine’s kinematic and acceleration limits, predicts short-term trajectories for each candidate, and selects the trajectory scoring highest on a multi-objective evaluation function. The team’s evaluation function jointly weighs heading deviation from the target, obstacle proximity, vehicle speed, trajectory smoothness, and minimum avoidance distance. What makes it distinctive is a dynamic obstacle penalty term that accumulates, with exponential decay, the predicted future positions of all tracked obstacles along each candidate trajectory, incorporating obstacle velocity, position prediction covariance, and an uncertainty weighting. This means the planner is not merely dodging where obstacles are—it is reasoning about where they will be and how confident that prediction is.

The remaining weakness of DWA is that its evaluation weights are typically fixed by hand, which makes the algorithm prone to local optima in cluttered, multi-obstacle scenes. The researchers addressed this with Soft Actor-Critic (SAC), a maximum-entropy deep reinforcement learning algorithm well suited to continuous action spaces. In the SAC-DWA framework, the policy network observes environmental state information—obstacle data, vehicle speed, target position—and outputs a set of DWA weight parameters as its action. The environment returns a reward and the next state, and experience tuples are stored in a replay buffer for training. The reward function balances path-tracking accuracy, heading deviation, speed maintenance, obstacle safety with a hard penalty below a safety distance threshold, and trajectory smoothness, with safety weighted most heavily. Because SAC’s objective includes an entropy regularization term, the policy explores broadly during training and resists premature convergence to suboptimal, overly conservative behaviors. Training was carried out entirely in a Gazebo simulation environment over up to 50,000 steps on an RTX 3090 workstation, avoiding the cost and safety risks of learning directly in the field. Sensitivity analyses identified robust settings for the key hyperparameters: a temperature coefficient of 0.20, a discount factor of 0.99, and a soft update coefficient of 0.005.

Simulation results were striking. In a 30-meter scenario containing five static and five randomly moving dynamic obstacles, SAC-DWA reached the target in a mean of about 45 seconds—roughly 25 percent faster than conventional DWA—along paths of about 45 meters, with mean path smoothness improved from 0.35 to 0.20 radians per meter and the proportion of time violating safety constraints reduced from around 12 percent to just 2 percent.

The decisive test, however, took place in real fields at the Zhuozhou Experimental Station of China Agricultural University, over 40-meter courses with human testers simulating dynamic obstacles. Three scenarios were evaluated: a single obstacle moving in the same direction as the tractor, a single obstacle approaching head-on, two obstacles crossing the machine’s route, and multiple obstacles moving randomly. In every case, SAC-DWA outperformed four comparison methods: conventional DWA, the timed elastic band (TEB) planner, an offline grid-search-optimized DWA, and a TD3-DWA reinforcement learning variant. Notably, in the same-direction scenario—where the obstacle moved faster than the tractor and posed no collision risk—SAC-DWA correctly recognized that no avoidance maneuver was necessary and held a straight course, something position-only methods could not do. Across the full field test series, SAC-DWA achieved the shortest times to goal, the shortest paths, and the lowest trajectory curvature, with safety violation ratios near zero. In the most demanding multi-obstacle scenario, it reached the target in 40.34 seconds on a 47.83-meter path, reducing time by at least 3.49 percent and path length by at least 2.79 percent even against the strongest competitor, TD3-DWA, and by roughly 14 to 20 percent against conventional methods. One-way ANOVA followed by Tukey’s post-hoc testing confirmed the improvements were statistically significant at P < 0.001, and additional speed trials from 0.8 to 2.0 meters per second showed SAC-DWA degrading least as operating speed increased.

The authors note that the system retains the interpretable, real-time structure of DWA while adding adaptive optimization, making it suitable for deployment on resource-constrained edge computing platforms and readily integrated with existing intelligent machinery. Limitations remain: the multi-sensor configuration raises hardware cost and calibration complexity, and the study acknowledges that the safety margin can shrink when human motion is highly random, since the planner may favor efficiency as long as collisions are avoided. Future work, the team suggests, will incorporate risk-aware planning, adaptive safety buffers, more robust motion prediction, crop occlusion, uneven terrain, and sensor degradation, moving toward reinforcement learning that spans the full loop of perception, decision, and control. For now, the study offers one of the most complete demonstrations to date that sensor fusion and reinforcement learning can work together to make full-size farm machines navigate the unpredictable choreography of a living, working field.