AI Scheduler and Blockchain Sync Promise Real-Time Accounting Overhaul

Real-time accounting systems face a stubborn trio of problems: tasks pile up faster than they can be scheduled, computing resources sit unevenly loaded across clusters, and distributed ledgers drift out of sync when concurrent writes arrive from multiple business units. A new study published in Discover Artificial Intelligence by Xiaoo Liu and Weiwei Bao of Heilongjiang Vocational College proposes a unified answer, weaving together Apache Spark Streaming, YARN resource scheduling, deep reinforcement learning, graph neural networks, and a permissioned blockchain into a single execution pipeline. The result, tested on 5,000 enterprise-derived accounting task instances, reaches 91.3 percent scheduling accuracy, 88.0 percent cross-ledger synchronization accuracy, and an average response time of just 1.7 seconds under replayed high-concurrency workloads.

The core insight of the research is that task scheduling and ledger consistency should not be treated as separate engineering problems. Traditional accounting platforms typically handle synchronization as an external reconciliation step that runs after execution has finished. Under bursts of concurrent traffic, that separation makes end-to-end performance sensitive to workload spikes, resource contention, dependency propagation, and delayed consistency confirmation. The new framework instead converts ledger synchronization status into a scheduling feedback signal, so that task priority, container allocation, dependency handling, and cross-ledger commit verification are all adjusted within the same execution loop.

The pipeline begins with an accounting task decomposer that partitions each incoming stream request, whether a transaction record or a ledger update, into validation, aggregation, and synchronization stages. These tasks are pushed into a distributed message queue where heterogeneous work is buffered and reordered according to system load. A scheduler continuously monitors YARN resource status, queue priority, and ledger synchronization urgency, dispatching validation and aggregation tasks to stream executor workers while routing cross-ledger commit operations to dedicated synchronization workers. A retry and backoff mechanism between the queue and the scheduler improves robustness when transient execution failures occur.

At the heart of the scheduling engine sits a deep reinforcement learning agent built on an actor-critic architecture. The state vector, 24 dimensions in total, concatenates queue statistics, cluster utilization ratios for CPU, memory, and I/O, task descriptors such as estimated execution time and historical latency, and ledger-related indicators including whether a task triggers cross-ledger synchronization and how large the pending commit backlog has grown. The action space is deliberately compact: the agent assigns each task to a high, medium, or low priority queue and selects a coarse-grained container allocation profile on YARN, either single-container or multi-container execution. The Q-network uses two hidden layers of 128 and 64 neurons with ReLU activations and layer normalization, trained with the Adam optimizer at a learning rate of 0.001 and a discount factor of 0.99. An epsilon-greedy strategy decays exploration from 1.0 to 0.05 over 50,000 steps, while a 100,000-entry replay buffer and a target network synchronized every 1,000 gradient updates stabilize learning.

Two additional machine learning components sharpen the scheduler’s judgment. A graph neural network models task dependency structures, treating tasks as nodes and dependencies as edges, and updating node features through weighted aggregation of neighboring information. This captures the spatial relationships among concurrently executing tasks that rule-based schedulers typically ignore. A long short-term memory network then predicts execution-time correlations across scheduling steps, learning from historical traces to anticipate future resource demands. Together, the GNN-LSTM pair feeds dependency and temporal features into the reinforcement learning policy, which converts them into adaptive queue and container decisions.

Cross-ledger synchronization runs on a permissioned consortium blockchain rather than a public network, a choice driven by enterprise requirements for controllable access, low confirmation delay, and auditable transaction traces. The blockchain layer comprises one ordering node, four peer nodes, and two ledger gateway nodes connected to the Spark Streaming workers, with Raft-based consensus avoiding mining delays. When a ledger-update request reaches a synchronization worker, the smart contract registers transaction metadata, ledger identifiers, version numbers, hash digests, priority labels, and timeout thresholds. If the commit condition is satisfied, a zero-knowledge proof module generates a non-interactive consistency proof over hash commitments and range constraints of sensitive accounting fields. Validators confirm the proof and public commitments without ever seeing the underlying values, preserving privacy while guaranteeing atomicity.

The evaluation dataset was assembled from eight medium- and large-scale enterprises that piloted or deployed the system between January and December 2023, yielding 5,000 anonymized task instances. The workload spans four accounting operations: transaction validation accounts for 44 percent of instances, balance aggregation and reconciliation for 30 percent, report generation for 16 percent, and cross-ledger synchronization for 10 percent. Expert-derived priority labels divide tasks into high, medium, and low classes at 29.6, 46.8, and 23.6 percent respectively. High-concurrency scenarios were generated by compressing the arrival intervals of these real records, a replay protocol that stresses queue pressure and ledger contention without inventing new labeled samples.

The benchmark comparisons are striking. A static scheduler achieved 78.2 percent scheduling accuracy, 74.5 percent synchronization accuracy, and a 2.1-second response time. A conventional DRL scheduler without graph modeling or blockchain feedback improved those figures to 85.6 percent, 81.3 percent, and 1.9 seconds. The full DRL-GNN-BC scheduler pushed all three metrics to their best values. Ablation experiments confirmed that each module matters: removing the reinforcement learning unit, the GNN dependency modeling, or the blockchain and zero-knowledge verification degraded response time by 0.28, 0.17, and 0.09 seconds respectively, corresponding to 16.5, 10.0, and 5.3 percent of the full-stack mean. Crucially, a decoupled variant that retained all modules but severed the feedback path from ledger commit status back to the scheduler demonstrated that live synchronization feedback delivers benefits beyond the simple sum of independent components.

Resource costs remain measurable but controlled. The complete framework sustained an average CPU utilization of 74.6 percent and peak memory utilization of 68.9 percent, with the DRL component adding 11.4 percentage points of CPU load and the GNN module a further 6.8 points. The blockchain and zero-knowledge proof layer produced the largest communication overhead, raising network traffic from 12.3 to 18.7 megabytes per second and per-request communication from 0.91 to 1.84 megabytes, though cryptographic verification is triggered only for the 10 percent of tasks involving cross-ledger commits. Throughput held at 60.8 megabytes per second with 96.0 percent stability. Controlled fault-injection tests, including peer-node interruption, network degradation with 300-millisecond delays and 3 percent packet loss, ledger contention, and workload drift, showed recovery times between 12.6 and 18.5 seconds, with the worst degradation occurring under ledger contention, where synchronization accuracy fell 7.6 percentage points and the rollback rate rose to 8.7 percent.

The authors are careful to scope their claims. The 5,000-instance dataset represents a limited enterprise-derived sample, and replayed concurrency does not constitute additional independent observations, so the reported figures characterize the evaluated cluster and replay protocol rather than universal performance. Cross-cluster, cross-platform, and cross-region validation remain future work, as does integration with software-defined networking controllers that could add network-state features to the scheduler’s state vector. Even so, the study offers a concrete demonstration that treating scheduling, resource allocation, and ledger consistency as one optimization problem, rather than loosely coupled subsystems, can yield faster, more auditable, and more resilient financial infrastructure, a design philosophy likely to resonate well beyond the accounting systems where it was first tested.

Subject of Research: Real-time task scheduling and cross-ledger synchronization in enterprise accounting systems using Spark Streaming, YARN, deep reinforcement learning, and blockchain verification

Article Title: Research on cross-ledger synchronization of real-time accounting system based on Spark Streaming and YARN scheduling

Article References: Liu, X., & Bao, W. (2026). Research on cross-ledger synchronization of real-time accounting system based on Spark Streaming and YARN scheduling. Discover Artificial Intelligence, 6(1), Article 1261. https://doi.org/10.1007/s44163-026-02303-y

Image Credits: AI Generated

DOI: 10.1007/s44163-026-02303-y

Keywords: real-time accounting, Spark Streaming, YARN scheduling, deep reinforcement learning, graph neural networks, blockchain, smart contracts, zero-knowledge proofs, cross-ledger synchronization, distributed ledgers, task scheduling, enterprise data systems