Interactive Flow-Models Repository Expands Online Network Traffic Analysis Tools

A new release of the open-source flow-models framework introduces an interactive web platform designed to streamline the exploration and analysis of internet protocol network traffic data directly within a web browser. Developed by researcher Piotr Jurkiewicz and published in SoftwareX, version 3.0 builds upon earlier analytical tools by transforming static data archives into an accessible, client-side environment. This update allows researchers and network engineers to inspect large libraries of flow statistics, examine fitted probability distributions, and evaluate experimental results without downloading extensive raw data repositories for local computation.

Modern data networks experience expanding demands, prompting network operators to engineer traffic at the granular level of individual network flows. Under this paradigm, routing and capacity decisions rely on observed or predicted traffic loads across interconnected links. The operational feasibility of per-flow traffic management depends substantially on the presence of elephant flows, which represent a very small fraction of total flow counts yet transport a predominant share of total byte volume. Efficiently identifying these high-volume flows allows systems to balance traffic loads and optimize forwarding paths without incurring prohibitive overhead across every minor data transaction.

The flow-models framework was originally created in Python to quantify the statistical properties of internet protocol flows. Following its initial publication, version 2.0 expanded the toolset to model and detect elephant flows using machine learning algorithms trained via the scikit-learn library. That prior milestone established early-stage flow classification based on initial packet observations and released an anonymized binary flow dataset prepared using prefix-preserving algorithms. While subsequent research reused those benchmark records, accessing the associated histograms, fitted mixture models, and validation outcomes previously required researchers to retrieve files manually and write bespoke local scripts to parse and visualize the results.

The 3.0 release addresses these access limitations by introducing a client-side single-page application deployed openly online. The browser interface is structured around a three-panel layout that organizes datasets, interactive visualizations, and accompanying documentation. On the left panel, users navigate dataset hierarchies using cascading selectors that dynamically filter criteria such as internet protocol version, transport protocols like TCP and UDP, or domain name query exclusions. The left panel also supplies metadata and global statistics, reporting total flow counts, transmitted packet counts, octet volumes, and mean flow durations alongside dataset summaries.

Model visualizations within the central panel take advantage of responsive vector graphics equipped with uniform, fixed axis limits. By standardizing axis boundaries across different datasets and analytical views, the interface allows direct side-by-side graphical comparisons. Embedded interactive controls enable users to hover over curves to retrieve exact numerical coordinates for overall distributions and individual mixture components. Researchers can also pin coordinate markers, select distinct ranges to inspect regional coverage statistics, toggle individual plot lines on and off, and zoom into specific distribution regions across cumulative distribution functions and probability density views.

Beyond displaying pre-computed distribution curves, the interface incorporates dynamic viewers capable of generating visualizations on the fly from structured source records. A specialized time-series tool calculates thirty-second rolling averages of flow, packet, and bit transfer rates directly from underlying counter files, displaying these rates alongside shaded minimum-maximum envelopes. A companion histogram tool renders step charts straight from comma-separated data tables, while summary metrics are retrieved dynamically from newly introduced structured JavaScript Object Notation outputs created by the platform’s command-line utilities.

For repositories distributing raw packet traces, the system adds a dedicated preview tool backed by a new software module. This component decodes binary flow directories to present field names, data types, and byte sizes alongside a tabular preview of the first one hundred decoded records. This mechanism enables users to confirm data formatting, examine field configurations, and understand anonymization parameters before downloading large binary files. To minimize storage requirements and accelerate transmission, the updated binary writer automatically omits empty field files containing only zero values.

The platform also integrates visualization modules for theoretical calculations and practical simulations of elephant flow detection strategies. Users can review flowcharts and comparative performance curves for three prominent detection architectures: first-packet machine learning classification, counter-based volume thresholds, and packet sampling mechanisms. The interface plots theoretical performance boundaries derived from mathematical models and trace replays, establishing the upper limits of achievable detection accuracy against which practical implementations, such as scikit-learn classifiers, can be benchmarked.

Complementing detection evaluations, the platform visualizes packet-mirroring experiments relevant to software-defined networking architectures. Simulating switch behavior by replaying sorted network traces, the framework calculates the control-plane burden incurred when mirroring the first N packets of every observed flow. The interactive dashboard plots mirrored packet and bit rates across various mirror counts, tracking their proportional share of total network volume over daily cycles. These benchmarks provide quantitative measures of the processing and telemetry overhead placed on network controllers under alternative flow-inspection thresholds.

Technically, the version 3.0 update incorporates seventeen repository commits adding 8,068 lines of code and removing 340 lines, while maintaining compatibility with Python 3.10 through updated standard library references. All web navigation states, including dataset paths, active tabs, and filter configurations, are encoded directly into query parameters to support deep linking and persistent browser history. By consolidating raw measurement previews, fitted statistical models, and experimental control-plane simulations into an interactive browser environment, the updated framework provides an open operational resource for network traffic research and classification benchmarks.

The release of version 3.0 represents a direct progression from version 2.2, expanding the analytical framework first described in Piotr Jurkiewicz’s 2022 SoftwareX publication. Published under the permissive MIT open-source license and tracked through Git, the codebase combines Python and JavaScript to power its modeling tools and browser interface. To support data processing and mathematical routines, the environment depends on NumPy, SciPy, Matplotlib, pandas, scikit-learn, and the cryptography library. Detailed technical documentation and user manuals are maintained online at flow-models.readthedocs.io to assist researchers in navigating the software. Direct inquiries regarding the software implementation are directed through the published academic contact address at piotr.jurkiewicz@agh.edu.pl, preserving transparent developer support for network traffic researchers.

Within the web interface, the layout organizes assets using a dedicated right panel that categorizes available artifacts into model documentation, analysis results, and experiment outputs. In the left panel, the metadata table provides specific operational context by reporting the active selector path, the measurement interval, the effective filter expression, and the availability of complete binary flow records. The summary statistics panel reports aggregate counts of flows, packets, and octets, along with average flow length, average flow byte size, and mean packet size. The hierarchical navigation itself is structured around a directory convention where nested folders represent traffic subsets such as IPv4, TCP, UDP, or domain name exclusions. Selecting a node displays child categories and applies filter or datetime conditions cumulatively, generating an effective filter expression via the logical conjunction of all path predicates. The exact syntax for filter tokens, transformation rules, and metadata files consumed by the interface is documented on the platform’s datasets.rst page.

The interactive plot interface incorporates dedicated display tabs that permit users to toggle between a unified single-figure view and individual distribution charts. These individual tabs provide cumulative distribution functions, probability density functions for flows, packets, and octets, and average value plots. Because the inline Scalable Vector Graphics employ fixed axis limits across every quantity and plot type, researchers can compare figures across different datasets without visual distortion. Directly below each time-series chart, integrated links allow analysts to open or download raw counter files and their static renderings. The time-series viewer computes thirty-second rolling means for flow, packet, and bit rates, displaying shaded min-max envelopes with optional per-subset series overlays. Both the time-series and histogram viewers support hover inspection, enabling users to isolate mixture curves, pin coordinate markers, or select chart spans to review interval coverage.

To inspect binary trace files before downloading complete archives, the interface integrates a new records_preview module that reads binary flow directories to construct an intermediate preview.json file. This module presents the binary schema, including field names, data types, and byte sizes, alongside a decoded table displaying the first one hundred records, as demonstrated for UDP and non-DNS traffic selections. In parallel, storage requirements are reduced through optimizations in the binary flow writer, which automatically omits field files containing solely zero values. Technical maintenance in version 3.0 also corrected the fit.save to_json function call to explicitly pass mixture fields during serialization. In addition, the codebase replaced datetime.UTC with datetime.timezone.utc, preserving full operational compatibility for environments running Python 3.10.

The experimental visualizations systematically measure control-plane loads for software-defined networks through packet-mirroring simulations evaluated across mirror counts of one, two, three, four, and eight packets. By replaying sorted flow records at recorded timestamps, the system tracks per-second counters of mirrored packets and octets, calculating daily traffic patterns and cumulative bandwidth shares for each mirror count. For elephant flow detection, the platform analyzes three method families, encompassing first-packet classification, counter thresholds, and packet sampling, measuring traffic coverage, flow-table occupancy, and flow-operations costs. The interface establishes performance benchmarks by plotting theoretical boundary curves derived analytically from histograms and mixture models, as well as empirically from full trace replays, bounding practical scikit-learn classifier performance. Key operating points are also exported as TeX tables, while the author declares no competing financial interests or personal relationships influencing the work reported in this paper.

Subject of Research: Interactive web-based exploration and modeling of IP network flow data and elephant flow detection

Article Title: flow-models 3.0: Interactive repository of network flows data and experiments

Article References: Jurkiewicz, P. (2026). flow-models 3.0: Interactive repository of network flows data and experiments. SoftwareX, 36, Article 103201. https://doi.org/10.1016/j.softx.2026.103201

Image Credits: AI Generated

DOI: 10.1016/j.softx.2026.103201

Keywords: network flows, traffic engineering, elephant flows, flow-models, open-source software, machine learning, scikit-learn, packet mirroring, software-defined networking, traffic classification, data visualization, single-page application, time-series analysis, packet sampling, Python, network monitoring, load balancing