Skip to content

Benchmarks

Performance baseline and profiling of the SimPy engine, as defined in ADR-0006. This page is the reference point for later performance work: optimisations (ADR-0006 §5), the Scenario IR refactor (ADR-0007) and the Rust engine (ADR-0008) are all measured against it.

Running the suite

The suite lives in benchmarks/ at the repository root. It is not part of the PyPI package.

# full suite: B1-B3 × 100k / 1M / 10M frames × metrics off/on (~3 h, sequential)
uv run python -m benchmarks run --label baseline

# a subset
uv run python -m benchmarks run --scenario B1 B2 --frames 100k 1M --metrics off

# B4 (10 000 nodes) at research scale — one run, ~1.5-2 h on SimPy
uv run python -m benchmarks run --scenario B4 --frames 100M --metrics off --timeout 14400

# batch throughput: N independent runs, J at a time
uv run python -m benchmarks batch --scenario B3 --frames 1M --runs 16 --jobs 8

# markdown summary of a results file
uv run python -m benchmarks summarize benchmarks/results/<file>.json

# Modbus and MQTT on ns-3's TCP against SimPy, as a shared link loads up
# (needs ns-3; see Execution backends)
uv run python -m benchmarks load --label tcp-under-load
uv run python -m benchmarks load-summary benchmarks/results/<file>.json

# profiling (reports go to benchmarks/profiles/, not committed)
uv run python -m benchmarks profile --scenario B3 --frames 1M --tool cprofile
uv run python -m benchmarks profile --scenario B1 --frames 1M --metrics on --tool tracemalloc
uv run python -m benchmarks profile --scenario B1 --frames 1M --tool py-spy   # needs py-spy installed

Every measurement runs in a fresh subprocess, so that its peak RSS belongs to one run. Measurements run one at a time, because parallel runs would compete for CPU and distort the timings.

Results are written as JSON to benchmarks/results/, one record per run. Each record includes: - the machine, Python, SimPy and SMOLPy versions, and the git commit; - the backend, the number of events and the run time, as reported by the engine itself in SimulationResult.stats (ADR-0007). The run time covers the engine run only. Building the network and its Scenario IR is reported separately as build_s; - an output hash of the event count and all metric samples. It confirms that a run computed exactly the same thing as the runs it is compared against.

Scenarios

All traffic sources are deterministic (constant rates, periodic protocols), so the frame and event counts do not depend on the seed. Each scenario is sized by the number of injected frames, which is rate × simulated duration, exact up to ±1 frame per periodic source.

Scenario Topology Injected frames / simulated s Simulated time for 1 M frames
B1 — single path sender → switch → receiver, one 50 000 fps constant flow (512 B, 1 Gb/s links) 50 000 20 s
B2 — industrial cell 1 switch, 35 nodes: PLC with 10 PROFINET ARs (2 ms), 10 Modbus slaves (10 Hz), 10 MQTT publishers (10 Hz) → broker → historian, plus a 2 000 fps bulk flow 12 400 80.6 s
B3 — large plant core switch + 50 access switches × 20 nodes (1 051 nodes); paired access switches, every node exchanges 20 fps with its counterpart across the core (4 hops) 20 000 50 s
B4 — very large plant as B3 with 200 access switches × 50 nodes (10 201 nodes); switches start converged (mac_table="topology") 200 000 5 s

With metrics on, each scenario adds a representative set of observe() statements, sampled every 100 ms. B3 has 151 observations.

B1–B3 make up the regular suite (run without --scenario). B4 is the research-scale scenario for the extended Rust decision gate (ADR-0006 §6). It is run on its own, at 100 M frames.

Why B4 starts converged. With address learning, every switch floods the first frames to each of the 10 000 destinations. Each flooded frame reaches ~200 access switches × ~50 ports. Queues are unbounded, so all these copies wait in memory. At 1 M frames the result is 329 events per frame instead of 25, 45 min instead of 1 min, and 12 GB peak memory. That cost grows with stations × ports, so it is a start-up artefact of the model, not the scale effect B4 is meant to measure. With mac_table="topology", B4 at 1 M frames takes 60 s and 203 MB, at 24.8 events per frame.

Baseline

Measured on Intel Core Ultra 7 255H (16 threads), 31 GB RAM, Linux, Python 3.14, SimPy 4.1.2.

Raw results: benchmarks/results/2026-11-13-0921-simpy-jaws-zbook-baseline.json. 78 runs, no failures or timeouts; every repetition of every variant produced the same output hash.

Scenario Frames Metrics Runs Wall time (median, min–max) Events Events/s Frames/s Peak RSS Output
B1 100k off 5 2.74 s (2.69–2.76) 1,699,992 620,312 36,489 41 MB identical
B1 100k on 5 2.95 s (2.93–2.98) 1,700,072 576,497 33,910 40 MB identical
B2 100k off 5 4.43 s (4.41–4.47) 1,940,224 437,845 22,567 46 MB identical
B2 100k on 5 4.50 s (4.49–4.56) 1,940,872 430,894 22,201 46 MB identical
B3 100k off 5 15.80 s (14.46–15.93) 7,380,145 467,159 6,330 70 MB identical
B3 100k on 5 16.05 s (15.93–16.82) 7,387,695 460,244 6,230 71 MB identical
B1 1M off 5 27.06 s (26.98–27.53) 16,999,992 628,175 36,952 75 MB identical
B1 1M on 5 29.74 s (23.71–29.93) 17,000,792 571,707 33,628 75 MB identical
B2 1M off 5 44.20 s (43.01–45.04) 19,403,129 438,950 22,623 123 MB identical
B2 1M on 5 44.84 s (43.60–45.00) 19,409,585 432,848 22,301 124 MB identical
B3 1M off 5 79.03 s (78.57–79.66) 37,080,145 469,197 12,654 105 MB identical
B3 1M on 5 81.93 s (80.35–82.58) 37,155,645 453,509 12,206 115 MB identical
B1 10M off 3 271.97 s (271.49–272.23) 169,999,992 625,076 36,769 419 MB identical
B1 10M on 3 300.90 s (298.90–305.07) 170,007,992 565,002 33,234 420 MB identical
B2 10M off 3 443.04 s (439.23–443.28) 194,032,022 437,953 22,571 880 MB identical
B2 10M on 3 448.43 s (448.10–448.51) 194,096,542 432,836 22,300 896 MB identical
B3 10M off 3 704.60 s (704.08–718.26) 334,080,145 474,141 14,192 451 MB identical
B3 10M on 3 752.29 s (750.99–752.88) 334,835,145 445,087 13,293 550 MB identical

What the baseline shows:

  • Throughput is flat in run length. Each scenario keeps its event rate from 100 k to 10 M frames: B1 ~625 k events/s, B2 ~438 k events/s, B3 ~470 k events/s. Wall time grows linearly, with no slowdown in long runs.
  • Frames/s depend on events per frame, not on the event rate. Those ratios are 17 for B1 (2 hops), ~19.4 for B2 and ~33–37 for B3 (4 hops).
  • Metrics cost 1–10 % of run time. The largest share is in B1, where the engine itself does the least work per frame.
  • Peak memory grows with the number of frames. From 1 M to 10 M frames: B1 75 → 420 MB, B2 123 → 896 MB, B3 105 → 550 MB. This matches the latency_us finding below. B2 grows fastest, because Modbus RTT and PROFINET latency and jitter lists grow alongside it.
  • B3 at 100 k is inflated by start-up flooding (~74 instead of ~37 events per frame, see Findings outside performance). Compare B3 at 1 M and 10 M.
  • One B1 1 M metrics-on repetition ran in 23.7 s against a median of 29.7 s, with an identical hash. This is most likely CPU frequency or core scheduling; the median is the reported value.

Baseline against the Rust decision gate (ADR-0006 §6). The gate is evaluated formally after the §5 optimisations. The baseline starting point is:

Target Baseline Status
B3 at 10 M frames, metrics on, under 10 min 752 s (12.5 min) missed by 25 %
Peak memory at 10 M frames under 4 GB max 896 MB (B2, metrics on) met
100 × B2 at 1 M frames under 1 h on 8 cores ~13 × 45 s ≈ 10 min estimated not measured yet (needs ADR-0009 batch runner)

Profiling: where the time and memory go

Profiled at 1 M frames, metrics off. CPU times below are under cProfile, which slows the run 3–5×; the shares are what matter, not the absolute seconds.

CPU: SimPy machinery dominates

B1 (2 hops) B3 (4 hops)
Events processed 17.0 M 37.1 M
Events per injected frame 17.0 37.1
Python function calls 297 M 649 M

One hop costs about 8.5 SimPy events. The events are, in order: 1. a put into the link's egress queue (Store put event); 2. the get by the link process (Store get event); 3. the transmission timeout; 4. the start of a new SimPy process just to wait out the propagation delay: a Process object, its Initialize event and the propagation timeout; 5. the put into the next node's inbound store; 6. that node's get.

On top of that, every injected frame costs one timeout in its traffic generator.

In both profiles, most of the run time is spent inside SimPy itself: - the event loop (Environment.step, Process._resume, schedule, heap push/pop); - Store bookkeeping (_trigger_put, _trigger_get, event constructors).

SMOLPy's own code is only about 12–15 % of own time: the link process _LinkChannel._run, _propagate_and_deliver, enqueue and the traffic generator. Switch forwarding and receivers do not appear among the top consumers. The cost is per event, not per protocol feature.

Memory: one list grows with every frame

B1, 1 M frames metrics off metrics on
Python heap at end of run 39.1 MiB 39.3 MiB
of which _AdapterCounters.latency_us (engine.py, self.latency_us.append(...)) 30.9 MiB (1 M floats) 30.9 MiB
everything else (topology, queues, metric samples) < 1 MiB < 1 MiB

Every adapter appends the latency of every received frame to a list that is never trimmed, whether or not anything observes it. That is 32 B per received frame: - about 310 MiB at 10 M frames; - about 3 GB at 100 M frames.

modbus_rtt_us and the PROFINET latency and jitter lists behave the same way for their traffic. Sampling windows only ever read the newest slice of these lists.

ADR-0006 hypotheses

Hypothesis Result
Several SimPy events per frame per hop Confirmed: ~8.5 per hop, plus 1 per injected frame
A new process per frame per hop for propagation Confirmed: one Process + Initialize event + timeout per hop; the single largest avoidable cost
A new _Frame object per hop Confirmed: one copy per delivery. Cheaper than the SimPy events, but still one allocation per hop.
Memory grows with frames, not network size Confirmed: latency_us holds ~79 % of the Python heap at 1 M frames, with metrics on or off

Findings outside performance

Building the benchmarks exposed two behaviours of the model that matter for results, not just for speed:

  • One-way traffic floods. A switch learns a MAC address only from frames the station sends. If a destination never sends, every access switch on the path keeps flooding frames for it to all its ports for the whole run. This is real Ethernet behaviour. It was the reason B3 was changed to paired, two-way traffic, because the one-way version measured 116 instead of ~35 events per frame.
  • Adapters accepted frames not addressed to them (fixed). _adapter_receiver counted every frame that arrived, without checking dst_mac. Under flooding, throughput, bytes_received and latency of every node in the flood domain included traffic meant for someone else. A real NIC drops these frames. All end stations (adapters, Modbus slaves, MQTT brokers, PROFINET devices) now accept only frames addressed to their own MAC. B1 and B2 are unaffected. In B3 the change is confined to the first 100 ms sampling window, where start-up flooding used to show up, for example as 20.48 Mb/s instead of 0.04 Mb/s on an observed node. From 200 ms on, every metric is identical.

A third, smaller effect: B3 at 100 k frames shows ~74 events per frame instead of ~37. At start-up no switch knows any of the 1 000 destinations, so the first frame to each one is flooded through the core and the access switches. This one-off cost is ~4 M events; it is significant at 100 k frames and negligible at 1 M and above.

Optimisation log

Each optimisation is checked with the metrics hash, a hash of all metric samples without the event count. Removing engine events changes the output hash by design. The metrics hash must stay identical to the baseline engine in every scenario with metrics on.

Timings below are single runs, metrics on, on the reference machine. The full suite was then re-run on the result; see Optimised engine vs baseline.

1. Propagation as a timed callback

_LinkChannel no longer starts a SimPy process for every frame on every hop. Each frame is now a single timeout that carries it, with delivery attached as a callback.

Scenario, 1 M frames Events before → after Run time before → after Metrics hash
B1 17.0 M → 13.0 M (−24 %) 30.4 s → 24.7 s (−19 %) identical
B2 19.4 M → 15.4 M (−21 %) 44.8 s → 37.6 s (−16 %) identical
B3 37.2 M → 28.1 M (−24 %) 86.9 s → 68.4 s (−21 %) identical

One hop now costs ~6.5 events instead of ~8.5.

2. Keep only the current window of per-frame values

Latency, Modbus RTT and PROFINET latency and jitter used to be appended to lists that lived for the whole run, observed or not. Now each sampler registers its own window buffer with the node and clears it after every sample. With no observer, the value is not even computed.

The window keeps the raw values and aggregates them as before, rather than using a running sum. Since Python 3.12 sum() over floats uses compensated summation, and a running sum could differ in the last bits.

Scenario, metrics on Peak RSS before → after Run time before → after Metrics hash
B1, 1 M 75 MB → 36 MB 24.7 s → 23.9 s identical
B2, 1 M 124 MB → 37 MB 37.6 s → 35.0 s identical
B3, 1 M 115 MB → 56 MB 68.4 s → 66.9 s identical
B2, 10 M 896 MB → 45 MB 448 s → 323 s (optimisations 1 + 2) —

Peak memory no longer grows with run length. It is now set by the topology and the number of observations.

3. @dataclass(slots=True) for _Frame

Optimisations 3 and 4 do not change the order of events, so the full output hash, event count included, has to match as well. It does in all six scenario and size combinations. Their effect is a few per cent, which is about the noise of a single run, so they were measured A/B, alternating the two variants three times. The table shows medians.

Scenario, 1 M frames, metrics on Without → with slots
B1 24.4 s → 23.9 s (−2 %)
B3 71.4 s → 69.8 s (−2 %)

4. No frame copy per hop

Delivery used to build a new _Frame on every hop, so that flooded copies would not share in_port. A frame object is only ever in one place, because every sender builds a new frame per transmission. So delivery now sets in_port in place. Only flooding (switch, unknown destination) and hub broadcast copy the frame, once per output port.

Scenario, 1 M frames, metrics on Copy → no copy
B1 23.9 s → 22.4 s (−6 %)
B3 70.2 s → 66.8 s (−5 %)

B3 at 100 k frames, which floods heavily at start-up, also gives an identical output hash.

Optimised engine vs baseline

The full suite was re-run after all four optimisations, at commit d3eb8a2, on the same machine. Raw results are in benchmarks/results/2026-11-13-1527-simpy-jaws-zbook-optimised.json. All 78 runs completed, and every repetition of every variant produced the same output hash. The metrics hash with metrics on matches the baseline engine at 100 k and 1 M frames.

Scenario Frames Metrics Wall time baseline → optimised Speed-up Events/frame Peak RSS baseline → optimised
B1 100k off 2.74 s → 1.59 s 1.73× 17.0 → 13.0 41 MB → 36 MB
B1 100k on 2.95 s → 2.21 s 1.34× 17.0 → 13.0 40 MB → 37 MB
B2 100k off 4.43 s → 3.35 s 1.32× 19.4 → 15.4 46 MB → 37 MB
B2 100k on 4.50 s → 3.53 s 1.27× 19.4 → 15.4 46 MB → 37 MB
B3 100k off 15.80 s → 11.55 s 1.37× 73.8 → 55.6 70 MB → 47 MB
B3 100k on 16.05 s → 13.30 s 1.21× 73.9 → 55.7 71 MB → 48 MB
B1 1M off 27.06 s → 19.71 s 1.37× 17.0 → 13.0 75 MB → 36 MB
B1 1M on 29.74 s → 22.03 s 1.35× 17.0 → 13.0 75 MB → 37 MB
B2 1M off 44.20 s → 33.40 s 1.32× 19.4 → 15.4 123 MB → 37 MB
B2 1M on 44.84 s → 35.19 s 1.27× 19.4 → 15.4 124 MB → 38 MB
B3 1M off 79.03 s → 61.60 s 1.28× 37.1 → 28.1 105 MB → 47 MB
B3 1M on 81.93 s → 64.59 s 1.27× 37.2 → 28.1 115 MB → 56 MB
B1 10M off 271.97 s → 198.06 s 1.37× 17.0 → 13.0 419 MB → 36 MB
B1 10M on 300.90 s → 220.48 s 1.36× 17.0 → 13.0 420 MB → 38 MB
B2 10M off 443.04 s → 331.66 s 1.34× 19.4 → 15.4 880 MB → 37 MB
B2 10M on 448.43 s → 353.43 s 1.27× 19.4 → 15.4 896 MB → 45 MB
B3 10M off 704.60 s → 543.93 s 1.30× 33.4 → 25.3 451 MB → 47 MB
B3 10M on 752.29 s → 566.59 s 1.33× 33.5 → 25.4 550 MB → 145 MB
  • The Python-level work gave about 1.3× across the board and removed ~24 % of engine events. The event rate itself barely moved (B1 ~655 k events/s, B2 and B3 ~440–480 k events/s). What remains is SimPy's per-event cost.
  • Memory is now flat in run length for engine state: 36–47 MB with metrics off, at every size. The 145 MB of B3 at 10 M frames with metrics on is the result itself: 151 observations × ~5 000 samples each, held in SimulationResult.metrics.
  • Runs of 1–3 s are noisy. B1 at 100 k frames ranges 0.96–2.09 s, probably due to CPU frequency scaling. Compare at 1 M frames and above.

Integer nanoseconds (ADR-0007 step 7)

The SimPy engine moved from float µs to integer nanoseconds (Model Semantics §1). It was measured A/B against the float engine of the previous commit: 200 k frames, metrics on and off, 3 alternating repetitions per variant, medians.

Scenario Metrics Float µs Integer ns Ratio Events Output hash
B1 on 4.27 s 4.16 s 0.97 same same
B1 off 3.79 s 3.90 s 1.03 same same
B2 on 6.83 s 6.50 s 0.95 same changed
B2 off 6.60 s 6.38 s 0.97 same same
B3 on 18.64 s 18.63 s 1.00 same same
B3 off 17.66 s 18.20 s 1.03 same same
B4 on 12.72 s 13.01 s 1.02 same same
B4 off 12.40 s 12.88 s 1.04 same same
  • No slowdown. The ratios lie between 0.95 and 1.04, with a geometric mean of 1.00, which is within the noise of runs this short.
  • Events and sample times are unchanged in every scenario.
  • Metric values differ only in the last digits. The output hash rounds values to 9 significant digits, so it changed in one case only.
  • The one changed hash is float noise disappearing. In B2 with metrics on, pn_jitter:plc was 3.6·10⁻¹¹ to 9.3·10⁻⁹ µs in some windows. That jitter came from adding a float cycle to a float clock. With integer time it is exactly 0.
  • New reference hashes, 200 k frames, seed 42. The previous ones remain valid for the float engine.
Scenario Metrics Events Output hash Metrics hash
B1 on 2 600 156 f048830c28eeea8e d6e8b361315d7870
B1 off 2 599 996 e8a33879a5bbf6b7 44136fa355b3678a
B2 on 3 081 985 7121957abe6fe8b5 77f68c90f91c7d95
B2 off 3 080 689 b1777c3ad5ff40af 44136fa355b3678a
B3 on 8 076 247 6d18affb11375afb 2eadb14fc7e594fc
B3 off 8 061 147 01f72baf86c5101d 44136fa355b3678a
B4 on 4 806 383 ebcf601c4d085453 15127f118dae037b
B4 off 4 800 373 02af5a54a6a9b70d 44136fa355b3678a

Ethernet frames on the wire (ADR-0010)

Frames now carry their FCS and, with a priority, an 802.1Q tag. Every frame also takes a preamble, SFD and inter-frame gap on the link (ADR-0010).

Run time. - The gap costs no extra event. It is added to the next frame's transmission timeout. - No slowdown. B3 at 200 k frames with metrics on, measured A/B against the previous commit with 3 alternating repetitions, has a median of 18.3 s against 18.9 s. The single runs range from 14.8 s to 19.9 s, so that difference is noise. - A first version was 3 % slower. It read the clock through SimPy's now property twice per frame and hop. The loop now reads _now directly.

Results. - Metrics change, so every metrics-on hash changes: latencies include the preamble and SFD on every hop, and frames are larger. - Metrics-off hashes of B1 and B2 are unchanged. They cover only the event count, which is the same. - B3 and B4 process 6 events fewer (8 076 241 against 8 076 247 for B3). Frames now take longer on the wire, so a few more deliveries fall after the end of the run. B4 builds converged MAC tables, so there is no flooding that could explain it otherwise.

New reference hashes, 200 k frames, seed 42:

Scenario Metrics Events Output hash Metrics hash
B1 on 2 600 156 fa12188e00b30f2f f560f3e269367339
B1 off 2 599 996 e8a33879a5bbf6b7 44136fa355b3678a
B2 on 3 081 985 b7553cb6509ab6c8 de8b9a94b6b4c10d
B2 off 3 080 689 b1777c3ad5ff40af 44136fa355b3678a
B3 on 8 076 241 0f71bc321d823748 aec5aaa9d9372139
B3 off 8 061 141 bc694547192bc981 44136fa355b3678a
B4 on 4 806 377 65ca251c2c3cd9db 06a64eac51c98358
B4 off 4 800 367 5eeb1fb6fffa3dac 44136fa355b3678a

Rust engine: stop criterion (ADR-0008)

B1 at 10 M frames, metrics off, on both engines in one session, 3 alternating repetitions. Median run time: SimPy 200.8 s, Rust 4.93 s, which is 40.8× faster. The required factor was 10×. Event counts (129 999 996) and output hashes (9eda076fe235df25) are identical. Details are in ADR-0008. Raw results are in benchmarks/results/2026-11-15-1255-rust-jaws-zbook-stop-criterion.json.

python -m benchmarks.worker --backend rust runs one measurement on the Rust backend. The record has call_s, the whole backend call, next to the backend's own run_s.

Rust decision gate (ADR-0006 §6)

Target Optimised SimPy Result
B3 at 10 M frames, metrics on, under 10 min 566.6 s (9.4 min); baseline 752 s met, by a 6 % margin
Peak memory at 10 M frames under 4 GB max 145 MB met
100 × B2 at 1 M frames under 1 h on 8 cores not measured; 100 × 35.2 s / 8 ≈ 7–8 min if runs scale, ~15 min allowing 2× for shared cores met (estimate), to be confirmed by the ADR-0009 batch runner

By the rule in ADR-0006 §6, optimised SimPy meets all three targets, so the Rust engine (ADR-0008) does not proceed on these targets. These targets measure whether SimPy is enough for today's scenarios, not for larger networks or many independent runs. The gate was therefore extended with research targets (ADR-0006 §6, revision).

Research targets

Measured at commit 80a493f. Raw results are in benchmarks/results/2026-11-13-2119-simpy-jaws-zbook-research-b4.json and …-2116-…-research-batch.json.

Target Optimised SimPy Result
B4 (10 201 nodes) at 100 M frames, metrics off, under 30 min 6 361 s (106 min): 2.50 G events, 393 k events/s, 15.7 k frames/s, peak 204 MB missed, 3.5× over
1 000 × B3 at 1 M frames, metrics on, under 30 min on 8 cores 16 runs, 8 at a time: 183 s, i.e. 315 runs/h; 1 000 runs ≈ 3.2 h missed, 6.4× over
  • B4 scales linearly: 24.8 → 25.0 events per frame from 1 M to 100 M frames. Memory stays flat at ~204 MB, of which the 10 201-node topology is most.
  • The event rate is lower than in B3: 393 k against ~465 k events/s. Larger MAC tables and more processes make each event slightly more expensive.
  • Parallel runs barely slow each other: ~8 % per run at 8 at a time. Batch throughput is limited by the cost of one run.

Optimised SimPy misses both research targets, so the gate is open for ADR-0008. Work there starts after ADR-0007 and is subject to its stop criterion: B1 at least 10× faster than optimised SimPy.

Research targets on the Rust engine (ADR-0008, step 6)

Measured on 2026-11-15 on the same machine with --backend rust, the same harness and the same core budget. Raw results are in benchmarks/results/2026-11-15-1333-rust-jaws-zbook-research-b4.json and …-1346-…-research-batch.json.

Target Limit SimPy (2026-11-13) Rust Result
B4 (10 201 nodes) at 100 M frames, metrics off 30 min 6 361 s (106 min) 310 s (5.2 min): 2.50 G events, 8.1 M events/s, 322 k frames/s, peak 170 MB met, 5.8× under the limit
1 000 × B3 at 1 M frames, metrics on, 8 cores 30 min ≈ 3.2 h (315 runs/h) 413 s (6.9 min): all 1 000 runs, 8 714 runs/h, median 2.6 s per run met, 4.4× under the limit
  • The same model. The SimPy figures predate integer time and ADR-0010, so the speed-ups of about 20× for B4 and 28× for batches compare two slightly different models. The targets are absolute, and the Rust engine meets them on the current model.
  • Bit-exact with SimPy. At 200 k frames, B3 and B4 give the same output and metrics hashes on both engines, with metrics on and off.
  • The batch ran exactly as for SimPy. Each run is its own process, 8 at a time, as in the SimPy measurement. All 1 000 runs have the same seed and give one output hash.
  • B4 at 100 M frames was not repeated on SimPy: it takes 106 min. Its event count, 2 499 800 367, is consistent with B4's 25 events per frame.
  • Two fixes on the way. Building B4's converged MAC tables was quadratic in the number of stations, and forwarding searched the port list. Both are now constant time per lookup. B4 at 200 k frames went from 7.1 s to 1.7 s, with unchanged hashes.

python -m benchmarks run … --backend rust and python -m benchmarks batch … --backend rust measure the Rust backend. The default stays simpy.