Performance & Throughput Validation over QSFP28 Fibre Optics
100GBASE-SR4 · 100GBASE-LR4
Field
Detail
Document title
100GbE Inter-Switch Link — Performance & Throughput Test Plan
Version
1.0 (Draft for review)
Prepared by
Brett Rogers, Connec — Industrial Power Connectors
Date
17 August 2026
Test scope
Point-to-point 100GbE link between Switch-A and Switch-B over QSFP28 optics
Focus
Throughput, latency, jitter, frame loss, MTU, buffering and sustained load
Assumption
Link is physically installed and reaches UP state; commissioning of L2/L3 services is out of scope
Status
Pending peer review and change approval
1. Purpose and Scope
This plan defines the acceptance testing required to prove that a 100 Gigabit Ethernet link between two switches, carried over QSFP28 fibre optic transceivers, performs at line rate and within specification before it is placed into production service.
1.1 Terminology
Throughout this document the link is referred to as 100GbE (100 gigabits per second), not 100 GB. 100 Gbps equates to approximately 12.5 GB/s of theoretical payload capacity. This distinction matters when interpreting host-based test results, which are frequently reported in bytes rather than bits.
1.2 In scope
Optical baseline of both QSFP28 transceivers and the fibre path (measured, not assumed).
Forward Error Correction (FEC) behaviour and pre-/post-FEC bit error rate.
RFC 2544 throughput, latency, frame loss and back-to-back burst testing across the standard frame size range.
Bidirectional full-duplex line rate validation.
Frame delay variation (jitter) per RFC 3393.
MTU and jumbo frame validation up to the platform maximum.
Buffer, microburst and incast behaviour under congestion.
TCP goodput validation per RFC 6349 where host-based testing is used.
Sustained soak test with error counter monitoring.
1.3 Out of scope
Initial cabling, patching and physical installation works.
Routing protocol convergence, spanning tree and control plane failover testing.
Multi-hop or end-to-end application performance beyond the two switches under test.
Security, ACL and QoS policy validation, other than the buffer behaviour tests noted below.
1.4 Success statement
The link is accepted when every mandatory test case returns a Pass, the post-FEC bit error rate is zero across the soak period, and bidirectional throughput reaches 100% of theoretical line rate at frame sizes of 512 bytes and above with zero frame loss.
2. Test Topology
A single point-to-point 100GbE link is under test. Two topologies are used depending on the test phase.
2.1 Phase A — Instrumented (preferred)
A hardware traffic generator is connected to a 100GbE port on each switch. Traffic is injected at Switch-A, traverses the link under test, and is received and analysed at Switch-B. This is the only method that reliably produces true line rate at small frame sizes and provides accurate one-way latency measurement.
Tester Port 1 → Switch-A [Port X] ══ QSFP28 LINK UNDER TEST ══ Switch-B [Port Y] → Tester Port 2
2.2 Phase B — Host-based (fallback)
Where a hardware tester is unavailable, two servers fitted with 100GbE NICs are attached, one to each switch. Host-based testing cannot achieve line rate at 64-byte frames and will under-report performance if not tuned; results must be interpreted against the caveats in Section 7.
2.3 Link under test — record before starting
Attribute
Switch-A
Switch-B
Hostname
Make / model / OS version
Interface ID
Transceiver type (SR4 / LR4)
Transceiver part / serial
Fibre type (OM4 / OS2)
Connector (MPO-12 / duplex LC)
Measured path length
Patch panel / cross-connect IDs
FEC mode configured
3. Prerequisites and Equipment
3.1 Equipment
Item
Requirement
Traffic generator
Hardware tester with two 100GbE QSFP28 ports capable of line rate at 64-byte frames, RFC 2544 automation and hardware timestamping. Examples: Keysight/Ixia, Spirent TestCenter, Viavi. Software alternative: TRex or DPDK-pktgen on a tuned server.
Host NICs (Phase B)
Two 100GbE NICs (e.g. NVIDIA/Mellanox ConnectX-5/6, Intel E810) in PCIe 3.0 x16 or better slots. PCIe 3.0 x16 gives roughly 126 Gbps usable — adequate for one direction only. Use PCIe 4.0 x16 for full-duplex host testing.
Optical power meter
Calibrated meter supporting 850 nm and 1310 nm, plus a reference patch lead of matching type.
Fibre inspection scope
Video inspection probe with MPO and LC tips. Mandatory — contamination is the single most common cause of 100GbE link faults.
Spare transceivers
At least one matched spare pair of the same type, for fault isolation.
Variable attenuator
Optional, for optical margin stress testing (TC-14).
Cleaning kit
MPO and LC one-click cleaners plus lint-free wipes and IPA.
3.2 Entry conditions
Change record raised and approved; maintenance window confirmed.
Both switches running the intended production software version.
Transceivers are a matched pair of the same standard at both ends. SR4 will not link to LR4.
Both ends configured with the same FEC mode, speed and MTU.
All fibre endfaces inspected and cleaned, and inspected again after cleaning.
A rollback plan exists and the pre-change configuration is backed up.
Monitoring and alerting suppressed for the ports under test to avoid noise during load generation.
Baseline configuration of both interfaces captured to file.
3.3 Optical reference values
Use these figures to judge whether measured DOM readings are healthy. Always confirm against the specific transceiver vendor datasheet, which takes precedence.
To support this test programme we are seeking advice on equipment availability, technical support and reporting capabilities. Based on our initial review, we believe the following equipment may be required. The optical loss test set, OFDR and OTDR support the fibre path characterisation underpinning TC-01 and TC-02; the dual-port tester supports TC-04 through TC-09 and TC-15.
Reference leads for loss measurement on LR4 spans.
Connec receptacle-to-LC adaptors / pigtails
Interface between the Connec receptacle system and test equipment.
Environmental monitoring equipment
Temperature and humidity logging across the soak period (TC-15).
Laptop and reporting software (where applicable)
Data capture, counter polling and report compilation.
3.5 Information requested from supplier
What equipment do you have available that would be suitable for this testing?
Can you provide engineering support and a formal test report?
What is the earliest availability of the equipment and/or personnel in Brisbane?
Do you have any recommendations regarding the most appropriate test methodology to satisfy BHP acceptance requirements?
4. Line Rate Reference Data
Theoretical maximum frame rates for 100GbE, accounting for the 20 bytes of per-frame overhead (12-byte inter-frame gap plus 8-byte preamble and start-of-frame delimiter). Use these as the denominator when calculating percentage of line rate achieved.
Frame size (bytes)
Frames per second
L1 rate (Gbps)
L2 payload throughput (Gbps)
64
148,809,524
100.000
76.19
128
84,459,459
100.000
86.49
256
45,289,855
100.000
92.75
512
23,496,241
100.000
96.24
1024
11,973,180
100.000
98.08
1280
9,615,385
100.000
98.46
1518
8,127,438
100.000
98.70
9216 (jumbo)
1,353,398
100.000
99.78
Small-frame performance is the honest test of a switch fabric. A link that passes at 1518 bytes but fails at 64 bytes has a forwarding or fabric limitation, not an optical one.
5. Test Cases
Order matters. Do not proceed to traffic generation until TC-01 through TC-03 have passed. Running load across a marginal optical path produces misleading results and wastes window time.
5.1 Baseline and optical integrity
TC-01 — Fibre endface inspection and cleanliness
Objective
Confirm every optical endface in the path is clean and undamaged before any measurement is taken.
Tooling
Video inspection probe with MPO and LC tips; one-click cleaners.
Method
Inspect both transceiver ports, both patch lead ends, and every patch panel adapter in the path. Clean any endface that fails IEC 61300-3-35 criteria, then re-inspect. For MPO connectors, inspect all 12 fibres and confirm correct polarity (Type B crossover is standard for SR4).
Pass criteria
All endfaces pass IEC 61300-3-35 with no contamination, scratches or pits in the core zone. MPO polarity confirmed correct.
Notes
Never inspect a live fibre without confirming the far-end transmitter is disabled.
TC-02 — Optical power and DOM baseline
Objective
Record transmit and receive optical power on all four lanes at both ends and confirm adequate margin above receiver sensitivity.
Tooling
Switch CLI DOM/DDM output; calibrated optical power meter for independent verification.
Method
Read per-lane Tx and Rx power, temperature, supply voltage and bias current at both ends. Calculate path loss as (near-end Tx) minus (far-end Rx) for each lane. Compare against Section 3.3 and the vendor datasheet. Independently verify at least one lane with the power meter to validate the DOM readings.
Pass criteria
All lanes within vendor Tx and Rx specification. Rx power at least 3 dB above stated receiver sensitivity on every lane. Lane-to-lane Rx variance no greater than 2 dB. No DOM alarms or warnings.
Notes
Lane imbalance greater than 2–3 dB usually indicates a dirty or misaligned MPO connector, or a partially seated ferrule.
Characterise every optical event along the fibre path — connectors, splices, adaptors and coupler interfaces — and confirm that per-event loss and reflectance, and total path loss, fall within the acceptance thresholds.
Tooling
OFDR (preferred) — millimetre-scale spatial resolution, required because the coupler interfaces within the Connec receptacle system are spaced more closely than a conventional OTDR can resolve. High-resolution OTDR with launch and receive leads as the fallback. 1310 / 1550 nm optical loss test set for the reference end-to-end insertion loss measurement.
Method
Establish the end-to-end insertion loss with the loss test set using the one-cord reference method at both 1310 nm and 1550 nm. Then run OFDR or OTDR traces at both wavelengths, from both ends, using matched launch and receive leads. Produce an event table listing distance, insertion loss and reflectance for every event. Reconcile the summed event losses against the measured end-to-end insertion loss and against the path loss calculated in TC-02. Retain all raw traces as test evidence.
Pass criteria
Every mated connector pair at or below 0.5 dB insertion loss (0.75 dB absolute maximum). Reflectance at or below −45 dB for UPC and −55 dB for APC interfaces. Fusion splices at or below 0.1 dB. Total measured path loss within the link power budget in Section 3.3, with at least 3 dB of margin retained. Bidirectional averaged event losses agree to within 0.2 dB. No unexplained or undocumented events on the trace.
Notes
This is the test case that justifies the OFDR request in Section 3.4. A standard OTDR has an event dead zone of roughly 0.5–2 m, so closely spaced coupler interfaces merge into a single event and individual faults are masked — OFDR resolves them separately. Compare the 1310 nm and 1550 nm traces: loss that appears at 1550 nm but not 1310 nm indicates a macrobend rather than a connector fault. Confirm the specific reflectance and loss thresholds against the BHP acceptance specification, which takes precedence over the generic values above.
TC-03 — Link establishment, negotiation and FEC baseline
Objective
Confirm the link comes up cleanly at 100 Gbps with matching FEC configuration and no error accumulation at idle.
Tooling
Switch CLI (interface, transceiver and FEC counters).
Method
Clear all interface counters. Bring the link up and confirm both ends report 100 Gbps full duplex. Verify FEC mode matches (RS-FEC mandatory for SR4). Leave the link idle for 30 minutes and re-read counters. Record pre-FEC BER, corrected codewords, uncorrected codewords, CRC errors, runts, giants and symbol errors.
Pass criteria
Link UP, 100 Gbps, no flaps. FEC mode identical at both ends. Zero uncorrected FEC codewords. Zero CRC, input and output errors. Pre-FEC BER at or below 1E-6.
Notes
RS-FEC(528,514) corrects to approximately 2E-4 pre-FEC BER before the correction waterfall. A pre-FEC BER above 1E-5 indicates a marginal path that will fail under thermal drift — investigate before proceeding.
5.2 Throughput and performance
TC-04 — RFC 2544 throughput (unidirectional)
Objective
Determine the maximum frame rate the link forwards with zero frame loss, across the standard frame size range.
Tooling
Hardware traffic generator running an automated RFC 2544 throughput suite.
Method
Run a binary search for the zero-loss rate at frame sizes 64, 128, 256, 512, 1024, 1280, 1518 and 9216 bytes. Trial duration 60 seconds per frame size (RFC 2544 minimum), resolution 0.1%. Repeat in the reverse direction. Record achieved fps, throughput in Gbps and percentage of theoretical line rate against Section 4.
Pass criteria
100% of line rate with zero loss at 512 bytes and above. At or above 98% of line rate at 64 and 128 bytes. Results consistent in both directions to within 0.5%.
Notes
RFC 2544 was written for lab characterisation of a device under test. It is used here as a repeatable acceptance benchmark, not as a production traffic profile.
TC-05 — Bidirectional full-duplex line rate
Objective
Confirm the link sustains full line rate simultaneously in both directions — 200 Gbps aggregate.
Offer 100% line rate simultaneously in both directions at frame sizes 64, 512 and 1518 bytes. Hold each for 5 minutes. Monitor FEC and interface counters throughout and read transceiver temperature at the start and end of each run.
Pass criteria
Zero frame loss in both directions. Zero uncorrected FEC codewords and zero CRC errors. Transceiver temperature remains within vendor operating range.
Notes
This is where undersized optics cooling and marginal power delivery reveal themselves. Watch transceiver temperature closely.
TC-06 — RFC 2544 latency
Objective
Measure the forwarding latency introduced by the link and the two switch ports.
Tooling
Hardware traffic generator with hardware timestamping (sub-microsecond resolution).
Method
At the zero-loss throughput rate determined in TC-04, measure latency at each frame size. Run 20 iterations of 120 seconds each, per RFC 2544. Record minimum, average, maximum and 99.9th percentile. Note whether switches are store-and-forward or cut-through, as this materially changes the expected figure.
Pass criteria
Latency consistent with vendor published figures for the platform and forwarding mode. Maximum latency no greater than three times the average. No latency outliers correlating with FEC correction events.
Notes
Add approximately 5 µs per kilometre of fibre for propagation delay on LR4 spans.
TC-07 — Frame delay variation (jitter)
Objective
Confirm delay variation is low enough to support latency-sensitive traffic such as storage, voice and video.
Offer a constant-rate stream at 90% of line rate for 15 minutes at 512 and 1518 byte frame sizes. Record the inter-packet delay variation distribution.
Pass criteria
Average frame delay variation below 10 µs. 99.9th percentile below 50 µs. No sustained upward drift over the measurement period.
TC-08 — RFC 2544 frame loss rate
Objective
Characterise how the link degrades when offered load exceeds the zero-loss threshold.
Tooling
Hardware traffic generator.
Method
Offer load at 100%, 90%, 80%, 70%, 60% and 50% of line rate at each frame size, 60 seconds per step. Plot the resulting loss curve.
Pass criteria
Zero loss at every step up to and including 100% of line rate for frames of 512 bytes and above. Loss curve degrades gracefully and predictably, with no discontinuities or cliff edges below 100%.
TC-09 — Back-to-back frames and burst tolerance
Objective
Determine the maximum burst of minimum-gap frames the link absorbs without loss.
Transmit bursts of frames at minimum inter-frame gap, increasing burst length until loss occurs. Repeat 50 times per frame size and record the average maximum burst length in frames and in bytes.
Pass criteria
Burst tolerance at or above the vendor published port buffer depth. Results repeatable to within 10% across iterations.
TC-10 — MTU and jumbo frame validation
Objective
Confirm the configured MTU is honoured end to end and that oversized frames are correctly dropped.
Tooling
Traffic generator, or host with ping and DF bit set.
Method
Configure the platform maximum MTU (commonly 9216 bytes) on both interfaces. Transmit frames at exactly the MTU, one byte below, and one byte above. From a host, run ping with the do-not-fragment bit set at increasing sizes to find the effective path MTU.
Pass criteria
Frames at and below the configured MTU forward without loss or fragmentation. Frames above the MTU are dropped and counted as giants. Effective path MTU matches the configured value on both ends.
Notes
MTU accounting differs between vendors — some count the L2 header and FCS, some do not. Verify empirically rather than trusting the configured number.
TC-11 — Buffer, microburst and incast behaviour
Objective
Confirm the link and egress port handle oversubscription and bursty traffic without excessive loss.
Tooling
Hardware traffic generator with multiple source ports, or multiple hosts.
Method
Generate a many-to-one incast pattern where aggregate offered load exceeds 100 Gbps on the egress port. Test at 2:1 and 4:1 oversubscription for 5 minutes each. Then generate microbursts — short spikes to 100% line rate against a 40% background load. Record output discards, queue depth and any pause frame or PFC activity.
Pass criteria
Loss occurs only when offered load genuinely exceeds egress capacity, and is proportionate. Flow control behaves as configured. No head-of-line blocking observed on unrelated ports.
Notes
If the link will carry storage or RDMA traffic, extend this test to validate PFC and ECN marking behaviour.
5.3 Host-based validation (Phase B)
TC-12 — Multi-stream host throughput
Objective
Validate achievable throughput from real hosts, reflecting what applications will actually see.
Tooling
iperf3 (multiple parallel instances) or preferably a DPDK-based generator.
Method
A single iperf3 process will not saturate 100GbE — it is single-threaded and CPU bound at roughly 20–40 Gbps. Run 8 to 16 parallel iperf3 server and client instances pinned to separate CPU cores on the correct NUMA node for the NIC. Enable jumbo frames, increase socket buffers, and confirm the PCIe slot provides sufficient bandwidth. Run for 300 seconds and sum the results.
Pass criteria
Aggregate throughput at or above 90 Gbps TCP, or at or above 95 Gbps UDP, with loss below 0.001%. A shortfall here is far more likely to be a host tuning limit than a network fault.
Notes
Record CPU utilisation per core. If cores are saturated, the result measures the host, not the link — escalate to hardware-based testing before recording a failure.
TC-13 — RFC 6349 TCP goodput
Objective
Establish achievable TCP goodput against the theoretical maximum for the path.
Tooling
RFC 6349 capable tester, or iperf3 with a calculated window size.
Method
Measure baseline round-trip time. Calculate the bandwidth-delay product and required TCP window size. Confirm the path MTU. Run the RFC 6349 test and record TCP efficiency, buffer delay percentage and the transfer time ratio.
Pass criteria
TCP efficiency at or above 99.5%. Transfer time ratio at or below 1.05. Buffer delay percentage below 10%.
Notes
For a 10 km LR4 span the bandwidth-delay product is substantial — window scaling must be enabled or results will be meaningless.
5.4 Stress and endurance
TC-14 — Optical margin stress test
Objective
Confirm the link retains usable margin and degrades predictably as optical power falls.
With a constant 50% line rate load running, insert a variable attenuator and increase attenuation in 0.5 dB steps. At each step record Rx power, pre-FEC BER, corrected and uncorrected codewords, and frame loss. Continue until the link errors, then restore.
Pass criteria
At least 3 dB of usable margin above the point where uncorrected FEC codewords first appear. Degradation is gradual, and the link recovers fully and automatically when attenuation is removed.
Notes
Optional but strongly recommended for LR4 spans and any link crossing third-party infrastructure. This is the test that predicts whether the link survives a hot summer or a slightly dirty future patch.
TC-15 — Sustained soak test
Objective
Prove stability under continuous load over an extended period and across a full thermal cycle.
Tooling
Traffic generator with a mixed frame size profile; scripted counter polling.
Method
Run a mixed frame size profile (IMIX or equivalent) bidirectionally at 80% of line rate for a minimum of 24 hours, ideally 72 hours. Poll interface counters, FEC counters and transceiver DOM readings every 5 minutes and log to file. Include at least one full day/night cycle so thermal variation is captured.
Pass criteria
Zero link flaps. Zero uncorrected FEC codewords. Zero CRC or input errors. Pre-FEC BER stable and at or below 1E-6 throughout. Rx power variance under 1 dB across the period. No upward trend in corrected codeword rate.
Notes
This is the most valuable single test in the plan. Marginal optics and poor terminations frequently pass a 60-second test and fail overnight.
6. Results Record
Complete during execution. Attach raw tester output, counter logs and DOM captures as appendices.
ID
Test
Priority
Result
Evidence / comments
TC-01
Fibre endface inspection
Mandatory
TC-02
Optical power and DOM baseline
Mandatory
TC-02a
Fibre path characterisation (OFDR / OTDR)
Mandatory
TC-03
Link establishment and FEC baseline
Mandatory
TC-04
RFC 2544 throughput
Mandatory
TC-05
Bidirectional full-duplex line rate
Mandatory
TC-06
RFC 2544 latency
Mandatory
TC-07
Frame delay variation
Recommended
TC-08
RFC 2544 frame loss rate
Recommended
TC-09
Back-to-back burst tolerance
Recommended
TC-10
MTU and jumbo frame validation
Mandatory
TC-11
Buffer, microburst and incast
Recommended
TC-12
Multi-stream host throughput
Conditional
TC-13
RFC 6349 TCP goodput
Conditional
TC-14
Optical margin stress test
Recommended
TC-15
Sustained soak test
Mandatory
6.1 Throughput results — record per frame size
Frame size
Target fps
Achieved fps
% line rate
Loss (%)
Avg latency (µs)
64
148,809,524
128
84,459,459
256
45,289,855
512
23,496,241
1024
11,973,180
1280
9,615,385
1518
8,127,438
9216
1,353,398
7. Interpreting Results and Common Failure Modes
Symptom
Most likely cause
Action
Link will not come up
FEC mismatch between ends; mismatched transceiver types; incorrect MPO polarity; unsupported optic on the platform
Verify FEC mode and optic type match; check the compatibility matrix; test MPO polarity
Link up but high pre-FEC BER
Contaminated or damaged endface; excessive path loss; bent or stressed fibre
Re-inspect and clean; re-measure path loss; check bend radius
One or two lanes low on Rx power
Partially seated MPO; damaged fibre in the ribbon; dirty individual fibre
Reseat and re-inspect all 12 fibres; swap the patch lead
Passes at 1518 B, fails at 64 B
Switch fabric or forwarding engine limitation, not optical
Confirm against vendor datasheet packets-per-second figure; may be expected behaviour
Host test caps at 20–40 Gbps
Single-threaded generator; CPU bound; wrong NUMA node; insufficient PCIe lanes
Use parallel streams pinned across cores; verify NIC NUMA affinity; check PCIe link width
Errors appear only under load
Thermal issue in the transceiver or cage; marginal power delivery; inadequate airflow
Monitor DOM temperature during load; check chassis airflow and fan status
Errors appear after hours of stability
Thermal drift on a link with insufficient optical margin
Run TC-14 to quantify remaining margin; likely needs a cleaner or shorter path
Corrected codewords rising over time
Progressive degradation — contamination, connector wear or a failing laser
Do not accept the link. Re-terminate or replace the transceiver and repeat TC-02 and TC-15
8. Risks, Safety and Rollback
8.1 Safety
Never look directly into a fibre endface or an active transceiver port. 100GBASE-LR4 operates at 1310 nm, which is invisible and can cause permanent eye damage.
Disable the far-end transmitter or unplug the transceiver before inspecting an endface.
Fit dust caps to all unused ports and unplugged leads immediately.
8.2 Risks
Risk
Impact
Mitigation
Load testing saturates a shared fabric
Production traffic on other ports degraded
Test within an approved window; verify the port is isolated from production VLANs before generating load
Repeated transceiver insertion damages the cage
Intermittent faults introduced during testing
Minimise insertion cycles; handle by the body, never the fibre
Test traffic leaks into the production network
Broadcast storm or MAC table pollution
Place test ports in an isolated VLAN with no uplink; confirm before starting
Soak test runs unattended and fails silently
Window lost, retest required
Script counter polling with alerting; check in at defined intervals
Monitoring alerts fire during load generation
Unnecessary incident escalation
Suppress alerting on the ports under test for the duration of the window
8.3 Rollback
Stop all traffic generation and disconnect tester ports.
Shut down the interfaces under test at both ends.
Restore the pre-change interface configuration from the backup taken at Section 3.2.
Remove the test VLAN and any temporary configuration.
Re-enable monitoring and alerting on the affected ports.
Confirm production traffic paths are unchanged and error-free before closing the window.
9. Acceptance and Sign-off
The link is accepted into production service only when all mandatory test cases return a Pass and the soak test completes with zero uncorrected FEC codewords and zero CRC errors.