[bmwg] Packet generator precision

[Paul Emmerich]

Hi,

we've had a (very) short discussion on reliability and precision of software packet generators at the session at IETF 100.

I've wanted to revive this discussion because I might be at IETF 102 (travel grant application pending, fingers crossed),
if anyone here is still interested in this topic.

To get the discussion started, here's our research paper from last year where we looked at rate control of packet generators:
https://www.net.in.tum.de/fileadmin/bibtex/publications/papers/comparison_of_software_packet_generators.pdf

I want to draw your attention to Figure 1 on page 2. It shows a measurement of the median latency of packet forwarding with Open vSwitch
under increasing load. We repeated this test several times and modified the burst size of the packet generator while keeping the average
packet rate the same. The base line is true CBR traffic.

The result is an increase of the measured latency of up to 400% even though we only changed the spacing between packets on the wire.

Why is this important? Packet generators often generate small bursts in order to increase performance.
Usually the default is bursts of 16 to 256 packets and that can ruin latency tests in some cases.

The short discussion at IETF 100 was whether it might be helpful to have some kind of test for packet generators to measure their precision.


Paul


--
Chair of Network Architectures and Services
Department of Informatics
Technical University of Munich
Boltzmannstr. 3
85748 Garching bei MÃŒnchen, Germany

[MORTON, ALFRED C (AL)]

Hi Paul,

Thanks for your note and re-starting this discussion.

Post-IETF-100, BMWG had the opportunity to review the traffic
generator specifications in ETSI NFV draft specification, TST009.
https://docbox.etsi.org/ISG/NFV/Open/Drafts/TST009_NFVI_Benchmarks
I think what you have in mind goes beyond this specification in
several ways.

The Working group is in the luxurious position of considering
new work, following the approval of our new charter. This gives
us flexibility to take-up obviously related items like generator
precision requirements, as long as there is interest in the
working group.

I'll be re-reading your paper in the next few days, and
good luck with your travel grant proposal!

regards,
Al
bmwg co-chair

-----Original Message-----
Sent: Thursday, June 14, 2018 4:11 PM
Subject: [bmwg] Packet generator precision
Hi,
we've had a (very) short discussion on reliability and precision of
software packet generators at the session at IETF 100.
I've wanted to revive this discussion because I might be at IETF 102
(travel grant application pending, fingers crossed),
if anyone here is still interested in this topic.
To get the discussion started, here's our research paper from last year
https://www.net.in.tum.de/fileadmin/bibtex/publications/papers/comparison_
of_software_packet_generators.pdf
I want to draw your attention to Figure 1 on page 2. It shows a
measurement of the median latency of packet forwarding with Open vSwitch
under increasing load. We repeated this test several times and modified
the burst size of the packet generator while keeping the average
packet rate the same. The base line is true CBR traffic.
The result is an increase of the measured latency of up to 400% even
though we only changed the spacing between packets on the wire.
Why is this important? Packet generators often generate small bursts in
order to increase performance.
Usually the default is bursts of 16 to 256 packets and that can ruin
latency tests in some cases.
The short discussion at IETF 100 was whether it might be helpful to have
some kind of test for packet generators to measure their precision.
Paul
--
Chair of Network Architectures and Services
Department of Informatics
Technical University of Munich
Boltzmannstr. 3
85748 Garching bei München, Germany

[Paul Emmerich]

Post by MORTON, ALFRED C (AL)
Hi Paul,
Thanks for your note and re-starting this discussion.
Post-IETF-100, BMWG had the opportunity to review the traffic
generator specifications in ETSI NFV draft specification, TST009.
https://docbox.etsi.org/ISG/NFV/Open/Drafts/TST009_NFVI_Benchmarks
I think what you have in mind goes beyond this specification in
several ways.

That looks interesting, I'll have a look at that.

Post by MORTON, ALFRED C (AL)
I'll be re-reading your paper in the next few days, and
good luck with your travel grant proposal!

I'll be at IETF 102, ISOC invited me :)
I can give a short talk/summary at the meeting if you like.


Paul

Post by MORTON, ALFRED C (AL)
regards,
Al
bmwg co-chair

-----Original Message-----
Sent: Thursday, June 14, 2018 4:11 PM
Subject: [bmwg] Packet generator precision
Hi,
we've had a (very) short discussion on reliability and precision of
software packet generators at the session at IETF 100.
I've wanted to revive this discussion because I might be at IETF 102
(travel grant application pending, fingers crossed),
if anyone here is still interested in this topic.
To get the discussion started, here's our research paper from last year
https://www.net.in.tum.de/fileadmin/bibtex/publications/papers/comparison_
of_software_packet_generators.pdf
I want to draw your attention to Figure 1 on page 2. It shows a
measurement of the median latency of packet forwarding with Open vSwitch
under increasing load. We repeated this test several times and modified
the burst size of the packet generator while keeping the average
packet rate the same. The base line is true CBR traffic.
The result is an increase of the measured latency of up to 400% even
though we only changed the spacing between packets on the wire.
Why is this important? Packet generators often generate small bursts in
order to increase performance.
Usually the default is bursts of 16 to 256 packets and that can ruin
latency tests in some cases.
The short discussion at IETF 100 was whether it might be helpful to have
some kind of test for packet generators to measure their precision.
Paul
--
Chair of Network Architectures and Services
Department of Informatics
Technical University of Munich
Boltzmannstr. 3
85748 Garching bei MÃŒnchen, Germany

--
Chair of Network Architectures and Services
Department of Informatics
Technical University of Munich
Boltzmannstr. 3
85748 Garching bei MÃŒnchen, Germany

[MORTON, ALFRED C (AL)]

...

Post by Paul Emmerich

Post by MORTON, ALFRED C (AL)
good luck with your travel grant proposal!

I'll be at IETF 102, ISOC invited me :)
I can give a short talk/summary at the meeting if you like.

[acm]
Great, thanks. Since all of our proposals end up in
Internet drafts at some point, think about how the doc
might be organized. That way if there is interest,
you could possibly divide the writing assignments among
an author-group, if formed.

Al

Post by Paul Emmerich
Paul

Post by MORTON, ALFRED C (AL)
regards,
Al
bmwg co-chair

-----Original Message-----
Sent: Thursday, June 14, 2018 4:11 PM
Subject: [bmwg] Packet generator precision
Hi,
we've had a (very) short discussion on reliability and precision of
software packet generators at the session at IETF 100.
I've wanted to revive this discussion because I might be at IETF 102
(travel grant application pending, fingers crossed),
if anyone here is still interested in this topic.
To get the discussion started, here's our research paper from last year

https://www.net.in.tum.de/fileadmin/bibtex/publications/papers/comparison_

Post by MORTON, ALFRED C (AL)

of_software_packet_generators.pdf
I want to draw your attention to Figure 1 on page 2. It shows a
measurement of the median latency of packet forwarding with Open

vSwitch

Post by MORTON, ALFRED C (AL)

under increasing load. We repeated this test several times and modified
the burst size of the packet generator while keeping the average
packet rate the same. The base line is true CBR traffic.
The result is an increase of the measured latency of up to 400% even
though we only changed the spacing between packets on the wire.
Why is this important? Packet generators often generate small bursts in
order to increase performance.
Usually the default is bursts of 16 to 256 packets and that can ruin
latency tests in some cases.
The short discussion at IETF 100 was whether it might be helpful to

have

Post by MORTON, ALFRED C (AL)

some kind of test for packet generators to measure their precision.
Paul
--
Chair of Network Architectures and Services
Department of Informatics
Technical University of Munich
Boltzmannstr. 3
85748 Garching bei München, Germany

--
Chair of Network Architectures and Services
Department of Informatics
Technical University of Munich
Boltzmannstr. 3
85748 Garching bei München, Germany