0

Is there any way to divide outgoing channel between different traffic class, for example, with relation 30/70, if I don't know channel width? HTB requires exact numbers, and CBQ too.

Selivanov Pavel
  • 2,206
  • 3
  • 26
  • 48
  • 2
    If you do not know the width, what should 30% or 70% relate to? To the number of connections? Traffic class is QoS-class or do you use different VLANs on that channel? – Nils Sep 04 '13 at 21:05
  • Are you working with a link with variable bandwidth? Or trying to write a script that will work regardless of the (fixed) bandwidth of the system running it? – Dan Pritts Sep 10 '13 at 18:52

2 Answers2

4

I thought this would do the trick but unless it has some mechanism controlling its outflow, the queue empties too fast and it never exhibits drr weightings (not exactly sure why this is). As soon as there is selection pressure, however, it works exactly as it is supposed to.

tc qdisc add dev eth0 handle 1 root drr
tc class add dev eth0 parent 1: classid 1:1 drr quantum 600  # 30%
tc class add dev eth0 parent 1: classid 1:2 drr quantum 1400 # 70%
tc class add dev eth0 parent 1: classid 1:3 drr # everything else....

tc qdisc add dev eth0 parent 1:1 handle 10: sfq perturb 120 limit 1024
tc qdisc add dev eth0 parent 1:2 handle 20: sfq perturb 120 limit 1024
tc qdisc add dev eth0 parent 1:3 handle 30: sfq perturb 120 limit 1024

# outgoing to port 8111 will go to 30% queue, 8112 will go to 70% queue...
tc filter add dev eth0 protocol ip parent 1:0 prio 1 u32 match ip dport 8111 0xffff classid 1:1
tc filter add dev eth0 protocol ip parent 1:0 prio 1 u32 match ip dport 8112 0xffff classid 1:2
# a default so my connection doesn't die when doing this.
tc filter add dev eth0 protocol all parent 1:0 prio 2 u32 match u8 0x00 0x00 at 0 classid 1:3

If you wrap it all in an HTB with a 800kbps flow rate, you get a nice 70KB/30KB split, tested by running parallel instances of pv -ar /dev/zero | nc differenthost port.

Incidentally, don't try to test this from a remote machine at line speed without out-of-band control mechanisms. (oops)

Maybe this answer will help anyway.

Edit: Searching the internet, it turns out there is no other linux deficit round robin (DRR) qdisc example, other than the manpage-included one, which seemed incomplete to me, omitting the filter rules (and caveats thereof) and being generally non-obvious.

What is DRR

Deficit round robin is a scheduling algorithm which can be imagined as a number of parallel queues. The scheduler iterates these queues in sequence. Each time a queue comes up, if the queue is not empty, it checks the packet size of the next packet in the queue against a number it tracks per queue called the deficit counter. If the packet size is smaller than the deficit counter, DRR removes the packet from the queue and sends it along to be put on the wire and subtracts its size from the deficit counter (hence the deficit). It then repeats with the next packet in the queue until either the packet is bigger than the deficit counter or the queue is empty. If the queue is not empty when the loop ends, the queue-specific value quantum is added to the deficit counter before moving on to the next queue.

This is actually not all that different from HTB, except HTB limits the maximum amount to be added to a given queue in any particular time interval (Pretty sure linux measures this in bytes-per-jiffie, though that might not be true anymore with tickless) and also has a deficit counter for all queues combined (which is also filled with some number of byte per time interval)

Example usage description

So what the above example does is create a drr qdisc as root and add 3 queues to it, one with a quantum of 600 bytes per pass, a second with 1400 bytes per pass, and a third which defaults to the MTU size. For reasons I will explain in a bit doesn't matter too much what the values are, only what the ratio is.

Because I like being fair, I added some sfqs to the leaves; this is not necessary because if you don't, it should default to pfifo fast (or whatever your default scheduler is, I think. I would have to read the sch_drr.c source to be 100% sure). It also doesn't change my tests since I was using a single TCP connection per port.

tc filter rules for testing description

When I was testing the above, it actually gave me some trouble in the filter rules. drr doesn't have a default flow like a lot of the other qdiscs have, nor does it allow you to assign one as a qdisc option (that I'm aware of, if it does, edit this answer). So it's rather amusing in a frustrating way when it starts dropping your packets on the floor because it can't queue things like arp requests or replies and it doesn't say why your interface is spontaneously down.

So the first two rules perform the test activity, match the tcp/udp (it's in the same spot) dport against 8111 and 8112, putting matches in the appropriate queue, stopping matching if an appropriate rule is found.

The third rule says "match any protocol where the first byte (0 offset) matches 0x0 with a mask of 0" and put it in the third queue. It is prio 2 to be evaluated after the first-pass guys and then catch any unmatched packets. If you know a better way to make a default classid, I would surely like to know.

Quantum selection

As I mentioned earlier, the actual values don't matter so much as the ratio, though it'll probably make the queues more bursty (by which I mean stick in one queue for X packets per pass) if they are bigger than they should be, and use more cpu time if they are smaller. For that reason, I chose values near the same order of magnitude as MTU whose relation to the 30/70 target ratio is visible. Since quantum determines the fill-rate per pass, and thus number of bytes per pass, the ratio of quantums will be the ratio of bytes per pass relative to each other. If one queue is empty, the others don't so much absorb its bandwidth as just spend more time skipping the empty queue and filling themselves.

The relative dearth of documentation compared to HTB or CBQ suggests to me that DRR isn't a particularly popular qdisc, so unfortunately, if you do decide to go this route, I expect support will be pretty sparse, which makes it hard to recommend for use.

Andrew Domaszek
  • 5,163
  • 1
  • 15
  • 27
  • Good idea, thank you :) I'll test this as soon as I have any time – Selivanov Pavel Sep 11 '13 at 12:07
  • Why is HTB around this all required? Didn't work for me. But, still, you made great job. I wanted to give you more bounty, but SF says I can't do it for 23 hours – Selivanov Pavel Sep 12 '13 at 17:03
  • Didn't work = both pv instances show equal performance. Kernel module sch_drr is loaded – Selivanov Pavel Sep 12 '13 at 17:09
  • I'm not sure why it doesn't work using the link queue instead of some other arbitrary queue. I'm thinking it's because a given queue will be empty while it is waiting for TCP acks to come back; qdiscs don't really understand waiting for packets. Another notable assumption that must be made to use something like DRR is bandwidth congestion is occurring somewhere on a route common to everyone under the qdisc, and that's not always true. – Andrew Domaszek Sep 13 '13 at 12:32
  • Here is new question about DRR: http://serverfault.com/questions/539045/linux-qos-tc-drr-qdisc-does-not-work Any ideas? – Selivanov Pavel Sep 15 '13 at 11:39
0

Did you check this link ?

http://www.tldp.org/HOWTO/html_single/Traffic-Control-HOWTO/#r-unknown-bandwidth

>     8.2. Handling a link with a known bandwidth
>     
>     HTB is an ideal qdisc to use on a link with a known bandwidth, because the innermost (root-most) class can be set to the maximum
> bandwidth available on a given link. Flows can be further subdivided
> into children classes, allowing either guaranteed bandwidth to
> particular classes of traffic or allowing preference to specific kinds
> of traffic
Nikolaidis Fotis
  • 2,032
  • 11
  • 13
  • Yes, I'v read LARTC. HTC is for **known bandwidth** :( I need solution for unknown bandwidth – Selivanov Pavel Sep 04 '13 at 14:31
  • Sorry. Wrong copy-paste 8.3. Handling a link with a variable (or unknown) bandwidth In theory, the PRIO scheduler is an ideal match for links with variable bandwidth, because it is a work-conserving qdisc (which means that it provides no shaping). In the case of a link with an unknown or fluctuating bandwidth, the PRIO scheduler simply prefers to dequeue any available packet in the highest priority band first, then falling to the lower priority queues – Nikolaidis Fotis Sep 04 '13 at 14:47
  • PRIO gives only preemptive solution: traffic of class 1 is always higher priority than traffic class 2. If class 1 uses 100% of channel, class 2 is fully displaced. I want to divide channel with unknown width in some predefined proportion, for example 30/70. But as I can see, it's impossible :( – Selivanov Pavel Sep 04 '13 at 18:19
  • You can follow another method but it's more tricky. you can make a custom script to mark the packets {70% label A, 30% label B}. It's up to you to make the logic for that. After that, you can use the labels for QOS in your switch. You can take some ideas from http://unix.stackexchange.com/questions/2259/default-mark-for-packets-using-iptables – Nikolaidis Fotis Sep 05 '13 at 01:01
  • If I have 70% of packets for type A and 30% packets for type B, than I need no shaping, just SFQ. But if I have 90% of packets type A, and 10% of packets type B, how can I remark them to be 70/30? – Selivanov Pavel Sep 05 '13 at 11:13