Mikrotik Queues, Bandwidth Throttling, and Bursting

Bandwidth Throttling, a system you would think would be easier to pull off. Hoping you find this article helpful.

I’m installing a CCR-1036-12G-4S for a minor league baseball stadium, and am expecting about 600-800 devices to be connected during a game. The stadium has a symmetrical (up/down) gigabit fiber line, and I have to ration bandwidth properly between them. This means bridges to isolate devices from seeing where they shouldn’t, bandwidth pools / cap-limits, and per-device throttling. Fun stuff!

 

I spent about 4 hours practicing to get Queue Trees and Packet Marketing working, much to my frustration. I eventually found they are intended for ISPs to deliver bandwidth to client sites, as in, building internet connections, not individual devices.

 

I found the tool I needed in Simple Queues, applied to an interface with a PCQ (Per Connection [device] queue), and using Queue Types as my rate limiters.

 

Oh boy, what a fun router, 16Gbps of raw traffic, 4x SFP ports, 12x GigE ports, a fun little interactive touchscreen, and a 36-core CPU to number crunch it all, all for under $1k, wowzers, eat that Cisco/Sonicwall/Palo Alto Networks, hah!

Part 1 – The Bridges

Bridges are used to connect one interface to another. An interface could be an ethernet port, fiber-port, antenna, heck, even a VLAN. Just as a switch connects together ethernet cables in the physical world, so a bridge connects different interfaces in the digital world. We are using bridges to manage the traffic across multiple parts of the stadium. Whether a guest is on the ground-floor Wireless Access Points, or on the third-floor, they should be sharing the same total pool of bandwidth. However, within this pool, each device should only be allowed so much of it.

Let’s pretend we have two bridges, and they should not be able to talk to each other.

  • bridge-stadium
    • Ports ETH2-4
      • ETH2 = Staff
      • ETH3 = Broadcasters
      • ETH4 = Ticket Scanners
  • bridge-guest
    • Ports ETH10-12
      • ETH10 = First Floor
      • ETH11 = Second Floor
      • ETH12 = Third Floor

Part 2 – The Simple Queues

We will make one queue for each bridge.

Note – The terms upload and download are from the perspective of the router — not the client device! For example, if you want your client to only download at 20Mbps. The router will upload 20Mbps to your client. The terms are reversed in queue menus!

  •  queue-stadium
    • 100 devices.
    • 300M Download / 100M Upload Bandwidth Pool  (Called the Max Limit of the Target Interface/Bridge on the General Tab)
      • 20M Max Limit – Download / 10M Upload per client
        • 40M Burst Limit – Download / 20M Upload per client
  • queue-guest
    • 500 devices.
    • 500M Download / 400M Upload Bandwidth Pool (Called the Max Limit of the Target Interface/Bridge on the General Tab)
      • 10M Max Limit – Download / 5M Upload per client
        • 20M Burst Limit – Download / 10M Upload per client

Mikrotik Simple Queues 1

Part 3 – Queue Types

The Queue Types are your definitions of speed limits. They are applied on a per-client basis.

Terminology

  • Rate = Speed in bit/s (the one you really want)
  • Limit = Packet limit in KiB (don’t touch unless you know what you are doing).
  • Total Limit = Packet buffer limit in KiB (don’t touch unless you know what you are doing).
  • Burst Threshold = The speed limit, once passed, the client will be considered to have started their burst timer.
    • Note that a burst time of 30s, does not mean the client will download at burst speed for 30s in real-time.
    • It means the calculation for burst speed will be spread across 30s of calculation.
    • For example, a 30s burst time calculation, may yield 10-12s of real-time high-speed. The router is averaging 30s of the speed that client would otherwise use, and crunch it down until they use it all up, then back to the max-limit they go. Just search “mikrotik burst spreadsheet” to find a graphical way of understanding it.
  • Max Limit = After burst is used up, the maximum speed the client is allowed to go.

For our scenario, we will need four queue type definitions:

  • Stadium
    • 20M-40B-Download (Called the rate in bits/s)
      • 20Mbps max limit, 40Mbps burst limit, for stadium client download/receive traffic.
    • 10M-20B-Upload (Called the rate in bits/s)
      • 10Mbps max limit, 20Mbps burst limit, for stadium client upload/send traffic.
  • Guest
    • 10M-20B-Download (Called the rate in bits/s)
      • 10Mbps max limit, 20MBps burst limit, for guest client download/receive traffic.
    • 5M-10B-Upload (Called the rate in bits/s)
      • 5Mbps max limit, 10Mbps burst limit, for stadium client upload/send traffic.

Below is an image of the Queue Types listed above, as programmed in Winbox.

Note – You MUST use the classifiers if you want per-device bandwidth throttling. If you leave the classifiers blank, then these limiters are spread equally across all devices, making effectively the same as the bandwidth pool, which you really don’t want.

Note 2 – Fasttrack can bypass your throttling queues, depending on where it is located in your firewall rules (nearing the top, the more likely your rules will be skipped). For testing purposes to better understand this process, disable your fasttrack firewall rule.

Classifiers

Remember, perspective of the router for these definitions. (In parenthesis, is the client’s perspective / wording everyone else uses).

  • Source Address (Client Download/Receive)
  • Destination Address (Client Upload/Send)

Mikrotik Queue Types 2

Part 4 – Assign the Queue Types to the Simple Queues

In the advanced tab of your simple queue, change the Queue Types.

Remember, the definitions are reversed, the Router’s Target, is your client device. So the Target Upload = Client Download. Target Download = Client Upload). Leave the “Limit At” fields at unlimited, unless you know what you are doing.

 

Another benefit of the Advanced field, is the “Priority” Field, also known by the rest of the world as QoS — Quality of Service. On a scale of 1-8, where 1 is the highest/most-important priority, and 8 is the lowest/least-important priority.

Generally VOIP traffic would be a 1, and bulk file-sharing/downloading would be priority 8 (last).

Mikrotik Simple Queues 0

Part 5 – Testing

Gee Dan, this guide was very concise, now time to test if it really works!

Enter: iPerf3.

iPerf is a tool that give you the no-bs, second to second speeds you are running at. You could also use speedof.me  or speedtest.net to get “average” speeds or to throw load on the connection, the downside is they average out their numbers across the whole test.

 

I recommend for your testing, to have your OWN iPerf3 server, rather than using an internet-based server. iPerf3 will only allow one client to connect at a time. Pretty crazy right? A publicly available internet testing service, that only one person can test at a time….. not to mention the bandwidth caps, they won’t let you saturate their entire 10Gbit line, sorry :-(.

 

To test that the pool limit evenly splits speed across devices, I’ve had the best experience running iPerf to my own server on the WAN side of the router, and at the same time, an internet-based server for my second client to test. You could also just use two PCs on the WAN for two personal iPerf servers you can max out.

program.exe, -format (m = megabit) (M = MegaByte), -port (5201 TCP), -client (host), -time (120 seconds)

  • Download from an internet server for 15 seconds (Max speed of 25Mbps receive, bummer).
    • iperf3.exe -f m -p 5201 -c iperf.he.net -t 15
  • Send to an internet server for 15 seconds (-R = reverse traffic flow)
    • iperf3.exe -f m -p 5201 -c iperf.scottlinux.com -t 15 -R
  • Download from a local server on your WAN for 120 seconds (to test full gigabit speed)
    • iperf3.exe -f m -p 5201 -c 10.10.200.123 -t 120

I recommend practicing with Max Limits in your queue-types first, before moving on to setting Burst Limits.

Play with it, spend a few hours fiddling with the settings until you understand it.

That’s all folks, let me know if this guide was helpful, and have a brilliant week 🙂

Leave a Reply

Your email address will not be published. Required fields are marked *