The total outgoing traffic of ftp.fau.de increased from 5.04 PiB in 2021 to 6.57 PiB in 2022, an increase of 30%.

The percentage of IPv6 traffic dropped slightly, it is now at 26% (1.68 PiB), after 27% in 2021, 27% in 2020, 25% in 2019, 22% in 2018, 22% in 2017, 14% in 2016 and 11% in 2015.

There is very little change in the Top 10 overall. Sure, some mirrors got a little more or a little less traffic, and thus changed position in the Top 10, but fluctuations are little and nothing really stands out. There is not a single new entry into the Top 10.

The notable exception seems to be CentOS, which jumped from rank 10 to 2. We’re a bit surprised by that, considering what has happened to this project in the last 2 years (you’ll need to read up on that elsewhere). The likely explanation is probably, that after RedHat turned this into an abomination of the original project, most other mirrors stopped mirroring – and as one of the few remaining mirrors we now see a lot more traffic to the mirror. That won’t last forever though: Newer versions of “CentOS stream” (starting with 9) are distributed through a different mirror network now, and we neither are nor plan to be a part of that. We will only keep the existing mirror running as long as there are still CentOS 7 updates to distribute (supposedly on 2024-06-30, but after spontanously pulling the plug on CentOS 8 with just a few months notice, would you really trust that company to keep their promises for CentOS 7?).

We will end this post by explaining why “gimp” is almost the only mirror whose total traffic (in TB) dropped from 2021 to 2022: In September 2022, we noticed that we were accidently listed twice in gimps automatic mirror redirector, meaning we served twice as many downloads as the other mirrors. Since that error has (sadly) been corrected now, we’re getting a lot less traffic from gimp.

The total outgoing traffic of ftp.fau.de increased from 4.12 PiB in 2020 to 5.04 PiB in 2021, an increase of 22%.

The percentage of IPv6 traffic stagnated, with 27% (1.36 PiB), after 27% in 2020, 25% in 2019, 22% in 2018, 22% in 2017, 14% in 2016 and 11% in 2015.

There is very little change in the Top 10 overall. Sure, some mirrors got a little more or less traffic, and thus changed position in the Top 10, but fluctuations are little and nothing really stands out.

The biggest change is with mint/iso, which already was in the Top 10 on Rank 2 last year, but assured Rank 1 in 2021 quite clearly, with a large gap to Rank 2. We are not sure what caused the traffic increase for that mirror, our best guess is that they changed their download-page to prefer Torrents with webseeds, thus we are now seing a certain percentage of each download.

It seems that I have not posted the traffic stats for 2020 yet. Well, better late than never, so here they are.

The total outgoing traffic of ftp.fau.de increased from 3.52 PiB in 2019 to 4.12 PiB in 2020, an increase of 17%.

The percentage of IPv6 traffic increased a tiny bit again, to 27% (1.13 PB), after 25% in 2019, 22% in 2018, 22% in 2017, 14% in 2016 and 11% in 2015.

There is some change in the Top 10 overall, with some mirrors traffic volumes (and thus placement in the Top 10) changing significantly, but most staying pretty much the same.

The most eye-catching change is the Gimp-mirror. We have been mirroring gimp since 2015, but this mirror never generated a lot of traffic. This changed rapidly in April 2020: Instead of less than 10 gigabytes per day, outgoing traffic jumped to over 3000 gigabytes a day average. The reason for this seems to be that they changed their download-page: While it would previously just give you a list of mirrors and ask you to select one manually, they would then send you to a mirror chosen randomly from a list of known-stable mirrors – which happened to include us. Later that year, traffic went down again, because they added more mirrors to said list – but around 800 gigabytes a day average still is quite good.

And as already anticipated last year, the mirror for F-Droid, a community-maintained Android software repository only hosting free/libre software (sort of an alternative to Googles “Play Store”), had no problem to reach the Top 10 in 2020. The reason is that since mid-2019 they made their clients use the mirrors instead of central servers far more often. As a result, this is now our 5th most requested mirror.

The total traffic of ftp.fau.de increased from 3.22 PB in 2018 to 3.52 PB in 2019, an unusually small increase of only 9%.

The percentage of IPv6 traffic increased a tiny bit again, to 25% (0.87 PB), after 22% in 2018, 22% in 2017, 14% in 2016 and 11% in 2015.

There is little change in the Top 10 overall, with traffic volumes staying pretty much the same, only some mirrors swapped their places in the table.

Our mirror of cdn.media.ccc.de managed to reach the Traffic Top 10 again, but it was a close call. This mirror makes most of its yearly traffic at the end of December / beginning of January, when the recordings of the annual Chaos Communication Congress are put online. On the other hand, the CTAN mirror that was on rank 10 last year did not make the Top 10 this year, missing it by a few TB (106).

A notable new entry that would be on rank 11 is F-Droid, a community-maintained Android software repository only hosting free/libre software (sort of an alternative to Googles “Play Store”). We started mirroring this at the end of 2018, but only recently they added functionality that makes the clients use the mirrors more often. As a result, this mirror has seen some a lot more usage in recent months, and is likely to reach the Top 10 in 2020.

The total traffic of ftp.fau.de increased from 2.51 PB in 2017 to 3.22 PB in 2018, a 28% increase.

Even though the absolute amount of IPv6 traffic increased a bit, its percentage of all traffic stagnated, with 22% (0.72 PB) in 2018, after 22% in 2017, 14% in 2016 and 11% in 2015.

Our mirror of cdn.media.ccc.de is no longer in the Traffic Top 10, it only ranked in 11th place with a measly 94 TB, a large part of that at the end of December / beginning of January, when the recordings of the annual Chaos Communication Congress are put online.

The total traffic of ftp.fau.de increased from 1.89 PB in 2016 to 2.51 PB in 2017, by 33%. 203 TB alone was accounted to one of our newest additions from late may 2017: lineageos, a free and open-source android distribution and successor of cyanogenmod (which was discontinued).

The total IPv6 traffic increased significantly, to 22% (0.55 PB) of all traffic in 2017, from 14% in 2016 and 11% in 2015.

It seems I haven’t posted the traditional annual mirror stats for 2016 yet. Well, lets fix that: Here are the most used mirrors on ftp.fau.de in 2016:

Comparing this list to last year, the first thing one notices is the new entry on rank 3: OSMC generates a steady amount of traffic, with visible peaks whenever they do a release. They were not in the 2015 top ten because we only started mirroring it in Q4 of 2015.

Linux mint dropped from rank 3 last year to rank 5 this year, with much less traffic than last year. And that is despite a slightly different counting that should actually have increased their numbers: We are now summing up the two parts of the mirror, the ISOs and the packages. Not because we want to do that, but for technical reasons – we cannot always distinguish between the two in the stats, and not summing them up would make the stats even more wrong. Now judging from the stats, we must have been dropped from their mirror list for ISO downloads around June of 2016. From there to the end of the year, almost no requests for the Mint ISOs have hit our server. As to why we were dropped, we haven’t got the slightest clue – we got no notification about the removal. We did get a notice about being readded at the beginning of 2017 though.

Last years rank 10, videolan, dropped off the list – it would be on rank 13 this year.

Rank 1 and 2, Qt and kiwix, are really close head to head.

For all other mirrors, they swapped positions here and there, and all of them generated a little more traffic than the year before, but there were no big changes.

Lets take a look at IPv6 traffic only:

With the exception of Linux Mint, where I’ve already explained the reason above, all mirrors had more IPv6 traffic, sometimes significantly more.

This is also visible in the total IPv6 traffic over all mirros: 13.68% of all traffic in 2016 was IPv6, up from 10.54% in 2015.

There are still huge differences in the IPv6 traffic share between the different projects mirrored, and most of the time it isn’t really clear why. One example where it is clear though is cygwin, with an IPv6 share of pretty much 0%: They use a setup-tool that downloads individual packages from the mirrors, and it seems this tool only does IPv4.

People operate mirrors out of different reasons, this could be purely altruistic to support open source communities, or to reduce incoming traffic by providing local clients with an internal package source, or to tilt the in/out ratio of the internet uplink. What ever the reasons are, they are all associated with the usage of outbound traffic and local storage size, in addition to occupation of other hardware and administrative resources. We will consider traffic and storage requirements as the costs and outgoing traffic as the profit (to the general public or an organization) a mirror service provides.

Now looking around at what other mirror operators publish, it is very hard to find useful information to guide the mirror selection. Which means, that if you plan on hosting a new project, you have to rely on guesswork to estimate cost vs profit.

Although cost and profit are terms usually associated with currency based units, we will not estimate them using any such metric. We will consider Gigabytes as the unit for storage and traffic and therefore cost and profit.

By combining logged data from the ~50 mirrors we are currently hosting, we are in the position to quantify both cost and profit for each one of them. Be aware that the profit often depends on the geographic location, since many projects use a topology- and geography-aware load balancer like mirrorbrain.

Lets start of with the incoming traffic cost. The average number of GBs that need to be synced from the upstream mirror to ftp.fau.de per day:

We calculated this by summing up all positive changes in mirror size, which is calculated and logged on a daily basis. Since we sync up to hourly, files that get synced and deleted within a day are not covered, nor are files that change.

How about storage costs? Since most mirrors stay around the same size, with a slight increase taking the average size works for most. But some have a tendency to constantly increase (or even decrease), this won’t be correct in all cases.

Let’s have a look at the profit side of things, the outgoing traffic, and put that into relation with the costs. The scores on the right hands side, are calculated by dividing the average outgoing traffic by the average incoming traffic and by the average storage requirement:

What can we learn from this? Small projects can produce a lot of traffic (osmc), especially if they tend to have a small mirror network (libreelec) or a humongous user base (videolan). Big mirrors on the other hand, often produce a lot of traffic in an absolute sense (kiwix, cdn.media.ccc.de), but they are unable to compensate for their storage requirement. For example, the CCC media mirror would have to constantly saturate 2 Gbit/s to get the same traffic/storage ratio as videolan. Although we love outgoing traffic, thankfully they do not. 🙂

Here are the most used mirrors in 2015:

Kiwix on rank 2 has only been mirrored since March 2015, so it generated that amount of traffic in only 9 and a half months – it might well take over the top spot next year. The Ubuntu release ISOs dropped out of the Top 10 for IPv4. As is to be expected, the CCC conference recordings generate extremely peaky traffic after CCC events – setting a new record high with 11.3 TB in just one day on December 30 (after 32C3). It might even have made a little bit more, but unfortunately the webserver ran out of threads because someone apparently distributed web-torrent-files with 64 KB of chunksize.

The list is slightly different for IPv6.

IPv6 traffic is up – from 7.86% over all mirrors in 2014 to 10.54% in 2015. Absolute IPv6 traffic is also up in 2015 for all mirrors in the Top 10. The share of IPv6 traffic varies greatly between the mirrors.

While we publish daily stats for all of our mirrors at https://ftp.fau.de/cgi-bin/show-ftp-stats.cgi, there is currently no aggregation over a whole year. So here are the most used mirrors over the whole year:

The list is slightly different for IPv6.

Still only a small fraction of the traffic is via IPv6, 7.86% over all mirrors and the whole year. CTAN for some reason drops out of the IPv6 list (it would be at #17), its Top10 slot is taken over by Debian (#16 in the total list) which has an unusually large IPv6 share. That large share can be attributed mostly to the local Computer Science Department which has large pools with IPv6 capable Debian machines.