evelyn
evelyn is named Evelyn Berezin, inventor of the computer word processor, who developed the first computerized banking system.
Colocataires setup
The Digital Ocean setup was torn down because they sponsor fascists.
Instead, I've rebuilt it at https://colocataires.dev/ which seems like a cool DIY / grassroots cheapo thingie in Ottawa.
Performance tests
Performance feels slow. Turns out the VMs are on SAS 10k HDDs, so of
course it's slow. My baseline is the remaining VM (louise) from
linode.
On linear writes, I get at most 40MB/s, but a random read/write test with fio shows much worse performance, below 1MB/s.
root@evelyn:~# grep 'model name' /proc/cpuinfo
model name : Intel(R) Xeon(R) CPU E5-2640 v2 @ 2.00GHz
root@evelyn:~# head -1 /proc/meminfo
MemTotal: 2021228 kB
root@evelyn:~# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
sr0 11:0 1 1024M 1 rom
vda 254:0 0 12G 0 disk
└─vda1 254:1 0 12G 0 part /
root@evelyn:~# dd bs=1M count=512 conv=fdatasync if=/dev/zero of=/zero
512+0 records in
512+0 records out
536870912 bytes (537 MB, 512 MiB) copied, 10.175 s, 52.8 MB/s
root@evelyn:~# fio --name=stressant --group_reporting --directory=test --size=100M --readwrite=randrw --direct=1 --numjobs=4
fio: test is not a directory
fio: failed parsing directory=test
fio: file:options.c:1586, func=lstat, error=No such file or directory
root@evelyn:~# mkdir test
root@evelyn:~# fio --name=stressant --group_reporting --directory=test --size=100M --readwrite=randrw --direct=1 --numjobs=4
stressant: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1
...
fio-3.39
Starting 4 processes
stressant: Laying out IO file (1 file / 100MiB)
stressant: Laying out IO file (1 file / 100MiB)
stressant: Laying out IO file (1 file / 100MiB)
stressant: Laying out IO file (1 file / 100MiB)
Jobs: 4 (f=4): [m(4)][60.5%][r=484KiB/s,w=528KiB/s][r=121,w=132 IOPS][eta 03m:15s]
Jobs: 1 (f=1): [_(1),m(1),_(2)][100.0%][r=139KiB/s,w=135KiB/s][r=34,w=33 IOPS][eta 00m:00s]
stressant: (groupid=0, jobs=4): err= 0: pid=26497: Wed Oct 28 11:41:21 2026
read: IOPS=110, BW=444KiB/s (455kB/s)(199MiB/459315msec)
clat (usec): min=758, max=697023, avg=33524.74, stdev=56298.06
lat (usec): min=758, max=697024, avg=33525.51, stdev=56298.06
clat percentiles (usec):
| 1.00th=[ 1254], 5.00th=[ 1582], 10.00th=[ 1795], 20.00th=[ 4080],
| 30.00th=[ 8717], 40.00th=[ 11338], 50.00th=[ 13960], 60.00th=[ 16909],
| 70.00th=[ 21890], 80.00th=[ 33817], 90.00th=[102237], 95.00th=[168821],
| 99.00th=[265290], 99.50th=[299893], 99.90th=[375391], 99.95th=[425722],
| 99.99th=[557843]
bw ( KiB/s): min= 63, max= 1504, per=100.00%, avg=471.34, stdev=50.92, samples=3462
iops : min= 15, max= 376, avg=117.52, stdev=12.74, samples=3462
write: IOPS=111, BW=448KiB/s (459kB/s)(201MiB/459315msec); 0 zone resets
clat (usec): min=133, max=8630, avg=464.98, stdev=189.82
lat (usec): min=133, max=8631, avg=465.92, stdev=189.98
clat percentiles (usec):
| 1.00th=[ 204], 5.00th=[ 251], 10.00th=[ 285], 20.00th=[ 334],
| 30.00th=[ 375], 40.00th=[ 416], 50.00th=[ 457], 60.00th=[ 494],
| 70.00th=[ 529], 80.00th=[ 578], 90.00th=[ 627], 95.00th=[ 676],
| 99.00th=[ 848], 99.50th=[ 1057], 99.90th=[ 2769], 99.95th=[ 3392],
| 99.99th=[ 4555]
bw ( KiB/s): min= 31, max= 1528, per=100.00%, avg=475.82, stdev=60.70, samples=3459
iops : min= 7, max= 382, avg=118.64, stdev=15.18, samples=3459
lat (usec) : 250=2.49%, 500=28.66%, 750=18.00%, 1000=0.81%
lat (msec) : 2=7.46%, 4=2.67%, 10=7.33%, 20=16.12%, 50=8.86%
lat (msec) : 100=2.52%, 250=4.38%, 500=0.68%, 750=0.01%
cpu : usr=0.14%, sys=0.49%, ctx=102766, majf=0, minf=42
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=50982,51418,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
READ: bw=444KiB/s (455kB/s), 444KiB/s-444KiB/s (455kB/s-455kB/s), io=199MiB (209MB), run=459315-459315msec
WRITE: bw=448KiB/s (459kB/s), 448KiB/s-448KiB/s (459kB/s-459kB/s), io=201MiB (211MB), run=459315-459315msec
Disk stats (read/write):
vda: ios=51021/52429, sectors=412344/517080, merge=0/336, ticks=1706041/74307, in_queue=1828328, util=97.29%
My baseline is a Linode Vm with a single core out of a (claimed) AMD EPYC 7713 64-Core Processor, with 1GB of RAM and a 25GB disk. There I performed the following tests:
root@louise:~# grep 'model name' /proc/cpuinfo
model name : AMD EPYC 7713 64-Core Processor
root@louise:~# head -1 /proc/meminfo
MemTotal: 990572 kB
root@louise:~# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
sda 8:0 0 24.5G 0 disk /
sdb 8:16 0 512M 0 disk [SWAP]
root@louise:~# dd bs=1M count=512 conv=fdatasync if=/dev/zero of=/zero
512+0 records in
512+0 records out
536870912 bytes (537 MB, 512 MiB) copied, 3.12335 s, 172 MB/s
root@louise:~# fio --name=stressant --group_reporting --directory=test --size=100M --readwrite=randrw --direct=1 --numjobs=4
stressant: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1
...
fio-3.39
Starting 4 processes
stressant: Laying out IO file (1 file / 100MiB)
stressant: Laying out IO file (1 file / 100MiB)
stressant: Laying out IO file (1 file / 100MiB)
stressant: Laying out IO file (1 file / 100MiB)
Jobs: 4 (f=4): [m(4)][83.3%][r=35.6MiB/s,w=35.4MiB/s][r=9115,w=9073 IOPS][eta 00m:01s]
stressant: (groupid=0, jobs=4): err= 0: pid=883628: Wed Oct 28 15:32:59 2026
read: IOPS=10.9k, BW=42.4MiB/s (44.5MB/s)(199MiB/4697msec)
clat (usec): min=90, max=23106, avg=216.73, stdev=319.03
lat (usec): min=90, max=23106, avg=216.87, stdev=319.08
clat percentiles (usec):
| 1.00th=[ 110], 5.00th=[ 120], 10.00th=[ 126], 20.00th=[ 135],
| 30.00th=[ 143], 40.00th=[ 153], 50.00th=[ 161], 60.00th=[ 176],
| 70.00th=[ 194], 80.00th=[ 229], 90.00th=[ 326], 95.00th=[ 441],
| 99.00th=[ 963], 99.50th=[ 1483], 99.90th=[ 4621], 99.95th=[ 6259],
| 99.99th=[ 9896]
bw ( KiB/s): min=23344, max=60577, per=98.64%, avg=42827.56, stdev=3738.16, samples=36
iops : min= 5836, max=15144, avg=10706.44, stdev=934.50, samples=36
write: IOPS=10.9k, BW=42.8MiB/s (44.8MB/s)(201MiB/4697msec); 0 zone resets
clat (usec): min=42, max=22925, avg=147.04, stdev=289.04
lat (usec): min=42, max=22925, avg=147.44, stdev=289.10
clat percentiles (usec):
| 1.00th=[ 56], 5.00th=[ 64], 10.00th=[ 71], 20.00th=[ 80],
| 30.00th=[ 87], 40.00th=[ 95], 50.00th=[ 104], 60.00th=[ 116],
| 70.00th=[ 133], 80.00th=[ 161], 90.00th=[ 235], 95.00th=[ 314],
| 99.00th=[ 725], 99.50th=[ 1106], 99.90th=[ 4047], 99.95th=[ 5342],
| 99.99th=[10028]
bw ( KiB/s): min=22908, max=61914, per=98.54%, avg=43147.67, stdev=3902.77, samples=36
iops : min= 5727, max=15478, avg=10786.44, stdev=975.69, samples=36
lat (usec) : 50=0.10%, 100=22.78%, 250=64.45%, 500=9.95%, 750=1.52%
lat (usec) : 1000=0.43%
lat (msec) : 2=0.46%, 4=0.19%, 10=0.11%, 20=0.01%, 50=0.01%
cpu : usr=1.59%, sys=10.19%, ctx=103588, majf=0, minf=53
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=50982,51418,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
READ: bw=42.4MiB/s (44.5MB/s), 42.4MiB/s-42.4MiB/s (44.5MB/s-44.5MB/s), io=199MiB (209MB), run=4697-4697msec
WRITE: bw=42.8MiB/s (44.8MB/s), 42.8MiB/s-42.8MiB/s (44.8MB/s-44.8MB/s), io=201MiB (211MB), run=4697-4697msec
Disk stats (read/write):
sda: ios=48669/49014, sectors=389352/392112, merge=0/0, ticks=8766/5501, in_queue=14267, util=90.61%
But let's see if it works well enough for DNS.
DNS benchmarks
I setup the following dnsperf data file:
anarc.at A
marcos.anarc.at A
louise.anarc.at A
evelyn.anarc.at A
tubman.anarc.at A
anarc.at SOA
anarc.at NS
_node-exporter-targets.anarc.at SRV
non-exist.anarc.at A
Then ran this benchmark from my home network:
anarcat@angela:~> dnsperf -l 10 -s evelyn.anarc.at < dnsperf.txt
DNS Performance Testing Tool
Version 2.14.0
[Status] Command line: dnsperf -l 10 -s evelyn.anarc.at
[Status] Sending queries (to 23.133.40.50:53)
[Status] Started at: Wed Oct 28 19:40:01 2026
[Status] Stopping after 10.000000 seconds
[Status] Testing complete (time limit)
Statistics:
Queries sent: 46224
Queries completed: 46224 (100.00%)
Queries lost: 0 (0.00%)
Response codes: NOERROR 41088 (88.89%), NXDOMAIN 5136 (11.11%)
Average packet size: request 32, response 96
Run time (s): 10.015053
Queries per second: 4615.452360
Average Latency (s): 0.021596 (min 0.007401, max 0.036570)
Latency StdDev (s): 0.003565
Then against louise:
anarcat@angela:~> dnsperf -l 10 -s louise.anarc.at < dnsperf.txt
DNS Performance Testing Tool
Version 2.14.0
[Status] Command line: dnsperf -l 10 -s louise.anarc.at
[Status] Sending queries (to 172.105.130.156:53)
[Status] Started at: Wed Oct 28 19:39:38 2026
[Status] Stopping after 10.000000 seconds
[Status] Testing complete (time limit)
Statistics:
Queries sent: 8542
Queries completed: 8542 (100.00%)
Queries lost: 0 (0.00%)
Response codes: NOERROR 7593 (88.89%), NXDOMAIN 949 (11.11%)
Average packet size: request 32, response 96
Run time (s): 10.080910
Queries per second: 847.344139
Average Latency (s): 0.117020 (min 0.104996, max 0.133216)
Latency StdDev (s): 0.003442
Evelyn was 5x faster!! But locally, it's another story: the linode VM performs about 2x better.
root@evelyn:~# dnsperf -l 10 < dnsperf.txt
DNS Performance Testing Tool
Version 2.14.0
[Status] Command line: dnsperf -l 10
[Status] Sending queries (to 127.0.0.1:53)
[Status] Started at: Wed Oct 28 19:43:19 2026
[Status] Stopping after 10.000000 seconds
[Status] Testing complete (time limit)
Statistics:
Queries sent: 158217
Queries completed: 158217 (100.00%)
Queries lost: 0 (0.00%)
Response codes: NOERROR 140638 (88.89%), NXDOMAIN 17579 (11.11%)
Average packet size: request 32, response 96
Run time (s): 10.006066
Queries per second: 15812.108375
Average Latency (s): 0.005862 (min 0.000335, max 0.024470)
Latency StdDev (s): 0.002053
vs linode:
root@louise:~# dnsperf -l 10 < dnsperf.txt
DNS Performance Testing Tool
Version 2.14.0
[Status] Command line: dnsperf -l 10
[Status] Sending queries (to 127.0.0.1:53)
[Status] Started at: Wed Oct 28 23:43:22 2026
[Status] Stopping after 10.000000 seconds
[Status] Testing complete (time limit)
Statistics:
Queries sent: 387578
Queries completed: 387578 (100.00%)
Queries lost: 0 (0.00%)
Response codes: NOERROR 344514 (88.89%), NXDOMAIN 43064 (11.11%)
Average packet size: request 32, response 96
Run time (s): 10.001152
Queries per second: 38753.335616
Average Latency (s): 0.002226 (min 0.000034, max 0.018695)
Latency StdDev (s): 0.000947
Still, it seems acceptable.
Original DO setup
It's my third cloud VM, replacing fumiko, designed to handle outages or overloads on marcos..
I picked Digital Ocean because it's cheap. I picked the NYC PoP because it was cheaper than the Toronto one.
I hope to eventually shut down this shit.
I set it up with a plain Debian 11 bullseye install from DO, then went through the bullseye upgrade, after a very basic bootstrap:
apt install foot-terminfo tmux
Then went with the Puppet bootstrap:
tmux
apt install puppet
puppet agent --test --server puppet.anarc.at
then on the master:
puppetserver ca list | grep $HASH_FROM_ABOVE
puppetserver ca sign fumiko.anarc.at
then on the client again:
puppet agent --test
Created .
Edited .