POKI_PUT_TOC_HERE
I did simple experiments in several languages. In one experiment (cat
tests) I just read lines and printed them back out — a line-oriented
cat
. In another (cut tests) I consumed input lines like
x=1,y=2,z=3
one at a time, split them on commas and equals signs
to populate hash maps, transformed them (e.g. remove the y
field),
and emitted them. Basically mlr cut -x -f y
with DKVP format. I
didn’t do anything fancy — just using each language’s
getline
, string-split, hashmap-put, etc. (For C, the hashmap logic
was my own and has turned into Miller per se.) And nothing was as fast
as C, so I used C. Here are the experiments I kept (I failed to keep the Lua
code, for example). Note that I re-ran the timings in 2019 using code written
in 2015, for purposes of creating this page.
/bin/cat
is quite a bit faster than my line-oriented C code.
This shows that my simple experiments here don’t exhaust the
possibilities of clever programming. Rather, they serve to compare and contrast
the performance of straightforward coding in various languages.
I didn’t get any of the other languages to do line-oriented
cat
as fast as C. But Go was in the ballpark and was worth a
further look.
The cut
experiments separated it out. I iteratively
commented out/in various steps of the cut
steps and found (as
noted in the source code below) big chunks of cutgo.go
’s
time in the simple map and write-to-stdout library-code operations. I failed
to retain cutc.c
as I originally implemented it in 2015 (it
morphed gradually into Miller per se) but it was sufficiently faster than
cutgo.go
that I stuck with C for implementing Miller.
The full Miller executable running mlr cat
or mlr
cut
is slower than these bare-bones C executables — I fully
implemented Miller only in one language so there’s no apples-to-apples
comparison with other languages.
While C was the clear winner in these very simple tests, it’s quite possible
that had I committed to another language for full-featured Miller, it would have been
faster in that language. That was a gamble I did not take.
$ gcc --version gcc (Ubuntu 7.3.0-27ubuntu1~18.04) 7.3.0 Copyright (C) 2017 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. $ dmd --version DMD64 D Compiler v2.088.0 Copyright (C) 1999-2019 by The D Language Foundation, All Rights Reserved written by Walter Bright $ rustc --version rustc 1.36.0 $ go version go version go1.10.4 linux/amd64 $ nim --version Nim Compiler Version 0.17.2 (2018-02-05) [Linux: amd64] Copyright (c) 2006-2018 by Andreas Rumpf active boot switches: -d:release
$ uname -a Linux sprax 4.15.0-20-generic #21-Ubuntu SMP Tue Apr 24 06:16:15 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux $ cat /proc/cpuinfo processor : 0 vendor_id : GenuineIntel cpu family : 6 model : 142 model name : Intel(R) Core(TM) i7-8550U CPU @ 1.80GHz stepping : 10 microcode : 0x96 cpu MHz : 796.615 cache size : 8192 KB physical id : 0 siblings : 8 core id : 0 cpu cores : 4 apicid : 0 initial apicid : 0 fpu : yes fpu_exception : yes cpuid level : 22 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf tsc_known_freq pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb invpcid_single pti tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid mpx rdseed adx smap clflushopt intel_pt xsaveopt xsavec xgetbv1 xsaves ibpb ibrs stibp dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp bugs : cpu_meltdown spectre_v1 spectre_v2 bogomips : 3984.00 clflush size : 64 cache_alignment : 64 address sizes : 39 bits physical, 48 bits virtual power management: ... (8 CPUs total)