LanguageCompilationSpeed
Written between April and May 2021. The world may have changed since then.
Other interesting data from Jan 2023: https://quick-lint-js.com/blog/cpp-vs-rust-build-times/
Introduction
One of the annoying things about programming in Rust is waiting for things to compile. “Why is Rust slow to compile” is a complicated question that has lots of moving parts that I won’t go into right now, but either way, rustc takes a while to do its work. But I’ve always felt like it was partially a problem of expectations. People used to fast-turnaround languages like JS and Python seem to complain about it loudest (even though I’ve had Typescript projects that take minutes to compile anyway). Meanwhile I don’t seem to see as many people used to C++ bitching about Rust’s compile times. Regardless of expectations though, everyone can agree on the fundamental issue: Faster compilation times are better. Nobody’s ever said “boy I wish I had to wait an extra ten minutes to see if this thing I made even works”. What people argue about is how MUCH better fast compiles are, and what the cost/benefit tradeoffs are.
So for a long time I have wanted to actually benchmark the speed of compiling Rust code vs. other programming languages, but it’s been stymied by one major problem: Lack of suitable benchmark code. A good benchmark for this sort of thing would be as direct a translation of a program from one language to another as possible by someone who is an expert in both, but in many ways Rust is unique enough to make that pretty hard. It has a type system much more like Haskell than C++, and the borrow checker encourages some patterns that are a little odd compared to both manual memory management and garbage collected laguages. A good benchmark would thus have to be a nontrivial program written in Rust and then translated to C++ as accurately as possible, or vice versa, and then one could still get tangled up in what is idiomatic code or whether the compilers are actually doing the same thing. A better benchmark would be several of these programs with different problem domains, with proper statistics and stuff, but frankly, I am not going to put that much work into trying to figure this out when I could be doing something actually useful instead.
But then I had a thought! A thought that is silly enough to be fun but also might be realistic enough to be useful. Each language I’m interested in looking at has at least one large, real-world codebase that solves similar problems in fairly similar ways, and is written by experts in a style idiomatic to that language: its compiler. So, I’m going to look at the compilation speed of different languages by benchmarking how fast various compilers compile themselves.
Of course this is not actually an apples-to-apples comparison. Different compilers can be very different beasts that potentially use very different technology under the hood. There’s also a vast range in the size of compilers: clang+LLVM is millions of lines of code, while chibicc is barely more than 10,000, and both are functional C compilers. rustc is written in Rust… except for the giant chunk of C++ that is LLVM. We could look at the Roslyn C# compiler as well… except C# is generally executed by a JIT runtime, where a one-and-done compiler execution is kinda the worst case for the JIT’s optimization capabilities. So this isn’t going to be a perfect comparison. But hopefully digging into what the differences are and what impact they have is usually also pretty interesting. Fortunately, most of these languages also have multiple implementations, so we can get at least a couple data points for each language and see what the spread looks like.
Methods
Hardware doesn’t matter. It’s a Ryzen 2400G in case someone cares. What I care about is relative speeds, not absolute, and all the tests are done on the same hardware.
We will measure speed in terms of lines compiled per second, using a single process/thread. “Lines” will be measured by the tokei program, and will include comments. Most of these compilers do whole-module or whole-program optimizations, which makes “lines per second” rather meaningless since the cost to optimize it varies based on the total complexity of the program, not a constant multiplier that scales with lines of input. Many algorithms used in compilers are apparently also super-linear in complexity, so doubling the size of the program may increase the amount of time the compiler spends on it by more than two. So this is already a hopeless idea, but I’m going to try to scrape a few fragments of meaning from it, but I don’t bother too hard to try to analyze and correct for any of these confounding factors.
So on the scale of “lies, damned lies, and statistics”, the results of this experiment will hover somewhere around the region of “damned lies”. Like cosmology and theoretical physics, if we get something within 10x of the correct answer, I’m going to be happy with it. If I tried to get entirely rigorous statistics for everything I would probably be at this for months; I would rather skim lightly over a lot of things and accept the results will be vague guidelines at best.
C/C++
I know these are separate languages, but it appears the main C compilers, gcc and clang, both compile both C and C++, and gcc at least is written mostly in C with a fair amount of C++ mixed in. So I’m going to pretend they overlap.
gcc
gcc 10.2.0 compiled with the gcc 10.2.0 provided on my Debian system. Requires libgmp, libmpfr, and libmpc. And libgcc for multilib??? To build I did ./configure --enable-languages=c --disable-multilib --disable-bootstrap, followed by time make -j1
GNU docs kinda suck ass, as always. And the build system spends a significant amount of time running configure scripts, even when I try to make it do all that up front. And apparently gcc is written largely in C++ these days so this isn’t really an apples-to-apples test; I could have sworn it used to be mostly C. There’s still a lot of C in the auxiliary libraries though. The problem here is figuring out what the flying fuck is and isn’t actually getting compiled. Per these docs (findable in gcc/doc/install.texi) by default it does a full 3-stage bootstrap build: builds itself with the system C compiler, builds itself with the version of itself just built, then builds itself again and compares tests against the stage two compiler. Configuring with --disable-bootstrap turns this off, apparently, so we only compile the compiler once. Parts of the build process appear to be interested in the location of numpy, clang, and Eris only knows what else, and it also builds some amount of runtime library junk. It includes or fetches a copy of zlib, it generates about 1.5 million lines of code for various specifications and state machines like an instruction parser, builds the obtusely-named libiberty at least twice, and all sorts of hairy things like that.
So in general it does everything possible to make it impossible to count what I want to count. I’m going to try anyway, but don’t expect these numbers to be remotely correct.
Executed in 18.49 mins fish external
usr time 1012.76 secs 467.00 micros 1012.76 secs
sys time 98.19 secs 92.00 micros 98.19 secs Okay, now can I tell it to bootstrap itself once by hand? It should give identical results but I want to see what actually happens. Instead I… can’t even find where it put the gcc output executable. RIP. Heck, let’s build gcc with clang and see what happens! Is that even possible? Apparently! Works flawlessly, just by setting the CC environment variable. Building with clang we get:
Executed in 19.88 mins fish external
usr time 1072.10 secs 550.00 micros 1072.10 secs
sys time 121.03 secs 108.00 micros 121.03 secs tokei gcc/ libgcc/ libatomic/ libiberty/ turns up:
-------------------------------------------------------------------------------
Language Files Lines Code Comments Blanks
-------------------------------------------------------------------------------
Ada 6233 1473225 808366 379355 285504
Alex 215 6623 6623 0 0
Assembly 505 80595 68559 2639 9397
Autoconf 46 30229 20932 4202 5095
BASH 1 138 110 8 20
C 56442 4781658 3276207 846600 658851
C Header 2047 567076 387633 90201 89242
C++ 285 182221 131712 24270 26239
D 2285 173438 120269 23466 29703
FORTRAN Legacy 504 19447 12668 5509 1270
FORTRAN Modern 6321 229706 167504 34321 27881
Go 1088 82178 64885 7864 9429
Haskell 59 217 191 0 26
Makefile 14 622 439 49 134
Markdown 422 463941 463941 0 0
Module-Definition 154 42887 37848 1 5038
OCaml 1 358 285 29 44
Objective C 528 24726 16837 3004 4885
Objective C++ 368 17491 11818 2218 3455
Perl 4 1580 1107 265 208
Python 5 1493 953 259 281
ReStructuredText 65 61884 61884 0 0
Shell 25 3234 2028 788 418
Standard ML (SML) 1 277 215 28 34
TeX 3 10601 6477 3307 817
Plain Text 28 5344 5344 0 0
-------------------------------------------------------------------------------
Total 77649 8261189 5674835 1428383 1157971
-------------------------------------------------------------------------------So if we only take the 5,530,955 lines of C, C++ and header files, our results are something like:
- gcc: 5,000 loc/second
- clang: 4,600 loc/second
Again, these numbers are absolute BS because there’s tons of weird stuff going on here besides “just” building a compiler. But it’s a number.
g++
Same deal as above, except configured with ./configure --enable-languages=c++ --disable-multilib --disable-bootstrap. It took almost exactly the same amount of time, so I’m going to assume they’re doing pretty much the same thing and the results are redundant.
clang
Ok, we are using clang version 9.0.1, and building the same version. We are going to build the LLVM lib and clang compiler frontend, since that seems the best apples-to-apples comparison that involves the whole compiler. This… is going to take a while.
…and, it got killed with an out of memory error while linking clang, after 196 minutes. Silly me, trying to build LLVM on a system with only 13 GB of memory free. How dare I? I tried another time or two with the same results. I could try to link it with lld, but that doesn’t sound very rewarding. And now I know why the LLVM people are building their own linker at all, it’s because their code is so gigantic they actually need it. After some wizardly help from someone on Discord, it turns out that building clang without debug info enabled works, and doesn’t OOM my entire system. Have to include -DCMAKE_BUILD_TYPE=Release in the cmake command line. I guess including debug symbols for a 157 megabyte binary is a lot of work for the linker to sort out.
So the working build string is: mkdir build; cd build; cmake -DLLVM_ENABLE_PROJECTS=clang -DCMAKE_BUILD_TYPE=Release -G "Unix Makefiles" ../llvm; time make -j1
Time results are:
Executed in 201.62 mins fish external
usr time 192.27 mins 1118.00 micros 192.27 mins
sys time 9.09 mins 190.00 micros 9.09 minsSo that was building it with the default compiler on my system, which is gcc. Let’s build it with clang:
cmake -DCMAKE_CXX_COMPILER=/usr/bin/clang++ -DCMAKE_C_COMPILER=/usr/bin/clang -DLLVM_ENABLE_PROJECTS=clang -G "Unix Makefiles" -DCMAKE_BUILD_TYPE=Release ../llvm; time make -j1
Executed in 191.03 mins fish external
usr time 184.09 mins 551.00 micros 184.09 mins
sys time 6.10 mins 100.00 micros 6.10 mins(Sidetrack: Does using Ninja instead of makefiles result in any speedup? Turns out it makes it 1 minute faster for single-process builds. Less than 1%, so, not enough to be interesting to me here.)
Ok, measuring with tokei, the llvm and clang directories contain 4,635,178 lines of C/C++/header code. …and 1,360,633 of assembly that I’m going to ignore. So our results are:
- gcc: 4,635,178 / (201 * 60) = 384 lines/second
- clang: 4,635,178 / (191 / 60) = 404 lines/second
If we include all the C-family code in the entire project directory that’s 7,385,315 lines, which brings the numbers up to:
- gcc: 612 lines/second
- clang: 644 lines/second
So the “real” number is probably somewhere between those extremes. Of course we’re ignoring assembly language, we’re ignoring the demonstrably not-ignorable fact that the linker is a non-trivial part of this process, and so on. But still, 600 lines of code per second is probably an underestimate, it is impressive. And not in a great way. I blame C++.
chibicc
Chibicc is an educational but functional C compiler designed to be small and readable. I needed something small and satisfying to try to compile. Building it is easy, basically no configuration is required. Times measured with time make -j1.
Building with GCC:
Executed in 781.51 millis fish external
usr time 661.18 millis 0.00 micros 661.18 millis
sys time 120.50 millis 713.00 micros 119.79 millis With chibicc built by GCC:
Executed in 900.55 millis fish external
usr time 753.50 millis 746.00 micros 752.76 millis
sys time 147.35 millis 131.00 micros 147.22 millis With chibicc built by itself:
Executed in 1.66 secs fish external
usr time 1502.81 millis 609.00 micros 1502.20 millis
sys time 154.26 millis 107.00 micros 154.15 millis SLOC measured by tokei:
-------------------------------------------------------------------------------
Language Files Lines Code Comments Blanks
-------------------------------------------------------------------------------
C 50 11270 8694 771 1805
C Header 13 718 537 69 112
Makefile 1 50 31 3 16
Markdown 1 209 209 0 0
Shell 6 363 255 43 65
-------------------------------------------------------------------------------
Total 71 12610 9726 886 1998
-------------------------------------------------------------------------------Our lines of C+header code is 11,918. So the results are:
- gcc: 11918 loc / 781 ms (has to parse comments and stuff as well, so!) = 15,300 loc/second
- chibicc built with gcc: 13,200 loc/second
- chibicc built with itself: 7,200 loc/second
So we see that chibicc compiles code a little bit slower than gcc, but its optimizations suck in comparison.
lcc
Sure why not try some other C compilers; I have fond memories of using lcc back in the days as the one free C compiler that actually worked on Windows. Building it is a little bit of a pain though, you have to set Magic Environment Variables correctly or else it will just do something random. It’s a real throwback to the Good Ol’ Days of of the late 90’s/early oughts, when men were men, errors were silent, and computers expected you to tell them information they could figure out themselves. Read the docs. All of them.
For me the build command ended up being:
mkdir out
HOSTFILE=etc/linux.c BUILDDIR=out make all -j1Building with gcc:
Executed in 4.06 secs fish external
usr time 3.43 secs 787.00 micros 3.43 secs
sys time 0.63 secs 129.00 micros 0.63 secs
Compiling it with itself didn’t work ’cause it looks for all the various bits of itself in the wrong places. Not sure how to fix that yet.
tokei output for the following directories: src/ include/ lburg/ cpp/ etc/:
-------------------------------------------------------------------------------
Language Files Lines Code Comments Blanks
-------------------------------------------------------------------------------
C 50 16356 14821 566 969
C Header 72 3637 3191 49 397
Happy 1 202 202 0 0
Markdown 6 5796 5796 0 0
Shell 1 52 42 4 6
-------------------------------------------------------------------------------
Total 130 26043 24052 619 1372
-------------------------------------------------------------------------------Total lines of C+header: 19,993
- gcc: 19,993 loc / 4.06 sec = 4,900 loc/second
tcc
I wasn’t going to do this but it turned out to be really easy, so here it is. We are using version 0.9.27. Just do configure --cc=gcc or whatever. I did have issues bootstrapping it; for some reason it had trouble finding header files if it wasn’t installed in the correct place. So I compiled it with the Debian packaged version of tcc, which has the same version number, and then installed that copy to /usr/local with its make install command. Then it could build itself fine.
- building with gcc: 6.02 seconds
- building with system tcc, presumably built with gcc: 0.48 seconds
- building with tcc built with itself: 0.495 seconds
Tokei output:
-------------------------------------------------------------------------------
Language Files Lines Code Comments Blanks
-------------------------------------------------------------------------------
Assembly 9 1368 1183 0 185
Batch 1 189 171 0 18
C 160 50572 41058 4489 5025
C Header 93 40384 34289 1553 4542
Makefile 5 926 665 105 156
Module-Definition 5 3398 3346 0 52
Perl 1 427 306 63 58
Shell 2 560 488 28 44
Plain Text 1 168 168 0 0
-------------------------------------------------------------------------------
Total 277 97992 81674 6238 10080
-------------------------------------------------------------------------------This includes a couple example programs but they’re like 150 lines total. More importantly, the win32/ dir has 400 lines of C and 32750 lines of header files. That also includes the code for the ARM targets and such, which is another 8000ish lines. So, omitting those, our total is 60,244 lines, which may still be an overestimate. That is indeed quite tiny; not exactly a weekend project, but well within the reach of a reasonably motivated person doing this for fun.
So our results are:
- gcc: 10,000 lines/sec
- tcc built w/ gcc: 125,500 lines/sec
- tcc bootstrapped with itself: 121,700 lines/sec
Dang, that’s fast. These are also one of the few sets of numbers where I’m sure enough of what is and isn’t actually being compiled to trust that it’s pretty accurate.
Rust
rustc
Finally, something with a toolchain that wasn’t built by rabid baboons. …well, maybe it was, but they’re my rabid baboons, and I know how they think, and where they write things down. So, following the README.md file, we just cp config.toml.example config.toml, and do time ./x.py build. …okay, that starts by cloning git repos for a big pile of documentation, a small pile of auxiliary crates, and LLVM, and building a build tool. However, if you do ./x.py help then it does all the downloading and configuring and such, getting everything ready for you to do the actual run without needing to download anything more.
Let’s use the rustc 1.52 version tag, since that’s the version of rustc I have on my computer already. First off, we need to make it build LLVM or use a pre-built version of LLVM, so we only look at the time spent compiling the Rust part of the compiler. The full commands are:
cp config.toml.example config.toml
# Get everything downloaded
./x.py help
# Build only LLVM first, since we only want to measure rust
./x.py build llvm
# Build the rest of the compiler.
time ./x.py build -j 1 --stage 1This doesn’t seem to build clang with debug symbols, so it all works out. We are doing a “stage 1” build. This still re-does a bit of work: it builds the Rust stdlib with your existing Rust compiler, builds a new Rust compiler using that stdlib, and then re-builds the stdlib with the new compiler. For now that’s fine, I suppose. Results
Executed in 43.05 mins fish external
usr time 42.49 mins 552.00 micros 42.49 mins
sys time 0.54 mins 115.00 micros 0.54 minsBy now the rust repo directory contains everything, including all of LLVM, various tools, unit tests, etc. I THINK that what actually gets built with stage1 is only the code in compiler/ and library/. There is a src/ directory as well but it appears full of unit tests, CI stuff, checked out source for tools such as cargo and rustdoc, etc. The outputs in build/x86_64-unknown-linux-gnu/stage1 are only for rustc, rustdoc and the standard lib, I think, so I am going to omit src/. This might not be correct, but most of the bulk of code in src/ is unit tests which I know aren’t getting built, so hopefully this is close-ish. So the tokei output for compiler/ and library/ summed together is:
-------------------------------------------------------------------------------
Language Files Lines Code Comments Blanks
-------------------------------------------------------------------------------
Assembly 1 372 347 0 25
Autoconf 4 2683 2167 180 336
C 32 15620 11217 1984 2419
C Header 6 1512 906 277 329
C++ 6 3968 3248 261 459
Dockerfile 35 508 428 20 60
HTML 1 93399 84614 32 8753
JSON 1 34 34 0 0
Markdown 508 18309 18309 0 0
Python 1 236 196 5 35
Rust 2325 921697 642237 195016 84444
Shell 28 10437 7694 1581 1162
Plain Text 1 8465 8465 0 0
TOML 89 2033 1733 64 236
XML 1 148137 147871 0 266
YAML 1 2 2 0 0
-------------------------------------------------------------------------------
Total 3040 1227412 929468 199420 98524
-------------------------------------------------------------------------------Or, 921,697 lines of Rust code. So, our results are:
- rustc: 357 lines/sec
I’m preeeeeetty sure this is an underestimate: the stage0 build that happens before stage1 builds the stdlib again, and seems to build a few other minor tools as well. I could try to omit that from the calculations, but frankly I didn’t do that with any of the other compilers, so in the interests of fairness I’m not going to try. A bit of noodling around suggests it might shave a couple minutes off the time at most, anyway.
mrustc
The Other Rust Compiler, written in C++ to be able to bootstrap rustc without needing a copy of rustc to bootstrap itself. It only implements a subset of Rust, but it’s the subset used by rustc itself so it’s a pretty broad and slightly weird one; the main shortcut is it assumes the code it’s given is valid, so does very little actual checking. It’s designed basically to be a clean-room way to bootstrap rustc without needing rustc to already exist on a system, so this is actually a perfect test for it.
So let’s build the sucker. make -j1 takes 448 seconds with g++. It’s about 108,000 lines of C++. So, that’s about 240 lines/second, with gcc. And since it’s not written in the language it compiles, for once I know for sure it’s not sneakily doing any bootstrapping stuff to make my life hard. mrustc, written in C++, compiles more slowly in terms of lines/second than rustc written in Rust does, which is just hilarious.
So how fast does mrustc build rustc, I wonder? Once mrustc is built we do make RUSTCSRC to download the rustc source (it builds rustc 1.29, so this isn’t quite apples-to-apples with what we did in the previous section). Then we do time make -f minicargo.mk -j1 to build the build tool, the Rust standard lib, and rustc. …However, this also needs to build LLVM as well, so we have to sneak into the source dir and build it by hand, THEN go back and tell mrustc to try to build everything. It correctly detects that LLVM is already built, and so just keeps on trucking with everything else.
And we have:
Executed in 44.28 mins fish external
usr time 42.07 mins 490.00 micros 42.07 mins
sys time 0.83 mins 127.00 micros 0.83 minsAnd tokei reports…. 2.3 million lines of Rust in the rustc-1.29.0-src directory, which seems a little concerning given that rustc 1.52 was about 900k lines. It seems that most of those lines are in the vendor and tests directory, so if I remove those, then tokei measures rustc as weighing in at… 965k lines. Seems about right. This calls into question how accurate the previous result for rustc actually is; it presumably still uses most of these vendored libs, but doesn’t actually include their source in the main repository anymore but instead downloads them from crates.io. But if the previous results are underestimating the amount of work done by about 2.5x, well, that’s still “correct” to within a factor of 10 so I’m going to stick with it for now. So that would give us:
- mrustc: 965k lines / 44 minutes = 363 lines/second
Free Pascal
Let’s try some other languages! How bad could it be? …holy hell, I thought lcc was obtuse. Latest source seems to be Free Pascal version 3.2.0, same as what’s on my Debian system. It claims it does some kind of bootstrapping and testing automatically, we’re just gonna have to keep an eye out for that, but I’m not going to try too hard to bootstrap the thing with itself. In the end though, while the docs and build system are somewhat obtuse to describe it seems pretty straightforward to actually use. Just make rtl for the runtime lib, make compiler for the actual compiler.
make rtl: 3.70 seconds. Hard to tell how much code it’s actually building, the runtime lib is full of platform-specific code. It’s 99264 lines of Pascal total though. …And 118,643 lines of Makefile, which is absolutely hilarious, even if they’re generated by the fpcmake program. It looks like the x86_64+linux backend plus common code is something along the lines of 10,000 lines, and it also takes a significant amount of time just for the make parts of it to run. So I think I’m going to ignore this bit.
make compiler: 7.84 seconds. The compiler dir has 430,960 lines of Pascal in it, but a lot of that is for non-x86 backends. I don’t know much about FPC so I don’t really have a great grip on how many of those lines are actually used, since it doesn’t appear to build all possible cross-compilers. Some rough digging to look at the different backends suggests that the actual compiler + x86_64 backend is more like 350,000 lines of code.
So the final result is:
- fpc: 44,600 loc/second
Tragically, I don’t seem to be able to find another Pascal compiler to test against, at least not easily. GNU Pascal seems dead and Delphi is Windows only? Either way, I’m done for now.
Go
Go claims it compiles quickly. Does it? Let’s find out.
Go’s building docs are less complete than Rust’s, and it doesn’t seem to have a way to tell it to not do a full three-stage bootstrap. So I just stopped the build when it printed out “Building Go toolchain3”. Took 41 seconds.
tokei output for go/src:
-------------------------------------------------------------------------------
Language Files Lines Code Comments Blanks
-------------------------------------------------------------------------------
Alex 2 118 118 0 0
Assembly 502 140814 127778 70 12966
Autoconf 9 283 274 0 9
BASH 12 768 448 220 100
Batch 5 314 186 71 57
C 70 6122 4757 600 765
C Header 10 373 148 168 57
C++ 1 34 17 9 8
CSS 1 1 1 0 0
Go 4386 1822009 1439338 238118 144553
HTML 1 1 1 0 0
JSON 13 1712 1712 0 0
Makefile 4 17 7 7 3
Markdown 6 945 945 0 0
Objective C 2 21 15 3 3
Perl 9 1343 1019 169 155
Python 1 606 404 70 132
Shell 7 1842 1057 653 132
Plain Text 704 44107 44107 0 0
-------------------------------------------------------------------------------
Total 5745 2021430 1622332 240158 158940
-------------------------------------------------------------------------------So the results are indeed pretty quick:
- 1.82 million lines / 41 seconds = 44k lines/second
Others
- Swift? A rumor mentions it’s bulky and slow to work on. It seems to be about 50% C++ though, so I’m not going to try to untangle it the way I did for rustc.
- ocamlc – Ok this is actually impossible to measure; it builds a C program that is its bytecode interpreter, uses that to run a pre-compiled compiler bytecode file, and then uses that to compile the actual compiler to build itself again as native code. I could measure something, but Eris only knows what it would be.
- Haskell? F#? Zig isn’t self-hosted yet. Nim?
- Modula-2? Apparently the ADW compiler is nice, though Windows-only?
Conclusions
What conclusions can we make?
This was a terrible idea!
And most of these numbers are absolute frothing nonsense. Compilers are complicated! They use nontrivial libraries! They involve lots of weird code generation and bootstrapping stuff! Each one has its own boutique toolchain to make the bootstrapping cycle easy, and isn’t designed for someone to take them apart and poke around in the middle of the stages! And most of these toolchains are among the least user-friendly imaginable! Even if you manage to do it, most of the time you have tons of confounding factors like the overhead of configure scripts or make itself, the exact libraries and settings you use, and what languages you’re building for! Good heckin’ luck actually untangling that all.
For better benchmarks, you might want to check out this project which takes a different approach: it tries to generate programs in each language and compile them all. It has the eternal problem of “synthetic benchmarks vs. real benchmarks”, but at least you can measure synthetic benchmarks.
This was kinda cool to do though. tcc pleased me the most, because I expected it to be difficult and old-fashioned as heck like lcc was, and it was actually really nice. It is apparently unmaintained these days; someone should pick it up and dust it off!
This data is too terrible to be honored with a graph, but let’s make a table for at least some kind of summary:
| Language | Compiler | compiled with | Speed it was compiled at | Error bounds |
|---|---|---|---|---|
| C and C++ | gcc | gcc | 5k loc/sec | big |
| C and C++ | gcc | clang | 4.6k loc/sec | big |
| C++ | clang | gcc | 380 loc/sec | biggish |
| C++ | clang | clang | 400 loc/sec | biggish |
| C | chibicc | gcc | 15k loc/sec | chibi-sized |
| C | chibicc | gcc-built chibicc | 13k loc/sec | chibi-sized |
| C | chibicc | bootstrapped chibicc | 7k loc/sec | chibi-sized |
| C | lcc | gcc | 5k loc/sec | no idea |
| C | tcc | gcc | 10k loc/sec | smallish |
| C | tcc | gcc-built tcc | 125k loc/sec | smallish |
| C | tcc | bootstrapped tcc | 121k loc/sec | smallish |
| C++ | mrustc | gcc | 240 loc/sec | smallish |
| Rust | rustc | rustc | 350 loc/sec | big |
| Rust | rustc | mrustc | 360 loc/sec | big |
| Pascal | fpc | fpc | 45k loc/sec | biggish |
| Go | go | go | 44k loc/sec | mediumish |
So what have we learned from this? Nothing terribly useful, but:
- Can you compile C fast? – Yes.
- Can you compile C++ fast? – No information
- Can you compile Rust fast? – No information
- Can you compile Pascal fast? – Yes.
- Can you compile Go fast? – Yes.
On the flip side:
- Can you compile C slowly? – Yes but the worst case I found was still 10x faster than Rust/C++’s worst case
- Can you compile C++ slowly? – Hell yeah
- Can you compile Rust slowly? – I would be disappointed if you couldn’t
- Can you compile Pascal slowly? – No information
- Can you compile Go slowly? – No information
