For reference this is how I would to the same task using dd:

sudo dd if=image.wic of=/dev/sda status=progress conv=fsync

Interestingly enough, the reported transfer speed at the end is 38 MB/s, which is almost exactly what etcher reports (it reports 41MB/s), but the dd invocation takes 2m49s, while etcher is done in 1m21s.

Excellent question! So, file systems are stored in "blocks", and whenever the file system needs some space for a file, it allocates a new block. But up until that point, the block is just sitting there idle, with "garbage" data (the data might be old or just zeroes if it's a new disk). Even disk images straight from the creator will have "holes" (or unallocated blocks) in the disk image, and the only way to tell which blocks actually contain real data is by "asking" the file system.

dd is a dumb tool that just copies file-like entities and knows nothing about the holes in the disk, whereas Etcher "trims" the partition (meaning disregards the unallocated blocks) before it copies the data. This is most of the reason why Etcher is so fast, although there are other reasons as well. But the main bottleneck in flashing disks is the disk I/O, and the only way to increase that is by getting a faster disk, or reducing the amount of blocks you are copying.

The codebase is rather complex and difficult to understand, but all the "magic" happens in node-ext2fs, where the partitions are trimmed.

How exactly all of this works is rather complicated, so much so that I made a video explanation for internal use, but you're welcome to check it out if you're interested.

This might sound self-serving, but the easiest way to understand it is usually to contribute to it, so PR's welcome :)

Wow! Thanks for the detailed explanation! I'll definitely consider contributing!

On the contrary I might ask why etcher is slower than usbimager writing compressed images?

Compressed write time (4)

BalenaEtcher
01:12:51

USBImager
00:30:47

(4) - measurements performed by @CaptainMidnight on Windows 10 Pro using a SanDisk Ulta 32GB A1 device. Raw image file size was 31,166,976 Kb, the bzip2 compressed image size was 1,887,044 Kb.

Source: https://gitlab.com/bztsrc/usbimager#comparison

Writing a 32G compressed image to the device takes half an hour in USBImager, and one and a quater hour (!) in etcher. 🚮

Looking at the table you linked to, the raw (uncompressed) write time is fairly similar, so I guess the difference is because Javascript is much slower at decompressing a bzip2 file than a native C application? 🤷

From the same source (which is a competitor, so I guess they picked their test image to look good): flashing a Raw image is faster with etcher...

Is etcher the best/fastest for all cases? Probably not, but it's not its main purpose.

The main question was a comparison with DD, so this is off-topic anyway.