More in-depth discussion of topics around reproducible builds. WORK IN PROGRESS.

Why bit-for-bit

As we've done so ourselves already with diffoscope, we can diff two build outputs, and we can review it to see that this only contains "build date" or "build path" differences.

However, it is not feasible to write an algorithm that reads this diff and can decide if the run-time behaviour of both outputs would be identical. It is even fundamentally impossible to do this in the case of Turing-complete data formats such as executables, since the real behaviour of the result could easily change based on a piece of data embedded in the file (even if the data is itself static or immutable).

So, any small diff like this would ultimately have to be reviewed by a human. This doesn't scale and we can't automate this for thousands of packages, e.g. in trying to achieve whole-distro reproducibility. This is not acceptable, and bit-for-bit reproducibility is really the only solution if you want any real level of security.

Buildinfo files

Originally, we wanted to produce buildinfo-like files that describe "the minimum information needed to reproduce a binary". But it turns out that defining this precisely (and therefore writing algorithms to calculate it) was more difficult than we thought. We have a much better understanding of how to achieve this now, and it is definitely possible, but also understand that it is best done not when generating a buildinfo file after 1 single build.

Instead, buildinfo files' primary purpose is to record the build environment for that particular build, and it should be an over-estimate of the information needed to reproduce that build. Using this primitive data descriptor, we can then devise a distributed algorithm involving multiple independent builders, that can tell us with high probability the "minimum information needed to reproduce a binary" which was the original target that we wanted.

See also https://anonscm.debian.org/cgit/reproducible/buildinfo-spec.git/tree/notes/buildinfo.rst (also a WIP).