Yes, it is a matter of scale, but it is also a matter of what you promise. If you are going to promise 100% reliable detection of duplicates, just using a hash (no matter how good) is a bug, because it ISN’T 100% reliable. It might be “Reliable Enough”, but don’t say it is 100%.

Please also review your math, as your argument actually proved its flaw. The number 2.71 x 10^18 ISN’T a chance of collision, or even a chance of any kind. It is the number of UUIDs (of 128 bits) that would need to be generated to have a 50% chance of collision.

I would actually argue that the idea of using a very high quality hash, like SHA256 is taking the problem in the wrong direction. It is a somewhat expensive value to compute, and being 256 bits long (32 bytes) somewhat big to add, and index, for every file. I would be much more inclined to use a simple hash (like CRC) that gives maybe a 32 bit result, lookup the files that match (somewhat likely to have one) and do the bit by bit compare to see if it is a duplicate. That is probably faster than your method of computing the SHA256 and (possibly even if unlikely, incorrectly) assuming they are unique, the time saved in using a simpler hash, and quicker index, likely makes up for the time needed to compare the two files.