compact: reclaim superseded duplicates left after a full check --repair, fixes #10471

[mr-raj12]

Description

A superseded duplicate is an unindexed object in a pack whose chunk id the index maps to another location (see superseded_gap_ranges). compact_packs only counted indexed-but-unused bytes, so a pack holding used objects and superseded duplicates, or only superseded duplicates, was never rewritten or deleted. This is the state after a crashed borg create is re-run.

• Pass 1 walks the gaps of every pack that has unindexed bytes and is not recorded corrupt, and counts its superseded duplicates.
• A pack's reclaimable bytes are its unused indexed bytes plus its superseded duplicates: they decide the --threshold rewrite and the all-packs gate. A pack holding only unused indexed objects and superseded duplicates is deleted.
• The unindexed-bytes hint no longer counts superseded duplicates.

Tested with the repro from the issue without the borg check --repair step, which leaves the same superseded duplicates (800 MB random file, kill -9 after 3s, re-run kept): packs go from 1.1G to 764M in one borg compact, the next run logs no hint, borg check is rc 0.

Depends on #10443 and #10475, whose commits this branch contains.

fixes #10471

Checklist

• PR is against master (or maintenance branch if only applicable there)
• New code has tests and docs where appropriate
• Tests pass (run tox or the relevant test subset)
• Commit messages are clean and reference related issues

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 88.77%. Comparing base (8274470) to head (399dea2).
⚠️ Report is 13 commits behind head on master.
✅ All tests successful. No failed tests found.

Additional details and impacted files

@@            Coverage Diff             @@
##           master   #10476      +/-   ##
==========================================
+ Coverage   88.73%   88.77%   +0.03%     
==========================================
  Files         103      103              
  Lines       19387    19413      +26     
  Branches     3023     3026       +3     
==========================================
+ Hits        17204    17233      +29     
+ Misses       1514     1512       -2     
+ Partials      669      668       -1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.