repoobj: add OBJ_MAGIC blob header for pack file format#9684
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #9684 +/- ##
==========================================
- Coverage 83.87% 83.78% -0.10%
==========================================
Files 93 93
Lines 15586 15604 +18
Branches 2335 2343 +8
==========================================
+ Hits 13073 13074 +1
- Misses 1785 1794 +9
- Partials 728 736 +8 ☔ View full report in Codecov by Sentry. |
ThomasWaldmann
left a comment
There was a problem hiding this comment.
Looks good.
Next after this, I guess you need to decouple the legacyrepository_test yourself (can't do that if you keep changing it although that test should only test legacy format repos).
The legacy repo (borg 1.x) shall use RepoObj1 format, not RepoObj. It can still use fchunk ("format a chunk") and p(d?)chunk ("parse a chunk"), but needs to be adapted so it works with the old format.
|
Thanks! Understood on legacyrepository_test, I'll fix fchunk/pchunk there to use RepoObj1.obj_header in the next PR. |
|
If you decouple the legacy tests first, you can remove the changes to the legacy test file from this one. |
Description
RepoObjblob header grows from 8 bytes (<II) to 17 bytes (<8sBII): 8-byte magic (BORG_OBJ), 1-byte version, then the existing meta/data sizes.The magic is a re-sync point. If a pack is corrupted, borg scans forward for the next
BORG_OBJto find the next valid blob. The version byte lets borg reject unknown formats rather than misreading them.This is the N=1 pack format (one chunk per pack file). No chunk id in the header yet -- for N=1 the pack filename is the chunk id (SHA-256 hex), so it is implicit. Chunk id in the header comes later when chunk id != pack id.
Changes:
repoobj.py: addOBJ_MAGIC/OBJ_VERSIONconstants, extend struct to<8sBII, validate magic and version informat(),parse(),parse_meta(),extract_crypted_data().repository.py: importOBJ_MAGIC/OBJ_VERSION, updatecheck_object()to check magic/version before sizes, fixget()partial-read path for 17-byte header.repository_test.py,legacyrepository_test.py: importOBJ_MAGIC/OBJ_VERSION, updatefchunk()to include magic and version, fixpchunk()slice to[2:4], updatetest_read_data().refs #8572
Checklist
master(or maintenance branch if only applicable there)toxor the relevant test subset)