NAME
gitformat-chunk - Chunk-based file formatsSYNOPSIS
Used by gitformat-commit-graph(5) and the "MIDX" format (see the pack format documentation in gitformat-pack(5)).DESCRIPTION
Some file formats in Git use a common concept of "chunks" to describe sections of the file. This allows structured access to a large file by scanning a small "table of contents" for the remaining data. This common format is used by the commit-graph and multi-pack-index files. See the multi-pack-index format in gitformat-pack(5) and the commit-graph format in gitformat-commit-graph(5) for how they use the chunks to describe structured data.| Chunk ID (4 bytes) | Chunk Offset (8 bytes) | |--------------------|------------------------| | ID[0] | OFFSET[0] | | ... | ... | | ID[C] | OFFSET[C] | | 0x0000 | OFFSET[C+1] |
WRITING CHUNK-BASED FILE FORMATS
To write a chunk-based file format, create a struct chunkfile by calling init_chunkfile() and pass a struct hashfile pointer. The caller is responsible for opening the hashfile and writing header information so the file format is identifiable before the chunk-based format begins.READING CHUNK-BASED FILE FORMATS
To read a chunk-based file format, the file must be opened as a memory-mapped region. The chunk-format API expects that the entire file is mapped as a contiguous memory region.•pair_chunk() assigns a given
pointer with the location inside the memory-mapped file corresponding to that
chunk’s offset. If the chunk does not exist, then the pointer is not
modified.
•read_chunk() takes a
chunk_read_fn function pointer and calls it with the appropriate
initial pointer and size information. The function is not called if the chunk
does not exist. Use this method to read chunks if you need to perform
immediate parsing or if you need to execute logic based on the size of the
chunk.
EXAMPLES
These file formats use the chunk-format API, and can be used as examples for future formats:•commit-graph: see
write_commit_graph_file() and parse_commit_graph() in
commit-graph.c for how the chunk-format API is used to write and parse
the commit-graph file format documented in the commit-graph file format in
gitformat-commit-graph(5).
•multi-pack-index: see
write_midx_internal() and load_multi_pack_index() in
midx.c for how the chunk-format API is used to write and parse the
multi-pack-index file format documented in the multi-pack-index file format
section of gitformat-pack(5).
GIT
Part of the git(1) suite02/28/2023 | Git 2.39.2 |