NAME
gitprotocol-pack - How packs are transferred over-the-wireSYNOPSIS
<over-the-wire-protocol>
DESCRIPTION
Git supports transferring data in packfiles over the ssh://, git://, http:// and file:// transports. There exist two sets of protocols, one for pushing data from a client to a server and another for fetching data from a server to a client. The three transports (ssh, git, file) use the same protocol to transfer data. http is documented in gitprotocol-http(5).PKT-LINE FORMAT
The descriptions below build on the pkt-line format described in gitprotocol-common(5). When the grammar indicate PKT-LINE(...), unless otherwise noted the usual pkt-line LF rules apply: the sender SHOULD include a LF, but the receiver MUST NOT complain if it is not present.error-line = PKT-LINE("ERR" SP explanation-text)
TRANSPORTS
There are three transports over which the packfile protocol is initiated. The Git transport is a simple, unauthenticated server that takes the command (almost always upload-pack, though Git servers can be configured to be globally writable, in which receive- pack initiation is also allowed) with which the client wishes to communicate and executes it and connects it to the requesting process.EXTRA PARAMETERS
The protocol provides a mechanism in which clients can send additional information in its first message to the server. These are called "Extra Parameters", and are supported by the Git, SSH, and HTTP protocols.GIT TRANSPORT
The Git transport starts off by sending the command and repository on the wire using the pkt-line format, followed by a NUL byte and a hostname parameter, terminated by a NUL byte.0033git-upload-pack /project.git\0host=myserver.com\0
003egit-upload-pack /project.git\0host=myserver.com\0\0version=1\0
git-proto-request = request-command SP pathname NUL [ host-parameter NUL ] [ NUL extra-parameters ] request-command = "git-upload-pack" / "git-receive-pack" / "git-upload-archive" ; case sensitive pathname = *( %x01-ff ) ; exclude NUL host-parameter = "host=" hostname [ ":" port ] extra-parameters = 1*extra-parameter extra-parameter = 1*( %x01-ff ) NUL
$ echo -e -n \ "003agit-upload-pack /schacon/gitbook.git\0host=example.com\0" | nc -v example.com 9418
SSH TRANSPORT
Initiating the upload-pack or receive-pack processes over SSH is executing the binary on the server via SSH remote execution. It is basically equivalent to running this:$ ssh git.example.com "git-upload-pack '/project.git'"
git clone ssh://[email protected]/project.git | v ssh [email protected] "git-upload-pack '/project.git'"
git clone [email protected]:project.git | v ssh [email protected] "git-upload-pack 'project.git'"
ssh://[email protected]/~alice/project.git, | v ssh [email protected] "git-upload-pack '~alice/project.git'"
•The "command name" is
spelled with dash (e.g. git-upload-pack), but this can be overridden by the
client;
•The repository path is always quoted
with single quotes.
FETCHING DATA FROM A SERVER
When one Git repository wants to get data that a second repository has, the first can fetch from the second. This operation determines what data the server has that the client does not then streams that data down to the client in packfile format.REFERENCE DISCOVERY
When the client initially connects the server will immediately respond with a version number (if "version=1" is sent as an Extra Parameter), and a listing of each reference it has (all branches and tags) along with the object name that each reference currently points to.$ echo -e -n "0045git-upload-pack /schacon/gitbook.git\0host=example.com\0\0version=1\0" | nc -v example.com 9418 000eversion 1 00887217a7c7e582c46cec22a130adf4b9d7d950fba0 HEAD\0multi_ack thin-pack side-band side-band-64k ofs-delta shallow no-progress include-tag 00441d3fcd5ced445d1abc402225c0b8a1299641f497 refs/heads/integration 003f7217a7c7e582c46cec22a130adf4b9d7d950fba0 refs/heads/master 003cb88d2441cac0977faf98efc80305012112238d9d refs/tags/v0.9 003c525128480b96c89e6418b1e40909bf6c5b2d580f refs/tags/v1.0 003fe92df48743b7bc7d26bcaabfddde0a1e20cae47c refs/tags/v1.0^{} 0000
advertised-refs = *1("version 1") (no-refs / list-of-refs) *shallow flush-pkt no-refs = PKT-LINE(zero-id SP "capabilities^{}" NUL capability-list) list-of-refs = first-ref *other-ref first-ref = PKT-LINE(obj-id SP refname NUL capability-list) other-ref = PKT-LINE(other-tip / other-peeled) other-tip = obj-id SP refname other-peeled = obj-id SP refname "^{}" shallow = PKT-LINE("shallow" SP obj-id) capability-list = capability *(SP capability) capability = 1*(LC_ALPHA / DIGIT / "-" / "_") LC_ALPHA = %x61-7A
PACKFILE NEGOTIATION
After reference and capabilities discovery, the client can decide to terminate the connection by sending a flush-pkt, telling the server it can now gracefully terminate, and disconnect, when it does not need any pack data. This can happen with the ls-remote command, and also can happen when the client already is up to date.upload-request = want-list *shallow-line *1depth-request [filter-request] flush-pkt want-list = first-want *additional-want shallow-line = PKT-LINE("shallow" SP obj-id) depth-request = PKT-LINE("deepen" SP depth) / PKT-LINE("deepen-since" SP timestamp) / PKT-LINE("deepen-not" SP ref) first-want = PKT-LINE("want" SP obj-id SP capability-list) additional-want = PKT-LINE("want" SP obj-id) depth = 1*DIGIT filter-request = PKT-LINE("filter" SP filter-spec)
shallow-update = *shallow-line *unshallow-line flush-pkt shallow-line = PKT-LINE("shallow" SP obj-id) unshallow-line = PKT-LINE("unshallow" SP obj-id)
upload-haves = have-list compute-end have-list = *have-line have-line = PKT-LINE("have" SP obj-id) compute-end = flush-pkt / PKT-LINE("done")
•the server will respond with ACK
obj-id continue for any common commits.
•once the server has found an
acceptable common base commit and is ready to make a packfile, it will blindly
ACK all have obj-ids back to the client.
•the server will then send a NAK
and then wait for another response from the client - either a done or
another list of have lines.
•the server will differentiate the ACKs
where it is signaling that it is ready to send data with ACK obj-id
ready lines, and signals the identified common commits with ACK obj-id
common lines.
•upload-pack sends "ACK
obj-id" on the first common object it finds. After that it says nothing
until the client gives it a "done".
•upload-pack sends "NAK" on a
flush-pkt if no common object has been found yet. If one has been found, and
thus an ACK was already sent, it’s silent on the flush-pkt.
server-response = *ack_multi ack / nak ack_multi = PKT-LINE("ACK" SP obj-id ack_status) ack_status = "continue" / "common" / "ready" ack = PKT-LINE("ACK" SP obj-id) nak = PKT-LINE("NAK")
C: 0054want 74730d410fcb6603ace96f1dc55ea6196122532d multi_ack \ side-band-64k ofs-delta\n C: 0032want 7d1665144a3a975c05f1f43902ddaf084e784dbe\n C: 0032want 5a3f6be755bbb7deae50065988cbfa1ffa9ab68a\n C: 0032want 7e47fe2bd8d01d481f44d7af0531bd93d3b21c01\n C: 0032want 74730d410fcb6603ace96f1dc55ea6196122532d\n C: 0000 C: 0009done\n S: 0008NAK\n S: [PACKFILE]
C: 0054want 74730d410fcb6603ace96f1dc55ea6196122532d multi_ack \ side-band-64k ofs-delta\n C: 0032want 7d1665144a3a975c05f1f43902ddaf084e784dbe\n C: 0032want 5a3f6be755bbb7deae50065988cbfa1ffa9ab68a\n C: 0000 C: 0032have 7e47fe2bd8d01d481f44d7af0531bd93d3b21c01\n C: [30 more have lines] C: 0032have 74730d410fcb6603ace96f1dc55ea6196122532d\n C: 0000 S: 003aACK 7e47fe2bd8d01d481f44d7af0531bd93d3b21c01 continue\n S: 003aACK 74730d410fcb6603ace96f1dc55ea6196122532d continue\n S: 0008NAK\n C: 0009done\n S: 0031ACK 74730d410fcb6603ace96f1dc55ea6196122532d\n S: [PACKFILE]
PACKFILE DATA
Now that the client and server have finished negotiation about what the minimal amount of data that needs to be sent to the client is, the server will construct and send the required data in packfile format.PUSHING DATA TO A SERVER
Pushing data to a server will invoke the receive-pack process on the server, which will allow the client to tell it which references it should update and then send all the data the server will need for those new references to be complete. Once all the data is received and validated, the server will then update its references to what the client specified.AUTHENTICATION
The protocol itself contains no authentication mechanisms. That is to be handled by the transport, such as SSH, before the receive-pack process is invoked. If receive-pack is configured over the Git transport, those repositories will be writable by anyone who can access that port (9418) as that transport is unauthenticated.REFERENCE DISCOVERY
The reference discovery phase is done nearly the same way as it is in the fetching protocol. Each reference obj-id and name on the server is sent in packet-line format to the client, followed by a flush-pkt. The only real difference is that the capability listing is different - the only possible values are report-status, report-status-v2, delete-refs, ofs-delta, atomic and push-options.REFERENCE UPDATE REQUEST AND PACKFILE TRANSFER
Once the client knows what references the server is at, it can send a list of reference update requests. For each reference on the server that it wants to update, it sends a line listing the obj-id currently on the server, the obj-id the client would like to update it to and the name of the reference.update-requests = *shallow ( command-list | push-cert ) shallow = PKT-LINE("shallow" SP obj-id) command-list = PKT-LINE(command NUL capability-list) *PKT-LINE(command) flush-pkt command = create / delete / update create = zero-id SP new-id SP name delete = old-id SP zero-id SP name update = old-id SP new-id SP name old-id = obj-id new-id = obj-id push-cert = PKT-LINE("push-cert" NUL capability-list LF) PKT-LINE("certificate version 0.1" LF) PKT-LINE("pusher" SP ident LF) PKT-LINE("pushee" SP url LF) PKT-LINE("nonce" SP nonce LF) *PKT-LINE("push-option" SP push-option LF) PKT-LINE(LF) *PKT-LINE(command LF) *PKT-LINE(gpg-signature-lines LF) PKT-LINE("push-cert-end" LF) push-option = 1*( VCHAR | SP )
push-options = *PKT-LINE(push-option) flush-pkt
packfile = "PACK" 28*(OCTET)
PUSH CERTIFICATE
A push certificate begins with a set of header lines. After the header and an empty line, the protocol commands follow, one per line. Note that the trailing LF in push-cert PKT-LINEs is not optional; it must be present.Identify the GPG key in "Human Readable
Name < email@address[1]>" format.
pushee url
The repository URL (anonymized, if the URL
contains authentication material) the user who ran git push intended to
push into.
nonce nonce
The nonce string the receiving
repository asked the pushing user to include in the certificate, to prevent
replay attacks.
REPORT STATUS
After receiving the pack data from the sender, the receiver sends a report if report-status or report-status-v2 capability is in effect. It is a short listing of what happened in that update. It will first list the status of the packfile unpacking as either unpack ok or unpack [error]. Then it will list the status for each of the references that it tried to update. Each line is either ok [refname] if the update was successful, or ng [refname] [error] if the update was not.report-status = unpack-status 1*(command-status) flush-pkt unpack-status = PKT-LINE("unpack" SP unpack-result) unpack-result = "ok" / error-msg command-status = command-ok / command-fail command-ok = PKT-LINE("ok" SP refname) command-fail = PKT-LINE("ng" SP refname SP error-msg) error-msg = 1*(OCTET) ; where not "ok"
report-status-v2 = unpack-status 1*(command-status-v2) flush-pkt unpack-status = PKT-LINE("unpack" SP unpack-result) unpack-result = "ok" / error-msg command-status-v2 = command-ok-v2 / command-fail command-ok-v2 = command-ok *option-line command-ok = PKT-LINE("ok" SP refname) command-fail = PKT-LINE("ng" SP refname SP error-msg) error-msg = 1*(OCTET) ; where not "ok" option-line = *1(option-refname) *1(option-old-oid) *1(option-new-oid) *1(option-forced-update) option-refname = PKT-LINE("option" SP "refname" SP refname) option-old-oid = PKT-LINE("option" SP "old-oid" SP obj-id) option-new-oid = PKT-LINE("option" SP "new-oid" SP obj-id) option-force = PKT-LINE("option" SP "forced-update")
S: 006274730d410fcb6603ace96f1dc55ea6196122532d refs/heads/local\0report-status delete-refs ofs-delta\n S: 003e7d1665144a3a975c05f1f43902ddaf084e784dbe refs/heads/debug\n S: 003f74730d410fcb6603ace96f1dc55ea6196122532d refs/heads/master\n S: 003d74730d410fcb6603ace96f1dc55ea6196122532d refs/heads/team\n S: 0000 C: 00677d1665144a3a975c05f1f43902ddaf084e784dbe 74730d410fcb6603ace96f1dc55ea6196122532d refs/heads/debug\n C: 006874730d410fcb6603ace96f1dc55ea6196122532d 5a3f6be755bbb7deae50065988cbfa1ffa9ab68a refs/heads/master\n C: 0000 C: [PACKDATA] S: 000eunpack ok\n S: 0018ok refs/heads/debug\n S: 002ang refs/heads/master non-fast-forward\n
GIT
Part of the git(1) suiteNOTES
- 1.
- email@address
mailto:email@address
02/28/2023 | Git 2.39.2 |