.TH XPAK 5 "Oct 2011" "Portage VERSION" "Portage"
.SH NAME
xpak \- The XPAK Data Format used with Portage binary packages
.SH DESCRIPTION
Every Gentoo binary package has a xpak attached to it which contains build
time information like the USE flags it was built with, the ebuild it was
built from, the environmental variables, CFLAGS, CXXFLAGS, etc...
.SH NOTES
.SS Data Types
The following conventions cover all occurrences in this documentation
.IP Integer
All offsets/lengths are big endian unsigned 32bit integers
.IP String
All strings are ASCII encoded, and not NUL terminated (quotes are for illustration only)
.IP Values
The actual values of the individual xpak entries are stored as Strings
.P
.SS Vertical Bars
The vertical bars '|' are not part of the file format; they are merely used to
illustrate how the offset values apply to the data.
.SH SYNOPSIS
.IP "binpkg (tbz2)"
      |<-xpak_offset->|
 <tar>|<    xpak     >|<xpak_offset>"STOP"
.IP xpak
 "XPAKPACK"<index_len><data_len><index><data>"XPAKSTOP"
.IP index
 |<-------------index_len------------->|
 |<index1><index2><index3><index4><...>|
.IP indexN
           |<-name_len->|
 <name_len>|<   name   >|<data_offset><data_len>
.IP data
 |<--------------data_len------------->|
 |<-dataN_offset->|<-dataN_len->|
 |<     data     >|<  data_N   >|<data>|
.SH DETAILS
.SS xpak

If you look at a Gentoo binary package (binpkg) with a hex-editor you'll
notice that after the tarball of files, you find a binary blob - the
\fIxpak\fR, an offset which holds the bytes from the start of the
\fIxpak\fR to the end of the file - \fIxpak_offset\fR and finally the
String \fI"STOP"\fR.

      |<xpak_offset>|
 <tar>|<---xpak---->|<xpak_offset>"STOP"

Here you see the \fItar\fR archive, the attached \fIxpak\fR blob, the 
\fIxpak_offset\fR and the string \fI"STOP"\fR at the end.  This metadata
is not considered "part" of the \fIxpak\fR, but rather part of the binpkg.

If we read the offset value and count \fIoffset\fR bytes backwards from
the start of \fIxpak_offset\fR, we have found the start of the \fIxpak\fR
block which starts with the String \fI"XPAKPACK"\fR.

This xpak block consists of the string \fI"XPAKPACK"\fR, the length of the
\fIindex\fR block (\fIindex_len\fR), the length of the \fIdata\fR block
(\fIdata_len\fR), an \fIindex_len\fR bytes long binary blob with the
\fIindex\fR, a \fIdata_len\fR bytes long binary blob with the \fIdata\fR,
and the string \fI"XPAKSTOP"\fR at the end:

                                |<index_len>|<data_len>|
 "XPAKPACK"<index_len><data_len>|<--index-->|<--data-->|"XPAKSTOP"

To actually get the \fIindex\fR and the \fIdata\fR, we cut out \fIindex_len\fR
bytes after the end of \fIdata_len\fR for the \fIindex\fR block, and then cut
out the next \fIdata_len\fR bytes for the \fIdata\fR block.  If we have done
everything right up to this point, the following bytes would be the ASCII
formatted string \fI"XPAKSTOP"\fR.

The actual \fIdata\fR is merged into one big block; so if we want to read it,
we need the actual positions of each information in this big data block.  This
information can be obtained using the indices which are stored in the
\fIindex\fR block.

.SS Index block
The \fIindex\fR block consists of several indices:

 |<-----------------------index_len---------------------->|
 |<index1><index2><index3><index4><index5><index6><index7>|

The \fIindex\fR block holds all the information we need to find the data we
want in the \fIdata\fR block.  It consists of multiple index elements, all of
which add up to the total length \fIindex_len\fR.  It is not zero delimited
or anything else.

Each of those elements corresponds to one chunk of data in the \fIdata\fR
block: the length of that block's name (\fIname_len\fR), a \fIname_len\fR
bytes long string, the offset of that block (\fIdataN_offset\fR) and the
length of that block (\fIdataN_len\fR):

           |<name_len>|
 <name_len>|<--name-->|<dataN_offset><dataN_len>

.SS Data block
The \fIdata\fR block contains multiple chunks of data with a total length of
\fIdata_len\fR:

 |<------------------------data_len------------------------>|
 |<data1><data2><data3><data4><data5><data6><data7><data...>|

To select one data element, we need the \fIdata_offset\fR and the
\fIdata_len\fR from the \fIindex\fR.  With those, we can count
\fIdata_offset\fR bytes from the start of the \fIdata\fR block,
and then cut out the next \fIdata_len\fR bytes.  Then we got our
data block:

 |<-----dataN_offset----->|<--dataN_len->|
 |<data1data2data3data...>|<data-we-want>|
.SH EXAMPLES
Let's say we have an xpak with two chunks of data.  The first has the name
"file1" with the contents "ddDddDdd" and the second has the name "file2" with
the contents "jjJjjJjj".  There is no \fI"STOP"\fR or \fIxpak_offset\fR as
this xpak is not part of a binpkg.

Here is the hexdump output (we will break it down line by line below):
 00  58 50 41 4b 50 41 43 4b  00 00 00 20 00 00 00 10  |XPAKPACK... ....|
 10  00 00 00 04 66 69 6c 31  00 00 00 00 00 00 00 08  |....fil1........|
 20  00 00 00 04 66 69 6c 32  00 00 00 08 00 00 00 08  |....fil2........|
 30  64 64 44 64 64 44 64 64  6a 6a 4a 6a 6a 4a 6a 6a  |ddDddDddjjJjjJjj|
 40  58 50 41 4b 53 54 4f 50                           |XPAKSTOP|

The \fIindex_len\fR is 32 and the \fIdata_len\fR 16 (as there are 16 bytes:
"ddDddDdd" and "jjJjjJjj").
    |<------"XPAKPACK"----->||    32     |    16     |
 00  58 50 41 4b 50 41 43 4b  00 00 00 20 00 00 00 10

Now we have the first index element with a \fIname_len\fR of 4, followed
by the name string "fil1", followed by the data1 offset of 0 and a data1
len of 8 (since data1 has 8 bytes: "ddDddDdd").
    |     4     |<--"fil1"->||data1_off:0|data1_len:8|
 10  00 00 00 04 66 69 6c 31  00 00 00 00 00 00 00 08

Now we have the second index element with a \fIname_len\fR of 4, followed
by the name string "fil2", followed by the data2 offset of 8 and a data2
len of 8 (since data2 has 8 bytes: "jjJjjJjj").
    |     4     |<--"fil2"->||data2_off:8|data2_len:8|
 20  00 00 00 04 66 69 6c 32  00 00 00 08 00 00 00 08

    |<------"XPAKSTOP"----->|
 40  58 50 41 4b 53 54 4f 50
.SH AUTHORS
.nf
Lars Hartmann <lars@chaotika.org>
Mike Frysinger <vapier@gentoo.org>
.fi
.SH "SEE ALSO"
.BR qtbz2 (1),
.BR quickpkg (1),
.BR qxpak (1)