debbisect

NAME

debbisect - bisect snapshot.debian.org

DESCRIPTION

usage: debbisect [-h] [-d] [-v] [--cache CACHE] [--nocache] [--port PORT]

[--depends DEPENDS] [--qemu QEMU] [--architecture ARCHITECTURE] [--suite SUITE] [--components COMPONENTS] [--no-find-exact-package] [--ignore-cached-results] good bad script

Execute a script or a shell snippet for a known good timestamp and a known bad timestamp and then bisect the timestamps until a timestamp from snapshot.debian.org is found where the script first fails. Environment variables are used to tell the script which timestamp to test. See ENVIRONMENT VARIABLES below. At the end of the execution, the files debbisect.log.good and debbisect.log.bad are the log files of the last good and last bad run, respectively. By default, a temporary caching mirror is executed to reduce bandwidth usage on snapshot.debian.org. If you plan to run debbisect multiple times on a similar range of timestamps, consider setting a non-temporary cache directory with the --cache option. The program has three basic modes of operation. In the first, the given script is responsible to set up everything as needed:

$ ./debbisect "last week" today script.sh

 

$ diff -u debbisect.log.good debbisect.log.bad

If also the --depends option is given, then a chroot of the correct timestamp will be created each time and the script will receive as first argument the path to that chroot. Additionally, this mode allows debbisect to figure out the exact package that was responsible for the failure instead of only presenting you the last good and first bad timestamp. Lastly, you can also provide the --qemu option. In this mode, your test will be create a qemu virtual machine of the correct timestamp each time. The script will receive the correct ssh config to log into a host named qemu and execute arbitrary commands.

positional arguments:

good

good timestamp -- see section TIMESTAMPS for valid formats

bad

bad timestamp -- see section TIMESTAMPS for valid formats

script

test script -- can either be a shell code snippet or an executable script. A non zero exit code indicates failure. When also --depends is used, then the first argument to the script will be the chroot directory. When --qemu is used, then the first argument to the script will an ssh config for a host named qemu.

options:

-h, --help

show this help message and exit

-d, --debug

Print lots of debugging statements

-v, --verbose

Be verbose

--cache CACHE

cache directory -- by default $TMPDIR is used

--nocache

disable cache

--port PORT

manually choose port number for the apt cache instead of automatically choosing a free port

--depends DEPENDS

Comma separated list of binary packages the test script requires. With this option, the test script will run inside a chroot with the requested packages installed.

--qemu QEMU

Create qemu virtual machine and pass a ssh config file to the test script. This argument takes a commaseparated series of key=value pairs to specify the virtual machine memory size (via memsize) and the virtual machine disksize (via disksize). Sizes are measured in bytes or with common unit suffixes like M or G. To pick the default values (disksize=4G,memsize=1G) the shorthand 'defaults' can be passed.

--architecture ARCHITECTURE

Chosen architecture when creating the chroot with --depends or --qemu (default: native architecture)

--suite SUITE

Chosen suite when creating the chroot with --depends or --qemu (default: unstable)

--components COMPONENTS

Chosen components (separated by comma or whitespace) when creating the chroot with --depends or --qemu (default: main)

--no-find-exact-package

Normally, when the --depends argument is given so that debbisect takes care of managing dependencies, the precise package that introduced the problem is determined after bisection by installing the packages that differ between the last good and first bad timestamp one by one. This option disables this feature.

--ignore-cached-results

Perform a run for a timestamp even if a log file for it exists in the current directory

TIMESTAMPS

Valid good and bad timestamp formats are either:

> the format used by snapshot.debian.org

 

> ISO 8601 (with or without time, seconds and timezone)

 

> RFC 2822 (used in debian/changelog)

 

> all formats understood by the Python dateutil module (if installed)

 

> all formats understood by the Python parsedatetime module (if installed)

Without specifying the timezone explicitly, the local offset is used. Examples (corresponding to the items in above list, respectively):

> 20200313T065326Z

 

> 2020-03-13T06:53:26+00:00

 

> Fri, 29 Nov 2019 14:00:08 +0100

 

> 5:50 A.M. on June 13, 1990

 

> two weeks ago

The earliest timestamp that works with debbisect should be 2006-08-10.

ENVIRONMENT VARIABLES

The following environment variables are available to the test script: DEBIAN_BISECT_MIRROR Contains the caching mirror address. DEBIAN_BISECT_EPOCH Contains an integer representing the unix epoch of the

current timestamp. The value of this variable can

directly be assigned to SOURCE_DATE_EPOCH.

DEBIAN_BISECT_TIMESTAMP Contains a timestamp in the format used by

snapshot.debian.org. Can also be generated from

DEBIAN_BISECT_EPOCH via: date --utc --date=@$DEBIAN_BISECT_EPOCH +%Y%m%dT%H%M%SZ

DEBIAN_BISECT_* All environment variables starting with DEBIAN_BISECT_

are passed to the test script.

EXAMPLES

Just run "do_something" which runs the test and returns a non-zero exit on failure.

$ ./debbisect "last week" today "mmdebstrap --customize-hook='chroot \"\$1\" do_something' unstable - \$DEBIAN_BISECT_MIRROR >/dev/null"

 

$ diff -u debbisect.log.good debbisect.log.bad

Since the command can easily become very long and quoting very involved, lets instead use a script:

$ cat << END > script.sh

> #!/bin/sh

 

> set -exu

 

> mmdebstrap \

 

> --verbose \

 

> --aptopt='Acquire::Check-Valid-Until "false"' \

 

> --variant=apt \

 

> --include=pkga,pkgb,pkgc \

 

> --customize-hook='chroot "$1" dpkg -l' \

 

> --customize-hook='chroot "$1" do_something' \

 

> unstable \

 

> - \

 

> $DEBIAN_BISECT_MIRROR \

 

> >/dev/null

 

> END

$ chmod +x script.sh

 

$ ./debbisect --verbose --cache=./cache "two years ago" yesterday ./script.sh

 

$ diff -u debbisect.log.good debbisect.log.bad

 

$ rm -r ./cache

This example sets Acquire::Check-Valid-Until to not fail on snapshot timestamps from "two years ago", uses the "apt" variant (only Essential:yes plus apt), installs the packages required for the test using --include, runs "dpkg -l" so that we can see which packages differed in the logs at the end and uses --cache=cache so that the apt cache does not get discarded at the end and the command can be re-run without downloading everything from snapshot.debian.org again. If you want to build a source package you can use the script shipped by devscripts as /usr/share/doc/devscripts/examples/debbisect_buildsrc.sh and either use it unmodified like this:

$ DEBIAN_BISECT_SRCPKG=mysrc ./debbisect "two years ago" yesterday > /usr/share/doc/devscripts/examples/debbisect_buildsrc.sh

or use the script as a starting point to do your own custom builds. Once debbisect has finished bisecting and figured out the last good and the first bad timestamp, there might be more than one package that differs in version between these two timestamps. debbisect can figure out which package is the culprit if you hand it control over installing dependencies for you via the --depends option. With that option active, the script will not be responsible to set up a chroot itself but is given the path to an existing chroot as the first argument. Here is a real example that verifies the package responsible for Debian bug #912935:

$ ./debbisect --depends=botch "2018-11-17" "2018-11-22" 'chroot "$1" botch-dose2html --packages=/dev/null --help'

[...] test upgrading python3-minimal 3.6.7-1 -> 3.7.1-2...

upgrading python3-minimal triggered the problem

If you want to run above test under qemu, then you would run:

$ ./debbisect --depends=botch --qemu=defaults "2018-11-17" "2018-11-22" 'ssh -F "$1" qemu botch-dose2html --packages=/dev/null --help'

In the last two examples we omitted the --cache argument for brevity. But please make use of it to reduce the load on snapshot.debian.org.

AUTHOR

Written by Johannes Schauer Marin Rodrigues <[email protected]>