guestfs-hacking - extending and contributing to libguestfs
This manual page is for hackers who want to extend libguestfs itself.
Libguestfs source is located in the github repository
https://github.com/libguestfs/libguestfs
Large amounts of boilerplate code in libguestfs (RPC, bindings, documentation)
are generated. This means that many source files will appear to be missing
from a straightforward git checkout. You have to run the generator
("./configure && make -C generator") in order to create
those files.
Libguestfs uses an autotools-based build system, with the main files being
configure.ac and
Makefile.am. See "THE BUILD SYSTEM".
The
generator subdirectory contains the generator, plus files describing
the API. The
lib subdirectory contains source for the library. The
appliance and
daemon subdirectories contain the source for the
code that builds the appliance, and the code that runs in the appliance
respectively. Other directories are covered in the section "SOURCE CODE
SUBDIRECTORIES" below.
Apart from the fact that all API entry points go via some generated code, the
library is straightforward. (In fact, even the generated code is designed to
be readable, and should be read as ordinary code). Some actions run entirely
in the library, and are written as C functions in files under
lib.
Others are forwarded to the daemon where (after some generated RPC
marshalling) they appear as C functions in files under
daemon.
To build from source, first read the
guestfs-building(1).
There are a lot of subdirectories in the source tree! Which ones should you
concentrate on first?
lib and
daemon which contain the source
code of the core library.
generator is the code generator described
above, so that is important. The
Makefile.am in the root directory will
tell you in which order the subdirectories get built. And then if you are
looking at a particular tool (eg.
customize) or language binding (eg.
python), go straight to that subdirectory, but remember that if you
didn't run the generator yet, then you may find files which appear to be
missing.
- align
-
virt-alignment-scan(1)
のコマンドおよびドキュメント。
- appliance
- libguestfs
アプライアンス、スクリプトなどを構築します。
- bash
- Bash tab-completion scripts.
- build-aux
- autotools
により使用されるさまざまなビルドスクリプト。
- builder
-
virt-builder(1)
コマンドおよびドキュメント。
- bundled
- Embedded copies of other libraries, mostly for convenience
(and the embedded library is not widespread enough).
- cat
- The virt-cat(1), virt-filesystems(1),
virt-log(1), virt-ls(1) and virt-tail(1) commands and
documentation.
- common
- Various libraries of internal code can be found in the
common subdirectory:
- common/edit
- Common code for interactively and non-interactively editing
files within a libguestfs filesystem.
- common/errnostring
- The communication protocol used between the library and the
daemon running inside the appliance has to encode errnos as strings, which
is handled by this library.
- common/mlcustomize
- Library code associated with "virt-customize" but
also used in other tools.
- common/mlgettext
- Small, generated wrapper which allows libguestfs to be
compiled with or without ocaml-gettext. This is generated by
./configure.
- common/mlpcre
- Lightweight OCaml bindings for Perl Compatible Regular
Expressions (PCRE). Note this is not related in any way to Markus Mottl's
ocaml-pcre library.
- common/mlprogress
- OCaml bindings for the progress bar functions (see
common/progress).
- common/mlstdutils
- A library of pure OCaml utility functions used in many
places.
- common/mltools
- OCaml utility functions only used by the OCaml virt tools
(like "virt-sysprep", "virt-customize" etc.)
- common/mlutils
- OCaml bindings for C functions in "common/utils",
and some POSIX bindings which are missing from the OCaml stdlib.
- common/mlvisit
- OCaml bindings for the visit functions (see
common/visit).
- common/mlxml
- OCaml bindings for the libxml2 library.
- common/options
- Common options parsing for guestfish, guestmount and some
virt tools.
- common/parallel
- A framework used for processing multiple libvirt domains in
parallel.
- common/progress
- Common code for printing progress bars.
- common/protocol
- The XDR-based communication protocol used between the
library and the daemon running inside the appliance is defined here.
- common/qemuopts
- Mini-library for writing qemu command lines and qemu config
files.
- common/structs
- Common code for printing and freeing libguestfs structs,
used by the library and some tools.
- common/utils
- Various utility functions used throughout the library and
tools.
- common/visit
- Recursively visit a guestfs filesystem hierarchy.
- common/windows
- Utility functions for handling Windows drive letters.
- contrib
- 外部の貢献、実験的な部分です。
- customize
-
virt-customize(1) command and documentation.
- daemon
- libguestfs
アプライアンスの中で実行され、アクションを実行しているデーモン。
- df
-
virt-df(1)
のコマンドおよびドキュメント。
- dib
-
virt-dib(1) command and documentation.
- diff
-
virt-diff(1) command and documentation.
- docs
- Miscellaneous manual pages.
- edit
-
virt-edit(1)
のコマンドおよびドキュメント。
- examples
- C API のコード例。
- fish
-
guestfish(1)
コマンドラインシェル、および
virt-copy-in(1), virt-copy-out(1), virt-tar-in(1),
virt-tar-out(1)
のようなさまざまなシェルスクリプト。
- format
-
virt-format(1)
のコマンドおよびドキュメント。
- fuse
-
guestmount(1), libguestfs
の上に組み立てられた
FUSE
(ユーザー空間ファイルシステム)。
- generator
- The crucially important generator, used to automatically
generate large amounts of boilerplate C code for things like RPC and
bindings.
- get-kernel
-
virt-get-kernel(1) command and documentation.
- inspector
-
virt-inspector(1),
仮想マシンイメージ検査ツール。
- lib
- C
ライブラリーのソースコード。
- logo
- Logo used on the website. The fish is called Arthur by the
way.
- m4
- M4 macros used by autoconf. See "THE BUILD
SYSTEM".
- make-fs
-
virt-make-fs(1) command and documentation.
- po
- シンプルな gettext
文字列の翻訳。
- po-docs
- The build infrastructure and PO files for translations of
manpages and POD files. Eventually this will be combined with the
po directory, but that is rather complicated.
- rescue
-
virt-rescue(1)
のコマンドおよびドキュメント。
- resize
-
virt-resize(1)
のコマンドおよびドキュメント。
- sparsify
-
virt-sparsify(1)
のコマンドおよびドキュメント。
- sysprep
-
virt-sysprep(1)
コマンドおよびドキュメント。
- tests
- テストします。
- test-data
- Files and other test data used by the tests.
- test-tool
- エンドユーザーが
QEMU/カーネルの組み合わせが
libguestfs
で動作するかどうかを確認するためのテストツールです。
- tmp
- Used for temporary files when running the tests (instead of
/tmp etc). The reason is so that you can run multiple parallel
tests of libguestfs without having one set of tests overwriting the
appliance created by another.
- tools
- Perl
で書かれたコマンドラインツール
( virt-win-reg(1)
および他の多くのもの)。
- utils
- Miscellaneous utilities, such as
"boot-benchmark".
- v2v
- Up to libguestfs > 1.42 this contained the
virt-v2v(1) tool, but this has now moved into a separate
repository: https://github.com/libguestfs/virt-v2v
- website
- The http://libguestfs.org website files.
- csharp
- erlang
- gobject
- golang
- haskell
- java
- lua
- ocaml
- php
- perl
- python
- ruby
- 言語バインディング。
Libguestfs uses the GNU autotools build system (autoconf, automake, libtool).
The
./configure script is generated from
configure.ac and
m4/guestfs-*.m4. Most of the configure script is split over many m4
macro files by topic, for example
m4/guestfs-daemon.m4 deals with the
dependencies of the daemon.
The job of the top level
Makefile.am is mainly to list the subdirectories
("SUBDIRS") in the order they should be compiled.
common-rules.mk is included in every
Makefile.am (top level and
subdirectories).
subdir-rules.mk is included only in subdirectory
Makefile.am files.
There are many make targets. Use this command to list them all:
make help
Because large amounts of boilerplate code in libguestfs are generated, this
makes it easy to extend the libguestfs API.
To add a new API action there are two changes:
- 1.
- You need to add a description of the call (name,
parameters, return type, tests, documentation) to
generator/actions_*.ml and possibly generator/proc_nr.ml.
There are two sorts of API action, depending on whether the call goes
through to the daemon in the appliance, or is serviced entirely by the
library (see "ARCHITECTURE" in guestfs-internals(1)).
"guestfs_sync" in guestfs(3) is an example of the former,
since the sync is done in the appliance. "guestfs_set_trace" in
guestfs(3) is an example of the latter, since a trace flag is
maintained in the handle and all tracing is done on the library side.
Most new actions are of the first type, and get added to the
"daemon_functions" list. Each function has a unique procedure
number used in the RPC protocol which is assigned to that action when we
publish libguestfs and cannot be reused. Take the latest procedure number
and increment it.
For library-only actions of the second type, add to the
"non_daemon_functions" list. Since these functions are serviced
by the library and do not travel over the RPC mechanism to the daemon,
these functions do not need a procedure number, and so the procedure
number is set to "-1".
- 2.
- Implement the action (in C):
For daemon actions, implement the function "do_<name>" in
the "daemon/" directory.
For library actions, implement the function
"guestfs_impl_<name>" in the "lib/" directory.
In either case, use another function as an example of what to do.
- 3.
- As an alternative to step 2: Since libguestfs 1.38, daemon
actions can be implemented in OCaml. You have to set the "impl =
OCaml ..." flag in the generator. Take a look at
daemon/file.ml for an example.
これらの変更をした後、コンパイルするために
"make"
を使用してください。
Note that you don’t need to implement the RPC, language bindings, manual
pages or anything else. It’s all automatically generated from the OCaml
description.
Adding tests for an API
You can supply zero or as many tests as you want per API call. The tests can
either be added as part of the API description (
generator/actions_*.ml), or in some rarer cases you may want to drop a
script into "tests/*/". Note that adding a script to
"tests/*/" is slower, so if possible use the first method.
The following describes the test environment used when you add an API test in
actions_*.ml.
テスト環境は 4
個のブロックデバイスを持ちます:
-
/dev/sda 2 GB
- テスト用の一般的なブロックデバイス。
-
/dev/sdb 2 GB
-
/dev/sdb1 is an ext2 filesystem used for testing
filesystem write operations.
-
/dev/sdc 10 MB
- 2
つのブロックデバイスが必要となるいくつかのテストにおいて使用されます。
- /dev/sdd
- ISO with fixed content (see images/test.iso).
To be able to run the tests in a reasonable amount of time, the libguestfs
appliance and block devices are reused between tests. So don't try testing
"guestfs_kill_subprocess" in
guestfs(3) :-x
Each test starts with an initial scenario, selected using one of the
"Init*" expressions, described in
generator/types.ml. These
initialize the disks mentioned above in a particular way as documented in
types.ml. You should not assume anything about the previous contents of
other disks that are not initialized.
You can add a prerequisite clause to any individual test. This is a run-time
check, which, if it fails, causes the test to be skipped. Useful if testing a
command which might not work on all variations of libguestfs builds. A test
that has prerequisite of "Always" means to run unconditionally.
In addition, packagers can skip individual tests by setting environment
variables before running "make check".
SKIP_TEST_<CMD>_<NUM>=1
eg: "SKIP_TEST_COMMAND_3=1" skips test #3 of
"guestfs_command" in
guestfs(3).
または:
SKIP_TEST_<CMD>=1
eg: "SKIP_TEST_ZEROFREE=1" skips all "guestfs_zerofree" in
guestfs(3) tests.
Packagers can run only certain tests by setting for example:
TEST_ONLY="vfs_type zerofree"
See
tests/c-api/tests.c for more details of how these environment
variables work.
Debugging new APIs
Test new actions work before submitting them.
新しいコマンドを試すために
guestfish
を使うことができます。
Debugging the daemon is a problem because it runs inside a minimal environment.
However you can fprintf messages in the daemon to stderr, and they will show
up if you use "guestfish -v".
All language bindings must be generated by the generator (see the
generator subdirectory).
There is no documentation for this yet. We suggest you look at an existing
binding, eg.
generator/ocaml.ml or
generator/perl.ml.
Adding tests for language bindings
Language bindings should come with tests. Previously testing of language
bindings was rather ad-hoc, but we have been trying to formalize the set of
tests that every language binding should use.
Currently only the OCaml and Perl bindings actually implement the full set of
tests, and the OCaml bindings are canonical, so you should emulate what the
OCaml tests do.
This is the numbering scheme used by the tests:
- 000+ basic tests:
010 load the library
020 create
030 create-flags
040 create multiple handles
050 test setting and getting config properties
060 explicit close
065 implicit close (in GC'd languages)
070 optargs
080 version
090 retvalues
- 100 launch, create partitions and LVs and filesystems
- 400+ events:
410 close event
420 log messages
430 progress messages
- 800+ regression tests (specific to the language)
- 900+ any other custom tests for the language
To save time when running the tests, only 100, 430, 800+, 900+ should launch the
handle.
Our C source code generally adheres to some basic code-formatting conventions.
The existing code base is not totally consistent on this front, but we do
prefer that contributed code be formatted similarly. In short, use
spaces-not-TABs for indentation, use 2 spaces for each indentation level, and
other than that, follow the K&R style.
If you use Emacs, add the following to one of your start-up files (e.g.,
~/.emacs), to help ensure that you get indentation right:
;;; In libguestfs, indent with spaces everywhere (not TABs).
;;; Exceptions: Makefile and ChangeLog modes.
(add-hook 'find-file-hook
'(lambda () (if (and buffer-file-name
(string-match "/libguestfs\\>"
(buffer-file-name))
(not (string-equal mode-name "Change Log"))
(not (string-equal mode-name "Makefile")))
(setq indent-tabs-mode nil))))
;;; When editing C sources in libguestfs, use this style.
(defun libguestfs-c-mode ()
"C mode with adjusted defaults for use with libguestfs."
(interactive)
(c-set-style "K&R")
(setq c-indent-level 2)
(setq c-basic-offset 2))
(add-hook 'c-mode-hook
'(lambda () (if (string-match "/libguestfs\\>"
(buffer-file-name))
(libguestfs-c-mode))))
Turn warnings into errors when developing to make warnings hard to ignore:
./configure --enable-werror
有用なターゲットは次のとおりです:
- "make check"
- 一般的なテスト群を実行します。
This is implemented using the regular automake "TESTS" target. See
the automake documentation for details.
- "make check-valgrind"
- valgrind
にあるテスト群のサブセットを実行します。
See "VALGRIND" below.
- "make check-valgrind-local-guests"
- ローカルにインストールされた
libvirt 仮想マシン
(読み込み専用)
を使用して、valgrind
にあるテスト群のサブセットを実行します。
- "make check-direct"
- Runs all tests using default appliance back-end. This only
has any effect if a non-default backend was selected using
"./configure --with-default-backend=..."
- "make check-valgrind-direct"
- Run a subset of the test suite under valgrind using the
default appliance back-end.
- "make check-with-upstream-qemu"
- Runs all tests using a local qemu binary. It looks for the
qemu binary in QEMUDIR (defaults to $HOME/d/qemu),
but you can set this to another directory on the command line, eg:
make check-with-upstream-qemu QEMUDIR=/usr/src/qemu
- "make check-with-upstream-libvirt"
- Runs all tests using a local libvirt. This only has any
effect if the libvirt backend was selected using "./configure
--with-default-backend=libvirt"
It looks for libvirt in LIBVIRTDIR (defaults to
$HOME/d/libvirt), but you can set this to another
directory on the command line, eg:
make check-with-upstream-libvirt LIBVIRTDIR=/usr/src/libvirt
- "make check-slow"
- Runs some slow/long-running tests which are not run by
default.
To mark a test as slow/long-running:
- •
- Add it to the list of "TESTS" in the
Makefile.am, just like a normal test.
- •
- Modify the test so it checks if the "SLOW=1"
environment variable is set, and if not set it skips (ie. returns
with exit code 77). If using $TEST_FUNCTIONS, you can call the function
"slow_test" for this.
- •
- Add a variable "SLOW_TESTS" to the
Makefile.am listing the slow tests.
- •
- Add a rule to the Makefile.am:
check-slow:
$(MAKE) check TESTS="$(SLOW_TESTS)" SLOW=1
- "sudo make check-root"
- Runs some tests which require root privileges. These are
supposed to be safe, but take care. You have to run this as root (eg.
using sudo(8) explicitly).
To mark a test as requiring root:
- •
- Add it to the list of "TESTS" in the
Makefile.am, just like a normal test.
- •
- Modify the test so it checks if euid == 0, and if
not set it skips (ie. returns with exit code 77). If using
$TEST_FUNCTIONS, you can call the function "root_test" for
this.
- •
- Add a variable "ROOT_TESTS" to the
Makefile.am listing the root tests.
- •
- Add a rule to the Makefile.am:
check-root:
$(MAKE) check TESTS="$(ROOT_TESTS)"
- "make check-all"
- Equivalent to running all "make check*" rules
except "check-root".
- "make check-release"
- Runs a subset of "make check*" rules that are
required to pass before a tarball can be released. Currently this is:
- •
- check
- •
- check-valgrind
- •
- check-direct
- •
- check-valgrind-direct
- •
- check-slow
- "make installcheck"
- Run "make check" on the installed copy of
libguestfs.
The version of installed libguestfs being tested, and the version of the
libguestfs source tree must be the same.
Do:
./configure
make clean ||:
make
make installcheck
When you do "make check-valgrind", it searches for any
Makefile.am in the tree that has a "check-valgrind:" target
and runs it.
Writing the
Makefile.am and tests correctly to use valgrind and working
with automake parallel tests is subtle.
If your tests are run via a shell script wrapper, then in the wrapper use:
$VG virt-foo
and in the
Makefile.am use:
check-valgrind:
make VG="@VG@" check
However, if your binaries run directly from the "TESTS" rule, you have
to modify the
Makefile.am like this:
LOG_COMPILER = $(VG)
check-valgrind:
make VG="@VG@" check
In either case, check that the right program is being tested by examining the
tmp/valgrind* log files carefully.
パッチをメーリングリストに提出します:
http://www.redhat.com/mailman/listinfo/libguestfs および
[email protected] (Cc)。
You do not need to subscribe to the mailing list if you don’t want to.
There may be a short delay while your message is moderated.
We support i18n (gettext anyhow) in the library.
However many messages come from the daemon, and we don’t translate those
at the moment. One reason is that the appliance generally has all locale files
removed from it, because they take up a lot of space. So we'd have to readd
some of those, as well as copying our PO files into the appliance.
Debugging messages are never translated, since they are intended for the
programmers.
Mostly this section is "how we make automake & ocamlopt work
together" since OCaml programs themselves are easy to compile.
Automake has no native support for OCaml programs, ocamlc nor ocamlopt. What we
do instead is to treat OCaml programs as C programs which happen to contain
these "other objects" ("DEPENDENCIES" in automake-speak)
that happen to be the OCaml objects. This works because OCaml programs usually
have C files for native bindings etc.
So a typical program is described as just its C sources:
virt_customize_SOURCES = ... crypt-c.c perl_edit-c.c
For programs that have no explicit C sources, we create an empty
dummy.c
file, and list that instead:
virt_resize_SOURCES = dummy.c
The OCaml objects which contain most of the code are listed as automake
dependencies (other dependencies may also be listed):
virt_customize_DEPENDENCIES = ... customize_main.cmx
The only other special thing we need to do is to provide a custom link command.
This is needed because automake won't assemble the ocamlopt command, the list
of objects and the "-cclib" libraries in the correct order
otherwise.
virt_customize_LINK = \
$(top_builddir)/ocaml-link.sh -cclib '-lutils' -- ...
The actual rules, which you can examine in
customize/Makefile.am, are a
little bit more complicated than this because they have to handle:
- •
- Compiling for byte code or native code.
- •
- The pattern rules needed to compile the OCaml sources to
objects.
These are now kept in subdir-rules.mk at the top level, which is
included in every subdirectory Makefile.am.
- •
- Adding OCaml sources files to "EXTRA_DIST".
Automake isn't aware of the complete list of sources for a binary, so it
will not add them all automatically.
These "make" targets probably won’t work and aren't useful
unless you are a libguestfs maintainer.
make maintainer-commit
This commits everything in the working directory with the commit message
"Version $(VERSION).". You must update
configure.ac, clean
and rebuild first.
make maintainer-tag
This tags the current HEAD commit with the tag "v$(VERSION)" and one
of the messages:
Version $(VERSION) stable
Version $(VERSION) development
(See "LIBGUESTFS VERSION NUMBERS" in
guestfs(3) for the
difference between a stable and development release.)
make maintainer-check-authors
Check that all authors (found in git commit messages) are included in the
generator/authors.ml file.
make maintainer-check-extra-dist
This rule must be run after "make dist" (so there is a tarball in the
working directory). It compares the contents of the tarball with the contents
of git to ensure that no files have been missed from
Makefile.am
"EXTRA_DIST" rules.
make maintainer-upload-website
This is used by the software used to automate libguestfs releases to copy the
libguestfs website to another git repository before it is uploaded to the web
server.
When we make a stable release, there are several steps documented here. See
"LIBGUESTFS VERSION NUMBERS" in
guestfs(3) for general
information about the stable branch policy.
- •
- Check "make && make check" works on at
least:
- Fedora (x86-64)
- Debian (x86-64)
- Ubuntu (x86-64)
- Fedora (aarch64)
- Fedora (ppc64)
- Fedora (ppc64le)
- •
- Check "./configure --without-libvirt" works.
- •
- Finalize guestfs-release-notes.pod
- •
- http://libguestfs.org/download
の下に新しい安定版および開発版のディレクトリーを作成します。
- •
- Edit website/index.html.in.
- •
- Set the version (in configure.ac) to the new
stable version, ie. 1.XX.0, and commit it:
./localconfigure
make distclean -k
./localconfigure
make && make dist
make maintainer-commit
make maintainer-tag
- •
- Create the stable branch in git:
git branch stable-1.XX
git push origin stable-1.XX
- •
- Do a full release of the stable branch.
- •
- Set the version to the next development version and commit
that. Optionally do a full release of the development branch.
This section documents internal functions inside libguestfs and various
utilities. It is intended for libguestfs developers only.
This section is autogenerated from "/**" comments in source files,
which are marked up in POD format.
These functions are not publicly exported, and may change or be removed at
any time.
__INTERNAL_DOCUMENTATION__
guestfs(3),
guestfs-building(1),
guestfs-examples(3),
guestfs-internals(1),
guestfs-performance(1),
guestfs-release-notes(1),
guestfs-testing(1),
libguestfs-test-tool(1),
libguestfs-make-fixed-appliance(1),
http://libguestfs.org/.
Richard W.M. Jones ("rjones at redhat dot com")
Copyright (C) 2009-2020 Red Hat Inc.
To get a list of bugs against libguestfs, use this link:
https://bugzilla.redhat.com/buglist.cgi?component=libguestfs&product=Virtualization+Tools
To report a new bug against libguestfs, use this link:
https://bugzilla.redhat.com/enter_bug.cgi?component=libguestfs&product=Virtualization+Tools
When reporting a bug, please supply:
- •
- The version of libguestfs.
- •
- Where you got libguestfs (eg. which Linux distro, compiled
from source, etc)
- •
- Describe the bug accurately and give a way to reproduce
it.
- •
- Run libguestfs-test-tool(1) and paste the
complete, unedited output into the bug report.