Usage
Imagine that you are creating a 101 Python
tutorial and you want to explain
the basics of the language writing a blog.
So you open your favorite editor, write some comments, you enhance them with
real Python
examples and you get something like this:
$ cat test/ds/python-tutorial.v1.md # byexample: +rm=
This is a 101 Python tutorial
The following is an example written in Python about arithmetics
```
>>> from __future__ import print_function
>>> 1 + 2
3
```
The next examples show you about complex numbers in Python
```
>>> 2j * 2
4
>>> 2j + 4j
6
```
Now we want to be sure that the examples in the blog are correct.
Just run byexample
selecting python
as the language target
$ byexample -l python test/ds/python-tutorial.v1.md
<...>
Failed example:
2j * 2
Expected:
4
Got:
4j
<...>
File test/ds/python-tutorial.v1.md, 4/4 test ran in <...> seconds
[FAIL] Pass: 2 Fail: 2 Skip: 0
byexample
will show the differences for each failing test and at the end
will print a summary showing how many examples were executed, how many passed,
failed or where skipped.
In normal circumstances there are two possible status: PASS
and FAIL
.
If something strange happen like the user pressed ctrl-c
, the underlying
runner crashed or an example couldn’t get parsed, the status will be ABORT
.
For quick regression you may want to stop byexample
at the first failing
example using --ff
or --fail-fast
to skip all the remaining examples.
$ byexample --ff -l python test/ds/python-tutorial.v1.md
<...>
File test/ds/python-tutorial.v1.md, 4/4 test ran in <...> seconds
[FAIL] Pass: 2 Fail: 1 Skip: 1
Fix the bugs an re-run
$ byexample --ff -l python test/ds/python-tutorial.v2.md
<...>
File test/ds/python-tutorial.v2.md, 4/4 test ran in <...> seconds
[PASS] Pass: 4 Fail: 0 Skip: 0
Help included
The help included in byexample
should give you a quick overview of its
capabilities
$ byexample -h # byexample: +norm-ws -tags +rm= +diff=ndiff
usage: byexample -l <languages> [--ff] [--timeout <secs>] [-j <n>] [--dry]
[--skip <file> [<file> ...]] [--capture-env-var <var names>]
[-d {none,unified,ndiff,context,tool}] [--difftool <cmd>]
[--no-enhance-diff] [-o <options>] [--show-options]
[-m <dir>] [--encoding <enc>[:<error>]] [--show-failures <n>]
[--pretty {none,all}] [--no-progress-bar] [-V] [-v | -q]
[-h | -xh]
Write snippets of code in C++, Python, Ruby, and others as documentation and
execute them as regression tests.
positional arguments:
<file> files that have the examples to run.
Language Selection:
-l <languages>, --language <languages>, --languages <languages>
select which languages to parse and run. Comma
separated syntax is also accepted.
Execution Options:
--ff, --fail-fast if an example fails, fail and stop all the execution.
--timeout <secs> timeout in seconds to complete each example (2 by
default); this can be changed per example with this
option.
-j <n>, --jobs <n> run <n> jobs in parallel (1 by default); <n> can be an
integer or the string "cpu" or "cpu<n>": "cpu" means
use all the cpus available; "cpu<n>" multiply it by
<n> the cpus available.
--dry do not run any example, only parse them.
--skip <file> [<file> ...]
skip these files
--capture-env-var <var names>, --capture-env-vars <var names>
capture some environment variables and put them in the
clipboard so they can be pasted and used in
conditional executions. Comma separated syntax is also
accepted.
Diff Options:
-d {none,unified,ndiff,context,tool}, --diff {none,unified,ndiff,context,tool}
select diff algorithm (none by default).
--difftool <cmd> command line to the external diff program; the tokens
%e and %g are replaced by the file names with the
expected and the got outputs to compare. Enabled only
if "--diff tool".
--no-enhance-diff by default, improves are made so the diff are easier
to to understand: non-printable characters are
visible; captured string shown, and more; this flag
disables all of that.
Miscellaneous Options:
-o <options>, --options <options>
add additional options; see --show-options to list
them.
--show-options show the available options for the selected languages
(with -l)
-m <dir>, --modules <dir>
append a directory for searching modules there.
--encoding <enc>[:<error>]
select the encoding and optionally the error handler
(default: utf-8:strict); valid error handlers are
"strict", "ignore" and "replace".
--show-failures <n> show up to <n> failures per file (all by default) and
suppress the rest (the execution of the examples is
not stopped, only the failures are not shown)
--pretty {none,all} control how to pretty print the output.
--no-progress-bar do not show the progress bar.
-V, --version show byexample's version and license, then exit
Logging:
-v verbosity level, add more flags to increase the level.
-q, --quiet quiet mode, do not print anything even if an example
fails; suppress the progress output.
Help Options:
-h, --help show this help message and exit
-xh show this help message plus the one for the advanced
options and exit
Examples:
byexample -l python file.py
byexample -l python,ruby --ff --timeout=8 file.md
byexample -l python,ruby --show-options
See https://byexamples.github.io for the full documentation
and more examples.