Shell
byexample
can execute shell commands using by default bash
configured
to be POSIX-conformant
but other shells are supported like dash
and ksh
.
Run the Shell
examples calling byexample
as:
$ byexample -l shell your-file-here # byexample: +skip
$ byexample -l shell -o '+shell=dash' your-file-here # byexample: +skip
Stability:
provisional
- low impact non backward compatibility changes may occur between versions; but in general a change like that will happen only between major versions.
Versions tested
We tested byexample
with the following versions of the language
and the underlying runner or interpreter:
Language | Runner/Interpreter |
---|---|
latest | 5.0.17 |
POSIX-conformant
byexample
configures bash
to run in POSIX-conformant by default.
If you execute an shell example and you get a syntax error, you may be using a non-POSIX syntax.
You can disable the POSIX-conformant from within bash
with set
+o posix
:
$ echo $POSIXLY_CORRECT # this Bash's variable says yes if we are in POSIX
y
$ set +o posix
$ echo $POSIXLY_CORRECT # we are not longer in POSIX mode, happy hacking
$ set -o posix
$ echo $POSIXLY_CORRECT # back to the default of byexample
y
Find interactive examples
For Shell
, we use the simple $
marker as the primary prompt
and >
as the secondary prompt to found examples in a document.
$ g () {
> c=$3
> c=$(( $c + $1 ))
> c=$(( $c + $2 ))
>
> echo $c
> }
$ g 1 2 3
6
Running in background
byexample
executes each example in sequence, one after the other, moving to
the next one only when the previous finished.
That means that you need to relay in the shell’s job control to put some process to work in background.
Moreover you need to make sure that the process doesn’t write anything to the console, otherwise its output will be mixed with the output of the next examples.
It is easy to do this using a redirection:
$ sleep 2 >/dev/null 2>&1 &
[<job-id>] <pid>
Notice how the &
operator tells the shell to put the process in the
background and the >/dev/null 2>&1
discards both standard output and
error streams.
Depending of the underlying shell when you send a process to the background the job control may print the job id and process id linked to that process.
bash
andksh
do this always but others likedash
does not.
You can get the process id of the process running in background and its job id to control it later.
$ echo "$!" # byexample: +paste
<pid>
$ jobs -l # byexample: +paste +norm-ws
[<job-id>]+ <pid> Running <...>
When a process finishes (or dies), the shell will print a message.
It may happen asynchronously unless you control, and probably kill the background process, and wait for it to finish.
Here is an example (the %%
is replaced by the job id by the shell):
$ kill %% ; wait # byexample: +timeout=4 +norm-ws +paste
[<job-id>]+ Terminated<...>
Subshells
If waiting for a process is not an option, you can start the process in background inside of a subshell:
$ ( sleep 1 & ) >/dev/null 2>&1
The &
operator tells the shell to put the process in the background
and because we grouped it in a subshell (using the parenthesis), any
job-control message will be suppressed.
The downside of this is that we cannot access to neither the process id nor the job id.
Therefore the job list will be empty:
$ jobs -l
Disabling the job monitor
An alternative solution is disabling the monitoring that the shell does.
This basically suppress any message about the end of a process that was running in background without loosing its process id or job id.
$ set +m
$ sleep 1 >/dev/null 2>&1 &
[<job-id>] <pid>
$ echo "$!" # byexample: +paste
<pid>
$ jobs -l # byexample: +paste +norm-ws
[<job-id>]+ <pid> Running <...>
To prove this we can wait enough and see that no asynchronous message is print.
$ sleep 2 # byexample: +timeout=4
$ echo "foreground"
foreground
You can re-enable it later
$ set -m # byexample: +pass -skip
The downside of this solution is that disabling the monitoring also prevent us
to send ^Z
and ^Y
signals from an interactive session among
other side effects.
See the documentation of your shell.
The following
stackoverflow
thread explains all of this in a very concise manner: https://stackoverflow.com/questions/11097761/is-there-a-way-to-make-bash-job-control-quiet/11111042#11111042
Killing any background process
This has nothing to do with byexample
but I found quite useful this trick.
To kill any running background process you can write:
$ kill -9 $(jobs -p) && wait # byexample: -skip +pass
The -skip
will make sure
that this example gets executed while the +pass
will ignore any output
so it will work even if there is no process to kill.
Stopping a process on inactivity or silence
Sometimes is useful to run a long-running process in foreground and after some period of inactivity or silence, stop it and get back the control of the shell.
For example, imagine that we want to read the new entries of a log file as soon as they are saved in the log file.
$ echo "some log line" > w/msg.log
We could use tail -f
for this. But if we do that, tail
will never end,
blocking the whole execution.
In these cases we can use the +stop-on-silence
option.
After some period of inactivity of the process, byexample
will stop it
returning back the control of the shell.
$ tail -f w/msg.log # byexample: +stop-on-silence
some log line
The process will be stopped, if you want that the process keeps running
in background execute bg
.
Or you can resume it in foreground with fg
, this enable us
to keep reading the new entries in the log.
$ echo "another log line" >> w/msg.log
$ fg # byexample: +stop-on-silence
tail -f w/msg.log
another log line
By default, +stop-on-silence
waits for 0.2
seconds of inactivity.
If your process is a little slower and sends data to the output
less frequently you can increase the wait time:
$ (sleep 0.4 ; echo "a slow line" >> w/msg.log) &
[2] <pid>
$ fg %1 # byexample: +stop-on-silence=0.5
tail -f w/msg.log
a slow line
$ jobs
[1]<...>Stopped<...>
Note:
+stop-on-silence
requires the job control and monitoring to be enabled (set -m
). This should be the default in your shell.
Changed in
byexample 8.0.0
: before the+stop-on-silence
had the same behaviour than+stop-on-timeout
stopping the process always on timeout. But in8.0.0
this option was fixed and the old behaviour can be achieved using+stop-on-timeout
.
Stopping on timeout
+stop-on-silence
will stop a process if this one times out, which
it is basically another kind of inactivity.
But sometimes you have a process that it is continually sending data and you want to stop it after some period of time.
+stop-on-silence
will work, but it is not the correct tool for this job.
Instead, +stop-on-timeout
is the correct one: instead of checking
periodically if there is activity or not, byexample
will wait for the process
to timeout and it will stop it later.
New in
byexample 8.0.0
.
Note:
+stop-on-timeout
requires the job control and monitoring to be enabled (set -m
). This should be the default in your shell.
Stop signal
+stop-on-timeout
and +stop-on-silence
stop the running process but
what does it mean?
By default byexample
sends a suspend
signal (SIGTSTP
or ^Z
in
Linux parlance) which stops the process and sends it to background.
The signal used can be changed with +stop-signal
.
If you want to interrupt the process and very likely finish it, send
an interrupt
signal (SIGINT
or ^C
):
$ sleep 300 # byexample: +stop-on-silence +stop-signal=interrupt
If the process is waiting for input perhaps closing the standard input is more adequate than interrupting the process.
For that send an eof
signal (^D
):
$ cat - # byexample: +stop-on-silence +stop-signal=eof
New in
byexample 10.1.0
.
Signals like SIGTERM
and SIGKILL
cannot be used from
+stop-signal
; the closest thing is the quit
signal (SIGQUIT
or ^\
):
$ cat - # byexample: +stop-on-silence +stop-signal=quit
New in
byexample 10.5.2
.
Arbitrary signals on stop
If you want to kill a process with SIGKILL
or SIGTERM
you will have to stop the process first with a suspend
signal (the
default for +stop-signal
), capture the process’ id or job’s id and
kill it calling kill
:
$ cat - # byexample: +stop-on-silence
$ # The following sends a SIGTERM. Replace the -15 by -9 to send
$ # a SIGKILL. See signal(7).
$ kill -15 %%
<...>
In fact, you can use this trick to send any signal like SIGUSR1
.
In some cases you will have to bring the process to foreground (with
fg
to give it a chance to receive the signal).
$ fg # byexample: +pass
Using other shells
byexample
supports bash
, dash
and ksh
and the shell
by default is set to bash
in POSIX-conformant mode.
Changed in
byexample 8.1.0
: before the default shell wassh
. However different Linux distros have different shells behind the name ofsh
: in Debian it isdash
while in Red Hat it isbash
. And this changed over the time: Ubuntu hadbash
but in Ubuntu 6.10 it changed the shell todash
.To have a stable shell, since
byexample 8.1.0
it is explicitly set tobash
.
You can change the default shell from the command line with the
+shell
option.
$ byexample -l shell -o '+shell=bash' test/ds/shell-example # byexample: +timeout=8
<...>
[PASS] Pass: 14 Fail: 0 Skip: 0
$ byexample -l shell -o '+shell=dash' test/ds/shell-example # byexample: +if=dash-installed +timeout=8
<...>
[PASS] Pass: 14 Fail: 0 Skip: 0
$ byexample -l shell -o '+shell=ksh' test/ds/shell-example # byexample: +if=ksh-installed +timeout=8
<...>
[PASS] Pass: 14 Fail: 0 Skip: 0
The option can only be set from the command line and it will affect all the shell examples (you cannot change the shell only for a single example)
New in
byexample 8.1.0
.For backward compatibility with
byexample 8.0.0
and earlier versions, you can use+shell=sh
; however, we encourage to you to set a more specific shell.
If another shell is needed, byexample
allows
you to use -x-shebang
to control
how to spawn a runner, in this case,
a shell.
You could run ssh
to have a remote shell for example.
Or you could run bash
without the constraint to be POSIX-conformant
by default (but see set +o posix
as mentioned above before trying to do this):
$ byexample -l shell -x-shebang 'shell:%e bash --norc --noprofile --noediting' test/ds/shell-example
<...>
[PASS] Pass: 14 Fail: 0 Skip: 0
Bash history expansion
Since byexample 11.0.0
the history expansion of Bash is disabled
(calling set +H
at the begin of the shell execution).
This feature had troubles in some old versions of Bash and it is usefulness is limited anyways.
Shell specific options
$ byexample -l shell --show-options # byexample: +norm-ws
<...>
shell's specific options
------------------------
<...>:
+stop-on-timeout stop the process if it timeout.
+stop-on-silence [secs]
stop the process if no output is read in the last
<secs> seconds (0.2 secs by default).
+stop-signal {suspend,eof,interrupt,quit}
signal to send when stop-on-timeout/stop-on-silence is
used (suspend ^Z by default).
+shell {bash,dash,ksh,sh}
shell to use with default settings ('bash' by
default). For full control use -x-shebang)