The following tooling is provided as part of the
The tools are validated on the Ubuntu 18.04.6 LTS Linux distribution running Python 3.6. Any particular dependencies required to run the tools are detailed in their relevant sections, below.
A script automating the documentation build process is available under
tools/build/doc-build.py. It will generate an HTML version of the
public/. To use this script and generate the
documentation, you should use Python 3.6 or higher:
For more information about the parameters, call the help function of the script:
To render and explore the documentation, simply open
public/index.html in a
Quality Assurance Checks¶
The project provides tooling for running quality assurance (QA) checks on the
repository, in order to automatically validate that patches adhere to the
project’s minimal code, submission, and documentation standards. The tooling is
provided as a set of Python scripts that can be found within the
tools/qa-checks/ directory of the repository.
run-checks.py script can be used to perform the validation, for example:
Detailed usage instructions can be found by passing
--help to the script:
python3 ./tools/qa-checks/run-checks.py --help
In order to run the tool, the system must have installed Python 3 (version 3.6 or greater), the PyYAML Python package available via pip (5.4.1 is the project’s currently supported version), and Git.
The tooling runs a set of modular checks to validate different aspects of the repository state. These are briefly described as follows:
- Commit message validation
This check ensures that the commit message submitted with the patch adheres to the project’s expected format.
- License and copyright header validation
This check validates the inclusion of correctly formatted copyright and license headers for relevant project files.
- Python code quality validation
This check ensures that all Python files within the project are compliant with the code-style conventions in PEP8 as validated by the pycodestyle utility.
- Shell script code quality validation
This check ensures that all shell scripts and BATS files within the project produce no warnings when passed to the ShellCheck static analysis tool.
- Spelling validation
This check aims to validate that there are no English word misspellings within the relevant project files (in particular, the appropriate files within the
documentation/directory). A custom dictionary is maintained within the
meta-ewaol-config/qa-checksdirectory that may be used to hold exceptions, should the check erroneously highlight valid technical terminology.
More detail on the validation steps performed by each check are included at the top of each check Python module as in-source documentation. In addition, any failed validation will output the specific reason for the failure, enabling it to be fixed.
CI Build Tool¶
The repository provides a Python script to support building EWAOL images for Continuous Integration (CI) purposes:
Given a space-separated list of build targets (each consisting of one or more
colon-separated YAML files present in the
this script will run a containerized kas command for each target. The container
instance will configure and build the image, then package and deploy it along
with relevant build artifacts for use by later CI pipeline stages.
All other dependencies of bitbake and kas will be handled by the docker container image provided for kas:
The container engine, image source, and version are configurable as command line arguments to the Python script. However, only Docker and the aforementioned image and version are currently supported.
The script can be passed one or more build targets as its list of positional arguments, where these build targets will be processed sequentially. Each build target consists of one or more kas config files, concatenated via a colon (:), which defines a desired build.
For example, to build an image for the n1sdp machine that includes the tests build configuration, run:
The available build config YAML files can be queried by passing to the script:
While all specified build config files must be available in the
meta-ewaol-config/kas directory, if
all is provided as the build target
, then all default CI build targets will be built. These default CI build
targets can be queried by passing:
The results of these options are defined in
meta-ewaol-config/ci/build-defs.yml, meaning that changing this build
definitions file will update the default CI targets produced by
By default, the script will set:
Target build directory:
Above directories can be changed with build script parameters:
for setting build output,
--sstate-dir for setting sstate cache directory
--dl-dir for setting downloads directory.
[build_id] is given by replacing each colon in the list of YAML
files for the build target with an underscore (_), and excluding all
file extensions. For example, the
n1sdp.yml:tests.yml build target above
would result in a default build folder:
The script can also be passed a set of optional named arguments, where these
arguments and their defaults can be found by passing
Note that by default no cache mirrors will be configured, and no artifacts will be deployed.
Interactive Build Container¶
The script allows the user to customize both the container engine arguments as well as the kas commands that will be executed, via the following script options:
--engine-arguments '--foo bar="baz"' ['--bar' [...]]allows for additional engine-specific options to be passed to the container engine, e.g.
-itto enable interactive access to a docker container.
--kas-arguments ARGScustomizes the arguments string passed to the kas command, allowing the user to run custom commands (e.g. to enter the bitbake environment by passing the
shellstring, or to pass specific options for running tests). The default string is
The Python build script supports read-only mirrors for the
SOURCE_MIRROR_URL mounted as local filepaths. Currently there is no
support for http(s) paths.
These paths can be provided using:
[path] refers to the path on the local machine, not a path internal
to the container image execution.
As an alternative option, the environment variables
SOURCE_MIRROR_URL will be carried through to the containerized bitbake
build if set in the build environment.
BB_GENERATE_MIRROR_TARBALLS are also passed through to bitbake using
--deploy-artifacts to the build script will package and compress
files produced in the bitbake build directories:
conf.tgzcontaining build config files stored within
logs.tgzcontaining the following log files:
All package build logs found in
pseudo.logfiles found in
images.tgzcontaining all files found in
By default the artifacts will be deployed in
ci-build/artifacts/, but this
can be configured by passing:
The artifacts path will be created if it does not exist.
The script produces a significant volume of build output to the terminal by default. Much of this output can be redirected to a log file using:
This will cause
STDOUT to contain only important messages, while the log
file will receive the full output. If enabled, the log file should be consulted
to check the current progress of the containerized build.
The log path will be created if it does not exist.