pax_global_header00006660000000000000000000000064151356130600014512gustar00rootroot0000000000000052 comment=99bfac9a0b7154548a99553426742e9b8be40429 avocado-113.0/000077500000000000000000000000001513561306000131325ustar00rootroot00000000000000avocado-113.0/.copr/000077500000000000000000000000001513561306000141535ustar00rootroot00000000000000avocado-113.0/.copr/Makefile000066400000000000000000000016251513561306000156170ustar00rootroot00000000000000pkg_install := $(shell dnf -y install git rpm-build) pwd := $(shell pwd) git_config_safe_directory := $(shell git config --global --add safe.directory $(pwd)) commit := $(shell git log --pretty=format:'%H' -n 1) commit_date := $(shell git log --pretty='format:%cd' --date='format:%Y%m%d' -n 1) short_commit := $(shell git rev-parse --short=9 HEAD) srpm: if test ! -d SOURCES; then mkdir SOURCES; fi if test ! -d SPEC; then mkdir SPEC; fi git archive --prefix="avocado-$(commit)/" -o "SOURCES/avocado-$(short_commit).tar.gz" HEAD cp python-avocado.spec SPEC sed -i -e 's/\%global rel_build .*/\%global rel_build 0/' SPEC/python-avocado.spec sed -i -e 's/\%global commit .*/\%global commit $(commit)/' SPEC/python-avocado.spec sed -i -e 's/\%global commit_date .*/\%global commit_date $(commit_date)/' SPEC/python-avocado.spec rpmbuild -D '_topdir .' -bs SPEC/python-avocado.spec mv SRPMS/*.src.rpm $(outdir) avocado-113.0/.coveragerc000066400000000000000000000001411513561306000152470ustar00rootroot00000000000000[run] source = avocado/, optional_plugins/ concurrency = multiprocessing, thread parallel = true avocado-113.0/.gitattributes000066400000000000000000000000561513561306000160260ustar00rootroot00000000000000*.min.css binary *.min.js binary *.svg binary avocado-113.0/.github/000077500000000000000000000000001513561306000144725ustar00rootroot00000000000000avocado-113.0/.github/ISSUE_TEMPLATE/000077500000000000000000000000001513561306000166555ustar00rootroot00000000000000avocado-113.0/.github/ISSUE_TEMPLATE/bug_report.md000066400000000000000000000012631513561306000213510ustar00rootroot00000000000000--- name: Bug report about: Create a report to help us improve title: '' labels: bug assignees: '' --- **Describe the bug** A clear and concise description of what the bug is. **Steps to reproduce** Steps to reproduce the behavior with a reproducible whenever possible. **Expected behavior** A clear and concise description of what you expected to happen. **Current behavior** What you are getting? **System information (please complete the following information):** - OS: `lsb_release -a` - Avocado version: `avocado -v` - Avocado installation method: pip, rpm, github ? **Additional information** Add any other information about the problem here. Test details can be added here. avocado-113.0/.github/ISSUE_TEMPLATE/feature_request.md000066400000000000000000000011631513561306000224030ustar00rootroot00000000000000--- name: Feature request about: Suggest an idea for this project title: '' labels: enhancement assignees: '' --- **Is your feature request related to a problem? Please describe.** A clear and concise description of what the problem is. Ex. I'm always missing this feature when [...] **Describe the solution you'd like** A clear and concise description of what you want to happen. **Describe alternatives you've considered** A clear and concise description of any alternative solutions or features you've considered. **Additional information** Add any additional information or screenshots about the feature request here. avocado-113.0/.github/actions/000077500000000000000000000000001513561306000161325ustar00rootroot00000000000000avocado-113.0/.github/actions/egg/000077500000000000000000000000001513561306000166745ustar00rootroot00000000000000avocado-113.0/.github/actions/egg/action.yml000066400000000000000000000011661513561306000207000ustar00rootroot00000000000000name: Egg description: Test running avocado from eggs runs: using: composite steps: - name: Test running avocado from eggs shell: sh run: | python3 setup.py bdist_egg mv dist/avocado_framework-*egg /tmp python3 setup.py clean --all python3 -c 'import sys; import glob; sys.path.insert(0, glob.glob("/tmp/avocado_framework-*.egg")[0]); from avocado.core.main import main; sys.exit(main())' run /bin/true cd /tmp python3 -c 'import sys; from pkg_resources import require; require("avocado-framework"); from avocado.core.main import main; sys.exit(main())' run /bin/true avocado-113.0/.github/actions/version/000077500000000000000000000000001513561306000176175ustar00rootroot00000000000000avocado-113.0/.github/actions/version/action.yml000066400000000000000000000003701513561306000216170ustar00rootroot00000000000000name: Version description: Installs and run avocado --version runs: using: composite steps: - name: Install and run avocado --version shell: sh run: | python3 setup.py develop --user python3 -m avocado --version avocado-113.0/.github/workflows/000077500000000000000000000000001513561306000165275ustar00rootroot00000000000000avocado-113.0/.github/workflows/ansible.yml000066400000000000000000000017621513561306000206750ustar00rootroot00000000000000name: Ansible deployment test on: schedule: - cron: "0 5 * * 1" # Allows you to run this workflow manually from the Actions tab workflow_dispatch: jobs: podman: name: Ansible (${{matrix.extra}}) runs-on: ubuntu-latest container: image: fedora:40 env: RUN_BEFORE: 'dnf install -y git ansible' GIT_URL: 'https://github.com/${{github.repository}}' INVENTORY: 'selftests/deployment/inventory' PLAYBOOK: 'selftests/deployment/deployment.yml' strategy: matrix: extra: ["method=pip", "method=copr", "method=official", "method=pip avocado_vt=true", "method=copr avocado_vt=true"] fail-fast: false steps: - uses: actions/checkout@v4 with: ref: 'master' fetch-depth: 0 - name: Install Ansible run: dnf install -y git ansible - name: Run Ansible playbook run: ansible-pull -v -U ${{env.GIT_URL}} -i ${{env.INVENTORY}} -c local ${{env.PLAYBOOK}} -e "${{matrix.extra}}" avocado-113.0/.github/workflows/autils_migration_announcement.yml000066400000000000000000000033541513561306000254030ustar00rootroot00000000000000name: AAutils migration announcement on: pull_request_target: types: - opened paths: - '**/ar.py' - '**/astring.py' - '**/data_structures.py' - '**/external/gdbmi_parser.py' - '**/external/spark.py' - '**/gdb.py' - '**/network/ports.py' - '**/output.py' - '**/path.py' - '**/process.py' - '**/script.py' - '**/wait.py' jobs: commnet-to-pr: name: Do an announcement to PR runs-on: ubuntu-latest steps: - name: Generate token id: generate_token uses: tibdex/github-app-token@021a2405c7f990db57f5eae5397423dcc554159c with: app_id: ${{ secrets.MR_AVOCADO_ID }} installation_id: ${{ secrets.MR_AVOCADO_INSTALLATION_ID }} private_key: ${{ secrets.MR_AVOCADO_PRIVATE_KEY }} - name: Get PR ID env: GITHUB_TOKEN: ${{ steps.generate_token.outputs.token }} run: | pr_data=$(gh api graphql -f query='query { repository(owner:"avocado-framework", name:"avocado") { pullRequest(number:${{ github.event.number }}) { id } } }') echo 'PR_ID='$(echo $pr_data | jq .data.repository.pullRequest.id) >> $GITHUB_ENV - name: comment on PR env: GITHUB_TOKEN: ${{ steps.generate_token.outputs.token }} run: | gh api graphql -f query='mutation { addComment(input: { subjectId: ${{ env.PR_ID }}, body: """${{ vars.AUTILS_MIGRATION_ANNOUNCEMENT }}"""}) { clientMutationId } }' avocado-113.0/.github/workflows/ci.yml000066400000000000000000000425741513561306000176610ustar00rootroot00000000000000name: GH Actions on: pull_request: workflow_dispatch: jobs: static-checks: name: Static checks runs-on: ubuntu-22.04 container: image: quay.io/avocado-framework/avocado-ci-fedora-40 steps: - run: echo "Job triggered by a ${{ github.event_name }} event on branch is ${{ github.ref }} in repository is ${{ github.repository }}, runner on ${{ runner.os }}" - name: Check out repository code uses: actions/checkout@v4 with: fetch-depth: 0 ref: ${{ github.event.pull_request.head.sha }} submodules: true - name: Allow git to operate on directory checked out by GH Actions run: git config --global --add safe.directory `pwd` - name: Installing Avocado development dependencies run: make requirements-static-checks - name: Installing Avocado in develop mode run: python3 setup.py develop --user - name: Run static checks env: COMMIT_COUNT: ${{ github.event.pull_request.commits }} run: python3 setup.py test --select=static-checks - name: Archive failed tests logs if: failure() uses: actions/upload-artifact@v4 with: name: static-checks-logs path: /home/runner/avocado/job-results/ retention-days: 1 - run: echo "đŸĨ‘ This job's status is ${{ job.status }}." smokecheck-linux: name: Smokecheck on Linux with Python ${{ matrix.python-version }} runs-on: ubuntu-22.04 strategy: matrix: python-version: [3.9, '3.10', 3.11, 3.12, 3.13] fail-fast: false steps: - run: echo "Job triggered by a ${{ github.event_name }} event on branch is ${{ github.ref }} in repository is ${{ github.repository }}, runner on ${{ runner.os }}" - name: Check out repository code uses: actions/checkout@v4 with: fetch-depth: 0 - name: Set up Python ${{ matrix.python-version }} uses: actions/setup-python@v5 with: python-version: ${{ matrix.python-version }} - name: Install setuptools on Python >= 3.12 run: python3 -c 'import setuptools' || python3 -m pip install "setuptools<80" - name: Avocado smokecheck run: make smokecheck check-linux: name: Linux with Python ${{ matrix.python-version }} runs-on: ubuntu-22.04 needs: smokecheck-linux strategy: matrix: python-version: [3.9, '3.10', 3.11, 3.12, 3.13] fail-fast: false steps: - run: echo "Job triggered by a ${{ github.event_name }} event on branch is ${{ github.ref }} in repository is ${{ github.repository }}, runner on ${{ runner.os }}" - name: Check out repository code uses: actions/checkout@v4 with: fetch-depth: 0 - name: Set up Python ${{ matrix.python-version }} uses: actions/setup-python@v5 with: python-version: ${{ matrix.python-version }} - name: Display Python version run: python -V --version - name: Install setuptools on Python >= 3.12 run: python3 -c 'import setuptools' || python3 -m pip install "setuptools<80" - name: Install dependencies run: pip install -r requirements-dev.txt - name: Installing Avocado in develop mode run: python3 setup.py develop --user - name: Avocado version run: avocado --version - name: Unittests and fast functional tests run: python3 setup.py test --skip=static-checks - name: Archive failed tests logs if: failure() uses: actions/upload-artifact@v4 with: name: linux-logs-${{ matrix.python-version }} path: /home/runner/avocado/job-results/ retention-days: 1 - run: echo "đŸĨ‘ This job's status is ${{ job.status }}." # macOS check on latest Python check-macos: name: macOS with Python ${{ matrix.python-version }} runs-on: macos-14 strategy: matrix: python-version: [3.11] steps: - run: echo "Job triggered by a ${{ github.event_name }} event on branch is ${{ github.ref }} in repository is ${{ github.repository }}, runner on ${{ runner.os }}" - name: Check out repository code uses: actions/checkout@v4 - name: Set up Python ${{ matrix.python-version }} uses: actions/setup-python@v5 with: python-version: ${{ matrix.python-version }} - name: Display Python version run: python -V --version - name: Install avocado run: | python3 -m pip install -r requirements-dev.txt python setup.py develop --user - name: Show avocado help run: python -m avocado --help - name: Example test run: python -m avocado run examples/tests/passtest.py - name: List test run: python -m avocado --verbose list selftests/unit/* selftests/functional/* selftests/*sh - name: Run a subset of avocado's selftests run: PATH=~/Library/Python/3.11/bin:$PATH ./selftests/check.py --skip=static-checks - run: echo "đŸĨ‘ This job's status is ${{ job.status }}." # Windows checks on latest Python smokecheck-windows: name: Windows with Python ${{ matrix.python-version }} runs-on: windows-2025 strategy: matrix: python-version: [3.9, '3.10', 3.11, 3.12, 3.13] steps: - run: echo "Job triggered by a ${{ github.event_name }} event on branch is ${{ github.ref }} in repository is ${{ github.repository }}, runner on ${{ runner.os }}" - name: Check out repository code uses: actions/checkout@v4 - name: Set up Python ${{ matrix.python-version }} uses: actions/setup-python@v5 with: python-version: ${{ matrix.python-version }} - name: Display Python version run: python -V --version - name: Install setuptools on Python >= 3.12 run: python -c 'import setuptools' || python -m pip install "setuptools<80" - name: Install avocado run: python setup.py develop --user - name: Show avocado help run: python -m avocado --help - name: Example test run: python -m avocado run examples\tests\passtest.py - run: echo "đŸĨ‘ This job's status is ${{ job.status }}." # Checks that Python packages builds succeeds package-build: name: Build Package (wheel/tarball) for Python ${{ matrix.python-version }} runs-on: ubuntu-22.04 strategy: matrix: python-version: [3.9, '3.10', 3.11, 3.12, 3.13] fail-fast: false steps: - uses: actions/checkout@v4 - name: Set up Python uses: actions/setup-python@v5 with: python-version: ${{ matrix.python-version }} - name: Install setuptools on Python >= 3.12 run: python3 -c 'import setuptools' || python3 -m pip install "setuptools<80" - name: Build tarballs and wheels run: make -f Makefile.gh build-wheel check-wheel - name: Save tarballs and wheels as artifacts uses: actions/upload-artifact@v4 with: name: tarballs_and_wheels-${{ matrix.python-version }} path: ${{github.workspace}}/PYPI_UPLOAD/ retention-days: 1 - run: echo "đŸĨ‘ This job's status is ${{ job.status }}." egg-build: name: Build Egg for Python ${{ matrix.python-version }} runs-on: ubuntu-22.04 strategy: matrix: python-version: [3.9, '3.10', 3.11, 3.12, 3.13] fail-fast: false steps: - uses: actions/checkout@v4 - name: Set up Python uses: actions/setup-python@v5 with: python-version: ${{ matrix.python-version }} - name: Install setuptools on Python >= 3.12 run: python3 -c 'import setuptools' || python3 -m pip install "setuptools<80" - name: Build eggs run: make -f Makefile.gh build-egg - name: Save eggs as artifacts uses: actions/upload-artifact@v4 with: name: eggs-${{ matrix.python-version }} path: ${{github.workspace}}/EGG_UPLOAD/ retention-days: 1 - run: echo "đŸĨ‘ This job's status is ${{ job.status }}." experimental-checks: name: Experimental checks runs-on: ubuntu-22.04 steps: - run: echo "Job triggered by a ${{ github.event_name }} event on branch is ${{ github.ref }} in repository is ${{ github.repository }}, runner on ${{ runner.os }}" - name: Check out repository code uses: actions/checkout@v4 with: fetch-depth: 0 - name: Run Codespell Check run: make -f Makefile.gh codespell - name: Run bandit check run: make -f Makefile.gh bandit continue-on-error: True - name: Save bandit output as artifacts uses: actions/upload-artifact@v4 with: name: bandit-results path: /home/runner/work/avocado/avocado/bandit-output.txt retention-days: 1 - run: echo "đŸĨ‘ This job's status is ${{ job.status }}." version_task_fedora_40: name: Version task fedora:40 runs-on: ubuntu-22.04 container: image: fedora:40 steps: - name: Install Python dependencies run: dnf -y install python3 python3-setuptools - name: Check out repository code uses: actions/checkout@v4 - uses: ./.github/actions/version version_task_fedora_41: name: Version task fedora:41 runs-on: ubuntu-22.04 container: image: fedora:41 steps: - name: Install Python dependencies run: dnf -y install python3-setuptools - name: Check out repository code uses: actions/checkout@v4 - uses: ./.github/actions/version version_task_ubi_8: name: Version task ubi:8.8 runs-on: ubuntu-22.04 container: image: registry.access.redhat.com/ubi8/ubi:8.8 steps: - name: Install Python dependencies run: dnf -y install python3.11 python3.11-setuptools python3.11-setuptools-rust - name: Check out repository code uses: actions/checkout@v4 - uses: ./.github/actions/version version_task_ubi_9: name: Version task ubi:9.2 runs-on: ubuntu-22.04 container: image: registry.access.redhat.com/ubi9/ubi:9.2 steps: - name: Check out repository code uses: actions/checkout@v4 - uses: ./.github/actions/version version_task_debian_12: name: Version task debian:12.4 runs-on: ubuntu-22.04 container: image: debian:12.4 steps: - name: Install Python dependencies run: apt update && apt -y install python3 python3-setuptools - name: Check out repository code uses: actions/checkout@v4 - uses: ./.github/actions/version version_task_debian_11: name: Version task debian:11.0 runs-on: ubuntu-22.04 container: image: debian:11.0 steps: - name: Install Python dependencies run: apt update && apt -y install python3 python3-setuptools - name: Check out repository code uses: actions/checkout@v4 - uses: ./.github/actions/version version_task_ubuntu_24: name: Version task ubuntu:24.04 runs-on: ubuntu-22.04 container: image: ubuntu:24.04 steps: - name: Install Python dependencies run: apt update && apt -y install python3 python3-setuptools ca-certificates - name: Check out repository code uses: actions/checkout@v4 - uses: ./.github/actions/version version_task_ubuntu_22: name: Version task ubuntu:22.04 runs-on: ubuntu-22.04 container: image: ubuntu:22.04 steps: - name: Install Python dependencies run: apt update && apt -y install python3 python3-setuptools ca-certificates - name: Check out repository code uses: actions/checkout@v4 - uses: ./.github/actions/version egg_task_fedora_40: name: Egg task fedora:40 runs-on: ubuntu-22.04 container: image: fedora:40 steps: - name: Install Python dependencies run: dnf -y install python3 python3-setuptools - name: Check out repository code uses: actions/checkout@v4 - uses: ./.github/actions/egg egg_task_fedora_41: name: Egg task fedora:41 runs-on: ubuntu-22.04 container: image: fedora:41 steps: - name: Install Python dependencies run: dnf -y install python3-setuptools - name: Check out repository code uses: actions/checkout@v4 - uses: ./.github/actions/egg egg_task_ubi_8: name: Egg task ubi:8.8 runs-on: ubuntu-22.04 container: image: registry.access.redhat.com/ubi8/ubi:8.8 steps: - name: Install Python dependencies run: dnf -y install python39 python39-setuptools - name: Check out repository code uses: actions/checkout@v4 - uses: ./.github/actions/egg egg_task_ubi_9: name: Egg task ubi:9.2 runs-on: ubuntu-22.04 container: image: registry.access.redhat.com/ubi9/ubi:9.2 steps: - name: Check out repository code uses: actions/checkout@v4 - uses: ./.github/actions/egg egg_task_debian_12: name: Egg task debian:12.4 runs-on: ubuntu-22.04 container: image: debian:12.4 steps: - name: Install Python dependencies run: apt update && apt -y install python3 python3-setuptools - name: Check out repository code uses: actions/checkout@v4 - uses: ./.github/actions/egg egg_task_debian_11: name: Egg task debian:11.0 runs-on: ubuntu-22.04 container: image: debian:11.0 steps: - name: Install Python dependencies run: apt update && apt -y install python3 python3-setuptools - name: Check out repository code uses: actions/checkout@v4 - uses: ./.github/actions/egg egg_task_ubuntu_24: name: Egg task ubuntu:24.04 runs-on: ubuntu-22.04 container: image: ubuntu:24.04 steps: - name: Install Python dependencies run: apt update && apt -y install python3 python3-setuptools - name: Check out repository code uses: actions/checkout@v4 - uses: ./.github/actions/egg egg_task_ubuntu_22: name: Egg task ubuntu:22.04 runs-on: ubuntu-22.04 container: image: ubuntu:22.04 steps: - name: Install Python dependencies run: apt update && apt -y install python3 python3-setuptools - name: Check out repository code uses: actions/checkout@v4 - uses: ./.github/actions/egg podman_egg_task: name: Podman Egg task runs-on: ubuntu-22.04 steps: - name: Check out repository code uses: actions/checkout@v4 - name: Test running avocado from eggs under Podman spawner run: | apt update && apt -y install python3 python3-setuptools python3 setup.py bdist_egg mv dist/avocado_framework-*egg /tmp/avocado_framework.egg python3 setup.py clean --all python3 -c 'import sys; sys.path.insert(0, "/tmp/avocado_framework.egg"); from avocado.core.main import main; sys.exit(main())' run --spawner=podman --spawner-podman-image=fedora:40 --spawner-podman-avocado-egg=file:///tmp/avocado_framework.egg -- /bin/true podman_external_runner_task: name: Podman spawner with 3rd party runner plugin runs-on: ubuntu-22.04 steps: - name: Check out repository code uses: actions/checkout@v4 - name: Test running avocado from released eggs under Podman spawner with 3rd party plugins run: | apt update && apt -y install python3 python3-setuptools python3 setup.py develop --user cd examples/plugins/tests/magic python3 setup.py develop --user cd ../../../../ python3 -m avocado -V list -- magic:pass magic:fail | grep "magic: 2" podman pull quay.io/avocado-framework/avocado-ci-magic python3 -m avocado run --spawner=podman --spawner-podman-image=quay.io/avocado-framework/avocado-ci-magic -- magic:pass tail -n1 ~/avocado/job-results/latest/results.tap | grep "ok 1 magic:pass" python3 -m avocado run --spawner=podman --spawner-podman-image=quay.io/avocado-framework/avocado-ci-magic -- magic:fail || true tail -n1 ~/avocado/job-results/latest/results.tap | grep "not ok 1 magic:fail" fedora_develop_install_uninstall_task: name: Fedora develop install/uninstall task runs-on: ubuntu-latest container: image: fedora:40 steps: - name: Check out repository code uses: actions/checkout@v4 - name: Test avocado install/uninstall run: | python3 -c 'import setuptools' || dnf -y install python3 python3-setuptools python3 setup.py develop --user test `python3 -m avocado plugins | grep ^html | wc -l` -eq "3" test `python3 -m avocado plugins | grep ^robot | wc -l` -eq "2" python3 setup.py develop --user --uninstall python3 setup.py clean --all python3 setup.py develop --user --skip-optional-plugins python3 -m avocado --version test `python3 -m avocado plugins | grep ^html | wc -l` -eq "0" test `python3 -m avocado plugins | grep ^robot | wc -l` -eq "0" fedora_selftests_task: name: Fedora selftests runs-on: ubuntu-latest timeout-minutes: 18 # max + 3*std over the last 400 successful runs container: image: quay.io/avocado-framework/avocado-ci-fedora-40 steps: - name: Check out repository code uses: actions/checkout@v4 - name: Run unittests and functional tests run: | make develop PATH=$HOME/.local/bin:$PATH LANG=en_US.UTF-8 AVOCADO_CHECK_LEVEL=0 python3 selftests/check.py --skip static-checks avocado-113.0/.github/workflows/pr_announcement.yml000066400000000000000000000026221513561306000224470ustar00rootroot00000000000000name: PR announcement on: pull_request_target: types: - opened jobs: commnet-to-pr: name: Do an announcement to PR runs-on: ubuntu-latest steps: - name: Generate token id: generate_token uses: tibdex/github-app-token@021a2405c7f990db57f5eae5397423dcc554159c with: app_id: ${{ secrets.MR_AVOCADO_ID }} installation_id: ${{ secrets.MR_AVOCADO_INSTALLATION_ID }} private_key: ${{ secrets.MR_AVOCADO_PRIVATE_KEY }} - name: Get PR ID env: GITHUB_TOKEN: ${{ steps.generate_token.outputs.token }} run: | pr_data=$(gh api graphql -f query='query { repository(owner:"avocado-framework", name:"avocado") { pullRequest(number:${{ github.event.number }}) { id } } }') echo 'PR_ID='$(echo $pr_data | jq .data.repository.pullRequest.id) >> $GITHUB_ENV - name: comment on PR env: GITHUB_TOKEN: ${{ steps.generate_token.outputs.token }} run: | gh api graphql -f query='mutation { addComment(input: { subjectId: ${{ env.PR_ID }}, body: """${{ vars.PR_ANNOUNCEMENT }}"""}) { clientMutationId } }' avocado-113.0/.github/workflows/prerelease.yml000066400000000000000000000074551513561306000214140ustar00rootroot00000000000000name: Pre-release tests on: # Runs at 5:00 UTC on Mondays schedule: - cron: "0 5 * * 1" workflow_dispatch: jobs: make-check: name: Run 'make check' runs-on: ubuntu-22.04 steps: - uses: actions/checkout@v4 with: fetch-depth: 0 submodules: true - name: Set up Python 3.9 uses: actions/setup-python@v5 with: python-version: 3.9 - name: Install dependencies run: | pip install -r requirements-dev.txt pip install -r static-checks/requirements.txt - name: Installing Avocado in develop mode run: python3 setup.py develop --user - name: Run make check run: make check - name: Save artifacts if: failure() uses: actions/upload-artifact@v4 with: name: job-results-make-check path: /home/runner/avocado/job-results/ retention-days: 1 avocado-pre-release: name: Avocado pre-release job runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 with: fetch-depth: 0 - name: Install avocado run: | pip install -r requirements-dev.txt python3 setup.py develop --user - name: Avocado pre-release job run: python3 setup.py test --select=pre-release,vmimage - name: Save artifacts if: failure() uses: actions/upload-artifact@v4 with: name: job-results-pre-release path: /home/runner/avocado/job-results/ retention-days: 1 check-rpm-available-copr: name: Check the right RPM packages are available in COPR runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 with: ref: 'master' - name: Check the right RPM packages are available run: ./selftests/pre_release/tests/check-copr-rpm-version.sh avocado-deployment-copr: name: Avocado deployment runs-on: ubuntu-latest container: image: fedora:42 env: GIT_URL: 'https://github.com/avocado-framework/avocado' INVENTORY: 'selftests/deployment/inventory' PLAYBOOK: 'selftests/deployment/deployment.yml' steps: - uses: actions/checkout@v4 with: fetch-depth: 0 - name: Install Ansible run: dnf install -y git ansible - name: Run Ansible playbook run: ansible-pull -v -U ${{env.GIT_URL}} -i ${{env.INVENTORY}} -c local ${{env.PLAYBOOK}} -e "method=copr" - name: Run test run: avocado run /usr/bin/true - name: Save artifacts if: failure() uses: actions/upload-artifact@v4 with: name: job-results-deployment path: /github/home/avocado/job-results/ retention-days: 1 package-build: name: Build Package (wheel/tarball) runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - name: Build tarballs and wheels run: make -f Makefile.gh build-wheel check-wheel - name: Save tarballs and wheels as artifacts uses: actions/upload-artifact@v4 with: name: tarballs_and_wheels path: ${{github.workspace}}/PYPI_UPLOAD/ retention-days: 1 - run: echo "đŸĨ‘ This job's status is ${{ job.status }}." publish-to-test-pypi: name: Publish Avocado to TestPyPI needs: - package-build runs-on: ubuntu-latest environment: name: testpypi url: https://test.pypi.org/p/avocado-framework permissions: id-token: write # IMPORTANT: mandatory for trusted publishing steps: - name: Download all the wheels uses: actions/download-artifact@v4 with: name: tarballs_and_wheels path: dist/ - name: Publish avocado to test PyPI uses: pypa/gh-action-pypi-publish@release/v1 with: repository-url: https://test.pypi.org/legacy/ avocado-113.0/.github/workflows/project.yml000066400000000000000000000074261513561306000207310ustar00rootroot00000000000000name: Add new issue/PR to project on: issues: types: - opened - reopened pull_request_target: types: - opened jobs: add-to-project: name: Add issue or PR to project runs-on: ubuntu-latest steps: - name: Generate token id: generate_token uses: tibdex/github-app-token@021a2405c7f990db57f5eae5397423dcc554159c with: app_id: ${{ secrets.MR_AVOCADO_ID }} installation_id: ${{ secrets.MR_AVOCADO_INSTALLATION_ID }} private_key: ${{ secrets.MR_AVOCADO_PRIVATE_KEY }} - name: Get project data env: GITHUB_TOKEN: ${{ steps.generate_token.outputs.token }} ORGANIZATION: avocado-framework PROJECT_NUMBER: 1 run: | gh api graphql -f query=' query($org: String!, $number: Int!) { organization(login: $org){ projectV2(number: $number) { id fields(first:20) { nodes { ... on ProjectV2Field { id name } ... on ProjectV2SingleSelectField { id name options { id name } } } } } } }' -f org=$ORGANIZATION -F number=$PROJECT_NUMBER > project_data.json echo 'PROJECT_ID='$(jq '.data.organization.projectV2.id' project_data.json) >> $GITHUB_ENV echo 'STATUS_FIELD_ID='$(jq '.data.organization.projectV2.fields.nodes[] | select(.name== "Status") | .id' project_data.json) >> $GITHUB_ENV if [[ ${{ github.event_name }} == pull_request_target ]]; then echo 'EVENT_ID='${{ github.event.pull_request.node_id }} >> $GITHUB_ENV echo 'FIELD_VALUE='$(jq '.data.organization.projectV2.fields.nodes[] | select(.name== "Status") | .options[] | select(.name=="Review Requested") |.id' project_data.json) >> $GITHUB_ENV else echo 'EVENT_ID='${{ github.event.issue.node_id }} >> $GITHUB_ENV echo 'FIELD_VALUE='$(jq '.data.organization.projectV2.fields.nodes[] | select(.name== "Status") | .options[] | select(.name=="Triage") |.id' project_data.json) >> $GITHUB_ENV fi - name: Add Item to project env: GITHUB_TOKEN: ${{ steps.generate_token.outputs.token }} run: | item_id="$( gh api graphql -f query=' mutation($project:ID!, $item:ID!) { addProjectV2ItemById(input: {projectId: $project, contentId: $item}) { item { id } } }' -f project=$PROJECT_ID -f item=$EVENT_ID --jq '.data.addProjectV2ItemById.item.id')" echo 'ITEM_ID='$item_id >> $GITHUB_ENV - name: Set fields env: GITHUB_TOKEN: ${{ steps.generate_token.outputs.token }} run: | gh api graphql -f query=' mutation ( $project: ID! $item: ID! $status_field: ID! $status_value: String! ) { set_status: updateProjectV2ItemFieldValue(input: { projectId: $project itemId: $item fieldId: $status_field value: { singleSelectOptionId: $status_value } }) { projectV2Item { id } } }' -f project=$PROJECT_ID -f item=$ITEM_ID -f status_field=$STATUS_FIELD_ID -f status_value=${{ env.FIELD_VALUE }} --silent avocado-113.0/.github/workflows/push_ci.yml000066400000000000000000000101521513561306000207030ustar00rootroot00000000000000name: "PR and Push checks" on: push: branches: [ "master", "103lts" ] pull_request: branches: [ "master", "103lts" ] jobs: analyze: name: Analyze # Runner size impacts CodeQL analysis time. To learn more, please see: # - https://gh.io/recommended-hardware-resources-for-running-codeql # - https://gh.io/supported-runners-and-hardware-resources # - https://gh.io/using-larger-runners # Consider using larger runners for possible analysis time improvements. runs-on: ${{ (matrix.language == 'swift' && 'macos-latest') || 'ubuntu-latest' }} timeout-minutes: ${{ (matrix.language == 'swift' && 120) || 360 }} permissions: # required for all workflows security-events: write # only required for workflows in private repositories actions: read contents: read strategy: fail-fast: false matrix: language: [ 'python' ] # CodeQL supports [ 'c-cpp', 'csharp', 'go', 'java-kotlin', 'javascript-typescript', 'python', 'ruby', 'swift' ] # Use only 'java-kotlin' to analyze code written in Java, Kotlin or both # Use only 'javascript-typescript' to analyze code written in JavaScript, TypeScript or both # Learn more about CodeQL language support at https://aka.ms/codeql-docs/language-support steps: - name: Checkout repository uses: actions/checkout@v4 # Initializes the CodeQL tools for scanning. - name: Initialize CodeQL uses: github/codeql-action/init@v3 with: languages: ${{ matrix.language }} # If you wish to specify custom queries, you can do so here or in a config file. # By default, queries listed here will override any specified in a config file. # Prefix the list here with "+" to use these queries and those in the config file. # For more details on CodeQL's query packs, refer to: https://docs.github.com/en/code-security/code-scanning/automatically-scanning-your-code-for-vulnerabilities-and-errors/configuring-code-scanning#using-queries-in-ql-packs # queries: security-extended,security-and-quality # Autobuild attempts to build any compiled languages (C/C++, C#, Go, Java, or Swift). # If this step fails, then you should remove it and run the build manually (see below) - name: Autobuild uses: github/codeql-action/autobuild@v3 # ℹī¸ Command-line programs to run using the OS shell. # 📚 See https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstepsrun # If the Autobuild fails above, remove it and uncomment the following three lines. # modify them (or add more) to build your code if your project, please refer to the EXAMPLE below for guidance. # - run: | # echo "Run, Build Application using script" # ./location_of_script_within_repo/buildscript.sh - name: Perform CodeQL Analysis uses: github/codeql-action/analyze@v3 with: category: "/language:${{matrix.language}}" code-coverage: name: Code Coverage runs-on: ubuntu-22.04 strategy: matrix: python-version: [3.11] steps: - run: echo "Job triggered by a ${{ github.event_name }} event on branch is ${{ github.ref }} in repository is ${{ github.repository }}, runner on ${{ runner.os }}" - name: Check out repository code uses: actions/checkout@v4 - name: Set up Python ${{ matrix.python-version }} uses: actions/setup-python@v5 with: python-version: ${{ matrix.python-version }} - name: Install system dependencies run: | sudo apt-get update sudo apt-get install -y gcc gdb gdbserver - name: Install run: pip install -r requirements-dev.txt - name: Avocado build run: python setup.py develop --user - name: Run script run: ./selftests/run_coverage - name: Push results to codecov if: always() uses: codecov/codecov-action@v5 with: files: ./coverage.xml token: ${{ secrets.CODECOV_TOKEN }} - run: echo "đŸĨ‘ This job's status is ${{ job.status }}." avocado-113.0/.github/workflows/release.yml000066400000000000000000000116531513561306000207000ustar00rootroot00000000000000# must do before using this action: # - set up RELEASE_TOKEN in secrets of the repository avocado-framework/avocado # - set up RTD_TOKEN in secrets to update readthedocs # - set up two tokens for twine: PYPI_USER and PYPI_PASSWD name: Release tests on: workflow_dispatch: inputs: version: description: 'Release version' required: true default: '0.0' devel_name: description: 'Developer Name' required: true default: 'Avocado Developer' devel_mail: description: 'Developer mail' required: true default: 'avocado@redhat.com' rtd_project: description: 'readthedocs project name' required: true default: 'avocado-framework' jobs: release: name: Release pipeline runs-on: ubuntu-latest container: image: fedora:40 env: VERSION: ${{ github.event.inputs.version }} DEVEL_NAME: ${{ github.event.inputs.devel_name }} DEVEL_MAIL: ${{ github.event.inputs.devel_mail }} PYTHON: /usr/bin/python3 TOKEN_RTD: ${{ secrets.RTD_TOKEN }} URL: "https://readthedocs.org/api/v3/projects/${{ github.event.inputs.rtd_project }}" steps: - name: Generate token id: generate_token uses: tibdex/github-app-token@021a2405c7f990db57f5eae5397423dcc554159c with: app_id: ${{ secrets.MR_AVOCADO_ID }} installation_id: ${{ secrets.MR_AVOCADO_INSTALLATION_ID }} private_key: ${{ secrets.MR_AVOCADO_PRIVATE_KEY }} - name: install required packages run: dnf -y install rpmdevtools git python3-pip make - uses: actions/checkout@v4 with: persist-credentials: false fetch-depth: 0 - name: Update VERSION files and python-avocado.spec run: | make -f Makefile.gh propagate-version make -f Makefile.gh release-update-spec - name: Commit files and tag run: | git config --global --add safe.directory `pwd` git config --local user.email "${{ github.event.inputs.devel_mail }}" git config --local user.name "${{ github.event.inputs.devel_name }}" make -f Makefile.gh release-commit-tag - name: Push changes to github uses: ad-m/github-push-action@master with: github_token: ${{ steps.generate_token.outputs.token }} branch: ${{ github.ref }} - name: Build wheel run: make -f Makefile.gh build-wheel check-wheel - name: Save wheel as artifact uses: actions/upload-artifact@v4 with: name: wheel path: ${{github.workspace}}/PYPI_UPLOAD/ retention-days: 3 - name: Build documentation readthedocs run: | make -f Makefile.gh build-update-readthedocs - run: >- echo In a few minutes the release documentation will be available at https://${{ github.event.inputs.rtd_project }}.readthedocs.io/en/${{ github.event.inputs.version }}/ | tee -a "${GITHUB_STEP_SUMMARY}" publish-to-pypi: name: Publish Avocado to PyPI needs: - release runs-on: ubuntu-latest environment: name: pypi url: https://pypi.org/project/avocado-framework/${{ github.event.inputs.version }} permissions: id-token: write # IMPORTANT: mandatory for trusted publishing steps: - name: Download all the wheels uses: actions/download-artifact@v4 with: name: wheel path: dist/ - name: Publish avocado to PyPI uses: pypa/gh-action-pypi-publish@release/v1 build-and-publish-eggs: name: Build eggs and publish them runs-on: ubuntu-22.04 needs: release strategy: matrix: python-version: [3.9, '3.10', 3.11, 3.12, 3.13] fail-fast: false steps: - name: Generate token id: generate_token uses: tibdex/github-app-token@021a2405c7f990db57f5eae5397423dcc554159c with: app_id: ${{ secrets.MR_AVOCADO_ID }} installation_id: ${{ secrets.MR_AVOCADO_INSTALLATION_ID }} private_key: ${{ secrets.MR_AVOCADO_PRIVATE_KEY }} - uses: actions/checkout@v4 with: persist-credentials: false fetch-depth: 0 ref: ${{ github.event.inputs.version }} - name: Set up Python uses: actions/setup-python@v5 with: python-version: ${{ matrix.python-version }} - name: Build eggs run: | if python -c 'import sys; exit(0) if sys.version_info.minor > 11 else exit(1)' ; then pip install "setuptools<80" fi make -f Makefile.gh build-egg - name: Upload binaries to release uses: svenstaro/upload-release-action@v2 with: repo_token: ${{ steps.generate_token.outputs.token }} file: ${{ github.workspace }}/EGG_UPLOAD/avocado_framework*egg tag: ${{ github.event.inputs.version }} overwrite: true file_glob: true avocado-113.0/.github/workflows/setup.yml000066400000000000000000000174511513561306000204220ustar00rootroot00000000000000name: Setup tests on: # Runs at 5:00 UTC on Mondays schedule: - cron: "0 5 * * 1" # Allows you to run this workflow manually from the Actions tab workflow_dispatch: jobs: user-installation: name: User installation commands runs-on: ubuntu-22.04 strategy: matrix: python-version: [3.x] fail-fast: false steps: - run: echo "Job triggered by a ${{ github.event_name }} event on branch is ${{ github.ref }} in repository is ${{ github.repository }}, runner on ${{ runner.os }}" - name: Check out repository code uses: actions/checkout@v4 with: fetch-depth: 0 - name: Set up Python ${{ matrix.python-version }} uses: actions/setup-python@v5 with: python-version: ${{ matrix.python-version }} - name: Installing Avocado run: python3 setup.py install --user - name: Installing plugins run: | python3 setup.py plugin --install=golang --user python3 setup.py plugin --install=html --user python3 setup.py plugin --install=result_upload --user python3 setup.py plugin --install=resultsdb --user python3 setup.py plugin --install=robot --user python3 setup.py plugin --install=varianter_cit --user python3 setup.py plugin --install=varianter_pict --user python3 setup.py plugin --install=varianter_yaml_to_mux --user - name: Avocado version run: avocado --version - name: Avocado smoketest run: avocado run examples/tests/passtest.py - name: Avocado two tests run: avocado run /usr/bin/true examples/tests/passtest.py - run: echo "đŸĨ‘ This job's status is ${{ job.status }}." system-wide-installation: name: System wide installation commands runs-on: ubuntu-22.04 strategy: matrix: python-version: [3.x] fail-fast: false steps: - run: echo "Job triggered by a ${{ github.event_name }} event on branch is ${{ github.ref }} in repository is ${{ github.repository }}, runner on ${{ runner.os }}" - name: Check out repository code uses: actions/checkout@v4 with: fetch-depth: 0 - name: Set up Python ${{ matrix.python-version }} uses: actions/setup-python@v5 with: python-version: ${{ matrix.python-version }} - name: Installing Avocado run: python3 setup.py install - name: Installing plugins run: | python3 setup.py plugin --install=golang python3 setup.py plugin --install=html python3 setup.py plugin --install=result_upload python3 setup.py plugin --install=resultsdb python3 setup.py plugin --install=robot python3 setup.py plugin --install=varianter_cit python3 setup.py plugin --install=varianter_pict python3 setup.py plugin --install=varianter_yaml_to_mux - name: Avocado version run: avocado --version - name: Avocado smoketest run: avocado run examples/tests/passtest.py - name: Avocado two tests run: avocado run /usr/bin/true examples/tests/passtest.py - run: echo "đŸĨ‘ This job's status is ${{ job.status }}." devel-user-installation: name: Developer user commands runs-on: ubuntu-22.04 strategy: matrix: python-version: [3.x] fail-fast: false steps: - run: echo "Job triggered by a ${{ github.event_name }} event on branch is ${{ github.ref }} in repository is ${{ github.repository }}, runner on ${{ runner.os }}" - name: Check out repository code uses: actions/checkout@v4 with: fetch-depth: 0 - name: Set up Python ${{ matrix.python-version }} uses: actions/setup-python@v5 with: python-version: ${{ matrix.python-version }} - name: Install dependencies run: pip3 install -r requirements-dev.txt - name: Installing Avocado in develop mode run: python3 setup.py develop --user - name: Avocado version run: avocado --version - name: Avocado smoketest run: avocado run examples/tests/passtest.py - name: Avocado build manpage run: python3 setup.py man - name: Tree static check, unittests and fast functional tests run: python3 setup.py test - name: Uninstall avocado run: python3 setup.py develop --user --uninstall - run: echo "đŸĨ‘ This job's status is ${{ job.status }}." devel-wide-system-installation: name: Developer system wide commands runs-on: ubuntu-22.04 strategy: matrix: python-version: [3.x] fail-fast: false steps: - run: echo "Job triggered by a ${{ github.event_name }} event on branch is ${{ github.ref }} in repository is ${{ github.repository }}, runner on ${{ runner.os }}" - name: Check out repository code uses: actions/checkout@v4 with: fetch-depth: 0 - name: Set up Python ${{ matrix.python-version }} uses: actions/setup-python@v5 with: python-version: ${{ matrix.python-version }} - name: Install dependencies run: pip3 install -r requirements-dev.txt - name: Installing Avocado in develop mode run: python3 setup.py develop - name: Avocado version run: avocado --version - name: Avocado smoketest run: avocado run examples/tests/passtest.py - name: Avocado build manpage run: python3 setup.py man - name: Tree static check, unittests and fast functional tests run: python3 setup.py test - name: Uninstall avocado run: python3 setup.py develop --uninstall - run: echo "đŸĨ‘ This job's status is ${{ job.status }}." virtualenv-installation: name: Virtualenv installation runs-on: ubuntu-22.04 steps: - run: echo "Job triggered by a ${{ github.event_name }} event on branch is ${{ github.ref }} in repository is ${{ github.repository }}, runner on ${{ runner.os }}" - name: Check out repository code uses: actions/checkout@v4 with: fetch-depth: 0 - name: Install packages run: | sudo apt-get update sudo apt-get install -y python3-virtualenv - name: virtualenv run: | python3 -m venv env source env/bin/activate python3 setup.py install python3 setup.py plugin --install=golang python3 setup.py plugin --install=html python3 setup.py plugin --install=result_upload python3 setup.py plugin --install=resultsdb python3 setup.py plugin --install=robot python3 setup.py plugin --install=varianter_cit python3 setup.py plugin --install=varianter_pict python3 setup.py plugin --install=varianter_yaml_to_mux avocado --version avocado run examples/tests/passtest.py deactivate - run: echo "đŸĨ‘ This job's status is ${{ job.status }}." devel-virtualenv-installation: name: Developer Virtualenv installation runs-on: ubuntu-22.04 steps: - run: echo "Job triggered by a ${{ github.event_name }} event on branch is ${{ github.ref }} in repository is ${{ github.repository }}, runner on ${{ runner.os }}" - name: Check out repository code uses: actions/checkout@v4 with: fetch-depth: 0 - name: Install packages run: | sudo apt-get update sudo apt-get install -y python3-virtualenv - name: virtualenv run: | python3 -m venv env source env/bin/activate pip install -r requirements-dev.txt python3 setup.py develop avocado --version avocado run examples/tests/passtest.py python3 setup.py test --skip static-checks deactivate - run: echo "đŸĨ‘ This job's status is ${{ job.status }}." avocado-113.0/.github/workflows/vmimage.yml000066400000000000000000000030061513561306000206760ustar00rootroot00000000000000name: "VMImage CI" on: push: branches: [ "master", "103lts" ] paths: - 'avocado/utils/vmimage.py' - 'selftests/vmimage/**' - 'avocado/plugins/runners/vmimage.py' pull_request: branches: [ "master", "103lts" ] paths: - 'avocado/utils/vmimage.py' - 'selftests/vmimage/**' - 'avocado/plugins/runners/vmimage.py' jobs: vmimage-test-coverage: name: Code Coverage runs-on: ubuntu-22.04 strategy: matrix: python-version: [3.11] steps: - run: echo "Job triggered by a ${{ github.event_name }} event on branch is ${{ github.ref }} in repository is ${{ github.repository }}, runner on ${{ runner.os }}" - name: Check out repository code uses: actions/checkout@v4 - name: Set up Python ${{ matrix.python-version }} uses: actions/setup-python@v5 with: python-version: ${{ matrix.python-version }} - name: Install system dependencies run: | sudo apt-get update sudo apt-get install -y qemu-utils - name: Install run: pip install -r requirements-dev.txt - name: Avocado build run: python setup.py develop --user - name: Run script run: ./selftests/run_coverage --select=vmimage - name: Push results to codecov if: always() uses: codecov/codecov-action@v5 with: files: ./coverage.xml token: ${{ secrets.CODECOV_TOKEN }} - run: echo "đŸĨ‘ This job's status is ${{ job.status }}." avocado-113.0/.github/workflows/weekly.yml000066400000000000000000000071071513561306000205570ustar00rootroot00000000000000name: Weekly tests on: # Runs at 5:00 UTC on Mondays schedule: - cron: "0 5 * * 1" # Allows you to run this workflow manually from the Actions tab workflow_dispatch: jobs: latest-python: name: Linux with Python ${{ matrix.python-version }} runs-on: ubuntu-22.04 strategy: matrix: # see list of available Python versions at https://github.com/actions/python-versions/blob/main/versions-manifest.json python-version: [3.11] fail-fast: false outputs: job_status: ${{ job.status }} steps: - run: echo "Job triggered by a ${{ github.event_name }} event on branch is ${{ github.ref }} in repository is ${{ github.repository }}, runner on ${{ runner.os }}" - name: Check out repository code uses: actions/checkout@v4 with: fetch-depth: 0 - name: Set up Python ${{ matrix.python-version }} uses: actions/setup-python@v5 with: python-version: ${{ matrix.python-version }} - name: Display Python version run: python -V --version - name: Install dependencies run: pip install -r requirements-dev.txt - name: Installing Avocado in develop mode run: python3 setup.py develop --user - name: Avocado version run: avocado --version - name: Avocado smoketest run: python -m avocado run examples/tests/passtest.py - name: Tree static check, unittests and fast functional tests run: | export AVOCADO_CHECK_LEVEL="1" python3 selftests/check.py - name: Archive test logs if: failure() uses: actions/upload-artifact@v4 with: name: job-results-plugins path: /home/runner/avocado/job-results/ - run: echo "đŸĨ‘ This job's status is ${{ job.status }}." without-plugins-latest-python: name: Test with Python without plugins ${{ matrix.python-version }} runs-on: ubuntu-22.04 needs: latest-python if: "always()&&(needs.latest-python.outputs.job_status=='failure')" strategy: matrix: # see list of available Python versions at https://github.com/actions/python-versions/blob/main/versions-manifest.json python-version: [3.11] fail-fast: false steps: - run: echo "Job triggered by a ${{ github.event_name }} event on branch is ${{ github.ref }} in repository is ${{ github.repository }}, runner on ${{ runner.os }}" - name: Check out repository code uses: actions/checkout@v4 with: fetch-depth: 0 - name: Set up Python ${{ matrix.python-version }} uses: actions/setup-python@v5 with: python-version: ${{ matrix.python-version }} - name: Display Python version run: python -V --version - name: Install dependencies run: pip install -r requirements-dev.txt - name: Installing Avocado in develop mode run: python3 setup.py develop --user --skip-optional-plugins --skip-examples-plugins-tests - name: Avocado version run: avocado --version - name: Tree static check, unittests and fast functional tests without plugins run: | export AVOCADO_CHECK_LEVEL="1" python3 selftests/check.py --disable-plugin-checks golang,html,resultsdb,result_upload,robot,varianter_cit,varianter_pict,varianter_yaml_to_mux - name: Archive test logs if: failure() uses: actions/upload-artifact@v4 with: name: job-results-without-plugins path: /home/runner/avocado/job-results/ - run: echo "đŸĨ‘ This job's status is ${{ job.status }}." avocado-113.0/.gitignore000066400000000000000000000007031513561306000151220ustar00rootroot00000000000000.project .pydevproject .settings .coverage .idea/* coverage.xml xunit.xml tmp/* logs/* *.pyc *~ tags cscope.* build sysinfo-* MANIFEST dist *.egg-info* docs/source/api/core docs/source/api/test docs/source/api/utils docs/source/api/plugins docs/source/api/optional-plugins docs/source/guides/writer/libs/data/vmimage # This file should be generated with: # avocado config reference > docs/source/config/reference.rst docs/source/config/reference.rst avocado-113.0/.gitmodules000066400000000000000000000001641513561306000153100ustar00rootroot00000000000000[submodule "static-checks"] path = static-checks url = https://github.com/avocado-framework/avocado-static-checks avocado-113.0/.packit.yml000066400000000000000000000005161513561306000152100ustar00rootroot00000000000000specfile_path: python-avocado.spec downstream_package_name: python-avocado srpm_build_deps: [] jobs: - job: copr_build trigger: pull_request metadata: targets: - fedora-all - centos-stream-9 - epel-9 - fedora-41-aarch64 - fedora-41-ppc64le - fedora-41-s390x enable_net: False avocado-113.0/.pylintrc000066400000000000000000000370761513561306000150140ustar00rootroot00000000000000[MASTER] # A comma-separated list of package or module names from where C extensions may # be loaded. Extensions are loading into the active Python interpreter and may # run arbitrary code. extension-pkg-whitelist=netifaces # Add files or directories to the blacklist. They should be base names, not # paths. ignore=CVS,gdbmi_parser.py,spark.py # Add files or directories matching the regex patterns to the blacklist. The # regex matches against base names, not paths. ignore-patterns=.git # Python code to execute, usually for sys.path manipulation such as # pygtk.require(). #init-hook= # Use multiple processes to speed up Pylint. Specifying 0 will auto-detect the # number of processors available to use. jobs=0 # Control the amount of potential inferred values when inferring a single # object. This can help the performance when dealing with large functions or # complex, nested conditions. limit-inference-results=100 # List of plugins (as comma separated values of python modules names) to load, # usually to register additional checkers. load-plugins=pylint.extensions.no_self_use # Pickle collected data for later comparisons. persistent=yes # Specify a configuration file. #rcfile= # When enabled, pylint would attempt to guess common misconfiguration and emit # user-friendly hints instead of false-positive error messages. suggestion-mode=yes # Allow loading of arbitrary C extensions. Extensions are imported into the # active Python interpreter and may run arbitrary code. unsafe-load-any-extension=no [MESSAGES CONTROL] # Only show warnings with the listed confidence levels. Leave empty to show # all. Valid levels: HIGH, INFERENCE, INFERENCE_FAILURE, UNDEFINED. confidence= # Enable the message, report, category or checker with the given id(s). You can # either give multiple identifier separated by comma (,) or put this option # multiple time (only on the command line, not in the configuration file where # it should appear only once). See also the "--disable" option for examples. enable=all # Disable the message, report, category or checker with the given id(s). You # can either give multiple identifiers separated by comma (,) or put this # option multiple times (only on the command line, not in the configuration # file where it should appear only once). You can also use "--disable=all" to # disable everything first and then re-enable specific checks. For example, if # you want to run only the similarities checker, you can use "--disable=all # --enable=similarities". If you want to run only the classes checker, but have # no Warning level messages displayed, use "--disable=all --enable=classes # --disable=W". disable=C0103, C0114, C0115, C0116, C0200, C0201, C0301, C0302, C0321, C0325, C0413, C0415, C1804, C1805, E0401, E0611, E1101, E1120, I0011, I0020, I0021, I0022, I0023, R0201, R0801, R0901, R0902, R0903, R0904, R0911, R0912, R0913, R0914, R0915, R1702, R1703, R1704, R1705, R1706, R1710, R1711, R1714, R1717, R1718, R1719, R1720, R1721, R1722, R1723, R1724, R1729, R1730, R1732, W0212, W0511, W0703, W0707, W1203 [REPORTS] # Python expression which should return a note less than 10 (10 is the highest # note). You have access to the variables errors warning, statement which # respectively contain the number of errors / warnings messages and the total # number of statements analyzed. This is used by the global evaluation report # (RP0004). evaluation=10.0 - ((float(5 * error + warning + refactor + convention) / statement) * 10) # Template used to display messages. This is a python new-style format string # used to format the message information. See doc for all details. #msg-template= # Set the output format. Available formats are text, parseable, colorized, json # and msvs (visual studio). You can also give a reporter class, e.g. # mypackage.mymodule.MyReporterClass. output-format=text # Tells whether to display a full report or only the messages. reports=no # Activate the evaluation score. score=yes [REFACTORING] # Maximum number of nested blocks for function / method body max-nested-blocks=5 # Complete name of functions that never returns. When checking for # inconsistent-return-statements if a never returning function is called then # it will be considered as an explicit return statement and no message will be # printed. never-returning-functions=sys.exit [STRING] # This flag controls whether the implicit-str-concat-in-sequence should # generate a warning on implicit string concatenation in sequences defined over # several lines. check-str-concat-over-line-jumps=no [TYPECHECK] # List of decorators that produce context managers, such as # contextlib.contextmanager. Add to this list to register other decorators that # produce valid context managers. contextmanager-decorators=contextlib.contextmanager # List of members which are set dynamically and missed by pylint inference # system, and so shouldn't trigger E1101 when accessed. Python regular # expressions are accepted. generated-members= # Tells whether missing members accessed in mixin class should be ignored. A # mixin class is detected if its name ends with "mixin" (case insensitive). ignore-mixin-members=yes # Tells whether to warn about missing members when the owner of the attribute # is inferred to be None. ignore-none=yes # This flag controls whether pylint should warn about no-member and similar # checks whenever an opaque object is returned when inferring. The inference # can return multiple potential results while evaluating a Python object, but # some branches might not be evaluated, which results in partial inference. In # that case, it might be useful to still emit no-member and other checks for # the rest of the inferred objects. ignore-on-opaque-inference=yes # List of class names for which member attributes should not be checked (useful # for classes with dynamically set attributes). This supports the use of # qualified names. ignored-classes=optparse.Values,thread._local,_thread._local # List of module names for which member attributes should not be checked # (useful for modules/projects where namespaces are manipulated during runtime # and thus existing member attributes cannot be deduced by static analysis. It # supports qualified module names, as well as Unix pattern matching. ignored-modules= # Show a hint with possible names when a member name was not found. The aspect # of finding the hint is based on edit distance. missing-member-hint=yes # The minimum edit distance a name should have in order to be considered a # similar match for a missing member name. missing-member-hint-distance=1 # The total number of similar names that should be taken in consideration when # showing a hint for a missing member. missing-member-max-choices=1 [BASIC] # Naming style matching correct argument names. argument-naming-style=snake_case # Regular expression matching correct argument names. Overrides argument- # naming-style. #argument-rgx= # Naming style matching correct attribute names. attr-naming-style=snake_case # Regular expression matching correct attribute names. Overrides attr-naming- # style. #attr-rgx= # Bad variable names which should always be refused, separated by a comma. bad-names=foo, bar, baz, toto, tutu, tata # Naming style matching correct class attribute names. class-attribute-naming-style=any # Regular expression matching correct class attribute names. Overrides class- # attribute-naming-style. #class-attribute-rgx= # Naming style matching correct class names. class-naming-style=PascalCase # Regular expression matching correct class names. Overrides class-naming- # style. #class-rgx= # Naming style matching correct constant names. const-naming-style=UPPER_CASE # Regular expression matching correct constant names. Overrides const-naming- # style. #const-rgx= # Minimum line length for functions/classes that require docstrings, shorter # ones are exempt. docstring-min-length=-1 # Naming style matching correct function names. function-naming-style=snake_case # Regular expression matching correct function names. Overrides function- # naming-style. #function-rgx= # Good variable names which should always be accepted, separated by a comma. good-names=i, j, k, ex, Run, _ # Include a hint for the correct naming format with invalid-name. include-naming-hint=no # Naming style matching correct inline iteration names. inlinevar-naming-style=any # Regular expression matching correct inline iteration names. Overrides # inlinevar-naming-style. #inlinevar-rgx= # Naming style matching correct method names. method-naming-style=snake_case # Regular expression matching correct method names. Overrides method-naming- # style. #method-rgx= # Naming style matching correct module names. module-naming-style=snake_case # Regular expression matching correct module names. Overrides module-naming- # style. #module-rgx= # Colon-delimited sets of names that determine each other's naming style when # the name regexes allow several styles. name-group= # Regular expression which should only match function or class names that do # not require a docstring. no-docstring-rgx=^_ # List of decorators that produce properties, such as abc.abstractproperty. Add # to this list to register other decorators that produce valid properties. # These decorators are taken in consideration only for invalid-name. property-classes=abc.abstractproperty # Naming style matching correct variable names. variable-naming-style=snake_case # Regular expression matching correct variable names. Overrides variable- # naming-style. #variable-rgx= [VARIABLES] # List of additional names supposed to be defined in builtins. Remember that # you should avoid defining new builtins when possible. additional-builtins= # Tells whether unused global variables should be treated as a violation. allow-global-unused-variables=yes # List of strings which can identify a callback function by name. A callback # name must start or end with one of those strings. callbacks=cb_, _cb # A regular expression matching the name of dummy variables (i.e. expected to # not be used). dummy-variables-rgx=_+$|(_[a-zA-Z0-9_]*[a-zA-Z0-9]+?$)|dummy|^ignored_|^unused_ # Argument names that match this expression will be ignored. Default to name # with leading underscore. ignored-argument-names=_.*|^ignored_|^unused_ # Tells whether we should check for unused import in __init__ files. init-import=no # List of qualified module names which can have objects that can redefine # builtins. redefining-builtins-modules=six.moves,past.builtins,future.builtins,builtins,io [FORMAT] # Expected format of line ending, e.g. empty (any line ending), LF or CRLF. expected-line-ending-format= # Regexp for a line that is allowed to be longer than the limit. ignore-long-lines=^\s*(# )??$ # Number of spaces of indent required inside a hanging or continued line. indent-after-paren=4 # String used as indentation unit. This is usually " " (4 spaces) or "\t" (1 # tab). indent-string=' ' # Maximum number of characters on a single line. max-line-length=100 # Maximum number of lines in a module. max-module-lines=1000 # Allow the body of a class to be on the same line as the declaration if body # contains single statement. single-line-class-stmt=no # Allow the body of an if to be on the same line as the test if there is no # else. single-line-if-stmt=no [SIMILARITIES] # Ignore comments when computing similarities. ignore-comments=yes # Ignore docstrings when computing similarities. ignore-docstrings=yes # Ignore imports when computing similarities. ignore-imports=no # Minimum lines number of a similarity. min-similarity-lines=4 [MISCELLANEOUS] # List of note tags to take in consideration, separated by a comma. notes=FIXME, XXX, TODO [SPELLING] # Limits count of emitted suggestions for spelling mistakes. max-spelling-suggestions=4 # Spelling dictionary name. Available dictionaries: en_ZM (myspell), en_ZW # (myspell), en_US (myspell), en_PH (myspell), en_IN (myspell), en_BW # (myspell), en_TT (myspell), en_GB (myspell), en_SG (myspell), en_AG # (myspell), en_NG (myspell), en_CA (myspell), en_BS (myspell), en_AU # (myspell), en_JM (myspell), en_ZA (myspell), en_HK (myspell), en_BZ # (myspell), en_IE (myspell), en_NZ (myspell), en_DK (myspell), en_MW # (myspell), en_NA (myspell), en_GH (myspell).. spelling-dict= # List of comma separated words that should not be checked. spelling-ignore-words= # A path to a file that contains private dictionary; one word per line. spelling-private-dict-file= # Tells whether to store unknown words to indicated private dictionary in # --spelling-private-dict-file option instead of raising a message. spelling-store-unknown-words=no [LOGGING] # Format style used to check logging format string. `old` means using % # formatting, while `new` is for `{}` formatting. logging-format-style=old # Logging modules to check that the string format arguments are in logging # function parameter format. logging-modules=logging [IMPORTS] # Allow wildcard imports from modules that define __all__. allow-wildcard-with-all=no # Analyse import fallback blocks. This can be used to support both Python 2 and # 3 compatible code, which means that the block might have code that exists # only in one or another interpreter, leading to false positives when analysed. analyse-fallback-blocks=no # Deprecated modules which should not be used, separated by a comma. deprecated-modules=optparse,tkinter.tix # Create a graph of external dependencies in the given file (report RP0402 must # not be disabled). ext-import-graph= # Create a graph of every (i.e. internal and external) dependencies in the # given file (report RP0402 must not be disabled). import-graph= # Create a graph of internal dependencies in the given file (report RP0402 must # not be disabled). int-import-graph= # Force import order to recognize a module as part of the standard # compatibility libraries. known-standard-library= # Force import order to recognize a module as part of a third party library. known-third-party=enchant [DESIGN] # Maximum number of arguments for function / method. max-args=5 # Maximum number of attributes for a class (see R0902). max-attributes=7 # Maximum number of boolean expressions in an if statement. max-bool-expr=5 # Maximum number of branch for function / method body. max-branches=12 # Maximum number of locals for function / method body. max-locals=15 # Maximum number of parents for a class (see R0901). max-parents=7 # Maximum number of public methods for a class (see R0904). max-public-methods=20 # Maximum number of return / yield for function / method body. max-returns=6 # Maximum number of statements in function / method body. max-statements=50 # Minimum number of public methods for a class (see R0903). min-public-methods=2 [CLASSES] # List of method names used to declare (i.e. assign) instance attributes. defining-attr-methods=__init__, __new__, setUp # List of member names, which should be excluded from the protected access # warning. exclude-protected=_asdict, _fields, _replace, _source, _make # List of valid names for the first argument in a class method. valid-classmethod-first-arg=cls # List of valid names for the first argument in a metaclass class method. valid-metaclass-classmethod-first-arg=cls [EXCEPTIONS] # Exceptions that will emit a warning when being caught. Defaults to # "BaseException, Exception". overgeneral-exceptions=builtin.BaseException, builtin.Exception avocado-113.0/.pylintrc_utils000066400000000000000000000377471513561306000162410ustar00rootroot00000000000000[MASTER] # A comma-separated list of package or module names from where C extensions may # be loaded. Extensions are loading into the active Python interpreter and may # run arbitrary code. extension-pkg-whitelist=netifaces # Add files or directories to the blacklist. They should be base names, not # paths. ignore=CVS,archive.py,asset.py,astring.py,aurl.py,build.py,cloudinit.py,cpu.py,crypto.py,data_factory.py,datadrainer.py,debug.py,diff_validator.py,disk.py,distro.py,dmesg.py,download.py,exit_codes.py,file_utils.py,filelock.py,genio.py,git.py,iso9660.py,kernel.py,linux.py,linux_modules.py,lv_utils.py,memory.py,multipath.py,nvme.py,partition.py,pci.py,pmem.py,podman.py,service.py,softwareraid.py,ssh.py,stacktrace.py,sysinfo.py,vmimage.py,wait.py,gdbmi_parser.py,spark.py,distro_packages.py,inspector.py,main.py,manager.py,apt.py,base.py,dnf.py,dpkg.py,rpm.py,yum.py,zypper.py,deprecation.py # regex matches against base names, not paths. ignore-patterns=.git # Python code to execute, usually for sys.path manipulation such as # pygtk.require(). #init-hook= # Use multiple processes to speed up Pylint. Specifying 0 will auto-detect the # number of processors available to use. jobs=0 # Control the amount of potential inferred values when inferring a single # object. This can help the performance when dealing with large functions or # complex, nested conditions. limit-inference-results=100 # List of plugins (as comma separated values of python modules names) to load, # usually to register additional checkers. load-plugins=pylint.extensions.no_self_use,pylint.extensions.docparams,pylint.extensions.docstyle # Pickle collected data for later comparisons. persistent=yes # Specify a configuration file. #rcfile= # When enabled, pylint would attempt to guess common misconfiguration and emit # user-friendly hints instead of false-positive error messages. suggestion-mode=yes # Allow loading of arbitrary C extensions. Extensions are imported into the # active Python interpreter and may run arbitrary code. unsafe-load-any-extension=no [MESSAGES CONTROL] # Only show warnings with the listed confidence levels. Leave empty to show # all. Valid levels: HIGH, INFERENCE, INFERENCE_FAILURE, UNDEFINED. confidence= # Enable the message, report, category or checker with the given id(s). You can # either give multiple identifier separated by comma (,) or put this option # multiple time (only on the command line, not in the configuration file where # it should appear only once). See also the "--disable" option for examples. enable=all # Disable the message, report, category or checker with the given id(s). You # can either give multiple identifiers separated by comma (,) or put this # option multiple times (only on the command line, not in the configuration # file where it should appear only once). You can also use "--disable=all" to # disable everything first and then re-enable specific checks. For example, if # you want to run only the similarities checker, you can use "--disable=all # --enable=similarities". If you want to run only the classes checker, but have # no Warning level messages displayed, use "--disable=all --enable=classes # --disable=W". disable=line-too-long, locally-disabled, suppressed-message, useless-suppression, use-symbolic-message-instead, too-few-public-methods, fixme [REPORTS] # Python expression which should return a note less than 10 (10 is the highest # note). You have access to the variables errors warning, statement which # respectively contain the number of errors / warnings messages and the total # number of statements analyzed. This is used by the global evaluation report # (RP0004). evaluation=10.0 - ((float(5 * error + warning + refactor + convention) / statement) * 10) # Template used to display messages. This is a python new-style format string # used to format the message information. See doc for all details. #msg-template= # Set the output format. Available formats are text, parseable, colorized, json # and msvs (visual studio). You can also give a reporter class, e.g. # mypackage.mymodule.MyReporterClass. output-format=text # Tells whether to display a full report or only the messages. reports=no # Activate the evaluation score. score=yes [REFACTORING] # Maximum number of nested blocks for function / method body max-nested-blocks=5 # Complete name of functions that never returns. When checking for # inconsistent-return-statements if a never returning function is called then # it will be considered as an explicit return statement and no message will be # printed. never-returning-functions=sys.exit [STRING] # This flag controls whether the implicit-str-concat-in-sequence should # generate a warning on implicit string concatenation in sequences defined over # several lines. check-str-concat-over-line-jumps=no [TYPECHECK] # List of decorators that produce context managers, such as # contextlib.contextmanager. Add to this list to register other decorators that # produce valid context managers. contextmanager-decorators=contextlib.contextmanager # List of members which are set dynamically and missed by pylint inference # system, and so shouldn't trigger E1101 when accessed. Python regular # expressions are accepted. generated-members= # Tells whether missing members accessed in mixin class should be ignored. A # mixin class is detected if its name ends with "mixin" (case insensitive). ignore-mixin-members=yes # Tells whether to warn about missing members when the owner of the attribute # is inferred to be None. ignore-none=yes # This flag controls whether pylint should warn about no-member and similar # checks whenever an opaque object is returned when inferring. The inference # can return multiple potential results while evaluating a Python object, but # some branches might not be evaluated, which results in partial inference. In # that case, it might be useful to still emit no-member and other checks for # the rest of the inferred objects. ignore-on-opaque-inference=yes # List of class names for which member attributes should not be checked (useful # for classes with dynamically set attributes). This supports the use of # qualified names. ignored-classes=optparse.Values,thread._local,_thread._local # List of module names for which member attributes should not be checked # (useful for modules/projects where namespaces are manipulated during runtime # and thus existing member attributes cannot be deduced by static analysis. It # supports qualified module names, as well as Unix pattern matching. ignored-modules= # Show a hint with possible names when a member name was not found. The aspect # of finding the hint is based on edit distance. missing-member-hint=yes # The minimum edit distance a name should have in order to be considered a # similar match for a missing member name. missing-member-hint-distance=1 # The total number of similar names that should be taken in consideration when # showing a hint for a missing member. missing-member-max-choices=1 [BASIC] # Naming style matching correct argument names. argument-naming-style=snake_case # Regular expression matching correct argument names. Overrides argument- # naming-style. #argument-rgx= # Naming style matching correct attribute names. attr-naming-style=snake_case # Regular expression matching correct attribute names. Overrides attr-naming- # style. #attr-rgx= # Bad variable names which should always be refused, separated by a comma. bad-names=foo, bar, baz, toto, tutu, tata # Naming style matching correct class attribute names. class-attribute-naming-style=any # Regular expression matching correct class attribute names. Overrides class- # attribute-naming-style. #class-attribute-rgx= # Naming style matching correct class names. class-naming-style=PascalCase # Regular expression matching correct class names. Overrides class-naming- # style. #class-rgx= # Naming style matching correct constant names. const-naming-style=UPPER_CASE # Regular expression matching correct constant names. Overrides const-naming- # style. #const-rgx= # Minimum line length for functions/classes that require docstrings, shorter # ones are exempt. docstring-min-length=-1 # Naming style matching correct function names. function-naming-style=snake_case # Regular expression matching correct function names. Overrides function- # naming-style. #function-rgx= # Good variable names which should always be accepted, separated by a comma. good-names=i, j, k, ex, Run, _ # Include a hint for the correct naming format with invalid-name. include-naming-hint=no # Naming style matching correct inline iteration names. inlinevar-naming-style=any # Regular expression matching correct inline iteration names. Overrides # inlinevar-naming-style. #inlinevar-rgx= # Naming style matching correct method names. method-naming-style=snake_case # Regular expression matching correct method names. Overrides method-naming- # style. #method-rgx= # Naming style matching correct module names. module-naming-style=snake_case # Regular expression matching correct module names. Overrides module-naming- # style. #module-rgx= # Colon-delimited sets of names that determine each other's naming style when # the name regexes allow several styles. name-group= # Regular expression which should only match function or class names that do # not require a docstring. no-docstring-rgx=^_ # List of decorators that produce properties, such as abc.abstractproperty. Add # to this list to register other decorators that produce valid properties. # These decorators are taken in consideration only for invalid-name. property-classes=abc.abstractproperty # Naming style matching correct variable names. variable-naming-style=snake_case # Regular expression matching correct variable names. Overrides variable- # naming-style. #variable-rgx= [VARIABLES] # List of additional names supposed to be defined in builtins. Remember that # you should avoid defining new builtins when possible. additional-builtins= # Tells whether unused global variables should be treated as a violation. allow-global-unused-variables=yes # List of strings which can identify a callback function by name. A callback # name must start or end with one of those strings. callbacks=cb_, _cb # A regular expression matching the name of dummy variables (i.e. expected to # not be used). dummy-variables-rgx=_+$|(_[a-zA-Z0-9_]*[a-zA-Z0-9]+?$)|dummy|^ignored_|^unused_ # Argument names that match this expression will be ignored. Default to name # with leading underscore. ignored-argument-names=_.*|^ignored_|^unused_ # Tells whether we should check for unused import in __init__ files. init-import=no # List of qualified module names which can have objects that can redefine # builtins. redefining-builtins-modules=six.moves,past.builtins,future.builtins,builtins,io [FORMAT] # Expected format of line ending, e.g. empty (any line ending), LF or CRLF. expected-line-ending-format= # Regexp for a line that is allowed to be longer than the limit. ignore-long-lines=^\s*(# )??$ # Number of spaces of indent required inside a hanging or continued line. indent-after-paren=4 # String used as indentation unit. This is usually " " (4 spaces) or "\t" (1 # tab). indent-string=' ' # Maximum number of characters on a single line. max-line-length=100 # Maximum number of lines in a module. max-module-lines=1000 # Allow the body of a class to be on the same line as the declaration if body # contains single statement. single-line-class-stmt=no # Allow the body of an if to be on the same line as the test if there is no # else. single-line-if-stmt=no [SIMILARITIES] # Ignore comments when computing similarities. ignore-comments=yes # Ignore docstrings when computing similarities. ignore-docstrings=yes # Ignore imports when computing similarities. ignore-imports=no # Minimum lines number of a similarity. min-similarity-lines=4 [MISCELLANEOUS] # List of note tags to take in consideration, separated by a comma. notes=FIXME, XXX, TODO [SPELLING] # Limits count of emitted suggestions for spelling mistakes. max-spelling-suggestions=4 # Spelling dictionary name. Available dictionaries: en_ZM (myspell), en_ZW # (myspell), en_US (myspell), en_PH (myspell), en_IN (myspell), en_BW # (myspell), en_TT (myspell), en_GB (myspell), en_SG (myspell), en_AG # (myspell), en_NG (myspell), en_CA (myspell), en_BS (myspell), en_AU # (myspell), en_JM (myspell), en_ZA (myspell), en_HK (myspell), en_BZ # (myspell), en_IE (myspell), en_NZ (myspell), en_DK (myspell), en_MW # (myspell), en_NA (myspell), en_GH (myspell).. spelling-dict= # List of comma separated words that should not be checked. spelling-ignore-words= # A path to a file that contains private dictionary; one word per line. spelling-private-dict-file= # Tells whether to store unknown words to indicated private dictionary in # --spelling-private-dict-file option instead of raising a message. spelling-store-unknown-words=no [LOGGING] # Format style used to check logging format string. `old` means using % # formatting, while `new` is for `{}` formatting. logging-format-style=old # Logging modules to check that the string format arguments are in logging # function parameter format. logging-modules=logging [IMPORTS] # Allow wildcard imports from modules that define __all__. allow-wildcard-with-all=no # Analyse import fallback blocks. This can be used to support both Python 2 and # 3 compatible code, which means that the block might have code that exists # only in one or another interpreter, leading to false positives when analysed. analyse-fallback-blocks=no # Deprecated modules which should not be used, separated by a comma. deprecated-modules=optparse,tkinter.tix # Create a graph of external dependencies in the given file (report RP0402 must # not be disabled). ext-import-graph= # Create a graph of every (i.e. internal and external) dependencies in the # given file (report RP0402 must not be disabled). import-graph= # Create a graph of internal dependencies in the given file (report RP0402 must # not be disabled). int-import-graph= # Force import order to recognize a module as part of the standard # compatibility libraries. known-standard-library= # Force import order to recognize a module as part of a third party library. known-third-party=enchant [DESIGN] # Maximum number of arguments for function / method. max-args=5 # Maximum number of attributes for a class (see R0902). max-attributes=7 # Maximum number of boolean expressions in an if statement. max-bool-expr=5 # Maximum number of branch for function / method body. max-branches=12 # Maximum number of locals for function / method body. max-locals=15 # Maximum number of parents for a class (see R0901). max-parents=7 # Maximum number of public methods for a class (see R0904). max-public-methods=20 # Maximum number of return / yield for function / method body. max-returns=6 # Maximum number of statements in function / method body. max-statements=50 # Minimum number of public methods for a class (see R0903). min-public-methods=2 [CLASSES] # List of method names used to declare (i.e. assign) instance attributes. defining-attr-methods=__init__, __new__, setUp # List of member names, which should be excluded from the protected access # warning. exclude-protected=_asdict, _fields, _replace, _source, _make # List of valid names for the first argument in a class method. valid-classmethod-first-arg=cls # List of valid names for the first argument in a metaclass class method. valid-metaclass-classmethod-first-arg=cls [EXCEPTIONS] # Exceptions that will emit a warning when being caught. Defaults to # "BaseException, Exception". overgeneral-exceptions=builtins.BaseException, builtins.Exception [tool.pylint.parameter_documentation] accept-no-param-doc = false accept-no-raise-doc = false accept-no-return-doc = false accept-no-yields-doc = false # Possible choices: ['sphinx', 'epytext', 'google', 'numpy', 'default'] default-docstring-type = sphinx avocado-113.0/.readthedocs.yaml000066400000000000000000000011731513561306000163630ustar00rootroot00000000000000# .readthedocs.yaml # Read the Docs configuration file # See https://docs.readthedocs.io/en/stable/config-file/v2.html for details # Required version: 2 # Set the version of Python and other tools you might need build: os: ubuntu-22.04 tools: python: "3.11" # Build documentation in the docs/ directory with Sphinx sphinx: configuration: docs/source/conf.py fail_on_warning: true # We recommend specifying your dependencies to enable reproducible builds: # https://docs.readthedocs.io/en/stable/guides/reproducible-builds.html python: install: - requirements: requirements-doc.txt - method: pip path: . avocado-113.0/CONTRIBUTING.rst000077700000000000000000000000001513561306000255112docs/source/guides/contributor/index.rstustar00rootroot00000000000000avocado-113.0/LICENSE000066400000000000000000000441561513561306000141510ustar00rootroot00000000000000Most files in Avocado are released under the GPLv2 with a "and any later version" clause (GPLv2+), but some files are explicitly bound to the version 2 of the license and therefore that's the official license of the project itself. The header on each individual file contains information about the GPL version that applies to it. Unless explicitly otherwise stated, all files in the avocado project are covered by the GPLv2+ (i.e: v2, or any later version). GNU GENERAL PUBLIC LICENSE Version 2, June 1991 Copyright (C) 1989, 1991 Free Software Foundation, Inc. 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed. Preamble The licenses for most software are designed to take away your freedom to share and change it. By contrast, the GNU General Public License is intended to guarantee your freedom to share and change free software--to make sure the software is free for all its users. This General Public License applies to most of the Free Software Foundation's software and to any other program whose authors commit to using it. (Some other Free Software Foundation software is covered by the GNU Library General Public License instead.) You can apply it to your programs, too. When we speak of free software, we are referring to freedom, not price. Our General Public Licenses are designed to make sure that you have the freedom to distribute copies of free software (and charge for this service if you wish), that you receive source code or can get it if you want it, that you can change the software or use pieces of it in new free programs; and that you know you can do these things. To protect your rights, we need to make restrictions that forbid anyone to deny you these rights or to ask you to surrender the rights. These restrictions translate to certain responsibilities for you if you distribute copies of the software, or if you modify it. For example, if you distribute copies of such a program, whether gratis or for a fee, you must give the recipients all the rights that you have. You must make sure that they, too, receive or can get the source code. And you must show them these terms so they know their rights. We protect your rights with two steps: (1) copyright the software, and (2) offer you this license which gives you legal permission to copy, distribute and/or modify the software. Also, for each author's protection and ours, we want to make certain that everyone understands that there is no warranty for this free software. If the software is modified by someone else and passed on, we want its recipients to know that what they have is not the original, so that any problems introduced by others will not reflect on the original authors' reputations. Finally, any free program is threatened constantly by software patents. We wish to avoid the danger that redistributors of a free program will individually obtain patent licenses, in effect making the program proprietary. To prevent this, we have made it clear that any patent must be licensed for everyone's free use or not licensed at all. The precise terms and conditions for copying, distribution and modification follow. GNU GENERAL PUBLIC LICENSE TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION 0. This License applies to any program or other work which contains a notice placed by the copyright holder saying it may be distributed under the terms of this General Public License. The "Program", below, refers to any such program or work, and a "work based on the Program" means either the Program or any derivative work under copyright law: that is to say, a work containing the Program or a portion of it, either verbatim or with modifications and/or translated into another language. (Hereinafter, translation is included without limitation in the term "modification".) Each licensee is addressed as "you". Activities other than copying, distribution and modification are not covered by this License; they are outside its scope. The act of running the Program is not restricted, and the output from the Program is covered only if its contents constitute a work based on the Program (independent of having been made by running the Program). Whether that is true depends on what the Program does. 1. You may copy and distribute verbatim copies of the Program's source code as you receive it, in any medium, provided that you conspicuously and appropriately publish on each copy an appropriate copyright notice and disclaimer of warranty; keep intact all the notices that refer to this License and to the absence of any warranty; and give any other recipients of the Program a copy of this License along with the Program. You may charge a fee for the physical act of transferring a copy, and you may at your option offer warranty protection in exchange for a fee. 2. You may modify your copy or copies of the Program or any portion of it, thus forming a work based on the Program, and copy and distribute such modifications or work under the terms of Section 1 above, provided that you also meet all of these conditions: a) You must cause the modified files to carry prominent notices stating that you changed the files and the date of any change. b) You must cause any work that you distribute or publish, that in whole or in part contains or is derived from the Program or any part thereof, to be licensed as a whole at no charge to all third parties under the terms of this License. c) If the modified program normally reads commands interactively when run, you must cause it, when started running for such interactive use in the most ordinary way, to print or display an announcement including an appropriate copyright notice and a notice that there is no warranty (or else, saying that you provide a warranty) and that users may redistribute the program under these conditions, and telling the user how to view a copy of this License. (Exception: if the Program itself is interactive but does not normally print such an announcement, your work based on the Program is not required to print an announcement.) These requirements apply to the modified work as a whole. If identifiable sections of that work are not derived from the Program, and can be reasonably considered independent and separate works in themselves, then this License, and its terms, do not apply to those sections when you distribute them as separate works. But when you distribute the same sections as part of a whole which is a work based on the Program, the distribution of the whole must be on the terms of this License, whose permissions for other licensees extend to the entire whole, and thus to each and every part regardless of who wrote it. Thus, it is not the intent of this section to claim rights or contest your rights to work written entirely by you; rather, the intent is to exercise the right to control the distribution of derivative or collective works based on the Program. In addition, mere aggregation of another work not based on the Program with the Program (or with a work based on the Program) on a volume of a storage or distribution medium does not bring the other work under the scope of this License. 3. You may copy and distribute the Program (or a work based on it, under Section 2) in object code or executable form under the terms of Sections 1 and 2 above provided that you also do one of the following: a) Accompany it with the complete corresponding machine-readable source code, which must be distributed under the terms of Sections 1 and 2 above on a medium customarily used for software interchange; or, b) Accompany it with a written offer, valid for at least three years, to give any third party, for a charge no more than your cost of physically performing source distribution, a complete machine-readable copy of the corresponding source code, to be distributed under the terms of Sections 1 and 2 above on a medium customarily used for software interchange; or, c) Accompany it with the information you received as to the offer to distribute corresponding source code. (This alternative is allowed only for noncommercial distribution and only if you received the program in object code or executable form with such an offer, in accord with Subsection b above.) The source code for a work means the preferred form of the work for making modifications to it. For an executable work, complete source code means all the source code for all modules it contains, plus any associated interface definition files, plus the scripts used to control compilation and installation of the executable. However, as a special exception, the source code distributed need not include anything that is normally distributed (in either source or binary form) with the major components (compiler, kernel, and so on) of the operating system on which the executable runs, unless that component itself accompanies the executable. If distribution of executable or object code is made by offering access to copy from a designated place, then offering equivalent access to copy the source code from the same place counts as distribution of the source code, even though third parties are not compelled to copy the source along with the object code. 4. You may not copy, modify, sublicense, or distribute the Program except as expressly provided under this License. Any attempt otherwise to copy, modify, sublicense or distribute the Program is void, and will automatically terminate your rights under this License. However, parties who have received copies, or rights, from you under this License will not have their licenses terminated so long as such parties remain in full compliance. 5. You are not required to accept this License, since you have not signed it. However, nothing else grants you permission to modify or distribute the Program or its derivative works. These actions are prohibited by law if you do not accept this License. Therefore, by modifying or distributing the Program (or any work based on the Program), you indicate your acceptance of this License to do so, and all its terms and conditions for copying, distributing or modifying the Program or works based on it. 6. Each time you redistribute the Program (or any work based on the Program), the recipient automatically receives a license from the original licensor to copy, distribute or modify the Program subject to these terms and conditions. You may not impose any further restrictions on the recipients' exercise of the rights granted herein. You are not responsible for enforcing compliance by third parties to this License. 7. If, as a consequence of a court judgment or allegation of patent infringement or for any other reason (not limited to patent issues), conditions are imposed on you (whether by court order, agreement or otherwise) that contradict the conditions of this License, they do not excuse you from the conditions of this License. If you cannot distribute so as to satisfy simultaneously your obligations under this License and any other pertinent obligations, then as a consequence you may not distribute the Program at all. For example, if a patent license would not permit royalty-free redistribution of the Program by all those who receive copies directly or indirectly through you, then the only way you could satisfy both it and this License would be to refrain entirely from distribution of the Program. If any portion of this section is held invalid or unenforceable under any particular circumstance, the balance of the section is intended to apply and the section as a whole is intended to apply in other circumstances. It is not the purpose of this section to induce you to infringe any patents or other property right claims or to contest validity of any such claims; this section has the sole purpose of protecting the integrity of the free software distribution system, which is implemented by public license practices. Many people have made generous contributions to the wide range of software distributed through that system in reliance on consistent application of that system; it is up to the author/donor to decide if he or she is willing to distribute software through any other system and a licensee cannot impose that choice. This section is intended to make thoroughly clear what is believed to be a consequence of the rest of this License. 8. If the distribution and/or use of the Program is restricted in certain countries either by patents or by copyrighted interfaces, the original copyright holder who places the Program under this License may add an explicit geographical distribution limitation excluding those countries, so that distribution is permitted only in or among countries not thus excluded. In such case, this License incorporates the limitation as if written in the body of this License. 9. The Free Software Foundation may publish revised and/or new versions of the General Public License from time to time. Such new versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns. Each version is given a distinguishing version number. If the Program specifies a version number of this License which applies to it and "any later version", you have the option of following the terms and conditions either of that version or of any later version published by the Free Software Foundation. If the Program does not specify a version number of this License, you may choose any version ever published by the Free Software Foundation. 10. If you wish to incorporate parts of the Program into other free programs whose distribution conditions are different, write to the author to ask for permission. For software which is copyrighted by the Free Software Foundation, write to the Free Software Foundation; we sometimes make exceptions for this. Our decision will be guided by the two goals of preserving the free status of all derivatives of our free software and of promoting the sharing and reuse of software generally. NO WARRANTY 11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION. 12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. END OF TERMS AND CONDITIONS How to Apply These Terms to Your New Programs If you develop a new program, and you want it to be of the greatest possible use to the public, the best way to achieve this is to make it free software which everyone can redistribute and change under these terms. To do so, attach the following notices to the program. It is safest to attach them to the start of each source file to most effectively convey the exclusion of warranty; and each file should have at least the "copyright" line and a pointer to where the full notice is found. Copyright (C) This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA Also add information on how to contact you by electronic and paper mail. If the program is interactive, make it output a short notice like this when it starts in an interactive mode: Gnomovision version 69, Copyright (C) year name of author Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'. This is free software, and you are welcome to redistribute it under certain conditions; type `show c' for details. The hypothetical commands `show w' and `show c' should show the appropriate parts of the General Public License. Of course, the commands you use may be called something other than `show w' and `show c'; they could even be mouse-clicks or menu items--whatever suits your program. You should also get your employer (if you work as a programmer) or your school, if any, to sign a "copyright disclaimer" for the program, if necessary. Here is a sample; alter the names: Yoyodyne, Inc., hereby disclaims all copyright interest in the program `Gnomovision' (which makes passes at compilers) written by James Hacker. , 1 April 1989 Ty Coon, President of Vice This General Public License does not permit incorporating your program into proprietary programs. If your program is a subroutine library, you may consider it more useful to permit linking proprietary applications with the library. If this is what you want to do, use the GNU Library General Public License instead of this License. avocado-113.0/MANIFEST.in000066400000000000000000000003471513561306000146740ustar00rootroot00000000000000include LICENSE include Makefile include README.rst include VERSION recursive-include avocado/etc * recursive-include avocado/libexec * recursive-include avocado/schemas * recursive-include selftests * recursive-include examples * avocado-113.0/Makefile000066400000000000000000000051721513561306000145770ustar00rootroot00000000000000all: @echo @echo "Development related targets:" @echo "check: Runs tree static check, unittests and fast functional tests" @echo "smokecheck: Runs the simplest possible avocado test execution" @echo "develop: Runs 'python setup.py --develop' on this tree alone" @echo "develop-external: Install Avocado's external plugins in develop mode." @echo " You need to set AVOCADO_EXTERNAL_PLUGINS_PATH" @echo "clean: Get rid of build scratch from this project and subprojects" @echo "variables: Show the value of variables as defined in this Makefile or" @echo " given as input to make" @echo @echo "Package requirements related targets" @echo "requirements-dev: Install development requirements" @echo "requirements-static-checks: Install development requirements for static checks" @echo @echo "Platform independent distribution/installation related targets:" @echo "install: Install on local system" @echo "uninstall: Uninstall Avocado and also subprojects" @echo "man: Generate the avocado man page" @echo "pip: Auxiliary target to install pip. (It's not recommended to run this directly)" @echo include Makefile.include DESTDIR=/ AVOCADO_DIRNAME=$(shell basename ${PWD}) AVOCADO_OPTIONAL_PLUGINS=$(shell find ./optional_plugins -maxdepth 1 -mindepth 1 -type d) clean: $(PYTHON) setup.py clean --all install: $(PYTHON) setup.py install --root $(DESTDIR) $(COMPILE) uninstall: $(PYTHON) setup.py develop --uninstall $(PYTHON_DEVELOP_ARGS) requirements-dev: pip $(PYTHON) -m pip install -r requirements-dev.txt $(PYTHON_DEVELOP_ARGS) requirements-static-checks: pip $(PYTHON) -m pip install -r static-checks/requirements.txt $(PYTHON_DEVELOP_ARGS) smokecheck: clean uninstall develop $(PYTHON) -m avocado run examples/tests/passtest.py check: clean uninstall develop # Unless manually set, this is equivalent to AVOCADO_CHECK_LEVEL=0 $(PYTHON) selftests/check.py develop: $(PYTHON) setup.py develop $(PYTHON_DEVELOP_ARGS) develop-external: ifndef AVOCADO_EXTERNAL_PLUGINS_PATH $(error AVOCADO_EXTERNAL_PLUGINS_PATH is not defined) endif $(PYTHON) setup.py develop $(PYTHON_DEVELOP_ARGS) --external man: man/avocado.1 %.1: %.rst $(PYTHON) setup.py man variables: @echo "PYTHON: $(PYTHON)" @echo "VERSION: $(VERSION)" @echo "PYTHON_DEVELOP_ARGS: $(PYTHON_DEVELOP_ARGS)" @echo "DESTDIR: $(DESTDIR)" @echo "AVOCADO_DIRNAME: $(AVOCADO_DIRNAME)" @echo "AVOCADO_OPTIONAL_PLUGINS: $(AVOCADO_OPTIONAL_PLUGINS)" .PHONY: pip install clean uninstall requirements-dev smokecheck check develop develop-external variables man avocado-113.0/Makefile.gh000066400000000000000000000057001513561306000151710ustar00rootroot00000000000000# This Makefile contains targets used by GitHub Actions include Makefile.include ifndef AVOCADO_OPTIONAL_PLUGINS AVOCADO_OPTIONAL_PLUGINS=$(shell find ./optional_plugins -maxdepth 1 -mindepth 1 -type d) endif ifndef DEVEL_NAME DEVEL_NAME=$(shell git config --get user.name) endif ifndef DEVEL_MAIL DEVEL_NAME=$(shell git config --get user.email) endif all: @echo @echo "This file contains targets used by GitHub Actions." @echo codespell: pip install codespell==2.2.4 codespell --check-filenames --check-hidden --skip ".git,*.js,./selftests/unit/utils/cpu.py.data/*,./avocado/utils/pmem.py,./avocado/utils/cpu.py,./avocado/utils/linux_modules.py,./selftests/functional/output.py,./selftests/unit/utils/vmimage.py" bandit: pip install bandit bandit -o bandit-output.txt -r --skip B101,B105,B311,B404,B603 . propagate-version: echo "$(VERSION)" > VERSION for DIR in $(AVOCADO_OPTIONAL_PLUGINS); do\ if test -f "$$DIR/VERSION"; then\ echo ">> Updating $$DIR"; echo "$(VERSION)" > "$$DIR/VERSION";\ else echo ">> Skipping $$DIR"; fi;\ done release-update-spec: rpmdev-bumpspec -n "$(VERSION)" -u "$(DEVEL_NAME) <$(DEVEL_MAIL)>" -c "New release" python-avocado.spec sed -i 's/^Release:.*/Release: 1%{?gitrel}%{?dist}/' python-avocado.spec release-commit-tag: git commit -m "Changes for release $(VERSION)" -a git tag "$(VERSION)" -m "Release $(VERSION)" build-update-readthedocs: ifndef TOKEN_RTD $(error TOKEN_RTD is undefined) endif ifndef URL $(error URL is undefined) endif # Activate version curl -X PATCH \ -H "Authorization: Token $(TOKEN_RTD)" "$(URL)/versions/$(VERSION)/" \ -H "Content-Type: application/json" -d '{"active": true, "hidden": false }' # Build new version curl -X POST \ -H "Authorization: Token $(TOKEN_RTD)" "$(URL)/versions/$(VERSION)/builds/" build-wheel: pip $(PYTHON) -m pip install $(PYTHON_DEVELOP_ARGS) build if test ! -d PYPI_UPLOAD; then mkdir PYPI_UPLOAD; fi $(PYTHON) -m build -o PYPI_UPLOAD for PLUGIN in $(AVOCADO_OPTIONAL_PLUGINS); do\ if test -f $$PLUGIN/setup.py; then\ cd $$PLUGIN;\ $(PYTHON) -m build -o ../../PYPI_UPLOAD;\ cd -;\ fi;\ done check-wheel: build-wheel $(PYTHON) -m pip install twine==6.1.0 packaging==24.2 twine check --strict ./PYPI_UPLOAD/* build-egg: if test ! -d EGG_UPLOAD; then mkdir EGG_UPLOAD; fi $(PYTHON) setup.py bdist_egg -d EGG_UPLOAD for PLUGIN in $(AVOCADO_OPTIONAL_PLUGINS); do\ if test -f $$PLUGIN/setup.py; then\ cd $$PLUGIN;\ $(PYTHON) setup.py bdist_egg -d ../../EGG_UPLOAD;\ cd -;\ fi;\ done update-pypi: ifndef TWINE_USERNAME $(error TWINE_USERNAME is undefined) endif ifndef TWINE_PASSWORD $(error TWINE_PASSWORD is undefined) endif pip install twine packaging>=24.2 export TWINE_NON_INTERACTIVE=true twine upload PYPI_UPLOAD/*.{tar.gz,whl} variables: @echo "PYTHON: $(PYTHON)" @echo "VERSION: $(VERSION)" @echo "AVOCADO_OPTIONAL_PLUGINS: $(AVOCADO_OPTIONAL_PLUGINS)" @echo "DEVEL_NAME: $(DEVEL_NAME)" @echo "DEVEL_MAIL: $(DEVEL_MAIL)" avocado-113.0/Makefile.include000066400000000000000000000007451513561306000162220ustar00rootroot00000000000000# This Makefile contains snippets used on other Makefiles ifndef PYTHON PYTHON=$(shell which python3 2>/dev/null || which python 2>/dev/null) endif ifndef VERSION VERSION=$(shell $(PYTHON) setup.py --version 2>/dev/null) endif ifndef PYTHON_DEVELOP_ARGS PYTHON_DEVELOP_ARGS=$(shell if ($(PYTHON) setup.py develop --help 2>/dev/null | grep -q '\-\-user'); then echo "--user"; else echo ""; fi) endif pip: $(PYTHON) -m pip --version || $(PYTHON) -m ensurepip $(PYTHON_DEVELOP_ARGS) avocado-113.0/Makefile.pkg000066400000000000000000000053661513561306000153640ustar00rootroot00000000000000# This Makefile contains RPM related targets include Makefile.include COMMIT=$(shell git log --pretty=format:'%H' -n 1) COMMIT_DATE=$(shell git log --pretty='format:%cd' --date='format:%Y%m%d' -n 1) SHORT_COMMIT=$(shell git rev-parse --short=9 HEAD) MOCK_CONFIG=default ARCHIVE_BASE_NAME=avocado PYTHON_MODULE_NAME=avocado-framework RPM_BASE_NAME=python-avocado all: @echo @echo "RPM related targets:" @echo "source: Create single source package with commit info, suitable for RPMs" @echo "srpm: Generate a source RPM package (.srpm)" @echo "rpm: Generate binary RPMs" @echo @echo "Release related targets:" @echo "source-release: Create source package for the latest tagged release" @echo "srpm-release: Generate a source RPM package (.srpm) for the latest tagged release" @echo "rpm-release: Generate binary RPMs for the latest tagged release" @echo @echo "variables: Show the value of variables as defined in this Makefile or" @echo " given as input to make" @echo "clean: Clean previous build directories" @echo clean: rm -rf SOURCES BUILD source: clean if test ! -d SOURCES; then mkdir SOURCES; fi git archive --prefix="$(ARCHIVE_BASE_NAME)-$(COMMIT)/" -o "SOURCES/$(ARCHIVE_BASE_NAME)-$(SHORT_COMMIT).tar.gz" HEAD source-release: clean if test ! -d SOURCES; then mkdir SOURCES; fi git archive --prefix="$(ARCHIVE_BASE_NAME)-$(VERSION)/" -o "SOURCES/$(ARCHIVE_BASE_NAME)-$(VERSION).tar.gz" $(VERSION) srpm: source if test ! -d BUILD/SRPM; then mkdir -p BUILD/SRPM; fi mock -r $(MOCK_CONFIG) --resultdir BUILD/SRPM -D "rel_build 0" -D "commit $(COMMIT)" -D "commit_date $(COMMIT_DATE)" --buildsrpm --spec $(RPM_BASE_NAME).spec --sources SOURCES rpm: srpm if test ! -d BUILD/RPM; then mkdir -p BUILD/RPM; fi mock -r $(MOCK_CONFIG) --no-clean --resultdir BUILD/RPM -D "rel_build 0" -D "commit $(COMMIT)" -D "commit_date $(COMMIT_DATE)" --rebuild BUILD/SRPM/$(RPM_BASE_NAME)-$(VERSION)-*.src.rpm srpm-release: source-release if test ! -d BUILD/SRPM; then mkdir -p BUILD/SRPM; fi mock -r $(MOCK_CONFIG) --resultdir BUILD/SRPM -D "rel_build 1" --buildsrpm --spec $(RPM_BASE_NAME).spec --sources SOURCES rpm-release: srpm-release if test ! -d BUILD/RPM; then mkdir -p BUILD/RPM; fi mock -r $(MOCK_CONFIG) --no-clean --resultdir BUILD/RPM -D "rel_build 1" --rebuild BUILD/SRPM/$(RPM_BASE_NAME)-$(VERSION)-*.src.rpm variables: @echo "VERSION: $(VERSION)" @echo "COMMIT: $(COMMIT)" @echo "COMMIT_DATE: $(COMMIT_DATE)" @echo "SHORT_COMMIT: $(SHORT_COMMIT)" @echo "MOCK_CONFIG: $(MOCK_CONFIG)" @echo "ARCHIVE_BASE_NAME: $(ARCHIVE_BASE_NAME)" @echo "PYTHON_MODULE_NAME: $(PYTHON_MODULE_NAME)" @echo "RPM_BASE_NAME: $(RPM_BASE_NAME)" .PHONY: clean source source-release sprm rpm srpm-release rpm-release variables avocado-113.0/README000077700000000000000000000000001513561306000154742README.rstustar00rootroot00000000000000avocado-113.0/README.rst000077700000000000000000000000001513561306000230772docs/source/quickstart/index.rstustar00rootroot00000000000000avocado-113.0/VERSION000066400000000000000000000000061513561306000141760ustar00rootroot00000000000000113.0 avocado-113.0/avocado-static-checks.conf000066400000000000000000000001461513561306000201410ustar00rootroot00000000000000[lint] avocado/utils:static-checks/default_configs/pylintrc avocado/utils:.pylintrc_utils .:.pylintrc avocado-113.0/avocado/000077500000000000000000000000001513561306000145465ustar00rootroot00000000000000avocado-113.0/avocado/__init__.py000066400000000000000000000022261513561306000166610ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2013-2014 # Author: Lucas Meneghel Rodrigues __all__ = [ "Test", "VERSION", "fail_on", "cancel_on", "skip", "skipIf", "skipUnless", "TestError", "TestFail", "TestCancel", ] from avocado.core import initialize_plugins, register_core_options from avocado.core.settings import settings register_core_options() settings.merge_with_configs() initialize_plugins() settings.merge_with_configs() from avocado.core.decorators import cancel_on, fail_on, skip, skipIf, skipUnless from avocado.core.exceptions import TestCancel, TestError, TestFail from avocado.core.test import Test from avocado.core.version import VERSION avocado-113.0/avocado/__main__.py000066400000000000000000000002261513561306000166400ustar00rootroot00000000000000""" Main entry point when called by 'python -m'. """ import sys from avocado.core.main import main if __name__ == "__main__": sys.exit(main()) avocado-113.0/avocado/core/000077500000000000000000000000001513561306000154765ustar00rootroot00000000000000avocado-113.0/avocado/core/__init__.py000066400000000000000000000175501513561306000176170ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2013-2014 # Author: Lucas Meneghel Rodrigues import os import pkg_resources from avocado.core.dispatcher import InitDispatcher from avocado.core.settings import settings as stgs from avocado.core.streams import BUILTIN_STREAM_SETS, BUILTIN_STREAMS_DESCRIPTION from avocado.core.utils.path import prepend_base_path def register_core_options(): streams = [ f'"{stream}": {description}' for stream, description in BUILTIN_STREAMS_DESCRIPTION.items() ] + [ f'"{stream}": {description}' for stream, description in BUILTIN_STREAM_SETS.items() ] streams = "; ".join(streams) help_msg = ( f"List of comma separated builtin logs, or logging streams " f"optionally followed by LEVEL (DEBUG,INFO,...). Builtin " f"streams are: {streams}. By default: 'app'" ) stgs.register_option( section="core", key="show", key_type=lambda x: set(x.split(",")), metavar="STREAM[:LEVEL]", nargs="?", default=set(["app"]), help_msg=help_msg, ) help_msg = ( "The amount of time to wait between asking nicely for a task " "to be terminated (say sending a signal) and proceeding with " "a more forceful termination. This may allow runners within " "tasks to perform clean ups. Spawners are free to implement " "a behavior that is suitable to their isolation model, " "including ignoring this configuration." ) stgs.register_option( section="runner.task.interval", key="from_soft_to_hard_termination", key_type=int, help_msg=help_msg, default=1, ) help_msg = ( "The amount of time to wait between executing a more forceful " "termination of a task, and the verification of the actual " "termination. This may allow spawners to give the necessary " "time for their isolation models to fully terminate a task. " "Spawners are free to implement a behavior that is suitable " "to their isolation model, including ignoring this " "configuration." ) stgs.register_option( section="runner.task.interval", key="from_hard_termination_to_verification", key_type=int, help_msg=help_msg, default=0, ) # Let's assume that by default, cache will be located under the user's # umbrella. This will make it easy for our deployments and it is a common # place for other applications too. help_msg = "Cache directories to be used by the avocado test" default = prepend_base_path("~/avocado/data/cache") stgs.register_option( section="datadir.paths", key="cache_dirs", key_type=list, default=[default], help_msg=help_msg, ) help_msg = "Base directory for Avocado tests and auxiliary data" default = prepend_base_path("~/avocado") stgs.register_option( section="datadir.paths", key="base_dir", key_type=prepend_base_path, default=default, help_msg=help_msg, ) help_msg = "Test directory for Avocado tests" default = prepend_base_path("/usr/share/doc/avocado/tests") stgs.register_option( section="datadir.paths", key="test_dir", key_type=prepend_base_path, default=default, help_msg=help_msg, ) help_msg = "Data directory for Avocado" default = prepend_base_path("~/avocado/data") stgs.register_option( section="datadir.paths", key="data_dir", key_type=prepend_base_path, default=default, help_msg=help_msg, ) help_msg = "Logs directory for Avocado" default = prepend_base_path("~/avocado/job-results") stgs.register_option( section="datadir.paths", key="logs_dir", key_type=prepend_base_path, default=default, help_msg=help_msg, ) help_msg = "Whether to display colored output in terminals that support it" stgs.register_option( section="runner.output", key="colored", key_type=bool, default=True, help_msg=help_msg, ) help_msg = ( "Whether to force colored output to non-tty outputs " "(e.g. log files). Allowed values: auto, always, never" ) stgs.register_option( section="runner.output", key="color", default="auto", help_msg=help_msg ) help_msg = ( "Suppress notification about broken plugins in the app " "standard error. Add the name of each broken plugin you " "want to suppress the notification in the list. (e.g. " '"avocado_result_html")' ) stgs.register_option( section="plugins", key="skip_broken_plugin_notification", key_type=list, default=[], help_msg=help_msg, ) # All settings starting with 'runner.' will be passed to runner # exec-test runner config help_msg = ( "Use a custom exit code list to consider a test as skipped. " "This is only used by exec-test runners. Default is []." ) stgs.register_option( section="runner.exectest.exitcodes", key="skip", default=[], key_type=list, help_msg=help_msg, ) help_msg = ( "If and how to clear test environment. Two possible options, " "`system` and `all`. " "`system` = no variables set, besides avocados default and kwargs. " "`all` = only kwargs are set." ) stgs.register_option( section="runner.exectest", key="clear_env", key_type=str, default=None, help_msg=help_msg, ) help_msg = ( "By default Avocado runners will use the {uri} of a test as " "its identifier. Use a custom f-string identifier in order to " "change it." ) stgs.register_option( section="runner", key="identifier_format", default="{uri}", help_msg=help_msg ) help_msg = "List of test references (aliases or paths)" stgs.register_option( section="resolver", key="references", key_type=list, default=[], help_msg=help_msg, ) help_msg = ( "Selects the runner implementation from one of the " "installed and active implementations. You can run " '"avocado plugins" and find the list of valid runners ' 'under the "Plugins that run test suites on a job ' '(runners)" section. Defaults to "nrunner", which is ' "the only runner that ships with core Avocado at this moment." ) stgs.register_option( section="run", key="suite_runner", default="nrunner", help_msg=help_msg ) def initialize_plugin_infrastructure(): help_msg = "Plugins that will not be loaded and executed" stgs.register_option( section="plugins", key="disable", key_type=list, default=[], help_msg=help_msg ) kinds = list(pkg_resources.get_entry_map("avocado-framework").keys()) plugin_types = [kind[8:] for kind in kinds if kind.startswith("avocado.plugins.")] for plugin_type in plugin_types: help_msg = f'Execution order for "{plugin_type}" plugins' stgs.register_option( section=plugin_type, key="order", key_type=list, default=[], help_msg=help_msg, ) def initialize_plugins(): initialize_plugin_infrastructure() InitDispatcher().map_method("initialize") avocado-113.0/avocado/core/app.py000066400000000000000000000076611513561306000166420ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2013-2014 # Author: Lucas Meneghel Rodrigues """ The core Avocado application. """ import os import signal import sys from avocado.core import output from avocado.core.dispatcher import CLICmdDispatcher, CLIDispatcher from avocado.core.output import STD_OUTPUT from avocado.core.parser import Parser from avocado.core.settings import settings from avocado.utils import process class AvocadoApp: """ Avocado application. """ def __init__(self): # Catch all libc runtime errors to STDERR os.environ["LIBC_FATAL_STDERR_"] = "1" self._cli_dispatcher = None self._cli_cmd_dispatcher = None self._setup_signals() self.parser = Parser() self.parser.start() output.early_start() show = getattr(self.parser.args, "core.show") reconfigure_settings = {"core.paginator": False, "core.show": show} try: self._load_cli_plugins() self._configure_cli_plugins() self.parser.finish() settings.merge_with_configs() settings.merge_with_arguments(self.parser.config) self.parser.config.update(settings.as_dict()) self._run_cli_plugins() except SystemExit as detail: # If someone tries to exit Avocado, we should first close the # STD_OUTPUT and only then exit. output.reconfigure(reconfigure_settings) STD_OUTPUT.close() sys.exit(detail.code) except: # For any other exception we also need to close the STD_OUTPUT. output.reconfigure(reconfigure_settings) STD_OUTPUT.close() raise else: # In case of no exceptions, we just reconfigure the output. output.reconfigure(self.parser.config) def _load_cli_plugins(self): self._cli_dispatcher = CLIDispatcher() self._cli_cmd_dispatcher = CLICmdDispatcher() output.log_plugin_failures( self._cli_dispatcher.load_failures + self._cli_cmd_dispatcher.load_failures ) def _configure_cli_plugins(self): if self._cli_cmd_dispatcher.extensions: self._cli_cmd_dispatcher.map_method("configure", self.parser) if self._cli_dispatcher.extensions: self._cli_dispatcher.map_method("configure", self.parser) def _run_cli_plugins(self): if self._cli_dispatcher.extensions: self._cli_dispatcher.map_method("run", self.parser.config) @staticmethod def _setup_signals(): def sigterm_handler(signum, frame): # pylint: disable=W0613 children = process.get_children_pids(os.getpid()) for child in children: process.kill_process_tree(int(child)) raise SystemExit("Terminated") signal.signal(signal.SIGTERM, sigterm_handler) if hasattr(signal, "SIGTSTP"): signal.signal(signal.SIGTSTP, signal.SIG_IGN) # ignore ctrl+z def run(self): try: try: subcmd = self.parser.config.get("subcommand") extension = self._cli_cmd_dispatcher[subcmd] except KeyError: return method = extension.obj.run return method(self.parser.config) finally: # This makes sure we cleanup the console (stty echo). The only way # to avoid cleaning it is to kill the less (paginator) directly STD_OUTPUT.close() avocado-113.0/avocado/core/data_dir.py000066400000000000000000000275601513561306000176310ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2013-2015 # Author: Lucas Meneghel Rodrigues """ Library used to let avocado tests find important paths in the system. The general reasoning to find paths is: * When running in tree, don't honor avocado.conf. Also, we get to run/display the example tests shipped in tree. * When avocado.conf is in /etc/avocado, or ~/.config/avocado, then honor the values there as much as possible. If they point to a location where we can't write to, use the next best location available. * The next best location is the default system wide one. * The next best location is the default user specific one. """ import atexit import glob import os import shutil import sys import tempfile import time import warnings from avocado.core import exit_codes, job_id from avocado.core.output import LOG_JOB, LOG_UI from avocado.core.settings import settings from avocado.utils import path as utils_path from avocado.utils.data_structures import Borg if "VIRTUAL_ENV" in os.environ: SYSTEM_BASE_DIR = os.environ["VIRTUAL_ENV"] USER_BASE_DIR = SYSTEM_BASE_DIR else: SYSTEM_BASE_DIR = "/var/lib/avocado" USER_BASE_DIR = os.path.expanduser("~/avocado") SYSTEM_TEST_DIR = os.path.join(SYSTEM_BASE_DIR, "tests") SYSTEM_DATA_DIR = os.path.join(SYSTEM_BASE_DIR, "data") SYSTEM_LOG_DIR = os.path.join(SYSTEM_BASE_DIR, "job-results") USER_TEST_DIR = os.path.join(USER_BASE_DIR, "tests") USER_DATA_DIR = os.path.join(USER_BASE_DIR, "data") USER_LOG_DIR = os.path.join(USER_BASE_DIR, "job-results") def _get_settings_dir(dir_name): """ Returns a given "datadir" directory as set by the configuration system """ namespace = f"datadir.paths.{dir_name}" path = settings.as_dict().get(namespace) return os.path.abspath(path) def _get_rw_dir(settings_location, system_location, user_location): if utils_path.usable_rw_dir(settings_location): return settings_location if utils_path.usable_rw_dir(system_location): return system_location user_location = os.path.expanduser(user_location) if utils_path.usable_rw_dir(user_location): return user_location def get_base_dir(): """ Get the most appropriate base dir. The base dir is the parent location for most of the avocado other important directories. Examples: * Log directory * Data directory * Tests directory """ warnings.warn( ( "get_base_dir() is deprecated, get values from " "settings.as_dict() or self.config" ), DeprecationWarning, ) return settings.as_dict().get("datadir.paths.base_dir") def get_test_dir(): """ Get the most appropriate test location. The test location is where we store tests written with the avocado API. The heuristics used to determine the test dir are: 1) If an explicit test dir is set in the configuration system, it is used. 2) If user is running Avocado from its source code tree, the example test dir is used. 3) System wide test dir is used. 4) User default test dir (~/avocado/tests) is used. """ configured = _get_settings_dir("test_dir") if utils_path.usable_ro_dir(configured): return configured source_tree_root = os.path.dirname(os.path.dirname(os.path.dirname(__file__))) if os.path.exists(os.path.join(source_tree_root, "examples")): return os.path.join(source_tree_root, "examples", "tests") if utils_path.usable_ro_dir(SYSTEM_TEST_DIR): return SYSTEM_TEST_DIR if utils_path.usable_rw_dir(USER_TEST_DIR): return USER_TEST_DIR def get_data_dir(): """ Get the most appropriate data dir location. The data dir is the location where any data necessary to job and test operations are located. Examples: * ISO files * GPG files * VM images * Reference bitmaps .. warning:: This method is deprecated, get values from settings.as_dict() or self.config """ warnings.warn( ( "get_data_dir() is deprecated, get values from " "settings.as_dict() or self.config" ), DeprecationWarning, ) return settings.as_dict().get("datadir.paths.data_dir") def get_datafile_path(*args): """ Get a path relative to the data dir. :param args: Arguments passed to os.path.join. Ex ('images', 'jeos.qcow2') """ config = settings.as_dict() new_args = tuple([config.get("datadir.paths.data_dir")] + list(args)) return os.path.join(*new_args) def get_logs_dir(): """ Get the most appropriate log dir location. The log dir is where we store job/test logs in general. .. warning:: This method is deprecated, get values from settings.as_dict() or self.config """ warnings.warn( ( "get_logs_dir() is deprecated, get values from " "settings.as_dict() or self.config" ), DeprecationWarning, ) return settings.as_dict().get("datadir.paths.logs_dir") def create_job_logs_dir(base_dir=None, unique_id=None): """ Create a log directory for a job, or a stand alone execution of a test. :param base_dir: Base log directory, if `None`, use value from configuration. :param unique_id: The unique identification. If `None`, create one. :rtype: str """ start_time = time.strftime("%Y-%m-%dT%H.%M") if base_dir is None: base_dir = get_logs_dir() if not base_dir: LOG_UI.error( "No writable location for logs found, use " "'avocado config --datadir' to get the " "locations and check system permissions." ) sys.exit(exit_codes.AVOCADO_FAIL) if not os.path.exists(base_dir): utils_path.init_dir(base_dir) # Stand alone tests handling if unique_id is None: unique_id = job_id.create_unique_job_id() logdir = os.path.join(base_dir, f"job-{start_time}-{unique_id[:7]}") for i in range(7, len(unique_id)): try: os.mkdir(logdir) except OSError: logdir += unique_id[i] continue return logdir logdir += "." for i in range(1000): try: os.mkdir(logdir + str(i)) except OSError: continue return logdir + str(i) raise IOError(f"Unable to create unique logdir" f" in 1000 iterations: {logdir}") def get_cache_dirs(): """ Returns the list of cache dirs, according to configuration and convention. This will be deprecated. Please use settings.as_dict() or self.config. .. warning:: This method is deprecated, get values from settings.as_dict() or self.config """ warnings.warn( ( "get_cache_dirs() is deprecated, get values from " "settings.as_dict() or self.config" ), DeprecationWarning, ) return settings.as_dict().get("datadir.paths.cache_dirs") class _TmpDirTracker(Borg): def __init__(self): Borg.__init__(self) self.basedir = None def get(self, basedir): if not hasattr(self, "tmp_dir"): if basedir is not None: self.basedir = basedir self.tmp_dir = tempfile.mkdtemp( # pylint: disable=W0201 prefix="avocado_", dir=self.basedir ) elif basedir is not None and basedir != self.basedir: LOG_JOB.error( "The tmp_dir was already created. The new basedir " "you're trying to provide will have no effect." ) return self.tmp_dir def unittest_refresh_dir_tracker(self): """ This force-removes the tmp_dir and refreshes the tracker to create new """ if not hasattr(self, "tmp_dir"): return shutil.rmtree(self.__dict__.pop("tmp_dir")) def __del__(self): tmp_dir = getattr(self, "tmp_dir", None) if tmp_dir is not None and self.basedir is None: try: if os.path.isdir(tmp_dir): shutil.rmtree(tmp_dir) # Need catch AttributeError and TypeError. except (AttributeError, TypeError): pass _tmp_tracker = _TmpDirTracker() def get_tmp_dir(basedir=None): """ Get the most appropriate tmp dir location. The tmp dir is where artifacts produced by the test are kept. Examples: * Copies of a test suite source code * Compiled test suite source code """ tmp_dir = _tmp_tracker.get(basedir) # This assert is a security mechanism for avoiding re-creating # the temporary directory, since that's a security breach. msg = ( f"Temporary dir {tmp_dir} no longer exists. This likely means the " f"directory was incorrectly deleted before the end of the job" ) assert os.path.isdir(tmp_dir), msg return tmp_dir def clean_tmp_files(): """ Try to clean the tmp directory by removing it. This is a useful function for avocado entry points looking to clean after tests/jobs are done. If OSError is raised, silently ignore the error. """ try: tmp_dir = get_tmp_dir() except AssertionError: return try: shutil.rmtree(tmp_dir, ignore_errors=True) except OSError: pass def get_job_results_dir(job_ref, logs_dir=None): """ Get the job results directory from a job reference. :param job_ref: job reference, which can be: * an valid path to the job results directory. In this case it is checked if 'id' file exists * the path to 'id' file * the job id, which can be 'latest' * an partial job id :param logs_dir: path to base logs directory (optional), otherwise it uses the value from settings. """ # Check if job_ref is actually the path to either the job logs # directory itself or the job id file. path_ref = os.path.expanduser(job_ref) if os.path.isdir(path_ref): # The id file should exists otherwise it is not the expected # directory. if os.path.isfile(os.path.join(path_ref, "id")): return os.path.abspath(path_ref) return None elif os.path.isfile(path_ref): if os.path.basename(path_ref) == "id": return os.path.abspath(os.path.dirname(path_ref)) return None # At this point job_ref is expected to be an id (can be partial) or # the 'latest' symlink. # if logs_dir is None: logs_dir = get_logs_dir() else: logs_dir = os.path.expanduser(logs_dir) if job_ref == "latest": try: actual_dir = os.readlink(os.path.join(logs_dir, "latest")) return os.path.join(logs_dir, actual_dir) except IOError: return None matches = 0 short_jobid = job_ref[:7] if len(short_jobid) < 7: short_jobid += "*" idfile_pattern = os.path.join(logs_dir, f"job-*-{short_jobid}", "id") for id_file in glob.glob(idfile_pattern): with open(id_file, "r", encoding="utf-8") as fid: line = fid.read().strip("\n") if line.startswith(job_ref): match_file = id_file matches += 1 if matches > 1: raise ValueError(f"hash '{job_ref}' is not unique enough") if matches == 1: return os.path.dirname(match_file) return None atexit.register(clean_tmp_files) avocado-113.0/avocado/core/decorators.py000066400000000000000000000122021513561306000202120ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2017 # Author: Amador Pahim import types from functools import wraps from avocado.core import exceptions as core_exceptions def deco_factory(behavior, signal): """ Decorator factory. Returns a decorator used to signal the test when specified exception is raised. :param behavior: expected test result behavior. :param signal: delegating exception. """ def signal_on(exceptions=None): """ {0} the test when decorated function produces exception of the specified type. :param exceptions: Tuple or single exception to be assumed as test {1} [Exception]. :note: self.error, self.cancel and self.fail remain intact. :note: to allow simple usage param 'exceptions' must not be callable. """ func = False if exceptions is None: exceptions = Exception elif isinstance(exceptions, types.FunctionType): func = exceptions exceptions = Exception def decorate(func): @wraps(func) def wrapper(*args, **kwargs): try: return func(*args, **kwargs) except core_exceptions.TestBaseException as exc: raise exc except exceptions as details: raise signal(repr(details)) from details return wrapper return decorate(func) if func else decorate name = f"{behavior}_on" signal_on.__name__ = signal_on.__qualname__ = name signal_on.__doc__ = signal_on.__doc__.format( behavior.capitalize(), behavior.upper() ) return signal_on fail_on = deco_factory("fail", core_exceptions.TestFail) cancel_on = deco_factory("cancel", core_exceptions.TestCancel) def _skip_method_decorator(function, message, condition, negate): """Creates a skip decorator for a method.""" @wraps(function) def wrapper(obj, *args, **kwargs): # pylint: disable=W0613 if negate: if callable(condition): if not condition(obj): raise core_exceptions.TestSkipError(message) else: if not condition: raise core_exceptions.TestSkipError(message) else: if callable(condition): if condition(obj): raise core_exceptions.TestSkipError(message) else: if condition: raise core_exceptions.TestSkipError(message) return function(obj, *args, **kwargs) wrapper.__skip_test_condition__ = condition wrapper.__skip_test_condition_negate__ = negate return wrapper def _skip_class_decorator(cls, message, condition, negate): """Creates a skip decorator for a class.""" for key in cls.__dict__: if key.startswith("test") and callable(getattr(cls, key)): wrapped = _skip_method_decorator( getattr(cls, key), message, condition, negate ) setattr(cls, key, wrapped) return cls def _get_skip_method_or_class_decorator(message, condition, negate): """Returns a method or class decorator, according to decorated type.""" def decorator(obj): if isinstance(obj, type): return _skip_class_decorator(obj, message, condition, negate) else: return _skip_method_decorator(obj, message, condition, negate) return decorator def skip(message=None): """ Decorator to skip a test. :param message: the message given when the test is skipped :type message: str """ return _get_skip_method_or_class_decorator(message, True, False) def skipIf(condition, message=None): """ Decorator to skip a test if a condition is True. :param condition: a condition that will be evaluated as either True or False, if it's a callable, it will be called with the class instance as a parameter :type condition: bool or callable :param message: the message given when the test is skipped :type message: str """ return _get_skip_method_or_class_decorator(message, condition, False) def skipUnless(condition, message=None): """ Decorator to skip a test if a condition is False. :param condition: a condition that will be evaluated as either True or False, if it's a callable, it will be called with the class instance as a parameter :type condition: bool or callable :param message: the message given when the test is skipped :type message: str """ return _get_skip_method_or_class_decorator(message, condition, True) avocado-113.0/avocado/core/dependencies/000077500000000000000000000000001513561306000201245ustar00rootroot00000000000000avocado-113.0/avocado/core/dependencies/__init__.py000066400000000000000000000000001513561306000222230ustar00rootroot00000000000000avocado-113.0/avocado/core/dependencies/dependency.py000066400000000000000000000031601513561306000226140ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2024 # Authors: Jan Richter class Dependency: """ Data holder for dependency. """ def __init__(self, kind=None, uri=None, args=(), kwargs=None): self._kind = kind self._uri = uri self._args = args self._kwargs = kwargs or {} @property def kind(self): return self._kind @property def uri(self): return self._uri @property def args(self): return self._args @property def kwargs(self): return self._kwargs def __hash__(self): return hash( ( self.kind, self.uri, tuple(sorted(self.args)), tuple(sorted(self.kwargs.items())), ) ) def __eq__(self, other): if isinstance(other, Dependency): return hash(self) == hash(other) return False @classmethod def from_dictionary(cls, dictionary): return cls( dictionary.pop("type", None), dictionary.pop("uri", None), dictionary.pop("args", ()), dictionary, ) avocado-113.0/avocado/core/dependencies/requirements/000077500000000000000000000000001513561306000226475ustar00rootroot00000000000000avocado-113.0/avocado/core/dependencies/requirements/__init__.py000066400000000000000000000000001513561306000247460ustar00rootroot00000000000000avocado-113.0/avocado/core/dependencies/requirements/cache/000077500000000000000000000000001513561306000237125ustar00rootroot00000000000000avocado-113.0/avocado/core/dependencies/requirements/cache/__init__.py000066400000000000000000000005441513561306000260260ustar00rootroot00000000000000# The sqlite based backend is the only implementation from avocado.core.dependencies.requirements.cache.backends.sqlite import ( delete_environment, delete_requirement, get_all_environments_with_requirement, is_environment_prepared, is_requirement_in_cache, set_requirement, update_environment, update_requirement_status, ) avocado-113.0/avocado/core/dependencies/requirements/cache/backends/000077500000000000000000000000001513561306000254645ustar00rootroot00000000000000avocado-113.0/avocado/core/dependencies/requirements/cache/backends/__init__.py000066400000000000000000000000001513561306000275630ustar00rootroot00000000000000avocado-113.0/avocado/core/dependencies/requirements/cache/backends/sqlite.py000066400000000000000000000266701513561306000273520ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2020 # Authors: Cleber Rosa """ Test requirements module. """ import os import sqlite3 from avocado.core.data_dir import get_datafile_path #: The location of the requirements cache database CACHE_DATABASE_PATH = get_datafile_path("cache", "requirements.sqlite") sqlite3.register_adapter(bool, int) sqlite3.register_converter("BOOLEAN", lambda v: bool(int(v))) #: The definition of the database schema SCHEMA = [ "CREATE TABLE IF NOT EXISTS requirement_type (requirement_type TEXT UNIQUE)", "CREATE TABLE IF NOT EXISTS environment_type (environment_type TEXT UNIQUE)", ( "CREATE TABLE IF NOT EXISTS environment (" "environment_type TEXT," "environment TEXT," "FOREIGN KEY(environment_type) REFERENCES " "environment_type(environment_type)" ")" ), ( "CREATE UNIQUE INDEX IF NOT EXISTS " "environment_idx ON environment (environment, environment_type)" ), ( "CREATE TABLE IF NOT EXISTS requirement (" "environment_type TEXT," "environment TEXT," "requirement_type TEXT," "requirement TEXT," "saved BOOLEAN," "FOREIGN KEY(environment_type) REFERENCES environment(environment_type)," "FOREIGN KEY(environment) REFERENCES environment(environment)," "FOREIGN KEY(requirement_type) REFERENCES requirement_type(requirement_type)" ")" ), ( "CREATE UNIQUE INDEX IF NOT EXISTS requirement_idx ON requirement " "(environment_type, environment, requirement_type, requirement)" ), ] def _create_requirement_cache_db(): os.makedirs(os.path.dirname(CACHE_DATABASE_PATH), exist_ok=True) with sqlite3.connect(CACHE_DATABASE_PATH) as conn: cursor = conn.cursor() for entry in SCHEMA: _ = cursor.execute(entry) conn.commit() def set_requirement( environment_type, environment, requirement_type, requirement, saved=True ): if not os.path.exists(CACHE_DATABASE_PATH): _create_requirement_cache_db() with sqlite3.connect(CACHE_DATABASE_PATH) as conn: cursor = conn.cursor() sql = "INSERT OR IGNORE INTO environment_type VALUES (?)" cursor.execute(sql, (environment_type,)) sql = "INSERT OR IGNORE INTO environment VALUES (?, ?)" cursor.execute(sql, (environment_type, environment)) sql = "INSERT OR IGNORE INTO requirement_type VALUES (?)" cursor.execute(sql, (requirement_type,)) sql = "INSERT OR IGNORE INTO requirement VALUES (?, ?, ?, ?, ?)" cursor.execute( sql, (environment_type, environment, requirement_type, requirement, saved) ) conn.commit() def is_requirement_in_cache( environment_type, environment, requirement_type, requirement ): """Checks if requirement is in cache. :rtype: True if requirement is in cache False if requirement is not in cache None if requirement is in cache but it is not saved yet. """ if not os.path.exists(CACHE_DATABASE_PATH): return False sql = ( "SELECT r.saved FROM requirement r WHERE (" "environment_type = ? AND " "environment = ? AND " "requirement_type = ? AND " "requirement = ?)" ) with sqlite3.connect(CACHE_DATABASE_PATH) as conn: cursor = conn.cursor() result = cursor.execute( sql, (environment_type, environment, requirement_type, requirement) ) row = result.fetchone() if row is not None: if row[0]: return True return None return False def is_environment_prepared(environment): """Checks if environment has all requirements saved.""" if not os.path.exists(CACHE_DATABASE_PATH): return False sql = ( "SELECT COUNT(*) FROM requirement r JOIN " "environment e ON e.environment = r.environment " "WHERE (r.environment = ? AND " "r.saved = 0)" ) with sqlite3.connect( CACHE_DATABASE_PATH, detect_types=sqlite3.PARSE_DECLTYPES ) as conn: cursor = conn.cursor() result = cursor.execute(sql, (environment,)) row = result.fetchone() if row is not None: return row[0] == 0 return False def update_environment(environment_type, old_environment, new_environment): """Updates environment information for each requirement in one environment. It will remove the old environment and add the new one to the cache. :param environment_type: Type of fetched environment :type environment_type: str :param old_environment: Environment which should be updated :type environment: str :param new_environment: Environment, which will be a reimbursement for the old one. :type environment: str """ if not os.path.exists(CACHE_DATABASE_PATH): return False with sqlite3.connect(CACHE_DATABASE_PATH) as conn: cursor = conn.cursor() sql = "INSERT OR IGNORE INTO environment VALUES (?, ?)" cursor.execute(sql, (environment_type, new_environment)) sql = ( "UPDATE requirement SET environment = ? WHERE (" "environment_type = ? AND " "environment = ? )" ) cursor.execute(sql, (new_environment, environment_type, old_environment)) sql = ( "DELETE FROM environment WHERE (" "environment_type = ? AND " "environment = ? )" ) cursor.execute(sql, (environment_type, old_environment)) conn.commit() def update_requirement_status( environment_type, environment, requirement_type, requirement, new_status ): """Updates status of selected requirement in cache. The status has two values, save=True or not_save=False. :param environment_type: Type of fetched environment :type environment_type: str :param environment: Environment where the requirement is :type environment: str :param requirement_type: Type of the requirement in environment :type requirement_type: str :param requirement: Name of requirement :type requirement: str :param new_status: Requirement status which will be updated :type new_status: bool """ if not os.path.exists(CACHE_DATABASE_PATH): return False sql = ( "UPDATE requirement SET saved = ? WHERE (" "environment_type = ? AND " "environment = ? AND " "requirement_type = ? AND " "requirement = ?)" ) with sqlite3.connect(CACHE_DATABASE_PATH) as conn: cursor = conn.cursor() cursor.execute( sql, (new_status, environment_type, environment, requirement_type, requirement), ) conn.commit() return True def delete_environment(environment_type, environment): """Deletes environment with all its requirements from cache. :param environment_type: Type of environment :type environment_type: str :param environment: Environment which will be deleted :type environment: str """ if not os.path.exists(CACHE_DATABASE_PATH): return False with sqlite3.connect(CACHE_DATABASE_PATH) as conn: sql = ( "DELETE FROM requirement WHERE (" "environment_type = ? AND " "environment = ? )" ) cursor = conn.cursor() cursor.execute(sql, (environment_type, environment)) sql = ( "DELETE FROM environment WHERE (" "environment_type = ? AND " "environment = ? )" ) cursor.execute(sql, (environment_type, environment)) conn.commit() def delete_requirement(environment_type, environment, requirement_type, requirement): """Deletes requirement from cache. :param environment_type: Type of environment :type environment_type: str :param environment: Environment where the requirement is. :type environment: str :param requirement_type: Type of the requirement in environment :type requirement_type: str :param requirement: Name of requirement which will be deleted :type requirement: str """ if not os.path.exists(CACHE_DATABASE_PATH): return False with sqlite3.connect(CACHE_DATABASE_PATH) as conn: sql = ( "DELETE FROM requirement WHERE (" "environment_type = ? AND " "environment = ? AND " "requirement_type = ? AND " "requirement = ?)" ) cursor = conn.cursor() cursor.execute( sql, (environment_type, environment, requirement_type, requirement) ) conn.commit() def get_all_environments_with_requirement( environment_type, requirement_type, requirement ): """Fetches all environments with selected requirement from cache. :param environment_type: Type of fetched environment :type environment_type: str :param requirement_type: Type of the requirement in environment :type requirement_type: str :param requirement: Name of requirement :type requirement: str :return: Dict with all environments which has selected requirements. """ requirements = {} if not os.path.exists(CACHE_DATABASE_PATH): return requirements environment_select = ( "SELECT e.environment FROM requirement r JOIN " "environment e ON e.environment = r.environment " "WHERE (r.environment_type = ? AND " "r.requirement_type = ? AND " "r.requirement = ?)" ) sql = ( f"SELECT r.environment, r.requirement_type, r.requirement " f"FROM requirement AS r, ({environment_select}) AS e " f"WHERE r.environment = e.environment" ) with sqlite3.connect( CACHE_DATABASE_PATH, detect_types=sqlite3.PARSE_DECLTYPES ) as conn: cursor = conn.cursor() result = cursor.execute(sql, (environment_type, requirement_type, requirement)) for row in result.fetchall(): if row[0] in requirements: requirements[row[0]].append((row[1], row[2])) else: requirements[row[0]] = [(row[1], row[2])] return requirements def get_all_requirements(): """Fetches all requirements from database. :return: Dict with all environments which has requirements. """ requirements = {} if not os.path.exists(CACHE_DATABASE_PATH): return requirements sql = "SELECT * FROM requirement" with sqlite3.connect( CACHE_DATABASE_PATH, detect_types=sqlite3.PARSE_DECLTYPES ) as conn: cursor = conn.cursor() result = cursor.execute(sql) for row in result.fetchall(): environment_type = row[0] if environment_type not in requirements: requirements[environment_type] = [] requirements[environment_type].append( { "environment": row[1], "requirement_type": row[2], "requirement": row[3], } ) return requirements avocado-113.0/avocado/core/dispatcher.py000066400000000000000000000152351513561306000202040ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2015 # Author: Cleber Rosa """ Extensions/plugins dispatchers Besides the dispatchers listed here, there's also a lower level dispatcher that these depend upon: :class:`avocado.core.settings_dispatcher.SettingsDispatcher` """ import inspect import sys from avocado.core.enabled_extension_manager import EnabledExtensionManager def get_dispatchers(module_name): """Returns the classes that implement plugin dispatching These should inherit from the *ExtensionManager* base classes and contain suitable descriptions. The produced values are tuples that contain the dispatcher class and two booleans that indicates whether the configuration and job is needed to instantiate the class. """ module = sys.modules[module_name] for _, klass in inspect.getmembers(module): if ( inspect.isclass(klass) and issubclass(klass, EnabledExtensionManager) and hasattr(klass, "PLUGIN_DESCRIPTION") ): params = list(inspect.signature(klass.__init__).parameters) if len(params) == 1: yield (klass, False, False) elif len(params) == 2 and params[1] == "config": yield (klass, True, False) elif len(params) == 3 and params[1] == "config" and params[2] == "job": yield (klass, True, True) class CLIDispatcher(EnabledExtensionManager): """ Calls extensions on configure/run Automatically adds all the extension with entry points registered under 'avocado.plugins.cli' """ PLUGIN_DESCRIPTION = "Plugins that add new options to commands (cli)" def __init__(self): super().__init__("avocado.plugins.cli") class CLICmdDispatcher(EnabledExtensionManager): """ Calls extensions on configure/run Automatically adds all the extension with entry points registered under 'avocado.plugins.cli.cmd' """ PLUGIN_DESCRIPTION = "Plugins that add new commands (cli.cmd)" def __init__(self): super().__init__("avocado.plugins.cli.cmd") class JobPrePostDispatcher(EnabledExtensionManager): """ Calls extensions before Job execution Automatically adds all the extension with entry points registered under 'avocado.plugins.job.prepost' """ PLUGIN_DESCRIPTION = ( "Plugins that run before/after the execution of jobs (job.prepost)" ) def __init__(self): super().__init__("avocado.plugins.job.prepost") class TestPreDispatcher(EnabledExtensionManager): """ Calls extensions before Test execution Automatically adds all the extension with entry points registered under 'avocado.plugins.test.pre' """ PLUGIN_DESCRIPTION = "Plugins that run before the execution of each test (test.pre)" def __init__(self): super().__init__("avocado.plugins.test.pre") class TestPostDispatcher(EnabledExtensionManager): """ Calls extensions after Test execution Automatically adds all the extension with entry points registered under 'avocado.plugins.test.post' """ PLUGIN_DESCRIPTION = "Plugins that run after the execution of each test (test.post)" def __init__(self): super().__init__("avocado.plugins.test.post") class ResultDispatcher(EnabledExtensionManager): PLUGIN_DESCRIPTION = ( "Plugins that generate job result in different formats (result)" ) def __init__(self): super().__init__("avocado.plugins.result") class ResultEventsDispatcher(EnabledExtensionManager): PLUGIN_DESCRIPTION = ( "Plugins that generate job result based on job/test events (result_events)" ) def __init__(self, config): super().__init__( "avocado.plugins.result_events", invoke_kwds={"config": config} ) class VarianterDispatcher(EnabledExtensionManager): PLUGIN_DESCRIPTION = "Plugins that generate test variants (varianter)" def __init__(self): super().__init__("avocado.plugins.varianter") def __getstate__(self): """ Very fragile pickle which works when all Varianter plugins are available on both machines. TODO: Replace this with per-plugin-refresh-mechanism """ return {"extensions": getattr(self, "extensions")} def __setstate__(self, state): """ Very fragile pickle which works when all Varianter plugins are available on both machines. TODO: Replace this with per-plugin-refresh-mechanism """ self.__init__() self.extensions = state.get("extensions") def map_method_with_return(self, method_name, *args, **kwargs): return super().map_method_with_return( method_name, deepcopy=False, *args, **kwargs ) def map_method_with_return_copy(self, method_name, *args, **kwargs): """ The same as map_method_with_return, but use copy.deepcopy on each passed arg """ return super().map_method_with_return( method_name, deepcopy=True, *args, **kwargs ) class SuiteRunnerDispatcher(EnabledExtensionManager): PLUGIN_DESCRIPTION = "Plugins that run test suites on a job (suite.runner)" def __init__(self): super().__init__("avocado.plugins.suite.runner") class InitDispatcher(EnabledExtensionManager): PLUGIN_DESCRIPTION = "Plugins that always need to be initialized (init)" def __init__(self): super().__init__("avocado.plugins.init") class SpawnerDispatcher(EnabledExtensionManager): PLUGIN_DESCRIPTION = ( "Plugins that spawn tasks and know about their status (spawner)" ) def __init__(self, config=None, job=None): super().__init__( "avocado.plugins.spawner", invoke_kwds={"job": job, "config": config} ) class RunnableRunnerDispatcher(EnabledExtensionManager): PLUGIN_DESCRIPTION = ( "Plugins that run runnables (under a task and spawner) (runnable.runner)" ) def __init__(self): super().__init__("avocado.plugins.runnable.runner") class CacheDispatcher(EnabledExtensionManager): PLUGIN_DESCRIPTION = "Plugins that manipulates with avocado cache (cache)" def __init__(self): super().__init__("avocado.plugins.cache") avocado-113.0/avocado/core/enabled_extension_manager.py000066400000000000000000000032701513561306000232320ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2015-2019 # Author: Cleber Rosa """ Extension manager with disable/ordering support """ from avocado.core.extension_manager import ExtensionManager from avocado.core.settings import settings class EnabledExtensionManager(ExtensionManager): def __init__(self, namespace, invoke_kwds=None): super().__init__(namespace, invoke_kwds) namespace = f"{self.settings_section()}.order" configured_order = settings.as_dict().get(namespace) ordered = [] if configured_order: for name in configured_order: for ext in self.extensions: if name == ext.name: ordered.append(ext) for ext in self.extensions: if ext not in ordered: ordered.append(ext) self.extensions = ordered def enabled(self, extension): """ Checks configuration for explicit mention of plugin in a disable list If configuration section or key doesn't exist, it means no plugin is disabled. """ disabled = settings.as_dict().get("plugins.disable") return self.fully_qualified_name(extension) not in disabled avocado-113.0/avocado/core/exceptions.py000066400000000000000000000073351513561306000202410ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2013-2014 # Author: Lucas Meneghel Rodrigues """ Exception classes, useful for tests, and other parts of the framework code. """ class JobBaseException(Exception): """ The parent of all job exceptions. It should never be raised directly. """ class JobError(JobBaseException): """ A generic error happened during a job execution. """ class JobTestSuiteError(JobBaseException): """ Generic error happened during the creation of a job's test suite """ class JobTestSuiteEmptyError(JobTestSuiteError): """ Error raised when the creation of a test suite results in an empty suite """ class JobTestSuiteDuplicateNameError(JobTestSuiteError): """ Error raised when a test suite name is not unique in a job """ class JobTestSuiteReferenceResolutionError(JobTestSuiteError): """ Test References did not produce a valid reference by any resolver """ class JobFailFast(JobBaseException): """ Indicates that the test has failed because failfast is enabled. Should be thrown when a test has failed and failfast is enabled. This will indicate that other tests will be skipped. """ class OptionValidationError(Exception): """ An invalid option was passed to the test runner """ class TestBaseException(Exception): """ The parent of all test exceptions. You should be never raising this, but just in case, we'll set its status' as FAIL. """ status = "FAIL" class TestSetupFail(TestBaseException): """ Indicates an error during a setup or cleanup procedure. """ status = "ERROR" class TestError(TestBaseException): """ Indicates that the test was not fully executed and an error happened. This is the sort of exception you raise if the test was partially executed and could not complete due to a setup, configuration, or another fatal condition. """ status = "ERROR" class TestAbortError(TestBaseException): """ Indicates that the test was prematurely aborted. """ status = "ERROR" class TestSkipError(TestBaseException): """ Indicates that the test is skipped. Should be thrown when various conditions are such that the test is inappropriate. For example, inappropriate architecture, wrong OS version, program being tested does not have the expected capability (older version). """ status = "SKIP" class TestFail(TestBaseException, AssertionError): """ Indicates that the test failed. TestFail inherits from AssertionError in order to keep compatibility with vanilla python unittests (they only consider failures the ones deriving from AssertionError). """ status = "FAIL" class TestWarn(TestBaseException): """ Indicates that bad things (may) have happened, but not an explicit failure. """ status = "WARN" class TestCancel(TestBaseException): """ Indicates that a test was canceled. Should be thrown when the cancel() test method is used. """ status = "CANCEL" class TestInterrupt(TestBaseException): """ Indicates that a test was interrupted. Should be thrown when the test method is interrupted by user or timeout. """ status = "INTERRUPTED" avocado-113.0/avocado/core/exit_codes.py000066400000000000000000000031541513561306000202010ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. """ Avocado exit codes. These codes are returned on the command line and may be used by applications that interface (that is, run) the Avocado command line application. Besides main status about the execution of the command line application, these exit status may also give extra, although limited, information about test statuses. """ #: Both job and tests PASSed AVOCADO_ALL_OK = 0x0000 #: Job went fine, but some tests FAILed or ERRORed AVOCADO_TESTS_FAIL = 0x0001 #: Something went wrong with an Avocado Job execution, usually by #: an explicit :class:`avocado.core.exceptions.JobError` exception. AVOCADO_JOB_FAIL = 0x0002 #: Something else went wrong and avocado failed (or crashed). Commonly #: used on command line validation errors. AVOCADO_FAIL = 0x0004 #: The job was explicitly interrupted. Usually this means that a user #: hit CTRL+C while the job was still running. AVOCADO_JOB_INTERRUPTED = 0x0008 #: Some internal avocado routine has not completed and probably it wasn't at #: the JOB level. Usually this means that something it was skipped but it is #: safe to ignore. AVOCADO_WARNING = 0x0016 #: Avocado generic crash AVOCADO_GENERIC_CRASH = -1 avocado-113.0/avocado/core/extension_manager.py000066400000000000000000000171461513561306000215670ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2015-2019 # Author: Cleber Rosa """ Base extension manager This is a mix of stevedore-like APIs and behavior, with Avocado's own look and feel. """ import copy import enum import logging import sys import pkg_resources from avocado.core.exceptions import JobBaseException from avocado.utils import stacktrace # This is also defined in avocado.core.output, but this avoids a # circular import LOG_UI = logging.getLogger("avocado.app") class PluginPriority(enum.IntEnum): VERY_HIGH = 100 HIGH = 70 NORMAL = 50 LOW = 30 VERY_LOW = 1 LAST_RESORT = 0 class Extension: """ This is a verbatim copy from the stevedore.extension class with the same name """ def __init__(self, name, entry_point, plugin, obj): self.name = name self.entry_point = entry_point self.plugin = plugin self.obj = obj self.priority = getattr(self.obj, "priority", PluginPriority.NORMAL) def __lt__(self, other): if self.priority == other.priority: return self.name < other.name return self.priority > other.priority def __le__(self, other): if self.priority == other.priority: return self.name <= other.name return self.priority >= other.priority def __ge__(self, other): if self.priority == other.priority: return self.name >= other.name return self.priority <= other.priority def __gt__(self, other): if self.priority == other.priority: return self.name > other.name return self.priority < other.priority class ExtensionManager: #: Default namespace prefix for Avocado extensions NAMESPACE_PREFIX = "avocado.plugins." def __init__(self, namespace, invoke_kwds=None): self.namespace = namespace self.extensions = [] self.load_failures = [] if invoke_kwds is None: invoke_kwds = {} # load plugins for ep in pkg_resources.iter_entry_points(self.namespace): try: plugin = ep.load() obj = plugin(**invoke_kwds) except ImportError as exception: self.load_failures.append((ep, exception)) else: ext = Extension(ep.name, ep, plugin, obj) if self.enabled(ext): # lgtm [py/init-calls-subclass] self.extensions.append(ext) self.extensions = sorted(self.extensions) def enabled(self, extension): # pylint: disable=W0613,R0201 """ Checks if a plugin is enabled Sub classes can change this implementation to determine their own criteria. """ return True def plugin_type(self): """ Subset of entry points namespace for this dispatcher Given an entry point `avocado.plugins.foo`, plugin type is `foo`. If entry point does not conform to the Avocado standard prefix, it's returned unchanged. """ if self.namespace.startswith(self.NAMESPACE_PREFIX): return self.namespace[len(self.NAMESPACE_PREFIX) :] else: return self.namespace def fully_qualified_name(self, extension): """ Returns the Avocado fully qualified plugin name :param extension: an Extension instance :type extension: :class:`Extension` """ return f"{self.plugin_type()}.{extension.entry_point.name}" def settings_section(self): """ Returns the config section name for the plugin type handled by itself """ return f"plugins.{self.plugin_type()}" def names(self): """ Returns the names of the discovered extensions This differs from :func:`stevedore.extension.ExtensionManager.names` in that it returns names in a predictable order, by using standard :func:`sorted`. """ return sorted(super().names()) def get_extentions_by_priority(self): """ Returns extensions in execution order """ return self.extensions def get_extentions_by_name(self): """ Returns extensions in alphabetical order """ return sorted(self.extensions, key=lambda x: x.name) def map_method_with_return(self, method_name, *args, **kwargs): """ The same as `map_method` but additionally reports the list of returned values and optionally deepcopies the passed arguments :param method_name: Name of the method to be called on each ext :param args: Arguments to be passed to all called functions :param kwargs: Key-word arguments to be passed to all called functions if `"deepcopy" == True` is present in kwargs the args and kwargs are deepcopied before passing it to each called function. """ deepcopy = kwargs.pop("deepcopy", False) ret = [] for ext in self.extensions: try: if hasattr(ext.obj, method_name): method = getattr(ext.obj, method_name) if deepcopy: copied_args = [copy.deepcopy(arg) for arg in args] copied_kwargs = copy.deepcopy(kwargs) ret.append(method(*copied_args, **copied_kwargs)) else: ret.append(method(*args, **kwargs)) except SystemExit: raise except KeyboardInterrupt: raise except JobBaseException: raise except: # catch any exception pylint: disable=W0702 LOG_UI.error( 'Error running method "%s" of plugin "%s": %s', method_name, ext.name, sys.exc_info()[1], ) stacktrace.log_exc_info(sys.exc_info(), logger=LOG_UI) return ret def map_method(self, method_name, *args): """ Maps method_name on each extension in case the extension has the attr :param method_name: Name of the method to be called on each ext :param args: Arguments to be passed to all called functions """ for ext in self.extensions: try: if hasattr(ext.obj, method_name): method = getattr(ext.obj, method_name) method(*args) except SystemExit: raise except KeyboardInterrupt: raise except JobBaseException: raise except: # catch any exception pylint: disable=W0702 LOG_UI.error( 'Error running method "%s" of plugin "%s": %s', method_name, ext.name, sys.exc_info()[1], ) stacktrace.log_exc_info(sys.exc_info(), logger=LOG_UI) def __getitem__(self, name): for ext in self.extensions: if ext.name == name: return ext raise KeyError def __iter__(self): return iter(self.extensions) avocado-113.0/avocado/core/job.py000066400000000000000000000634161513561306000166340ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2013-2015 # Authors: Lucas Meneghel Rodrigues # Ruda Moura """ Job module - describes a sequence of automated test operations. """ import datetime import logging import os import pprint import shutil import sys import tempfile import time import traceback import warnings from copy import deepcopy from avocado.core import ( data_dir, dispatcher, exceptions, exit_codes, jobdata, output, result, version, ) from avocado.core.job_id import create_unique_job_id from avocado.core.output import LOG_JOB, LOG_UI, split_loggers_and_levels from avocado.core.settings import settings from avocado.core.suite import TestSuite, TestSuiteError from avocado.core.utils.version import get_avocado_git_version from avocado.utils import astring from avocado.utils.data_structures import CallbackRegister, time_to_seconds _NEW_ISSUE_LINK = "https://github.com/avocado-framework/avocado/issues/new" def register_job_options(): """Register the few core options that the support the job operation.""" msg = ( "Sets the base log level of the output generated by the job, which " "is also the base logging level for the --show command line option. " "Any of the Python logging levels names are allowed here. Examples:" " DEBUG, INFO, WARNING, ERROR, CRITICAL. For more information refer" " to: https://docs.python.org/3/library/logging.html#levels" ) settings.register_option( section="job.output", key="loglevel", default="DEBUG", help_msg=msg ) help_msg = ( "Set the maximum amount of time (in SECONDS) that tests are " 'allowed to execute. Values <= zero means "no timeout". You ' "can also use suffixes, like: s (seconds), m (minutes), h " "(hours). " ) settings.register_option( section="job.run", key="timeout", default=0, key_type=time_to_seconds, help_msg=help_msg, ) help_msg = 'Store given logging STREAMs in "$JOB_RESULTS_DIR/$STREAM.$LEVEL".' settings.register_option( section="job.run", key="store_logging_stream", nargs="+", help_msg=help_msg, default=[], metavar="STREAM[:LEVEL]", key_type=list, ) help_msg = ( "Whether to add a layer of memory based buffering with a given number of " "entries. If set to 0 or less than 0, buffering disabled." ) settings.register_option( section="job.run", key="logging_buffer_size", help_msg=help_msg, default=0, metavar="SIZE", key_type=int, ) register_job_options() class Job: """A Job is a set of operations performed on a test machine. Most of the time, we are interested in simply running tests, along with setup operations and event recording. A job has multiple test suites attached to it. Please keep in mind that when creating jobs from the constructor (`Job()`), we are assuming that you would like to have control of the test suites and you are going to build your own TestSuites. If you would like any help to create the job's test_suites from the config provided, please use `Job.from_config()` method and we are going to do our best to create the test suites. So, basically, as described we have two "main ways" to create a job: 1. Automatic discovery, using `from_config()` method:: job = Job.from_config(job_config=job_config, suites_configs=[suite_cfg1, suite_cfg2]) 2. Manual or Custom discovery, using the constructor:: job = Job(config=config, test_suites=[suite1, suite2, suite3]) """ def __init__(self, config=None, test_suites=None): """Creates an instance of Job class. Note that `config` and `test_suites` are optional, if not passed you need to change this before running your tests. Otherwise nothing will run. If you need any help to create the test_suites from the config, then use the `Job.from_config()` method. :param config: the job configuration, usually set by command line options and argument parsing :type config: dict :param test_suites: A list with TestSuite objects. If is None the job will have an empty list and you can add suites after init accessing job.test_suites. :type test_suites: list """ self.config = settings.as_dict() if config: self.config.update(config) self.log = LOG_UI self.loglevel = self.config.get("job.output.loglevel") if self.config.get("run.dry_run.enabled"): # Modify config for dry-run unique_id = self.config.get("run.unique_job_id") if unique_id is None: self.config["run.unique_job_id"] = "0" * 40 self.config["sysinfo.collect.enabled"] = "off" self.test_suites = test_suites or [] self._check_test_suite_name_uniqueness() #: The log directory for this job, also known as the job results #: directory. If it's set to None, it means that the job results #: directory has not yet been created. self.logdir = None self.logfile = None self.tmpdir = None self.__keep_tmpdir = True self._base_tmpdir = None self.status = "RUNNING" self.result = None self.interrupted_reason = None self._timeout = None self._unique_id = None # The time at which the job has started or `None` if it has not been # started by means of the `run()` method. self._time_start = None # The time at which the job has finished or `None` if it has not been # started by means of the `run()` method. self._time_end = None #: The total amount of time the job took from start to finish, #: or None if it has not been started by means of the `run()` method self.time_elapsed = None self.funcatexit = CallbackRegister(f"JobExit {self.unique_id}", LOG_JOB) self._stdout_stderr = None self.replay_sourcejob = self.config.get("replay_sourcejob") self.exitcode = exit_codes.AVOCADO_ALL_OK self._result_events_dispatcher = None @property def result_events_dispatcher(self): # The result events dispatcher is shared with the test runner. # Because of our goal to support using the phases of a job # freely, let's get the result events dispatcher on first usage. # A future optimization may load it on demand. if self._result_events_dispatcher is None: self._result_events_dispatcher = dispatcher.ResultEventsDispatcher( self.config ) output.log_plugin_failures(self._result_events_dispatcher.load_failures) return self._result_events_dispatcher @property def test_results_path(self): return os.path.join(self.logdir, "test-results") def __enter__(self): self.setup() return self def __exit__(self, _exc_type, _exc_value, _traceback): self.cleanup() def __start_job_logging(self): # Enable test logger full_log = os.path.join(self.logdir, "full.log") fmt = "%(asctime)s %(name)s %(module)-16.16s L%(lineno)-.4d %(levelname)-5.5s| %(message)s" buffer_size = self.config.get("job.run.logging_buffer_size") output.add_log_handler( LOG_JOB, logging.FileHandler, self.logfile, self.loglevel, fmt, output.FilterTestMessage(), buffer_size, ) output.add_log_handler( logging.getLogger(""), logging.FileHandler, full_log, self.loglevel, fmt, output.FilterTestMessage(), buffer_size, ) output.add_log_handler( logging.getLogger(""), logging.FileHandler, full_log, self.loglevel, "", output.FilterTestMessageOnly(), buffer_size, ) # --store-logging-stream files store_loggers = set(self.config.get("job.run.store_logging_stream")) for enabled_logger, level in split_loggers_and_levels(store_loggers): if level: logfile = os.path.join( self.logdir, f"{enabled_logger}.{logging.getLevelName(level)}.log" ) else: level = logging.DEBUG logfile = os.path.join(self.logdir, f"{enabled_logger}.log") output.add_log_handler( enabled_logger, logging.FileHandler, logfile, level, fmt, output.FilterTestMessage(), buffer_size, ) output.add_log_handler( enabled_logger, logging.FileHandler, logfile, level, "", output.FilterTestMessageOnly(), buffer_size, ) def __stop_job_logging(self): output.del_last_configuration() def _log_avocado_config(self): LOG_JOB.info("Avocado config:") LOG_JOB.info("") for line in pprint.pformat(self.config).splitlines(): LOG_JOB.info(line) LOG_JOB.info("") def _log_avocado_datadir(self): LOG_JOB.info("Avocado Data Directories:") LOG_JOB.info("") LOG_JOB.info("base %s", self.config.get("datadir.paths.base_dir")) LOG_JOB.info("tests %s", data_dir.get_test_dir()) LOG_JOB.info("data %s", self.config.get("datadir.paths.data_dir")) LOG_JOB.info("logs %s", self.logdir) LOG_JOB.info("") @staticmethod def _log_avocado_version(): version_log = version.VERSION git_version = get_avocado_git_version() if git_version is not None: version_log += git_version LOG_JOB.info("Avocado version: %s", version_log) LOG_JOB.info("") @staticmethod def _log_cmdline(): cmdline = " ".join(sys.argv) LOG_JOB.info("Command line: %s", cmdline) LOG_JOB.info("") def _log_job_debug_info(self): """ Log relevant debug information to the job log. """ self._log_cmdline() self._log_avocado_version() self._log_avocado_config() self._log_avocado_datadir() for suite in self.test_suites: self._log_variants(suite.variants) self._log_tmp_dir() self._log_job_id() def _log_job_id(self): LOG_JOB.info("Job ID: %s", self.unique_id) if self.replay_sourcejob is not None: LOG_JOB.info("Replay of Job ID: %s", self.replay_sourcejob) LOG_JOB.info("") def _log_tmp_dir(self): LOG_JOB.info("Temporary dir: %s", self.tmpdir) LOG_JOB.info("") @staticmethod def _log_variants(variants): lines = variants.to_str(summary=1, variants=1, use_utf8=False) for line in lines.splitlines(): LOG_JOB.info(line) def _setup_job_category(self): """ This has to be called after self.logdir has been defined It attempts to create a directory one level up from the job results, with the given category name. Then, a symbolic link is created to this job results directory. This should allow a user to look at a single directory for all jobs of a given category. """ category = self.config.get("run.job_category") if category is None: return if category != astring.string_to_safe_path(category): msg = ( f"Unable to set category in job results: name is not " f"filesystem safe: {category}" ) LOG_UI.warning(msg) LOG_JOB.warning(msg) return # we could also get "base_logdir" from config, but I believe this is # the best choice because it reduces the dependency surface (depends # only on self.logdir) category_path = os.path.join(os.path.dirname(self.logdir), category) try: os.mkdir(category_path) except FileExistsError: pass try: os.symlink( os.path.relpath(self.logdir, category_path), os.path.join(category_path, os.path.basename(self.logdir)), ) except NotImplementedError: msg = f"Unable to link this job to category {category}" LOG_UI.warning(msg) LOG_JOB.warning(msg) except OSError: msg = f"Permission denied to link this job to category {category}" LOG_UI.warning(msg) LOG_JOB.warning(msg) def _setup_job_results(self): """ Prepares a job result directory, also known as logdir, for this job """ base_logdir = self.config.get("run.results_dir") if base_logdir is None: self.logdir = data_dir.create_job_logs_dir(unique_id=self.unique_id) else: base_logdir = os.path.abspath(base_logdir) self.logdir = data_dir.create_job_logs_dir( base_dir=base_logdir, unique_id=self.unique_id ) if not self.config.get("run.dry_run.enabled"): self._update_latest_link() self.logfile = os.path.join(self.logdir, "job.log") idfile = os.path.join(self.logdir, "id") with open(idfile, "w", encoding="utf-8") as id_file_obj: id_file_obj.write(f"{self.unique_id}\n") id_file_obj.flush() os.fsync(id_file_obj) def _update_latest_link(self): """ Update the latest job result symbolic link [avocado-logs-dir]/latest. """ def soft_abort(msg): """Only log the problem""" LOG_JOB.warning("Unable to update the latest link: %s", msg) basedir = os.path.dirname(self.logdir) basename = os.path.basename(self.logdir) proc_latest = os.path.join(basedir, f"latest.{os.getpid()}") latest = os.path.join(basedir, "latest") if os.path.exists(latest) and not os.path.islink(latest): soft_abort(f'"{latest}" already exists and is not a symlink') return if os.path.exists(proc_latest): try: os.unlink(proc_latest) except OSError as details: soft_abort(f"Unable to remove {proc_latest}: {details}") return try: os.symlink(basename, proc_latest) os.rename(proc_latest, latest) except OSError as details: soft_abort(f"Unable to create create latest symlink: {details}") return finally: if os.path.exists(proc_latest): os.unlink(proc_latest) @classmethod def from_config(cls, job_config, suites_configs=None): """Helper method to create a job from config dicts. This is different from the Job() initialization because here we are assuming that you need some help to build the test suites. Avocado will try to resolve tests based on the configuration information instead of assuming pre populated test suites. Keep in mind that here we are going to replace the suite.name with a counter. If you need create a custom Job with your own TestSuites, please use the Job() constructor instead of this method. :param job_config: A config dict to be used on this job and also as a 'global' config for each test suite. :type job_config: dict :param suites_configs: A list of specific config dict to be used on each test suite. Each suite config will be merged with the job_config dict. If None is passed then this job will have only one test_suite with the same config as job_config. :type suites_configs: list """ suites_configs = suites_configs or [deepcopy(job_config)] suites = [] for index, config in enumerate(suites_configs, start=1): suites.append( TestSuite.from_config(config, name=index, job_config=job_config) ) return cls(job_config, suites) @property def size(self): """Job size is the sum of all test suites sizes.""" return sum(suite.size for suite in self.test_suites) @property def test_suite(self): """This is the first test suite of this job (deprecated). Please, use test_suites instead. """ if self.test_suites: return self.test_suites[0] @test_suite.setter def test_suite(self, var): """Temporary setter. Suites should be set from test_suites.""" if self.test_suites: self.test_suites[0] = var else: self.test_suites = [var] @property def timeout(self): if self._timeout is None: self._timeout = self.config.get("job.run.timeout") return self._timeout @property def unique_id(self): if self._unique_id is None: self._unique_id = ( self.config.get("run.unique_job_id") or create_unique_job_id() ) return self._unique_id def cleanup(self): """ Cleanup the temporary job handlers (dirs, global setting, ...) """ self.__stop_job_logging() if not self.__keep_tmpdir and os.path.exists(self.tmpdir): shutil.rmtree(self.tmpdir) shutil.rmtree(self._base_tmpdir) cleanup_conditionals = ( self.config.get("run.dry_run.enabled"), not self.config.get("run.dry_run.no_cleanup"), ) if all(cleanup_conditionals): # Also clean up temp base directory created because of the dry-run base_logdir = self.config.get("run.results_dir") if base_logdir is not None: try: shutil.rmtree(base_logdir) except FileNotFoundError: pass def create_test_suite(self): msg = ( "create_test_suite() is deprecated. You can also create your " "own suites with TestSuite() or TestSuite.from_config()." ) warnings.warn(msg, DeprecationWarning) try: self.test_suite = TestSuite.from_config(self.config) if self.test_suite and self.test_suite.size == 0: refs = self.test_suite.references msg = ( f"No tests found for given test references, try " f"'avocado -V list {' '.join(refs)}' for details" ) raise exceptions.JobTestSuiteEmptyError(msg) except TestSuiteError as details: raise exceptions.JobBaseException(details) if self.test_suite: self.result.tests_total = self.test_suite.size def post_tests(self): """ Run the post tests execution hooks By default this runs the plugins that implement the :class:`avocado.core.plugin_interfaces.JobPostTests` interface. """ self.result_events_dispatcher.map_method("post_tests", self) def pre_tests(self): """ Run the pre tests execution hooks By default this runs the plugins that implement the :class:`avocado.core.plugin_interfaces.JobPreTests` interface. """ self.result_events_dispatcher.map_method("pre_tests", self) def render_results(self): """Render test results that depend on all tests having finished. By default this runs the plugins that implement the :class:`avocado.core.plugin_interfaces.Result` interface. """ result_dispatcher = dispatcher.ResultDispatcher() if result_dispatcher.extensions: result_dispatcher.map_method("render", self.result, self) def get_failed_tests(self): """Gets the tests with status 'FAIL' and 'ERROR' after the Job ended. :return: List of failed tests """ tests = [] if self.result: for test in self.result.tests: if test.get("status") in ["FAIL", "ERROR"]: tests.append(test) return tests def run(self): """ Runs all job phases, returning the test execution results. This method is supposed to be the simplified interface for jobs, that is, they run all phases of a job. :return: Integer with overall job status. See :mod:`avocado.core.exit_codes` for more information. """ assert self.tmpdir is not None, "Job.setup() not called" self.result.job_start_date_time = datetime.datetime.now() if self._time_start is None: self._time_start = time.monotonic() try: self.result.tests_total = self.size pre_post_dispatcher = dispatcher.JobPrePostDispatcher() output.log_plugin_failures(pre_post_dispatcher.load_failures) pre_post_dispatcher.map_method("pre", self) self.pre_tests() return self.run_tests() except exceptions.JobBaseException as details: fail_class = details.__class__.__name__ self.log.error("\nAvocado job failed: %s: %s", fail_class, details) self.exitcode |= exit_codes.AVOCADO_JOB_FAIL return self.exitcode except exceptions.OptionValidationError as details: self.log.error("\n%s", str(details)) self.exitcode |= exit_codes.AVOCADO_JOB_FAIL return self.exitcode except Exception as details: # pylint: disable=W0703 self.status = "ERROR" exc_type, exc_value, exc_traceback = sys.exc_info() tb_info = traceback.format_exception( exc_type, exc_value, exc_traceback.tb_next ) fail_class = details.__class__.__name__ self.log.error("\nAvocado crashed: %s: %s", fail_class, details) for line in tb_info: self.log.debug(line) self.log.error( "Please include the traceback info and command line" " used on your bug report" ) self.log.error("Report bugs visiting %s", _NEW_ISSUE_LINK) self.exitcode |= exit_codes.AVOCADO_FAIL return self.exitcode finally: self.post_tests() if self._time_end is None: self._time_end = time.monotonic() self.time_elapsed = self._time_end - self._time_start self.render_results() pre_post_dispatcher.map_method("post", self) def run_tests(self): """ The actual test execution phase """ self._log_job_debug_info() jobdata.record(self, sys.argv) if self.size == 0: msg = 'Unable to resolve any reference or "resolver.references" is empty.' LOG_UI.error(msg) if not self.test_suites: self.exitcode |= exit_codes.AVOCADO_JOB_FAIL return self.exitcode summary = set() for suite in self.test_suites: summary |= suite.run(self) # If it's all good so far, set job status to 'PASS' if self.status == "RUNNING": self.status = "PASS" LOG_JOB.info("Test results available in %s", self.logdir) if "INTERRUPTED" in summary: self.exitcode |= exit_codes.AVOCADO_JOB_INTERRUPTED if "FAIL" in summary or "ERROR" in summary: self.exitcode |= exit_codes.AVOCADO_TESTS_FAIL return self.exitcode def _check_test_suite_name_uniqueness(self): all_names = [suite.name for suite in self.test_suites] duplicate_names = set([name for name in all_names if all_names.count(name) > 1]) if duplicate_names: duplicate_names = ", ".join(duplicate_names) msg = ( f"Job contains suites with the following duplicate " f"name(s) {duplicate_names}. Test suite names must be " f"unique to guarantee that results will not be overwritten" ) raise exceptions.JobTestSuiteDuplicateNameError(msg) def setup(self): """ Setup the temporary job handlers (dirs, global setting, ...) """ output.reconfigure(self.config) assert self.tmpdir is None, "Job.setup() already called" if self.config.get("run.dry_run.enabled"): # Create the dry-run dirs if self.config.get("run.results_dir") is None: tmp_dir = tempfile.mkdtemp(prefix="avocado-dry-run-") self.config["run.results_dir"] = tmp_dir self._setup_job_results() self.result = result.Result(self.unique_id, self.logfile) self.__start_job_logging() self._setup_job_category() # Use "logdir" in case "keep_tmp" is enabled if self.config.get("run.keep_tmp"): self._base_tmpdir = self.logdir else: self._base_tmpdir = tempfile.mkdtemp(prefix="avocado_tmp_") self.__keep_tmpdir = False self.tmpdir = tempfile.mkdtemp(prefix="avocado_job_", dir=self._base_tmpdir) avocado-113.0/avocado/core/job_id.py000066400000000000000000000015771513561306000173100ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2013-2014 # Authors: Lucas Meneghel Rodrigues import hashlib import random _RAND_POOL = random.SystemRandom() def create_unique_job_id(): """ Create a 40 digit hex number to be used as a job ID string. (similar to SHA1) :return: 40 digit hex number string :rtype: str """ return hashlib.sha1(hex(_RAND_POOL.getrandbits(160)).encode()).hexdigest() avocado-113.0/avocado/core/jobdata.py000066400000000000000000000124541513561306000174620ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2016 # Author: Amador Pahim """ Record/retrieve job information """ import ast import json import os from avocado.core.nrunner.config import ConfigDecoder, ConfigEncoder from avocado.core.output import LOG_JOB, LOG_UI from avocado.core.varianter import VARIANTS_FILENAME from avocado.utils.astring import string_to_safe_path from avocado.utils.path import init_dir JOB_DATA_DIR = "jobdata" TEST_REFERENCES_FILENAME = "test_references" PWD_FILENAME = "pwd" JOB_CONFIG_FILENAME = "args.json" CMDLINE_FILENAME = "cmdline" def json_bad_variants_obj(item): for log in [LOG_UI, LOG_JOB]: log.warning("jobdata.variants: Unable to serialize '%s'", item) return str(item) def record_suite_variant(path_variants, suite): with open(path_variants, "w", encoding="utf-8") as variants_file: variants = [] variants += suite.variants.dump() json.dump(variants, variants_file, default=json_bad_variants_obj) variants_file.flush() os.fsync(variants_file) def record(job, cmdline=None): """ Records all required job information. """ base_dir = init_dir(job.logdir, JOB_DATA_DIR) path_references = os.path.join(base_dir, TEST_REFERENCES_FILENAME) path_pwd = os.path.join(base_dir, PWD_FILENAME) path_job_config = os.path.join(base_dir, JOB_CONFIG_FILENAME) path_cmdline = os.path.join(base_dir, CMDLINE_FILENAME) references = job.config.get("resolver.references") if references: with open(path_references, "w", encoding="utf-8") as references_file: references_file.write(f"{references}") references_file.flush() os.fsync(references_file) for idx, suite in enumerate(job.test_suites, 1): if suite.name: suite_var_name = f"variants-{idx}-{suite.name}.json" else: suite_var_name = f"variants-{idx}.json" suite_var_name = string_to_safe_path(suite_var_name) path_suite_variant = os.path.join(base_dir, suite_var_name) record_suite_variant(path_suite_variant, suite) with open(path_pwd, "w", encoding="utf-8") as pwd_file: pwd_file.write(f"{os.getcwd()}") pwd_file.flush() os.fsync(pwd_file) with open(path_job_config, "w", encoding="utf-8") as job_config_file: json.dump(job.config, job_config_file, cls=ConfigEncoder) job_config_file.flush() os.fsync(job_config_file) with open(path_cmdline, "w", encoding="utf-8") as cmdline_file: cmdline_file.write(f"{cmdline}") cmdline_file.flush() os.fsync(cmdline_file) def _retrieve(resultsdir, resource): path = os.path.join(resultsdir, JOB_DATA_DIR, resource) if not os.path.exists(path): return None return path def retrieve_pwd(resultsdir): """ Retrieves the job pwd from the results directory. """ recorded_pwd = _retrieve(resultsdir, PWD_FILENAME) if recorded_pwd is None: return None with open(recorded_pwd, "r", encoding="utf-8") as pwd_file: return pwd_file.read() def retrieve_references(resultsdir): """ Retrieves the job test references from the results directory. """ recorded_references = _retrieve(resultsdir, TEST_REFERENCES_FILENAME) if recorded_references is None: return None with open(recorded_references, "r", encoding="utf-8") as references_file: return ast.literal_eval(references_file.read()) def get_variants_path(resultsdir): """ Retrieves the variants path from the results directory. """ return _retrieve(resultsdir, VARIANTS_FILENAME) def retrieve_job_config(resultsdir): """ Retrieves the job config from the results directory. """ recorded_job_config = _retrieve(resultsdir, JOB_CONFIG_FILENAME) if recorded_job_config: with open(recorded_job_config, "r", encoding="utf-8") as job_config_file: return json.load(job_config_file, cls=ConfigDecoder) def retrieve_cmdline(resultsdir): """ Retrieves the job command line from the results directory. """ recorded_cmdline = _retrieve(resultsdir, CMDLINE_FILENAME) if recorded_cmdline is None: # Attempt to restore cmdline from log try: with open( os.path.join(resultsdir, "job.log"), "r", encoding="utf-8" ) as log: import re cmd = re.search( r"# \d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2},\d{3} " r"\w{17}\w\d{4} INFO | Command line: (.*)", log.read(), ) if cmd: import shlex return shlex.split(cmd.group(1)) except IOError: pass return None with open(recorded_cmdline, "r", encoding="utf-8") as cmdline_file: return ast.literal_eval(cmdline_file.read()) avocado-113.0/avocado/core/main.py000077500000000000000000000056051513561306000170050ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; specifically version 2 of the License. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: RedHat 2013-2014 # Author: Lucas Meneghel Rodrigues import multiprocessing import os import sys import tempfile import time import traceback try: from avocado.core import exit_codes from avocado.core.settings import settings except ImportError: sys.stderr.write( "Unable to import Avocado libraries, please verify " "your installation, and if necessary reinstall it.\n" ) # This exit code is replicated from avocado/core/exit_codes.py and not # imported because we are dealing with import failures sys.exit(-1) def get_crash_dir(): config = settings.as_dict() crash_dir_path = os.path.join(config.get("datadir.paths.data_dir"), "crashes") try: os.makedirs(crash_dir_path) except OSError: pass return crash_dir_path def handle_exception(*exc_info): tb = "".join(traceback.format_exception(*exc_info)) # Store traceback in data_dir or TMPDIR prefix = "avocado-traceback-" prefix += time.strftime("%F_%T") + "-" tmp, name = tempfile.mkstemp(".log", prefix, get_crash_dir()) os.write(tmp, tb.encode("utf-8")) os.close(tmp) if exc_info[0] is KeyboardInterrupt: os.write( 2, f"{exc_info[0].__doc__}\nYou can find details in {name}\n".encode("utf-8"), ) exit_code = exit_codes.AVOCADO_JOB_INTERRUPTED else: # Print friendly message and traceback in console-like output os.write(2, f"Avocado crashed unexpectedly: {exc_info[1]}\n".encode("utf-8")) os.write(2, tb.encode("utf-8")) os.write(2, f"\nYou can also find details in {name}\n".encode("utf-8")) exit_code = exit_codes.AVOCADO_GENERIC_CRASH sys.exit(exit_code) def main(): if sys.platform == "darwin": multiprocessing.set_start_method("fork") sys.excepthook = handle_exception from avocado.core.app import AvocadoApp # pylint: disable=E0611 # Override tmp in case it's not set in env for attr in ("TMP", "TEMP", "TMPDIR"): if attr in os.environ: break else: # TMP not set by user, use /var/tmp if exists # TMP not set by user in environment. Try to use /var/tmp to avoid # possible problems with "/tmp" being mounted as TMPFS without the # support for O_DIRECT if os.path.exists("/var/tmp"): os.environ["TMP"] = "/var/tmp" app = AvocadoApp() return app.run() if __name__ == "__main__": sys.exit(main()) avocado-113.0/avocado/core/messages.py000066400000000000000000000442551513561306000176710ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2021 # Authors: Jan Richter import logging import os import time from avocado.core.nrunner.task import TASK_DEFAULT_CATEGORY from avocado.core.output import LOG_UI from avocado.core.test_id import TestID DEFAULT_LOG_FILE = "debug.log" class BaseMessageHandler: """ Base interface for resolving runner messages. This is the interface a job uses to deal with messages from runners. """ def handle(self, message, task, job): """ Handle message from runner. :param message: message from runner. :type message: dict :param task: runtime_task which message is related to :type task: :class:`avocado.core.nrunner.Task` :param job: job which task is related to :type job: :class:`avocado.core.job.Job` """ def process_message(self, message, task, job): """ It transmits the message to the right handler. :param message: message from runner :type message: dict :param task: runtime_task which message is related to :type task: :class:`avocado.core.nrunner.Task` :param job: job which task is related to :type job: :class:`avocado.core.job.Job` """ self.handle(message, task, job) class MessageHandler(BaseMessageHandler): """Entry point for handling messages.""" def __init__(self): self._handlers = { "started": [StartMessageHandler()], "finished": [FinishMessageHandler()], "running": [RunningMessageHandler()], } def process_message(self, message, task, job): for handler in self._handlers.get(message.get("status"), []): handler.process_message(message, task, job) class RunningMessageHandler(BaseMessageHandler): """Entry point for handling running messages.""" def __init__(self): self._handlers = { "log": [LogMessageHandler()], "stdout": [StdoutMessageHandler()], "stderr": [StderrMessageHandler()], "whiteboard": [WhiteboardMessageHandler()], "output": [OutputMessageHandler()], "file": [FileMessageHandler()], } def process_message(self, message, task, job): for handler in self._handlers.get(message.get("type"), []): handler.process_message(message, task, job) class StartMessageHandler(BaseMessageHandler): """ Handler for started message. It will create the test base directories and triggers the 'start_test' event. This have to be triggered when the runner starts the test. :param status: 'started' :param time: start time of the test :type time: float example: {'status': 'started', 'time': 16444.819830573} """ @staticmethod def prepare_metadata(task, job, start_time): """ Creates metadata for task. :param task: runtime_task which message is related to :type task: :class:`avocado.core.nrunner.Task` :param job: job which task is related to :type job: :class:`avocado.core.job.Job` :param start_time: time when the task started :type start_time: float """ task_id = TestID.from_identifier(task.identifier) base_path = job.test_results_path task_path = task.runnable.output_dir logfile = os.path.join(task_path, DEFAULT_LOG_FILE) os.makedirs(task_path, exist_ok=True) params = [] symlink_dir = task.metadata.get("symlink") if symlink_dir: os.makedirs( os.path.abspath(os.path.join(symlink_dir, os.pardir)), exist_ok=True ) os.symlink(task_path, symlink_dir, target_is_directory=True) if task.runnable.variant is not None: # convert variant into the list of parameters params = [ param for params in task.runnable.variant.get("variant", []) for param in params[1] ] metadata = { "job_logdir": job.logdir, "job_unique_id": job.unique_id, "base_path": base_path, "logfile": logfile, "task_path": task_path, "time_start": start_time, "actual_time_start": time.time(), "name": task_id, "params": params, } return metadata def handle(self, message, task, job): metadata = self.prepare_metadata(task, job, message["time"]) if task.category == TASK_DEFAULT_CATEGORY: job.result.start_test(metadata) job.result_events_dispatcher.map_method("start_test", job.result, metadata) task.metadata.update(metadata) class FinishMessageHandler(BaseMessageHandler): """ Handler for finished message. It will report the test status and triggers the 'end_test' event. This is triggered when the runner ends the test. :param status: 'finished' :param result: test result :type result: `avocado.core.teststatus.STATUSES` :param time: end time of the test :type time: float :param fail_reason: Parameter for brief specification, of the failed result. [Optional] :type fail_reason: string :param returncode: Exit status of runner. [Optional] :type returncode: int :param class_name: class name of the test. [Optional] :type class_name: string :param returncode: Exit status of runner. [Optional] :type returncode: int :param fail_class: Exception class of the failure. [Optional] :type fail_class: string :param traceback: Traceback of the exception. [Optional] :type traceback: string example: {'status': 'finished', 'result': 'pass', 'time': 16444.819830573} """ def handle(self, message, task, job): if not task.metadata: task.metadata.update( StartMessageHandler.prepare_metadata(task, job, message["time"]) ) message.update(task.metadata) message["name"] = TestID.from_identifier(task.identifier) message["status"] = message.get("result").upper() time_start = message["time_start"] time_end = message["time"] time_elapsed = time_end - time_start message["time_end"] = time_end message["actual_time_end"] = time.time() message["time_elapsed"] = time_elapsed message["logdir"] = task.metadata["task_path"] message["tags"] = task.runnable.get_serializable_tags() if task.category == "test": job.result.check_test(message) job.result_events_dispatcher.map_method("end_test", job.result, message) class BaseRunningMessageHandler(BaseMessageHandler): """Base interface for resolving running messages.""" _tag = b"" def __init__(self): self.line_buffer = b"" def _split_complete_lines(self, data): """ It will split data into list of lines. If the data don't finish with the new line character, the last line is marked as incomplete, and it is saved to the line buffer for next usage. When the buffer is not empty the buffer will be added at the beginning of data. :param data: massage log :type data: bytes :return: list of lines """ data_lines = data.splitlines(True) if len(data_lines) <= 1 and not data.endswith(b"\n"): self.line_buffer += data return [] else: data_lines[0] = self.line_buffer + data_lines[0] self.line_buffer = b"" if not data.endswith(b"\n"): self.line_buffer = data_lines.pop() return data_lines def _save_to_default_file(self, message, task): """ It will save message log into the default log file. The default log file is based on `DEFAULT_LOG_FILE` variable and every line of log will be saved with prefix based on `_tag` variable :param message: message from runner :type message: dict :param task: runtime_task which message is related to :type task: :class:`avocado.core.nrunner.Task` """ if message.get("encoding"): data = message.get("log", b"").splitlines(True) else: data = self._split_complete_lines(message.get("log", b"")) if data: data = self._tag + self._tag.join(data) self._save_message_to_file( DEFAULT_LOG_FILE, data, task, message.get("encoding") ) @staticmethod def _message_to_line(message, encoding): """ Converts the message to string. When the message doesn't end with a new line, the new line is added. :param message: message for decoding :type message: bytes :param encoding: encoding of the message :type encoding: str :return: encoded message with new line character :rtype: str """ message = message.decode(encoding) if not message.endswith("\n"): message = f"{message}\n" return message @staticmethod def _save_message_to_file(filename, buff, task, encoding=None): """ Method for saving messages into the file It can decode and save messages.The message will be decoded when encoding is not None. When the decoded message doesn't end with a new line the new line will be added. Every message is saved in the append mode. :param filename: name of the file :type filename: str :param buff: message to be saved :type buff: bytes :param task: message related task. :type task: :class:`avocado.core.nrunner.Task` :param encoding: encoding of buff, default is None :type encoding: str """ def _save_to_file(file_name, mode): with open(file_name, mode) as fp: # pylint: disable=W1514 fp.write(buff) file = os.path.join(task.metadata["task_path"], filename) if encoding: buff = BaseRunningMessageHandler._message_to_line(buff, encoding) _save_to_file(file, "a") else: _save_to_file(file, "ab") class LogMessageHandler(BaseRunningMessageHandler): """ Handler for log message. It will save the log to the debug.log file in the task directory. :param status: 'running' :param type: 'log' :param log: log message :type log: string :param time: Time stamp of the message :type time: float :param log_name: optional name of the logger, such as "avocado.test.foo" :type log_name: string :param log_levelname: level of the logger, such as "INFO", required if "log_name" is set :type log_levelname: string :param log_level: numeric log level to be used as a fallback in cases where the log level name does not exist on the main Avocado side (and only on the avocado-runner-$type side). :type log_level: int example: {'status': 'running', 'type': 'log', 'log': 'log message', 'time': 18405.55351474} or: example: {'status': 'running', 'type': 'log', 'log': 'log message', 'time': 18405.55351474, 'log_name': 'avocado.test.foo', 'log_levelname': 'INFO'} """ _tag = b"[stdlog] " def handle(self, message, task, job): """Logs a textual message to a file. This assumes that the log message will not contain a newline, and thus one is explicitly added here. """ if task.metadata.get("logfile") is None: task.metadata["logfile"] = os.path.join( task.metadata["task_path"], "debug.log" ) self._save_to_default_file(message, task) # For messages from specific loggers, we preserve their names # and levels so that they are handled appropriately based on # the Avocado job logging configuration log_name = message.get("log_name") if log_name is not None and log_name != "avocado.app": logger = logging.getLogger(log_name) level = logging.getLevelName(message.get("log_levelname")) log_message = f"{task.identifier}: {message.get('log').decode(message.get('encoding'))}" logger_level = logger.level try: logger.setLevel(level) except ValueError: logging.addLevelName( message.get("log_level"), message.get("log_levelname") ) level = logging.getLevelName(message.get("log_levelname")) logger.log(level, log_message) logger.setLevel(logger_level) class StdoutMessageHandler(BaseRunningMessageHandler): """ Handler for stdout message. It will save the stdout to the debug file in the task directory, and optionally (depending on "log_only" value) to the stdout file. :param status: 'running' :param type: 'stdout' :param log: stdout message :type log: bytes :param log_only: whether to save the "log" message only to the standard test log (and not to the "stdout" file) :type log_only: bool :param encoding: optional value for decoding messages :type encoding: str :param time: Time stamp of the message :type time: float example: {'status': 'running', 'type': 'stdout', 'log': 'stdout message', 'log_only': False, 'time': 18405.55351474} """ _tag = b"[stdout] " def handle(self, message, task, job): self._save_to_default_file(message, task) if not message.get("log_only", False): self._save_message_to_file( "stdout", message["log"], task, message.get("encoding", None) ) class StderrMessageHandler(BaseRunningMessageHandler): """ Handler for stderr message. It will save the stderr to the debug file in the task directory, and optionally (depending on "log_only" value) to the stderr file. :param status: 'running' :param type: 'stderr' :param log: stderr message :type log: bytes :param log_only: whether to save the "log" message only to the standard test log (and not to the "stderr" file) :type log_only: bool :param encoding: optional value for decoding messages :type encoding: str :param time: Time stamp of the message :type time: float example: {'status': 'running', 'type': 'stderr', 'log': 'stderr message', 'log_only': False, 'time': 18405.55351474} """ _tag = b"[stderr] " def handle(self, message, task, job): self._save_to_default_file(message, task) if not message.get("log_only", False): self._save_message_to_file( "stderr", message["log"], task, message.get("encoding", None) ) class WhiteboardMessageHandler(BaseRunningMessageHandler): """ Handler for whiteboard message. It will save the stderr to the whiteboard file in the task directory. :param status: 'running' :param type: 'whiteboard' :param log: whiteboard message :type log: bytes :param encoding: optional value for decoding messages :type encoding: str :param time: Time stamp of the message :type time: float example: {'status': 'running', 'type': 'whiteboard', 'log': 'whiteboard message', 'time': 18405.55351474} """ def handle(self, message, task, job): encoding = message.get("encoding", "utf-8") whiteboard = task.metadata.get("whiteboard", "") whiteboard += message["log"].decode(encoding) task.metadata["whiteboard"] = whiteboard self._save_message_to_file("whiteboard", message["log"], task, encoding) class FileMessageHandler(BaseRunningMessageHandler): """ Handler for file message. In task directory will save log into the runner specific file. When the file doesn't exist, the file will be created. If the file exist, the message data will be appended at the end. :param status: 'running' :param type: 'file' :param path: relative path to the file. The file will be created under the Task directory and the absolute path will be created as `absolute_task_directory_path/relative_file_path`. :type path: string :param log: data to be saved inside file :type log: bytes :param time: Time stamp of the message :type time: float example: {'status': 'running', 'type': 'file', 'path':'foo/runner.log', 'log': 'this will be saved inside file', 'time': 18405.55351474} """ def handle(self, message, task, job): filename = os.path.relpath(os.path.join("/", message["path"]), "/") file = os.path.join(task.metadata["task_path"], filename) if not os.path.exists(file): os.makedirs(os.path.dirname(file), exist_ok=True) self._save_message_to_file( filename, message["log"], task, message.get("encoding", None) ) class OutputMessageHandler(BaseRunningMessageHandler): """ Handler for displaying messages in UI. It will show the message content in avocado UI. :param status: 'running' :param type: 'output' :param log: output message :type log: bytes :param encoding: optional value for decoding messages :type encoding: str :param time: Time stamp of the message :type time: float example: {'status': 'running', 'type': 'output', 'log': 'this is the output', 'time': 18405.55351474} """ def handle(self, message, task, job): encoding = message.get("encoding", "utf-8") output = message["log"].decode(encoding) task_id = TestID.from_identifier(task.identifier) output = f"{task_id}: {output}" LOG_UI.debug(output) avocado-113.0/avocado/core/nrunner/000077500000000000000000000000001513561306000171655ustar00rootroot00000000000000avocado-113.0/avocado/core/nrunner/__init__.py000066400000000000000000000000001513561306000212640ustar00rootroot00000000000000avocado-113.0/avocado/core/nrunner/app.py000066400000000000000000000237761513561306000203360ustar00rootroot00000000000000import argparse import inspect import json import os import re import sys import pkg_resources from avocado.core.nrunner.runnable import Runnable from avocado.core.nrunner.task import TASK_DEFAULT_CATEGORY, Task def _get_kind_options_from_executable_name(): executable_name = os.path.basename(sys.argv[0]) match = re.match(r"^avocado\-runner\-(.+)$", executable_name) options = {"type": str, "help": "Kind of runnable"} if match: options["required"] = False options["default"] = match.group(1) options["help"] += ', defaults to "%(default)s"' else: options["required"] = True return options def _parse_key_val(argument): key_value = argument.split("=", 1) if len(key_value) < 2: msg = ( f'Invalid keyword parameter: "{argument}". Valid option must ' f'be a "KEY=VALUE" like expression' ) raise argparse.ArgumentTypeError(msg) return tuple(key_value) class BaseRunnerApp: """ Helper base class for common runner application behavior """ #: The name of the command line application given to the command line #: parser PROG_NAME = "" #: The description of the command line application given to the #: command line parser PROG_DESCRIPTION = "" #: The names of types of runnables that this runner can handle. RUNNABLE_KINDS_CAPABLE = [] #: The command line arguments to the "runnable-run" command CMD_RUNNABLE_RUN_ARGS = ( (("-k", "--kind"), _get_kind_options_from_executable_name()), (("-u", "--uri"), {"type": str, "default": None, "help": "URI of runnable"}), ( ("-c", "--config"), {"type": str, "default": "{}", "help": "A config JSON data"}, ), ( ("-a", "--arg"), {"action": "append", "default": [], "help": "Simple arguments to runnable"}, ), ( ("kwargs",), { "default": [], "type": _parse_key_val, "nargs": "*", "metavar": "KEY_VAL", "help": "Keyword (key=val) arguments to runnable", }, ), ) CMD_RUNNABLE_RUN_RECIPE_ARGS = ( (("recipe",), {"type": str, "help": "Path to the runnable recipe file"}), ) CMD_TASK_RUN_ARGS = ( ( ("-i", "--identifier"), {"type": str, "required": True, "help": "Task unique identifier"}, ), ( ("-t", "--category"), { "type": str, "required": False, "default": TASK_DEFAULT_CATEGORY, "help": ( f"The category for tasks. Only tasks with category set " f'to "{TASK_DEFAULT_CATEGORY}" (the default) will be ' f"included in the test results of its parent job. Other " f"categories may be used for purposes that do include " f"test results such as requirements resolution tasks" ), }, ), ( ("-s", "--status-uri"), { "action": "append", "default": None, "help": "URIs of status services to report to", }, ), ( ("-j", "--job-id"), { "type": str, "required": False, "metavar": "JOB_ID", "help": "Identifier of Job this task belongs to", }, ), ) CMD_TASK_RUN_ARGS += CMD_RUNNABLE_RUN_ARGS CMD_TASK_RUN_RECIPE_ARGS = ( (("recipe",), {"type": str, "help": "Path to the task recipe file"}), ) CMD_STATUS_SERVER_ARGS = ( (("uri",), {"type": str, "help": "URI to bind a status server to"}), ) def __init__(self, echo=print, prog=None, description=None): self.echo = echo self.parser = None if prog is None: prog = self.PROG_NAME if description is None: description = self.PROG_DESCRIPTION self._class_commands_method = self._get_commands_method() self._setup_parser(prog, description) def _setup_parser(self, prog, description): self.parser = argparse.ArgumentParser(prog=prog, description=description) subcommands = self.parser.add_subparsers(dest="subcommand") subcommands.required = True for command, method in self._class_commands_method.items(): command_args = f"CMD_{command.upper().replace('-', '_')}_ARGS" command_parser = subcommands.add_parser( command, help=self._get_command_method_help_message(method) ) if hasattr(self, command_args): for arg in getattr(self, command_args): command_parser.add_argument(*arg[0], **arg[1]) def _get_commands_method(self): prefix = "command_" return { c[0][len(prefix) :].replace("_", "-"): getattr(self, c[0]) for c in inspect.getmembers(self, inspect.ismethod) if c[0].startswith(prefix) } @staticmethod def _get_command_method_help_message(command_method): help_message = "" docstring = command_method.__doc__ if docstring: docstring_lines = docstring.strip().splitlines() if docstring_lines: help_message = docstring_lines[0] return help_message def run(self): """ Runs the application by finding a suitable command method to call """ args = vars(self.parser.parse_args()) subcommand = args.get("subcommand") kallable = self._class_commands_method.get(subcommand, None) if kallable is not None: return kallable(args) def get_commands(self): """ Return the command names, as seen on the command line application For every method whose name starts with "command", and the name of the command follows, with underscores replaced by dashes. So, a method named "command_foo_bar", will be a command available on the command line as "foo-bar". :rtype: list """ return list(self._class_commands_method.keys()) def get_capabilities(self): """ Returns the runner capabilities, including runnables and commands This can be used by higher level tools, such as the entity spawning runners, to know which runner can be used to handle each runnable type. :rtype: dict """ return { "runnables": self.RUNNABLE_KINDS_CAPABLE, "commands": self.get_commands(), "configuration_used": self.get_configuration_used_by_runners(), } def get_runner_from_runnable(self, runnable): """ Returns a runner that is suitable to run the given runnable :rtype: instance of class inheriting from :class:`BaseRunner` :raises: ValueError if runnable is now supported """ runner = runnable.pick_runner_class() if runner is not None: return runner() raise ValueError(f"Unsupported kind of runnable: {runnable.kind}") def get_configuration_used_by_runners(self): """Returns the configuration keys used by capable runners. :returns: the configuration keys (aka namespaces) used by known runners :rtype: list """ config_used = [] for kind in self.RUNNABLE_KINDS_CAPABLE: for ep in pkg_resources.iter_entry_points( "avocado.plugins.runnable.runner", kind ): try: runner = ep.load() config_used += runner.CONFIGURATION_USED except ImportError: continue return list(set(config_used)) def command_capabilities(self, _): """ Outputs capabilities, including runnables and commands The output is intended to be consumed by upper layers of Avocado, such as the Job layer selecting the right runner script to handle a runnable of a given kind, or identifying if a runner script has a given feature (as implemented by a command). """ self.echo(json.dumps(self.get_capabilities())) def command_runnable_run(self, args): """ Runs a runnable definition from arguments This defines a Runnable instance purely from the command line arguments, then selects a suitable Runner, and runs it. :param args: parsed command line arguments turned into a dictionary :type args: dict """ runnable = Runnable.from_args(args) runner = self.get_runner_from_runnable(runnable) for status in runner.run(runnable): self.echo(status) def command_runnable_run_recipe(self, args): """ Runs a runnable definition from a recipe :param args: parsed command line arguments turned into a dictionary :type args: dict """ runnable = Runnable.from_recipe(args.get("recipe")) runner = self.get_runner_from_runnable(runnable) for status in runner.run(runnable): self.echo(status) def command_task_run(self, args): """ Runs a task from arguments :param args: parsed command line arguments turned into a dictionary :type args: dict """ runnable = Runnable.from_args(args) task = Task( runnable, args.get("identifier"), args.get("status_uri", []), category=args.get("category", TASK_DEFAULT_CATEGORY), job_id=args.get("job_id"), ) for status in task.run(): self.echo(status) def command_task_run_recipe(self, args): """ Runs a task from a recipe :param args: parsed command line arguments turned into a dictionary :type args: dict """ task = Task.from_recipe(args.get("recipe")) for status in task.run(): self.echo(status) avocado-113.0/avocado/core/nrunner/config.py000066400000000000000000000017251513561306000210110ustar00rootroot00000000000000import json class ConfigDecoder(json.JSONDecoder): """ JSON Decoder for config options. """ @staticmethod def decode_set(config_dict): for k, v in config_dict.items(): if isinstance(v, dict): if "__encoded_set__" in v: config_dict[k] = set(v["__encoded_set__"]) return config_dict def decode(self, config_str): # pylint: disable=W0221 config_dict = json.JSONDecoder.decode(self, config_str) return self.decode_set(config_dict) class ConfigEncoder(json.JSONEncoder): """ JSON Encoder for config options. """ def default(self, o): if isinstance(o, set): return {"__encoded_set__": list(o)} try: return json.JSONEncoder.default(self, o) except TypeError: # Probably this is a not JSON serializable data. To keep the same # behavior as before, lets return None return None avocado-113.0/avocado/core/nrunner/runnable.py000066400000000000000000000612641513561306000213560ustar00rootroot00000000000000import base64 import collections import copy import json import logging import os import subprocess import sys import warnings import pkg_resources try: import jsonschema JSONSCHEMA_AVAILABLE = True except ImportError: JSONSCHEMA_AVAILABLE = False from avocado.core.dependencies.dependency import Dependency from avocado.core.nrunner.config import ConfigDecoder, ConfigEncoder from avocado.core.settings import settings from avocado.core.utils.eggenv import get_python_path_env_if_egg LOG = logging.getLogger(__name__) #: All known runner commands, capable of being used by a #: SpawnMethod.STANDALONE_EXECUTABLE compatible spawners RUNNERS_REGISTRY_STANDALONE_EXECUTABLE = {} #: The configuration that is known to be used by standalone runners STANDALONE_EXECUTABLE_CONFIG_USED = {} #: Location used for schemas when packaged (as in RPMs) SYSTEM_WIDE_SCHEMA_PATH = "/usr/share/avocado/schemas" #: Configuration used by all runnables, no matter what its kind. The #: configuration that a kind uses in addition to this is set in their #: own class attribute "CONFIGURATION_USED" CONFIGURATION_USED = ["runner.identifier_format"] class RunnableRecipeInvalidError(Exception): """Signals that a runnable recipe is not well formed, contains missing or bad data""" def _arg_decode_base64(arg): """ Decode arguments possibly encoded as base64 :param arg: the possibly encoded argument :type arg: str :returns: the decoded argument :rtype: str """ prefix = "base64:" if arg.startswith(prefix): content = arg[len(prefix) :] return base64.decodebytes(content.encode()).decode() return arg def _kwarg_decode_json(value): """ Decode arguments possibly encoded as base64 :param value: the possibly encoded argument :type value: str :returns: the decoded keyword argument as Python object """ prefix = "json:" if value.startswith(prefix): content = value[len(prefix) :] return json.loads(content) return value def _key_val_args_to_kwargs(kwargs): result = {} for key, val in kwargs: result[key] = _kwarg_decode_json(val) return result class Runnable: """ Describes an entity that be executed in the context of a task A instance of :class:`BaseRunner` is the entity that will actually execute a runnable. """ def __init__(self, kind, uri, *args, config=None, identifier=None, **kwargs): self.kind = kind #: The main reference to what needs to be run. This is free #: form, but commonly set to the path to a file containing the #: test or being the test, or an actual URI with multiple #: parts self.uri = uri #: This attributes holds default configuration values that the #: runner has determined that has interest in by setting it in #: attr:`avocado.core.nrunner.runner.BaseRunner.CONFIGURATION_USED` self._default_config = self.filter_runnable_config(kind, {}) #: This attributes holds configuration from Avocado proper #: that is passed to runners, as long as a runner declares #: its interest in using them with #: attr:`avocado.core.nrunner.runner.BaseRunner.CONFIGURATION_USED` self.config = config or {} self.args = args self.tags = kwargs.pop("tags", None) self.dependencies = self.read_dependencies(kwargs.pop("dependencies", None)) self.variant = kwargs.pop("variant", None) self.output_dir = kwargs.pop("output_dir", None) #: list of (:class:`ReferenceResolutionAssetType`, str) tuples #: expressing assets that the test will require in order to run. self.assets = kwargs.pop("assets", None) self.kwargs = kwargs self._identifier = identifier def __repr__(self): fmt = ( '' ) return fmt.format( self.kind, self.uri, self.config, self.args, self.kwargs, self.tags, self.dependencies, self.variant, ) @property def identifier(self): """Runnable identifier respecting user's format string. This is still experimental and we have room for improvements. This property it will return an unique identifier for this runnable. Please use this property in order to respect user's customization. By default runnables has its '{uri}' as identifier. Custom formatter can be configured and currently we accept the following values as normal f-strings replacements: {uri}, {args}, and {kwargs}. "args" and "kwargs" are special cases. For args, since it is a list, you can use in two different ways: "{args}" for the entire list, or "{args[n]}" for a specific element inside this list. The same is valid when using "{kwargs}". With kwargs, since it is a dictionary, you have to specify a key as index and then the values are used. For instance if you have a kwargs value named 'DEBUG', a valid usage could be: "{kwargs[DEBUG]}" and this will print the current value to this variable (i.e: True or False). Since this is formatter, combined values can be used. Example: "{uri}-{args}". """ if not self._identifier: fmt = self.config.get("runner.identifier_format", "{uri}") # Optimize for the most common scenario if fmt == "{uri}": return self.uri # For args we can use the entire list of arguments or with a specific # index. args = "-".join(self.args) if "args" in fmt and "[" in fmt: args = self.args # For kwargs we can use the entire list of values or with a specific # index. kwargs = "-".join(str(self.kwargs.values())) if "kwargs" in fmt and "[" in fmt: kwargs = self.kwargs options = {"uri": self.uri, "args": args, "kwargs": kwargs} self._identifier = fmt.format(**options) return self._identifier @property def config(self): if not self._config: return self._default_config config_with_defaults = copy.copy(self._default_config) config_with_defaults.update(self._config) return config_with_defaults @property def default_config(self): return self._default_config def _config_setter_warning(self, config, default_config=False): configuration_used = Runnable.get_configuration_used_by_kind(self.kind) if default_config: if set(configuration_used) == (set(config.keys())): return else: if set(config.keys()).issubset(set(configuration_used)): return LOG.warning( "The runnable config should have only values " "essential for its runner. In the next version of " "avocado, this will raise a ValueError. Please " "use avocado.core.nrunner.runnable.Runnable.filter_runnable_config" ) @config.setter def config(self, config): """Sets the config values based on the runnable kind. This is not avocado config, it is a runnable config which is a subset of avocado config based on `STANDALONE_EXECUTABLE_CONFIG_USED` which describes essential configuration values for each runner kind. :param config: A config dict with new values for Runnable. :type config: dict """ self._config_setter_warning(config) self._config = config @default_config.setter def default_config(self, config): """Sets the default config values based on the runnable kind. This is not avocado config, it is a runnable config which is a subset of avocado config based on `STANDALONE_EXECUTABLE_CONFIG_USED` which describes essential configuration values for each runner kind. These values are used as convenience if other values are not set in the actual :attr:`config` itself. :param config: A config dict with default values for this Runnable. :type config: dict """ self._config_setter_warning(config, True) self._default_config = config @classmethod def from_args(cls, args): """Returns a runnable from arguments""" decoded_args = [_arg_decode_base64(arg) for arg in args.get("arg", ())] return cls( args.get("kind"), args.get("uri"), *decoded_args, config=json.loads(args.get("config", "{}"), cls=ConfigDecoder), **_key_val_args_to_kwargs(args.get("kwargs", [])), ) @staticmethod def _validate_recipe_json_schema(recipe): """Attempts to validate the runnable recipe using a JSON schema :param recipe: the recipe already parsed from JSON into a dict :type recipe: dict :returns: whether the runnable recipe JSON was attempted to be validated with a JSON schema :rtype: bool :raises: RunnableRecipeInvalidError if the recipe is invalid """ if not JSONSCHEMA_AVAILABLE: return False schema_filename = "runnable-recipe.schema.json" schema_path = pkg_resources.resource_filename( "avocado", os.path.join("schemas", schema_filename) ) if not os.path.exists(schema_path): schema_path = os.path.join(SYSTEM_WIDE_SCHEMA_PATH, schema_filename) if not os.path.exists(schema_path): return False with open(schema_path, "r", encoding="utf-8") as schema: try: jsonschema.validate(recipe, json.load(schema)) except jsonschema.exceptions.ValidationError as details: raise RunnableRecipeInvalidError(details) return True @classmethod def _validate_recipe(cls, recipe): """Validates a recipe using either JSON schema or builtin logic :param recipe: the recipe already parsed from JSON into a dict :type recipe: dict :returns: None :raises: RunnableRecipeInvalidError if the recipe is invalid """ if not cls._validate_recipe_json_schema(recipe): # This is a simplified validation of the recipe allowed = set(["kind", "uri", "args", "kwargs", "config", "identifier"]) if not "kind" in recipe: raise RunnableRecipeInvalidError('Missing required property "kind"') if not set(recipe.keys()).issubset(allowed): raise RunnableRecipeInvalidError( "Additional properties are not allowed" ) @classmethod def from_dict(cls, recipe_dict): """ Returns a runnable from a runnable dictionary :param recipe_dict: a dictionary with runnable keys and values :rtype: instance of :class:`Runnable` """ cls._validate_recipe(recipe_dict) config = ConfigDecoder.decode_set(recipe_dict.get("config", {})) return cls( recipe_dict.get("kind"), recipe_dict.get("uri"), *recipe_dict.get("args", ()), config=config, identifier=recipe_dict.get("identifier"), **recipe_dict.get("kwargs", {}), ) @classmethod def from_recipe(cls, recipe_path): """ Returns a runnable from a runnable recipe file :param recipe_path: Path to a recipe file :rtype: instance of :class:`Runnable` """ with open(recipe_path, encoding="utf-8") as recipe_file: recipe_dict = json.load(recipe_file) return cls.from_dict(recipe_dict) @classmethod def from_avocado_config( cls, kind, uri, *args, config=None, identifier=None, **kwargs ): """Creates runnable with only essential config for runner of specific kind.""" warnings.warn( "from_avocado_config() is deprecated, please use the regular " "class initialization as it has the same behavior.", DeprecationWarning, ) return cls(kind, uri, *args, config=config, identifier=identifier, **kwargs) @classmethod def get_configuration_used_by_kind(cls, kind): """Returns the configuration used by a runner of a given kind :param kind: Kind of runner which should use the configuration. :type kind: str :returns: the configuration used by a runner of a given kind :rtype: list """ configuration_used = [] klass = cls.pick_runner_class_from_entry_point_kind(kind) if klass is not None: configuration_used = klass.CONFIGURATION_USED else: command = Runnable.pick_runner_command(kind) if command is not None: command = " ".join(command) configuration_used = STANDALONE_EXECUTABLE_CONFIG_USED.get(command) return configuration_used + CONFIGURATION_USED @classmethod def filter_runnable_config(cls, kind, config): """ Returns only essential values for specific runner. It will use configuration from argument completed by values from config file and avocado default configuration. :param kind: Kind of runner which should use the configuration. :type kind: str :param config: Configuration values for runner. If some values will be missing the default ones and from config file will be used. :type config: dict :returns: Config dict, which has only values essential for runner based on STANDALONE_EXECUTABLE_CONFIG_USED :rtype: dict """ whole_config = settings.as_dict() filtered_config = {} config_items = cls.get_configuration_used_by_kind(kind) for config_item in config_items: filtered_config[config_item] = config.get( config_item, whole_config.get(config_item) ) return filtered_config def read_dependencies(self, dependencies_dict): """ Converts dependencies from json to avocado.core.dependencies.dependency.Dependency :param dependencies: Runnable dependencies :type dependencies: list of dict, or list of Dependency :returns: Runnable dependencies in avocado.core.dependencies.dependency.Dependency format. :rtype: list of Dependency """ if isinstance(dependencies_dict, list): return list( map( lambda d: ( Dependency.from_dictionary(d) if isinstance(d, dict) else d ), dependencies_dict, ) ) def get_command_args(self): """ Returns the command arguments that adhere to the runner interface This is useful for building 'runnable-run' and 'task-run' commands that can be executed on a command line interface. :returns: the arguments that can be used on an avocado-runner command :rtype: list """ args = ["-k", self.kind] if self.uri is not None: args.append("-u") args.append(self.uri) if self.config: args.append("-c") args.append(json.dumps(self.config, cls=ConfigEncoder)) for arg in self.args: args.append("-a") if arg.startswith("-"): arg = f"base64:{base64.b64encode(arg.encode()).decode('ascii')}" args.append(arg) if self.tags is not None: args.append(f"tags=json:{json.dumps(self.get_serializable_tags())}") if self.variant is not None: args.append(f"variant=json:{json.dumps(self.variant)}") if self.output_dir is not None: args.append(f"output_dir={self.output_dir}") for key, val in self.kwargs.items(): if not isinstance(val, str) or isinstance(val, int): val = f"json:{json.dumps(val)}" args.append(f"{key}={val}") return args def get_dict(self): """ Returns a dictionary representation for the current runnable This is usually the format that will be converted to a format that can be serialized to disk, such as JSON. :rtype: :class:`collections.OrderedDict` """ recipe = collections.OrderedDict(kind=self.kind) if self.uri is not None: recipe["uri"] = self.uri recipe["config"] = self.config recipe["identifier"] = self.identifier if self.args is not None: recipe["args"] = self.args kwargs = self.kwargs.copy() if self.tags is not None: kwargs["tags"] = self.get_serializable_tags() if self.variant is not None: kwargs["variant"] = self.variant if self.output_dir is not None: kwargs["output_dir"] = self.output_dir if kwargs: recipe["kwargs"] = kwargs return recipe def get_json(self): """ Returns a JSON representation :rtype: str """ return json.dumps(self.get_dict(), cls=ConfigEncoder) def get_serializable_tags(self): if self.tags is None: return {} tags = {} # sets are not serializable in json for key, val in self.tags.items(): if isinstance(val, set): val = list(val) tags[key] = val return tags def write_json(self, recipe_path): """ Writes a file with a JSON representation (also known as a recipe) """ with open(recipe_path, "w", encoding="utf-8") as recipe_file: recipe_file.write(self.get_json()) @staticmethod def get_capabilities_from_runner_command(runner_command, env=None): """Returns the capabilities of a given runner from a command. In case of failures, an empty capabilities dictionary is returned. When the capabilities are obtained, it also updates the :data:`STANDALONE_EXECUTABLE_CONFIG_USED` info. """ cmd = runner_command + ["capabilities"] try: process = subprocess.Popen( cmd, stdin=subprocess.DEVNULL, stdout=subprocess.PIPE, stderr=subprocess.DEVNULL, env=env, ) except (FileNotFoundError, PermissionError): return {} out, _ = process.communicate() try: capabilities = json.loads(out.decode()) except json.decoder.JSONDecodeError: capabilities = {} # lists are not hashable, and here it'd make more sense to have # a command as it'd be seen in a command line anyway cmd = " ".join(runner_command) if cmd not in STANDALONE_EXECUTABLE_CONFIG_USED: STANDALONE_EXECUTABLE_CONFIG_USED[cmd] = capabilities.get( "configuration_used", [] ) return capabilities @staticmethod def is_kind_supported_by_runner_command( kind, runner_cmd, capabilities=None, env=None ): """Checks if a runner command that seems a good fit declares support.""" if capabilities is None: capabilities = Runnable.get_capabilities_from_runner_command( runner_cmd, env ) return kind in capabilities.get("runnables", []) @staticmethod def pick_runner_command(kind, runners_registry=None): """Selects a runner command based on the runner kind. And when finding a suitable runner, keeps found runners in registry. This utility function will look at the given kind and try to find a matching runner. The matching runner probe results are kept in a registry (that is modified by this function) so that further executions take advantage of previous probes. This is related to the :data:`SpawnMethod.STANDALONE_EXECUTABLE` :param kind: runners' kind :type kind: str :param runners_registry: a registry with previously found (and not found) runners keyed by runnable kind :type runners_registry: dict :returns: command line arguments to execute the runner :rtype: list of str or None """ if runners_registry is None: runners_registry = RUNNERS_REGISTRY_STANDALONE_EXECUTABLE runner_cmd = runners_registry.get(kind) if runner_cmd is False: return None if runner_cmd is not None: return runner_cmd standalone_executable_cmd = [f"avocado-runner-{kind}"] if Runnable.is_kind_supported_by_runner_command( kind, standalone_executable_cmd ): runners_registry[kind] = standalone_executable_cmd return standalone_executable_cmd # attempt to find Python module files that are named after the # runner convention within the avocado.plugins.runners namespace dir. # Looking for the file only avoids an attempt to load the module # and should be a lot faster module_name = Runnable.pick_runner_module_from_entry_point_kind(kind) if module_name is not None: candidate_cmd = [sys.executable, "-m", module_name] if Runnable.is_kind_supported_by_runner_command( kind, candidate_cmd, env=get_python_path_env_if_egg() ): runners_registry[kind] = candidate_cmd return candidate_cmd # exhausted probes, let's save the negative on the cache and avoid # future similar problems runners_registry[kind] = False def runner_command(self, runners_registry=None): """Selects a runner command based on the runner. And when finding a suitable runner, keeps found runners in registry. This utility function will look at the given task and try to find a matching runner. The matching runner probe results are kept in a registry (that is modified by this function) so that further executions take advantage of previous probes. This is related to the :data:`SpawnMethod.STANDALONE_EXECUTABLE` :param runners_registry: a registry with previously found (and not found) runners keyed by runnable kind :type runners_registry: dict :returns: command line arguments to execute the runner :rtype: list of str or None """ return Runnable.pick_runner_command(self.kind, runners_registry) @staticmethod def pick_runner_module_from_entry_point_kind(kind): """Selects a runner module from entry points based on kind. This is related to the :data:`SpawnMethod.STANDALONE_EXECUTABLE`. The module found (if any) will be one that can be used with the Python interpreter using the "python -m $module" command. :param kind: Kind of runner :type kind: str :returns: a module that can be run with "python -m" or None""" namespace = "console_scripts" section = f"avocado-runner-{kind}" for ep in pkg_resources.iter_entry_points(namespace, section): return ep.module_name @staticmethod def pick_runner_class_from_entry_point_kind(kind): """Selects a runner class from entry points based on kind. This is related to the :data:`SpawnMethod.PYTHON_CLASS`. :param kind: Kind of runner :type kind: str :returns: a class that inherits from :class:`BaseRunner` or None """ namespace = "avocado.plugins.runnable.runner" for ep in pkg_resources.iter_entry_points(namespace, kind): try: obj = ep.load() return obj except ImportError: return def pick_runner_class_from_entry_point(self): """Selects a runner class from entry points based on kind. This is related to the :data:`SpawnMethod.PYTHON_CLASS`. :returns: a class that inherits from :class:`BaseRunner` or None """ return Runnable.pick_runner_class_from_entry_point_kind(self.kind) def pick_runner_class(self): """Selects a runner class from the registry based on kind. This is related to the :data:`SpawnMethod.PYTHON_CLASS` :returns: a class that inherits from :class:`BaseRunner` :raises: ValueError if kind there's no runner from kind of runnable """ runner = self.pick_runner_class_from_entry_point() if runner is not None: return runner raise ValueError(f"Unsupported kind of runnable: {self.kind}") avocado-113.0/avocado/core/nrunner/runner.py000066400000000000000000000063631513561306000210600ustar00rootroot00000000000000import time from avocado.core.nrunner.runnable import RUNNERS_REGISTRY_STANDALONE_EXECUTABLE from avocado.core.plugin_interfaces import RunnableRunner #: The amount of time (in seconds) between each internal status check RUNNER_RUN_CHECK_INTERVAL = 0.01 #: The amount of time (in seconds) between a status report from a #: runner that performs its work asynchronously RUNNER_RUN_STATUS_INTERVAL = 0.5 def check_runnables_runner_requirements(runnables, runners_registry=None): """ Checks if runnables have runner requirements fulfilled :param runnables: the tasks whose runner requirements will be checked :type runnables: list of :class:`Runnable` :param runners_registry: a registry with previously found (and not found) runners keyed by a task's runnable kind. Defaults to :attr:`RUNNERS_REGISTRY_STANDALONE_EXECUTABLE` :type runners_registry: dict :return: two list of tasks in a tuple, with the first being the tasks that pass the requirements check and the second the tasks that fail the requirements check :rtype: tuple of (list, list) """ if runners_registry is None: runners_registry = RUNNERS_REGISTRY_STANDALONE_EXECUTABLE ok = [] missing = [] for runnable in runnables: runner = runnable.pick_runner_command(runnable.kind, runners_registry) if runner: ok.append(runnable) else: missing.append(runnable) return (ok, missing) class BaseRunner(RunnableRunner): #: The "main Avocado" configuration keys (AKA namespaces) that #: this runners makes use of. CONFIGURATION_USED = [] @staticmethod def prepare_status(status_type, additional_info=None): """Prepare a status dict with some basic information. This will add the keyword 'status' and 'time' to all status. :param: status_type: The type of event ('started', 'running', 'finished') :param: addional_info: Any additional information that you would like to add to the dict. This must be a dict. :rtype: dict """ status = {} if isinstance(additional_info, dict): status = additional_info status.update({"status": status_type, "time": time.monotonic()}) return status def running_loop(self, condition): """Produces timely running messages until end condition is found. :param condition: a callable that will be evaluated as a condition for continuing the loop """ most_current_execution_state_time = None while not condition(): now = time.monotonic() if most_current_execution_state_time is not None: next_execution_state_mark = ( most_current_execution_state_time + RUNNER_RUN_STATUS_INTERVAL ) if ( most_current_execution_state_time is None or now > next_execution_state_mark ): most_current_execution_state_time = now yield self.prepare_status("running") time.sleep(RUNNER_RUN_CHECK_INTERVAL) avocado-113.0/avocado/core/nrunner/task.py000066400000000000000000000205261513561306000205060ustar00rootroot00000000000000import base64 import json import logging import socket import tempfile import time from uuid import uuid1 from avocado.core.nrunner.runnable import ( RUNNERS_REGISTRY_STANDALONE_EXECUTABLE, Runnable, ) LOG = logging.getLogger(__name__) #: The default category for tasks, and the value that will cause the #: task results to be included in the job results TASK_DEFAULT_CATEGORY = "test" class StatusEncoder(json.JSONEncoder): # pylint: disable=E0202 def default(self, o): if isinstance(o, bytes): return {"__base64_encoded__": base64.b64encode(o).decode("ascii")} return json.JSONEncoder.default(self, o) def json_dumps(data): return json.dumps(data, ensure_ascii=True, cls=StatusEncoder) class TaskStatusService: """ Implementation of interface that a task can use to post status updates TODO: make the interface generic and this just one of the implementations """ def __init__(self, uri): self.uri = uri self._connection = None @property def connection(self): if not self._connection: self._create_connection() return self._connection def _create_connection(self): """ Creates connection with `self.uri` based on `socket.create_connection` """ if ":" in self.uri: host, port = self.uri.split(":") port = int(port) for _ in range(600): try: self._connection = socket.create_connection((host, port)) break except ConnectionRefusedError as error: LOG.warning(error) time.sleep(1) else: self._connection = socket.create_connection((host, port)) else: self._connection = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) self._connection.connect(self.uri) def post(self, status): data = json_dumps(status) try: self.connection.send(data.encode("ascii") + "\n".encode("ascii")) except BrokenPipeError: try: self._create_connection() self.connection.send(data.encode("ascii") + "\n".encode("ascii")) except ConnectionRefusedError: LOG.warning(f"Connection with {self.uri} has been lost.") return False return True def close(self): if self.connection is not None: self.connection.close() def __repr__(self): return f'' class Task: """ Wraps the execution of a runnable While a runnable describes what to be run, and gets run by a runner, a task should be a unique entity to track its state, that is, whether it is pending, is running or has finished. """ def __init__( self, runnable, identifier=None, status_uris=None, category=TASK_DEFAULT_CATEGORY, job_id=None, ): """Instantiates a new Task. :param runnable: the "description" of what the task should run. :type runnable: :class:`avocado.core.nrunner.Runnable` :param identifier: any identifier that is guaranteed to be unique within the context of a Job. A recommended value is a :class:`avocado.core.test_id.TestID` instance when a task represents a test, because besides the uniqueness aspect, it's also descriptive. If an identifier is not given, an automatically generated one will be set. :param status_uris: the URIs for the status servers that this task should send updates to. :type status_uris: list :param category: category of this task. Defaults to :data:`TASK_DEFAULT_CATEGORY`. :type category: str :param job_id: the ID of the job, for authenticating messages that get sent to the destination job's status server and will make into the job's results. :type job_id: str """ # pylint: disable=W0201 self.runnable = runnable self.identifier = identifier or str(uuid1()) #: Category of the task. If the category is not "test", it #: will not be accounted for on a Job's test results. self.category = category self.job_id = job_id self.status_services = [] status_uris = status_uris or self.runnable.config.get("run.status_server_uri") if status_uris is not None: if not isinstance(status_uris, list): status_uris = [status_uris] for status_uri in status_uris: self.status_services.append(TaskStatusService(status_uri)) self.metadata = {} def __repr__(self): fmt = ( '' ) return fmt.format( self.identifier, self.runnable, self.status_services, self.category, self.job_id, ) def are_dependencies_available(self, runners_registry=None): """Verifies if dependencies needed to run this task are available. This currently checks the runner command only, but can be expanded once the handling of other types of dependencies are implemented. See :doc:`/blueprints/BP002`. """ if runners_registry is None: runners_registry = RUNNERS_REGISTRY_STANDALONE_EXECUTABLE return self.runnable.runner_command(runners_registry) def setup_output_dir(self, output_dir=None): if not self.runnable.output_dir: output_dir = output_dir or tempfile.mkdtemp(prefix=".avocado-task-") self.runnable.output_dir = output_dir @classmethod def from_recipe(cls, task_path): """ Creates a task (which contains a runnable) from a task recipe file :param task_path: Path to a recipe file :rtype: instance of :class:`Task` """ with open(task_path, encoding="utf-8") as recipe_file: recipe = json.load(recipe_file) identifier = recipe.get("id") runnable_recipe = recipe.get("runnable") runnable = Runnable( runnable_recipe.get("kind"), runnable_recipe.get("uri"), *runnable_recipe.get("args", ()), config=runnable_recipe.get("config"), ) status_uris = recipe.get("status_uris") category = recipe.get("category") return cls(runnable, identifier, status_uris, category) def get_command_args(self): """ Returns the command arguments that adhere to the runner interface This is useful for building 'task-run' commands that can be executed on a command line interface. :returns: the arguments that can be used on an avocado-runner command :rtype: list """ args = ["-i", str(self.identifier), "-j", str(self.job_id)] args += self.runnable.get_command_args() for status_service in self.status_services: args.append("-s") args.append(status_service.uri) return args def run(self): self.setup_output_dir() runner_klass = self.runnable.pick_runner_class() runner = runner_klass() running_status_services = self.status_services damaged_status_services = [] for status in runner.run(self.runnable): if status["status"] == "started": status.update({"output_dir": self.runnable.output_dir}) status.update({"id": self.identifier}) if self.job_id is not None: status.update({"job_id": self.job_id}) for status_service in running_status_services: if not status_service.post(status): damaged_status_services.append(status_service) if damaged_status_services: running_status_services = list( filter( lambda s: s not in damaged_status_services, running_status_services, ) ) damaged_status_services.clear() yield status avocado-113.0/avocado/core/output.py000066400000000000000000000551551513561306000174230ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2013-2014 # Author: Lucas Meneghel Rodrigues """ Manages output and logging in avocado applications. """ import errno import logging import logging.handlers import os import re import sys import traceback from avocado.core import exit_codes from avocado.core.settings import settings from avocado.core.streams import BUILTIN_STREAMS from avocado.utils import path as utils_path #: Handle cases of logging exceptions which will lead to recursion error logging.raiseExceptions = False #: Pre-defined Avocado root logger LOG_ROOT = logging.getLogger("avocado") #: Pre-defined Avocado human UI logger LOG_UI = logging.getLogger("avocado.app") #: Pre-defined Avocado job/test logger LOG_JOB = logging.getLogger("avocado.job") class TermSupport: COLOR_BLUE = "\033[94m" COLOR_GREEN = "\033[92m" COLOR_YELLOW = "\033[93m" COLOR_RED = "\033[91m" COLOR_DARKGREY = "\033[90m" CONTROL_END = "\033[0m" MOVE_BACK = "\033[1D" MOVE_FORWARD = "\033[1C" ESCAPE_CODES = [ COLOR_BLUE, COLOR_GREEN, COLOR_YELLOW, COLOR_RED, COLOR_DARKGREY, CONTROL_END, MOVE_BACK, MOVE_FORWARD, ] """ Class to help applications to colorize their outputs for terminals. This will probe the current terminal and colorize output only if the stdout is in a tty or the terminal type is recognized. """ def __init__(self): self.HEADER = self.COLOR_BLUE self.PASS = self.COLOR_GREEN self.SKIP = self.COLOR_YELLOW self.FAIL = self.COLOR_RED self.INTERRUPT = self.COLOR_RED self.ERROR = self.COLOR_RED self.WARN = self.COLOR_YELLOW self.CANCEL = self.COLOR_YELLOW self.PARTIAL = self.COLOR_YELLOW self.ENDC = self.CONTROL_END self.LOWLIGHT = self.COLOR_DARKGREY self.enabled = True allowed_terms = [ "linux", "rxvt-unicode", "rxvt-unicode-256color", "screen", "screen-256color", "screen.xterm-256color", "vt100", "xterm", "xterm-256color", "xterm-kitty", ] term = os.environ.get("TERM") config = settings.as_dict() colored = config.get("runner.output.colored") force_color = config.get("runner.output.color") if force_color == "never": self.disable() elif force_color == "auto": if not colored or not os.isatty(1) or term not in allowed_terms: self.disable() elif force_color != "always": raise ValueError( f"The value for runner.output.color must be one of " f"'always', 'never', 'auto' and not {force_color}" ) def disable(self): """ Disable colors from the strings output by this class. """ self.enabled = False self.HEADER = "" self.PASS = "" self.SKIP = "" self.FAIL = "" self.INTERRUPT = "" self.ERROR = "" self.WARN = "" self.CANCEL = "" self.PARTIAL = "" self.ENDC = "" self.LOWLIGHT = "" self.MOVE_BACK = "" self.MOVE_FORWARD = "" def header_str(self, msg): """ Print a header string (blue colored). If the output does not support colors, just return the original string. """ return f"{self.HEADER}{msg}{self.ENDC}" def fail_header_str(self, msg): """ Print a fail header string (red colored). If the output does not support colors, just return the original string. """ return f"{self.FAIL}{msg}{self.ENDC}" def warn_header_str(self, msg): """ Print a warning header string (yellow colored). If the output does not support colors, just return the original string. """ return f"{self.WARN}{msg}{self.ENDC}" def healthy_str(self, msg): """ Print a healthy string (green colored). If the output does not support colors, just return the original string. """ return f"{self.PASS}{msg}{self.ENDC}" def partial_str(self, msg): """ Print a string that denotes partial progress (yellow colored). If the output does not support colors, just return the original string. """ return f"{self.PARTIAL}{msg}{self.ENDC}" def pass_str(self, msg="PASS", move=MOVE_BACK): """ Print a pass string (green colored). If the output does not support colors, just return the original string. """ return f"{move}{self.PASS}{msg}{self.ENDC}" def skip_str(self, msg="SKIP", move=MOVE_BACK): """ Print a skip string (yellow colored). If the output does not support colors, just return the original string. """ return f"{move}{self.SKIP}{msg}{self.ENDC}" def fail_str(self, msg="FAIL", move=MOVE_BACK): """ Print a fail string (red colored). If the output does not support colors, just return the original string. """ return f"{move}{self.FAIL}{msg}{self.ENDC}" def error_str(self, msg="ERROR", move=MOVE_BACK): """ Print a error string (red colored). If the output does not support colors, just return the original string. """ return f"{move}{self.ERROR}{msg}{self.ENDC}" def interrupt_str(self, msg="INTERRUPT", move=MOVE_BACK): """ Print an interrupt string (red colored). If the output does not support colors, just return the original string. """ return f"{move}{self.INTERRUPT}{msg}{self.ENDC}" def warn_str(self, msg="WARN", move=MOVE_BACK): """ Print an warning string (yellow colored). If the output does not support colors, just return the original string. """ return f"{move}{self.WARN}{msg}{self.ENDC}" #: Transparently handles colored terminal, when one is used TERM_SUPPORT = TermSupport() #: A collection of mapping from test statuses to colors to be used #: consistently across the various plugins TEST_STATUS_MAPPING = { "PASS": TERM_SUPPORT.PASS, "ERROR": TERM_SUPPORT.ERROR, "FAIL": TERM_SUPPORT.FAIL, "SKIP": TERM_SUPPORT.SKIP, "WARN": TERM_SUPPORT.WARN, "INTERRUPTED": TERM_SUPPORT.INTERRUPT, "CANCEL": TERM_SUPPORT.CANCEL, } #: A collection of mapping from test status to formatting functions #: to be used consistently across the various plugins TEST_STATUS_DECORATOR_MAPPING = { "PASS": TERM_SUPPORT.pass_str, "ERROR": TERM_SUPPORT.error_str, "FAIL": TERM_SUPPORT.fail_str, "SKIP": TERM_SUPPORT.skip_str, "WARN": TERM_SUPPORT.warn_str, "INTERRUPTED": TERM_SUPPORT.interrupt_str, "CANCEL": TERM_SUPPORT.skip_str, } class _StdOutputFile: """ File-like object which stores (_is_stdout, content) into the provided list """ def __init__(self, is_stdout, records): """ :param is_stdout: Is this output stdout or stderr :param records: list to store (is_stdout, written_message) records """ self._is_stdout = is_stdout self._records = records def write(self, msg): """ Record the message """ self._records.append((self._is_stdout, msg)) def writelines(self, iterable): """ Record all messages """ for line in iterable: self.write(line) def close(self): """File-object methods""" def flush(self): """File-object methods""" @staticmethod def isatty(): """File-object methods""" return False def seek(self): """File-object methods""" def tell(self): """File-object methods""" def getvalue(self): """ Get all messages written to this "file" """ return "\n".join((_[1] for _ in self._records if _[0] == self._is_stdout)) class StdOutput: """ Class to modify sys.stdout/sys.stderr """ #: List of records of stored output when stdout/stderr is disabled records = [] def __init__(self): self.stdout = self._stdout = sys.stdout self.stderr = self._stderr = sys.stderr self.__configured = False @staticmethod def _paginator_in_use(): """ :return: True when we output into paginator """ return bool(isinstance(sys.stdout, Paginator)) @property def configured(self): """ Determines if a configuration of any sort has been performed """ return self.__configured def print_records(self): """ Prints all stored messages as they occurred into streams they were produced for. """ try: for stream, msg in self.records: if stream: sys.stdout.write(msg) else: sys.stderr.write(msg) del self.records[:] # IOError due to EPIPE is ignored. That is to avoid having to # handle all IOErrors at the main application loop # indiscriminately. By handling them here, we can be sure # that the failure was due to stdout or stderr not being # connected to an open PIPE. except IOError as detail: if detail.errno != errno.EPIPE: raise def fake_outputs(self): """ Replace sys.stdout/sys.stderr with in-memory-objects """ sys.stdout = _StdOutputFile(True, self.records) sys.stderr = _StdOutputFile(False, self.records) self.__configured = True def enable_outputs(self): """ Enable sys.stdout/sys.stderr (either with 2 streams or with paginator) """ sys.stdout = self.stdout sys.stderr = self.stderr self.__configured = True def enable_paginator(self): """ Enable paginator """ try: paginator = Paginator() except RuntimeError as details: # Paginator not available LOG_UI.error("Failed to enable paginator: %s", details) return self.stdout = self.stderr = paginator self.__configured = True def enable_stderr(self): """ Enable sys.stderr and disable sys.stdout """ sys.stdout = open(os.devnull, "w", encoding="utf-8") sys.stderr = self.stderr self.__configured = True def close(self): """ Enable original sys.stdout/sys.stderr and cleanup """ paginator = None if self._paginator_in_use(): paginator = sys.stdout self.enable_outputs() if paginator: paginator.close() #: Allows modifying the sys.stdout/sys.stderr STD_OUTPUT = StdOutput() def early_start(): """ Replace all outputs with in-memory handlers """ if os.environ.get("AVOCADO_LOG_EARLY"): add_log_handler(LOG_ROOT, logging.StreamHandler, sys.stdout) add_log_handler(LOG_JOB, logging.StreamHandler, sys.stdout) else: STD_OUTPUT.fake_outputs() add_log_handler(LOG_ROOT, MemStreamHandler, None) LOG_ROOT.level = logging.DEBUG CONFIG = [] def del_last_configuration(): if len(CONFIG) == 1: return configuration = CONFIG.pop() for logger_name in configuration: disable_log_handler(logger_name) configuration = CONFIG[-1] for logger_name, handlers in configuration.items(): logger = logging.getLogger(logger_name) for handler in handlers: logger.addHandler(handler) def split_loggers_and_levels(loggers): """Separates logger names and logger levels. :param loggers: Logger names with or without levels :type loggers: List of strings in format STREAM[:LEVEL][,STREAM[:LEVEL][,...]] :yields: Logger name and level :rtype: tuple(logger_name, logger_level) """ for stream_name in loggers: stream_level = re.split(r"(?= logging.WARN class FilterInfoAndLess(logging.Filter): def filter(self, record): return record.levelno <= logging.INFO class FilterTestMessage(logging.Filter): def filter(self, record): return record.module != "messages" and record.funcName != "handle" class FilterTestMessageOnly(logging.Filter): def filter(self, record): return record.module == "messages" and record.funcName == "handle" class ProgressStreamHandler(logging.StreamHandler): """ Handler class that allows users to skip new lines on each emission. """ def emit(self, record): try: msg = self.format(record) if record.levelno < logging.INFO: # Most messages are INFO pass elif record.levelno < logging.WARNING: msg = TERM_SUPPORT.header_str(msg) elif record.levelno < logging.ERROR: msg = TERM_SUPPORT.warn_header_str(msg) else: msg = TERM_SUPPORT.fail_header_str(msg) stream = self.stream skip_newline = False if hasattr(record, "skip_newline"): skip_newline = record.skip_newline stream.write(msg) if not skip_newline: stream.write("\n") self.flush() except (KeyboardInterrupt, SystemExit): # pylint: disable=W0706 raise except Exception: # pylint: disable=W0703 self.handleError(record) class MemStreamHandler(logging.StreamHandler): """ Handler that stores all records in self.log (shared in all instances) """ log = [] def emit(self, record): self.log.append(record) def flush(self): """ This is in-mem object, it does not require flushing """ class Paginator: """ Paginator that uses less to display contents on the terminal. Contains cleanup handling for when user presses 'q' (to quit less). """ def __init__(self): self.pipe = None paginator = os.environ.get("PAGER") if not paginator: try: paginator = f"{utils_path.find_command('less')} -FRX" except utils_path.CmdNotFoundError as details: raise RuntimeError(f"Unable to enable pagination: {details}") self.pipe = os.popen(paginator, "w") def __del__(self): self.close() def close(self): if self.pipe: try: self.pipe.close() except OSError: pass def write(self, msg): if self.pipe: try: self.pipe.write(msg) except (OSError, ValueError): pass def flush(self): if self.pipe and not self.pipe.closed: self.pipe.flush() def add_log_handler( logger, klass=logging.StreamHandler, stream=sys.stdout, level=logging.DEBUG, fmt="%(name)s: %(message)s", handler_filter=None, buffer_size=0, ): """ Add handler to a logger. :param logger_name: the name of a :class:`logging.Logger` instance, that is, the parameter to :func:`logging.getLogger` :param klass: Handler class (defaults to :class:`logging.StreamHandler`) :param stream: Logging stream, to be passed as an argument to ``klass`` (defaults to ``sys.stdout``) :param level: Log level (defaults to `INFO``) :param fmt: Logging format (defaults to ``%(name)s: %(message)s``) :param handler_filter: Logging filter class based on logging.Filter :param buffer_size: whether to add a layer of memory based buffering with a given number of entries. If <= 0, buffering is disabled. """ def save_handler(logger_name, handler): if not CONFIG: CONFIG.append({}) configuration = CONFIG[-1] if logger_name not in configuration: configuration[logger_name] = [] configuration[logger_name].append(handler) if isinstance(logger, str): logger = logging.getLogger(logger) handler = klass(stream) handler.setLevel(level) if isinstance(fmt, str): fmt = logging.Formatter(fmt=fmt) if handler_filter: handler.addFilter(handler_filter) handler.setFormatter(fmt) if handler_filter: handler.addFilter(handler_filter) if klass == logging.FileHandler and buffer_size > 0: buffered_wrapper = logging.handlers.MemoryHandler(buffer_size) buffered_wrapper.setTarget(handler) handler = buffered_wrapper logger.addHandler(handler) save_handler(logger.name, handler) return handler def disable_log_handler(logger): if not logger: return if isinstance(logger, str): logger = logging.getLogger(logger) # Handlers might be reused elsewhere, can't delete them while logger.handlers: logger.handlers.pop() logger.handlers.append(logging.NullHandler()) class Throbber: """ Produces a spinner used to notify progress in the application UI. """ STEPS = ["-", "\\", "|", "/"] # Only print a throbber when we're on a terminal if TERM_SUPPORT.enabled: MOVES = [ f"{TERM_SUPPORT.MOVE_BACK}{STEPS[0]}", f"{TERM_SUPPORT.MOVE_BACK}{STEPS[1]}", f"{TERM_SUPPORT.MOVE_BACK}{STEPS[2]}", f"{TERM_SUPPORT.MOVE_BACK}{STEPS[3]}", ] else: MOVES = ["", "", "", ""] def __init__(self): self.position = 0 def _update_position(self): if self.position == (len(self.MOVES) - 1): self.position = 0 else: self.position += 1 def render(self): result = self.MOVES[self.position] self._update_position() return result def log_plugin_failures(failures): """ Log in the application UI failures to load a set of plugins :param failures: a list of load failures, usually coming from a :class:`avocado.core.dispatcher.Dispatcher` attribute `load_failures` """ msg_fmt = 'Failed to load plugin from module "%s": %s :\n%s' config = settings.as_dict() silenced = config.get("plugins.skip_broken_plugin_notification") for failure in failures: if failure[0].module_name in silenced: continue if hasattr(failure[1], "__traceback__"): str_tb = "".join(traceback.format_tb(failure[1].__traceback__)) else: str_tb = "Traceback not available" LOG_UI.error(msg_fmt, failure[0].module_name, repr(failure[1]), str_tb) avocado-113.0/avocado/core/parameters.py000066400000000000000000000221431513561306000202150ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2014-2017 """ Module related to test parameters """ import logging import re class NoMatchError(KeyError): pass class AvocadoParams: """ Params object used to retrieve params from given path. It supports absolute and relative paths. For relative paths one can define multiple paths to search for the value. It contains compatibility wrapper to act as the original avocado Params, but by special usage you can utilize the new API. See ``get()`` docstring for details. You can also iterate through all keys, but this can generate quite a lot of duplicate entries inherited from ancestor nodes. It shouldn't produce false values, though. """ def __init__(self, leaves, paths, logger_name=None): """ :param leaves: List of TreeNode leaves defining current variant :param paths: list of entry points :param logger_name: the name of a logger to use to record attempts to get parameters :type logger_name: str """ self._rel_paths = [] leaves = list(leaves) for i, path in enumerate(paths): path_leaves = self._get_matching_leaves(path, leaves) self._rel_paths.append(AvocadoParam(path_leaves, f"{int(i)}: {path}")) # Don't use non-mux-path params for relative paths path_leaves = self._get_matching_leaves("/*", leaves) self._abs_path = AvocadoParam(path_leaves, "*: *") self._cache = {} # TODO: Implement something more efficient self._logger_name = logger_name def __eq__(self, other): if set(self.__dict__) != set(other.__dict__): return False for attr in self.__dict__: if getattr(self, attr) != getattr(other, attr): return False return True def __ne__(self, other): return not (self == other) def __repr__(self): return f"" def __str__(self): return f"params {self._str()}" def _str(self): out = ",".join(_.str_leaves_variant for _ in self._rel_paths) if out: return self._abs_path.str_leaves_variant + "," + out else: return self._abs_path.str_leaves_variant def _get_matching_leaves(self, path, leaves): """ Pops and returns list of matching nodes :param path: Path (str) :param leaves: list of TreeNode leaves """ path_re = self._greedy_path_to_re(path) path_leaves = [leaf for leaf in leaves if path_re.search(leaf.path + "/")] for leaf in path_leaves: leaves.remove(leaf) return path_leaves @staticmethod def _greedy_path_to_re(path): """ Converts user-friendly path with asterisk to a regex and compiles it :param path: a more natural, file-system-like/glob-like expression for the paths :type path: builtin.str :returns: a compiled regex """ if not path: return re.compile("^$") if path[-1] == "*": suffix = "" path = path[:-1] else: suffix = "$" return re.compile(path.replace("*", "[^/]*") + suffix) @staticmethod def _is_abspath(path): """Is this an absolute or relative path?""" if path.pattern and path.pattern[0] == "/": return True else: return False def get(self, key, path=None, default=None): """ Retrieve value associated with key from params :param key: Key you're looking for :param path: namespace ['*'] :param default: default value when not found :raise KeyError: In case of multiple different values (params clash) """ if path is None: # default path is any relative path path = "*" try: return self._cache[(key, path, default)] except (KeyError, TypeError): # KeyError - first query # TypeError - unable to hash value = self._get(key, path, default) if self._logger_name is not None: logger = logging.getLogger(self._logger_name) logger.debug( "PARAMS (key=%s, path=%s, default=%s) => %r", key, path, default, value, ) try: self._cache[(key, path, default)] = value except TypeError: pass return value def _get(self, key, path, default): """ Actual params retrieval :param key: key you're looking for :param path: namespace :param default: default value when not found :raise KeyError: In case of multiple different values (params clash) """ path_re = self._greedy_path_to_re(path) for param in self._rel_paths: try: return param.get_or_die(path_re, key) except NoMatchError: pass if self._is_abspath(path_re): try: return self._abs_path.get_or_die(path_re, key) except NoMatchError: pass return default def objects(self, key, path=None): """ Return the names of objects defined using a given key. :param key: The name of the key whose value lists the objects (e.g. 'nics'). """ return self.get(path, key, "").split() def iteritems(self): """ Iterate through all available params and yield origin, key and value of each unique value. """ env = [] for param in self._rel_paths: for path, key, value in param.iteritems(): if (path, key) not in env: env.append((path, key)) yield (path, key, value) for path, key, value in self._abs_path.iteritems(): if (path, key) not in env: env.append((path, key)) yield (path, key, value) class AvocadoParam: """ This is a single slice params. It can contain multiple leaves and tries to find matching results. """ def __init__(self, leaves, name): """ :param leaves: this slice's leaves :param name: this slice's name (identifier used in exceptions) """ # Basic initialization self._leaves = leaves # names cache (leaf.path is quite expensive) self._leaf_names = [leaf.path + "/" for leaf in leaves] self.name = name def __eq__(self, other): if self.__dict__ == other.__dict__: return True else: return False def __ne__(self, other): return not (self == other) @property def str_leaves_variant(self): """String with identifier and all params""" return f"{self.name} ({self._leaf_names})" def _get_leaves(self, path): """ Get all leaves matching the path """ return [ self._leaves[i] for i in range(len(self._leaf_names)) if path.search(self._leaf_names[i]) ] def get_or_die(self, path, key): """ Get a value or raise exception if not present :raise NoMatchError: When no matches :raise KeyError: When value is not certain (multiple matches) """ leaves = self._get_leaves(path) ret = [ (leaf.environment[key], leaf.environment.origin[key]) for leaf in leaves if key in leaf.environment ] if not ret: raise NoMatchError( f"No matches to {path.pattern} => " f" {key} in {self.str_leaves_variant}" ) # make sure all params come from the same origin if len(set([_[1].path for _ in ret])) == 1: return ret[0][0] else: raise ValueError( "Multiple %s leaves contain the key '%s'; %s" # pylint: disable=C0209 % ( path.pattern, key, [ "%s=>%s" % (_[1].path, _[0]) # pylint: disable=C0209 for _ in ret ], ) ) def iteritems(self): """ Very basic implementation which iterates through __ALL__ params, which generates lots of duplicate entries due to inherited values. """ for leaf in self._leaves: for key, value in leaf.environment.items(): yield (leaf.environment.origin[key].path, key, value) avocado-113.0/avocado/core/parser.py000066400000000000000000000217641513561306000173560ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2013-2014 # Author: Ruda Moura """ Avocado application command line parsing. """ import argparse from configparser import ConfigParser, NoOptionError from glob import glob from avocado.core import exit_codes from avocado.core.nrunner.runnable import Runnable from avocado.core.output import LOG_UI from avocado.core.resolver import ReferenceResolution, ReferenceResolutionResult from avocado.core.settings import ConfigFileNotFound, SettingsError, settings from avocado.core.version import VERSION PROG = "avocado" DESCRIPTION = "Avocado Test Runner" class ArgumentParser(argparse.ArgumentParser): """ Class to override argparse functions """ def error(self, message): LOG_UI.debug(self.format_help()) LOG_UI.error("%s: error: %s", self.prog, message) if "unrecognized arguments" in message: LOG_UI.warning( "Perhaps a plugin is missing; run 'avocado" " plugins' to list the installed ones" ) self.exit(exit_codes.AVOCADO_FAIL) def _get_option_tuples(self, option_string): return [] class FileOrStdoutAction(argparse.Action): """ Controls claiming the right to write to the application standard output """ def __call__(self, parser, namespace, values, option_string=None): if values == "-": stdout_claimed_by = getattr(namespace, "stdout_claimed_by", None) if stdout_claimed_by is not None: msg = ( f"Options {stdout_claimed_by} {option_string} " f"are trying to use stdout simultaneously." f" Please set at least one of them to a file to avoid " f"conflicts" ) raise argparse.ArgumentError(self, msg) else: setattr(namespace, "stdout_claimed_by", option_string) setattr(namespace, self.dest, values) class Parser: """ Class to Parse the command line arguments. """ def __init__(self): self.args = argparse.Namespace() self.config = {} self.subcommands = None self.application = ArgumentParser( prog=PROG, add_help=False, description=DESCRIPTION # see parent parsing ) self.application.add_argument( "-v", "--version", action="version", version=f"Avocado {VERSION}" ) self.application.add_argument( "--config", metavar="CONFIG_FILE", nargs="?", help="Use custom configuration from a file", ) help_msg = "Turn the paginator on. Useful when output is too long." settings.register_option( section="core", key="paginator", help_msg=help_msg, key_type=bool, default=False, action="store_true", parser=self.application, long_arg="--enable-paginator", ) help_msg = ( "Some commands can produce more information. This option " "will enable the verbosity when applicable." ) settings.register_option( section="core", key="verbose", help_msg=help_msg, default=False, key_type=bool, parser=self.application, long_arg="--verbose", short_arg="-V", ) settings.add_argparser_to_option( namespace="core.show", parser=self.application, long_arg="--show" ) def start(self): """ Start to parsing arguments. At the end of this method, the support for subparsers is activated. Side effect: update attribute `args` (the namespace). """ self.args, _ = self.application.parse_known_args() # Load settings from file, if user provides one if self.args.config is not None: settings.process_config_path(self.args.config) # Use parent parsing to avoid breaking the output of --help option self.application = ArgumentParser( prog=PROG, description=DESCRIPTION, parents=[self.application] ) # Subparsers where Avocado subcommands are plugged self.subcommands = self.application.add_subparsers( title="subcommands", description="valid subcommands", help="subcommand help", dest="subcommand", ) # On Python 2, required doesn't make a difference because a # subparser is considered an unconsumed positional arguments, # and not providing one will error with a "too few arguments" # message. On Python 3, required arguments are used instead. # Unfortunately, there's no way to pass this as an option when # constructing the sub parsers, but it is possible to set that # option afterwards. self.subcommands.required = True def finish(self): """ Finish the process of parsing arguments. Side effect: set the final value on attribute `config`. """ args, extra = self.application.parse_known_args(namespace=self.args) if extra: msg = f"unrecognized arguments: {' '.join(extra)}" for sub in self.application._subparsers._actions: # pylint: disable=W0212 if sub.dest == "subcommand": sub.choices[self.args.subcommand].error(msg) self.application.error(msg) # from this point on, config is a dictionary based on a argparse.Namespace self.config = vars(args) class HintParser: def __init__(self, filename): self.filename = filename self.config = None self.hints = [] self._parse() def _get_args_from_section(self, section): try: args = self.config.get(section, "args") if args == "$testpath": return [args] return args.split(",") except NoOptionError: return [] def _get_kwargs_from_section(self, section): result = {} kwargs = self.config.get(section, "kwargs", fallback="") for kwarg in kwargs.split(","): if kwarg == "": continue key, value = kwarg.split("=") result[key] = value return result def _get_resolutions_by_kind(self, kind, paths): self.validate_kind_section(kind) resolutions = [] success = ReferenceResolutionResult.SUCCESS config = { "uri": self._get_uri_from_section(kind), "args": self._get_args_from_section(kind), "kwargs": self._get_kwargs_from_section(kind), } for path in paths: uri = config.get("uri") args = config.get("args") kwargs = config.get("kwargs") if uri == "$testpath": uri = path if "$testpath" in args: args = [item.replace("$testpath", path) for item in args] if "$testpath" in kwargs.values(): kwargs = {k: v.replace("$testpath", path) for k, v in kwargs.items()} runnable = Runnable(kind, uri, *args, **kwargs) resolutions.append( ReferenceResolution( reference=path, result=success, resolutions=[runnable], origin=path ) ) return resolutions def _get_uri_from_section(self, section): return self.config.get(section, "uri") def _parse(self): self.config = ConfigParser() config_paths = self.config.read(self.filename) if not config_paths: raise ConfigFileNotFound(self.filename) def get_resolutions(self): """Return a list of resolutions based on the file definitions.""" resolutions = [] for kind in self.config["kinds"]: files = self.config.get("kinds", kind) resolutions.extend(self._get_resolutions_by_kind(kind, glob(files))) return resolutions def validate_kind_section(self, kind): """Validates a specific "kind section". This method will raise a `settings.SettingsError` if any problem is found on the file. :param kind: a string with the specific section. """ if kind not in self.config: msg = "Section {} is not defined. Please check your hint file." raise SettingsError(msg.format(kind)) uri = self._get_uri_from_section(kind) if uri is None: msg = f"uri needs to be defined inside {kind}" raise SettingsError(msg) avocado-113.0/avocado/core/parser_common_args.py000066400000000000000000000037251513561306000217370ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2019 # Author: Cleber Rosa from avocado.core.settings import settings def add_tag_filter_args(parser): section = "filter.by_tags" group = parser.add_argument_group("filtering parameters") settings.register_option( section=section, key="tags", help_msg="Filter tests based on tags", action="append", key_type=list, default=[], metavar="TAGS", parser=group, short_arg="-t", long_arg="--filter-by-tags", allow_multiple=True, ) help_msg = ( "Include all tests without tags during filtering. This " "effectively means they will be kept in the test suite " "found previously to filtering." ) settings.register_option( section=section, key="include_empty", default=False, key_type=bool, help_msg=help_msg, parser=group, long_arg="--filter-by-tags-include-empty", allow_multiple=True, ) help_msg = ( "Include all tests that do not have a matching key in its " "key:val tags. This effectively means those tests will be " "kept in the test suite found previously to filtering." ) settings.register_option( section=section, key="include_empty_key", default=False, key_type=bool, help_msg=help_msg, parser=group, long_arg="--filter-by-tags-include-empty-key", allow_multiple=True, ) avocado-113.0/avocado/core/plugin_interfaces.py000066400000000000000000000417731513561306000215650ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2015 # Author: Cleber Rosa import abc class Plugin(abc.ABC): """Base for all plugins.""" class Init(Plugin): """Base plugin interface for plugins that needs to initialize itself.""" @abc.abstractmethod def initialize(self): """Entry point for the plugin to perform its initialization.""" class Settings(Plugin): """Base plugin to allow modifying settings. Currently it only supports to extend/modify the default list of paths to config files. """ @abc.abstractmethod def adjust_settings_paths(self, paths): """Entry point where plugin can modify the list of configuration paths.""" class CLI(Plugin): """Base plugin interface for adding options (non-commands) to the command line. Plugins that want to add extra options to the core command line application or to sub commands should use the 'avocado.plugins.cli' namespace. """ @abc.abstractmethod def configure(self, parser): """Configures the command line parser with options specific to this plugin.""" @abc.abstractmethod def run(self, config): """Execute any action the plugin intends. Example of action may include activating a special features upon finding that the requested command line options were set by the user. Note: this plugin class is not intended for adding new commands, for that please use `CLICmd`. """ class CLICmd(Plugin): """Base plugin interface for adding new commands to the command line app. Plugins that want to add extensions to the run command should use the 'avocado.plugins.cli.cmd' namespace. """ name = None description = None def configure(self, parser): """Lets the extension add command line options and do early configuration. By default it will register its `name` as the command name and give its `description` as the help message. """ help_msg = self.description if help_msg is None: help_msg = f"Runs the {self.name} command" parser = parser.subcommands.add_parser(self.name, help=help_msg) return parser @abc.abstractmethod def run(self, config): """Entry point for actually running the command.""" class JobPre(Plugin): """Base plugin interface for adding actions before a job runs. Plugins that want to add actions to be run before a job runs, should use the 'avocado.plugins.job.prepost' namespace and implement the defined interface. """ @abc.abstractmethod def pre(self, job): """Entry point for actually running the pre job action.""" class JobPost(Plugin): """Base plugin interface for adding actions after a job runs. Plugins that want to add actions to be run after a job runs, should use the 'avocado.plugins.job.prepost' namespace and implement the defined interface. """ @abc.abstractmethod def post(self, job): """Entry point for actually running the post job action.""" class Result(Plugin): @abc.abstractmethod def render(self, result, job): """Entry point with method that renders the result. This will usually be used to write the result to a file or directory. :param result: the complete job result :type result: :class:`avocado.core.result.Result` :param job: the finished job for which a result will be written :type job: :class:`avocado.core.job.Job` """ class JobPreTests(Plugin): """Base plugin interface for adding actions before a job runs tests. This interface looks similar to :class:`JobPre`, but it's intended to be called at a very specific place, that is, between :meth:`avocado.core.job.Job.create_test_suite` and :meth:`avocado.core.job.Job.run_tests`. """ @abc.abstractmethod def pre_tests(self, job): """Entry point for job running actions before tests execution.""" class JobPostTests(Plugin): """Base plugin interface for adding actions after a job runs tests. Plugins using this interface will run at the a time equivalent to plugins using the :class:`JobPost` interface, that is, at :meth:`avocado.core.job.Job.post_tests`. This is because :class:`JobPost` based plugins will eventually be modified to really run after the job has finished, and not after it has run tests. """ @abc.abstractmethod def post_tests(self, job): """Entry point for job running actions after the tests execution.""" class PreTest(Plugin): """Base plugin interface for adding tasks before a test run. This interface helps to create `avocado.core.nrunner.Task` for running additional actions inside spawner environment before Test. """ @abc.abstractmethod def pre_test_runnables(self, test_runnable, suite_config=None): """Entry point for creating runnables, which will be run before test. :param test_runnable: Runnable of the Test itself. :param suite_config: Configuration dict relevant for the whole suite. :return: PreTest task runnables created by plugin. :rtype: list of :class:`avocado.core.nrunner.Runnable` """ class PostTest(Plugin): """Base plugin interface for adding tasks after a test run. This interface helps to create `avocado.core.nrunner.Task` for running additional actions inside spawner environment after Test. """ @abc.abstractmethod def post_test_runnables(self, test_runnable, suite_config=None): """Entry point for creating runnables, which will be run after test. :param test_runnable: Runnable of the Test itself. :param suite_config: Configuration dict relevant for the whole suite. :return: PostTest task runnables created by plugin. :rtype: list of tuple(:class:`avocado.core.nrunner.Runnable`, `avocado.core.task.runtime.RuntimeTask.possible_dependency_results`) """ class ResultEvents(JobPreTests, JobPostTests): """Base plugin interface for event based (stream-able) results. Plugins that want to add actions to be run after a job runs, should use the 'avocado.plugins.result_events' namespace and implement the defined interface. """ @abc.abstractmethod def start_test(self, result, state): """Event triggered when a test starts running.""" @abc.abstractmethod def test_progress(self, progress=False): """Interface to notify progress (or not) of the running test.""" @abc.abstractmethod def end_test(self, result, state): """Event triggered when a test finishes running.""" class Varianter(Plugin): """Base plugin interface for producing test variants.""" @abc.abstractmethod def __iter__(self): """Yields all variants. The variant is defined as dictionary with at least: * variant_id - name of the current variant * variant - AvocadoParams-compatible variant (usually a list) * paths - default path(s) :yield variant """ @abc.abstractmethod def __len__(self): """Report number of variants.""" @abc.abstractmethod def to_str(self, summary, variants, **kwargs): """Return human readable representation. The summary/variants accepts verbosity where 0 means silent and maximum is up to the plugin. :param summary: How verbose summary to output (int) :param variants: How verbose list of variants to output (int) :param kwargs: Other free-form arguments :rtype: str """ class ResolverMixin: """Common utilities for Resolver implementations.""" def __init__(self, config=None): from avocado.core.settings import settings self.config = config or settings.as_dict() class Resolver(Plugin, ResolverMixin): """Base plugin interface for resolving test references into resolutions.""" @abc.abstractmethod def resolve(self, reference): """Resolves the given reference into a reference resolution. :param reference: a specification that can eventually be resolved into a test (in the form of a :class:`avocado.core.nrunner.Runnable`) :type reference: str :returns: the result of the resolution process, containing the success, failure or error, along with zero or more :class:`avocado.core.nrunner.Runnable` objects :rtype: :class:`avocado.core.resolver.ReferenceResolution` """ class Discoverer(Plugin, ResolverMixin): """Base plugin interface for discovering tests without reference.""" @abc.abstractmethod def discover(self): """Discovers a test resolutions It will be used when the `test.references` variable is empty, but the discoverer will be able to use another data for gathering test resolutions. It work same as the Resolver, but without the test reference. :returns: the result of the resolution process, containing the success, failure or error, along with zero or more :class:`avocado.core.nrunner.Runnable` objects :rtype: :class:`avocado.core.resolver.ReferenceResolution` """ class SuiteRunner(Plugin): """Base plugin interface for runners for suites. This is the interface a job uses to drive the tests execution in a suite. NOTE: This interface is not to be confused with the internal interface or idiosyncrasies of the :ref:`nrunner`. """ @abc.abstractmethod def run_suite(self, job, test_suite): """Run one or more tests and report with test result. :param job: an instance of :class:`avocado.core.job.Job`. :param test_suite: an instance of TestSuite with some tests to run. :return: a set with types of test failures. """ class Spawner(Plugin): """Base plugin interface spawners of tasks. A spawner implementation will spawn Avocado in its intended location, and isolation model. Spawners that run on the same machine as Avocado is originally installed are possible children of this. """ #: The capabilities of a spawner implementation. Possible values #: in :class:`avocado.core.spawners.common.SpawnCapabilities` CAPABILITIES = [] @staticmethod @abc.abstractmethod def is_task_alive(runtime_task): """Determines if a task is alive or not. :param runtime_task: wrapper for a Task with additional runtime information. :type runtime_task: :class:`avocado.core.task.runtime.RuntimeTask` """ @abc.abstractmethod async def spawn_task(self, runtime_task): """Spawns a task return whether the spawning was successful. :param runtime_task: wrapper for a Task with additional runtime information. :type runtime_task: :class:`avocado.core.task.runtime.RuntimeTask` """ def create_task_output_dir(self, runtime_task): """Creates output directory in the intended location, before the task spawn. The output directory path will be saved to the `task.runnable.config` for the usage by runners. :param runtime_task: wrapper for a Task with additional runtime information. :type runtime_task: :class:`avocado.core.task.runtime.RuntimeTask` """ @abc.abstractmethod async def wait_task(self, runtime_task): """Waits for a task to finish. :param runtime_task: wrapper for a Task with additional runtime information. :type runtime_task: :class:`avocado.core.task.runtime.RuntimeTask` """ @abc.abstractmethod async def terminate_task(self, runtime_task): """Terminates a task before finish. :param runtime_task: wrapper for a Task with additional runtime information. :type runtime_task: :class:`avocado.core.task.runtime.RuntimeTask` :returns: whether the task has been fully terminated or not :rtype: bool """ @staticmethod @abc.abstractmethod async def check_task_requirements(runtime_task): """Checks if the requirements described within a task are available. :param runtime_task: wrapper for a Task with additional runtime information. :type runtime_task: :class:`avocado.core.task.runtime.RuntimeTask` """ @staticmethod @abc.abstractmethod async def is_requirement_in_cache(runtime_task): """Checks if it's necessary to run the requirement. There are occasions when the similar requirement has been run and its results are already saved in cache. In such occasion, it is not necessary to run the task again. For example, this might be useful for tasks which would install the same package to the same environment. :param runtime_task: runtime task with requirement :type runtime_task: :class:`avocado.core.task.runtime.RuntimeTask` :return: If the results are already in cache. :rtype: True if task is in cache False if task is not in cache None if task is running in different process and should be in cache soon. """ @staticmethod @abc.abstractmethod async def save_requirement_in_cache(runtime_task): """Saves the information about requirement in cache before the runtime_task is run. :param runtime_task: runtime task with requirement :type runtime_task: :class:`avocado.core.task.runtime.RuntimeTask` """ @staticmethod @abc.abstractmethod async def update_requirement_cache(runtime_task, result): """Updates the information about requirement in cache based on result. :param runtime_task: runtime task with requirement :type runtime_task: :class:`avocado.core.task.runtime.RuntimeTask` :param result: result of runtime_task :type result: `avocado.core.teststatus.STATUSES` """ @abc.abstractmethod def is_operational(self): """Checks whether this spawner is operationally capable to perform. :result: whether or not this spawner is operational on this system, that is, whether it has all its requirements set up and should be ready to operate successfully. :rtype: bool """ class DeploymentSpawner(Spawner): """Spawners that needs basic deployment are based on this class. Spawners that uses any type of isolation model would be a possible children of this. """ @abc.abstractmethod async def deploy_avocado(self, where): """Avocado deployment for the isolated environment. This method should be executed before spawning the task. :param where: handle for where the spawner should deploy avocado. You can use this parameter to pass any valid argument, like a container, vm, or any other isolated environment object or identifier. """ @abc.abstractmethod async def deploy_artifacts(self): """Basic artifacts deployment for the isolated environment. During this stage test references (i.e: mytest.py), data files (i.e: mytest.py.data/), and any other basic requirement should be deployed to the isolated environment. Please keep in mind that test requirements defined at the requirements resolver level are not part of this deployment and it will be fulfilled by the requirement resolver. This method should also be executed before spawning the task. """ class RunnableRunner(Plugin): """Base interface for a nrunner Runner.""" @abc.abstractmethod def run(self, runnable): """Runner main method Yields dictionary as output, containing status as well as relevant information concerning the results. :param runnable: a Runnable instance that describes what is to be run :type runnable: :class:`avocado.core.nrunner.Runnable` """ class Cache(Plugin): """Plugin for manipulating with cache metadata interface""" @abc.abstractmethod def list(self): """List all cache entries Creates list of cache entries which represents current cache state. :return: Human-readable list of entries which will be printed to console. :rtype: str """ @abc.abstractmethod def clear(self): """Clear all cache entries It removes every cache entire. """ avocado-113.0/avocado/core/references.py000066400000000000000000000021751513561306000201760ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2019 # Authors: Lucas Meneghel Rodrigues # Ruda Moura """ Test loader module. """ import os def reference_split(reference): """ Splits a test reference into a path and additional info This should be used dependent on the specific type of resolver. If a resolver is not expected to support multiple test references inside a given file, then this is not suitable. :returns: (path, additional_info) :type: (str, str or None) """ if not os.path.exists(reference): if ":" in reference: return reference.rsplit(":", 1) return (reference, None) avocado-113.0/avocado/core/resolver.py000066400000000000000000000261501513561306000177150ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2019 # Authors: Cleber Rosa """ Test resolver module. """ import glob import os import stat from enum import Enum from avocado.core.enabled_extension_manager import EnabledExtensionManager from avocado.core.exceptions import JobTestSuiteReferenceResolutionError from avocado.core.output import LOG_UI class ReferenceResolutionAssetType(Enum): #: The actual test file. Spawners may use this as the entry point #: to run the tests. Usually only one of this is given, and the #: behavior with either a single or multiple items given is spawner #: dependent. TEST_FILE = "test_file" #: Auxiliary data file placed alongside the test in a ".data" #: directory DATA_FILE = "data_file" class ReferenceResolutionResult(Enum): #: Given test reference was properly resolved SUCCESS = object() #: Given test reference might be resolved, but it is corrupted. CORRUPT = object() #: Given test reference was not properly resolved NOTFOUND = object() #: Internal error in the resolution process ERROR = object() class ReferenceResolutionAction(Enum): #: Stop trying to resolve the reference RETURN = object() #: Continue to resolve the given reference CONTINUE = object() class ReferenceResolution: """ Represents one complete reference resolution Note that the reference itself may result in many resolutions, or none. """ def __init__(self, reference, result, resolutions=None, info=None, origin=None): """ :param reference: a specification that can eventually be resolved into a test (in the form of a :class:`avocado.core.nrunner.Runnable`) :type reference: str :param result: if the complete resolution was a success, failure or error :type result: :class:`ReferenceResolutionResult` :param resolutions: the runnable definitions resulting from the resolution :type resolutions: list of :class:`avocado.core.nrunner.Runnable` :param info: free form information the resolver may add :type info: str :param origin: the name of the resolver that performed the resolution :type origin: str """ self.reference = reference self.result = result if resolutions is None: resolutions = [] self.resolutions = resolutions self.info = info self.origin = origin def __repr__(self): fmt = ( '' ) return fmt.format( self.reference, self.result, self.resolutions, self.info, self.origin ) class Resolver(EnabledExtensionManager): """ Main test reference resolution utility. This performs the actual resolution according to the active resolver plugins and a resolution policy. """ PLUGIN_DESCRIPTION = "Plugins that resolve test references (resolver)" DEFAULT_POLICY = { ReferenceResolutionResult.SUCCESS: ReferenceResolutionAction.RETURN, ReferenceResolutionResult.CORRUPT: ReferenceResolutionAction.RETURN, ReferenceResolutionResult.NOTFOUND: ReferenceResolutionAction.CONTINUE, ReferenceResolutionResult.ERROR: ReferenceResolutionAction.CONTINUE, } def __init__(self, config=None): super().__init__("avocado.plugins.resolver", invoke_kwds={"config": config}) def resolve(self, reference): resolution = [] for ext in self.extensions: try: result = ext.obj.resolve(reference) if not result.origin: result.origin = ext.name except Exception as exc: # pylint: disable=W0703 result = ReferenceResolution( reference, ReferenceResolutionResult.ERROR, info=exc, origin=ext.name, ) resolution.append(result) action = self.DEFAULT_POLICY.get( result.result, ReferenceResolutionAction.CONTINUE ) if action == ReferenceResolutionAction.RETURN: break return resolution class Discoverer(EnabledExtensionManager): """ Secondary test reference resolution utility. When the user didn't provide any test references, Discoverer will discover tests from different data according to active discoverer plugins. """ PLUGIN_DESCRIPTION = "Plugins that discover tests without references (discoverer)" def __init__(self, config=None): super().__init__("avocado.plugins.discoverer", invoke_kwds={"config": config}) def discover(self): resolutions = [] for ext in self.extensions: try: results = ext.obj.discover() except Exception as exc: # pylint: disable=W0703 results = [ ReferenceResolution( "", ReferenceResolutionResult.ERROR, info=exc, origin=ext.name ) ] for result in results: if not result.origin: result.origin = ext.name resolutions.append(result) return resolutions def check_file( path, reference, suffix=".py", type_check=os.path.isfile, type_name="regular file", access_check=os.R_OK, access_name="readable", ): if suffix is not None: if not path.endswith(suffix): return ReferenceResolution( reference, ReferenceResolutionResult.NOTFOUND, info=f'File name "{path}" does not end with suffix "{suffix}"', ) if not type_check(path): return ReferenceResolution( reference, ReferenceResolutionResult.NOTFOUND, info=f'File "{path}" does not exist or is not a {type_name}', ) st = os.stat(path) user_permissions = st.st_mode & (stat.S_IRUSR | stat.S_IWUSR | stat.S_IXUSR) # Initialize required permissions to 0, indicating no permissions are needed yet required_permissions = 0 # Build the required permissions based on access_check if access_check & os.R_OK: # If read access needs to be checked, set the corresponding user read permission bit required_permissions |= stat.S_IRUSR if access_check & os.W_OK: # If write access needs to be checked, set the corresponding user write permission bit required_permissions |= stat.S_IWUSR if access_check & os.X_OK: # If execute access needs to be checked, set the corresponding user execute permission bit required_permissions |= stat.S_IXUSR # Check if the user has the required permissions if (user_permissions & required_permissions) != required_permissions: # If the bitwise AND of user permissions and required permissions is not equal to required permissions, # it means the user is missing some permissions return ReferenceResolution( reference, ReferenceResolutionResult.NOTFOUND, info=f'File "{path}" does not have the required {access_name} permissions', ) return True def get_file_assets(test_file_path): """Gets asset files from the test file and its the ".data" directory :param test_file_path: the filesystem location of the test file :type test_file_path: str :returns: list of tuples with (asset type, path) """ file_assets = [(ReferenceResolutionAssetType.TEST_FILE, test_file_path)] for data_file in glob.glob(os.path.join(f"{test_file_path}.data", "*")): file_assets.append((ReferenceResolutionAssetType.DATA_FILE, data_file)) return file_assets def _extend_directory(path): if not os.path.isdir(path): return [path] paths = [] # no error handling so far for dirpath, dirs, filenames in os.walk(path): dirs.sort() for file_name in sorted(filenames): # does it make sense to ignore hidden files here? if file_name.startswith("."): continue pth = os.path.join(dirpath, file_name) paths.append(pth) if not paths: paths = [path] return paths def resolve(references, hint=None, ignore_missing=True, config=None): resolutions = [] hint_references = {} if hint: hint_references = {r.reference: r for r in hint.get_resolutions()} if not references and hint_references: references = list(hint_references.keys()) if references: # should be initialized with args, to define the behavior # of this instance as a whole resolver = Resolver(config) extended_references = [] for reference in references: # a reference extender is not (yet?) an extensible feature # here it walks directories if one is given, and extends # the original reference into final file paths extended_references.extend(_extend_directory(reference)) for reference in extended_references: if reference in hint_references: resolutions.append(hint_references[reference]) else: resolutions.extend(resolver.resolve(reference)) else: discoverer = Discoverer(config) resolutions.extend(discoverer.discover()) for res in resolutions: if res.result == ReferenceResolutionResult.CORRUPT: LOG_UI.warning( "Reference %s might be resolved by %s resolver, but the file is corrupted: %s", res.reference, res.origin, res.info or "", ) # This came up from a previous method and can be refactored to improve # performance since that we could merge with the loop above. if not ignore_missing: missing = [] for reference in references: results = [res.result for res in resolutions if res.reference == reference] if ReferenceResolutionResult.SUCCESS not in results: missing.append(reference) # directories are automatically expanded, and thus they can # not be considered a reference that needs to exist after the # resolution process missing = [_ for _ in missing if not os.path.isdir(_)] if missing: msg = ( f"No tests found for given test references: {', '.join(missing)}\n" f"Try 'avocado -V list {' '.join(missing)}' for details" ) raise JobTestSuiteReferenceResolutionError(msg) return resolutions avocado-113.0/avocado/core/result.py000066400000000000000000000076451513561306000174020ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2013-2014 # Authors: Lucas Meneghel Rodrigues # Ruda Moura """ Contains the Result class, used for result accounting. """ class Result: """ Result class, holder for job (and its tests) result information. """ def __init__(self, job_unique_id, job_logfile, job_start_date_time=None): """ Creates an instance of Result. :param job_unique_id: the job's unique ID, usually from :attr:`avocado.core.job.Job.unique_id` :param job_logfile: the job's unique ID, usually from :attr:`avocado.core.job.Job.logfile` :param job_start_date_time: the date and time a job started running """ self.job_unique_id = job_unique_id self.logfile = job_logfile self.job_start_date_time = job_start_date_time self.tests_total = 0 self.tests_run = 0 self.tests_total_time = 0.0 self.passed = 0 self.errors = 0 self.failed = 0 self.skipped = 0 self.warned = 0 self.interrupted = 0 self.cancelled = 0 self.tests = [] def _reconcile(self): """ Make sure job results are reconciled In situations such as job interruptions, some test results will be missing, but this is no excuse for giving wrong summaries of test results. """ valid_results_count = ( self.passed + self.errors + self.failed + self.warned + self.skipped + self.interrupted + self.cancelled ) other_skipped_count = self.tests_total - valid_results_count if other_skipped_count > 0: self.skipped += other_skipped_count else: self.tests_total -= other_skipped_count def end_tests(self): """ Called once after all tests are executed. """ self._reconcile() def start_test(self, state): """ Called when the given test is about to run. :param state: result of :class:`avocado.core.test.Test.get_state`. :type state: dict """ def end_test(self, state): """ Called when the given test has been run. :param state: result of :class:`avocado.core.test.Test.get_state`. :type state: dict """ self.tests_run += 1 self.tests_total_time += state.get("time_elapsed", -1) self.tests.append(state) def check_test(self, state): """ Called once for a test to check status and report. :param state: result of :class:`avocado.core.test.Test.get_state`. :type state: dict """ status = state.get("status") if status == "PASS": self.passed += 1 elif status == "SKIP": self.skipped += 1 elif status == "FAIL": self.failed += 1 elif status == "WARN": self.warned += 1 elif status == "INTERRUPTED": self.interrupted += 1 elif status == "CANCEL": self.cancelled += 1 else: self.errors += 1 self.end_test(state) @property def rate(self): total = float(self.tests_total - self.skipped - self.cancelled) if not total: return 0.0 succeeded = float(self.passed + self.warned) return 100 * (succeeded / total) avocado-113.0/avocado/core/safeloader/000077500000000000000000000000001513561306000176035ustar00rootroot00000000000000avocado-113.0/avocado/core/safeloader/__init__.py000066400000000000000000000014361513561306000217200ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2014-2016,2018 # Authors: Cleber Rosa # Lukas Doktor """ Safe (AST based) test loader module utilities """ from avocado.core.safeloader.core import find_avocado_tests, find_python_unittests __all__ = ["find_avocado_tests", "find_python_unittests"] avocado-113.0/avocado/core/safeloader/core.py000066400000000000000000000424151513561306000211130ustar00rootroot00000000000000import ast import collections import sys from importlib.machinery import PathFinder from avocado.core.safeloader.docstring import ( check_docstring_directive, get_docstring_directives, get_docstring_directives_dependencies, get_docstring_directives_tags, ) from avocado.core.safeloader.module import PythonModule def get_methods_info(statement_body, class_tags, class_dependencies): """Returns information on test methods. :param statement_body: the body of a "class" statement :param class_tags: the tags at the class level, to be combined with the tags at the method level. :param class_dependencies: the dependencies at the class level, to be combined with the dependencies at the method level. """ methods_info = [] for st in statement_body: if not (isinstance(st, (ast.FunctionDef, ast.AsyncFunctionDef))): continue decorators = getattr(st, "decorator_list", []) decorator_names = [getattr(_, "id", None) for _ in decorators] if "property" in decorator_names: continue if not st.name.startswith("test"): continue docstring = ast.get_docstring(st) mt_tags = get_docstring_directives_tags(docstring) mt_tags.update(class_tags) mt_dependencies = get_docstring_directives_dependencies(docstring) mt_dependencies.extend(class_dependencies) methods = [method for method, _, _ in methods_info] if st.name not in methods: methods_info.append((st.name, mt_tags, mt_dependencies)) return methods_info def _extend_test_list(current, new): for test in new: test_method_name = test[0] found = False for current_test in current: if test_method_name == current_test[0]: _exted_tests_tags([current_test], test[1]) found = True break if not found: current.append(test) def _exted_tests_tags(tests, tags, force_update=False): for test in tests: for tag, value in tags.items(): if force_update: test[1][tag] = value else: test[1].setdefault(tag, value) def _examine_same_module( parents, info, disabled, match, module, target_module, target_class, determine_match, info_class_tags, ): # Searching the parents in the same module for parent in parents[:]: # Looking for a 'class FooTest(Parent)' if not isinstance(parent, ast.Name): # 'class FooTest(bar.Bar)' not supported within # a module continue parent_class = parent.id # From this point we use `_$variable` to name temporary returns # from method calls that are to-be-assigned/combined with the # existing `$variable`. _info, _disable, parent_tags, _match = _examine_class( target_module, target_class, determine_match, module.path, parent_class, match, ) if _info: parents.remove(parent) _exted_tests_tags(info, parent_tags) _exted_tests_tags(_info, info_class_tags, True) _extend_test_list(info, _info) disabled.update(_disable) if _match is not match: match = _match return match class ClassNotSuitable(Exception): """Exception raised when examination of a class should not proceed.""" def _get_attributes_for_further_examination(parent, module): """Returns path, module and class for further examination. This looks at one of the parents of an interesting class, so for the example class Test below: >>> class Test(unittest.TestCase, MixIn): >>> pass This function should be called twice: once for unittest.TestCase, and once for MixIn. :param parent: parent is one of the possibly many parents from which the class being examined inherits from. :type parent: :class:`ast.Attribute` :param module: PythonModule instance with information about the module being inspected :type module: :class:`avocado.core.safeloader.module.PythonModule` :raises: ClassNotSuitable :returns: a tuple with three values: the class name, the imported symbol instance matching the further examination step, and a hint about the symbol name also being a module. :rtype: tuple of (str, :class:`avocado.core.safeloader.imported.ImportedSymbol`, bool) """ if hasattr(parent, "value"): # A "value" in an "attribute" in this context means that # there's a "module.class" notation. It may be called that # way, because it resembles "class" being an attribute of the # "module" object. In short, if "parent" has a "value" # attribute, it means that this is given as a # "module.parent_class" notation, meaning that: # - parent is a module # - parent.value *should* be a class, because there's # currently no support for the "module.module.parent_class" # notation. See issue #4706. klass = parent.value if not hasattr(klass, "id"): # We don't support multi-level 'parent.parent.Class' raise ClassNotSuitable else: # We know 'parent.Class' or 'asparent.Class' and need # to get path and original_module_name. Class is given # by parent definition. imported_symbol = module.imported_symbols.get(klass.id) if imported_symbol is None: # We can't examine this parent (probably broken module) raise ClassNotSuitable # We currently don't support classes whose parents are generics if isinstance(parent, ast.Subscript): raise ClassNotSuitable parent_class = parent.attr # Special situation: in this case, because we know the parent # class is given as, module.class notation, we know what the # module name is. The imported symbol, because of its knowledge # *only* about the imports, and not about the class definitions, # can not tell if an import is a "from module import other_module" # or a "from module import class" symbol_is_module = klass.id == imported_symbol.symbol_name else: # We only know 'Class' or 'AsClass' and need to get # path, module and original class_name klass = parent.id imported_symbol = module.imported_symbols.get(klass) if imported_symbol is None: # We can't examine this parent (probably broken module) raise ClassNotSuitable parent_class = imported_symbol.symbol symbol_is_module = False return parent_class, imported_symbol, symbol_is_module def _find_import_match(parent_path, parent_module): """Attempts to find an importable module.""" modules_paths = [parent_path] + sys.path found_spec = PathFinder.find_spec(parent_module, modules_paths) if found_spec is None: raise ClassNotSuitable return found_spec def _examine_class( target_module, target_class, determine_match, path, class_name, match ): """ Examine a class from a given path :param target_module: the name of the module from which a class should have come from. When attempting to find a Python unittest, the target_module will most probably be "unittest", as per the standard library module name. When attempting to find Avocado tests, the target_module will most probably be "avocado". :type target_module: str :param target_class: the name of the class that is considered to contain test methods. When attempting to find Python unittests, the target_class will most probably be "TestCase". When attempting to find Avocado tests, the target_class will most probably be "Test". :type target_class: str :param determine_match: a callable that will determine if a match has occurred or not :type determine_match: function :param path: path to a Python source code file :type path: str :param class_name: the specific class to be found :type path: str :param match: whether the inheritance from has been determined or not :type match: bool :returns: tuple where first item is a list of test methods detected for given class; second item is set of class names which look like avocado tests but are force-disabled; third is dict of class tags. :rtype: tuple """ module = PythonModule(path, target_module, target_class) info = [] class_tags = {} disabled = set() for klass in module.iter_classes(class_name): if class_name != klass.name: continue docstring = ast.get_docstring(klass) if match is False: match = module.is_matching_klass(klass) class_tags = get_docstring_directives_tags(docstring) info = get_methods_info( klass.body, class_tags, get_docstring_directives_dependencies(docstring), ) # Getting the list of parents of the current class parents = klass.bases match = _examine_same_module( parents, info, disabled, match, module, target_module, target_class, determine_match, class_tags, ) # If there are parents left to be discovered, they # might be in a different module. for parent in parents: try: ( parent_class, imported_symbol, symbol_is_module, ) = _get_attributes_for_further_examination(parent, module) found_spec = imported_symbol.get_importable_spec(symbol_is_module) if found_spec is None: continue except ClassNotSuitable: continue _info, _disabled, parent_tags, _match = _examine_class( target_module, target_class, determine_match, found_spec.origin, parent_class, match, ) if _info: _exted_tests_tags(info, parent_tags) _exted_tests_tags(_info, class_tags, True) _extend_test_list(info, _info) disabled.update(_disabled) if _match is not match: match = _match if not match and module.interesting_klass_found: imported_symbol = module.imported_symbols[class_name] if imported_symbol: found_spec = imported_symbol.get_importable_spec() if found_spec: _info, _disabled, _class_tags, _match = _examine_class( target_module, target_class, determine_match, found_spec.origin, class_name, match, ) if _info: _exted_tests_tags(info, _class_tags) _exted_tests_tags(_info, class_tags, True) _extend_test_list(info, _info) _class_tags.update(class_tags) class_tags = _class_tags disabled.update(_disabled) if _match is not match: match = _match return info, disabled, class_tags, match def find_python_tests(target_module, target_class, determine_match, path): """ Attempts to find Python tests from source files A Python test in this context is a method within a specific type of class (or that inherits from a specific class). :param target_module: the name of the module from which a class should have come from. When attempting to find a Python unittest, the target_module will most probably be "unittest", as per the standard library module name. When attempting to find Avocado tests, the target_module will most probably be "avocado". :type target_module: str :param target_class: the name of the class that is considered to contain test methods. When attempting to find Python unittests, the target_class will most probably be "TestCase". When attempting to find Avocado tests, the target_class will most probably be "Test". :type target_class: str :type determine_match: a callable that will determine if a given module and class is contains valid Python tests :type determine_match: function :param path: path to a Python source code file :type path: str :returns: tuple where first item is dict with class name and additional info such as method names and tags; the second item is set of class names which look like Python tests but have been forcefully disabled. :rtype: tuple """ module = PythonModule(path, target_module, target_class) # The resulting test classes result = collections.OrderedDict() disabled = set() for klass in module.iter_classes(): docstring = ast.get_docstring(klass) # Looking for a class that has in the docstring either # ":avocado: enable" or ":avocado: disable if check_docstring_directive(docstring, "disable"): disabled.add(klass.name) continue if check_docstring_directive(docstring, "enable"): info = get_methods_info( klass.body, get_docstring_directives_tags(docstring), get_docstring_directives_dependencies(docstring), ) result[klass.name] = info continue # From this point onwards we want to do recursive discovery, but # for now we don't know whether it is avocado.Test inherited # (Ifs are optimized for readability, not speed) # If "recursive" tag is specified, it is forced as test if check_docstring_directive(docstring, "recursive"): match = True else: match = module.is_matching_klass(klass) class_tags = get_docstring_directives_tags(docstring) info = get_methods_info( klass.body, class_tags, get_docstring_directives_dependencies(docstring), ) # Getting the list of parents of the current class parents = klass.bases match = _examine_same_module( parents, info, disabled, match, module, target_module, target_class, determine_match, class_tags, ) # If there are parents left to be discovered, they # might be in a different module. for parent in parents: try: ( parent_class, imported_symbol, symbol_is_module, ) = _get_attributes_for_further_examination(parent, module) found_spec = imported_symbol.get_importable_spec(symbol_is_module) if found_spec is None: continue except ClassNotSuitable: continue _info, _dis, parent_tags, _match = _examine_class( target_module, target_class, determine_match, found_spec.origin, parent_class, match, ) if _info: _exted_tests_tags(info, parent_tags) _exted_tests_tags(_info, class_tags, True) _extend_test_list(info, _info) disabled.update(_dis) if _match is not match: match = _match # Only update the results if this was detected as 'avocado.Test' if match: result[klass.name] = info return result, disabled def _determine_match_python(module, klass, docstring): """ Implements the match check for all Python based test classes Meaning that the enable/disabled/recursive tags are respected for Avocado Instrumented Tests and Python unittests. """ directives = get_docstring_directives(docstring) if "disable" in directives: return False if "enable" in directives: return True if "recursive" in directives: return True # Still not decided, try inheritance return module.is_matching_klass(klass) def find_avocado_tests(path): return find_python_tests("avocado", "Test", _determine_match_python, path) def find_python_unittests(path): found, _ = find_python_tests("unittest", "TestCase", _determine_match_python, path) return found avocado-113.0/avocado/core/safeloader/docstring.py000066400000000000000000000046561513561306000221640ustar00rootroot00000000000000import json import re #: Gets the docstring directive value from a string. Used to tweak #: test behavior in various ways DOCSTRING_DIRECTIVE_RE_RAW = ( r"\s*:avocado:[ \t]+(([a-zA-Z0-9]+?[a-zA-Z0-9_:,\=\-\.]*)|(dependency={.*}))\s*$" ) # the RE will match `:avocado: tags=category` or `:avocado: dependency={}` DOCSTRING_DIRECTIVE_RE = re.compile(DOCSTRING_DIRECTIVE_RE_RAW) def get_docstring_directives(docstring): """ Returns the values of the avocado docstring directives :param docstring: the complete text used as documentation :type docstring: str :rtype: builtin.list """ result = [] if docstring is None: return result for line in docstring.splitlines(): try: match = DOCSTRING_DIRECTIVE_RE.match(line) if match: result.append(match.groups()[0]) except TypeError: pass return result def check_docstring_directive(docstring, directive): """ Checks if there's a given directive in a given docstring :rtype: bool """ return directive in get_docstring_directives(docstring) def get_docstring_directives_tags(docstring): """ Returns the test categories based on a `:avocado: tags=category` docstring :rtype: dict """ result = {} for item in get_docstring_directives(docstring): if item.startswith("tags="): _, comma_tags = item.split("tags=", 1) for tag in comma_tags.split(","): if not tag: continue if ":" in tag: key, val = tag.split(":", 1) if key in result: result[key].add(val) else: result[key] = {val} else: result[tag] = None return result def get_docstring_directives_dependencies(docstring): """ Returns the test dependencies from docstring patterns like `:avocado: dependencies={}`. :rtype: list """ dependencies = [] for item in get_docstring_directives(docstring): if item.startswith("dependency="): _, dependency_str = item.split("dependency=", 1) try: dependencies.append(json.loads(dependency_str)) except json.decoder.JSONDecodeError: # ignore dependencies in case of malformed dictionary continue return dependencies avocado-113.0/avocado/core/safeloader/imported.py000066400000000000000000000240001513561306000217740ustar00rootroot00000000000000import ast import os import sys from importlib.machinery import PathFinder from avocado.core.safeloader.utils import get_statement_import_as class ImportedSymbol: """A representation of an importable symbol. Attributes: module_path : str symbol : str importer_fs_path: str or None """ def __init__( self, module_path, symbol="", importer_fs_path=None, module_alias="", symbol_alias="", ): #: Path from where the symbol was imported. On a statement such as #: "import os", module_path is "os" and there's no symbol. #: On a statement such as from unittest.mock import mock_open", #: the module_path is "unittest.mock". On a statement such as #: "from ..foo import bar", module_path is "..foo" (relative). self.module_path = module_path #: The name of the imported symbol. On a statement such as #: "import os", there's no symbol. On a statement such as #: "from unittest import mock"", the symbol is "mock" (even #: though it may actually also be a module, but it's impossible #: to know for sure). On a statement such as "from unittest.mock #: import mock_open", symbol is "mock_open". self.symbol = symbol #: The full, absolute filesystem path of the module importing #: this symbol. This is used for relative path calculations, #: but it's limited to relative modules that also share the #: filesystem location. An example is "/path/to/mytest.py", #: that can contain: #: #: from .base import BaseTestClass #: #: And thus will have a symbol of "BaseTestClass" and the #: module as ".base". The relative filesystem path of the #: module (which should contain the symbol) will be #: "/path/to". #: #: And if "/path/to/common/test.py" contains: #: #: from ..base import BaseTestClass #: #: The relative filesystem path of the module (which should #: contain the symbol) will be "/path/to". self.importer_fs_path = importer_fs_path #: An optional alias for the module, such as when a #: "import os as operating_system" statement is given. self.module_alias = module_alias #: An optional alias the symbol, such as when a #: "from os import path as os_path" is given self.symbol_alias = symbol_alias def _walk_importable_components(self, symbol_is_module=False): if symbol_is_module: full_name = f"{self.module_path}.{self.symbol}" else: full_name = self.module_path # Stripping leading dots, as relative paths will be handled with # insertion to module_paths with the relative path components = full_name.strip(".").split(".") for index, component in enumerate(components): if index > 0: previous = components[index - 1] else: previous = "" yield (component, previous) def get_importable_spec(self, symbol_is_module=False): """Returns the specification of an actual importable module. This is a check based on the limitations that we do not actually perform an import, and assumes a directory structure with modules. :param symbol_is_module: if it's known that the symbol is also a module, include it in the search for an importable spec :type symbol_is_module: bool """ modules_paths = sys.path modules_paths.insert(0, self.get_relative_module_fs_path()) spec = None for component, previous in self._walk_importable_components(symbol_is_module): if previous: modules_paths = [ os.path.join(mod, previous) for mod in modules_paths[:] ] spec = PathFinder.find_spec(component, modules_paths) if spec is None: break return spec def is_importable(self, symbol_is_module=False): """Checks whether this imported symbol seems to be importable. This is a check based on the limitations that we do not actually perform an import, and assumes a directory structure with modules. :param symbol_is_module: if it's known that the symbol is also a module, include it in the search for an importable spec :type symbol_is_module: bool """ return self.get_importable_spec(symbol_is_module) is not None @staticmethod def _split_last_module_path_component(module_path): """Splits a module path into a lower and topmost components. It also discards any information about relative location. :param module_path: a module path, such as "os" or "os.path" or "..selftests.utils" :type module_path: str :returns: the lower and topmost components :rtype: tuple """ non_relative = module_path.strip(".") if "." in non_relative: module_components = non_relative.rsplit(".", 1) if len(module_components) == 2: return (module_components[0], module_components[1]) return ("", non_relative) @staticmethod def _get_relative_prefix(statement): """Returns the string that represents to the relative import level. :param statement: an "import from" ast statement :type statement: :class:`ast.ImportFrom` :returns: the string that represents the relative import level. :rtype: str """ relative_level = getattr(statement, "level", 0) or 0 return "".join(["." for _ in range(relative_level)]) @staticmethod def get_symbol_from_statement(statement): return ImportedSymbol.get_symbol_module_path_from_statement(statement)[0] @staticmethod def get_module_path_from_statement(statement): return ImportedSymbol.get_symbol_module_path_from_statement(statement)[1] @staticmethod def get_symbol_module_path_from_statement(statement, name_index=0): symbol = "" module_path = "" module_alias = "" symbol_alias = "" import_as = get_statement_import_as(statement) names = list(import_as.keys()) as_names = list(import_as.values()) if isinstance(statement, ast.Import): # On an Import statement, it's impossible to import a symbol # so everything is the module_path module_path = names[name_index] module_alias = as_names[name_index] elif isinstance(statement, ast.ImportFrom): symbol = names[name_index] relative = ImportedSymbol._get_relative_prefix(statement) module_name = statement.module or "" module_path = relative + module_name symbol_alias = as_names[name_index] return symbol, module_path, module_alias, symbol_alias @property def module_name(self): """The final name of the module from its importer perspective. If a alias exists, it will be the alias name. If not, it will be the original name. """ if self.module_alias: return self.module_alias return self.module_path @property def symbol_name(self): """The final name of the symbol from its importer perspective. If a alias exists, it will be the alias name. If not, it will be the original name. """ if self.symbol_alias: return self.symbol_alias return self.symbol @classmethod def from_statement(cls, statement, importer_fs_path=None, index=0): ( symbol, module_path, module_alias, symbol_alias, ) = cls.get_symbol_module_path_from_statement(statement, index) return cls(module_path, symbol, importer_fs_path, module_alias, symbol_alias) def to_str(self): """Returns a string representation of the plausible statement used.""" if not self.symbol: return f"import {self.module_path}" return f"from {self.module_path} import {self.symbol}" def is_relative(self): """Returns whether the imported symbol is on a relative path.""" return self.module_path.startswith(".") def get_relative_module_fs_path(self): """Returns the module base dir, based on its relative path The base dir for the module is the directory where one is expected to find the first module of the module path. For a module path of "..foo.bar", and its importer being at "/abs/path/test.py", the base dir where "foo" is supposed to be found would be "/abs". And as a consequence, "bar" would be found at "/abs/foo/bar". This assumes that the module path is indeed related to the location of its importer. This may not be true if the namespaces match, but are distributed across different filesystem paths. """ path = os.path.dirname(self.importer_fs_path) for char in self.module_path[1:]: if char != ".": break path = os.path.dirname(path) return path def get_parent_fs_path(self): if self.is_relative(): return self.get_relative_module_fs_path() parent_path = os.path.dirname(self.importer_fs_path) if self.module_path: return os.path.join(parent_path, self.module_path) return parent_path def __repr__(self): return ( f'' ) def __eq__(self, other): return ( (self.module_path == other.module_path) and (self.symbol == other.symbol) and (self.importer_fs_path == other.importer_fs_path) ) avocado-113.0/avocado/core/safeloader/module.py000066400000000000000000000141641513561306000214500ustar00rootroot00000000000000import ast import os from avocado.core.safeloader.imported import ImportedSymbol from avocado.core.safeloader.utils import get_statement_import_as class PythonModule: """ Representation of a Python module that might contain interesting classes By default, it uses module and class names that matches Avocado instrumented tests, but it's supposed to be agnostic enough to be used for, say, Python unittests. """ __slots__ = ( "path", "klass_imports", "mod_imports", "mod", "module", "klass", "imported_symbols", "interesting_klass_found", ) def __init__(self, path, module="avocado", klass="Test"): """ Instantiates a new PythonModule representation :param path: path to a Python source code file :type path: str :param module: the original module name from where the possibly interesting class must have been imported from :type module: str :param klass: the possibly interesting class original name :type klass: str """ self.klass_imports = set() self.mod_imports = set() if os.path.isdir(path): path = os.path.join(path, "__init__.py") self.path = path self.module = module self.klass = klass self.imported_symbols = {} with open(self.path, encoding="utf-8") as source_file: self.mod = ast.parse(source_file.read(), self.path) self.interesting_klass_found = False def is_matching_klass(self, klass): """ Detect whether given class directly defines itself as . It can either be a that inherits from a test "symbol", like: ```class FooTest(Test)``` Or from an . symbol, like in: ```class FooTest(avocado.Test)``` :type klass: :class:`_ast.ClassDef` :rtype: bool """ # Is it inherited from Test? 'class FooTest(Test):' if self.klass_imports: base_ids = [base.id for base in klass.bases if isinstance(base, ast.Name)] # Looking for a 'class FooTest(Test):' if not self.klass_imports.isdisjoint(base_ids): return True # Is it inherited from avocado.Test? 'class FooTest(avocado.Test):' if self.mod_imports: for base in klass.bases: if not isinstance(base, ast.Attribute): # Check only 'module.Class' bases continue cls_module = base.value.id cls_name = base.attr if cls_module in self.mod_imports and cls_name == self.klass: return True return False @staticmethod def _get_adjusted_path_for_level(statement, path): level = getattr(statement, "level", 0) if level is None: level = 0 for _ in range(level - 1): path = os.path.dirname(path) return path def _get_imported_path_from_statement(self, statement): """Returns the imported path, from absolute or relative import.""" abs_path_of_module_dir = os.path.abspath(os.path.dirname(self.path)) imported_path = self._get_adjusted_path_for_level( statement, abs_path_of_module_dir ) if getattr(statement, "module", None) is not None: # Module has a name, so its path is absolute, and not relative # to the directory structure module_path = statement.module.replace(".", os.path.sep) imported_path = os.path.join(imported_path, module_path) return imported_path def add_imported_symbol(self, statement): """ Keeps track of symbol names and importable entities """ for index, name in enumerate(statement.names): final_name = self._get_name_from_alias_statement(name) imported_symbol = ImportedSymbol.from_statement( statement, os.path.abspath(self.path), index ) self.imported_symbols[final_name] = imported_symbol @staticmethod def _get_name_from_alias_statement(alias): """Returns the aliased name or original one.""" return alias.asname if alias.asname else alias.name def _handle_import_from(self, statement, interesting_klass): self.add_imported_symbol(statement) if interesting_klass in [name.name for name in statement.names]: self.interesting_klass_found = True if statement.module != self.module: return name = get_statement_import_as(statement).get(self.klass, None) if name is not None: self.klass_imports.add(name) @staticmethod def _all_module_level_names(full_module_name): result = [] components = full_module_name.split(".")[:-1] for topmost in range(len(components)): result.append(".".join(components[-topmost:])) components.pop() return result def _handle_import(self, statement): self.add_imported_symbol(statement) imported_as = get_statement_import_as(statement) name = imported_as.get(self.module, None) if name is not None: self.mod_imports.add(name) for as_name in imported_as.values(): for mod_name in self._all_module_level_names(as_name): if mod_name == self.module: self.mod_imports.add(mod_name) def iter_classes(self, interesting_klass=None): """ Iterate through classes and keep track of imported avocado statements """ for statement in self.mod.body: # Looking for a 'from import ' if isinstance(statement, ast.ImportFrom): self._handle_import_from(statement, interesting_klass) # Looking for a 'import ' elif isinstance(statement, ast.Import): self._handle_import(statement) # Looking for a 'class Anything(anything):' elif isinstance(statement, ast.ClassDef): yield statement avocado-113.0/avocado/core/safeloader/utils.py000066400000000000000000000014041513561306000213140ustar00rootroot00000000000000import ast import collections def get_statement_import_as(statement): """ Returns a mapping of imported module names whether using aliases or not :param statement: an AST import statement :type statement: ast.Import :returns: a mapping of names {: } of modules imported :rtype: collections.OrderedDict """ if not any( [isinstance(statement, ast.Import), isinstance(statement, ast.ImportFrom)] ): raise ValueError("Value given is not an ast import or import from statement") result = collections.OrderedDict() for name in statement.names: if name.asname is not None: result[name.name] = name.asname else: result[name.name] = name.name return result avocado-113.0/avocado/core/settings.py000066400000000000000000000671661513561306000177300ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; specifically version 2 of the License. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # This code was inspired in the autotest project, # client/shared/settings.py # # Authors: Travis Miller # Beraldo Leal """ This module is a new and experimental configuration handler. This will handle both, command line args and configuration files. Settings() = configparser + argparser Settings() is an attempt to implement part of BP001 and concentrate all default values in one place. This module will read the Avocado configuration options from many sources, in the following order: 1. Default values: This is a "source code" defined. When plugins or core needs a settings, basically needs to call settings.register_option() with default value as argument. Developers only need to register the default value once, here when calling this methods. 2. System/User configuration files (/etc/avocado or ~/.config/avocado/): This is configured by the user, in a more "permanent way". 3. Command-line options parsed in runtime. This is configured by the user, in a more "temporary way"; """ import ast import configparser import glob import json import os import platform import re from importlib.resources import files from avocado.core.settings_dispatcher import SettingsDispatcher def sorted_dict(dict_object): return sorted(dict_object.items(), key=lambda t: t[0]) class SettingsError(Exception): """ Base settings error. """ class ConfigFileNotFound(SettingsError): """ Error thrown when the main settings file could not be found. """ def __init__(self, path_list): super().__init__() self.path_list = path_list def __str__(self): return ( f"Could not find the avocado config file after looking in: " f"{self.path_list}" ) class DuplicatedNamespace(SettingsError): """ Raised when a namespace is already registered. """ class NamespaceNotRegistered(SettingsError): """ Raised when a namespace is not registered. """ class ConfigOption: def __init__( self, namespace, help_msg, key_type=str, default=None, parser=None, short_arg=None, long_arg=None, positional_arg=False, choices=None, nargs=None, metavar=None, required=None, action=None, argparse_type=None, argparse_help_msg=None, ): self.namespace = namespace self.help_msg = help_msg self.key_type = key_type self.default = default self.parser = parser self.short_arg = short_arg self.long_arg = long_arg self.positional_arg = positional_arg self.choices = choices self.nargs = nargs self._metavar = metavar self.required = required self._action = action self._value = None self._argparse_type = argparse_type self._argparse_help_msg = argparse_help_msg self._update_argparser() @property def action(self): if self.key_type is bool: # action is automatic when using bool types if self.default is False: return "store_true" else: return "store_false" return self._action @property def section(self): return ".".join(self.namespace.split(".")[:-1]) @property def key(self): return self.namespace.split(".")[-1] @property def value(self): if self._value is not None: return self._value return self.default @property def name_or_tags(self): if self.positional_arg: return self.key tags = [] if self.short_arg: tags.append(self.short_arg) if self.long_arg: tags.append(self.long_arg) return tags @property def argparse_type(self): if self._argparse_type is not None: return self._argparse_type # type is automatic when using lists because Argparse do not support # list on command-line. They are comma separated strings that will be # converted to list here. if self.key_type is list: return str else: return self.key_type @argparse_type.setter def argparse_type(self, value): self._argparse_type = value @property def argparse_help_msg(self): if self._argparse_help_msg is not None: return self._argparse_help_msg return self.help_msg @argparse_help_msg.setter def argparse_help_msg(self, value): self._argparse_help_msg = value @property def metavar(self): if self.positional_arg: if self._metavar is None: return self.key return self._metavar @property def arg_parse_args(self): args = {"help": self.argparse_help_msg, "default": None} if self.nargs: args["nargs"] = self.nargs if self.metavar: args["metavar"] = self.metavar if self.choices: args["choices"] = self.choices if self.action: args["action"] = self.action if self.key_type is not bool: # We don't specify type for bool args["type"] = self.argparse_type if not self.positional_arg: args["required"] = self.required args["dest"] = self.namespace # most of the magic is here return args @staticmethod def _as_list(value): if value == "": return [] if isinstance(value, str): try: return ast.literal_eval(value) except SyntaxError: pass if isinstance(value, list): return value raise ValueError(f"{value} could not be converted into a list") def _update_argparser(self): if not self.parser: return if self.positional_arg: self.parser.add_argument(self.namespace, **self.arg_parse_args) else: self.parser.add_argument(*self.name_or_tags, **self.arg_parse_args) def add_argparser( self, parser, long_arg, short_arg=None, positional_arg=False, choices=None, nargs=None, metavar=None, required=None, action=None, argparse_type=None, argparse_help_msg=None, ): """Add an command-line argparser to this option.""" self.parser = parser self.short_arg = short_arg self.long_arg = long_arg self.positional_arg = positional_arg self.choices = choices self.nargs = nargs self._metavar = metavar or self._metavar self.required = required self._action = action self._argparse_type = argparse_type self._argparse_help_msg = argparse_help_msg self._update_argparser() def set_value(self, value, convert=False): dst_type = self.key_type if convert is False: self._value = value else: if dst_type is list: self._value = self._as_list(value) elif dst_type is bool: self._value = value.lower() in ["true", "on", "y", "yes", "1"] else: self._value = dst_type(value) class AvocadoConfigParser: """AvocadoConfigParser uses ConfigParser to parse config from INI files. During the parsing process, it preserves the config files path. The configuration from files is stored in a dict-like structure where key is avocado config namespace and value is a pair of config value and config file where this value comes from. example: AvocadoConfigParser['namespace'] = tuple('value', 'config_file_path') """ def __init__(self): self._config = {} def read(self, filenames): """Parses the INI files to Avocado config :param filenames: One or more files which will be parsed. :type filenames: list or str :return: List of successfully parsed files. :rtype: list """ if isinstance(filenames, str): filenames = [filenames] read_ok = [] for filename in filenames: try: with open(filename, encoding="utf-8") as fp: self.read_file(fp) except OSError: continue read_ok.append(filename) return read_ok def read_file(self, file): """Parses file-like object to Avocado config :param file: File-like object which will be parsed. """ new_config = configparser.ConfigParser() new_config.read_file(file) for section in new_config: items = new_config.items(section) for key, value in items: self._config[f"{section}.{key}"] = (value, file.name) def __iter__(self): return self._config.__iter__() def __getitem__(self, namespace): return self._config[namespace] class Settings: """Settings is the Avocado configuration handler. It is a simple wrapper around configparser and argparse. Also, one object of this class could be passed as config to plugins and modules. Basically, if you are going to have options (configuration options), either via config file or via command line, you should use this class. You don't need to instantiate a new settings, just import and use `register_option()`. from avocado.core.settings import settings settings.register_option(...) And when you needs get the current value, check on your configuration for the namespace (section.key) that you registered. i.e: value = config.get('a.section.with.subsections.key') .. note:: Please, do not use a default value when using `get()` here. If you are using an existing namespace, get will always return a value, either the default value, or the value set by the user. Please, note that most of methods and attributes here are private. Only public methods and attributes should be used outside this module. """ def __init__(self): """Constructor. Tries to find the main settings files and load them.""" self.config = AvocadoConfigParser() self.all_config_paths = [] self.config_paths = [] self._namespaces = {} # 1. Prepare config paths self._prepare_base_dirs() self._append_config_paths() # 2. Parse/read all config paths self.config_paths = self.config.read(self.all_config_paths) def _append_config_paths(self): # Override with system config self._append_system_config() # Allow plugins to modify/extend the list of configs dispatcher = SettingsDispatcher() if dispatcher.extensions: dispatcher.map_method("adjust_settings_paths", self.all_config_paths) # Override with the user's local config self._append_user_config() def _append_system_config(self): if self._config_path_pkg is not None: self.all_config_paths.append(self._config_path_pkg) self.all_config_paths.append(self._config_path_system) configs = glob.glob(os.path.join(self._config_dir_system_extra, "*.conf")) for extra_file in configs: self.all_config_paths.append(extra_file) def _append_user_config(self): if os.path.exists(self._config_path_local): self.all_config_paths.append(self._config_path_local) def _prepare_base_dirs(self): if platform.system() == "FreeBSD": cfg_dir = "/usr/local/etc" else: cfg_dir = "/etc" user_dir = os.path.expanduser("~") if "VIRTUAL_ENV" in os.environ: cfg_dir = os.path.join(os.environ["VIRTUAL_ENV"], "etc") user_dir = os.environ["VIRTUAL_ENV"] config_file_name = "avocado.conf" conf_file = files("avocado") / "etc" / "avocado" / config_file_name if conf_file.is_file(): self._config_path_pkg = str(conf_file) else: self._config_path_pkg = None self._config_dir_system = os.path.join(cfg_dir, "avocado") self._config_dir_system_extra = os.path.join(cfg_dir, "avocado", "conf.d") self._config_dir_local = os.path.join(user_dir, ".config", "avocado") self._config_path_system = os.path.join( self._config_dir_system, config_file_name ) self._config_path_local = os.path.join(self._config_dir_local, config_file_name) def add_argparser_to_option( self, namespace, parser, long_arg=None, short_arg=None, positional_arg=False, choices=None, nargs=None, metavar=None, required=None, action=None, allow_multiple=False, argparse_type=None, help_msg=None, ): """Add a command-line argument parser to an existing option. This method is useful to add a parser when the option is registered without any command-line argument options. You should call the "register_option()" method for the namespace before calling this method. Arguments namespace : str What is the namespace of the option (section.key) parser : argparser parser Since that you would like to have a command-line option, you should specify what is the parser or parser group that we should add this option. long_arg: : str A long option for the command-line. i.e: `--debug` for debug. short_arg : str A short option for the command-line. i.e: `-d` for debug. positional_arg : bool If this option is an positional argument or not. Default is `False`. choices : tuple If you would like to limit the option to a few choices. i.e: ('foo', 'bar') nargs : int or str The number of command-line arguments that should be consumed. Could be a int, '?', '*' or '+'. For more information visit the argparser documentation. metavar : str String presenting available sub-commands in help, if None we will use the section+key as metavar. required : bool If this is a required option or not when on command-line. Default is False. action : The basic type of action to be taken when this argument is encountered at the command line. For more information visit the argparser documentation. allow_multiple : Whether the same option may be available on different parsers. This is useful when the same option is available on different commands, such as "avocado run" or "avocado list". argparse_type : A possibly different type for the command line handling of an option. For instance, when an option has a "key_type" of "list", its respective configuration file entry will expect a string that can be evaluated to a Python list, but that is far from convenient to set on the command line. With this argument, a function that will, for instance, split a comma separated list may be used, resulting in command line users being able to provide convenient input. help_msg : str A help message, different from the original ConfigOption help message, to be shown on the command line. To be used when the command line usage and the original help message do not make sense together. """ if not any([long_arg, short_arg, positional_arg]): raise SettingsError( "To add an argument parser to an option, it " "needs to have a long argument, a short " "argument or be a positional argument" ) try: option = self._namespaces[namespace] except KeyError: msg = f"Namespace not found: {namespace}" raise NamespaceNotRegistered(msg) if option and option.parser and not allow_multiple: msg = "Parser already registered for this namespace" raise SettingsError(msg) option.add_argparser( parser, long_arg, short_arg, positional_arg, choices, nargs, metavar, required, action, argparse_type, help_msg, ) def as_dict(self, regex=None): """Return an dictionary with the current active settings. This will return a dict with all parsed options (either via config file or via command-line). If regex is not None, this method will filter the current config matching regex with the namespaces. :param regex: A regular expression to be used on the filter. """ result = {} for namespace, option in sorted_dict(self._namespaces): result[namespace] = option.value return self.filter_config(result, regex) if regex else result def as_full_dict(self): result = {} for namespace, option in sorted_dict(self._namespaces): result[namespace] = { "help": option.help_msg, "type": option.key_type, "default": option.default, "section": option.section, "key": option.key, } return result def as_json(self, regex=None): """Return a JSON with the current active settings. This will return a JSON with all parsed options (either via config file or via command-line). If regex is not None, it will be used to filter namespaces. :param regex: A regular expression to be used on the filter. """ return json.dumps(self.as_dict(regex), indent=4) @staticmethod def filter_config(config, regex): """Utility to filter a config by namespaces based on a regex. :param config: dict object with namespaces and values :param regex: regular expression to use against the namespace """ result = {} for namespace, option in sorted_dict(config): if re.match(regex, namespace): result[namespace] = option return result def merge_with_arguments(self, arg_parse_config): """Merge the current settings with the command-line args. After parsing argument options this method should be executed to have an unified settings. :param arg_parse_config: argparse.config dictionary with all command-line parsed arguments. """ for namespace, value in arg_parse_config.items(): # This check is important! For argparse when an option is # not passed will return None, except for positional arguments # which will be an empty list. We need to update only the # options that the user has specified. config_option = self._namespaces.get(namespace, None) positional = getattr(config_option, "positional_arg", False) if positional and value == []: continue if value is not None: if namespace in self._namespaces: self.update_option(namespace, value) def merge_with_configs(self): """Merge the current settings with the config file options. After parsing config file options this method should be executed to have an unified settings. """ for namespace in self.config: value = self.config[namespace] path = value[1] value = value[0] try: self.update_option(namespace, value, convert=True) except (SyntaxError, ValueError): raise SyntaxError( f"Syntax error in config file {path}, please check the value {value} " ) def process_config_path(self, path): """Update list of config paths and process the given path.""" self.all_config_paths.append(path) self.config_paths.extend(self.config.read(path)) def register_option( self, section, key, default, help_msg, key_type=str, parser=None, positional_arg=False, short_arg=None, long_arg=None, choices=None, nargs=None, metavar=None, required=False, action=None, allow_multiple=False, ): """Method used to register a configuration option inside Avocado. This should be used to register a settings option (either config file option or command-line option). This is the central point that plugins and core should use to register a new configuration option. This method will take care of the 'under the hood dirt', registering the configparse option and, if desired, the argparse too. Instead of using argparse and/or configparser, Avocado's contributors should use this method. Using this method, you need to specify a "section", "key", "default" value and a "help_msg" always. This will create a relative configuration file option for you. For instance: settings.register_option(section='foo', key='bar', default='hello', help_msg='this is just a test') This will register a 'foo.bar' namespace inside Avocado internals settings. And this could be now, be changed by the users or system configuration option: [foo] bar = a different message replacing 'hello' If you would like to provide also the flexibility to the user change the values via command-line, you should pass the other arguments. Arguments section : str The configuration file section that your option should be present. You can specify subsections with dots. i.e: run.output.json key : str What is the key name of your option inside that section. default : typeof(key_type) The default value of an option. It sets the option value when the key is not defined in any configuration files or via command-line. The default value should be "processed". It means the value should match the type of key_type. Due to some internal limitations, the Settings module will not apply key_type to the default value. help_msg : str The help message that will be displayed at command-line (-h) and configuration file template. key_type : any method What is the type of your option? Currently supported: int, list, str or a custom method. Default is `str`. parser : argparser parser Since that you would like to have a command-line option, you should specify what is the parser or parser group that we should add this option. positional_arg : bool If this option is an positional argument or not. Default is `False`. short_arg : str A short option for the command-line. i.e: `-d` for debug. long_arg: : str A long option for the command-line. i.e: `--debug` for debug. choices : tuple If you would like to limit the option to a few choices. i.e: ('foo', 'bar') nargs : int or str The number of command-line arguments that should be consumed. Could be a int, '?', '*' or '+'. For more information visit the argparser documentation. metavar : str String presenting available sub-commands in help, if None we will use the section+key as metavar. required : bool If this is a required option or not when on command-line. Default is False. action : The basic type of action to be taken when this argument is encountered at the command line. For more information visit the argparser documentation. allow_multiple : Whether the same option may be available on different parsers. This is useful when the same option is available on different commands, such as "avocado run" or "avocado list". .. note:: Most of the arguments here (like parser, positional_arg, short_arg, long_arg, choices, nargs, metavar, required and action) are only necessary if you would like to add a command-line option. """ namespace = f"{section}.{key}" # Check if namespace is already registered if namespace in self._namespaces: if not allow_multiple: msg = f'Key "{key}" already registered under ' f'section "{section}"' raise DuplicatedNamespace(msg) else: self.add_argparser_to_option( namespace, parser, long_arg, short_arg, positional_arg, choices, nargs, metavar, required, action, allow_multiple, ) else: option = ConfigOption( namespace, help_msg, key_type, default, parser, short_arg, long_arg, positional_arg, choices, nargs, metavar, required, action, ) # Register the option to a dynamic in-memory namespaces self._namespaces[namespace] = option def update_option(self, namespace, value, convert=False): """Convenient method to change the option's value. This will update the value on Avocado internals and if necessary the type conversion will be realized. For instance, if an option was registered as bool and you call: settings.register_option(namespace='foo.bar', value='true', convert=True) This will be stored as True, because Avocado will get the 'key_type' registered and apply here for the conversion. This method is useful when getting values from config files where everything is stored as string and a conversion is needed. Arguments namespace : str Your section plus your key, separated by dots. The last part of the namespace is your key. i.e: run.outputs.json.enabled (section is `run.outputs.json` and key is `enabled`) value : any type This is the new value to update. convert : bool If Avocado should try to convert the value and store it as the 'key_type' specified during the register. Default is False. """ if namespace not in self._namespaces: return self._namespaces[namespace].set_value(value, convert) settings = Settings() # pylint: disable-msg=invalid-name avocado-113.0/avocado/core/settings_dispatcher.py000066400000000000000000000022771513561306000221260ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2018-2019 """ Settings Dispatcher This is a special case for the dispatchers that can be found in :mod:`avocado.core.dispatcher`. This one deals with settings that will be read by the other dispatchers, while still being a dispatcher for configuration sources. """ from avocado.core.extension_manager import ExtensionManager class SettingsDispatcher(ExtensionManager): """ Dispatchers that allows plugins to modify settings It's not the standard "avocado.core.dispatcher" because that one depends on settings. This dispatcher is the bare-stevedore dispatcher which is executed before settings is parsed. """ def __init__(self): super().__init__("avocado.plugins.settings") avocado-113.0/avocado/core/spawners/000077500000000000000000000000001513561306000173405ustar00rootroot00000000000000avocado-113.0/avocado/core/spawners/__init__.py000066400000000000000000000000001513561306000214370ustar00rootroot00000000000000avocado-113.0/avocado/core/spawners/common.py000066400000000000000000000032271513561306000212060ustar00rootroot00000000000000import enum from avocado.core.settings import settings class SpawnMethod(enum.Enum): """The method employed to spawn a runnable or task.""" #: Spawns by running executing Python code, that is, having access to #: a runnable or task instance, it calls its run() method. PYTHON_CLASS = object() #: Spawns by running a command, that is having either a path to an #: executable or a list of arguments, it calls a function that will #: execute that command (such as with os.system()) STANDALONE_EXECUTABLE = object() #: Spawns with any method available, that is, it doesn't declare or #: require a specific spawn method ANY = object() class SpawnCapabilities(enum.Enum): """The capabilities of a spawner implementation.""" #: The spawner can automatically provision the environment. There #: is no need to create environment before running the task. AUTOMATIC_ENVIRONMENT_PROVISIONING = "Automatic environment provisioning" #: The avocado will be preinstalled in the environment. AVOCADO_DEPLOYMENT = "Avocado deployment" #: The environment is accessible after the task is finished. ENVIRONMENT_PRESERVATION = "Environment preservation" #: The spawner can share the filesystem with the run task. FILESYSTEM_SHARING = "Filesystem sharing" class SpawnerMixin: """Common utilities for Spawner implementations.""" METHODS = [] def __init__(self, config=None, job=None): if config is None: config = settings.as_dict() self.config = config self._job = job def task_output_dir(self, runtime_task): return runtime_task.task.runnable.output_dir avocado-113.0/avocado/core/spawners/exceptions.py000066400000000000000000000000541513561306000220720ustar00rootroot00000000000000class SpawnerException(Exception): pass avocado-113.0/avocado/core/spawners/mock.py000066400000000000000000000042041513561306000206430ustar00rootroot00000000000000import asyncio import random from avocado.core.plugin_interfaces import Spawner from avocado.core.spawners.common import SpawnMethod class MockSpawner(Spawner): """A mocking spawner that performs no real operation. Tasks asked to be spawned by this spawner will initially reported to be alive, and on the next check, will report not being alive. """ METHODS = [SpawnMethod.PYTHON_CLASS, SpawnMethod.STANDALONE_EXECUTABLE] def __init__(self): # pylint: disable=W0231 self._known_tasks = {} def is_operational(self): return True def is_task_alive(self, runtime_task): # pylint: disable=W0221 alive = self._known_tasks.get(runtime_task, None) # task was never spawned if alive is None: return False # task was spawned and should signal it's alive for the first time if alive: self._known_tasks[runtime_task] = False return True else: # signal it's *not* alive after first check return False async def spawn_task(self, runtime_task): self._known_tasks[runtime_task] = True return True async def wait_task(self, runtime_task): while True: if self.is_task_alive(runtime_task): return await asyncio.sleep(0.1) async def terminate_task(self, runtime_task): self._known_tasks[runtime_task] = False @staticmethod async def check_task_requirements(runtime_task): return True @staticmethod async def is_requirement_in_cache(runtime_task): return False @staticmethod async def save_requirement_in_cache(runtime_task): pass @staticmethod async def update_requirement_cache(runtime_task, result): pass class MockRandomAliveSpawner(MockSpawner): """A mocking spawner that simulates randomness about tasks being alive.""" def is_task_alive(self, runtime_task): alive = self._known_tasks.get(runtime_task, None) # task was never spawned if alive is None: return False return random.choice([True, True, True, True, False]) avocado-113.0/avocado/core/status/000077500000000000000000000000001513561306000170215ustar00rootroot00000000000000avocado-113.0/avocado/core/status/__init__.py000066400000000000000000000000001513561306000211200ustar00rootroot00000000000000avocado-113.0/avocado/core/status/repo.py000066400000000000000000000146641513561306000203530ustar00rootroot00000000000000import heapq import logging from avocado.core.status.utils import json_loads from avocado.core.teststatus import STATUSES LOG = logging.getLogger(__name__) class StatusMsgMissingDataError(Exception): """Status message does not contain the required data.""" class StatusRepo: """Maintains tasks' status related data and provides aggregated info.""" def __init__(self, job_id): """Initializes a new StatusRepo :param job_id: the job unique identification for which the messages are destined to. :type job_id: str """ self.job_id = job_id #: Contains all received messages by a given task (by its ID) self._all_data = {} #: Contains the most up to date status of a task, and the time #: it was set in a tuple (status, time). This is keyed #: by the task ID, and the most up to date status is determined by #: the status type of message. self._status = {} #: Contains a global journal of status updates to be picked, each #: entry containing a tuple with (task_id, status, time). It discards #: status that have been superseded by newer status. self._status_journal_summary = [] #: Contains the task IDs keyed by the result received self._by_result = {} def _handle_task_finished(self, message): task_id = message["id"] result = message.get("result") if result is not None and result.upper() not in STATUSES: overridden = "error" message["result"] = overridden message[ "fail_reason" ] = f'Runner error occurred: Test reports unsupported status "{result}"' LOG.error( 'Task "%s" finished message with unsupported status ' '"%s", changing to "%s"', task_id, result, overridden, ) self._set_by_result(message) self._set_task_data(message) LOG.debug('Task "%s" finished message: "%s"', task_id, message) def _handle_task_started(self, message): if "output_dir" not in message: raise StatusMsgMissingDataError("output_dir") task_id = message["id"] LOG.debug('Task "%s" started message: "%s"', task_id, message) self._set_task_data(message) def _set_by_result(self, message): """Sets an entry in the aggregate by result. For messages that include a "result" key, expected for example, from a "finished" status message, this will allow users to query for tasks with a given result.""" result = message.get("result") if result not in self._by_result: self._by_result[result] = [] if message["id"] not in self._by_result[result]: self._by_result[result].append(message["id"]) def _set_task_data(self, message): """Appends all data on message to an entry keyed by the task's ID.""" task_id = message.pop("id") if task_id not in self._all_data: self._all_data[task_id] = [] self._all_data[task_id].append(message) def get_all_task_data(self, task_id): """Returns all data on a given task, by its ID.""" return self._all_data.get(task_id) def get_task_data(self, task_id, index): """Returns the data on the index of a given task, by its ID.""" task_data = self._all_data.get(task_id) return task_data[index] def get_latest_task_data(self, task_id): """Returns the latest data on a given task, by its ID.""" task_data = self._all_data.get(task_id) if task_data is None: return None return task_data[-1] def status_journal_summary_pop(self): return heapq.heappop(self._status_journal_summary) def _update_status(self, message): """Update the latest status of a task (by message).""" task_id = message.get("id") status = message.get("status") time = message.get("time") if not all((task_id, status, time)): return if task_id not in self._status: self._status[task_id] = (status, time) heapq.heappush(self._status_journal_summary, (time, task_id, status, 0)) else: current_status, _ = self._status[task_id] if current_status == "finished": LOG.warning( "Received a %s message after finished message: %s", status, message ) elif status == "started": LOG.warning( "Received a started message when the status is already %s: %s", current_status, message, ) else: self._status[task_id] = (status, time) index = len(self.get_all_task_data(task_id)) heapq.heappush(self._status_journal_summary, (time, task_id, status, index)) def process_message(self, message): for required_field in ("id", "job_id"): if required_field not in message: raise StatusMsgMissingDataError(required_field) job_id = message.get("job_id") if job_id != self.job_id: LOG.warning("Received a message destined for a different job: %s", message) return message.pop("job_id") self._update_status(message) handlers = { "started": self._handle_task_started, "finished": self._handle_task_finished, } meth = handlers.get(message.get("status"), self._set_task_data) meth(message) def process_raw_message(self, raw_message): raw_message = raw_message.strip() message = json_loads(raw_message) self.process_message(message) @property def result_stats(self): return {key: len(value) for key, value in self._by_result.items()} def get_task_status(self, task_id): return self._status.get(task_id, (None, None))[0] @staticmethod def _is_in_task(tasks, task_ids): """Returns True if any of the tasks is in task_ids.""" return any([True for task_id in task_ids if task_id in tasks]) def get_result_set_for_tasks(self, task_ids): """Returns a set of results for the given tasks.""" results = [ key for key, value in self._by_result.items() if self._is_in_task(value, task_ids) ] return set(results) avocado-113.0/avocado/core/status/server.py000066400000000000000000000033241513561306000207030ustar00rootroot00000000000000import asyncio import os from avocado.core.settings import settings class StatusServer: """Server that listens for status messages and updates a StatusRepo.""" def __init__(self, uri, repo): """Initializes a new StatusServer. :param uri: either a "host:port" string or a path to a UNIX socket :type uri: str :param repo: the repository to use to process received status messages :type repo: :class:`avocado.core.status.repo.StatusRepo` """ self._uri = uri self._repo = repo self._server_task = None @property def uri(self): return self._uri async def create_server(self): limit = settings.as_dict().get("run.status_server_buffer_size") if ":" in self._uri: host, port = self._uri.split(":") port = int(port) self._server_task = await asyncio.start_server( self.cb, host=host, port=port, limit=limit ) else: self._server_task = await asyncio.start_unix_server( self.cb, path=self._uri, limit=limit ) async def serve_forever(self): if self._server_task is None: await self.create_server() await self._server_task.serve_forever() def close(self): self._server_task.close() if os.path.exists(self._uri): os.unlink(self._uri) async def cb(self, reader, _): while True: try: raw_message = await reader.readline() except ConnectionResetError: continue if not raw_message: return self._repo.process_raw_message(raw_message) avocado-113.0/avocado/core/status/utils.py000066400000000000000000000014721513561306000205370ustar00rootroot00000000000000import base64 import json class StatusMsgInvalidJSONError(Exception): """Status message does not contain valid JSON.""" def json_base64_decode(dct): """base64 decode object hook for custom JSON encoding.""" key_name = "__base64_encoded__" if key_name in dct: return base64.b64decode(dct[key_name]) return dct def json_loads(data): """Loads and decodes JSON, with added base64 decoding. :param data: either bytes or a string. If bytes, will be decoded using the current default encoding. :raises: :returns: decoded Python objects """ if isinstance(data, bytes): data = data.decode() try: return json.loads(data, object_hook=json_base64_decode) except json.decoder.JSONDecodeError: raise StatusMsgInvalidJSONError(data) avocado-113.0/avocado/core/streams.py000066400000000000000000000010401513561306000175210ustar00rootroot00000000000000#: Builtin special keywords to enable set of logging streams BUILTIN_STREAMS = { "app": "avocado.app", "test": "avocado.test", "job": "avocado.job", "early": "avocado", } BUILTIN_STREAMS_DESCRIPTION = { "app": "application output", "test": "test output", "job": "job output", "early": "early logging of other streams, including test (very verbose)", } #: Groups of builtin streams BUILTIN_STREAM_SETS = { "all": "all builtin streams", "none": "disables regular output (leaving only errors enabled)", } avocado-113.0/avocado/core/suite.py000066400000000000000000000303641513561306000172070ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2020 # Author: Beraldo Leal import os from copy import deepcopy from enum import Enum from uuid import uuid4 from avocado.core.dispatcher import SuiteRunnerDispatcher from avocado.core.exceptions import ( JobTestSuiteReferenceResolutionError, OptionValidationError, ) from avocado.core.parser import HintParser from avocado.core.resolver import ReferenceResolutionResult, resolve from avocado.core.settings import settings from avocado.core.tags import filter_tags_on_runnables from avocado.core.tree import TreeNode from avocado.core.varianter import Varianter, dump_variant, is_empty_variant class TestSuiteError(Exception): pass class TestSuiteStatus(Enum): RESOLUTION_NOT_STARTED = object() TESTS_NOT_FOUND = object() TESTS_FOUND = object() UNKNOWN = object() def resolutions_to_runnables(resolutions, config): """ Transforms resolver resolutions into runnables suitable for a suite A resolver resolution (:class:`avocado.core.resolver.ReferenceResolution`) contains information about the resolution process (if it was successful or not) and in case of successful resolutions a list of resolutions. It's expected that the resolution contain one or more :class:`avocado.core.nrunner.Runnable`. This function sets the runnable specific configuration for each runnable. It also performs tag based filtering on the runnables for possibly excluding some of the Runnables. :param resolutions: possible multiple resolutions for multiple references :type resolutions: list of :class:`avocado.core.resolver.ReferenceResolution` :param config: job configuration :type config: dict :returns: the resolutions converted to runnables :rtype: list of :class:`avocado.core.nrunner.Runnable` """ filter_by_tags = config.get("filter.by_tags.tags") include_empty = config.get("filter.by_tags.include_empty") include_empty_key = config.get("filter.by_tags.include_empty_key") if filter_by_tags: return filter_tags_on_runnables( resolutions, filter_by_tags, include_empty, include_empty_key ) result = [] for resolution in resolutions: if resolution.result != ReferenceResolutionResult.SUCCESS: continue for runnable in resolution.resolutions: runnable.default_config = runnable.filter_runnable_config( runnable.kind, config ) result.append(runnable) return result class TestSuite: def __init__( self, name, config=None, tests=None, job_config=None, resolutions=None, enabled=True, ): self.name = name self.tests = tests self.resolutions = resolutions self.enabled = enabled # Create a complete config dict with all registered options + custom # config self.config = settings.as_dict() if job_config: self.config.update(job_config) if config: self.config.update(config) self._variants = None self._references = None self._runner = None self._test_parameters = None self._check_both_parameters_and_variants() if self.config.get("run.dry_run.enabled"): self._convert_to_dry_run() if self.size == 0: return def _has_both_parameters_and_variants(self): if not self.test_parameters: return False for variant in self.variants.itertests(): var = variant.get("variant") if not is_empty_variant(var): return True return False def _check_both_parameters_and_variants(self): if self._has_both_parameters_and_variants(): err_msg = ( "Specifying test parameters (with config entry " '"run.test_parameters" or command line "-p") along with ' 'any varianter plugin (run "avocado plugins" for a list)' " is not yet supported. Please use one or the other." ) raise TestSuiteError(err_msg) def __len__(self): """This is a convenient method to run `len()` over this object. With this you can run: len(a_suite) and will return the same as `len(a_suite.tests)`. """ return self.size def _convert_to_dry_run(self): if self.config.get("run.suite_runner") == "nrunner": for runnable in self.tests: runnable.kind = "dry-run" @classmethod def _from_config_with_resolver(cls, config, name=None): ignore_missing = config.get("run.ignore_missing_references") references = config.get("resolver.references") try: hint = None hint_filepath = ".avocado.hint" if os.path.exists(hint_filepath): hint = HintParser(hint_filepath) resolutions = resolve( references, hint=hint, ignore_missing=ignore_missing, config=config ) except JobTestSuiteReferenceResolutionError as details: raise TestSuiteError(details) runnables = resolutions_to_runnables(resolutions, config) if name is None: name = str(uuid4()) return cls(name=name, config=config, tests=runnables, resolutions=resolutions) def _get_stats_from_nrunner(self): stats = {} for test in self.tests: self._increment_dict_key_counter(stats, test.kind) return stats def _get_tags_stats_from_nrunner(self): stats = {} for runnable in self.tests: if runnable is None: continue tags = runnable.tags or {} for tag in tags: self._increment_dict_key_counter(stats, tag) return stats @staticmethod def _increment_dict_key_counter(dict_object, key): try: dict_object[key.lower()] += 1 except KeyError: dict_object[key.lower()] = 1 @property def references(self): if self._references is None: self._references = self.config.get("resolver.references") return self._references @property def runner(self): if self._runner is None: runner_name = self.config.get("run.suite_runner") try: runner_extension = SuiteRunnerDispatcher()[runner_name] self._runner = runner_extension.obj except KeyError: raise TestSuiteError("Runner not implemented.") return self._runner @property def size(self): """The overall length/size of this test suite.""" if self.tests is None: return 0 return len(self.tests) @property def stats(self): """Return a statistics dict with the current tests.""" runner_name = self.config.get("run.suite_runner") if runner_name == "nrunner": return self._get_stats_from_nrunner() return {} @property def status(self): if self.tests is None: return TestSuiteStatus.RESOLUTION_NOT_STARTED elif self.size == 0: return TestSuiteStatus.TESTS_NOT_FOUND elif self.size > 0: return TestSuiteStatus.TESTS_FOUND else: return TestSuiteStatus.UNKNOWN @property def tags_stats(self): """Return a statistics dict with the current tests tags.""" runner_name = self.config.get("run.suite_runner") if runner_name == "nrunner": return self._get_tags_stats_from_nrunner() return {} @property def test_parameters(self): """Placeholder for test parameters. This is related to --test-parameters command line option or (run.test_parameters). """ if self._test_parameters is None: self._test_parameters = { name: value for name, value in self.config.get("run.test_parameters", []) } return self._test_parameters @property def variants(self): if self._variants is None: variants = Varianter() if not variants.is_parsed(): try: variants.parse(self.config) except (IOError, ValueError) as details: raise OptionValidationError( f"Unable to parse variant: " f"{details}" ) self._variants = variants return self._variants def _get_test_variants(self): def add_variant(runnable, variant): runnable = deepcopy(runnable) runnable.variant = dump_variant(variant) runnable_with_variant.append(runnable) runnable_with_variant = [] if self.test_parameters: paths = ["/"] tree_nodes = TreeNode().get_node(paths[0], True) tree_nodes.value = self.test_parameters variant = {"variant": tree_nodes, "variant_id": None, "paths": paths} for runnable in self.tests: add_variant(runnable, variant) elif self.variants: # let's use variants when parameters are not available # define execution order execution_order = self.config.get("run.execution_order") if execution_order == "variants-per-test": for runnable in self.tests: for variant in self.variants.itertests(): add_variant(runnable, variant) elif execution_order == "tests-per-variant": for variant in self.variants.itertests(): for runnable in self.tests: add_variant(runnable, variant) return runnable_with_variant def get_test_variants(self): """Computes test variants based on the parameters""" return [(test, test.variant) for test in self._get_test_variants()] def run(self, job): """Run this test suite with the job context in mind. :param job: A :class:`avocado.core.job.Job` instance. :rtype: set """ return self.runner.run_suite(job, self) @classmethod def from_config(cls, config, name=None, job_config=None): """Helper method to create a TestSuite from config dicts. This is different from the TestSuite() initialization because here we are assuming that you need some help to build the test suite. Avocado will try to resolve tests based on the configuration information instead of assuming pre populated tests. If you need to create a custom TestSuite, please use the TestSuite() constructor instead of this method. :param config: A config dict to be used on the desired test suite. :type config: dict :param name: The name of the test suite. This is optional and default is a random uuid. :type name: str :param job_config: The job config dict (a global config). Use this to avoid huge configs per test suite. This is also optional. :type job_config: dict """ suite_config = config config = settings.as_dict() if job_config: config.update(job_config) config.update(suite_config) suite = cls._from_config_with_resolver(config, name) if suite.test_parameters or suite.variants: suite.tests = suite._get_test_variants() if not config.get("run.ignore_missing_references"): if not suite.tests: msg = ( "Test Suite could not be created. No test references " "provided nor any other arguments resolved into tests" ) raise TestSuiteError(msg) return suite avocado-113.0/avocado/core/sysinfo.py000066400000000000000000000233571513561306000175540ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; specifically version 2 of the License. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # This code was inspired in the autotest project, # client/shared/settings.py # Author: John Admanski import filecmp import logging import os import time from avocado.core import output from avocado.core.settings import settings from avocado.utils import genio from avocado.utils import path as utils_path from avocado.utils import sysinfo from avocado.utils.software_manager import manager log = logging.getLogger("avocado.sysinfo") def gather_collectibles_config(config): sysinfo_files = {} for collectible in ["commands", "files", "fail_commands", "fail_files"]: tmp_file = config.get(f"sysinfo.collectibles.{collectible}") if os.path.isfile(tmp_file): log.info("%s configured by file: %s", collectible.title(), tmp_file) sysinfo_files[collectible] = genio.read_all_lines(tmp_file) else: log.debug("File %s does not exist.", tmp_file) sysinfo_files[collectible] = [] if "fail_" in collectible: list1 = sysinfo_files[collectible] list2 = sysinfo_files[collectible.split("_")[1]] sysinfo_files[collectible] = [tmp for tmp in list1 if tmp not in list2] return sysinfo_files class SysInfo: """ Log different system properties at some key control points. Includes support for a start and stop event, with daemons running in between. An event may be a job, a test, or any other event with a beginning and end. """ def __init__(self, basedir=None, log_packages=None, profiler=None): """ Set sysinfo collectibles. :param basedir: Base log dir where sysinfo files will be located. :param log_packages: Whether to log system packages (optional because logging packages is a costly operation). If not given explicitly, tries to look in the config files, and if not found, defaults to False. :param profiler: Whether to use the profiler. If not given explicitly, tries to look in the config files. """ self.config = settings.as_dict() if basedir is None: basedir = utils_path.init_dir("sysinfo") self.basedir = basedir self._installed_pkgs = None if log_packages is None: packages_namespace = "sysinfo.collect.installed_packages" self.log_packages = self.config.get(packages_namespace) else: self.log_packages = log_packages self._get_collectibles(profiler) self.start_collectibles = set() self.end_collectibles = set() self.end_fail_collectibles = set() self.pre_dir = utils_path.init_dir(self.basedir, "pre") self.post_dir = utils_path.init_dir(self.basedir, "post") self.profile_dir = utils_path.init_dir(self.basedir, "profile") self._set_collectibles() def _get_collectibles(self, c_profiler): self.sysinfo_files = gather_collectibles_config(self.config) profiler = c_profiler if profiler is None: self.profiler = self.config.get("sysinfo.collect.profiler") else: self.profiler = profiler profiler_file = self.config.get("sysinfo.collectibles.profilers") if os.path.isfile(profiler_file): self.sysinfo_files["profilers"] = genio.read_all_lines(profiler_file) log.info("Profilers configured by file: %s", profiler_file) if not self.sysinfo_files["profilers"]: self.profiler = False if self.profiler is False: if not self.sysinfo_files["profilers"]: log.info("Profiler disabled: no profiler commands configured") else: log.info("Profiler disabled") else: log.debug("File %s does not exist.", profiler_file) self.sysinfo_files["profilers"] = [] @staticmethod def _get_syslog_watcher(): logpaths = ["/var/log/messages", "/var/log/syslog", "/var/log/system.log"] for logpath in logpaths: if os.path.exists(logpath): try: return sysinfo.LogWatcher(logpath) except PermissionError as e: log.debug(e.args[0]) raise ValueError(f"System log file not found (looked for {logpaths})") def _set_collectibles(self): timeout = self.config.get("sysinfo.collect.commands_timeout") locale = self.config.get("sysinfo.collect.locale") if self.profiler: for cmd in self.sysinfo_files["profilers"]: self.start_collectibles.add(sysinfo.Daemon(cmd, locale=locale)) for cmd in self.sysinfo_files["commands"]: self.start_collectibles.add( sysinfo.Command(cmd, timeout=timeout, locale=locale) ) self.end_collectibles.add( sysinfo.Command(cmd, timeout=timeout, locale=locale) ) for fail_cmd in self.sysinfo_files["fail_commands"]: self.end_fail_collectibles.add( sysinfo.Command(fail_cmd, timeout=timeout, locale=locale) ) for filename in self.sysinfo_files["files"]: self.start_collectibles.add(sysinfo.Logfile(filename)) self.end_collectibles.add(sysinfo.Logfile(filename)) for fail_filename in self.sysinfo_files["fail_files"]: self.end_fail_collectibles.add(sysinfo.Logfile(fail_filename)) try: self.end_collectibles.add(sysinfo.JournalctlWatcher()) except sysinfo.CollectibleException as e: log.debug(e.args[0]) def _get_installed_packages(self): sm = manager.SoftwareManager() installed_pkgs = sm.list_all() self._installed_pkgs = installed_pkgs return installed_pkgs def _log_installed_packages(self, path): installed_path = os.path.join(path, "installed_packages") installed_packages = "\n".join(self._get_installed_packages()) + "\n" genio.write_file(installed_path, installed_packages) def _log_modified_packages(self, path): """ Log any changes to installed packages. """ old_packages = set(self._installed_pkgs) new_packages = set(self._get_installed_packages()) added_path = os.path.join(path, "added_packages") added_packages = "\n".join(new_packages - old_packages) + "\n" genio.write_file(added_path, added_packages) removed_path = os.path.join(self.basedir, "removed_packages") removed_packages = "\n".join(old_packages - new_packages) + "\n" genio.write_file(removed_path, removed_packages) def _save_sysinfo(self, log_hook, sysinfo_dir, optimized=False): try: file_path = os.path.join(sysinfo_dir, log_hook.name) with open(file_path, "wb") as log_file: for data in log_hook.collect(): log_file.write(data) if optimized: self._optimize(log_hook) except sysinfo.CollectibleException as e: log.debug(e.args[0]) except Exception as exc: # pylint: disable=W0703 log.error("Collection %s failed: %s", type(log_hook), exc) def _optimize(self, log_hook): pre_file = os.path.join(self.pre_dir, log_hook.name) post_file = os.path.join(self.post_dir, log_hook.name) if filecmp.cmp(pre_file, post_file): os.remove(post_file) log.debug("Not logging %s (no change detected)", log_hook.name) def start(self): """Log all collectibles at the start of the event.""" os.environ["AVOCADO_SYSINFODIR"] = self.pre_dir for log_hook in self.start_collectibles: # log daemons in profile directory if isinstance(log_hook, sysinfo.Daemon): try: log_hook.run() except sysinfo.CollectibleException as e: log.debug(e.args[0]) else: self._save_sysinfo(log_hook, self.pre_dir) if self.log_packages: self._log_installed_packages(self.pre_dir) def end(self, status=""): """ Logging hook called whenever a job finishes. """ optimized = self.config.get("sysinfo.collect.optimize") os.environ["AVOCADO_SYSINFODIR"] = self.post_dir for log_hook in self.end_collectibles: self._save_sysinfo(log_hook, self.post_dir, optimized) if status == "FAIL": for log_hook in self.end_fail_collectibles: self._save_sysinfo(log_hook, self.post_dir, optimized) # Stop daemon(s) started previously for log_hook in self.start_collectibles: if isinstance(log_hook, sysinfo.Daemon): self._save_sysinfo(log_hook, self.post_dir) if self.log_packages: self._log_modified_packages(self.post_dir) def collect_sysinfo(basedir): """ Collect sysinfo to a base directory. """ output.add_log_handler(log.name) if not basedir: cwd = os.getcwd() timestamp = time.strftime("%Y-%m-%d-%H.%M.%S") basedir = os.path.join(cwd, f"sysinfo-{timestamp}") sysinfo_logger = SysInfo(basedir=basedir) sysinfo_logger.start() sysinfo_logger.end() log.info("Logged system information to %s", basedir) avocado-113.0/avocado/core/tags.py000066400000000000000000000113231513561306000170060ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2014-2019 """ Test tags utilities module """ from avocado.core.resolver import ReferenceResolutionResult def _parse_filter_by_tags(filter_by_tags): """ Parses the various filter by tags in "command line" format The filtering of tests usually happens my means of "--filter-by-tags" command line options, and many can be given. This parses the contents of those into a list of must/must_not pairs, which can be used directly for comparisons when filtering. :param filter_by_tags: params in the format given to "-t/--filter-by-tags" :type filter_by_tags: list of str :returns: list of tuples with (set, set) """ result = [] for raw_tags in filter_by_tags: required_tags = raw_tags.split(",") must = set() must_not = set() for tag in required_tags: if tag.startswith("-"): must_not.add(tag[1:]) else: must.add(tag) result.append((must, must_not)) return result def _must_split_flat_key_val(must): """ Splits the flat and key:val tags apart :returns: the flat set tags and the key:val tags :rtype: tuple(set, dict) """ key_val = {} flat = set() for i in must: if ":" in i: k, v = i.split(":", 1) key_val[k] = v else: flat.add(i) return flat, key_val def _must_key_val_matches(must_key_vals, test_tags, include_empty_key): """ Checks if the required key:vals are fulfilled by the test_tags :rtype: bool """ key_val_test_tags = {} for k, v in test_tags.items(): if v is None: continue key_val_test_tags[k] = v for k, v in must_key_vals.items(): if k not in test_tags: if include_empty_key: continue else: return False if v.startswith("-"): abs_v = v[1:] if abs_v in key_val_test_tags.get(k, set()): return False elif v not in key_val_test_tags.get(k, set()): return False return True def filter_tags_on_runnables( resolutions, filter_by_tags, include_empty=False, include_empty_key=False ): """ Filters out runnables that do not match the tags criteria given The filtering mechanism is agnostic to test type. It means that if users request filtering by tag and the specific test type does not populate the test tags, it will be considered to have empty tags. :param resolutions: possible multiple resolutions for multiple references :type resolutions: list of :class:`avocado.core.resolver.ReferenceResolution` :param filter_by_tags: the list of tag sets to use as filters :type filter_by_tags: list of comma separated tags (['foo,bar', 'fast']) :param include_empty: if true tests without tags will not be filtered out :type include_empty: bool :param include_empty_key: if true tests "keys" on key:val tags will be included in the filtered results :type include_empty_key: bool :returns: the resolutions converted to runnables filtered by tags :rtype: list of :class:`avocado.core.nrunner.Runnable` """ filtered = [] must_must_nots = _parse_filter_by_tags(filter_by_tags) for resolution in resolutions: if resolution.result != ReferenceResolutionResult.SUCCESS: continue for runnable in resolution.resolutions: test_tags = runnable.tags or {} if not test_tags: if include_empty: filtered.append(runnable) continue for must, must_not in must_must_nots: if must_not.intersection(test_tags): continue must_flat, must_key_val = _must_split_flat_key_val(must) if must_key_val: if not _must_key_val_matches( must_key_val, test_tags, include_empty_key ): continue if must_flat: if not must_flat.issubset(test_tags): continue filtered.append(runnable) break return filtered avocado-113.0/avocado/core/tapparser.py000066400000000000000000000157741513561306000200670ustar00rootroot00000000000000import enum import re from collections import namedtuple @enum.unique class TestResult(enum.Enum): PASS = "PASS" SKIP = "SKIP" FAIL = "FAIL" XFAIL = "XFAIL" XPASS = "XPASS" # TapParser is based on Meson's TAP parser, which were licensed under the # MIT (X11) license and were contributed to both Meson and Avocado by the # same author (Paolo). class TapParser: Plan = namedtuple("Plan", ["count", "late", "skipped", "explanation"]) Bailout = namedtuple("Bailout", ["message"]) Test = namedtuple("Test", ["number", "name", "result", "explanation"]) Error = namedtuple("Error", ["message"]) Version = namedtuple("Version", ["version"]) _MAIN = 1 _AFTER_TEST = 2 _YAML = 3 _RE_BAILOUT = re.compile(r"Bail out!\s*(.*)") _RE_DIRECTIVE = re.compile( r"(?:\s*(?:#|\\{2}#)\s*([Ss][Kk][Ii][Pp]\S*|[Tt][Oo][Dd][Oo])\b\s*(.*))?" ) _RE_PLAN = re.compile(r"(?: {4})*1\.\.([0-9]+)" + _RE_DIRECTIVE.pattern) _RE_TEST = re.compile( r"(?: {4})*((?:not )?ok)\s*(?:([0-9]+)\s*)?\s*(.*?(?=(? line_indent_size: self.last_line = line break self.lineno += 1 # YAML blocks are only accepted after a test if state == self._AFTER_TEST: if version >= 13: m = self._RE_YAML_START.match(line) if m: state = self._YAML yaml_lineno = self.lineno yaml_indent = m.group(1) continue state = self._MAIN elif state == self._YAML: if self._RE_YAML_END.match(line): state = self._MAIN continue if line.startswith(yaml_indent): continue yield self.Error( f"YAML block not terminated (started on line {int(yaml_lineno)})" ) state = self._MAIN assert state == self._MAIN if line.startswith("#"): continue m = self._RE_TEST.match(line) if m: if plan and plan.late and not found_late_test: yield self.Error("unexpected test after late plan") found_late_test = True num_tests += 1 num = num_tests if m.group(2) is None else int(m.group(2)) if num != num_tests: yield self.Error("out of order test numbers") yield from self.parse_test( m.group(1) == "ok", num, m.group(3), m.group(4), m.group(5) ) state = self._AFTER_TEST continue m = self._RE_PLAN.match(line) if m: if plan: yield self.Error("more than one plan found") else: count = int(m.group(1)) skipped = count == 0 if m.group(2): if m.group(2).upper().startswith("SKIP"): if count > 0: yield self.Error("invalid SKIP directive for plan") skipped = True else: yield self.Error("invalid directive for plan") plan = self.Plan( count=count, late=(num_tests > 0), skipped=skipped, explanation=m.group(3), ) yield plan continue m = self._RE_BAILOUT.match(line) if m: yield self.Bailout(m.group(1)) bailed_out = True continue m = self._RE_VERSION.match(line) if m: # The TAP version is only accepted as the first line if self.lineno != 1: yield self.Error("version number must be on the first line") continue version = int(m.group(1)) if version < 13: yield self.Error("version number should be at least 13") else: yield self.Version(version=version) continue if line == "": continue if state == self._YAML: yield self.Error( f"YAML block not terminated (started on line {int(yaml_lineno)})" ) if not bailed_out and plan and num_tests != plan.count: if num_tests < plan.count: yield self.Error( f"Too few tests run (expected " f"{int(plan.count)}, got {int(num_tests)})" ) else: yield self.Error( f"Too many tests run (expected " f"{int(plan.count)}, got {int(num_tests)})" ) def parse(self): self.lineno = 0 yield from self._parse() avocado-113.0/avocado/core/task/000077500000000000000000000000001513561306000164405ustar00rootroot00000000000000avocado-113.0/avocado/core/task/__init__.py000066400000000000000000000000001513561306000205370ustar00rootroot00000000000000avocado-113.0/avocado/core/task/runtime.py000066400000000000000000000341061513561306000205010ustar00rootroot00000000000000import itertools import os from enum import Enum from avocado.core.dispatcher import TestPostDispatcher, TestPreDispatcher from avocado.core.nrunner.task import TASK_DEFAULT_CATEGORY, Task from avocado.core.test_id import TestID class RuntimeTaskStatus(Enum): INTERRUPTED = "FINISHED INTERRUPTED" WAIT_DEPENDENCIES = "WAITING DEPENDENCIES" WAIT = "WAITING" FINISHED = "FINISHED" TIMEOUT = "FINISHED TIMEOUT" IN_CACHE = "FINISHED IN CACHE" FAILFAST = "FINISHED FAILFAST" FAIL_TRIAGE = "FINISHED WITH FAILURE ON TRIAGE" FAIL_START = "FINISHED FAILING TO START" STARTED = "STARTED" @staticmethod def finished_statuses(): return [ status for _, status in RuntimeTaskStatus.__members__.items() if "FINISHED" in status.value ] class RuntimeTaskMixin: """Common utilities for RuntimeTask implementations.""" @classmethod def get_identifier( cls, runnable, no_digits, index, test_suite_name=None, ): if test_suite_name: prefix = f"{test_suite_name}-{index}" else: prefix = index if cls.category is TASK_DEFAULT_CATEGORY: name = runnable.identifier else: name = f'{runnable.kind}-{runnable.kwargs.get("name")}' return TestID(prefix, name, runnable.variant, no_digits) @classmethod def from_runnable( cls, runnable, no_digits, index, base_dir, test_suite_name=None, status_server_uri=None, job_id=None, satisfiable_deps_execution_statuses=None, ): """Creates runtime task for test from runnable :param runnable: the "description" of what the task should run. :type runnable: :class:`avocado.core.nrunner.Runnable` :param no_digits: number of digits of the test uid :type no_digits: int :param index: index of tests inside test suite :type index: int :param base_dir: Path to the job base directory. :type base_dir: str :param test_suite_name: test suite name which this test is related to :type test_suite_name: str :param status_server_uri: the URIs for the status servers that this task should send updates to. :type status_server_uri: list :param job_id: the ID of the job, for authenticating messages that get sent to the destination job's status server and will make into the job's results. :type job_id: str :param satisfiable_deps_execution_statuses: The dependency result types that satisfy the execution of this RuntimeTask. :type satisfiable_deps_execution_statuses: list of test results. :returns: RuntimeTask of the test from runnable """ test_id = cls.get_identifier(runnable, no_digits, index, test_suite_name) if not runnable.output_dir: runnable.output_dir = os.path.join(base_dir, test_id.str_filesystem) # handles the test task task = Task( runnable, identifier=test_id, status_uris=status_server_uri, category=cls.category, job_id=job_id, ) return cls(task, satisfiable_deps_execution_statuses) class RuntimeTask(RuntimeTaskMixin): """Task with extra status information on its life cycle status. The :class:`avocado.core.nrunner.Task` class contains information that is necessary to describe its persistence and execution by itself. This class wraps a :class:`avocado.core.nrunner.Task`, with extra information about its execution by a spawner within a state machine. """ category = TASK_DEFAULT_CATEGORY def __init__(self, task, satisfiable_deps_execution_statuses=None): """Instantiates a new RuntimeTask. :param task: The task to keep additional information about :type task: :class:`avocado.core.nrunner.Task` :param satisfiable_deps_execution_statuses: The dependency result types that satisfy the execution of this RuntimeTask. :type satisfiable_deps_execution_statuses: list of test results. """ #: The :class:`avocado.core.nrunner.Task` self.task = task #: The task status, a value from the enum #: :class:`avocado.core.task.runtime.RuntimeTaskStatus` self.status = None #: Information about task result when it is finished self._result = None #: Timeout limit for the completion of the task execution self.execution_timeout = None #: A handle that may be set by a spawner, and that may be #: spawner implementation specific, to keep track the task #: execution. This may be a PID, a container ID, a FQDN+PID #: etc. self.spawner_handle = None #: The result of the spawning of a Task self.spawning_result = None self.dependencies = [] self._satisfiable_deps_execution_statuses = ["pass"] if satisfiable_deps_execution_statuses: self._satisfiable_deps_execution_statuses = [ status.lower() for status in satisfiable_deps_execution_statuses ] #: Flag to detect if the task should be save to cache self.is_cacheable = False def __repr__(self): if self.status is None: return f'' else: return ( f'' ) def __hash__(self): return hash(self.task.identifier) def __eq__(self, other): if isinstance(other, RuntimeTask): return hash(self) == hash(other) return False @property def result(self): return self._result @property def satisfiable_deps_execution_statuses(self): return self._satisfiable_deps_execution_statuses @result.setter def result(self, result): self._result = result.lower() def are_dependencies_finished(self): for dependency in self.dependencies: if dependency.status not in RuntimeTaskStatus.finished_statuses(): return False return True def get_finished_dependencies(self): """Returns all dependencies which already finished.""" return [ dep for dep in self.dependencies if dep.status in RuntimeTaskStatus.finished_statuses() ] def can_run(self): if not self.are_dependencies_finished(): return False for dependency in self.dependencies: if dependency.result not in self.satisfiable_deps_execution_statuses: return False return True class PrePostRuntimeTaskMixin(RuntimeTask): """Common utilities for PrePostRuntimeTask implementations.""" @classmethod def get_tasks_from_test_task( cls, test_task, no_digits, base_dir, test_suite_name=None, status_server_uri=None, job_id=None, suite_config=None, ): """Creates runtime tasks for preTest task from test task. :param test_task: Runtime test task. :type test_task: :class:`avocado.core.task.runtime.RuntimeTask` :param no_digits: number of digits of the test uid :type no_digits: int :param base_dir: Path to the job base directory. :type base_dir: str :param test_suite_name: test suite name which this test is related to :type test_suite_name: str :param status_server_uri: the URIs for the status servers that this task should send updates to. :type status_server_uri: list :param job_id: the ID of the job, for authenticating messages that get sent to the destination job's status server and will make into the job's results. :type job_id: str :param suite_config: Configuration dict relevant for the whole suite. :type suite_config: dict :returns: Pre/Post RuntimeTasks of the dependencies from runnable :rtype: list """ tasks = [] plugins = cls.dispatcher().get_extentions_by_priority() runnable = test_task.task.runnable prefix = f"{test_task.task.identifier.str_filesystem}" for plugin in plugins: plugin = plugin.obj is_cacheable = getattr(plugin, "is_cacheable", False) test_runnables_method = getattr(plugin, f"{cls.category}_runnables") runnables = test_runnables_method(runnable, suite_config) for runnable in runnables: satisfiable_deps_execution_statuses = None if isinstance(runnable, tuple): runnable, satisfiable_deps_execution_statuses = runnable output_dir_not_exists = runnable.output_dir is None task = cls.from_runnable( runnable, no_digits, prefix, base_dir, test_suite_name, status_server_uri, job_id, satisfiable_deps_execution_statuses, ) if output_dir_not_exists: runnable.output_dir = os.path.join( os.path.abspath(os.path.join(base_dir, os.pardir)), "dependencies", str(task.task.identifier), ) task.task.metadata["symlink"] = os.path.join( test_task.task.runnable.output_dir, "dependencies", f'{runnable.kind}-{runnable.kwargs.get("name")}', ) task.is_cacheable = is_cacheable tasks.append(task) return tasks class PreRuntimeTask(PrePostRuntimeTaskMixin): """Runtime task for tasks run before test""" category = "pre_test" dispatcher = TestPreDispatcher class PostRuntimeTask(PrePostRuntimeTaskMixin): """Runtime task for tasks run after test""" category = "post_test" dispatcher = TestPostDispatcher class RuntimeTaskGraph: """Graph representing dependencies between runtime tasks.""" def __init__( self, tests, test_suite_name, status_server_uri, job_id, base_dir, suite_config=None, ): """Instantiates a new RuntimeTaskGraph. From the list of tests, it will create runtime tasks and connects them inside the graph by its dependencies. :param tests: runnables from test suite :type tests: list :param test_suite_name: test suite name which this test is related to :type test_suite_name: str :param status_server_uri: the URIs for the status servers that this task should send updates to. :type status_server_uri: list :param job_id: the ID of the job, for authenticating messages that get sent to the destination job's status server and will make into the job's results. :type job_id: str :param base_dir: Path to the job base directory. :type base_dir: str :param suite_config: Configuration dict relevant for the whole suite. :type suite_config: dict """ self.graph = {} # create graph no_digits = len(str(len(tests))) for index, runnable in enumerate(tests, start=1): runtime_test = RuntimeTask.from_runnable( runnable, no_digits, index, base_dir, test_suite_name, status_server_uri, job_id, ) self.graph[runtime_test] = runtime_test # with --dry-run we don't want to run dependencies if runnable.kind != "dry-run": pre_tasks = PreRuntimeTask.get_tasks_from_test_task( runtime_test, no_digits, base_dir, test_suite_name, status_server_uri, job_id, suite_config, ) post_tasks = PostRuntimeTask.get_tasks_from_test_task( runtime_test, no_digits, base_dir, test_suite_name, status_server_uri, job_id, suite_config, ) if pre_tasks or post_tasks: self._connect_tasks(pre_tasks, [runtime_test], post_tasks) def _connect_tasks(self, pre_tasks, tasks, post_tasks): connections = list(itertools.product(pre_tasks, tasks)) connections += list(itertools.product(tasks, post_tasks)) for dependency, task in connections: self.graph[task] = task self.graph[dependency] = dependency task.dependencies.append(dependency) def get_tasks_in_topological_order(self): """Computes the topological order of runtime tasks in graph :returns: runtime tasks in topological order :rtype: list """ def topological_order_util(vertex, visited, topological_order): visited[vertex] = True for v in vertex.dependencies: if not visited[v]: topological_order_util(v, visited, topological_order) topological_order.append(vertex) visited = dict.fromkeys(self.graph, False) topological_order = [] for vertex in self.graph.values(): if not visited[vertex]: topological_order_util(vertex, visited, topological_order) return topological_order avocado-113.0/avocado/core/task/statemachine.py000066400000000000000000000470611513561306000214670ustar00rootroot00000000000000import asyncio import collections import logging import multiprocessing import time from avocado.core.exceptions import JobFailFast from avocado.core.task.runtime import RuntimeTaskStatus from avocado.core.teststatus import STATUSES_NOT_OK from avocado.core.utils import messages LOG = logging.getLogger(__name__) class TaskStateMachine: """Represents all phases that a task can go through its life.""" def __init__(self, tasks, status_repo): self._requested = collections.deque(tasks) self._status_repo = status_repo self._triaging = [] self._ready = [] self._started = [] self._monitored = [] self._finished = [] self._lock = asyncio.Lock() self._cache_lock = asyncio.Lock() self._task_size = len(tasks) self._tasks_by_id = { str(runtime_task.task.identifier): runtime_task.task for runtime_task in tasks } @property def requested(self): return self._requested @property def triaging(self): return self._triaging @property def ready(self): return self._ready @property def started(self): return self._started @property def monitored(self): return self._monitored @property def finished(self): return self._finished @property def lock(self): return self._lock @property def cache_lock(self): return self._cache_lock @property def task_size(self): return self._task_size @property async def complete(self): async with self._lock: pending = any([self._requested, self._triaging, self._ready, self._started]) return not pending @property def tasks_by_id(self): return self._tasks_by_id async def add_new_task(self, runtime_task): async with self.lock: self._requested.appendleft(runtime_task) self._tasks_by_id[str(runtime_task.task.identifier)] = runtime_task.task return async def abort(self, status_reason=None): """Abort all non-started tasks. This method will move all non-started tasks to finished with a specific reason. :param status_reason: string reason. Optional. """ await self.abort_queue("requested", status_reason) await self.abort_queue("triaging", status_reason) await self.abort_queue("ready", status_reason) async def abort_queue(self, queue_name, status_reason=None): """Abort all tasks inside a specific queue adding a status reason. :param queue_name: a string with the queue name. :param status_reason: string reason. Optional. """ to_remove = [] async with self._lock: queue = getattr(self, queue_name) for _ in range(len(queue)): if queue_name == "requested": runtime_task = queue.popleft() else: runtime_task = queue.pop(0) to_remove.append(runtime_task) if to_remove: if status_reason: LOG.debug( 'Aborting queue "%s" by finishing %u tasks: %s', queue_name, len(to_remove), status_reason, ) else: LOG.debug( 'Aborting queue "%s" by finishing %u tasks', queue_name, len(to_remove), ) for task in to_remove: await self.finish_task(task, status_reason) async def finish_task(self, runtime_task, status_reason=None): """Include a task to the finished queue with a specific reason. This method is assuming that you have removed (pop) the task from the original queue. :param runtime_task: A running task object. :param status_reason: string reason. Optional. """ async with self._lock: if runtime_task not in self.finished: if status_reason: runtime_task.status = status_reason LOG.debug( 'Task "%s" finished with status: %s', runtime_task.task.identifier, status_reason, ) else: LOG.debug('Task "%s" finished', runtime_task.task.identifier) self.finished.append(runtime_task) class Worker: def __init__( self, state_machine, spawner, max_triaging=None, max_running=None, task_timeout=None, failfast=False, ): self._state_machine = state_machine self._spawner = spawner if max_triaging is None: max_triaging = multiprocessing.cpu_count() self._max_triaging = max_triaging if max_running is None: max_running = 2 * multiprocessing.cpu_count() - 1 self._max_running = max_running self._task_timeout = task_timeout self._failfast = failfast LOG.debug("%s has been initialized", self) def __repr__(self): fmt = '' return fmt.format( self._spawner, self._max_triaging, self._max_running, self._task_timeout ) async def _send_finished_tasks_message(self, terminate_tasks, reason): """Sends messages related to tasks being terminated to status repository. On normal conditions, the "avocado-runner-*" will produce messages finishing each task. But, under some conditions (such as timeouts, interruptions requested by users, etc), it's necessary to do this on the runner's behalf. When a task is terminated, it is necessary to send a "finish" message with the correct fail reason to the status repository, which will close logging. This method will also send a "log" message with the reason (timeout, user interruption, etc). :param terminate_tasks: runtime_tasks which were terminated and need to have messages sent on their behalf :type terminate_tasks: list :param reason: a description of what caused the task interruption (timeout, user requested interruption, etc). :type reason: str """ for terminated_task in terminate_tasks: task_id = str(terminated_task.task.identifier) job_id = terminated_task.task.job_id log_message = messages.LogMessage.get( f"{time.strftime('%Y-%m-%d %H:%M:%S', time.localtime())} | Test interrupted: {reason}", id=task_id, job_id=job_id, ) finish_message = messages.FinishedMessage.get( "interrupted", f"Test interrupted: {reason}", id=task_id, job_id=job_id ) try: current_status, _ = self._state_machine._status_repo._status[task_id] except KeyError: return if current_status != "finished": self._state_machine._status_repo.process_message(log_message) self._state_machine._status_repo.process_message(finish_message) async def bootstrap(self): """Reads from requested, moves into triaging.""" try: async with self._state_machine.lock: if len(self._state_machine.triaging) < self._max_triaging: runtime_task = self._state_machine.requested.popleft() self._state_machine.triaging.append(runtime_task) LOG.debug( 'Task "%s": requested -> triaging', runtime_task.task.identifier ) else: return except IndexError: return async def triage(self): """Reads from triaging, moves into either: ready or finished.""" try: async with self._state_machine.lock: runtime_task = self._state_machine.triaging.pop(0) except IndexError: return # a task waiting requirements already checked its requirements if runtime_task.status != RuntimeTaskStatus.WAIT_DEPENDENCIES: # check for requirements a task may have requirements_ok = await self._spawner.check_task_requirements(runtime_task) if requirements_ok: LOG.debug( 'Task "%s": requirements OK (will proceed to check ' "dependencies)", runtime_task.task.identifier, ) else: await self._state_machine.finish_task( runtime_task, RuntimeTaskStatus.FAIL_TRIAGE ) return # handle task dependencies if runtime_task.dependencies: # check of all the dependency tasks finished if not runtime_task.are_dependencies_finished(): async with self._state_machine.lock: self._state_machine.triaging.append(runtime_task) runtime_task.status = RuntimeTaskStatus.WAIT_DEPENDENCIES await asyncio.sleep(0.1) return # dependencies finished, let's check if they finished # successfully, so we can move on with the parent task dependencies_ok = runtime_task.can_run() if not dependencies_ok: LOG.debug( 'Task "%s" has failed dependencies', runtime_task.task.identifier ) task_id = str(runtime_task.task.identifier) job_id = runtime_task.task.job_id reason = "Dependency was not fulfilled." start_message = messages.StartedMessage.get( output_dir=runtime_task.task.runnable.output_dir, id=task_id, job_id=job_id, ) log_message = messages.LogMessage.get( f"{time.strftime('%Y-%m-%d %H:%M:%S', time.localtime())} | {reason}", id=task_id, job_id=job_id, ) finish_message = messages.FinishedMessage.get( "skip", reason, id=task_id, job_id=job_id ) self._state_machine._status_repo.process_message(start_message) self._state_machine._status_repo.process_message(log_message) self._state_machine._status_repo.process_message(finish_message) runtime_task.result = "fail" await self._state_machine.finish_task( runtime_task, RuntimeTaskStatus.FAIL_TRIAGE ) return if runtime_task.task.category != "test": # save or retrieve task from cache if runtime_task.is_cacheable: async with self._state_machine.cache_lock: is_task_in_cache = await self._spawner.is_requirement_in_cache( runtime_task ) if is_task_in_cache is None: async with self._state_machine.lock: self._state_machine.triaging.append(runtime_task) runtime_task.status = RuntimeTaskStatus.WAIT await asyncio.sleep(0.1) return if is_task_in_cache: task_id = str(runtime_task.task.identifier) job_id = runtime_task.task.job_id start_message = messages.StartedMessage.get( output_dir=runtime_task.task.runnable.output_dir, id=task_id, job_id=job_id, ) log_message = messages.LogMessage.get( f"{time.strftime('%Y-%m-%d %H:%M:%S', time.localtime())} | " f"Dependency fulfilled from cache.", id=task_id, job_id=job_id, ) finish_message = messages.FinishedMessage.get( "pass", id=task_id, job_id=job_id ) self._state_machine._status_repo.process_message(start_message) self._state_machine._status_repo.process_message(log_message) self._state_machine._status_repo.process_message(finish_message) await self._state_machine.finish_task( runtime_task, RuntimeTaskStatus.IN_CACHE ) runtime_task.result = "pass" return await self._spawner.save_requirement_in_cache(runtime_task) # the task is ready to run async with self._state_machine.lock: self._state_machine.ready.append(runtime_task) async def start(self): """Reads from ready, moves into either: started or finished.""" try: async with self._state_machine.lock: runtime_task = self._state_machine.ready.pop(0) except IndexError: return # enforce a rate limit on the number of started (currently # running) tasks. this is a global limit, but the spawners # can also be queried with regards to their capacity to handle # new tasks should_wait = False async with self._state_machine.lock: if len(self._state_machine.started) >= self._max_running: self._state_machine.ready.insert(0, runtime_task) runtime_task.status = RuntimeTaskStatus.WAIT should_wait = True if should_wait: await asyncio.sleep(0.1) return LOG.debug( 'Task "%s": about to be spawned with "%s"', runtime_task.task.identifier, self._spawner, ) start_ok = await self._spawner.spawn_task(runtime_task) if start_ok: LOG.debug('Task "%s": spawned successfully', runtime_task.task.identifier) runtime_task.status = RuntimeTaskStatus.STARTED if self._task_timeout is not None: runtime_task.execution_timeout = time.monotonic() + self._task_timeout async with self._state_machine.lock: self._state_machine.started.append(runtime_task) else: await self._state_machine.finish_task( runtime_task, RuntimeTaskStatus.FAIL_START ) async def monitor(self): """Reads from started, moves into finished.""" try: async with self._state_machine.lock: runtime_task = self._state_machine.started.pop(0) except IndexError: return if self._spawner.is_task_alive(runtime_task): LOG.debug( 'Task "%s" is alive at monitor phase', runtime_task.task.identifier ) async with self._state_machine.lock: self._state_machine._monitored.append(runtime_task) try: if runtime_task.execution_timeout is None: remaining = None else: remaining = runtime_task.execution_timeout - time.monotonic() await asyncio.wait_for(self._spawner.wait_task(runtime_task), remaining) except asyncio.TimeoutError: await self._terminate_task(runtime_task, RuntimeTaskStatus.TIMEOUT) await self._send_finished_tasks_message( [runtime_task], "Timeout reached" ) async with self._state_machine.lock: try: self._state_machine.monitored.remove(runtime_task) except ValueError: # runtime_task has been terminated, there is no need to continue return else: LOG.debug( 'Task "%s" was very short lived, this may be ' "completely normal depending on the task itself. " "Please check the task logs", runtime_task.task.identifier, ) # from here, this `task` ran, so, let's check # its latest data in the status repo latest_task_data = ( self._state_machine._status_repo.get_latest_task_data( str(runtime_task.task.identifier) ) or {} ) # or maybe its results are not available yet while latest_task_data.get("result") is None: await asyncio.sleep(0.1) latest_task_data = ( self._state_machine._status_repo.get_latest_task_data( str(runtime_task.task.identifier) ) or {} ) if runtime_task.task.category != "test": async with self._state_machine.cache_lock: await self._spawner.update_requirement_cache( runtime_task, latest_task_data["result"].upper() ) runtime_task.result = latest_task_data["result"] result_stats = set( key.upper() for key in self._state_machine._status_repo.result_stats.keys() ) if self._failfast and not result_stats.isdisjoint(STATUSES_NOT_OK): await self._state_machine.abort(RuntimeTaskStatus.FAILFAST) raise JobFailFast("Interrupting job (failfast).") await self._state_machine.finish_task(runtime_task, RuntimeTaskStatus.FINISHED) async def _terminate_task(self, runtime_task, task_status): runtime_task.status = task_status terminate_result = await self._spawner.terminate_task(runtime_task) if not terminate_result: LOG.error('Could not terminate task "%s"', runtime_task.task.identifier) async def _terminate_tasks(self, task_status): await self._state_machine.abort(task_status) terminated = [] while True: async with self._state_machine.lock: try: runtime_task = self._state_machine.monitored.pop(0) await self._terminate_task(runtime_task, task_status) terminated.append(runtime_task) except IndexError: if ( len(self._state_machine.finished) + len(terminated) == self._state_machine.task_size ): break return terminated async def terminate_tasks_timeout(self): """Terminate all running tasks with a timeout message.""" task_status = RuntimeTaskStatus.TIMEOUT terminated = await self._terminate_tasks(task_status) await self._send_finished_tasks_message(terminated, "Timeout reached") async def terminate_tasks_interrupted(self): """Terminate all running tasks with an interrupted message.""" task_status = RuntimeTaskStatus.INTERRUPTED terminated = await self._terminate_tasks(task_status) await self._send_finished_tasks_message(terminated, "Interrupted by user") async def run(self): """Pushes Tasks forward and makes them do something with their lives.""" while True: is_complete = await self._state_machine.complete if is_complete: break await self.bootstrap() await self.triage() await self.start() await self.monitor() avocado-113.0/avocado/core/test.py000066400000000000000000000753251513561306000170430ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; specifically version 2 of the License. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # This code was inspired in the autotest project, # client/shared/test.py # Authors: Martin J Bligh , # Andy Whitcroft """ Contains the base test implementation, used as a base for the actual framework tests. """ import asyncio import functools import inspect import logging import multiprocessing import multiprocessing.pool import os import shutil import sys import tempfile import time import unittest import warnings from avocado.core import exceptions, parameters from avocado.core.settings import settings from avocado.core.test_id import TestID from avocado.core.teststatus import STATUSES_NOT_OK from avocado.core.version import VERSION from avocado.utils import asset, astring, data_structures from avocado.utils import path as utils_path from avocado.utils import stacktrace #: Environment variable used to store the location of a temporary #: directory which is preserved across all tests execution (usually in #: one job) COMMON_TMPDIR_NAME = "AVOCADO_TESTS_COMMON_TMPDIR" #: The list of test attributes that are used as the test state, which #: is given to the test runner via the queue they share TEST_STATE_ATTRIBUTES = ( "name", "logdir", "logfile", "status", "running", "paused", "time_start", "time_elapsed", "time_end", "actual_time_start", "actual_time_end", "fail_reason", "fail_class", "traceback", "tags", "timeout", "whiteboard", "phase", ) class TestData: """ Class that adds the ability for tests to have access to data files Writers of new test types can change the completely change the behavior and still be compatible by providing an :attr:`DATA_SOURCES` attribute and a meth:`get_data` method. """ #: Defines the name of data sources that this implementation makes #: available. Users may choose to pick data file from a specific #: source. DATA_SOURCES = ["variant", "test", "file"] #: The name of data directory associated with a file name is expected #: to match the file name, plus this suffix. SUFFIX = ".data" def __init__(self): file_datadir = None if ( self.filename is not None and len(os.path.basename(self.filename)) <= self._max_name_length() ): file_datadir = self.filename + self.SUFFIX self._data_sources_mapping = { "variant": [ lambda: file_datadir, lambda: (f"{self.__class__.__name__}." f"{self._testMethodName}"), lambda: self.name.variant, ], "test": [ lambda: file_datadir, lambda: (f"{self.__class__.__name__}." f"{self._testMethodName}"), ], "file": [lambda: file_datadir], } def _max_name_length(self): """Returns the maximum length for test file names The maximum file name is filesystem dependent, and besides that, standard data directory based on test file names has a suffix (:data:`SUFFIX`) that has to be accounted for. """ max_length = utils_path.get_max_file_name_length(self.filename) return max_length - len(self.SUFFIX) def _check_valid_data_source(self, source): """ Utility to check if user chose a specific data source :param source: either None for no specific selection or a source name :type source: None or str :raises: ValueError """ if source is not None and source not in self.DATA_SOURCES: msg = "Data file source requested (%s) is not one of: %s" msg %= (source, ", ".join(self.DATA_SOURCES)) raise ValueError(msg) def _get_datadir(self, source): path_components = self._data_sources_mapping.get(source) if path_components is None: return # evaluate lazily, needed when the class changes its own # information such as its datadir path_components = [func() for func in path_components] if None in path_components: return # if path components are absolute paths, let's believe that # they have already been treated (such as the entries that # return the self.datadir). If not, let's split the path # components so that they can be treated in the next loop split_path_components = [] for path_component in path_components: if not os.path.isabs(path_component): split_path_components += path_component.split(os.path.sep) else: split_path_components.append(path_component) # now, make sure each individual path component can be represented # in the filesystem. again, if it's an absolute path, do nothing paths = [] for path in split_path_components: if os.path.isabs(path): paths.append(path) else: paths.append(astring.string_to_safe_path(path)) return os.path.join(*paths) def get_data(self, filename, source=None, must_exist=True): """ Retrieves the path to a given data file. This implementation looks for data file in one of the sources defined by the :attr:`DATA_SOURCES` attribute. :param filename: the name of the data file to be retrieved :type filename: str :param source: one of the defined data sources. If not set, all of the :attr:`DATA_SOURCES` will be attempted in the order they are defined :type source: str :param must_exist: whether the existence of a file is checked for :type must_exist: bool :rtype: str or None """ log_fmt = "DATA (filename=%s) => %s (%s)" if source is None: sources = self.DATA_SOURCES else: self._check_valid_data_source(source) sources = [source] for attempt_source in sources: datadir = self._get_datadir(attempt_source) if datadir is not None: # avoid returning a slash after the data directory name # when a file was not requested (thus return the data # directory itself) if not filename: path = datadir else: path = os.path.join(datadir, filename) if not must_exist: self.log.debug( log_fmt, filename, path, (f"assumed to be located at " f"{attempt_source} source dir"), ) return path else: if os.path.exists(path): self.log.debug( log_fmt, filename, path, f"found at {attempt_source} source dir", ) return path self.log.debug( log_fmt, filename, "NOT FOUND", f"data sources: {', '.join(sources)}" ) class Test(unittest.TestCase, TestData): """ Base implementation for the test class. You'll inherit from this to write your own tests. Typically you'll want to implement setUp(), test*() and tearDown() methods on your own tests. """ #: Arbitrary string which will be stored in `$logdir/whiteboard` location #: when the test finishes. whiteboard = "" #: (unix) time when the test started, monotonic (could be forced from test) time_start = -1 #: (unix) time when the test finished, monotonic (could be forced from test) time_end = -1 #: duration of the test execution (always recalculated from time_end - #: time_start time_elapsed = -1 #: (unix) time when the test started, actual one to be shown to users actual_time_start = -1 #: (unix) time when the test finished, actual one to be shown to users actual_time_end = -1 #: Test timeout (the timeout from params takes precedence) timeout = None def __init__( self, methodName="test", name=None, params=None, base_logdir=None, config=None, tags=None, ): """ Initializes the test. :param methodName: Name of the main method to run. For the sake of compatibility with the original unittest class, you should not set this. :param name: Pretty name of the test name. For normal tests, written with the avocado API, this should not be set. This is reserved for internal Avocado use, such as when running random executables as tests. :type name: :class:`avocado.core.test.TestID` :param base_logdir: Directory where test logs should go. If None provided a temporary directory will be created. :param config: the job configuration, usually set by command line options and argument parsing :type config: dict """ self.__phase = "INIT" def record_and_warn(*args, **kwargs): """Record call to this function and log warning""" if not self.__log_warn_used: self.__log_warn_used = True return original_log_warn(*args, **kwargs) if name is not None: self.__name = name else: self.__name = TestID(0, self.__class__.__name__) self.__tags = tags self._config = config or settings.as_dict() self.__base_logdir = base_logdir self.__base_logdir_tmp = None if self.__base_logdir is None: prefix = "avocado_test_" self.__base_logdir_tmp = tempfile.TemporaryDirectory(prefix=prefix) self.__base_logdir = self.__base_logdir_tmp.name self.__logfile = os.path.join(self.logdir, "debug.log") self._stdout_file = os.path.join(self.logdir, "stdout") self._stderr_file = os.path.join(self.logdir, "stderr") self._output_file = os.path.join(self.logdir, "output") self._logging_handlers = {} self.__outputdir = utils_path.init_dir(self.logdir, "data") self.__log = logging.getLogger("avocado.test") original_log_warn = self.log.warning self.__log_warn_used = False self.log.warn = self.log.warning = record_and_warn self.log.info("INIT %s", self.name) paths = ["/test/*"] if params is None: params = [] elif isinstance(params, tuple): params, paths = params[0], params[1] self.__params = parameters.AvocadoParams(params, paths, self.__log.name) self.timeout = original_timeout = self.params.get( "timeout", default=self.timeout ) timeout_factor = float(self.params.get("timeout_factor", default=1.0)) if original_timeout is not None: self.timeout = float(original_timeout) * timeout_factor self.__status = None self.__fail_reason = None self.__fail_class = None self.__traceback = None self.__skip_test = False # Are initialized lazily self.__cache_dirs = None self.__base_tmpdir = None self.__workdir = None self.__running = False self.paused = False self.paused_msg = "" self.log.debug("Test metadata:") if self.filename: self.log.debug(" filename: %s", self.filename) try: teststmpdir = self.teststmpdir except EnvironmentError: pass else: self.log.debug(" teststmpdir: %s", teststmpdir) self.log.debug(" original timeout: %s", original_timeout) self.log.debug(" timeout factor: %s", timeout_factor) self.log.debug(" actual timeout: %s", self.timeout) unittest.TestCase.__init__(self, methodName=methodName) TestData.__init__(self) @property def _base_tmpdir(self): if self.__base_tmpdir is None: self.__base_tmpdir = tempfile.mkdtemp( prefix="tmp_dir", dir=self.__base_logdir ) return self.__base_tmpdir @property def name(self): """ Returns the Test ID, which includes the test name :rtype: TestID """ return self.__name @property def tags(self): """ The tags associated with this test """ return self.__tags @property def log(self): """ The enhanced test log """ return self.__log @property def logdir(self): """ Path to this test's logging dir """ return self.__base_logdir @property def logfile(self): """ Path to this test's main `debug.log` file """ return self.__logfile @property def outputdir(self): """ Directory available to test writers to attach files to the results """ return self.__outputdir @property def params(self): """ Parameters of this test (AvocadoParam instance) """ return self.__params @property def basedir(self): """ The directory where this test (when backed by a file) is located at """ if self.filename is not None: return os.path.dirname(self.filename) else: return None @property def filename(self): """ Returns the name of the file (path) that holds the current test """ try: possibly_compiled = inspect.getabsfile(self.__class__) if possibly_compiled.endswith(".pyc") or possibly_compiled.endswith(".pyo"): source = possibly_compiled[:-1] else: source = possibly_compiled except TypeError: return None if os.path.exists(source): return source else: return None @property def teststmpdir(self): """ Returns the path of the temporary directory that will stay the same for all tests in a given Job. """ env_var = COMMON_TMPDIR_NAME path = os.environ.get(env_var) if path is None: msg = f"Environment Variable {env_var} is not set." raise EnvironmentError(msg) return path @property def workdir(self): """ This property returns a writable directory that exists during the entire test execution, but will be cleaned up once the test finishes. It can be used on tasks such as decompressing source tarballs, building software, etc. """ if self.__workdir is None: self.__workdir = os.path.join(self._base_tmpdir, self.name.str_filesystem) utils_path.init_dir(self.__workdir) self.log.debug("Test workdir initialized at: %s", self.__workdir) return self.__workdir @property def cache_dirs(self): """ Returns a list of cache directories as set in config file. """ if self.__cache_dirs is None: self.__cache_dirs = self._config.get("datadir.paths.cache_dirs") return self.__cache_dirs @property def status(self): """ The result status of this test """ return self.__status @property def running(self): """ Whether this test is currently being executed """ return self.__running @property def fail_reason(self): return self.__fail_reason @property def fail_class(self): return self.__fail_class @property def traceback(self): return self.__traceback @property def phase(self): """ The current phase of the test execution Possible (string) values are: INIT, SETUP, TEST, TEARDOWN and FINISHED """ return self.__phase def __str__(self): return str(self.name) def __repr__(self): return f"Test({self.name!r})" def _tag_start(self): self.log.info("START %s", self.name) self.__running = True self.time_start = time.monotonic() self.actual_time_start = time.time() def _tag_end(self): self.__running = False self.time_end = time.monotonic() self.actual_time_end = time.time() # for consistency sake, always use the same stupid method self._update_time_elapsed(self.time_end) def _update_time_elapsed(self, current_time=None): if current_time is None: current_time = time.monotonic() self.time_elapsed = current_time - self.time_start def get_state(self): """ Serialize selected attributes representing the test state :returns: a dictionary containing relevant test state data :rtype: dict """ if self.running and self.time_start: self._update_time_elapsed() state = {key: getattr(self, key, None) for (key) in TEST_STATE_ATTRIBUTES} state["class_name"] = self.__class__.__name__ state["params"] = [ (path, key, value) for path, key, value in self.__params.iteritems() ] return state def _run_test(self): """ Auxiliary method to run setup and test method. """ self._tag_start() testMethod = getattr(self, self._testMethodName) skip_test_condition = getattr(testMethod, "__skip_test_condition__", False) skip_test_condition_negate = getattr( testMethod, "__skip_test_condition_negate__", False ) if skip_test_condition: if callable(skip_test_condition): if skip_test_condition_negate: self.__skip_test = not bool(skip_test_condition(self)) else: self.__skip_test = bool(skip_test_condition(self)) else: if skip_test_condition_negate: self.__skip_test = not bool(skip_test_condition) else: self.__skip_test = bool(skip_test_condition) else: self.__skip_test = bool(skip_test_condition) try: if self.__skip_test is False: self.__phase = "SETUP" self.setUp() except exceptions.TestSkipError as details: self.__skip_test = True stacktrace.log_exc_info(sys.exc_info(), logger=self.log) raise exceptions.TestSkipError(details) except (exceptions.TestCancel, exceptions.TestInterrupt): stacktrace.log_exc_info(sys.exc_info(), logger=self.log) raise except: # Old-style exceptions are not inherited from Exception() stacktrace.log_exc_info(sys.exc_info(), logger=self.log) details = sys.exc_info()[1] raise exceptions.TestSetupFail(details) else: try: self.__phase = "TEST" if inspect.iscoroutinefunction(testMethod): loop = asyncio.get_event_loop() loop.run_until_complete(testMethod()) else: testMethod() except exceptions.TestCancel: stacktrace.log_exc_info(sys.exc_info(), logger=self.log) raise except: # Old-style exceptions are not inherited from Exception() pylint: disable=W0702 stacktrace.log_exc_info(sys.exc_info(), logger=self.log) details = sys.exc_info()[1] if not isinstance(details, Exception): # Avoid passing nasty exc details = exceptions.TestError(f"{details!r}: {details}") self.log.debug("Local variables:") local_vars = inspect.trace()[1][0].f_locals for key, value in local_vars.items(): self.log.debug(" -> %s %s: %s", key, type(value), value) raise details self.__status = "PASS" def _tearDown(self): """ Auxiliary method to run tearDown. """ try: if self.__skip_test is False: self.__phase = "TEARDOWN" self.tearDown() except exceptions.TestSkipError as details: stacktrace.log_exc_info(sys.exc_info(), logger=self.log) skip_illegal_msg = ( f"Using skip decorators in tearDown() " f"is not allowed in " f"avocado, you must fix your " f"test. Original skip exception: {details}" ) raise exceptions.TestError(skip_illegal_msg) except (exceptions.TestCancel, exceptions.TestInterrupt): stacktrace.log_exc_info(sys.exc_info(), logger=self.log) raise except: # avoid old-style exception failures pylint: disable=W0702 stacktrace.log_exc_info(sys.exc_info(), logger=self.log) if self.status != "INTERRUPTED": details = sys.exc_info()[1] raise exceptions.TestSetupFail(details) def _setup_environment_variables(self): os.environ["AVOCADO_VERSION"] = VERSION if self.basedir is not None: os.environ["AVOCADO_TEST_BASEDIR"] = self.basedir if self.__workdir is not None: os.environ["AVOCADO_TEST_WORKDIR"] = self.workdir os.environ["AVOCADO_TEST_LOGDIR"] = self.logdir os.environ["AVOCADO_TEST_LOGFILE"] = self.logfile os.environ["AVOCADO_TEST_OUTPUTDIR"] = self.outputdir def _catch_test_status(self, method): """Wrapper around test methods for catching and logging failures.""" def set_new_event_loop_for_method(method): asyncio.set_event_loop(asyncio.new_event_loop()) method() try: pool = multiprocessing.pool.ThreadPool(1) res = pool.apply_async(set_new_event_loop_for_method, [method]) pool.close() try: res.get(self.timeout) if self.__log_warn_used and self.__status not in STATUSES_NOT_OK: raise exceptions.TestWarn( "Test passed but there were warnings " "during execution. Check the log for " "details." ) except multiprocessing.TimeoutError: raise exceptions.TestInterrupt("Test interrupted: Timeout reached") except exceptions.TestBaseException as detail: self.__status = detail.status self.__fail_class = detail.__class__.__name__ self.__fail_reason = astring.to_text(detail) self.__traceback = stacktrace.prepare_exc_info(sys.exc_info()) except AssertionError as detail: self.__status = "FAIL" self.__fail_class = detail.__class__.__name__ self.__fail_reason = astring.to_text(detail) self.__traceback = stacktrace.prepare_exc_info(sys.exc_info()) except Exception as detail: # pylint: disable=W0703 self.__status = "ERROR" tb_info = stacktrace.tb_info(sys.exc_info()) self.__traceback = stacktrace.prepare_exc_info(sys.exc_info()) try: self.__fail_class = astring.to_text(detail.__class__.__name__) self.__fail_reason = astring.to_text(detail) except TypeError: self.__fail_class = "Exception" self.__fail_reason = ( "Unable to get exception, check the traceback for details." ) for e_line in tb_info: self.log.error(e_line) finally: pool.terminate() def run_avocado(self): """ Wraps the run method, for execution inside the avocado runner. :result: Unused param, compatibility with :class:`unittest.TestCase`. """ self._setup_environment_variables() self._catch_test_status(self._run_test) self._catch_test_status(self._tearDown) self.__phase = "FINISHED" self._tag_end() self._report() self.log.info("") def _report(self): """ Report result to the logging system. """ if self.fail_reason is not None: self.log.error( "%s %s -> %s: %s", self.status, self.name, self.fail_class, self.fail_reason, ) else: if self.status is None: self.__status = "INTERRUPTED" self.log.info("%s %s", self.status, self.name) def _deprecate_params_message(func): # pylint: disable=E0213 """This decorator helps to deprecate parameter 'message' and replace it with 'msg'. """ @functools.wraps(func) def wrapper(*args, **kwargs): msg = None message = None if "msg" in kwargs: msg = kwargs.get("msg") elif args: msg = args[0] if "message" in kwargs: message = kwargs.get("message") if msg: warnings.warn( "Please favor parameter 'msg' and do not use " "it with the parameter 'message'", DeprecationWarning, ) else: warnings.warn( "The parameter 'message' has been deprecated." " Please use 'msg' instead", DeprecationWarning, ) actual_message = msg or message return func(actual_message) return wrapper @staticmethod @_deprecate_params_message def fail(msg=None): # pylint: disable=W0221 """ Fails the currently running test. After calling this method a test will be terminated and have its status as FAIL. :param msg: an optional message that will be recorded in the logs :type msg: str """ raise exceptions.TestFail(msg) @staticmethod @_deprecate_params_message def error(msg=None): """ Errors the currently running test. After calling this method a test will be terminated and have its status as ERROR. :param msg: an optional message that will be recorded in the logs :type msg: str """ raise exceptions.TestError(msg) @staticmethod @_deprecate_params_message def cancel(msg=None): """ Cancels the test. This method is expected to be called from the test method, not anywhere else, since by definition, we can only cancel a test that is currently under execution. If you call this method outside the test method, avocado will mark your test status as ERROR, and instruct you to fix your test in the error message. :param msg: an optional message that will be recorded in the logs :type msg: str """ raise exceptions.TestCancel(msg) def fetch_asset( self, name, asset_hash=None, algorithm=None, locations=None, expire=None, find_only=False, cancel_on_missing=False, ): """ Method o call the utils.asset in order to fetch and asset file supporting hash check, caching and multiple locations. :param name: the asset filename or URL :param asset_hash: asset hash (optional) :param algorithm: hash algorithm (optional, defaults to :data:`avocado.utils.asset.DEFAULT_HASH_ALGORITHM`) :param locations: list of URLs from where the asset can be fetched (optional) :param expire: time for the asset to expire :param find_only: When `True`, `fetch_asset` only looks for the asset in the cache, avoiding the download/move action. Defaults to `False`. :param cancel_on_missing: whether the test should be canceled if the asset was not found in the cache or if `fetch` could not add the asset to the cache. Defaults to `False`. :raises OSError: when it fails to fetch the asset or file is not in the cache and `cancel_on_missing` is `False`. :returns: asset file local path. """ if expire is not None: expire = data_structures.time_to_seconds(str(expire)) # If name has no protocol or network locations, attempt to find # the asset "by name" first. This is valid use case when the # asset has been previously put into any of the cache # directories, either manually or by the caching process # itself. parsed_name = asset.Asset.parse_name(name) if not (parsed_name.scheme or locations): try: return asset.Asset.get_asset_by_name( name, self.cache_dirs, expire, asset_hash ) except OSError as e: if cancel_on_missing: self.cancel(f"Missing asset {name}") raise e asset_obj = asset.Asset( name, asset_hash, algorithm, locations, self.cache_dirs, expire ) try: # return the path to the asset when it was found or fetched if find_only: return asset_obj.find_asset_file() else: return asset_obj.fetch() except OSError as e: # if asset is not in the cache or there was a problem fetching # the asset if cancel_on_missing: # cancel when requested self.cancel(f"Missing asset {name}") # otherwise re-throw OSError raise e def _cleanup(self): if self.__base_logdir_tmp is not None: self.__base_logdir_tmp.cleanup() self.__base_logdir_tmp = None if self.__base_tmpdir is not None: if not self._config.get("run.keep_tmp") and os.path.exists( self.__base_tmpdir ): shutil.rmtree(self.__base_tmpdir) def tearDown(self): self._cleanup() def __del__(self): self._cleanup() avocado-113.0/avocado/core/test_id.py000066400000000000000000000074021513561306000175060ustar00rootroot00000000000000from avocado.utils import astring class TestID: """ Test ID construction and representation according to specification This class wraps the representation of both Avocado's Test ID specification and Avocado's Test Name, which is part of a Test ID. """ def __init__(self, uid, name, variant=None, no_digits=None): """ Constructs a TestID instance :param uid: unique test id (within the job) :param name: test name, as returned by the Avocado test resolver :param variant: the variant applied to this Test ID :type variant: dict :param no_digits: number of digits of the test uid """ self.uid = uid if no_digits is not None and no_digits >= 0: self.str_uid = str(uid).zfill(no_digits if no_digits else 3) else: self.str_uid = str(uid) self.name = name or "" if variant is None or variant["variant_id"] is None: self.variant = None self.str_variant = "" else: self.variant = variant["variant_id"] self.str_variant = f";{self.variant}" def __str__(self): return f"{self.str_uid}-{self.name}{self.str_variant}" def __hash__(self): return id(self) def __repr__(self): return repr(str(self)) def __eq__(self, other): if isinstance(other, str): return str(self) == other else: return self.__dict__ == other.__dict__ @property def str_filesystem(self): """ Test ID in a format suitable for use in file systems The string returned should be safe to be used as a file or directory name. This file system version of the test ID may have to shorten either the Test Name or the Variant ID. The first component of a Test ID, the numeric unique test id, AKA "uid", will be used as a an stable identifier between the Test ID and the file or directory created based on the return value of this method. If the filesystem can not even represent the "uid", than an exception will be raised. For Test ID "001-mytest;foo", examples of shortened file system versions include "001-mytest;f" or "001-myte;foo". :raises: RuntimeError if the test ID cannot be converted to a filesystem representation. """ test_id = str(self) test_id_fs = astring.string_to_safe_path(test_id) if len(test_id) == len(test_id_fs): # everything fits in return test_id_fs idx_fit_variant = len(test_id_fs) - len(self.str_variant) if idx_fit_variant > len(self.str_uid): # full uid+variant return test_id_fs[:idx_fit_variant] + astring.string_to_safe_path( self.str_variant ) elif len(self.str_uid) <= len(test_id_fs): # full uid return astring.string_to_safe_path(self.str_uid + self.str_variant) else: # not even uid could be stored in fs raise RuntimeError( f"Test ID is too long to be stored on the " f'filesystem: "{self.str_uid}"\n' f'Full Test ID: "{str(self)}"' ) @classmethod def from_identifier(cls, identifier): """ It wraps an identifier by the TestID class. :param identifier: Any identifier that is guaranteed to be unique within the context of an avocado Job. :returns: TestID with `uid` as string representation of `identifier` and `name` "test". :rtype: :class:`avocado.core.test_id.TestID` """ if isinstance(identifier, cls): return identifier else: return cls(str(identifier), "test") avocado-113.0/avocado/core/teststatus.py000066400000000000000000000023251513561306000202750ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. """ Valid test statuses and whether they signal success (or failure). """ #: Maps the different status strings in avocado to booleans. STATUSES_MAPPING = { "SKIP": True, "ERROR": False, "FAIL": False, "WARN": True, "PASS": True, "INTERRUPTED": False, "CANCEL": True, } #: Valid test statuses, if a returned status is not listed here, it #: should be handled as error condition. STATUSES = [key for key in STATUSES_MAPPING.keys()] #: List of status that are considered OK (should not cause a job failure) STATUSES_OK = [key for (key, value) in STATUSES_MAPPING.items() if value] #: List of status that are NOT considered OK (should cause a job failure) STATUSES_NOT_OK = [key for (key, value) in STATUSES_MAPPING.items() if not value] avocado-113.0/avocado/core/tree.py000066400000000000000000000424151513561306000170150ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2014 # Copyright: Jaime Huerta-Cepas 2009 # # Authors: Ruda Moura # Lucas Meneghel Rodrigues # Jaime Huerta-Cepas # """ Tree data structure with nodes. This tree structure (Tree drawing code) was inspired in the base tree data structure of the ETE 2 project: http://pythonhosted.org/ete2/ A library for analysis of phylogenetics trees. Explicit permission has been given by the copyright owner of ETE 2 Jaime Huerta-Cepas to take ideas/use snippets from his original base tree code and re-license under GPLv2+, given that GPLv3 and GPLv2 (used in some avocado files) are incompatible. """ import collections import copy import itertools import locale from avocado.utils import astring class FilterSet(set): """Set of filters in standardized form""" @staticmethod def __normalize(item): if not item.endswith("/"): item = item + "/" return item def add(self, item): return super().add(self.__normalize(item)) def update(self, items): return super().update([self.__normalize(item) for item in items]) def __str__(self): fs = ", ".join(sorted([f"'{i}'" for i in self])) return f"FilterSet([{fs}])" class TreeEnvironment(dict): """TreeNode environment with values, origins and filters""" def __init__(self): super().__init__() # values self.origin = {} # origins of the values self.filter_only = FilterSet() # list of filter_only self.filter_out = FilterSet() # list of filter_out def copy(self): cpy = TreeEnvironment() cpy.update(self) cpy.origin = copy.copy(self.origin) cpy.filter_only = copy.copy(self.filter_only) cpy.filter_out = copy.copy(self.filter_out) return cpy def __str__(self): """ String representation using __str__ on items to improve readability """ return self.to_text(False) def to_text(self, sort=False): """ Human readable representation :param sort: Sorted to provide stable output :rtype: str """ if sort: sort_fn = sorted else: def sort_fn(x): return x # Use __str__ instead of __repr__ to improve readability if self: _values = [ # pylint: disable=C0209 "%s: %s" % _ for _ in sort_fn(list(self.items())) ] values = f"{{{', '.join(_values)}}}" _origin = [ f"{key}: {node.path}" for key, node in sort_fn(list(self.origin.items())) ] origin = f"{{{', '.join(_origin)}}}" else: values = "{}" origin = "{}" return ",".join( ( values, origin, astring.to_text(self.filter_only), astring.to_text(self.filter_out), ) ) class TreeNodeEnvOnly: """ Minimal TreeNode-like class providing interface for AvocadoParams """ def __init__(self, path, environment=None): """ :param path: Path of this node (must not end with '/') :param environment: List of pair/key/value items """ self.name = path.rsplit("/")[-1] self.path = path self.environment = TreeEnvironment() if environment: self.__load_environment(environment) def __load_environment(self, environment): nodes = {} for path, key, value in environment: self.environment[key] = value if path not in nodes: nodes[path] = TreeNodeEnvOnly(path) self.environment.origin[key] = nodes[path] def __eq__(self, other): if self.name != other.name: return False if self.path != other.path: return False if self.environment != other.environment: return False return True def fingerprint(self): return f"{self.path}{self.environment.to_text(True)}" def get_environment(self): return self.environment def get_path(self): return self.path class TreeNode: """ Class for bounding nodes into tree-structure. """ def __init__(self, name="", value=None, parent=None, children=None): """ :param name: a name for this node that will be used to define its path according to the name of its parents :type name: str :param value: a collection of keys and values that will be made into this node environment. :type value: dict :param parent: the node that is directly above this one in the tree structure :type parent: :class:`TreeNode` :param children: the nodes that are directly beneath this one in the tree structure :type children: builtin.list """ if value is None: value = {} if children is None: children = [] self.name = name self.value = value self.filters = [], [] # This node's filters, full filters are in env self.parent = parent self.children = [] self._environment = None for child in children: self.add_child(child) def __repr__(self): return f"TreeNode(name={self.name!r})" def __str__(self): variables = [f"{k}={v}" for k, v in self.environment.items()] return f"{self.path}: {', '.join(variables)}" def __len__(self): """Return number of descended leaf nodes""" return len(tuple(self.iter_leaves())) def __iter__(self): """Iterate through descended leaf nodes""" return self.iter_leaves() def __eq__(self, other): """Compares node to other node or string to name of this node""" if isinstance(other, str): # Compare names if self.name == other: return True else: for attr in ("name", "value", "children"): if getattr(self, attr) != getattr(other, attr): return False return True def __ne__(self, other): """Inverted eq""" return not self == other def __hash__(self): values = [] for item in self.value: try: values.append(hash(item)) except TypeError: values.append(hash(astring.to_text(item))) children = [] for item in self.children: try: children.append(hash(item)) except TypeError: children.append(hash(astring.to_text(item))) return hash((self.name,) + tuple(values) + tuple(children)) def fingerprint(self): """ Reports string which represents the value of this node. """ return f"{self.path}{self.environment.to_text(True)}" def add_child(self, node): """ Append node as child. Nodes with the same name gets merged into the existing position. """ if isinstance(node, TreeNode): if node.name in self.children: self.children[self.children.index(node.name)].merge(node) else: node.parent = self self.children.append(node) else: raise ValueError("Bad node type.") def merge(self, other): """ Merges `other` node into this one without checking the name of the other node. New values are appended, existing values overwritten and unaffected ones are kept. Then all other node children are added as children (recursively they get either appended at the end or merged into existing node in the previous position. """ self.value.update(other.value) self.filters[0].extend(other.filters[0]) self.filters[1].extend(other.filters[1]) for child in other.children: self.add_child(child) @property def is_leaf(self): """Is this a leaf node?""" return not self.children @property def root(self): """Root of this tree""" return self.get_root() def get_root(self): """Get root of this tree""" root = self for root in self.iter_parents(): pass return root def iter_parents(self): """Iterate through parent nodes to root""" node = self.parent while True: if node is None: return yield node node = node.parent @property def parents(self): """List of parent nodes""" return self.get_parents() def get_parents(self): """Get list of parent nodes""" return list(self.iter_parents()) @property def path(self): """Node path""" return self.get_path() def get_path(self, sep="/"): """Get node path""" if not self.parent: return sep + astring.to_text(self.name) path = [astring.to_text(self.name)] for node in self.iter_parents(): path.append(astring.to_text(node.name)) return sep.join(reversed(path)) @property def environment(self): """Node environment (values + preceding envs)""" return self.get_environment() def get_environment(self): """Get node environment (values + preceding envs)""" if self._environment is None: self._environment = ( self.parent.environment.copy() if self.parent else TreeEnvironment() ) for key, value in self.value.items(): if isinstance(value, list): if key in self._environment and isinstance( self._environment[key], list ): self._environment[key] = self._environment[key] + value else: self._environment[key] = value else: self._environment[key] = value self._environment.origin[key] = self self._environment.filter_only.update(self.filters[0]) self._environment.filter_out.update(self.filters[1]) return self._environment def set_environment_dirty(self): """ Set the environment cache dirty. You should call this always when you query for the environment and then change the value or structure. Otherwise you'll get the old environment instead. """ for child in self.children: child.set_environment_dirty() self._environment = None def get_node(self, path, create=False): """ :param path: Path of the desired node (relative to this node) :param create: Create the node (and intermediary ones) when not present :return: the node associated with this path :raise ValueError: When path doesn't exist and create not set """ node = self for name in path.split("/"): if not name: continue try: node = node.children[node.children.index(name)] except ValueError: if create: child = node.__class__(name) node.add_child(child) node = child else: raise ValueError( f"Path {path} does not exists in this " f"tree\n{tree_view(self.root)}" ) return node def iter_children_preorder(self): """Iterate through children""" queue = collections.deque() node = self while node is not None: yield node queue.extendleft(reversed(node.children)) try: node = queue.popleft() except IndexError: node = None def iter_leaves(self): """Iterate through leaf nodes""" for node in self.iter_children_preorder(): if node.is_leaf: yield node def get_leaves(self): """Get list of leaf nodes""" return list(self.iter_leaves()) def detach(self): """Detach this node from parent""" if self.parent: self.parent.children.remove(self) self.parent = None return self def tree_view(root, verbose=None, use_utf8=None): """ Generate tree-view of the given node :param root: root node :param verbose: verbosity (0, 1, 2, 3) :param use_utf8: Use utf-8 encoding (None=autodetect) :return: string representing this node's tree structure """ def prefixed_write(prefix1, prefix2, value): """ Split value's lines and prepend empty prefix to 2nd+ lines :return: list of lines """ value = astring.to_text(value) if "\n" not in value: return [prefix1 + prefix2 + value] value = value.splitlines() empty_prefix2 = " " * len(prefix2) return [prefix1 + prefix2 + value[0]] + [ prefix1 + empty_prefix2 + _ for _ in value[1:] ] def process_node(node): """ Generate this node's tree-view :return: list of lines """ if getattr(node, "multiplex", None): down = charset["DoubleDown"] down_right = charset["DoubleDownRight"] right = charset["DoubleRight"] else: down = charset["Down"] down_right = charset["DownRight"] right = charset["Right"] out = [node.name] if verbose is not None and verbose >= 2 and node.is_leaf: values = itertools.chain( iter(node.environment.items()), [("filter-only", _) for _ in node.environment.filter_only], [("filter-out", _) for _ in node.environment.filter_out], ) elif verbose in (1, 3): values = itertools.chain( iter(node.value.items()), [("filter-only", _) for _ in node.filters[0]], [("filter-out", _) for _ in node.filters[1]], ) else: values = None if values: val = charset["Value"] if node.children: val_prefix = down else: val_prefix = " " for key, value in values: out.extend(prefixed_write(val_prefix, f"{val}{key}: ", value)) if node.children: for child in node.children[:-1]: lines = process_node(child) out.append(down_right + lines[0]) out.extend(down + line for line in lines[1:]) lines = process_node(node.children[-1]) out.append(right + lines[0]) empty_down_right = " " * len(down_right) out.extend(empty_down_right + line for line in lines[1:]) return out if use_utf8 is None: use_utf8 = locale.getpreferredencoding() == "UTF-8" if use_utf8: charset = { "DoubleDown": " \u2551 ", "DoubleDownRight": " \u2560\u2550\u2550 ", "DoubleRight": " \u255a\u2550\u2550 ", "Down": " \u2503 ", "DownRight": " \u2523\u2501\u2501 ", "Right": " \u2517\u2501\u2501 ", "Value": "\u2192 ", } else: # ASCII fallback charset = { "Down": " | ", "DownRight": " |-- ", "Right": " \\-- ", "DoubleDown": " # ", "DoubleDownRight": " #== ", "DoubleRight": " #== ", "Value": " -> ", } if getattr(root, "multiplex", None): down = charset["DoubleDown"] down_right = charset["DoubleDownRight"] right = charset["DoubleRight"] else: down = charset["Down"] down_right = charset["DownRight"] right = charset["Right"] out = [] if verbose is not None and verbose >= 2 and root.is_leaf: values = root.environment.items() elif verbose in (1, 3): values = root.value.items() else: values = None if values: prefix = charset["Value"].lstrip() for key, value in values: out.extend(prefixed_write(prefix, key + ": ", value)) if root.children: for child in root.children[:-1]: lines = process_node(child) out.append(down_right + lines[0]) out.extend(down + line for line in lines[1:]) lines = process_node(root.children[-1]) out.append(right + lines[0]) out.extend(" " * len(down_right) + line for line in lines[1:]) # When not on TTY we need to force the encoding return "\n".join(out).encode( "utf-8" if use_utf8 else "ascii", errors="xmlcharrefreplace" ) avocado-113.0/avocado/core/utils/000077500000000000000000000000001513561306000166365ustar00rootroot00000000000000avocado-113.0/avocado/core/utils/__init__.py000066400000000000000000000000001513561306000207350ustar00rootroot00000000000000avocado-113.0/avocado/core/utils/eggenv.py000066400000000000000000000020011513561306000204540ustar00rootroot00000000000000import os import pkg_resources def get_python_path_env_if_egg(): """ Returns an environment mapping with an extra PYTHONPATH for the egg or None When running Avocado Python modules, the interpreter on the new process needs to know where Avocado can be found. This is usually handled by the Avocado installation being available to the standard Python installation, but if Avocado is running from an uninstalled egg, it needs extra help. :returns: environment mapping with an extra PYTHONPATH for the egg or None :rtype: os.environ mapping or None """ dist = pkg_resources.get_distribution("avocado-framework") if not (dist.location.endswith(".egg") and os.path.isfile(dist.location)): return None python_path = os.environ.get("PYTHONPATH", "") python_path_entries = python_path.split(":") if dist.location in python_path_entries: return None env = os.environ.copy() env["PYTHONPATH"] = f"{dist.location}:{python_path}" return env avocado-113.0/avocado/core/utils/loader.py000066400000000000000000000031341513561306000204570ustar00rootroot00000000000000import importlib import inspect import os import sys from avocado.core import test def load_test(test_factory): """ Load test from the test factory. :param test_factory: a pair of test class and parameters. :type test_factory: tuple :return: an instance of :class:`avocado.core.test.Test`. """ test_class, test_parameters = test_factory if "run.results_dir" in test_parameters: test_parameters["base_logdir"] = test_parameters.pop("run.results_dir") if "modulePath" not in test_parameters: raise RuntimeError( 'Test factory parameters is missing the module\'s path ("modulePath")' ) test_path = test_parameters.pop("modulePath") module_name = os.path.basename(test_path).split(".")[0] test_module_dir = os.path.abspath(os.path.dirname(test_path)) spec = importlib.util.spec_from_file_location(module_name, test_path) test_module = importlib.util.module_from_spec(spec) sys.modules[module_name] = test_module try: # Tests with local dir imports need this sys.path.insert(0, test_module_dir) spec.loader.exec_module(test_module) finally: if test_module_dir in sys.path: sys.path.remove(test_module_dir) for _, obj in inspect.getmembers(test_module): if ( inspect.isclass(obj) and obj.__name__ == test_class and inspect.getmodule(obj) == test_module and issubclass(obj, test.Test) ): return obj(**test_parameters) raise ImportError(f'Failed to find/load class "{test_class}" in "{test_path}"') avocado-113.0/avocado/core/utils/messages.py000066400000000000000000000220671513561306000210260ustar00rootroot00000000000000import gc import logging import sys import time from avocado.core.output import split_loggers_and_levels class GenericMessage: message_status = None @classmethod def _prepare_message(cls, additional_info=None): """Prepare a message dict with some basic information. This will add the keyword 'status' and 'time' to all messages. :param: addional_info: Any additional information that you would like to add to the message. :type additional_info: dict :return: message dict which can be send to avocado server :rtype: dict """ status = {} if additional_info is not None: status = additional_info status.update({"status": cls.message_status, "time": time.monotonic()}) return status @classmethod def get(cls, **kwargs): """Creates message base on it's type with all necessary information. :return: message dict which can be send to avocado server :rtype: dict """ kwargs = {key: value for (key, value) in kwargs.items() if value is not None} return cls._prepare_message(additional_info=kwargs) class StartedMessage(GenericMessage): message_status = "started" class RunningMessage(GenericMessage): """Creates running message without any additional info.""" message_status = "running" class FinishedMessage(GenericMessage): message_status = "finished" @classmethod def get( cls, result, fail_reason=None, returncode=None, class_name=None, fail_class=None, traceback=None, **kwargs, ): # pylint: disable=W0221 """Creates finished message with all necessary information. :param result: test result :type result: values for the statuses defined in :class: avocado.core.teststatus.STATUSES :param fail_reason: parameter for brief specification, of the failed result. :type fail_reason: str :param returncode: exit status of runner :param class_name: class name of the test :type class_name: str :param fail_class: Exception class of the failure :type fail_class: str :param traceback: Traceback of the exception :type traceback: str :return: finished message :rtype: dict """ kwargs["result"] = result kwargs["fail_reason"] = fail_reason kwargs["returncode"] = returncode kwargs["class_name"] = class_name kwargs["fail_class"] = fail_class kwargs["traceback"] = traceback return super().get(**kwargs) class GenericRunningMessage(GenericMessage): message_status = "running" message_type = None @classmethod def _get_running_message(cls, msg): """Prepare a message dict with necessary information for specific type. :param msg: message data. If the message is str, it will be encoded with utf-8. :type msg: str, bytes :return: message dict which can be send to avocado server :rtype: dict """ message = {"type": cls.message_type, "log": msg} if not isinstance(msg, bytes): msg = msg.encode("utf-8") message.update({"log": msg, "encoding": "utf-8"}) return message @classmethod def get(cls, msg, **kwargs): # pylint: disable=W0221 """Creates running message with all necessary information. :param msg: log of running message :type msg: str :return: running message :rtype: dict """ kwargs.update(cls._get_running_message(msg)) return super().get(**kwargs) class LogMessage(GenericRunningMessage): message_type = "log" class StdoutMessage(GenericRunningMessage): """Creates stdout message with all necessary information.""" message_type = "stdout" class StderrMessage(GenericRunningMessage): """Creates stderr message with all necessary information.""" message_type = "stderr" class WhiteboardMessage(GenericRunningMessage): """Creates whiteboard message with all necessary information.""" message_type = "whiteboard" class OutputMessage(GenericRunningMessage): """Creates output message with all necessary information.""" message_type = "output" class FileMessage(GenericRunningMessage): """Creates file message with all necessary information.""" message_type = "file" @classmethod def get(cls, msg, path): # pylint: disable=W0221 return super().get(msg=msg, path=path) _supported_types = { LogMessage.message_type: LogMessage, StdoutMessage.message_type: StdoutMessage, StderrMessage.message_type: StderrMessage, WhiteboardMessage.message_type: WhiteboardMessage, OutputMessage.message_type: OutputMessage, FileMessage.message_type: FileMessage, } class RunnerLogHandler(logging.Handler): def __init__(self, queue, message_type, kwargs=None): """ Runner logger which will put every log to the runner queue :param queue: queue for the runner messages :type queue: multiprocessing.SimpleQueue :param message_type: type of the log :type message_type: string """ super().__init__() self.queue = queue self.message = _supported_types[message_type] self.kwargs = kwargs or {} self._message_only_formatter = None @property def message_only_formatter(self): """ Property that initiates a :class:`logging.Formatter` lazily """ if self._message_only_formatter is None: self._message_only_formatter = logging.Formatter() return self._message_only_formatter def emit(self, record): msg = self.format(record) if self.message is LogMessage: kwargs = { "log_name": record.name, "log_level": record.levelno, "log_levelname": record.levelname, } kwargs.update(**self.kwargs) else: kwargs = self.kwargs gc.disable() self.queue.put(self.message.get(msg, **kwargs)) gc.enable() class StreamToQueue: def __init__(self, queue, message_type): """ Runner Stream which will transfer data to the runner queue :param queue: queue for the runner messages :type queue: multiprocessing.SimpleQueue :param message_type: type of the log :type message_type: string """ self.queue = queue self.message = _supported_types[message_type] def write(self, buf): gc.disable() self.queue.put(self.message.get(buf)) gc.enable() def flush(self): pass def start_logging(config, queue): """Helper method for connecting the avocado logging with avocado messages. It will add the logHandlers to the :class: avocado.core.output loggers, which will convert the logs to the avocado messages and sent them to processing queue. :param config: avocado configuration :type config: dict :param queue: queue for the runner messages :type queue: multiprocessing.SimpleQueue """ log_level = config.get("job.output.loglevel", logging.DEBUG) log_handler = RunnerLogHandler(queue, "log") stdout_handler = RunnerLogHandler(queue, "stdout") stderr_handler = RunnerLogHandler(queue, "stderr") fmt = "%(asctime)s %(name)s %(module)-16.16s L%(lineno)-.4d %(levelname)-5.5s| %(message)s" formatter = logging.Formatter(fmt=fmt) log_handler.setFormatter(formatter) stdout_handler.setFormatter(formatter) stderr_handler.setFormatter(formatter) # root log logger = logging.getLogger("") logger.addHandler(log_handler) logger.setLevel(log_level) # main log = 'avocado' logging.getLogger("avocado").setLevel(log_level) # 'avocado.test' logging.getLogger("avocado.test").setLevel(log_level) sys.stdout = StreamToQueue(queue, "stdout") sys.stderr = StreamToQueue(queue, "stderr") stdout_logger = logging.getLogger("avocado.test.stdout") stdout_logger.addHandler(stdout_handler) stdout_logger.propagate = False stderr_logger = logging.getLogger("avocado.test.stderr") stderr_logger.addHandler(stderr_handler) stderr_logger.propagate = False # store custom test loggers enabled_loggers = config.get("job.run.store_logging_stream", []) for enabled_logger, level in split_loggers_and_levels(enabled_loggers): log_path = f"{enabled_logger}.{logging.getLevelName(level)}.log" if not level: level = log_level log_path = f"{enabled_logger}.log" store_stream_handler = RunnerLogHandler(queue, "file", {"path": log_path}) store_stream_handler.setFormatter(formatter) output_logger = logging.getLogger(enabled_logger) output_logger.addHandler(store_stream_handler) output_logger.setLevel(level) if not enabled_logger.startswith("avocado."): output_logger.addHandler(log_handler) output_logger.propagate = False avocado-113.0/avocado/core/utils/path.py000066400000000000000000000034011513561306000201420ustar00rootroot00000000000000import os import platform from pkg_resources import get_distribution def prepend_base_path(value): expanded = os.path.expanduser(value) if not expanded.startswith(("/", "~", ".")): dist = get_distribution("avocado-framework") return os.path.join(dist.location, "avocado", expanded) return expanded def system_wide_or_base_path(file_path): """Returns either a system wide path, or one relative to the base. This utility function is helpful when locating configuration files and takes into account system specific conditions, such as whether configuration will normally be under "/etc" or "/usr/local/etc". If "etc/avocado/avocado.conf" is given as input, it checks for the existence of "/etc/avocado/avocado.conf". If that path does not exist, then a path starting with the avocado's Python's distribution is returned. In that case it'd return something like "/usr/lib/python3.9/site-packages/avocado/etc/avocado/avocado.conf". :param file_path: a filesystem path that can either be absolute, or relative. If relative, the absolute equivalent (that is, by prefixing the filesystem root location) is checked for existence. If it does not exist, a path relative to the Python's distribution base path is returned. :type file_path: str :rtype: str """ if os.path.isabs(file_path): abs_path = file_path else: if platform.system() == "FreeBSD": prefix = "/usr/local" else: prefix = os.path.sep abs_path = os.path.join(prefix, file_path) if os.path.exists(abs_path): return abs_path return prepend_base_path(file_path) avocado-113.0/avocado/core/utils/version.py000066400000000000000000000014541513561306000207010ustar00rootroot00000000000000import os from avocado.utils import path, process def get_avocado_git_version(): # if running from git sources, there will be a ".git" directory # 4 levels up dn = os.path.dirname base_dir = dn(dn(dn(dn(__file__)))) git_dir = os.path.join(base_dir, ".git") if not os.path.isdir(git_dir): return if not os.path.exists(os.path.join(base_dir, "python-avocado.spec")): return try: git = path.find_command("git") except path.CmdNotFoundError: return git_dir = os.path.abspath(base_dir) cmd = f"{git} -C {git_dir} show --summary --pretty='%H'" res = process.run(cmd, ignore_status=True, verbose=False) if res.exit_status == 0: top_commit = res.stdout_text.splitlines()[0][:8] return f" (GIT commit {top_commit})" avocado-113.0/avocado/core/varianter.py000066400000000000000000000305141513561306000200460ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2014 # # Authors: Ruda Moura # Ademar Reis # Lucas Meneghel Rodrigues """ Base classes for implementing the varianter interface """ import hashlib import json import os from avocado.core import dispatcher, output, tree from avocado.utils import astring VARIANTS_FILENAME = "variants.json" def is_empty_variant(variant): """ Reports whether the variant contains any data :param variant: Avocado test variant (list of TreeNode-like objects) :return: True when the variant does not contain (any useful) data """ return not variant or variant == [tree.TreeNode()] * len(variant) def generate_variant_id(variant): """ Basic function to generate variant-id from a variant :param variant: Avocado test variant (list of TreeNode-like objects) :return: String compounded of ordered node names and a hash of all values. """ def get_variant_name(variant): """ To get the variant full name string :param variant: Avocado test variant (list of TreeNode-like objects) :return: Complete variant name string """ full_name = [] for node in variant: var_str = [] while node: var_str.append(node.name) node = node.parent if hasattr(node, "parent") else None try: # Let's drop repeated node names and empty string full_name.extend([x for x in var_str[::-1][1:] if x not in full_name]) except IndexError: pass return "-".join(full_name) variant = sorted(variant, key=lambda x: x.path) fingerprint = "\n".join(_.fingerprint() for _ in variant) return ( get_variant_name(variant) + "-" + hashlib.sha1(fingerprint.encode(astring.ENCODING)).hexdigest()[:4] ) def variant_to_str(variant, verbosity, out_args=None, debug=False): """ Reports human readable representation of a variant :param variant: Valid variant (list of TreeNode-like objects) :param verbosity: Output verbosity where 0 means brief :param out_args: Extra output arguments (currently unused) :param debug: Whether the variant contains and should report debug info :return: Human readable representation """ del out_args out = [] if not debug: paths = ", ".join([x.path for x in variant["variant"]]) else: color = output.TERM_SUPPORT.LOWLIGHT cend = output.TERM_SUPPORT.ENDC paths = ", ".join( [ f"{_.name}{color}@{getattr(_, 'yaml', 'Unknown')}{cend}" for _ in variant["variant"] ] ) out.append( "%sVariant %s: %s" % ("\n" if verbosity else "", variant["variant_id"], paths) ) if verbosity: env = set() for node in variant["variant"]: for key, value in node.environment.items(): origin = node.environment.origin[key].path env.add((f"{origin}:{key}", astring.to_text(value))) if not env: return out fmt = " %%-%ds => %%s" % max(len(_[0]) for _ in env) # pylint: disable=C0209 for record in sorted(env): out.append(fmt % record) return out def dump_variant(variant): """Dump a variant into a json-serializable representation :param variant: Valid variant (list of TreeNode-like objects) :return: json-serializable representation """ def dump_tree_node(node): """ Turns TreeNode-like object into tuple(path, env_representation) """ return ( astring.to_text(node.path), [ ( astring.to_text(node.environment.origin[key].path), astring.to_text(key), value, ) for key, value in node.environment.items() ], ) safe_variant = {} safe_variant["paths"] = [astring.to_text(pth) for pth in variant.get("paths")] safe_variant["variant_id"] = variant.get("variant_id") safe_variant["variant"] = [dump_tree_node(_) for _ in variant.get("variant", [])] return safe_variant def dump_ivariants(ivariants): """ Walks the iterable variants and dumps them into json-serializable object """ variants = [] for variant in ivariants(): variants.append(dump_variant(variant)) return variants class FakeVariantDispatcher: """ This object can act instead of VarianterDispatcher to report loaded variants. """ def __init__(self, state): for variant in state: variant["variant"] = [ tree.TreeNodeEnvOnly(path, env) for path, env in variant["variant"] ] self.variants = state def map_method_with_return(self, method, *args, **kwargs): """ Reports list containing one result of map_method on self """ if hasattr(self, method): return [getattr(self, method)(*args, **kwargs)] else: return [] def to_str(self, summary=0, variants=0, **kwargs): # pylint: disable=W0613 if not self.variants: return "" out = [] for variant in self.variants: paths = ", ".join([x.path for x in variant["variant"]]) out.append(f"\nVariant {variant['variant_id']}: {paths}") env = set() for node in variant["variant"]: for key, value in node.environment.items(): origin = node.environment.origin[key].path env.add((f"{origin}:{key}", astring.to_text(value))) if not env: continue fmt = " %%-%ds => %%s" % max( # pylint: disable=C0209 len(_[0]) for _ in env ) for record in sorted(env): out.append(fmt % record) return "\n".join(out) def __iter__(self): return iter(self.variants) def __len__(self): return sum(1 for _ in self) class Varianter: """ This object takes care of producing test variants """ def __init__(self, debug=False, state=None): """ :param debug: Store whether this instance should debug varianter :param state: Force-varianter state :note: it's necessary to check whether variants debug is enable in order to provide the right results. """ if state is None: self.debug = debug self.node_class = tree.TreeNodeDebug if debug else tree.TreeNode self._variant_plugins = dispatcher.VarianterDispatcher() self._no_variants = None else: self.load(state) def parse(self, config): """ Apply options defined on the cmdline and initialize the plugins. :param config: Configuration received from configuration files, command line parser, etc. :type config: dict """ self._variant_plugins.map_method_with_return("initialize", config) self._no_variants = sum(self._variant_plugins.map_method_with_return("__len__")) def is_parsed(self): """ Reports whether the varianter was already parsed """ return self._no_variants is not None def to_str(self, summary=0, variants=0, **kwargs): """ Return human readable representation The summary/variants accepts verbosity where 0 means do not display at all and maximum is up to the plugin. :param summary: How verbose summary to output (int) :param variants: How verbose list of variants to output (int) :param kwargs: Other free-form arguments :rtype: str """ if self._no_variants == 0: # No variants return "" out = [ item for item in self._variant_plugins.map_method_with_return( "to_str", summary, variants, **kwargs ) if item ] return "\n\n".join(out) def get_number_of_tests(self, test_suite): """ :return: overall number of tests * number of variants """ # Currently number of tests is symmetrical if self._no_variants: return len(test_suite) * self._no_variants else: return len(test_suite) def dump(self): """ Dump the variants in loadable-state This is lossy representation which takes all yielded variants and replaces the list of nodes with TreeNodeEnvOnly representations:: [{'path': path, 'variant_id': variant_id, 'variant': dump_tree_nodes(original_variant)}, {'path': [str, str, ...], 'variant_id': str, 'variant': [(str, [(str, str, object), ...])], {'path': ['/run/*'], 'variant_id': 'cat-26c0' 'variant': [('/pig/cat', [('/pig', 'ant', 'fox'), ('/pig/cat', 'dog', 'bee')])]} ...] where `dump_tree_nodes` looks like:: [(node.path, environment_representation), (node.path, [(path1, key1, value1), (path2, key2, value2), ...]), ('/pig/cat', [('/pig', 'ant', 'fox')]) :return: loadable Varianter representation """ if not self.is_parsed(): raise NotImplementedError( "Dumping Varianter state before multiplexation is not supported." ) return dump_ivariants(self.itertests) def load(self, state): """ Load the variants state Current implementation supports loading from a list of loadable variants. It replaces the VariantDispatcher with fake implementation which reports the loaded (and initialized) variants. :param state: loadable Varianter representation """ self.debug = False self.node_class = tree.TreeNode self._variant_plugins = FakeVariantDispatcher(state) self._no_variants = sum(self._variant_plugins.map_method_with_return("__len__")) def itertests(self): """ Yields all variants of all plugins The variant is defined as dictionary with at least: * variant_id - name of the current variant * variant - AvocadoParams-compatible variant (usually a list of TreeNodes but dict or simply None are also possible values) * paths - default path(s) :yield variant """ if self._no_variants: # Copy template and modify it's params plugins_variants = self._variant_plugins.map_method_with_return("__iter__") iter_variants = ( variant for plugin_variants in plugins_variants for variant in plugin_variants ) for variant in iter(iter_variants): yield variant else: # No real variants, but currently *something* needs to be returned yield { "variant": self.node_class("").get_leaves(), "variant_id": None, "paths": ["/run/*"], } @classmethod def from_resultsdir(cls, resultsdir): """ Retrieves the job variants objects from the results directory. This will return a list of variants since a Job can have multiple suites and the variants is per suite. """ path = os.path.join(resultsdir, "jobdata", VARIANTS_FILENAME) if not os.path.exists(path): return None variants = [] with open(path, "r", encoding="utf-8") as variants_file: for variant in json.load(variants_file): variants.append(cls(state=variant)) return variants def __len__(self): return self._no_variants avocado-113.0/avocado/core/version.py000066400000000000000000000014551513561306000175420ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2013-2014 # Author: Lucas Meneghel Rodrigues __all__ = ["MAJOR", "MINOR", "VERSION"] import pkg_resources try: VERSION = pkg_resources.get_distribution("avocado-framework").version except pkg_resources.DistributionNotFound: VERSION = "unknown.unknown" MAJOR, MINOR = VERSION.split(".") avocado-113.0/avocado/etc/000077500000000000000000000000001513561306000153215ustar00rootroot00000000000000avocado-113.0/avocado/etc/avocado/000077500000000000000000000000001513561306000167355ustar00rootroot00000000000000avocado-113.0/avocado/etc/avocado/scripts/000077500000000000000000000000001513561306000204245ustar00rootroot00000000000000avocado-113.0/avocado/etc/avocado/scripts/job/000077500000000000000000000000001513561306000211765ustar00rootroot00000000000000avocado-113.0/avocado/etc/avocado/scripts/job/post.d/000077500000000000000000000000001513561306000224055ustar00rootroot00000000000000avocado-113.0/avocado/etc/avocado/scripts/job/post.d/README000066400000000000000000000003641513561306000232700ustar00rootroot00000000000000Put your post-job scripts here. They need to be readable and executable by the Avocado user running the jobs. The order of execution is based on their file names. If order is important, use a prefix, such as 001-myscript, 002-otherscript, etc. avocado-113.0/avocado/etc/avocado/scripts/job/pre.d/000077500000000000000000000000001513561306000222065ustar00rootroot00000000000000avocado-113.0/avocado/etc/avocado/scripts/job/pre.d/README000066400000000000000000000003631513561306000230700ustar00rootroot00000000000000Put your pre-job scripts here. They need to be readable and executable by the Avocado user running the jobs. The order of execution is based on their file names. If order is important, use a prefix, such as 001-myscript, 002-otherscript, etc. avocado-113.0/avocado/etc/avocado/sysinfo/000077500000000000000000000000001513561306000204275ustar00rootroot00000000000000avocado-113.0/avocado/etc/avocado/sysinfo/commands000066400000000000000000000002351513561306000221530ustar00rootroot00000000000000df -mP dmesg uname -a lspci -vvnn gcc --version ld --version hostname uptime dmidecode ifconfig -a brctl show ip link numactl --hardware show lscpu fdisk -l avocado-113.0/avocado/etc/avocado/sysinfo/files000066400000000000000000000004461513561306000214600ustar00rootroot00000000000000/proc/cmdline /proc/mounts /proc/pci /proc/meminfo /proc/slabinfo /proc/version /proc/cpuinfo /proc/modules /proc/interrupts /proc/partitions /sys/kernel/debug/sched_features /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor /sys/devices/system/clocksource/clocksource0/current_clocksource avocado-113.0/avocado/etc/avocado/sysinfo/profilers000066400000000000000000000000271513561306000223560ustar00rootroot00000000000000vmstat 1 journalctl -f avocado-113.0/avocado/libexec/000077500000000000000000000000001513561306000161615ustar00rootroot00000000000000avocado-113.0/avocado/libexec/avocado-bash-utils000077500000000000000000000011721513561306000215750ustar00rootroot00000000000000#!/bin/sh # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; specifically version 2 of the License. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2014 # Write to avocado_log # First argument is the log level avocado_log() { LEVEL=`printf '%-5s' $1` shift echo "`date '+%H:%M:%S'` $LEVEL| $*" } avocado-113.0/avocado/libexec/avocado_debug000077500000000000000000000000651513561306000206720ustar00rootroot00000000000000#!/bin/sh . avocado-bash-utils avocado_log DEBUG $* avocado-113.0/avocado/libexec/avocado_error000077500000000000000000000000651513561306000207350ustar00rootroot00000000000000#!/bin/sh . avocado-bash-utils avocado_log ERROR $* avocado-113.0/avocado/libexec/avocado_info000077500000000000000000000000641513561306000205360ustar00rootroot00000000000000#!/bin/sh . avocado-bash-utils avocado_log INFO $* avocado-113.0/avocado/libexec/avocado_warn000077500000000000000000000000641513561306000205520ustar00rootroot00000000000000#!/bin/sh . avocado-bash-utils avocado_log WARN $* avocado-113.0/avocado/plugins/000077500000000000000000000000001513561306000162275ustar00rootroot00000000000000avocado-113.0/avocado/plugins/__init__.py000066400000000000000000000010411513561306000203340ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2015 # Author: Cleber Rosa avocado-113.0/avocado/plugins/archive.py000066400000000000000000000031461513561306000202260ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2016 # Authors: Cleber Rosa """Result Archive Plugin""" from avocado.core.plugin_interfaces import CLI, Result from avocado.core.settings import settings from avocado.utils import archive class Archive(Result): name = "zip_archive" description = "Result archive (ZIP) support" def render(self, result, job): if job.config.get("run.results.archive"): archive.compress(f"{job.logdir}.zip", job.logdir) class ArchiveCLI(CLI): name = "zip_archive" description = "Result archive (ZIP) support to run command" def configure(self, parser): run_subcommand_parser = parser.subcommands.choices.get("run", None) if run_subcommand_parser is None: return help_msg = "Archive (ZIP) files generated by tests" settings.register_option( section="run.results", key="archive", default=False, help_msg=help_msg, key_type=bool, parser=run_subcommand_parser, short_arg="-z", long_arg="--archive", ) def run(self, config): pass avocado-113.0/avocado/plugins/assets.py000066400000000000000000000545211513561306000201120ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2019 # Authors: Willian Rampazzo """ Assets subcommand """ import ast import os from datetime import datetime from avocado.core import exit_codes, safeloader from avocado.core.nrunner.runnable import Runnable from avocado.core.output import LOG_UI from avocado.core.plugin_interfaces import CLICmd, JobPreTests from avocado.core.settings import settings from avocado.utils import data_structures from avocado.utils.asset import SUPPORTED_OPERATORS, Asset from avocado.utils.astring import iter_tabular_output from avocado.utils.data_structures import DataSize, InvalidDataSize from avocado.utils.output import display_data_size class FetchAssetHandler(ast.NodeVisitor): # pylint: disable=R0902 """ Handles the parsing of instrumented tests for `fetch_asset` statements. """ PATTERN = "fetch_asset" def __init__(self, file_name, test_file_parse_cache, klass=None, method=None): self.file_name = file_name # fetch assets from specific test using klass and method self.klass = klass # we need to make sure we cover the setUp method when fetching # assets for a specific test self.methods = [method, "setUp"] self.asgmts = {} self.calls = [] # hold current class and current method to make sure we have the # correct context for the assignment statement. # at this time, module constants and class attributes are discarded. self.current_klass = None self.current_method = None # check if we have valid instrumented tests # discards disabled tests if file_name not in test_file_parse_cache: test_file_parse_cache[file_name] = safeloader.find_avocado_tests( self.file_name )[0] self.tests = test_file_parse_cache[file_name] # create Abstract Syntax Tree from test source file with open(self.file_name, encoding="utf-8") as source_file: self.tree = ast.parse(source_file.read(), self.file_name) # build list of keyword arguments from calls that match pattern self.visit(self.tree) def _parse_args(self, node): """Parse the AST fetch_asset node and build the arguments dictionary. :param node: AST node to be evaluated :type node: ast.Attribute :returns: keywords and arguments from a fetch_asset call. :rtype: dict """ args = [] # variables to make lines shorter cur_klass = self.current_klass cur_method = self.current_method # parse args from call for arg in node.args: # handle string args if isinstance(arg, ast.Constant) and isinstance(arg.value, str): args.append(arg.value) # handle variable args elif isinstance(arg, ast.Name): # look for assignments at method if arg.id in self.asgmts[cur_klass][cur_method]: args.append(self.asgmts[cur_klass][cur_method][arg.id]) # right now we support just one level of variable as argument, # with a pure string assignment, in the same context, # just like `name = 'file.zip'` else: return None # starts building the keywords dictionary for the asset.Asset() # class constructor. keywords = ["name", "asset_hash", "algorithm", "locations", "expire"] fetch_args = dict(zip(keywords, args)) # parse keyword args for call for kwarg in node.keywords: # variable to make lines shorter kword = kwarg.arg # handle `keyword = string` if isinstance(kwarg.value, ast.Constant) and isinstance( kwarg.value.value, str ): fetch_args[kword] = kwarg.value.value # handle `keyword = variable` elif isinstance(kwarg.value, ast.Name): name = kwarg.value.id # look for assignments at method if name in self.asgmts[cur_klass][cur_method]: fetch_args[kword] = self.asgmts[cur_klass][cur_method][name] # right now we support just one level of variable as argument, # with a pure string assignment, in the same context, # just like `name = 'file.zip'` else: return None # Fill empty keywords with None for kword in keywords: if kword not in fetch_args: fetch_args[kword] = None return fetch_args def visit_ClassDef(self, node): # pylint: disable=C0103 """Visit ClassDef on AST and save current Class. :param node: AST node to be evaluated :type node: ast.* """ if node.name in self.tests: if self.klass and node.name != self.klass: return # reset the current method pointer self.current_method = None self.current_klass = node.name self.asgmts[self.current_klass] = {} self.generic_visit(node) def visit_FunctionDef(self, node): # pylint: disable=C0103 """Visit FunctionDef on AST and save current method. :param node: AST node to be evaluated :type node: ast.* """ # make sure we are into a class method and not a function if self.current_klass: if self.methods[0] and node.name not in self.methods: return self.current_method = node.name self.asgmts[self.current_klass][self.current_method] = {} self.generic_visit(node) @staticmethod def _ast_list_to_list(node): result = [] for item in node.value.elts: if hasattr(item, "value"): result.append(item.value) return result def visit_Assign(self, node): # pylint: disable=C0103 """Visit Assign on AST and build assignments. This method will visit and build list of assignments that matches the pattern pattern `name = string`. :param node: AST node to be evaluated :type node: ast.* """ if isinstance(node.value, ast.List) or ( isinstance(node.value, ast.Constant) and isinstance(node.value.value, str) ): # make sure we are into a class method, we are not supporting # attributes and module constant assignments at this time if self.current_klass and self.current_method: # variables to make dictionary assignment line shorter cur_klass = self.current_klass cur_method = self.current_method # if it is a class attribute, save the attribute name # otherwise, save the local variable name if isinstance(node.targets[0], ast.Attribute): name = node.targets[0].attr elif hasattr(node.targets[0], "id"): name = node.targets[0].id else: self.generic_visit(node) return if isinstance(node.value, ast.Constant) and isinstance( node.value.value, str ): self.asgmts[cur_klass][cur_method][name] = node.value.value elif isinstance(node.value, ast.List): self.asgmts[cur_klass][cur_method][name] = self._ast_list_to_list( node ) self.generic_visit(node) def visit_Call(self, node): # pylint: disable=C0103 """Visit Calls on AST and build list of calls that matches the pattern. :param node: AST node to be evaluated :type node: ast.* """ # make sure we are into a class method if self.current_klass and self.current_method: if isinstance(node.func, ast.Attribute): if self.PATTERN in node.func.attr: call = self._parse_args(node) if call: self.calls.append(call) def fetch_assets( test_file, test_file_parse_cache, klass=None, method=None, logger=None ): """Fetches the assets based on keywords listed on FetchAssetHandler.calls. :param test_file: File name of instrumented test to be evaluated :type test_file: str :returns: list of names that were successfully fetched and list of fails. """ cache_dirs = settings.as_dict().get("datadir.paths.cache_dirs") timeout = settings.as_dict().get("assets.fetch.timeout") success = [] fail = [] handler = FetchAssetHandler(test_file, test_file_parse_cache, klass, method) for call in handler.calls: expire = call.pop("expire", None) if expire is not None: expire = data_structures.time_to_seconds(str(expire)) try: asset_obj = Asset(**call, cache_dirs=cache_dirs, expire=expire) if logger is not None: logger.info("Fetching asset from %s:%s.%s", test_file, klass, method) asset_obj.fetch(timeout) success.append(call["name"]) except (OSError, ValueError) as failed: fail.append(failed) return success, fail class FetchAssetJob(JobPreTests): # pylint: disable=R0903 """Implements the assets fetch job pre tests. This has the same effect of running the 'avocado assets fetch INSTRUMENTED', but it runs during the test execution, before the actual test starts. """ name = "fetchasset" description = "Fetch assets before the test run" def __init__(self, config=None): # pylint: disable=W0231 pass def pre_tests(self, job): if not job.config.get("stdout_claimed_by", None): logger = job.log else: logger = None candidates = [] for suite in job.test_suites: for test in suite.tests: # nrunner/resolver based test should describe their requirements # explicitly using docstring statements. But, they can still # benefit from the parsing of "fetch_asset()" calls just like # the legacy runner. if isinstance(test, Runnable): if test.kind == "avocado-instrumented": module_path, klass_method = test.uri.split(":", 1) klass, method = klass_method.split(".", 1) candidate = (module_path, klass, method) if candidate not in candidates: candidates.append(candidate) test_file_parse_cache = {} for candidate in candidates: fetch_assets( candidate[0], test_file_parse_cache, candidate[1], candidate[2], logger ) class Assets(CLICmd): """ Implements the avocado 'assets' subcommand """ name = "assets" description = "Manage assets" @staticmethod def _count_filter_args(config): sub_command = config.get("assets_subcommand") args = [ config.get(f"assets.{sub_command}.days"), config.get(f"assets.{sub_command}.size_filter"), config.get(f"assets.{sub_command}.overall_limit"), ] return len([a for a in args if a is not None]) def configure(self, parser): """Add the subparser for the assets action. :param parser: The Avocado command line application parser :type parser: :class:`avocado.core.parser.ArgumentParser` """ def register_filter_options(subparser, section): help_msg = ( "Apply action based on a size filter (comparison " "operator + value) in bytes. Ex '>20', '<=200'. " "Supported operators: " + ", ".join(SUPPORTED_OPERATORS) ) settings.register_option( section=section, key="size_filter", help_msg=help_msg, default=None, metavar="FILTER", key_type=str, long_arg="--by-size-filter", parser=subparser, ) help_msg = "How old (in days) should Avocado look for assets?" settings.register_option( section=section, key="days", help_msg=help_msg, default=None, key_type=int, metavar="DAYS", long_arg="--by-days", parser=subparser, ) help_msg = ( "Filter will be based on a overall system limit " "threshold in bytes (with assets ordered by last " "access) or with a suffix unit. Valid suffixes are: " ) help_msg += ",".join(DataSize.MULTIPLIERS.keys()) settings.register_option( section=section, key="overall_limit", help_msg=help_msg, default=None, key_type=str, metavar="LIMIT", long_arg="--by-overall-limit", parser=subparser, ) parser = super().configure(parser) subcommands = parser.add_subparsers(dest="assets_subcommand") subcommands.required = True fetch_subcommand_parser = subcommands.add_parser( "fetch", help="Fetch assets from test source or config file if it's not" " already in the cache", ) help_msg = "Path to avocado instrumented test" settings.register_option( section="assets.fetch", key="references", help_msg=help_msg, default=[], metavar="AVOCADO_INSTRUMENTED", key_type=list, nargs="+", parser=fetch_subcommand_parser, positional_arg=True, ) help_msg = "always return success for the fetch command." settings.register_option( section="assets.fetch", key="ignore_errors", help_msg=help_msg, default=False, key_type=bool, parser=fetch_subcommand_parser, long_arg="--ignore-errors", ) help_msg = "Timeout to be used when download an asset." settings.register_option( section="assets.fetch", key="timeout", help_msg=help_msg, default=300, key_type=int, metavar="TIMEOUT", parser=fetch_subcommand_parser, long_arg="--timeout", ) register_subcommand_parser = subcommands.add_parser( "register", help="Register an asset directly to the cacche" ) help_msg = "Unique name to associate with this asset." settings.register_option( section="assets.register", key="name", help_msg=help_msg, default=None, key_type=str, parser=register_subcommand_parser, positional_arg=True, ) help_msg = "Path to asset that you would like to register manually." settings.register_option( section="assets.register", key="url", help_msg=help_msg, default=None, key_type=str, parser=register_subcommand_parser, positional_arg=True, ) help_msg = "SHA1 hash of this asset." settings.register_option( section="assets.register", key="sha1_hash", help_msg=help_msg, default=None, key_type=str, metavar="SHA1", long_arg="--hash", parser=register_subcommand_parser, ) purge_subcommand_parser = subcommands.add_parser( "purge", help="Removes assets cached locally." ) register_filter_options(purge_subcommand_parser, "assets.purge") help_msg = "List all cached assets." list_subcommand_parser = subcommands.add_parser("list", help=help_msg) register_filter_options(list_subcommand_parser, "assets.list") def handle_purge(self, config): days = config.get("assets.purge.days") size_filter = config.get("assets.purge.size_filter") overall_limit = config.get("assets.purge.overall_limit") if self._count_filter_args(config) != 1: msg = ( "You should choose --by-size-filter or --by-days. " "For help, run: avocado assets purge --help" ) LOG_UI.error(msg) return exit_codes.AVOCADO_FAIL cache_dirs = config.get("datadir.paths.cache_dirs") try: if days is not None: Asset.remove_assets_by_unused_for_days(days, cache_dirs) elif size_filter is not None: Asset.remove_assets_by_size(size_filter, cache_dirs) elif overall_limit is not None: try: size = DataSize(overall_limit).b Asset.remove_assets_by_overall_limit(size, cache_dirs) except InvalidDataSize: error_msg = "You are using an invalid suffix. " error_msg += "Use one of the following values: " error_msg += ",".join(DataSize.MULTIPLIERS.keys()) LOG_UI.error(error_msg) return exit_codes.AVOCADO_FAIL except (FileNotFoundError, OSError) as e: LOG_UI.error("Could not remove asset: %s", e) return exit_codes.AVOCADO_FAIL return exit_codes.AVOCADO_ALL_OK def handle_list(self, config): days = config.get("assets.list.days") size_filter = config.get("assets.list.size_filter") if self._count_filter_args(config) == 2: msg = ( "You should choose --by-size-filter or --by-days. " "For help, run: avocado assets list --help" ) LOG_UI.error(msg) return exit_codes.AVOCADO_FAIL cache_dirs = config.get("datadir.paths.cache_dirs") try: assets = None if days is not None: assets = Asset.get_assets_unused_for_days(days, cache_dirs) elif size_filter is not None: assets = Asset.get_assets_by_size(size_filter, cache_dirs) elif assets is None: assets = Asset.get_all_assets(cache_dirs) except (FileNotFoundError, OSError) as e: LOG_UI.error("Could get assets: %s", e) return exit_codes.AVOCADO_FAIL matrix = [] for asset in assets: stat = os.stat(asset) basename = os.path.basename(asset) hash_path = f"{asset}-CHECKSUM" atime = datetime.fromtimestamp(stat.st_atime) _, checksum = Asset.read_hash_from_file(hash_path) matrix.append( ( basename, str(checksum or "unknown")[:10], atime.strftime("%Y-%m-%d %H:%M:%S"), display_data_size(stat.st_size), ) ) header = ("asset", "checksum", "atime", "size") output = list(iter_tabular_output(matrix, header)) if len(output) == 1: LOG_UI.info("No asset found in your cache system.") else: for line in output: LOG_UI.info(line) @staticmethod def handle_fetch(config): exitcode = exit_codes.AVOCADO_ALL_OK # fetch assets from instrumented tests cache = {} for test_file in config.get("assets.fetch.references"): if os.path.isfile(test_file) and test_file.endswith(".py"): LOG_UI.debug("Fetching assets from %s.", test_file) success, fail = fetch_assets(test_file, cache) for asset_file in success: LOG_UI.debug(" File %s fetched or already on cache.", asset_file) for asset_file in fail: LOG_UI.error(asset_file) if fail: exitcode |= exit_codes.AVOCADO_FAIL else: LOG_UI.warning("No such file or file not supported: %s", test_file) exitcode |= exit_codes.AVOCADO_FAIL # check if we should ignore the errors if config.get("assets.fetch.ignore_errors"): return exit_codes.AVOCADO_ALL_OK return exitcode @staticmethod def handle_register(config): cache_dirs = config.get("datadir.paths.cache_dirs") name = config.get("assets.register.name") asset_hash = config.get("assets.register.sha1_hash") location = config.get("assets.register.url") # Adding a twice the location is a small hack due the current logic to # return "by_name". This needs to be improved soon. asset = Asset( name=name, asset_hash=asset_hash, locations=[location, location], cache_dirs=cache_dirs, ) try: asset.find_asset_file() LOG_UI.error("Asset with name %s already registered.", name) return exit_codes.AVOCADO_WARNING except OSError: try: asset.fetch() LOG_UI.info("Done. Now you can reference it by name %s", name) return exit_codes.AVOCADO_ALL_OK except OSError as e: LOG_UI.error(e) return exit_codes.AVOCADO_FAIL def run(self, config): subcommand = config.get("assets_subcommand") if subcommand == "fetch": return self.handle_fetch(config) elif subcommand == "register": return self.handle_register(config) elif subcommand == "purge": return self.handle_purge(config) elif subcommand == "list": return self.handle_list(config) else: return exit_codes.UTILITY_FAIL avocado-113.0/avocado/plugins/beaker_result.py000066400000000000000000000102371513561306000214330ustar00rootroot00000000000000""" beaker result plugin Sends results and logs to beaker using the harness API. https://beaker-project.org/docs/alternative-harnesses/index.html Finds the beaker API entry point using the BEAKER_LAB_CONTROLLER_URL environment variable. Does nothing in case the variable is not set. """ import glob import os import pprint import urllib.error import urllib.parse import urllib.request from avocado.core.output import LOG_UI from avocado.core.plugin_interfaces import ResultEvents class BeakerResult(ResultEvents): """ send test results to beaker test harness api """ name = "beaker" description = "report results to beaker" beaker_url = None job_id = None def __init__(self, config=None): # pylint: disable=W0613,W0231 baseurl = os.environ.get("BEAKER_LAB_CONTROLLER_URL") recipeid = os.environ.get("RSTRNT_RECIPEID") taskid = os.environ.get("RSTRNT_TASKID") if not all([baseurl, recipeid, taskid]): return baseurl = baseurl.rstrip("/") self.beaker_url = baseurl + "/recipes/" + recipeid + "/tasks/" + taskid LOG_UI.info("beaker: using API at %s (R:%s T:%s)", baseurl, recipeid, taskid) def send_request(self, req): LOG_UI.debug("beaker: %s %s ...", req.method, req.full_url) try: res = urllib.request.urlopen(req) # nosec return res except urllib.error.URLError as err: LOG_UI.info("beaker: %s %s failed: %s", req.method, req.full_url, err) return None except Exception as err: # should not happen LOG_UI.info("beaker: Oops: %s", err) return None def post_result(self, state): reqdict = { "path": f"{self.job_id}/{state.get('name')}", } result = state.get("status").lower().capitalize() if result == "Cancel": result = "Skip" if result not in ["Pass", "Fail", "Skip"]: result = "Fail" reqdict["result"] = result reason = state.get("fail_reason") if reason is not None: reqdict["message"] = reason secs = state.get("time_elapsed") if secs is not None: reqdict["score"] = str(int(secs)) reqdata = urllib.parse.urlencode(reqdict).encode("utf-8") url = self.beaker_url + "/results/" req = urllib.request.Request(url, method="POST", data=reqdata) res = self.send_request(req) if res is None: return None return res.getheader("Location") def put_data(self, location, name, content): url = location + "/logs/" + name req = urllib.request.Request(url, method="PUT", data=content) self.send_request(req) def put_file(self, location, name, filename): file = open(filename, encoding="utf-8") content = file.read().encode("utf-8") file.close() self.put_data(location, name, content) def put_file_list(self, location, prefix, filelist): for file in filelist: if os.path.isfile(file) and os.path.getsize(file) > 0: name = prefix + os.path.basename(file) self.put_file(location, name, file) def pre_tests(self, job): self.job_id = job.unique_id[:6] return def start_test(self, result, state): return def test_progress(self, progress=False): return def end_test(self, result, state): if self.beaker_url is None: return location = self.post_result(state) if location is None: return logfile = state.get("logfile") self.put_file(location, "logfile", logfile) ppstate = pprint.pformat(state).encode("utf8") self.put_data(location, "state", ppstate) pattern = os.path.join(state.get("logdir"), "*") filelist = [f for f in glob.glob(pattern) if f != logfile] self.put_file_list(location, "", filelist) def post_tests(self, job): if self.beaker_url is None: return pattern = os.path.join(job.logdir, "*") filelist = glob.glob(pattern) self.put_file_list(self.beaker_url, self.job_id + "-", filelist) avocado-113.0/avocado/plugins/bystatus.py000066400000000000000000000024411513561306000204600ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2022 # Authors: Beraldo Leal import os from avocado.core.plugin_interfaces import ResultEvents class ByStatusLink(ResultEvents): description = "Creates symlinks on file system grouped by status" def __init__(self, config): # pylint: disable=W0231 pass def pre_tests(self, job): pass def post_tests(self, job): pass def start_test(self, result, state): pass def test_progress(self, progress=False): pass def end_test(self, result, state): link = state.get("logdir") where = os.path.join(os.path.dirname(link), "by-status", state.get("status")) os.makedirs(where, exist_ok=True) os.symlink( os.path.relpath(link, where), os.path.join(where, os.path.basename(link)) ) avocado-113.0/avocado/plugins/cache.py000066400000000000000000000070661513561306000176550ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2022 # Author: Jan Richter from avocado.core.dispatcher import CacheDispatcher from avocado.core.output import LOG_UI from avocado.core.plugin_interfaces import CLICmd from avocado.core.settings import settings class Cache(CLICmd): """ Implements the avocado 'cache' subcommand """ name = "cache" description = "Interface for manipulating the Avocado cache metadata" def _list_plugin_entries(self, plugins, selected_plugins): if not selected_plugins: selected_plugins = [p.name for p in plugins] for plugin in plugins: if plugin.name in selected_plugins: LOG_UI.debug("%s:", plugin.obj.name) cache_list = "\t" + "\t".join(plugin.obj.list().splitlines(True)) LOG_UI.debug("%s\n", cache_list) def _clear_plugin_entries(self, plugins, selected_plugins): if not selected_plugins: selected_plugins = [p.name for p in plugins] for plugin in plugins: if plugin.name in selected_plugins: plugin.obj.clear() def configure(self, parser): """ Add the subparser for the cache action. :param parser: The Avocado command line application parser :type parser: :class:`avocado.core.parser.ArgumentParser` """ parser = super().configure(parser) subcommands = parser.add_subparsers(dest="cache_subcommand") subcommands.required = True list_help_msg = "List metadata in avocado cache" list_parser = subcommands.add_parser("list", help=list_help_msg) settings.register_option( section="cache", key="list", key_type=list, default=[], help_msg=list_help_msg, ) settings.add_argparser_to_option( namespace="cache.list", nargs="*", metavar="CACHE_TYPE", parser=list_parser, long_arg=None, positional_arg=True, allow_multiple=True, ) clear_help_msg = ( "Clear avocado cache, you can specify which part of cache" " will be removed." ) clear_parser = subcommands.add_parser("clear", help=clear_help_msg) settings.register_option( section="cache", key="clear", key_type=list, default=[], help_msg=clear_help_msg, ) settings.add_argparser_to_option( namespace="cache.clear", nargs="*", metavar="CACHE_TYPE", parser=clear_parser, long_arg=None, positional_arg=True, allow_multiple=True, ) def run(self, config): dispatcher = CacheDispatcher() plugins = dispatcher.get_extentions_by_priority() subcommand = config.get("cache_subcommand") if subcommand == "list": self._list_plugin_entries(plugins, config.get("cache.list")) if subcommand == "clear": self._clear_plugin_entries(plugins, config.get("cache.clear")) avocado-113.0/avocado/plugins/config.py000066400000000000000000000077531513561306000200620ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2013-2014 # Author: Lucas Meneghel Rodrigues # Beraldo Leal import textwrap from avocado.core import data_dir from avocado.core.output import LOG_UI from avocado.core.plugin_interfaces import CLICmd from avocado.core.settings import settings class Config(CLICmd): """ Implements the avocado 'config' subcommand """ name = "config" description = "Shows avocado config keys" def configure(self, parser): parser = super().configure(parser) help_msg = "Shows the data directories currently being used by Avocado" settings.register_option( section="config", key="datadir", key_type=bool, default=False, help_msg=help_msg, parser=parser, long_arg="--datadir", ) subcommands = parser.add_subparsers( dest="config_subcommand", metavar="sub-command" ) help_msg = "Show a configuration reference with all registered options" subcommands.add_parser("reference", help=help_msg) def run(self, config): if config.get("config_subcommand") == "reference": self.handle_reference(LOG_UI.debug) else: self.handle_default() @staticmethod def handle_reference(print_function): full = settings.as_full_dict() for namespace, option in full.items(): print_function(namespace) print_function("~" * len(namespace)) help_lines = textwrap.wrap(option.get("help")) for line in help_lines: print_function(line) print_function("") print_function(f"* Default: {option.get('default')}") print_function(f"* Type: {option.get('type')}") print_function("") @staticmethod def handle_default(): LOG_UI.info( "Config files read (in order, '*' means the file exists " "and had been read):" ) # Getting from settings because is already sorted config = settings.as_dict() for cfg_path in settings.all_config_paths: if cfg_path in settings.config_paths: LOG_UI.debug(" * %s", cfg_path) else: LOG_UI.debug(" %s", cfg_path) LOG_UI.debug("") if not config.get("config.datadir"): blength = 0 for namespace, value in config.items(): clength = len(namespace) if clength > blength: blength = clength format_str = " %-" + str(blength) + "s %s" LOG_UI.debug(format_str, "Section.Key", "Value") for namespace, value in config.items(): LOG_UI.debug(format_str, namespace, value) else: cache_dirs = config.get("datadir.paths.cache_dirs") LOG_UI.debug("Avocado replaces config dirs that can't be accessed") LOG_UI.debug("with sensible defaults. Please edit your local config") LOG_UI.debug("file to customize values") LOG_UI.debug("") LOG_UI.info("Avocado Data Directories:") LOG_UI.debug(" base %s", config.get("datadir.paths.base_dir")) LOG_UI.debug(" tests %s", data_dir.get_test_dir()) LOG_UI.debug(" data %s", config.get("datadir.paths.data_dir")) LOG_UI.debug(" logs %s", config.get("datadir.paths.logs_dir")) LOG_UI.debug(" cache %s", ", ".join(cache_dirs)) avocado-113.0/avocado/plugins/dependency.py000066400000000000000000000065771513561306000207360ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2021 # Authors: Willian Rampazzo import json from avocado.core.dependencies.dependency import Dependency from avocado.core.exceptions import JobBaseException from avocado.core.nrunner.runnable import Runnable from avocado.core.plugin_interfaces import JobPre, PreTest class DependencyResolverError(JobBaseException): """ Error raised when issue occurred during dependency file resolution. """ class DependencyResolver(PreTest): """Implements the dependency pre tests plugin. It will create pre-test tasks for managing dependencies based on the `:avocado: dependency=` definition inside the test’s docstring. """ name = "dependency" description = "Dependency resolver for tests with dependencies" is_cacheable = True @staticmethod def pre_test_runnables(test_runnable, suite_config=None): # pylint: disable=W0221 if not test_runnable.dependencies: return [] dependency_runnables = [] unique_dependencies = list(dict.fromkeys(test_runnable.dependencies)) for dependency in unique_dependencies: dependency_runnables.append( DependencyResolver._dependency_to_runnable( dependency, test_runnable.config ) ) return dependency_runnables @staticmethod def _dependency_to_runnable(dependency, config): return Runnable( dependency.kind, dependency.uri, *dependency.args, config=config, **dependency.kwargs, ) class SuiteDependency(JobPre): name = "suite-dependency" description = "Applies a set of dependencies to every test within the suite" def pre(self, job): for suite in job.test_suites: dependency_file_path = suite.config.get("job.run.dependency") if dependency_file_path: try: with open( dependency_file_path, encoding="utf-8" ) as dependency_file: try: dependencies_dict = json.load(dependency_file) except json.JSONDecodeError as e: raise DependencyResolverError( f"Issue with parsing dependency file at {dependency_file_path}: {e}" ) except FileNotFoundError as e: raise DependencyResolverError( f"Dependency file not found at {dependency_file_path}: {e}" ) dependencies = [] for dependency in dependencies_dict: dependencies.append(Dependency.from_dictionary(dependency)) for runnable in suite.tests: dependencies.extend(runnable.dependencies) runnable.dependencies = dependencies avocado-113.0/avocado/plugins/dict_variants.py000066400000000000000000000066471513561306000214500ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2020 # Authors: Cleber Rosa from avocado.core import varianter from avocado.core.plugin_interfaces import Init, Varianter from avocado.core.settings import settings from avocado.core.tree import TreeNode class DictVariantsInit(Init): name = "dict_variants" description = "Python Dictionary based varianter" def initialize(self): help_msg = "Load the Variants from Python dictionaries" settings.register_option( section="run", key="dict_variants", default=[], key_type=list, help_msg=help_msg, ) help_msg = ( "Configure the key that will be used to name the " "variant ID. If not set, will use all keys." ) settings.register_option( section="run.dict_variants", key="variant_id_keys", default=[], key_type=list, help_msg=help_msg, ) class DictVariants(Varianter): """ Turns (a list of) Python dictionaries into variants """ name = "dict_variants" description = "Python Dictionary based varianter" def initialize(self, config): # pylint: disable=W0201 self.variants = config.get("run.dict_variants") if self.variants: # pylint: disable=W0201 self.headers = [key for key in self.variants[0].keys()] self.headers_for_id = config.get("run.dict_variants.variant_id_keys") if not self.headers_for_id: self.headers_for_id = self.headers def __iter__(self): if self.variants is None: return variant_ids = [] for variant in self.variants: variant_ids.append( "-".join([str(variant.get(key)) for key in self.headers_for_id]) ) for vid, variant in zip(variant_ids, self.variants): yield { "variant_id": vid, "variant": [TreeNode("", variant)], "paths": ["/"], } def __len__(self): return sum(1 for _ in self.variants) if self.variants else 0 def to_str(self, summary, variants, **kwargs): """ Return human readable representation The summary/variants accepts verbosity where 0 means silent and maximum is up to the plugin. :param summary: How verbose summary to output (int) :param variants: How verbose list of variants to output (int) :param kwargs: Other free-form arguments :rtype: str """ if not self.variants: return "" out = [] if variants: # variants == 0 means disable, but in plugin it's brief out.append(f"Dict Variants ({len(self)}):") for variant in self: out.extend( varianter.variant_to_str(variant, variants - 1, kwargs, False) ) return "\n".join(out) avocado-113.0/avocado/plugins/diff.py000066400000000000000000000366241513561306000175240ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2016 # Author: Amador Pahim """ Job Diff """ import argparse import json import os import subprocess import sys import tempfile from difflib import HtmlDiff, unified_diff from avocado.core import data_dir, exit_codes, jobdata, output from avocado.core.output import LOG_UI from avocado.core.plugin_interfaces import CLICmd from avocado.core.settings import settings from avocado.core.varianter import Varianter class Diff(CLICmd): """ Implements the avocado 'diff' subcommand """ name = "diff" description = "Shows the difference between 2 jobs." def __init__(self): # pylint: disable=W0231 self.term = output.TERM_SUPPORT self.std_diff_output = True def configure(self, parser): """ Add the subparser for the diff action. :param parser: The Avocado command line application parser :type parser: :class:`avocado.core.parser.ArgumentParser` """ parser = super().configure(parser) parser.epilog = ( "By default, a textual diff report is generated in the standard output." ) help_msg = ( "A job reference, identified by a (partial) unique ID " "(SHA1) or test results directory." ) settings.register_option( section="diff", key="jobids", default=[], key_type=list, nargs=2, help_msg=help_msg, parser=parser, metavar="JOB", positional_arg="jobids", ) help_msg = "Enable HTML output to the FILE where the result should be written." settings.register_option( section="diff", key="html", default=None, metavar="FILE", help_msg=help_msg, parser=parser, long_arg="--html", ) help_msg = ( "Generate and open a HTML report in your preferred " "browser. If no --html file is provided, create a " "temporary file." ) settings.register_option( section="diff", key="open_browser", default=False, key_type=bool, help_msg=help_msg, parser=parser, long_arg="--open-browser", ) help_msg = ( "Comma separated filter of diff sections: " "(no)cmdline,(no)time,(no)variants,(no)results, " "(no)config,(no)sysinfo (defaults to all enabled)." ) settings.register_option( section="diff", key="filter", metavar="DIFF_FILTER", key_type=self._validate_filters, help_msg=help_msg, default=["cmdline", "time", "variants", "results", "config", "sysinfo"], parser=parser, long_arg="--diff-filter", ) help_msg = 'Strip the "id" from "id-name;variant" when comparing test results.' settings.register_option( section="diff", key="strip_id", default=False, key_type=bool, help_msg=help_msg, parser=parser, long_arg="--diff-strip-id", ) help_msg = ( "Create temporary files with job reports to be used by other diff tools" ) settings.register_option( section="diff", key="create_reports", default=False, key_type=bool, help_msg=help_msg, parser=parser, long_arg="--create-reports", ) def run(self, config): def _get_name(test): return str(test["id"]) def _get_name_no_id(test): return str(test["id"]).split("-", 1)[1] job1_dir, job1_id = self._setup_job(config.get("diff.jobids")[0]) job2_dir, job2_id = self._setup_job(config.get("diff.jobids")[1]) job1_data = self._get_job_data(job1_dir) job2_data = self._get_job_data(job2_dir) report_header = "Avocado Job Report\n" job1_results = [report_header] job2_results = [report_header] diff_filter = config.get("diff.filter") if "cmdline" in diff_filter: cmdline1 = self._get_command_line(job1_dir) cmdline2 = self._get_command_line(job2_dir) if str(cmdline1) != str(cmdline2): command_line_header = ["\n", "# COMMAND LINE\n"] job1_results.extend(command_line_header) job1_results.append(cmdline1) job2_results.extend(command_line_header) job2_results.append(cmdline2) if "time" in diff_filter: time1 = f"{job1_data['time']:.2f} s\n" time2 = f"{job2_data['time']:.2f} s\n" if str(time1) != str(time2): total_time_header = ["\n", "# TOTAL TIME\n"] job1_results.extend(total_time_header) job1_results.append(time1) job2_results.extend(total_time_header) job2_results.append(time2) if "variants" in diff_filter: variants1 = self._get_variants(job1_dir) variants2 = self._get_variants(job2_dir) if str(variants1) != str(variants2): variants_header = ["\n", "# VARIANTS\n"] job1_results.extend(variants_header) job1_results.extend(variants1) job2_results.extend(variants_header) job2_results.extend(variants2) if "results" in diff_filter: results1 = [] if config.get("diff.strip_id"): get_name = _get_name_no_id else: get_name = _get_name for test in job1_data["tests"]: test_result = f"{get_name(test)}: {str(test['status'])}\n" results1.append(test_result) results2 = [] for test in job2_data["tests"]: test_result = f"{get_name(test)}: {str(test['status'])}\n" results2.append(test_result) if str(results1) != str(results2): test_results_header = ["\n", "# TEST RESULTS\n"] job1_results.extend(test_results_header) job1_results.extend(results1) job2_results.extend(test_results_header) job2_results.extend(results2) if "config" in diff_filter: config1 = self._get_config(job1_dir) config2 = self._get_config(job2_dir) if str(config1) != str(config2): config_header = ["\n", "# SETTINGS\n"] job1_results.extend(config_header) job1_results.extend(config1) job2_results.extend(config_header) job2_results.extend(config2) if "sysinfo" in diff_filter: sysinfo_pre1 = self._get_sysinfo(job1_dir, "pre") sysinfo_pre2 = self._get_sysinfo(job2_dir, "pre") if str(sysinfo_pre1) != str(sysinfo_pre2): sysinfo_header_pre = ["\n", "# SYSINFO PRE\n"] job1_results.extend(sysinfo_header_pre) job1_results.extend(sysinfo_pre1) job2_results.extend(sysinfo_header_pre) job2_results.extend(sysinfo_pre2) sysinfo_post1 = self._get_sysinfo(job1_dir, "post") sysinfo_post2 = self._get_sysinfo(job2_dir, "post") if str(sysinfo_post1) != str(sysinfo_post2): sysinfo_header_post = ["\n", "# SYSINFO POST\n"] job1_results.extend(sysinfo_header_post) job1_results.extend(sysinfo_post1) job2_results.extend(sysinfo_header_post) job2_results.extend(sysinfo_post2) if config.get("diff.create_reports"): self.std_diff_output = False prefix = f"avocado_diff_{job1_id[:7]}_" tmp_file1 = tempfile.NamedTemporaryFile( mode="w", prefix=prefix, suffix=".txt", delete=False ) tmp_file1.writelines(job1_results) tmp_file1.close() prefix = f"avocado_diff_{job2_id[:7]}_" tmp_file2 = tempfile.NamedTemporaryFile( mode="w", prefix=prefix, suffix=".txt", delete=False ) tmp_file2.writelines(job2_results) tmp_file2.close() LOG_UI.info("%s %s", tmp_file1.name, tmp_file2.name) html_file = config.get("diff.html") open_browser = config.get("diff.open_browser") if open_browser and html_file is None: prefix = f"avocado_diff_{job1_id[:7]}_{job2_id[:7]}_" tmp_file = tempfile.NamedTemporaryFile( mode="w", prefix=prefix, suffix=".html", delete=False ) html_file = tmp_file.name if html_file is not None: self.std_diff_output = False try: html_diff = HtmlDiff() # pylint: disable=W0212 html_diff._legend = """
Colors
 Added 
Changed
Deleted
Links
(f)irst change
(n)ext change
(t)op
""" job_diff_html = html_diff.make_file( (_ for _ in job1_results), (_ for _ in job2_results), fromdesc=job1_id, todesc=job2_id, ) with open(html_file, "w", encoding="utf-8") as fp: fp.writelines(job_diff_html) LOG_UI.info(html_file) except IOError as exception: LOG_UI.error(exception) sys.exit(exit_codes.AVOCADO_FAIL) if open_browser: setsid = getattr(os, "setsid", None) if not setsid: setsid = getattr(os, "setpgrp", None) with open(os.devnull, "r+", encoding="utf-8") as in_out: cmd = ["xdg-open", html_file] subprocess.Popen( # pylint: disable=W1509 cmd, close_fds=True, stdin=in_out, stdout=in_out, stderr=in_out, preexec_fn=setsid, ) if self.std_diff_output: if self.term.enabled: for line in self._cdiff( unified_diff( job1_results, job2_results, fromfile=job1_id, tofile=job2_id ) ): LOG_UI.debug(line.strip()) else: for line in unified_diff( job1_results, job2_results, fromfile=job1_id, tofile=job2_id ): LOG_UI.debug(line.strip()) @staticmethod def _validate_filters(string): input_filter = set(string.split(",")) include_options = [ "cmdline", "time", "variants", "results", "config", "sysinfo", ] exclude_options = [ "nocmdline", "notime", "novariants", "noresults", "noconfig", "nosysinfo", ] invalid = input_filter.difference(include_options + exclude_options + ["all"]) if invalid: msg = f"Invalid option(s) '{','.join(invalid)}'" raise argparse.ArgumentTypeError(msg) if input_filter.intersection(exclude_options): output_filter = [ _ for _ in include_options if ("no" + _) not in input_filter ] elif "all" in input_filter: output_filter = include_options else: output_filter = input_filter return output_filter @staticmethod def _get_job_data(jobdir): results_json = os.path.join(jobdir, "results.json") with open(results_json, "r", encoding="utf-8") as json_file: data = json.load(json_file) return data @staticmethod def _setup_job(job_id): resultsdir = data_dir.get_job_results_dir(job_id) if resultsdir is None: LOG_UI.error("Can't find job results directory for '%s'", job_id) sys.exit(exit_codes.AVOCADO_FAIL) with open(os.path.join(resultsdir, "id"), "r", encoding="utf-8") as id_file: sourcejob = id_file.read().strip() return resultsdir, sourcejob @staticmethod def _get_command_line(resultsdir): command_line = jobdata.retrieve_cmdline(resultsdir) if command_line is not None: return f"{' '.join(command_line)}\n" return "Not found\n" @staticmethod def _get_variants(resultsdir): results = [] variants = Varianter.from_resultsdir(resultsdir) if variants is not None: for variant in variants: results.extend(variant.to_str(variants=2).splitlines()) else: results.append("Not found\n") return results @staticmethod def _get_config(resultsdir): config_file = os.path.join(resultsdir, "replay", "config") try: with open(config_file, "r", encoding="utf-8") as conf: return conf.readlines() except IOError: return ["Not found\n"] @staticmethod def _get_sysinfo(resultsdir, pre_post): sysinfo_dir = os.path.join(resultsdir, "sysinfo", pre_post) sysinfo = [] for path, _, files in os.walk(sysinfo_dir): for name in sorted(files): name_header = ["\n", f"** {name} **\n"] sysinfo.extend(name_header) with open( os.path.join(path, name), "r", encoding="utf-8" ) as sysinfo_file: try: sysinfo.extend(sysinfo_file.readlines()) except UnicodeDecodeError: msg = f"Ignoring file {name} as it cannot be decoded." LOG_UI.debug(msg) continue if sysinfo: del sysinfo[0] return sysinfo def _cdiff(self, diff): for line in diff: if line.startswith("+"): yield self.term.COLOR_GREEN + line elif line.startswith("-"): yield self.term.COLOR_RED + line elif line.startswith("@"): yield self.term.COLOR_BLUE + line else: yield line avocado-113.0/avocado/plugins/distro.py000066400000000000000000000341361513561306000201140ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2015 # Author: Cleber Rosa import bz2 import json import os import sys from avocado.core import exit_codes from avocado.core.output import LOG_UI from avocado.core.plugin_interfaces import CLICmd from avocado.core.settings import settings from avocado.utils import distro as utils_distro from avocado.utils import path as utils_path from avocado.utils import process class SoftwarePackage: """ Definition of relevant information on a software package """ def __init__(self, name, version, release, checksum, arch): self.name = name self.version = version self.release = release self.checksum = checksum self.arch = arch def to_dict(self): """ Returns the representation as a dictionary """ return { "name": self.name, "version": self.version, "release": self.release, "checksum": self.checksum, "arch": self.arch, } def to_json(self): """ Returns the representation of the distro as JSON """ return json.dumps(self.to_dict()) class DistroDef(utils_distro.LinuxDistro): """ More complete information on a given Linux Distribution Can and should include all the software packages that ship with the distro, so that an analysis can be made on whether a given package that may be responsible for a regression is part of the official set or an external package. """ def __init__(self, name, version, release, arch): super().__init__(name, version, release, arch) #: All the software packages that ship with this Linux distro self.software_packages = [] #: A simple text that denotes the software type that makes this distro self.software_packages_type = "unknown" def to_dict(self): """ Returns the representation as a dictionary """ d = { "name": self.name, "version": self.version, "release": self.release, "arch": self.arch, "software_packages_type": self.software_packages_type, "software_packages": [], } for package in self.software_packages: d["software_packages"].append(package.to_dict()) return d def to_json(self): """ Returns the representation of the distro as JSON """ return json.dumps(self.to_dict()) class DistroPkgInfoLoader: """ Loads information from the distro installation tree into a DistroDef It will go through all package files and inspect them with specific package utilities, collecting the necessary information. """ def __init__(self, path): self.path = path def get_packages_info(self): """ This method will go through each file, checking if it's a valid software package file by calling :meth:`is_software_package` and calling :meth:`load_package_info` if it's so. """ packages_info = set() for dirpath, _, filenames in os.walk(self.path): for filename in filenames: path = os.path.join(dirpath, filename) if self.is_software_package(path): packages_info.add(self.get_package_info(path)) # because we do not track of locations or how many copies of a given # package file exists in the installation tree, packages should be # comprised of unique entries return list(packages_info) def is_software_package(self, path): """ Determines if the given file at `path` is a software package This check will be used to determine if :meth:`load_package_info` will be called for file at `path`. This method should be implemented by classes inheriting from :class:`DistroPkgInfoLoader` and could be as simple as checking for a file suffix. :param path: path to the software package file :type path: str :return: either True if the file is a valid software package or False otherwise :rtype: bool """ raise NotImplementedError def get_package_info(self, path): """ Returns information about a given software package Should be implemented by classes inheriting from :class:`DistroDefinitionLoader`. :param path: path to the software package file :type path: str :returns: tuple with name, version, release, checksum and arch :rtype: tuple """ raise NotImplementedError class DistroPkgInfoLoaderRpm(DistroPkgInfoLoader): """ Loads package information for RPM files """ def __init__(self, path): super().__init__(path) try: utils_path.find_command("rpm") self.capable = True except utils_path.CmdNotFoundError: self.capable = False def is_software_package(self, path): """ Systems needs to be able to run the rpm binary in order to fetch information on package files. If the rpm binary is not available on this system, we simply ignore the rpm files found """ return self.capable and path.endswith(".rpm") def get_package_info(self, path): cmd = "rpm -qp --qf '%{NAME} %{VERSION} %{RELEASE} %{SIGMD5} %{ARCH}' " cmd += path info = process.system_output(cmd, ignore_status=True) info = tuple(info.split(" ")) return info class DistroPkgInfoLoaderDeb(DistroPkgInfoLoader): """ Loads package information for DEB files """ def __init__(self, path): super().__init__(path) try: utils_path.find_command("dpkg-deb") self.capable = True except utils_path.CmdNotFoundError: self.capable = False def is_software_package(self, path): return self.capable and (path.endswith(".deb") or path.endswith(".udeb")) def get_package_info(self, path): cmd = "dpkg-deb --showformat '${Package} ${Version} ${Architecture}' --show " cmd += path info = process.system_output(cmd, ignore_status=True) name, version, arch = info.split(" ") return (name, version, "", "", arch) #: the type of distro that will determine what loader will be used DISTRO_PKG_INFO_LOADERS = {"rpm": DistroPkgInfoLoaderRpm, "deb": DistroPkgInfoLoaderDeb} def save_distro(linux_distro, path): """ Saves the linux_distro to an external file format :param linux_distro: an :class:`DistroDef` instance :type linux_distro: DistroDef :param path: the location for the output file :type path: str :return: None """ with open(path, "wb") as output: buff = linux_distro.to_json() output.write(bz2.compress(buff.encode("utf-8"))) def load_distro(path): """ Loads the distro from an external file :param path: the location for the input file :type path: str :return: a dict with the distro definition data :rtype: dict """ with open(path, "rb") as distro_file: json_data = json.loads(bz2.decompress(distro_file.read())) return json_data def load_from_tree(name, version, release, arch, package_type, path): """ Loads a DistroDef from an installable tree :param name: a short name that precisely distinguishes this Linux Distribution among all others. :type name: str :param version: the major version of the distribution. Usually this is a single number that denotes a large development cycle and support file. :type version: str :param release: the release or minor version of the distribution. Usually this is also a single number, that is often omitted or starts with a 0 when the major version is initially release. It's often associated with a shorter development cycle that contains incremental a collection of improvements and fixes. :type release: str :param arch: the main target for this Linux Distribution. It's common for some architectures to ship with packages for previous and still compatible architectures, such as it's the case with Intel/AMD 64 bit architecture that support 32 bit code. In cases like this, this should be set to the 64 bit architecture name. :type arch: str :param package_type: one of the available package info loader types :type package_type: str :param path: top level directory of the distro installation tree files :type path: str """ distro_def = DistroDef(name, version, release, arch) loader_class = DISTRO_PKG_INFO_LOADERS.get(package_type, None) if loader_class is not None: loader = loader_class(path) distro_def.software_packages = [ SoftwarePackage(*args) for args in loader.get_packages_info() ] distro_def.software_packages_type = package_type return distro_def class Distro(CLICmd): """ Implements the avocado 'distro' subcommand """ name = "distro" description = "Shows detected Linux distribution" def configure(self, parser): parser = super().configure(parser) help_msg = "Creates a distro definition file based on the path given." settings.register_option( section="distro", key="distro_def_create", default=False, help_msg=help_msg, key_type=bool, parser=parser, long_arg="--distro-def-create", ) help_msg = "Distribution short name" settings.register_option( section="distro", key="distro_def_name", default="", help_msg=help_msg, parser=parser, long_arg="--distro-def-name", metavar="DISTRO_DEF_NAME", ) help_msg = "Distribution major version name" settings.register_option( section="distro", key="distro_def_version", default="", help_msg=help_msg, parser=parser, long_arg="--distro-def-version", metavar="DISTRO_DEF_VERSION", ) help_msg = "Distribution release version number" settings.register_option( section="distro", key="distro_def_release", default="", help_msg=help_msg, parser=parser, long_arg="--distro-def-release", metavar="DISTRO_DEF_RELEASE", ) help_msg = "Primary architecture that the distro targets" settings.register_option( section="distro", key="distro_def_arch", default="", help_msg=help_msg, parser=parser, long_arg="--distro-def-arch", metavar="DISTRO_DEF_ARCH", ) help_msg = "Top level directory of the distro installation files" settings.register_option( section="distro", key="distro_def_path", default="", help_msg=help_msg, parser=parser, long_arg="--distro-def-path", ) type_choices = tuple(DISTRO_PKG_INFO_LOADERS.keys()) type_choices_hlp = ", ".join(type_choices) help_msg = f"Distro type (one of: {type_choices_hlp})" settings.register_option( section="distro", key="distro_def_type", default="", help_msg=help_msg, choices=type_choices, parser=parser, long_arg="--distro-def-type", ) @staticmethod def _get_output_file_name(name, version, arch, release=None): """ Adapt the output file name based on given args It's not uncommon for some distros to not have a release number, so adapt the output file name to that """ if release: return f"{name}-{version}.{release}-{arch}.distro" else: return f"{name}-{version}-{arch}.distro" def run(self, config): name = config.get("distro.distro_def_name") version = config.get("distro.distro_def_version") release = config.get("distro.distro_def_release") arch = config.get("distro.distro_def_arch") distro_type = config.get("distro.distro_def_type") path = config.get("distro.distro_def_path") if config.get("distro.distro_def_create"): if not (name and version and arch and distro_type and path): LOG_UI.error("Required arguments: name, version, arch, type and path") sys.exit(exit_codes.AVOCADO_FAIL) output_file_name = self._get_output_file_name(name, version, arch, release) if os.path.exists(output_file_name): error_msg = ( 'Output file "%s" already exists, will not overwrite it', output_file_name, ) LOG_UI.error(error_msg) else: LOG_UI.debug("Loading distro information from tree... Please wait...") distro = load_from_tree(name, version, release, arch, distro_type, path) save_distro(distro, output_file_name) LOG_UI.debug('Distro information saved to "%s"', output_file_name) else: detected = utils_distro.detect() LOG_UI.debug( "Detected distribution: %s (%s) version %s release %s", detected.name, detected.arch, detected.version, detected.release, ) avocado-113.0/avocado/plugins/exec_path.py000066400000000000000000000022651513561306000205460ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2013-2015 """ Libexec PATHs modifier """ import os from pkg_resources import resource_filename from avocado.core.output import LOG_UI from avocado.core.plugin_interfaces import CLICmd class ExecPath(CLICmd): """ Implements the avocado 'exec-path' subcommand """ name = "exec-path" description = "Returns path to avocado bash libraries and exits." def run(self, config): """ Print libexec path and finish :param config: job configuration """ system_wide = "/usr/libexec/avocado" if os.path.isdir(system_wide): LOG_UI.debug(system_wide) else: LOG_UI.debug(resource_filename("avocado", "libexec")) avocado-113.0/avocado/plugins/human.py000066400000000000000000000124241513561306000177140ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat, Inc. 2016 # Author: Cleber Rosa """ Human result UI """ from avocado.core import output from avocado.core.output import LOG_UI from avocado.core.plugin_interfaces import Init, JobPost, JobPre, ResultEvents from avocado.core.settings import settings from avocado.core.teststatus import STATUSES #: STATUSES contains the finished status, but lacks the novel concept #: of nrunner having tests STARTED (that is, in progress). This contains #: a more complete list of statuses that includes "STARTED" COMPLETE_STATUSES = STATUSES + ["STARTED"] class HumanInit(Init): description = "Initialize human ui plugin settings" def initialize(self): help_msg = ( f"Status that will be omitted from the Human UI. " f"Valid statuses: {', '.join(COMPLETE_STATUSES)}" ) settings.register_option( section="human_ui.omit", key="statuses", key_type=list, default=[], help_msg=help_msg, ) class Human(ResultEvents): """ Human result UI """ name = "human" description = "Human Interface UI" def __init__(self, config): # pylint: disable=W0231 stdout_claimed_by = config.get("stdout_claimed_by", None) self.owns_stdout = not stdout_claimed_by self.omit_statuses = config.get("human_ui.omit.statuses") def pre_tests(self, job): if not self.owns_stdout: return LOG_UI.info("JOB ID : %s", job.unique_id) # TODO: this is part of the legacy implementation of the # replay plugin and should be removed soon. replay_enabled = replay_source_job = job.config.get("replay_sourcejob", False) # The "avocado replay" plugin sets a different namespace if not replay_source_job: replay_enabled = job.config.get("job.replay.enabled") replay_source_job = job.config.get("job.replay.source_job_id") if replay_enabled and replay_source_job: LOG_UI.info("SRC JOB ID : %s", replay_source_job) LOG_UI.info("JOB LOG : %s", job.logfile) def start_test(self, result, state): if not self.owns_stdout: return if "STARTED" in self.omit_statuses: return if "name" in state: name = state["name"] uid = name.str_uid name = name.name + name.str_variant else: name = "" uid = "?" LOG_UI.debug(" (%s/%s) %s: STARTED", uid, result.tests_total, name) def test_progress(self, progress=False): pass @staticmethod def get_colored_status(status, extra=None): out = ( output.TERM_SUPPORT.MOVE_BACK + output.TEST_STATUS_MAPPING[status] + status ) if extra: if len(extra) > 255: extra = extra[:255] + "..." extra = extra.replace("\n", "\\n") out += ": " + extra out += output.TERM_SUPPORT.ENDC return out def end_test(self, result, state): if not self.owns_stdout: return status = state.get("status", "ERROR") if status in self.omit_statuses: return if status == "TEST_NA": status = "SKIP" duration = ( f" ({state.get('time_elapsed', -1):.2f} s)" if status != "SKIP" else "" ) if "name" in state: name = state["name"] uid = name.str_uid name = name.name + name.str_variant else: name = "" uid = "?" msg = self.get_colored_status(status, state.get("fail_reason", None)) LOG_UI.debug( " (%s/%s) %s: ", uid, result.tests_total, name, extra={"skip_newline": True}, ) LOG_UI.debug(msg + duration) def post_tests(self, job): if not self.owns_stdout: return if job.interrupted_reason is not None: LOG_UI.warning(job.interrupted_reason) if job.status == "PASS": LOG_UI.info( "RESULTS : PASS %d | ERROR %d | FAIL %d | SKIP %d | " "WARN %d | INTERRUPT %s | CANCEL %s", job.result.passed, job.result.errors, job.result.failed, job.result.skipped, job.result.warned, job.result.interrupted, job.result.cancelled, ) class HumanJob(JobPre, JobPost): """ Human result UI """ name = "human" description = "Human Interface UI" def pre(self, job): pass def post(self, job): if job.status == "PASS": if not job.config.get("stdout_claimed_by", None): LOG_UI.info("JOB TIME : %.2f s", job.time_elapsed) avocado-113.0/avocado/plugins/jobs.py000066400000000000000000000164611513561306000175460ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2020 # Authors: Beraldo Leal """ Jobs subcommand """ import json import os from datetime import datetime from glob import glob from avocado.core import exit_codes, output from avocado.core.data_dir import get_job_results_dir, get_logs_dir from avocado.core.output import LOG_UI from avocado.core.plugin_interfaces import CLICmd from avocado.core.settings import settings from avocado.utils import astring class Jobs(CLICmd): """ Implements the avocado 'jobs' subcommand """ name = "jobs" description = "Manage Avocado jobs" @staticmethod def _get_data_from_file(filename): if not filename or not os.path.isfile(filename): raise FileNotFoundError(f"File not found {filename}") with open(filename, "r", encoding="utf-8") as fp: return json.load(fp) @staticmethod def _print_job_details(details): for key, value in details.items(): LOG_UI.info("%-12s: %s", key, value) @staticmethod def _format_test_status(status): """Format the test status with appropriate decorator.""" if status is None: raise ValueError("Test status is missing - corrupted test data") decorator = output.TEST_STATUS_DECORATOR_MAPPING.get(status) if decorator is None: raise ValueError(f"Unknown test status '{status}' - corrupted test data") return decorator(status, "") if decorator else "" @staticmethod def _format_end_time(test): """Format the test end time.""" date_fmt = "%Y/%m/%d %H:%M:%S" end_timestamp = test.get("actual_end", test.get("end")) if end_timestamp is not None: return datetime.fromtimestamp(end_timestamp).strftime(date_fmt) return "" @staticmethod def _format_run_time(test): """Format the test run time.""" time_taken = test.get("time") if time_taken is not None: return f"{float(time_taken):5f}" return "" @staticmethod def _create_matrix_row(test): """Create a matrix row for a single test.""" return ( test.get("id", ""), Jobs._format_end_time(test), Jobs._format_run_time(test), Jobs._format_test_status(test.get("status")), ) @staticmethod def _print_job_tests(tests): """Print formatted test results in a table.""" test_matrix = [Jobs._create_matrix_row(test) for test in tests] header = ( output.TERM_SUPPORT.header_str("Test ID"), output.TERM_SUPPORT.header_str("End Time"), output.TERM_SUPPORT.header_str("Run Time"), output.TERM_SUPPORT.header_str("Status"), ) for line in astring.iter_tabular_output(test_matrix, header=header, strip=True): LOG_UI.debug(line) def configure(self, parser): """ Add the subparser for the assets action. :param parser: The Avocado command line application parser :type parser: :class:`avocado.core.parser.ArgumentParser` """ parser = super().configure(parser) subcommands = parser.add_subparsers( dest="jobs_subcommand", metavar="sub-command" ) subcommands.required = True help_msg = "List all known jobs by Avocado" subcommands.add_parser("list", help=help_msg) help_msg = ( "Show details about a specific job. When passing a Job " 'ID, you can use any Job Reference (job_id, "latest", ' "or job results path)." ) show_parser = subcommands.add_parser("show", help=help_msg) settings.register_option( section="jobs.show", key="job_id", help_msg="JOB id", metavar="JOBID", default="latest", nargs="?", positional_arg=True, parser=show_parser, ) @staticmethod def handle_list_command(jobs_results): """Called when 'avocado jobs list' command is executed.""" for filename in jobs_results.values(): with open(filename, "r", encoding="utf-8") as fp: job = json.load(fp) LOG_UI.info( "%-40s %-26s %3s (%s/%s/%s/%s)", job.get("job_id", ""), job.get("start", ""), job.get("total", ""), job.get("pass", ""), job.get("skip", ""), job.get("errors", ""), job.get("failures", ""), ) return exit_codes.AVOCADO_ALL_OK def handle_show_command(self, config): """Called when 'avocado jobs show' command is executed.""" job_id = config.get("jobs.show.job_id") results_dir = get_job_results_dir(job_id) if results_dir is None: LOG_UI.error("Error: Job %s not found", job_id) return exit_codes.AVOCADO_GENERIC_CRASH results_file = os.path.join(results_dir, "results.json") try: results_data = self._get_data_from_file(results_file) except FileNotFoundError as ex: # Results data are important and should exit if not found LOG_UI.error(ex) return exit_codes.AVOCADO_GENERIC_CRASH data = { "JOB ID": job_id, "JOB LOG": results_data.get("debuglog"), } # We could improve this soon with more data and colors self._print_job_details(data) LOG_UI.info("") self._print_job_tests(results_data.get("tests")) results = ( "PASS %d | ERROR %d | FAIL %d | SKIP %d |" "WARN %d | INTERRUPT %s | CANCEL %s" ) results %= ( results_data.get("pass", 0), results_data.get("errors", 0), results_data.get("failures", 0), results_data.get("skip", 0), results_data.get("warn", 0), results_data.get("interrupt", 0), results_data.get("cancel", 0), ) self._print_job_details({"RESULTS": results}) return exit_codes.AVOCADO_ALL_OK def run(self, config): results = {} jobs_dir = get_logs_dir() for result in sorted( glob(os.path.join(jobs_dir, "*/results.json")), key=os.path.getmtime, reverse=True, ): with open(result, "r", encoding="utf-8") as fp: job = json.load(fp) results[job["job_id"]] = result subcommand = config.get("jobs_subcommand") if subcommand == "list": return self.handle_list_command(results) elif subcommand == "show": return self.handle_show_command(config) return exit_codes.AVOCADO_ALL_OK avocado-113.0/avocado/plugins/jobscripts.py000066400000000000000000000067471513561306000210010ustar00rootroot00000000000000import os from avocado.core.output import LOG_UI from avocado.core.plugin_interfaces import Init, JobPost, JobPre from avocado.core.settings import settings from avocado.core.utils.path import prepend_base_path from avocado.utils import process class JobScriptsInit(Init): name = "jobscripts-init" description = "Jobscripts plugin initialization" def initialize(self): help_msg = "Warn if configured (or default) directory does not exist" settings.register_option( section="plugins.jobscripts", key="warn_non_existing_dir", key_type=bool, default=False, help_msg=help_msg, ) help_msg = "Warn if any script run return non-zero status" settings.register_option( section="plugins.jobscripts", key="warn_non_zero_status", key_type=bool, default=True, help_msg=help_msg, ) help_msg = "Directory with scripts to be executed before a job is run" default = "/etc/avocado/scripts/job/pre.d/" settings.register_option( section="plugins.jobscripts", key="pre", key_type=prepend_base_path, help_msg=help_msg, default=default, ) help_msg = "Directory with scripts to be executed after a job is run" default = "/etc/avocado/scripts/job/post.d/" settings.register_option( section="plugins.jobscripts", key="post", key_type=prepend_base_path, help_msg=help_msg, default=default, ) class JobScripts(JobPre, JobPost): name = "jobscripts" description = "Runs scripts before/after the job is run" def _run_scripts(self, kind, scripts_dir, job): config = job.config if not os.path.isdir(scripts_dir): if config.get("plugins.jobscripts.warn_non_existing_dir"): LOG_UI.error( "Directory configured to hold %s-job scripts " "has not been found: %s", kind, scripts_dir, ) return dir_list = os.listdir(scripts_dir) scripts = [os.path.join(scripts_dir, f) for f in dir_list] scripts = [f for f in scripts if os.access(f, os.R_OK | os.X_OK)] scripts.sort() if not scripts: return env = self._job_to_environment_variables(job) non_zero_namespace = "plugins.jobscripts.warn_non_zero_status" warn_non_zero_status = config.get(non_zero_namespace) for script in scripts: result = process.run(script, ignore_status=True, env=env) if (result.exit_status != 0) and warn_non_zero_status: LOG_UI.error( '%s job script "%s" exited with status "%i"', kind.capitalize(), script, result.exit_status, ) @staticmethod def _job_to_environment_variables(job): env = {} env["AVOCADO_JOB_UNIQUE_ID"] = job.unique_id env["AVOCADO_JOB_STATUS"] = job.status if job.logdir is not None: env["AVOCADO_JOB_LOGDIR"] = job.logdir return env def pre(self, job): path = job.config.get("plugins.jobscripts.pre") self._run_scripts("pre", path, job) def post(self, job): path = job.config.get("plugins.jobscripts.post") self._run_scripts("post", path, job) avocado-113.0/avocado/plugins/journal.py000066400000000000000000000114201513561306000202510ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2014 # Author: Cleber Rosa """Journal Plugin""" import datetime import os import sqlite3 from avocado.core.plugin_interfaces import CLI, ResultEvents from avocado.core.settings import settings JOURNAL_FILENAME = ".journal.sqlite" SCHEMA = { "job_info": "CREATE TABLE job_info (unique_id TEXT UNIQUE)", "test_journal": ( "CREATE TABLE test_journal (" "tag TEXT, " "time TEXT, " "action TEXT, " "status TEXT, " "flushed BOOLEAN DEFAULT 0)" ), } class JournalResult(ResultEvents): """ Test Result Journal class. This class keeps a log of the test updates: started running, finished, etc. This information can be forwarded live to an avocado server and provide feedback to users from a central place. """ name = "journal" description = "Journal event based results implementation" def __init__(self, config): # pylint: disable=W0613, W0231 """ Creates an instance of ResultJournal. :param job: an instance of :class:`avocado.core.job.Job`. """ self.journal_initialized = False self.journal_path = "" self.journal = None self.journal_cursor = None self.config = config self.enabled = config.get("run.journal.enabled") def _init_journal(self, logdir): self.journal_path = os.path.join(logdir, JOURNAL_FILENAME) self.journal = sqlite3.connect(self.journal_path) self.journal_cursor = self.journal.cursor() for table in SCHEMA: # pylint: disable=C0206 res = self.journal_cursor.execute(f"PRAGMA table_info('{table}')") if res.fetchone() is None: self.journal_cursor.execute(SCHEMA[table]) self.journal.commit() def lazy_init_journal(self, state): # lazy init because we need the toplevel logdir for the job if not self.journal_initialized: self._init_journal(state["job_logdir"]) self._record_job_info(state) self.journal_initialized = True def _shutdown_journal(self): if self.journal_initialized: self.journal.close() def _record_job_info(self, state): res = self.journal_cursor.execute("SELECT unique_id FROM job_info") if res.fetchone() is None: sql = "INSERT INTO job_info (unique_id) VALUES (?)" self.journal_cursor.execute(sql, (state["job_unique_id"],)) self.journal.commit() def _record_status(self, state, action): sql = "INSERT INTO test_journal (tag, time, action, status) VALUES (?, ?, ?, ?)" # This shouldn't be required if action == "ENDED": status = state["status"] else: status = None self.journal_cursor.execute( sql, ( str(state["name"]), datetime.datetime(1, 1, 1).now().isoformat(), action, status, ), ) self.journal.commit() def pre_tests(self, job): pass def start_test(self, result, state): if not self.enabled: return self.lazy_init_journal(state) self._record_status(state, "STARTED") def test_progress(self, progress=False): pass def end_test(self, result, state): if not self.enabled: return self.lazy_init_journal(state) self._record_status(state, "ENDED") def post_tests(self, job): if not self.enabled: return self._shutdown_journal() class Journal(CLI): """ Test journal """ name = "journal" description = "Journal options for the 'run' subcommand" def configure(self, parser): run_subcommand_parser = parser.subcommands.choices.get("run", None) if run_subcommand_parser is None: return help_msg = ( "Records test status changes (for use with " "avocado-journal-replay and avocado-server)" ) settings.register_option( section="run.journal", key="enabled", default=False, key_type=bool, help_msg=help_msg, parser=run_subcommand_parser, long_arg="--journal", ) def run(self, config): pass avocado-113.0/avocado/plugins/json_variants.py000066400000000000000000000121031513561306000214560ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2018 # Authors: Amador Pahim import json import sys from avocado.core import exit_codes, varianter from avocado.core.output import LOG_UI from avocado.core.plugin_interfaces import CLI, Init, Varianter from avocado.core.settings import settings _NO_VARIANTS = -1 class JsonVariantsInit(Init): name = "json_variants" description = "JSON serialized based varianter initialization" def initialize(self): help_msg = "Load the Variants from a JSON serialized file" settings.register_option( section="json.variants", key="load", default=None, help_msg=help_msg ) class JsonVariantsCLI(CLI): """ Serialized based Varianter options """ name = "json variants" description = "JSON serialized based Varianter options for the 'run' subcommand" def configure(self, parser): for name in ("run", "variants"): # intentionally omitting "multiplex" subparser = parser.subcommands.choices.get(name, None) if subparser is None: continue sparser = subparser.add_argument_group( "JSON serialized based varianter options" ) settings.add_argparser_to_option( namespace="json.variants.load", parser=sparser, long_arg="--json-variants-load", allow_multiple=True, metavar="FILE", ) def run(self, config): pass class JsonVariants(Varianter): """ Processes the serialized file into variants """ name = "json variants" description = "JSON serialized based Varianter" variants = None def initialize(self, config): load_variants = config.get("json.variants.load") if load_variants is None: self.variants = _NO_VARIANTS return try: with open(load_variants, "r", encoding="utf-8") as var_file: self.variants = varianter.Varianter(state=json.load(var_file)) except IOError: LOG_UI.error( "JSON serialized file '%s' could not be found or is not readable", load_variants, ) if config.get("subcommand") == "run": sys.exit(exit_codes.AVOCADO_JOB_FAIL) else: sys.exit(exit_codes.AVOCADO_FAIL) def __iter__(self): if self.variants == _NO_VARIANTS: return elif self.variants is None: raise RuntimeError( "Iterating Varianter before initialization is not supported" ) for variant in self.variants.itertests(): yield variant def __len__(self): if self.variants == _NO_VARIANTS: return 0 elif self.variants is None: raise RuntimeError( "Calling Varianter __len__ before initialization is not supported" ) return len(self.variants) def to_str(self, summary, variants, **kwargs): """ Return human readable representation The summary/variants accepts verbosity where 0 means silent and maximum is up to the plugin. :param summary: How verbose summary to output (int) :param variants: How verbose list of variants to output (int) :param kwargs: Other free-form arguments :rtype: str """ if self.variants == _NO_VARIANTS: return "" out = [] verbose = variants > 1 if summary: # TODO: tree representation out.append("No tree representation for JSON serialized variants") if variants: out.append(f"JSON Serialized Variants ({len(self)}):") for variant in self: paths = ", ".join([x.path for x in variant["variant"]]) out.append( "%sVariant %s: %s" % ("\n" if verbose else "", variant["variant_id"], paths) ) if not verbose: continue env = set() for node in variant["variant"]: for key, value in node.environment.items(): origin = node.environment.origin[key].path env.add((f"{origin}:{key}", str(value))) if not env: return out fmt = " %%-%ds => %%s" % max( # pylint: disable=C0209 len(_[0]) for _ in env ) for record in sorted(env): out.append(fmt % record) return "\n".join(out) avocado-113.0/avocado/plugins/jsonresult.py000066400000000000000000000122741513561306000210170ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2014 # Authors: Ruda Moura # Cleber Rosa """ JSON output module. """ import json import os from avocado.core.output import LOG_UI from avocado.core.parser import FileOrStdoutAction from avocado.core.plugin_interfaces import CLI, Init, Result from avocado.core.settings import settings from avocado.core.test_id import TestID from avocado.utils import astring UNKNOWN = "" class JSONResult(Result): name = "json" description = "JSON result support" @staticmethod def _render(result): tests = [] for test in result.tests: fail_reason = test.get("fail_reason", UNKNOWN) if fail_reason is not None: fail_reason = astring.to_text(fail_reason) # Actually we are saving the TestID() there. test_id = test.get("name", UNKNOWN) if isinstance(test_id, TestID): name = f"{test_id.name}{test_id.str_variant}" tests.append( { "id": str(test_id), "name": str(name), "time_start": test.get("time_start", -1), "actual_time_start": test.get("actual_time_start", -1), "time_end": test.get("time_end", -1), "actual_time_end": test.get("actual_time_end", -1), "time_elapsed": test.get("time_elapsed", -1), "status": test.get("status", {}), "tags": test.get("tags") or {}, "whiteboard": test.get("whiteboard", UNKNOWN), "logdir": test.get("logdir", UNKNOWN), "logfile": test.get("logfile", UNKNOWN), "fail_reason": fail_reason, } ) content = { "job_id": result.job_unique_id, "debuglog": result.logfile, "tests": tests, "total": result.tests_total, "pass": result.passed, "errors": result.errors, "failures": result.failed, "skip": result.skipped, "cancel": result.cancelled, "warn": result.warned, "interrupt": result.interrupted, "time": result.tests_total_time, "start": str(result.job_start_date_time), } return json.dumps(content, sort_keys=True, indent=4, separators=(",", ": ")) def render(self, result, job): json_output = job.config.get("job.run.result.json.output") json_enabled = job.config.get("job.run.result.json.enabled") if not (json_enabled or json_output): return if not result.tests_total: return content = self._render(result) if json_enabled: json_path = os.path.join(job.logdir, "results.json") with open(json_path, "w", encoding="utf-8") as json_file: json_file.write(content) json_path = json_output if json_path is not None: if json_path == "-": LOG_UI.debug(content) else: with open(json_path, "w", encoding="utf-8") as json_file: json_file.write(content) class JSONInit(Init): name = "json" description = "JSON job result plugin initialization" def initialize(self): help_msg = ( "Enable JSON result format and write it to FILE. " 'Use "-" to redirect to the standard output.' ) settings.register_option( section="job.run.result.json", key="output", default=None, help_msg=help_msg ) help_msg = ( "Enables default JSON result in the job results " 'directory. File will be named "results.json".' ) settings.register_option( section="job.run.result.json", key="enabled", key_type=bool, default=True, help_msg=help_msg, ) class JSONCLI(CLI): """ JSON output """ name = "json" description = "JSON output options for 'run' command" def configure(self, parser): run_subcommand_parser = parser.subcommands.choices.get("run", None) if run_subcommand_parser is None: return settings.add_argparser_to_option( namespace="job.run.result.json.output", action=FileOrStdoutAction, metavar="FILE", parser=run_subcommand_parser, long_arg="--json", ) settings.add_argparser_to_option( namespace="job.run.result.json.enabled", parser=run_subcommand_parser, long_arg="--disable-json-job-result", ) def run(self, config): pass avocado-113.0/avocado/plugins/list.py000066400000000000000000000232021513561306000175530ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2013-2014 # Author: Lucas Meneghel Rodrigues # Author: Beraldo Leal import json import os from avocado.core import exit_codes, parser_common_args from avocado.core.output import LOG_UI, TERM_SUPPORT from avocado.core.plugin_interfaces import CLICmd from avocado.core.resolver import ReferenceResolutionResult from avocado.core.settings import settings from avocado.core.suite import TestSuite from avocado.core.tags import filter_tags_on_runnables from avocado.utils.astring import iter_tabular_output def _get_tags_as_string(tags): """Return a list of all tags but in a string format for output.""" tags_repr = [] for tag, values in tags.items(): if values: tags_repr.append(f"{tag}({','.join(values)})") else: tags_repr.append(tag) return ",".join(tags_repr) class List(CLICmd): """ Implements the avocado 'list' subcommand """ name = "list" description = "List available tests" @staticmethod def _prepare_matrix_for_display(matrix, verbose=False): colored_matrix = [] for item in matrix: kind = item[0] type_label = TERM_SUPPORT.healthy_str(kind) if verbose: colored_matrix.append( ( type_label, item[1], item[2], item[3], _get_tags_as_string(item[4] or {}), ) ) else: colored_matrix.append((type_label, item[1])) return colored_matrix def _display(self, suite, matrix): header = None verbose = suite.config.get("core.verbose") if verbose: header = ( TERM_SUPPORT.header_str("Type"), TERM_SUPPORT.header_str("Test"), TERM_SUPPORT.header_str("Uri"), TERM_SUPPORT.header_str("Resolver"), TERM_SUPPORT.header_str("Tag(s)"), ) matrix = self._prepare_matrix_for_display(matrix, verbose) for line in iter_tabular_output(matrix, header=header, strip=True): LOG_UI.debug(line) self._display_extra(suite, verbose) @staticmethod def _display_extra(suite, verbose=True): """Display extra data when in verbose mode.""" if not verbose: return if suite.resolutions: resolution_header = ( TERM_SUPPORT.header_str("Resolver"), TERM_SUPPORT.header_str("Reference"), TERM_SUPPORT.header_str("Info"), ) LOG_UI.info("") mapping = { ReferenceResolutionResult.SUCCESS: TERM_SUPPORT.healthy_str, ReferenceResolutionResult.CORRUPT: TERM_SUPPORT.warn_header_str, ReferenceResolutionResult.NOTFOUND: TERM_SUPPORT.fail_header_str, ReferenceResolutionResult.ERROR: TERM_SUPPORT.fail_header_str, } resolution_matrix = [] for r in suite.resolutions: decorator = mapping.get(r.result, TERM_SUPPORT.warn_header_str) if r.result == ReferenceResolutionResult.SUCCESS: continue resolution_matrix.append( (decorator(r.origin), r.reference, r.info or "") ) for line in iter_tabular_output( resolution_matrix, header=resolution_header, strip=True ): LOG_UI.info(line) LOG_UI.info("") LOG_UI.info("TEST TYPES SUMMARY") LOG_UI.info("==================") for key in sorted(suite.stats): LOG_UI.info("%s: %s", key, suite.stats[key]) if suite.tags_stats: LOG_UI.info("") LOG_UI.info("TEST TAGS SUMMARY") LOG_UI.info("=================") for key in sorted(suite.tags_stats): LOG_UI.info("%s: %s", key, suite.tags_stats[key]) @staticmethod def _get_resolution_matrix(suite): """Used for resolver. :returns: a list of tuples with either (kind, uri, tags) or (kind, uri) depending on whether verbose mode is on """ test_matrix = [] verbose = suite.config.get("core.verbose") tags = suite.config.get("filter.by_tags.tags") include_empty = suite.config.get("filter.by_tags.include_empty") include_empty_key = suite.config.get("filter.by_tags.include_empty_key") for resolution in suite.resolutions: if resolution.result != ReferenceResolutionResult.SUCCESS: continue if tags: runnables = filter_tags_on_runnables( [resolution], tags, include_empty, include_empty_key ) else: runnables = resolution.resolutions for runnable in runnables: if verbose: test_matrix.append( ( runnable.kind, runnable.identifier, runnable.uri, resolution.origin, runnable.tags or {}, ) ) else: test_matrix.append((runnable.kind, runnable.identifier)) return test_matrix @staticmethod def _save_to_json(matrix, filename, verbose=False): result = [] type_label_mapping = {} for line in matrix: try: test_type = type_label_mapping[line[0]] except KeyError: test_type = line[0] if verbose: tags = line[2] or {} result.append( { "Type": test_type, "Test": line[1], "Tags": {k: list(v or {}) for k, v in tags.items()}, } ) else: result.append({"Type": test_type, "Test": line[1]}) if filename == "-": LOG_UI.debug(json.dumps(result, indent=4)) else: with open(filename, "w", encoding="utf-8") as fp: json.dump(result, fp, indent=4) @staticmethod def save_recipes(suite, directory, matrix_len): fmt = f"%0{len(str(matrix_len))}u.json" index = 1 for resolution in suite.resolutions: if resolution.result == ReferenceResolutionResult.SUCCESS: for res in resolution.resolutions: res.write_json(os.path.join(directory, fmt % index)) index += 1 def configure(self, parser): """ Add the subparser for the list action. :param parser: The Avocado command line application parser :type parser: :class:`avocado.core.parser.ArgumentParser` """ parser = super().configure(parser) settings.add_argparser_to_option( namespace="resolver.references", nargs="*", metavar="TEST_REFERENCE", parser=parser, positional_arg=True, long_arg=None, allow_multiple=True, ) settings.add_argparser_to_option( namespace="resolver.run_executables", parser=parser, long_arg="--resolver-run-executables", allow_multiple=True, ) settings.add_argparser_to_option( namespace="resolver.exec_runnables_recipe.arguments", metavar="ARGS", parser=parser, long_arg="--resolver-exec-arguments", allow_multiple=True, ) help_msg = "Writes runnable recipe files to a directory." settings.register_option( section="list.recipes", key="write_to_directory", default=None, metavar="DIRECTORY", help_msg=help_msg, parser=parser, long_arg="--write-recipes-to-directory", ) help_msg = "Writes output to a json file." settings.register_option( section="list", key="write_to_json_file", default=None, metavar="JSON_FILE", help_msg=help_msg, parser=parser, long_arg="--json", ) parser_common_args.add_tag_filter_args(parser) def run(self, config): verbose = config.get("core.verbose") write_to_json_file = config.get("list.write_to_json_file") config["run.ignore_missing_references"] = True try: suite = TestSuite.from_config(config) matrix = self._get_resolution_matrix(suite) self._display(suite, matrix) directory = config.get("list.recipes.write_to_directory") if directory is not None: self.save_recipes(suite, directory, len(matrix)) if write_to_json_file: self._save_to_json(matrix, write_to_json_file, verbose) except KeyboardInterrupt: LOG_UI.error("Command interrupted by user...") return exit_codes.AVOCADO_FAIL avocado-113.0/avocado/plugins/plugins.py000066400000000000000000000100531513561306000202610ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2015 # Author: Cleber Rosa """ Plugins information plugin """ import itertools from avocado.core import dispatcher from avocado.core.output import LOG_UI from avocado.core.plugin_interfaces import CLICmd from avocado.core.settings import settings from avocado.core.spawners.common import SpawnCapabilities from avocado.utils import astring class Plugins(CLICmd): """ Plugins information """ name = "plugins" description = "Displays plugin information" def configure(self, parser): parser = super().configure(parser) help_msg = "Will list the plugins in execution order" settings.register_option( section="plugins", key="ordered_list", default=False, key_type=bool, action="store_true", help_msg=help_msg, parser=parser, long_arg="--ordered", short_arg="-o", ) help_msg = "Display the spawner plugins capabilities" settings.register_option( section="plugins", key="capabilities", default=False, key_type=bool, action="store_true", help_msg=help_msg, parser=parser, long_arg="--capabilities", ) def run(self, config): if config.get("plugins.capabilities"): spawners_dispatcher = dispatcher.SpawnerDispatcher(config, None) LOG_UI.info(f"{spawners_dispatcher.PLUGIN_DESCRIPTION}:") capabilities_matrix = [] for spawner in spawners_dispatcher.get_extentions_by_name(): capabilities_matrix.append((spawner.name, spawner.obj.description, "")) for capability in SpawnCapabilities: if capability in spawner.obj.CAPABILITIES: capabilities_matrix.append(("", f"{capability.value}:", "yes")) else: capabilities_matrix.append(("", f"{capability.value}:", "no")) capabilities_matrix.append(("", "", "")) if not capabilities_matrix: LOG_UI.debug("(No active spawner)") else: for line in astring.iter_tabular_output(capabilities_matrix): LOG_UI.debug(line) else: for plugin_dispatcher, config_needed, job_needed in itertools.chain( dispatcher.get_dispatchers("avocado.core.dispatcher"), dispatcher.get_dispatchers("avocado.core.resolver"), ): if not config_needed: plugins_active = plugin_dispatcher() elif config_needed and not job_needed: plugins_active = plugin_dispatcher(config) else: plugins_active = plugin_dispatcher(config, None) msg = f"{plugins_active.PLUGIN_DESCRIPTION}:" LOG_UI.info(msg) plugin_matrix = [] if config.get("plugins.ordered_list"): sorted_plugins = plugins_active.get_extentions_by_priority() else: sorted_plugins = plugins_active.get_extentions_by_name() for plugin in sorted_plugins: plugin_matrix.append((plugin.name, plugin.obj.description)) if not plugin_matrix: LOG_UI.debug("(No active plugin)") else: for line in astring.iter_tabular_output(plugin_matrix): LOG_UI.debug(line) LOG_UI.debug("") avocado-113.0/avocado/plugins/replay.py000066400000000000000000000057241513561306000201050ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat, Inc. 2020 # Authors: Cleber Rosa """Replay Job Plugin""" import json import sys from avocado.core import exit_codes, job, output from avocado.core.data_dir import get_job_results_dir from avocado.core.jobdata import retrieve_job_config from avocado.core.plugin_interfaces import CLICmd from avocado.core.settings import settings class Replay(CLICmd): """Implements the avocado 'replay' subcommand.""" name = "replay" description = "Runs a new job using a previous job as its configuration" def configure(self, parser): parser = super().configure(parser) help_msg = ( "Replays a job, identified by: complete or partial Job " 'ID, "latest" for the latest job, the job results path.' ) settings.register_option( section="job.replay", key="source_job_id", help_msg=help_msg, metavar="SOURCE_JOB_ID", default="latest", nargs="?", positional_arg=True, parser=parser, ) @staticmethod def _exit_fail(message): output.LOG_UI.error(message) sys.exit(exit_codes.AVOCADO_FAIL) @staticmethod def _retrieve_source_job_config(source_job_id): results_dir = get_job_results_dir(source_job_id) if not results_dir: msg = f"Could not find the results directory " f'for Job "{source_job_id}"' Replay._exit_fail(msg) try: return retrieve_job_config(results_dir) except OSError: msg = f"Could not open the {source_job_id} " f"Job configuration" Replay._exit_fail(msg) except json.decoder.JSONDecodeError: msg = f"Could not read a valid configuration " f'of Job "{source_job_id}"' Replay._exit_fail(msg) def run(self, config): namespace = "job.replay.source_job_id" source_job_id = config.get(namespace) source_job_config = self._retrieve_source_job_config(source_job_id) if hasattr(source_job_config, namespace): del source_job_config[namespace] # Flag that this is indeed a replayed job, which is impossible to # tell solely based on the job.replay.source_job_id given that it # has a default value of 'latest' for convenience reasons source_job_config["job.replay.enabled"] = True with job.Job.from_config(source_job_config) as job_instance: job_run = job_instance.run() return job_run avocado-113.0/avocado/plugins/requirement_cache.py000066400000000000000000000037271513561306000222750ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2022 # Author: Jan Richter import os from avocado.core import output from avocado.core.dependencies.requirements.cache.backends import sqlite from avocado.core.plugin_interfaces import Cache from avocado.utils import astring class RequirementCache(Cache): name = "requirement" description = "Provides requirement cache entries" def list(self): environments = sqlite.get_all_requirements() requirement_list = "" for enviroment_type, requirements in environments.items(): requirement_list += f"{enviroment_type}:\n" requirements_matrix = [ [ requirement["environment"], requirement["requirement_type"], requirement["requirement"], ] for requirement in requirements ] header = ( output.TERM_SUPPORT.header_str("Environment"), output.TERM_SUPPORT.header_str("Requirement_type"), output.TERM_SUPPORT.header_str("Requirement"), ) requirement_list += "\t" + "\t".join( astring.tabular_output( requirements_matrix, header=header, strip=True ).splitlines(True) ) requirement_list += "\n\n" return requirement_list def clear(self): if os.path.exists(sqlite.CACHE_DATABASE_PATH): os.remove(sqlite.CACHE_DATABASE_PATH) avocado-113.0/avocado/plugins/resolvers.py000066400000000000000000000231021513561306000206230ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2019 # Authors: Cleber Rosa """ Test resolver for builtin test types """ import json import os import re import shlex import subprocess from avocado.core.extension_manager import PluginPriority from avocado.core.nrunner.runnable import Runnable from avocado.core.plugin_interfaces import Init, Resolver from avocado.core.references import reference_split from avocado.core.resolver import ( ReferenceResolution, ReferenceResolutionResult, check_file, get_file_assets, ) from avocado.core.safeloader import find_avocado_tests, find_python_unittests from avocado.core.settings import settings class BaseExec: @staticmethod def check_exec(reference): criteria_check = check_file( reference, reference, suffix=None, type_name="executable file", access_check=os.R_OK | os.X_OK, access_name="executable", ) if criteria_check is not True: return criteria_check class ExecTestResolver(BaseExec, Resolver): name = "exec-test" description = "Test resolver for executable files to be handled as tests" priority = PluginPriority.VERY_LOW def resolve(self, reference): exec_criteria = self.check_exec(reference) if exec_criteria is not None: return exec_criteria runnable = Runnable("exec-test", reference, assets=get_file_assets(reference)) return ReferenceResolution( reference, ReferenceResolutionResult.SUCCESS, [runnable] ) def python_resolver(name, reference, find_tests): module_path, tests_filter = reference_split(reference) if tests_filter is not None: tests_filter = re.compile(tests_filter) criteria_check = check_file(module_path, reference) if criteria_check is not True: return criteria_check # disabled tests not needed here class_methods_info, _ = find_tests(module_path) runnables = [] for klass, methods_tags_depens in class_methods_info.items(): for method, tags, depens in methods_tags_depens: klass_method = f"{klass}.{method}" if tests_filter is not None and not tests_filter.search(klass_method): continue uri = f"{module_path}:{klass_method}" runnables.append( Runnable( name, uri=uri, tags=tags, dependencies=depens, assets=get_file_assets(module_path), ) ) if runnables: return ReferenceResolution( reference, ReferenceResolutionResult.SUCCESS, runnables ) return ReferenceResolution(reference, ReferenceResolutionResult.NOTFOUND) class PythonUnittestResolver(Resolver): name = "python-unittest" description = "Test resolver for Python Unittests" @staticmethod def _find_compat(module_path): """Used as compatibility for the :func:`python_resolver()` interface.""" return find_python_unittests(module_path), None def resolve(self, reference): return python_resolver( PythonUnittestResolver.name, reference, PythonUnittestResolver._find_compat ) class AvocadoInstrumentedResolver(Resolver): name = "avocado-instrumented" description = "Test resolver for Avocado Instrumented tests" priority = PluginPriority.HIGH def resolve(self, reference): return python_resolver( AvocadoInstrumentedResolver.name, reference, find_avocado_tests ) class TapResolver(BaseExec, Resolver): name = "tap" description = "Test resolver for executable files to be handled as TAP tests" priority = PluginPriority.LAST_RESORT def resolve(self, reference): exec_criteria = self.check_exec(reference) if exec_criteria is not None: return exec_criteria runnable = Runnable("tap", reference, assets=get_file_assets(reference)) return ReferenceResolution( reference, ReferenceResolutionResult.SUCCESS, [runnable] ) class RunnableRecipeResolver(Resolver): name = "runnable-recipe" description = "Test resolver for JSON runnable recipes" def resolve(self, reference): criteria_check = check_file( reference, reference, suffix=".json", type_name="JSON file" ) if criteria_check is not True: return criteria_check runnable = Runnable.from_recipe(reference) return ReferenceResolution( reference, ReferenceResolutionResult.SUCCESS, [runnable] ) class RunnablesRecipeResolver(Resolver): name = "runnables-recipe" description = "Test resolver for multiple runnables in a JSON recipe file" @staticmethod def _validate_and_load_runnables(reference): with open(reference, "r", encoding="utf-8") as json_file: runnables = json.load(json_file) if not ( isinstance(runnables, list) and all([isinstance(r, dict) for r in runnables]) ): return ReferenceResolution( reference, ReferenceResolutionResult.NOTFOUND, info="File {reference} does not look like a runnables recipe JSON file", ) return ReferenceResolution( reference, ReferenceResolutionResult.SUCCESS, [Runnable.from_dict(r) for r in runnables], ) def resolve(self, reference): criteria_check = check_file( reference, reference, suffix=".json", type_name="JSON file" ) if criteria_check is not True: return criteria_check return self._validate_and_load_runnables(reference) class ExecRunnablesRecipeInit(Init): name = "exec-runnables-recipe" description = 'Configuration for resolver plugin "exec-runnables-recipe" plugin' def initialize(self): help_msg = ( 'Whether resolvers (such as "exec-runnables-recipe") should ' "execute files given as test references that have executable " "permissions. This is disabled by default due to security " "implications of running executables that may not be trusted." ) settings.register_option( section="resolver", key="run_executables", key_type=bool, default=False, help_msg=help_msg, ) help_msg = ( "Command line options (space separated) that will be added " "to the executable when executing it as a producer of " "runnables-recipe JSON content." ) settings.register_option( section="resolver.exec_runnables_recipe", key="arguments", key_type=str, default="", help_msg=help_msg, ) class ExecRunnablesRecipeResolver(BaseExec, Resolver): name = "exec-runnables-recipe" description = "Test resolver for executables that output JSON runnable recipes" priority = PluginPriority.LOW def resolve(self, reference): if not self.config.get("resolver.run_executables"): return ReferenceResolution( reference, ReferenceResolutionResult.NOTFOUND, info=( "Running executables is not enabled. Refer to " '"resolver.run_executables" configuration option' ), ) exec_criteria = self.check_exec(reference) if exec_criteria is not None: return exec_criteria args = self.config.get("resolver.exec_runnables_recipe.arguments") if args: cmd = [reference] + shlex.split(args) else: cmd = reference try: process = subprocess.Popen( cmd, stdin=subprocess.DEVNULL, stdout=subprocess.PIPE, stderr=subprocess.PIPE, ) except (FileNotFoundError, PermissionError) as exc: return ReferenceResolution( reference, ReferenceResolutionResult.NOTFOUND, info=(f'Failure while running running executable "{reference}": {exc}'), ) content, _ = process.communicate() try: runnables = json.loads(content) except json.JSONDecodeError: return ReferenceResolution( reference, ReferenceResolutionResult.NOTFOUND, info=f'Content generated by running executable "{reference}" is not JSON', ) if not ( isinstance(runnables, list) and all([isinstance(r, dict) for r in runnables]) ): return ReferenceResolution( reference, ReferenceResolutionResult.NOTFOUND, info=f"Content generated by running executable {reference} does not look like a runnables recipe JSON content", ) return ReferenceResolution( reference, ReferenceResolutionResult.SUCCESS, [Runnable.from_dict(r) for r in runnables], ) avocado-113.0/avocado/plugins/run.py000066400000000000000000000237021513561306000174110ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2013-2014 # Author: Ruda Moura """ Base Test Runner Plugins. """ import argparse import sys from avocado.core import exit_codes, job, parser_common_args from avocado.core.output import LOG_UI from avocado.core.plugin_interfaces import CLICmd, Init from avocado.core.settings import settings from avocado.core.suite import TestSuite, TestSuiteError class RunInit(Init): name = "run" description = "Initializes the run options" def initialize(self): help_msg = "Defines the order of iterating through test suite and test variants" settings.register_option( section="run", key="execution_order", choices=("tests-per-variant", "variants-per-test"), default="variants-per-test", help_msg=help_msg, ) class Run(CLICmd): """ Implements the avocado 'run' subcommand """ name = "run" description = "Runs one or more tests (native test, test alias, binary or script)" @staticmethod def _test_parameter(string): param_name_value = string.split("=", 1) if len(param_name_value) < 2: msg = ( f'Invalid --test-parameter option: "{string}". ' f'Valid option must be a "NAME=VALUE" like expression' ) raise argparse.ArgumentTypeError(msg) return param_name_value def configure(self, parser): """ Add the subparser for the run action. :param parser: Main test runner parser. """ parser = super().configure(parser) settings.add_argparser_to_option( namespace="resolver.references", nargs="*", metavar="TEST_REFERENCE", parser=parser, positional_arg=True, long_arg=None, allow_multiple=True, ) help_msg = ( "Parameter name and value to pass to all tests. This is " "only applicable when not using a varianter plugin. " "This option format must be given in the NAME=VALUE " "format, and may be given any number of times, or per " "parameter." ) settings.register_option( section="run", key="test_parameters", action="append", default=[], key_type=self._test_parameter, metavar="NAME_VALUE", help_msg=help_msg, parser=parser, long_arg="--test-parameter", short_arg="-p", ) settings.add_argparser_to_option( namespace="run.suite_runner", parser=parser, long_arg="--suite-runner", metavar="SUITE_RUNNER", ) help_msg = "Instead of running the test only list them and log their params." settings.register_option( section="run.dry_run", key="enabled", default=False, key_type=bool, help_msg=help_msg, parser=parser, short_arg="-d", long_arg="--dry-run", ) help_msg = "Do not automatically clean up temporary directories used by dry-run" settings.register_option( section="run.dry_run", key="no_cleanup", help_msg=help_msg, default=False, key_type=bool, parser=parser, long_arg="--dry-run-no-cleanup", ) help_msg = ( "Forces the use of a particular job ID. Used internally " "when interacting with an avocado server. You should not " "use this option unless you know exactly what you're " "doing" ) settings.register_option( section="run", key="unique_job_id", default=None, help_msg=help_msg, parser=parser, long_arg="--force-job-id", metavar="UNIQUE_JOB_ID", ) help_msg = "Forces to use of an alternate job results directory." settings.register_option( section="run", key="results_dir", default=None, metavar="DIRECTORY", help_msg=help_msg, parser=parser, long_arg="--job-results-dir", ) help_msg = ( "Categorizes this within a directory with the same name, " "by creating a link to the job result directory" ) settings.register_option( section="run", key="job_category", help_msg=help_msg, parser=parser, default=None, metavar="CATEGORY", long_arg="--job-category", ) help_msg = ( "Path to the json file with list of dependencies needed for each test." ) settings.register_option( section="job.run", key="dependency", help_msg=help_msg, parser=parser, default=None, metavar="FILE", long_arg="--job-dependency", ) settings.add_argparser_to_option( namespace="job.run.timeout", metavar="SECONDS", parser=parser, long_arg="--job-timeout", ) help_msg = "Enable the job interruption on first failed test." settings.register_option( section="run", key="failfast", default=False, key_type=bool, action="store_true", help_msg=help_msg, parser=parser, long_arg="--failfast", ) help_msg = "Keep job temporary files (useful for avocado debugging)." settings.register_option( section="run", key="keep_tmp", default=False, key_type=bool, action="store_true", help_msg=help_msg, parser=parser, long_arg="--keep-tmp", ) help_msg = ( "Force the job execution, even if some of the test " "references are not resolved to tests." ) settings.register_option( section="run", key="ignore_missing_references", default=False, key_type=bool, help_msg=help_msg, parser=parser, long_arg="--ignore-missing-references", ) help_msg = "Disable sysinfo collection (like hardware details, profiles, etc)." settings.add_argparser_to_option( namespace="sysinfo.collect.enabled", parser=parser, action="store_false", long_arg="--disable-sysinfo", help_msg=help_msg, ) settings.add_argparser_to_option( "run.execution_order", parser=parser, long_arg="--execution-order" ) parser.output = parser.add_argument_group("output and result format") settings.add_argparser_to_option( "job.run.store_logging_stream", parser=parser.output, long_arg="--store-logging-stream", metavar="LOGGING_STREAM", argparse_type=lambda x: set(x.split(",")), ) help_msg = ( "Logs the possible data directories for each test. This " "is helpful when writing new tests and not being sure " 'where to put data files. Look for "Test data ' 'directories" in your test log' ) settings.register_option( section="run", key="log_test_data_directories", default=False, key_type=bool, help_msg=help_msg, parser=parser, long_arg="--log-test-data-directories", ) settings.add_argparser_to_option( namespace="resolver.run_executables", parser=parser, long_arg="--resolver-run-executables", allow_multiple=True, ) settings.add_argparser_to_option( namespace="resolver.exec_runnables_recipe.arguments", metavar="ARGS", parser=parser, long_arg="--resolver-exec-arguments", allow_multiple=True, ) parser_common_args.add_tag_filter_args(parser) def run(self, config): """ Runs an Avocado Job with a TestSuite created from test references :param config: Configuration received from command line parser and possibly other sources. :type config: dict """ unique_job_id = config.get("run.unique_job_id") if unique_job_id is not None: try: int(unique_job_id, 16) if len(unique_job_id) != 40: raise ValueError except ValueError: LOG_UI.error("Unique Job ID needs to be a 40 digit hex number") sys.exit(exit_codes.AVOCADO_FAIL) try: suite = TestSuite.from_config(config, name="") if suite.size == 0: msg = ( "Suite is empty. There is no tests to run. This usually " "happens when you pass --ignore-missing-references and " "there is no more references to process." ) LOG_UI.warning(msg) sys.exit(exit_codes.AVOCADO_FAIL) except TestSuiteError as err: LOG_UI.error(err) sys.exit(exit_codes.AVOCADO_JOB_FAIL) with job.Job(config, [suite]) as job_instance: return job_instance.run() avocado-113.0/avocado/plugins/runner_nrunner.py000066400000000000000000000332071513561306000216660ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2019-2020 # Authors: Cleber Rosa """ nrunner based implementation of job compliant runner """ import asyncio import multiprocessing import os import platform import random import tempfile from avocado.core.dispatcher import SpawnerDispatcher from avocado.core.exceptions import JobError, JobFailFast from avocado.core.messages import MessageHandler from avocado.core.nrunner.runner import check_runnables_runner_requirements from avocado.core.output import LOG_JOB from avocado.core.plugin_interfaces import CLI, Init, SuiteRunner from avocado.core.settings import settings from avocado.core.status.repo import StatusRepo from avocado.core.status.server import StatusServer from avocado.core.task.runtime import RuntimeTaskGraph from avocado.core.task.statemachine import TaskStateMachine, Worker DEFAULT_SERVER_URI = "127.0.0.1:8888" class RunnerInit(Init): name = "nrunner" description = "nrunner initialization" def initialize(self): section = "run" help_msg = "Shuffle the tasks to be executed" settings.register_option( section=section, key="shuffle", default=False, help_msg=help_msg, key_type=bool, ) help_msg = ( "If the status server should automatically choose " 'a "status_server_listen" and "status_server_uri" ' "configuration. Default is to auto configure a " "status server." ) settings.register_option( section=section, key="status_server_auto", default=True, key_type=bool, help_msg=help_msg, ) help_msg = ( 'URI where status server will listen on. Usually a "HOST:PORT" ' 'string. This is only effective if "status_server_auto" is disabled. ' 'If "status_server_uri" is not set, the value from "status_server_listen " ' "will be used." ) settings.register_option( section=section, key="status_server_listen", default=DEFAULT_SERVER_URI, metavar="HOST:PORT", help_msg=help_msg, ) help_msg = ( "URI for connecting to the status server, usually " 'a "HOST:PORT" string. Use this if your status server ' "is in another host, or different port. This is only " 'effective if "status_server_auto" is disabled. ' 'If "status_server_listen" is not set, the value from "status_server_uri" ' "will be used." ) settings.register_option( section=section, key="status_server_uri", default=DEFAULT_SERVER_URI, metavar="HOST:PORT", help_msg=help_msg, ) help_msg = ( "Buffer size that status server uses. This should " "generally not be a concern to most users, but " "it can be tuned in case a runner generates very large " "status messages, which is common if a test generates a " "lot of output. Default is 33554432 (32MiB)" ) settings.register_option( section=section, key="status_server_buffer_size", key_type=int, default=2**25, help_msg=help_msg, ) help_msg = ( "Number of maximum number tasks running in parallel. You " "can disable parallel execution by setting this to 1. " "Defaults to the amount of CPUs on this machine." ) settings.register_option( section=section, key="max_parallel_tasks", default=multiprocessing.cpu_count(), key_type=int, help_msg=help_msg, ) help_msg = ( "Spawn tasks in a specific spawner. Available spawners: " "'process' and 'podman'" ) settings.register_option( section=section, key="spawner", default="process", help_msg=help_msg ) help_msg = "The amount of time a test has to complete in seconds." settings.register_option( section="task.timeout", key="running", default=None, key_type=int, help_msg=help_msg, ) class RunnerCLI(CLI): name = "nrunner" description = 'nrunner command line options for "run"' def configure(self, parser): super().configure(parser) parser = parser.subcommands.choices.get("run", None) if parser is None: return parser = parser.add_argument_group("nrunner specific options") settings.add_argparser_to_option( namespace="run.shuffle", parser=parser, long_arg="--shuffle", action="store_true", ) settings.add_argparser_to_option( namespace="run.status_server_auto", parser=parser, long_arg="--status-server-disable-auto", action="store_false", ) settings.add_argparser_to_option( namespace="run.status_server_listen", parser=parser, long_arg="--status-server-listen", metavar="HOST_PORT", ) settings.add_argparser_to_option( namespace="run.status_server_uri", parser=parser, long_arg="--status-server-uri", metavar="HOST_PORT", ) settings.add_argparser_to_option( namespace="run.max_parallel_tasks", parser=parser, long_arg="--max-parallel-tasks", metavar="NUMBER_OF_TASKS", ) settings.add_argparser_to_option( namespace="run.spawner", parser=parser, long_arg="--spawner", metavar="SPAWNER", ) def run(self, config): pass class Runner(SuiteRunner): name = "nrunner" description = "nrunner based implementation of job compliant runner" def __init__(self): super().__init__() self.status_server_dir = None def _determine_status_server(self, test_suite, config_key): if test_suite.config.get("run.status_server_auto"): # no UNIX domain sockets on Windows if platform.system() != "Windows": if self.status_server_dir is None: self.status_server_dir = tempfile.TemporaryDirectory( prefix="avocado_" ) return os.path.join(self.status_server_dir.name, ".status_server.sock") return test_suite.config.get(config_key) def _sync_status_server_urls(self, config): server_listen = config.get("run.status_server_listen") server_uri = config.get("run.status_server_uri") if not config.get("run.status_server_auto"): if ( server_uri is not DEFAULT_SERVER_URI and server_listen is DEFAULT_SERVER_URI ): config["run.status_server_listen"] = server_uri if ( server_uri is DEFAULT_SERVER_URI and server_listen is not DEFAULT_SERVER_URI ): config["run.status_server_uri"] = server_listen def _create_status_server(self, test_suite, job): self._sync_status_server_urls(test_suite.config) listen = self._determine_status_server(test_suite, "run.status_server_listen") # pylint: disable=W0201 self.status_repo = StatusRepo(job.unique_id) # pylint: disable=W0201 self.status_server = StatusServer(listen, self.status_repo) async def _update_status(self, job): message_handler = MessageHandler() while True: try: (_, task_id, _, index) = self.status_repo.status_journal_summary_pop() except IndexError: await asyncio.sleep(0.05) continue message = self.status_repo.get_task_data(task_id, index) task = self.tsm.tasks_by_id.get(task_id) message_handler.process_message(message, task, job) @staticmethod def _abort_if_missing_runners(runnables): if runnables: missing_kinds = set([runnable.kind for runnable in runnables]) msg = ( f"Could not find runners for runnable(s) of kind(s): " f"{', '.join(missing_kinds)}" ) raise JobError(msg) def run_suite(self, job, test_suite): summary = set() if not test_suite.enabled: job.interrupted_reason = f"Suite {test_suite.name} is disabled." return summary spawner_name = test_suite.config.get("run.spawner") spawner = SpawnerDispatcher(test_suite.config, job)[spawner_name].obj if not spawner.is_operational(): suite_name = f" {test_suite.name}" if test_suite.name else "" msg = f'Spawner "{spawner_name}" is not operational, aborting execution of suite {suite_name}. Please check the logs for more information.' LOG_JOB.error(msg) job.interrupted_reason = msg summary.add("INTERRUPTED") return summary test_suite.tests, missing_requirements = check_runnables_runner_requirements( test_suite.tests ) self._abort_if_missing_runners(missing_requirements) job.result.tests_total = len(test_suite.tests) self._create_status_server(test_suite, job) graph = RuntimeTaskGraph( test_suite.tests, test_suite.name, self._determine_status_server(test_suite, "run.status_server_uri"), job.unique_id, job.test_results_path, test_suite.config, ) # pylint: disable=W0201 self.runtime_tasks = graph.get_tasks_in_topological_order() # Start the status server asyncio.set_event_loop(asyncio.new_event_loop()) asyncio.ensure_future(self.status_server.serve_forever()) if test_suite.config.get("run.shuffle"): random.shuffle(self.runtime_tasks) test_ids = [ rt.task.identifier for rt in self.runtime_tasks if rt.task.category == "test" ] self.tsm = TaskStateMachine(self.runtime_tasks, self.status_repo) max_running = min( test_suite.config.get("run.max_parallel_tasks"), len(self.runtime_tasks) ) timeout = test_suite.config.get("task.timeout.running") failfast = test_suite.config.get("run.failfast") workers = [ Worker( state_machine=self.tsm, spawner=spawner, max_running=max_running, task_timeout=timeout, failfast=failfast, ).run() for _ in range(max_running) ] asyncio.ensure_future(self._update_status(job)) loop = asyncio.get_event_loop() try: try: loop.run_until_complete( asyncio.wait_for( asyncio.shield(asyncio.gather(*workers)), job.timeout or None ) ) except asyncio.TimeoutError: terminate_worker = Worker( state_machine=self.tsm, spawner=spawner, max_running=max_running, task_timeout=timeout, failfast=failfast, ) loop.run_until_complete( asyncio.wait_for(terminate_worker.terminate_tasks_timeout(), None) ) raise except KeyboardInterrupt: terminate_worker = Worker( state_machine=self.tsm, spawner=spawner, max_running=max_running, task_timeout=timeout, failfast=failfast, ) loop.run_until_complete( asyncio.wait_for( terminate_worker.terminate_tasks_interrupted(), None ) ) raise except (KeyboardInterrupt, asyncio.TimeoutError, JobFailFast) as ex: LOG_JOB.info(str(ex)) job.interrupted_reason = str(ex) summary.add("INTERRUPTED") # Wait until all messages may have been processed by the # status_updater. This should be replaced by a mechanism # that only waits if there are missing status messages to # be processed, and, only for a given amount of time. # Tests with non received status will always show as SKIP # because of result reconciliation. loop.run_until_complete(asyncio.sleep(0.05)) job.result.end_tests() self.status_server.close() if self.status_server_dir is not None: self.status_server_dir.cleanup() # Update the overall summary with found test statuses, which will # determine the Avocado command line exit status summary.update( [ status.upper() for status in self.status_repo.get_result_set_for_tasks(test_ids) ] ) return summary avocado-113.0/avocado/plugins/runners/000077500000000000000000000000001513561306000177235ustar00rootroot00000000000000avocado-113.0/avocado/plugins/runners/__init__.py000066400000000000000000000000001513561306000220220ustar00rootroot00000000000000avocado-113.0/avocado/plugins/runners/asset.py000066400000000000000000000100061513561306000214110ustar00rootroot00000000000000import sys import time from multiprocessing import Process, SimpleQueue, set_start_method from avocado.core.nrunner.app import BaseRunnerApp from avocado.core.nrunner.runner import RUNNER_RUN_STATUS_INTERVAL, BaseRunner from avocado.core.settings import settings from avocado.utils import data_structures from avocado.utils.asset import Asset class AssetRunner(BaseRunner): """Runner for dependencies of type package This runner handles the fetch of files using the Avocado Assets utility. Runnable attributes usage: * kind: 'asset' * uri: not used * args: not used * kwargs: - name: the file name or uri (required) - asset_hash: hash of the file (optional) - algorithm: hash algorithm (optional) - locations: location(s) where the file can be fetched from (optional) - expire: time in seconds for the asset to expire (optional) """ name = "asset" description = "Runner for dependencies of type package" CONFIGURATION_USED = ["datadir.paths.cache_dirs"] @staticmethod def _fetch_asset(name, asset_hash, algorithm, locations, cache_dirs, expire, queue): asset_manager = Asset( name, asset_hash, algorithm, locations, cache_dirs, expire ) result = "pass" stdout = "" stderr = "" try: asset_file = asset_manager.fetch() stdout = f"File fetched at {asset_file}" except OSError as exc: result = "error" stderr = str(exc) output = {"result": result, "stdout": stdout, "stderr": stderr} queue.put(output) def run(self, runnable): # pylint: disable=W0201 self.runnable = runnable yield self.prepare_status("started") name = self.runnable.kwargs.get("name") # if name was passed correctly, run the Avocado Asset utility if name is not None: asset_hash = self.runnable.kwargs.get("asset_hash") algorithm = self.runnable.kwargs.get("algorithm") locations = self.runnable.kwargs.get("locations") expire = self.runnable.kwargs.get("expire") if expire is not None: expire = data_structures.time_to_seconds(str(expire)) cache_dirs = self.runnable.config.get("datadir.paths.cache_dirs") if cache_dirs is None: cache_dirs = settings.as_dict().get("datadir.paths.cache_dirs") # let's spawn it to another process to be able to update the # status messages and avoid the Asset to lock this process queue = SimpleQueue() process = Process( target=self._fetch_asset, args=( name, asset_hash, algorithm, locations, cache_dirs, expire, queue, ), ) process.start() while queue.empty(): time.sleep(RUNNER_RUN_STATUS_INTERVAL) yield self.prepare_status("running") output = queue.get() result = output["result"] stdout = output["stdout"] stderr = output["stderr"] else: # Otherwise, log the missing package name result = "error" stdout = "" stderr = 'At least name should be passed as kwargs using name="uri".' yield self.prepare_status("running", {"type": "stdout", "log": stdout.encode()}) yield self.prepare_status("running", {"type": "stderr", "log": stderr.encode()}) yield self.prepare_status("finished", {"result": result}) class RunnerApp(BaseRunnerApp): PROG_NAME = "avocado-runner-asset" PROG_DESCRIPTION = "nrunner application for dependencies of type asset" RUNNABLE_KINDS_CAPABLE = ["asset"] def main(): if sys.platform == "darwin": set_start_method("fork") app = RunnerApp(print) app.run() if __name__ == "__main__": main() avocado-113.0/avocado/plugins/runners/avocado_instrumented.py000066400000000000000000000147501513561306000245210ustar00rootroot00000000000000import multiprocessing import os import signal import sys import tempfile import time import traceback from avocado.core.exceptions import TestInterrupt from avocado.core.nrunner.app import BaseRunnerApp from avocado.core.nrunner.runner import RUNNER_RUN_CHECK_INTERVAL, BaseRunner from avocado.core.test import TestID from avocado.core.tree import TreeNodeEnvOnly from avocado.core.utils import loader, messages from avocado.core.varianter import is_empty_variant from avocado.utils.deprecation import log_deprecation class AvocadoInstrumentedTestRunner(BaseRunner): """ Runner for avocado-instrumented tests Runnable attributes usage: * uri: path to a test file, combined with an Avocado.Test inherited class name and method. The test file path and class and method names should be separated by a ":". One example of a valid uri is "mytest.py:Class.test_method". * args: not used """ name = "avocado-instrumented" description = "Runner for avocado-instrumented tests" CONFIGURATION_USED = [ "run.test_parameters", "datadir.paths.cache_dirs", "core.show", "job.output.loglevel", "job.run.store_logging_stream", ] @staticmethod def signal_handler(signum, frame): # pylint: disable=W0613 if signum == signal.SIGTERM.value: raise TestInterrupt("Test interrupted: Timeout reached") @staticmethod def _create_params(runnable): """Create params for the test""" if runnable.variant is None: return None # rebuild the variant tree variant_tree_nodes = [ TreeNodeEnvOnly(path, env) for path, env in runnable.variant["variant"] ] if not is_empty_variant(variant_tree_nodes): tree_nodes = variant_tree_nodes paths = runnable.variant["paths"] return tree_nodes, paths @staticmethod def _run_avocado(runnable, queue): def load_and_run_test(test_factory): instance = loader.load_test(test_factory) early_state = instance.get_state() early_state["type"] = "early_state" queue.put(early_state) log_deprecation.flush() instance.run_avocado() return instance.get_state() try: # This assumes that a proper resolution (see resolver module) # was performed, and that a URI contains: # 1) path to python module # 2) class # 3) method # # To be defined: if the resolution uri should be composed like # this, or broken down and stored into other data fields signal.signal(signal.SIGTERM, AvocadoInstrumentedTestRunner.signal_handler) module_path, klass_method = runnable.uri.split(":", 1) klass, method = klass_method.split(".", 1) params = AvocadoInstrumentedTestRunner._create_params(runnable) result_dir = runnable.output_dir or tempfile.mkdtemp(prefix=".avocado-task") test_factory = [ klass, { "name": TestID(1, runnable.uri, runnable.variant), "methodName": method, "config": runnable.config, "modulePath": module_path, "params": params, "tags": runnable.tags, "run.results_dir": result_dir, }, ] messages.start_logging(runnable.config, queue) # running the actual test if "COVERAGE_RUN" in os.environ: from coverage import Coverage coverage = Coverage(data_suffix=True) with coverage.collect(): state = load_and_run_test(test_factory) coverage.save() else: state = load_and_run_test(test_factory) fail_reason = state.get("fail_reason") queue.put(messages.WhiteboardMessage.get(state["whiteboard"])) queue.put( messages.FinishedMessage.get( state["status"].lower(), fail_reason=fail_reason, class_name=klass, fail_class=state.get("fail_class"), traceback=state.get("traceback"), ) ) except Exception as e: queue.put(messages.StderrMessage.get(traceback.format_exc())) queue.put( messages.FinishedMessage.get( "error", fail_reason=str(e), fail_class=e.__class__.__name__, traceback=traceback.format_exc(), ) ) @staticmethod def _monitor(queue): while True: time.sleep(RUNNER_RUN_CHECK_INTERVAL) if queue.empty(): yield messages.RunningMessage.get() else: message = queue.get() if message.get("type") != "early_state": yield message if message.get("status") == "finished": break def run(self, runnable): # pylint: disable=W0201 signal.signal(signal.SIGTERM, AvocadoInstrumentedTestRunner.signal_handler) self.runnable = runnable yield messages.StartedMessage.get() try: queue = multiprocessing.SimpleQueue() process = multiprocessing.Process( target=self._run_avocado, args=(self.runnable, queue) ) process.start() for message in self._monitor(queue): yield message except TestInterrupt: process.terminate() for message in self._monitor(queue): yield message except Exception as e: yield messages.StderrMessage.get(traceback.format_exc()) yield messages.FinishedMessage.get( "error", fail_reason=str(e), fail_class=e.__class__.__name__, traceback=traceback.format_exc(), ) class RunnerApp(BaseRunnerApp): PROG_NAME = "avocado-runner-avocado-instrumented" PROG_DESCRIPTION = "nrunner application for avocado-instrumented tests" RUNNABLE_KINDS_CAPABLE = ["avocado-instrumented"] def main(): if sys.platform == "darwin": multiprocessing.set_start_method("fork") app = RunnerApp(print) app.run() if __name__ == "__main__": main() avocado-113.0/avocado/plugins/runners/dry_run.py000066400000000000000000000016071513561306000217630ustar00rootroot00000000000000from avocado.core.nrunner.app import BaseRunnerApp from avocado.core.nrunner.runner import BaseRunner class DryRunRunner(BaseRunner): """ Runner for --dry-run. It performs no action before reporting FINISHED status with cancel result. Runnable attributes usage: * uri: not used * args: not used """ name = "dry-run" description = "Runner for --dry-run" def run(self, runnable): yield self.prepare_status("started") yield self.prepare_status( "finished", {"result": "cancel", "fail_reason": "Test cancelled due to " "--dry-run"}, ) class RunnerApp(BaseRunnerApp): PROG_NAME = "avocado-runner-dry-run" PROG_DESCRIPTION = "nrunner application for dry-run tests" RUNNABLE_KINDS_CAPABLE = ["dry-run"] def main(): app = RunnerApp(print) app.run() if __name__ == "__main__": main() avocado-113.0/avocado/plugins/runners/exec_test.py000066400000000000000000000175021513561306000222650ustar00rootroot00000000000000import multiprocessing import os import shutil import subprocess import sys import tempfile import pkg_resources from avocado.core.nrunner.app import BaseRunnerApp from avocado.core.nrunner.runner import BaseRunner class ExecTestRunner(BaseRunner): """ Runner for standalone executables treated as tests This is similar in concept to the Avocado "SIMPLE" test type, in which an executable returning 0 means that a test passed, and anything else means that a test failed. Runnable attributes usage: * uri: path to a binary to be executed as another process * args: arguments to be given on the command line to the binary given by path * kwargs: key=val to be set as environment variables to the process """ name = "exec-test" description = "Runner for standalone executables treated as tests" CONFIGURATION_USED = [ "run.keep_tmp", "runner.exectest.exitcodes.skip", "runner.exectest.clear_env", ] def _process_final_status( self, process, runnable, stdout=None, stderr=None ): # pylint: disable=W0613 # Since Runners are standalone, and could be executed on a remote # machine in an "isolated" way, there is no way to assume a default # value, at this moment. skip_codes = runnable.config.get("runner.exectest.exitcodes.skip", []) final_status = {} if skip_codes is None: skip_codes = [] if process.returncode in skip_codes: final_status["result"] = "skip" elif process.returncode == 0: final_status["result"] = "pass" else: final_status["result"] = "fail" final_status["returncode"] = process.returncode return self.prepare_status("finished", final_status) def _cleanup(self, runnable): """Cleanup method for the exec-test tests""" # cleanup temporary directories workdir = runnable.kwargs.get("AVOCADO_TEST_WORKDIR") if ( workdir is not None and runnable.config.get("run.keep_tmp") is not None and os.path.exists(workdir) ): shutil.rmtree(workdir) def _create_params(self, runnable): """Create params for the test""" if runnable.variant is None: return {} params = dict( [(str(key), str(val)) for _, key, val in runnable.variant["variant"][0][1]] ) return params @staticmethod def _get_avocado_version(): """Return the Avocado package version, if installed""" version = "unknown.unknown" try: version = pkg_resources.get_distribution("avocado-framework").version except pkg_resources.DistributionNotFound: pass return version def _get_env_variables(self, runnable): """Get the default AVOCADO_* environment variables These variables are available to the test environment during the test execution. """ # create the temporary work dir workdir = tempfile.mkdtemp(prefix=".avocado-workdir-") # create the avocado environment variable dictionary avocado_test_env_variables = { "AVOCADO_VERSION": self._get_avocado_version(), "AVOCADO_TEST_WORKDIR": workdir, "AVOCADO_TEST_BASEDIR": os.path.dirname(os.path.abspath(runnable.uri)), } if runnable.output_dir: avocado_test_env_variables["AVOCADO_TEST_LOGDIR"] = runnable.output_dir avocado_test_env_variables["AVOCADO_TEST_OUTPUTDIR"] = runnable.output_dir avocado_test_env_variables["AVOCADO_TEST_LOGFILE"] = os.path.join( runnable.output_dir, "debug.log" ) return avocado_test_env_variables @staticmethod def _is_uri_a_file_on_cwd(uri): if ( uri is not None and os.path.basename(uri) == uri and os.access(uri, os.R_OK | os.X_OK) ): return True return False def _get_env(self, runnable): clear_env = runnable.config.get("runner.exectest.clear_env", None) if clear_env not in ["all", "system", None]: raise ValueError( f"The `runner.exectest.clear_env` value {clear_env} is not supported." ) if clear_env in ["all", "system"]: env = {} else: env = dict(os.environ) if runnable.kwargs: env.update(runnable.kwargs) for key, value in runnable.kwargs.items(): if value is None: del env[key] # set default Avocado environment variables if running on a valid Task if runnable.uri is not None and clear_env != "all": avocado_test_env_variables = self._get_env_variables(runnable) # save environment variables for further cleanup runnable.kwargs.update(avocado_test_env_variables) if env is None: env = avocado_test_env_variables else: env.update(avocado_test_env_variables) params = self._create_params(runnable) if params: env.update(params) if env and "PATH" not in env: env["PATH"] = os.environ.get("PATH") # Support for running executable tests in the current working directory if self._is_uri_a_file_on_cwd(runnable.uri): env["PATH"] += f":{os.getcwd()}" return env def _run_proc(self, runnable): if runnable.output_dir is not None: stdout = open(os.path.join(runnable.output_dir, "stdout"), "xb") stderr = open(os.path.join(runnable.output_dir, "stderr"), "xb") else: stdout = subprocess.PIPE stderr = subprocess.PIPE return subprocess.Popen( [runnable.uri] + list(runnable.args), stdin=subprocess.DEVNULL, stdout=stdout, stderr=stderr, env=self._get_env(runnable), ) def run(self, runnable): yield self.prepare_status("started") try: process = self._run_proc(runnable) except Exception as e: yield self.prepare_status( "finished", {"result": "error", "fail_reason": str(e)} ) self._cleanup(runnable) return def poll_proc(): return process.poll() is not None yield from self.running_loop(poll_proc) if process.stdout is not None: stdout = process.stdout.read() yield self.prepare_status("running", {"type": "stdout", "log": stdout}) else: stdout_path = os.path.join(runnable.output_dir, "stdout") with open(stdout_path, "rb") as stdout_file: stdout = stdout_file.read() yield self.prepare_status( "running", {"type": "stdout", "log": stdout, "log_only": True} ) if process.stderr is not None: stderr = process.stderr.read() yield self.prepare_status("running", {"type": "stderr", "log": stderr}) else: stderr_path = os.path.join(runnable.output_dir, "stderr") with open(stderr_path, "rb") as stderr_file: stderr = stderr_file.read() yield self.prepare_status( "running", {"type": "stderr", "log": stderr, "log_only": True} ) yield self._process_final_status(process, runnable, stdout, stderr) self._cleanup(runnable) class RunnerApp(BaseRunnerApp): PROG_NAME = "avocado-runner-exec-test" PROG_DESCRIPTION = "nrunner application for exec-test tests" RUNNABLE_KINDS_CAPABLE = ["exec-test"] def main(): if sys.platform == "darwin": multiprocessing.set_start_method("fork") app = RunnerApp(print) app.run() if __name__ == "__main__": main() avocado-113.0/avocado/plugins/runners/noop.py000066400000000000000000000015021513561306000212460ustar00rootroot00000000000000from avocado.core.nrunner.app import BaseRunnerApp from avocado.core.nrunner.runner import BaseRunner class NoOpRunner(BaseRunner): """ Sample runner that performs no action before reporting FINISHED status Runnable attributes usage: * uri: not used * args: not used """ name = "noop" description = ( "Sample runner that performs no action before reporting FINISHED status" ) def run(self, runnable): yield self.prepare_status("started") yield self.prepare_status("finished", {"result": "pass"}) class RunnerApp(BaseRunnerApp): PROG_NAME = "avocado-runner-noop" PROG_DESCRIPTION = "nrunner application for noop tests" RUNNABLE_KINDS_CAPABLE = ["noop"] def main(): app = RunnerApp(print) app.run() if __name__ == "__main__": main() avocado-113.0/avocado/plugins/runners/package.py000066400000000000000000000135041513561306000216730ustar00rootroot00000000000000import sys import time from multiprocessing import Process, SimpleQueue, set_start_method from avocado.core.nrunner.app import BaseRunnerApp from avocado.core.nrunner.runner import RUNNER_RUN_STATUS_INTERVAL, BaseRunner from avocado.core.utils import messages from avocado.utils.software_manager.main import MESSAGES from avocado.utils.software_manager.manager import SoftwareManager class PackageRunner(BaseRunner): """Runner for dependencies of type package This runner handles, the installation, verification and removal of packages using the avocado-software-manager. Runnable attributes usage: * kind: 'package' * uri: not used * args: not used * kwargs: - name: the package name (required) - action: one of 'install', 'check', or 'remove' (optional, defaults to 'install') """ name = "package" description = "Runner for dependencies of type package" @staticmethod def _check(software_manager, package): if software_manager.check_installed(package): result = "pass" stdout = MESSAGES["check-installed"]["success"] % package stderr = "" else: result = "error" stdout = "" stderr = MESSAGES["check-installed"]["fail"] % package return result, stdout, stderr @staticmethod def _install(software_manager, cmd, package): result = "pass" stderr = "" if not software_manager.check_installed(package): if software_manager.install(package): stdout = MESSAGES[cmd]["success"] % package else: # check if the error is a false negative because of package # installation collision if software_manager.check_installed(package): stdout = MESSAGES[cmd]["success"] % package else: result = "error" stdout = "" stderr = MESSAGES[cmd]["fail"] % package else: stdout = MESSAGES["check-installed"]["success"] % package return result, stdout, stderr @staticmethod def _remove(software_manager, cmd, package): result = "pass" stderr = "" if software_manager.check_installed(package): if software_manager.remove(package): stdout = MESSAGES[cmd]["success"] % package else: # check if the error is a false negative because of package # installation collision if not software_manager.check_installed(package): stdout = MESSAGES[cmd]["success"] % package else: result = "error" stdout = "" stderr = MESSAGES[cmd]["fail"] % package else: stdout = MESSAGES["check-installed"]["fail"] % package return result, stdout, stderr def _run_software_manager(self, cmd, package, queue): software_manager = SoftwareManager() if not software_manager.is_capable(): output = { "result": "error", "stdout": "", "stderr": ("Package manager not supported or not available."), } queue.put(output) return if cmd == "install": result, stdout, stderr = self._install(software_manager, cmd, package) elif cmd == "remove": result, stdout, stderr = self._remove(software_manager, cmd, package) elif cmd == "check": result, stdout, stderr = self._check(software_manager, package) output = {"result": result, "stdout": stdout, "stderr": stderr} queue.put(output) def run(self, runnable): # pylint: disable=W0201 self.runnable = runnable yield messages.StartedMessage.get() # check if there is a valid 'action' argument cmd = self.runnable.kwargs.get("action", "install") # avoid invalid arguments if cmd not in ["install", "check", "remove"]: stderr = ( f"Invalid action {cmd}. Use one of 'install', 'check' " f"or 'remove'" ) yield messages.StderrMessage.get(stderr.encode()) yield messages.FinishedMessage.get("error") return package = self.runnable.kwargs.get("name") # if package was passed correctly, run avocado-software-manager if package is not None: # let's spawn it to another process to be able to update the # status messages and avoid the software-manager to lock this # process queue = SimpleQueue() process = Process( target=self._run_software_manager, args=(cmd, package, queue) ) process.start() while queue.empty(): time.sleep(RUNNER_RUN_STATUS_INTERVAL) yield messages.RunningMessage.get() output = queue.get() result = output["result"] stdout = output["stdout"] stderr = output["stderr"] else: # Otherwise, log the missing package name result = "error" stdout = "" stderr = ( 'Package name should be passed as kwargs using name="package_name".' ) yield messages.StdoutMessage.get(stdout.encode()) yield messages.StderrMessage.get(stderr.encode()) yield messages.FinishedMessage.get(result) class RunnerApp(BaseRunnerApp): PROG_NAME = "avocado-runner-package" PROG_DESCRIPTION = "nrunner application for dependencies of type package" RUNNABLE_KINDS_CAPABLE = ["package"] def main(): if sys.platform == "darwin": set_start_method("fork") app = RunnerApp(print) app.run() if __name__ == "__main__": main() avocado-113.0/avocado/plugins/runners/pip.py000066400000000000000000000051121513561306000210640ustar00rootroot00000000000000import sys import traceback from multiprocessing import set_start_method from avocado.core.nrunner.app import BaseRunnerApp from avocado.core.nrunner.runner import BaseRunner from avocado.core.utils import messages from avocado.utils import process class PipRunner(BaseRunner): """Runner for dependencies of type pip This runner handles, the installation, verification and removal of packages using the pip. Runnable attributes usage: * kind: 'pip' * uri: not used * args: not used * kwargs: - name: the package name (required) - action: one of 'install' or 'uninstall' (optional, defaults to 'install') """ name = "pip" description = "Runner for dependencies of type pip" def run(self, runnable): try: yield messages.StartedMessage.get() # check if there is a valid 'action' argument cmd = runnable.kwargs.get("action", "install") # avoid invalid arguments if cmd not in ["install", "uninstall"]: stderr = f"Invalid action {cmd}. Use one of 'install' or 'remove'" yield messages.StderrMessage.get(stderr.encode()) yield messages.FinishedMessage.get("error") return package = runnable.kwargs.get("name") # if package was passed correctly, run python -m pip if package is not None: try: cmd = f"python3 -m ensurepip && python3 -m pip {cmd} {package}" result = process.run(cmd, shell=True) except Exception as e: yield messages.StderrMessage.get(str(e)) yield messages.FinishedMessage.get("error") return yield messages.StdoutMessage.get(result.stdout) yield messages.StderrMessage.get(result.stderr) yield messages.FinishedMessage.get("pass") except Exception as e: yield messages.StderrMessage.get(traceback.format_exc()) yield messages.FinishedMessage.get( "error", fail_reason=str(e), fail_class=e.__class__.__name__, traceback=traceback.format_exc(), ) class RunnerApp(BaseRunnerApp): PROG_NAME = "avocado-runner-pip" PROG_DESCRIPTION = "nrunner application for dependencies of type pip" RUNNABLE_KINDS_CAPABLE = ["pip"] def main(): if sys.platform == "darwin": set_start_method("fork") app = RunnerApp(print) app.run() if __name__ == "__main__": main() avocado-113.0/avocado/plugins/runners/podman_image.py000066400000000000000000000046701513561306000227240ustar00rootroot00000000000000import asyncio import logging import sys import time from multiprocessing import Process, SimpleQueue, set_start_method from avocado.core.nrunner.app import BaseRunnerApp from avocado.core.nrunner.runner import RUNNER_RUN_STATUS_INTERVAL, BaseRunner from avocado.core.utils import messages from avocado.utils.podman import AsyncPodman, PodmanException class PodmanImageRunner(BaseRunner): """Runner for dependencies of type podman-image This runner handles download and verification. Runnable attributes usage: * kind: 'podman-image' * uri: the name of the image * args: not used """ name = "podman-image" description = f"Runner for dependencies of type {name}" def _run_podman_pull(self, uri, queue): # Silence the podman utility from outputting messages into # the regular handler, which will go to stdout. The # exceptions caught here still contain all the needed # information for debugging in case of errors. logging.getLogger("avocado.utils.podman").addHandler(logging.NullHandler()) try: podman = AsyncPodman() loop = asyncio.get_event_loop() loop.run_until_complete(podman.execute("pull", uri)) queue.put({"result": "pass"}) except PodmanException as ex: queue.put( {"result": "fail", "fail_reason": f"Could not pull podman image: {ex}"} ) def run(self, runnable): yield messages.StartedMessage.get() if not runnable.uri: reason = "uri identifying the podman image is required" yield messages.FinishedMessage.get("error", reason) else: queue = SimpleQueue() process = Process(target=self._run_podman_pull, args=(runnable.uri, queue)) process.start() while queue.empty(): time.sleep(RUNNER_RUN_STATUS_INTERVAL) yield messages.RunningMessage.get() output = queue.get() result = output.pop("result") yield messages.FinishedMessage.get(result, **output) class RunnerApp(BaseRunnerApp): PROG_NAME = f"avocado-runner-{PodmanImageRunner.name}" PROG_DESCRIPTION = PodmanImageRunner.description RUNNABLE_KINDS_CAPABLE = [PodmanImageRunner.name] def main(): if sys.platform == "darwin": set_start_method("fork") app = RunnerApp(print) app.run() if __name__ == "__main__": main() avocado-113.0/avocado/plugins/runners/python_unittest.py000066400000000000000000000153711513561306000235640ustar00rootroot00000000000000import io import multiprocessing import os import sys import time import traceback from unittest import TestLoader, TextTestRunner from avocado.core.nrunner.app import BaseRunnerApp from avocado.core.nrunner.runner import ( RUNNER_RUN_CHECK_INTERVAL, RUNNER_RUN_STATUS_INTERVAL, BaseRunner, ) from avocado.core.utils import messages class PythonUnittestRunner(BaseRunner): """ Runner for Python unittests The runnable uri is used as the test name that the native unittest TestLoader will use to find the test. A native unittest test runner (TextTestRunner) will be used to execute the test. Runnable attributes usage: * uri: a single test reference, that is "a test method within a test case class" within a test module. Example is: "./tests/foo.py:ClassFoo.test_bar". * args: not used * kwargs: not used """ name = "python-unittest" description = "Runner for Python unittests" @property def unittest(self): """Returns the unittest part of an uri as tuple. Ex: uri = './avocado.dev/selftests/.data/unittests/test.py:Class.test_foo' It will return ("test", "Class", "test_foo") """ uri = self.runnable.uri or "" if ":" in uri: module, class_method = uri.rsplit(":", 1) else: return None if module.endswith(".py"): module = module[:-3] module = module.rsplit(os.path.sep, 1)[-1] klass, method = class_method.rsplit(".", maxsplit=1) return module, klass, method @property def module_path(self): """Path where the module is located. Ex: uri = './avocado.dev/selftests/.data/unittests/test.py:Class.test_foo' It will return './avocado.dev/selftests/.data/unittests/' """ uri = self.runnable.uri if not uri: return None module_path = uri.rsplit(":", 1)[0] return os.path.dirname(module_path) @property def module_class_method(self): """Return a dotted name with module + class + method. Important to note here that module is only the module file without the full path. """ unittest = self.unittest if not unittest: return None return ".".join(unittest) @classmethod def _run_unittest(cls, module_path, module_class_method, queue): def run_and_load_test(): sys.path.insert(0, module_path) try: loader = TestLoader() suite = loader.loadTestsFromName(module_class_method) except ValueError as ex: msg = "loadTestsFromName error finding unittest {}: {}" queue.put(messages.StderrMessage.get(traceback.format_exc())) queue.put( messages.FinishedMessage.get( "error", fail_reason=msg.format(module_class_method, str(ex)), fail_class=ex.__class__.__name__, traceback=traceback.format_exc(), ) ) return None return runner.run(suite) stream = io.StringIO() runner = TextTestRunner(stream=stream, verbosity=0) # running the actual test if "COVERAGE_RUN" in os.environ: from coverage import Coverage coverage = Coverage(data_suffix=True) with coverage.collect(): unittest_result = run_and_load_test() coverage.save() else: unittest_result = run_and_load_test() if not unittest_result: return unittest_result_entries = None if len(unittest_result.errors) > 0: result = "error" unittest_result_entries = unittest_result.errors elif len(unittest_result.failures) > 0: result = "fail" unittest_result_entries = unittest_result.failures elif len(unittest_result.skipped) > 0: result = "skip" unittest_result_entries = unittest_result.skipped else: result = "pass" stream.seek(0) queue.put(messages.StdoutMessage.get(stream.read())) fail_reason = None if unittest_result_entries is not None: last_entry = unittest_result_entries[-1] lines = last_entry[1].splitlines() fail_reason = lines[-1] stream.close() queue.put(messages.FinishedMessage.get(result, fail_reason=fail_reason)) def run(self, runnable): # pylint: disable=W0201 self.runnable = runnable try: yield messages.StartedMessage.get() queue = multiprocessing.SimpleQueue() process = multiprocessing.Process( target=self._run_unittest, args=(self.module_path, self.module_class_method, queue), ) if not self.module_class_method: raise ValueError( "Invalid URI: could not be converted to an unittest dotted name." ) process.start() most_current_execution_state_time = None while True: if queue.empty(): time.sleep(RUNNER_RUN_CHECK_INTERVAL) now = time.monotonic() if most_current_execution_state_time is not None: next_execution_state_mark = ( most_current_execution_state_time + RUNNER_RUN_STATUS_INTERVAL ) if ( most_current_execution_state_time is None or now > next_execution_state_mark ): most_current_execution_state_time = now yield messages.RunningMessage.get() else: message = queue.get() yield message if message.get("status") == "finished": break except Exception as e: yield messages.StderrMessage.get(traceback.format_exc()) yield messages.FinishedMessage.get( "error", fail_reason=str(e), fail_class=e.__class__.__name__, traceback=traceback.format_exc(), ) class RunnerApp(BaseRunnerApp): PROG_NAME = "avocado-runner-python-unittest" PROG_DESCRIPTION = "nrunner application for python-unittest tests" RUNNABLE_KINDS_CAPABLE = ["python-unittest"] def main(): if sys.platform == "darwin": multiprocessing.set_start_method("fork") app = RunnerApp(print) app.run() if __name__ == "__main__": main() avocado-113.0/avocado/plugins/runners/sysinfo.py000066400000000000000000000162471513561306000220010ustar00rootroot00000000000000import multiprocessing import os import sys import time import traceback from avocado.core.nrunner.app import BaseRunnerApp from avocado.core.nrunner.runner import ( RUNNER_RUN_CHECK_INTERVAL, RUNNER_RUN_STATUS_INTERVAL, BaseRunner, ) from avocado.core.utils import messages from avocado.utils import sysinfo as sysinfo_collectible from avocado.utils.software_manager import manager class PreSysInfo: """ Log different system properties before start event. An event may be a job, a test, or any other event with a beginning and end. """ sysinfo_dir = os.path.join("sysinfo", "pre") def __init__(self, config, sysinfo_config, queue): """ Set sysinfo collectibles. :param config: avocado configuration :type config: dict :param sysinfo_config: dictionary with commands/tasks which should be performed during the sysinfo collection. :type sysinfo_config: dict :param queue: queue for the runner messages :type queue: multiprocessing.SimpleQueue """ self.config = config self.queue = queue self.log_packages = self.config.get("sysinfo.collect.installed_packages") self.timeout = self.config.get("sysinfo.collect.commands_timeout") self.locale = self.config.get("sysinfo.collect.locale") self.sysinfo_config = sysinfo_config self.collectibles = set() @property def installed_pkgs(self): sm = manager.SoftwareManager() return sm.list_all() def _set_collectibles(self): for cmd in self.sysinfo_config.get("commands", []): self.collectibles.add( sysinfo_collectible.Command( cmd, timeout=self.timeout, locale=self.locale ) ) for filename in self.sysinfo_config.get("files", []): self.collectibles.add(sysinfo_collectible.Logfile(filename)) def _save_sysinfo(self, log_hook): try: file_path = os.path.join(self.sysinfo_dir, log_hook.name) for data in log_hook.collect(): self.queue.put(messages.FileMessage.get(data, file_path)) except sysinfo_collectible.CollectibleException as e: self.queue.put(messages.LogMessage.get(e.args[0])) except Exception as exc: # pylint: disable=W0703 self.queue.put( messages.StderrMessage.get( f"Collection " f"{type(log_hook)} " f"failed: {exc}" ) ) def collect(self): """Log all collectibles at the start of the event.""" self._set_collectibles() for log_hook in self.collectibles: self._save_sysinfo(log_hook) if self.log_packages: self._log_packages(self.sysinfo_dir) self.queue.put(messages.FinishedMessage.get("pass")) def _log_packages(self, path): installed_path = os.path.join(path, "installed_packages") installed_packages = "\n".join(self.installed_pkgs) + "\n" self.queue.put(messages.FileMessage.get(installed_packages, installed_path)) class PostSysInfo(PreSysInfo): """ Log different system properties after end event. An event may be a job, a test, or any other event with a beginning and end. """ sysinfo_dir = os.path.join("sysinfo", "post") def __init__(self, config, sysinfo_config, queue, test_fail=False): """ :param test_fail: flag for fail tests. Default False :type test_fail: bool """ self.test_fail = test_fail super().__init__(config, sysinfo_config, queue) def _set_collectibles(self): super()._set_collectibles() if self.test_fail: for fail_cmd in self.sysinfo_config.get("fail_commands", []): self.collectibles.add( sysinfo_collectible.Command( fail_cmd, timeout=self.timeout, locale=self.locale ) ) for fail_filename in self.sysinfo_config.get("fail_files", []): self.collectibles.add(sysinfo_collectible.Logfile(fail_filename)) class SysinfoRunner(BaseRunner): """ Runner for gathering sysinfo Runnable attributes usage: * uri: sysinfo type pre/post. This variable decides if the sysinfo is collected before or after the test. * kwargs: "sysinfo" dictionary with commands/tasks which should be performed during the sysinfo collection. """ name = "sysinfo" description = "Runner for gathering sysinfo" CONFIGURATION_USED = [ "sysinfo.collect.installed_packages", "sysinfo.collect.commands_timeout", "sysinfo.collect.locale", ] def run(self, runnable): # pylint: disable=W0201 self.runnable = runnable yield self.prepare_status("started") sysinfo_config = self.runnable.kwargs.get("sysinfo", {}) test_fail = self.runnable.kwargs.get("test_fail", False) if self.runnable.uri not in ["pre", "post"]: yield messages.StderrMessage.get( f"Unsupported uri " f"{self.runnable.uri}. " f"Possible values, 'pre', 'post'" ) yield messages.FinishedMessage.get("error") try: queue = multiprocessing.SimpleQueue() if self.runnable.uri == "pre": sysinfo = PreSysInfo(self.runnable.config, sysinfo_config, queue) else: sysinfo = PostSysInfo( self.runnable.config, sysinfo_config, queue, test_fail ) sysinfo_process = multiprocessing.Process(target=sysinfo.collect) sysinfo_process.start() most_current_execution_state_time = None while True: time.sleep(RUNNER_RUN_CHECK_INTERVAL) now = time.monotonic() if queue.empty(): if most_current_execution_state_time is not None: next_execution_state_mark = ( most_current_execution_state_time + RUNNER_RUN_STATUS_INTERVAL ) if ( most_current_execution_state_time is None or now > next_execution_state_mark ): most_current_execution_state_time = now yield messages.RunningMessage.get() else: message = queue.get() yield message if message.get("status") == "finished": break except Exception: # pylint: disable=W0703 yield messages.StderrMessage.get(traceback.format_exc()) yield messages.FinishedMessage.get("error") class RunnerApp(BaseRunnerApp): PROG_NAME = "avocado-runner-sysinfo" PROG_DESCRIPTION = "nrunner application for gathering sysinfo" RUNNABLE_KINDS_CAPABLE = ["sysinfo"] def main(): if sys.platform == "darwin": multiprocessing.set_start_method("fork") app = RunnerApp(print) app.run() if __name__ == "__main__": main() avocado-113.0/avocado/plugins/runners/tap.py000066400000000000000000000062661513561306000210730ustar00rootroot00000000000000import io import multiprocessing import sys from avocado.core.nrunner.app import BaseRunnerApp from avocado.core.tapparser import TapParser, TestResult from avocado.core.utils.messages import FinishedMessage from avocado.plugins.runners.exec_test import ExecTestRunner class TAPRunner(ExecTestRunner): """Runner for standalone executables treated as TAP When creating the Runnable, use the following attributes: * kind: should be 'tap'; * uri: path to a binary to be executed as another process. This must provides a TAP output. * args: any runnable argument will be given on the command line to the binary given by path * kwargs: you can specify multiple key=val as kwargs. This will be used as environment variables to the process. Example: runnable = Runnable(kind='tap', uri='tests/foo.sh', 'bar', # arg 1 DEBUG='false') # kwargs 1 (environment) """ name = "tap" description = "Runner for standalone executables treated as TAP" @staticmethod def _get_tap_result(stdout): parser = TapParser(io.StringIO(stdout.decode())) result = "" fail_reason = None for event in parser.parse(): if isinstance(event, TapParser.Bailout): result = "error" break elif isinstance(event, TapParser.Error): result = "error" fail_reason = f"Tap format error: {event.message}" break elif isinstance(event, TapParser.Plan): if event.skipped: if not result: result = "skip" continue elif isinstance(event, TapParser.Test): if event.result == TestResult.FAIL: result = "fail" fail_reason = event.explanation break elif event.result == TestResult.SKIP: if not result: result = "skip" continue elif event.result == TestResult.XPASS: result = "warn" if event.name: tap_test_id = f"{event.number} ({event.name})" else: tap_test_id = f"{event.number}" fail_reason = f"TODO test {tap_test_id} unexpectedly passed." break else: result = "pass" return result, fail_reason def _process_final_status( self, process, runnable, stdout=None, stderr=None ): # pylint: disable=W0613 result, fail_reason = self._get_tap_result(stdout) return FinishedMessage.get(result, fail_reason, returncode=process.returncode) class RunnerApp(BaseRunnerApp): PROG_NAME = "avocado-runner-tap" PROG_DESCRIPTION = "nrunner application for executable tests that produce TAP" RUNNABLE_KINDS_CAPABLE = ["tap"] def main(): if sys.platform == "darwin": multiprocessing.set_start_method("fork") app = RunnerApp(print) app.run() if __name__ == "__main__": main() avocado-113.0/avocado/plugins/runners/vmimage.py000066400000000000000000000133451513561306000217300ustar00rootroot00000000000000import multiprocessing import signal import sys import time import traceback from multiprocessing import set_start_method from avocado.core.exceptions import TestInterrupt from avocado.core.nrunner.app import BaseRunnerApp from avocado.core.nrunner.runner import ( RUNNER_RUN_CHECK_INTERVAL, RUNNER_RUN_STATUS_INTERVAL, BaseRunner, ) from avocado.core.utils import messages from avocado.core.utils.messages import start_logging from avocado.plugins.vmimage import download_image from avocado.utils import vmimage class VMImageRunner(BaseRunner): """ Runner for dependencies of type vmimage. This runner uses the vmimage plugin's download_image function which handles: 1. Checking if the image exists in cache 2. Downloading the image if not in cache 3. Storing the image in the configured cache directory 4. Returning the cached image path """ name = "vmimage" description = "Runner for dependencies of type vmimage" @staticmethod def signal_handler(signum, frame): # pylint: disable=W0613 if signum == signal.SIGTERM.value: raise TestInterrupt("VM image operation interrupted: Timeout reached") @staticmethod def _run_vmimage_operation(runnable, queue): try: signal.signal(signal.SIGTERM, VMImageRunner.signal_handler) start_logging(runnable.config, queue) # Get parameters from runnable.kwargs provider = runnable.kwargs.get("provider") version = runnable.kwargs.get("version") arch = runnable.kwargs.get("arch") if not provider: stderr = "Missing required parameter: provider" queue.put(messages.StderrMessage.get(stderr.encode())) queue.put(messages.FinishedMessage.get("error")) return message = f"Getting VM image for {provider}" if version: message += f" {version}" if arch: message += f" {arch}" queue.put(messages.StdoutMessage.get(message.encode())) queue.put( messages.StdoutMessage.get( f"Attempting to download image for {provider} (version: {version or 'latest'}, arch: {arch})".encode() ) ) image = download_image(provider, version, arch) if not image: raise RuntimeError("Failed to get image") queue.put( messages.StdoutMessage.get( f"Successfully retrieved VM image from cache or downloaded to: {image['file']}".encode() ) ) queue.put(messages.FinishedMessage.get("pass")) except (AttributeError, RuntimeError, vmimage.ImageProviderError) as e: # AttributeError: provider not found # RuntimeError: failed to get image # ImageProviderError: provider-specific errors queue.put( messages.StderrMessage.get( f"Failed to download image: {str(e)}".encode() ) ) queue.put( messages.FinishedMessage.get( "error", fail_reason=str(e), fail_class=e.__class__.__name__, traceback=traceback.format_exc(), ) ) except (TestInterrupt, multiprocessing.TimeoutError) as e: queue.put( messages.StderrMessage.get(f"Operation interrupted: {str(e)}".encode()) ) queue.put( messages.FinishedMessage.get( "error", fail_reason=str(e), fail_class=e.__class__.__name__, ) ) @staticmethod def _monitor(queue): most_recent_status_time = None while True: time.sleep(RUNNER_RUN_CHECK_INTERVAL) if queue.empty(): now = time.monotonic() if ( most_recent_status_time is None or now >= most_recent_status_time + RUNNER_RUN_STATUS_INTERVAL ): most_recent_status_time = now yield messages.RunningMessage.get() continue message = queue.get() yield message if message.get("status") == "finished": break def run(self, runnable): signal.signal(signal.SIGTERM, VMImageRunner.signal_handler) yield messages.StartedMessage.get() queue = multiprocessing.SimpleQueue() process = multiprocessing.Process( target=self._run_vmimage_operation, args=(runnable, queue) ) try: process.start() for message in self._monitor(queue): yield message except TestInterrupt: process.terminate() for message in self._monitor(queue): yield message except (multiprocessing.ProcessError, OSError) as e: # ProcessError: Issues with process management # OSError: System-level errors (e.g. resource limits) yield messages.StderrMessage.get(f"Process error: {str(e)}".encode()) yield messages.FinishedMessage.get( "error", fail_reason=str(e), fail_class=e.__class__.__name__, ) class RunnerApp(BaseRunnerApp): PROG_NAME = "avocado-runner-vmimage" PROG_DESCRIPTION = "nrunner application for dependencies of type vmimage" RUNNABLE_KINDS_CAPABLE = ["vmimage"] def main(): if sys.platform == "darwin": set_start_method("fork") app = RunnerApp(print) app.run() if __name__ == "__main__": main() avocado-113.0/avocado/plugins/spawners/000077500000000000000000000000001513561306000200715ustar00rootroot00000000000000avocado-113.0/avocado/plugins/spawners/__init__.py000066400000000000000000000000001513561306000221700ustar00rootroot00000000000000avocado-113.0/avocado/plugins/spawners/lxc.py000066400000000000000000000254221513561306000212360ustar00rootroot00000000000000import asyncio import contextlib import logging import os import tempfile try: import lxc LXC_AVAILABLE = True except ImportError: lxc = None LXC_AVAILABLE = False from avocado.core.plugin_interfaces import Init, Spawner from avocado.core.settings import settings from avocado.core.spawners.common import SpawnCapabilities, SpawnerMixin, SpawnMethod LOG = logging.getLogger(__name__) class LXCSpawnerException(Exception): """Errors more closely related to the spawner functionality""" class LXCStreamsFile: def __init__(self): self.fd = None self.path = None def fileno(self): return self.fd def read(self): with open(self.path, "r", encoding="utf-8") as fp: return fp.read() def __enter__(self): self.fd, self.path = tempfile.mkstemp() return self def __exit__(self, *args): os.remove(self.path) class LXCSpawnerInit(Init): description = "LXC (container) based spawner initialization" def initialize(self): section = "spawner.lxc" help_msg = "List of already available container slots to spawn in" settings.register_option( section=section, key="slots", help_msg=help_msg, key_type=list, default=[] ) help_msg = "Distribution for the LXC container" settings.register_option( section=section, key="dist", help_msg=help_msg, default="fedora" ) help_msg = "Release of the LXC container (depends on the choice of distro)" settings.register_option( section=section, key="release", help_msg=help_msg, default="32" ) help_msg = "Architecture of the LXC container" settings.register_option( section=section, key="arch", help_msg=help_msg, default="i386" ) help_msg = ( "Container creation hook command to customize optional new containers" ) settings.register_option( section=section, key="create_hook", help_msg=help_msg, default="" ) def with_slot_reservation(fn): """ Decorator for slot cache context manager. :param fn: function to run with slot reservation :type fn: function :returns: same function with the slot now reserved :rtype: function The main reason for the decorator is to not have to indent the entire task running function in order to safely release the slot upon any error. """ async def wrapper(self, runtime_task): with LXCSpawner.reserve_slot(self, runtime_task) as slot: runtime_task.spawner_handle = slot return await fn(self, runtime_task) return wrapper class LXCSpawner(Spawner, SpawnerMixin): description = "LXC (container) based spawner" METHODS = [SpawnMethod.STANDALONE_EXECUTABLE] CAPABILITIES = [ SpawnCapabilities.AVOCADO_DEPLOYMENT, SpawnCapabilities.ENVIRONMENT_PRESERVATION, ] slots_cache = {} def is_operational(self): if not LXC_AVAILABLE: LOG.error("LXC python bindings not available on the system") return False return True @staticmethod def run_container_cmd(container, command): with LXCStreamsFile() as tmp_out, LXCStreamsFile() as tmp_err: exitcode = container.attach_wait( lxc.attach_run_command, command, stdout=tmp_out, stderr=tmp_err ) return exitcode, tmp_out.read(), tmp_err.read() @staticmethod async def run_container_cmd_async(container, command): with LXCStreamsFile() as tmp_out, LXCStreamsFile() as tmp_err: pid = container.attach( lxc.attach_run_command, command, stdout=tmp_out, stderr=tmp_err ) loop = asyncio.get_event_loop() _, exitcode = await loop.run_in_executor( None, os.waitpid, pid, os.WUNTRACED ) return exitcode, tmp_out.read(), tmp_err.read() @contextlib.contextmanager def reserve_slot(self, runtime_task): """ Reserve a free or custom container slot for the runtime task. :param runtime_task: runtime task to reserve the slot for :type runtime_task: :py:class:`avocado.core.task.runtime.RuntimeTask` :yields: a free slot to use if such was found :raises: :py:class:`RuntimeError` if no free slot could be found This will either use a runtime cache to find a free container slot to run the task in or use a custom container/slot ID to allow for custom schedulers to make their own decisions on which containers to run when. """ if len(LXCSpawner.slots_cache) == 0: # TODO: consider whether to provide persistence across runs via external storage LXCSpawner.slots_cache = { k: False for k in self.config.get("spawner.lxc.slots") if k } # TODO: spawner can look for free containers directly and populate these slots # for c in lxcontainer.list_containers(as_object=True): ... if runtime_task.spawner_handle is not None: slot = runtime_task.spawner_handle else: slots = LXCSpawner.slots_cache for key, value in slots.items(): if not value: slot = key slots[key] = True break else: raise RuntimeError( "No free slot available for the task, are " "you running with more processes than slots?" ) try: yield slot finally: LXCSpawner.slots_cache[slot] = False @staticmethod def is_task_alive(runtime_task): if runtime_task.spawner_handle is None: return False container = lxc.Container(runtime_task.spawner_handle) if not container.defined: LOG.debug(f"Container {runtime_task.spawner_handle} is not defined") return False if not container.running: LOG.debug( f"Container {runtime_task.spawner_handle} state is " f"{container.state} instead of RUNNING" ) return False status, _, _ = LXCSpawner.run_container_cmd( container, ["pgrep", "-r", "R,S", "-f", runtime_task.task.identifier] ) return status == 0 @with_slot_reservation async def spawn_task(self, runtime_task): self.create_task_output_dir(runtime_task) task = runtime_task.task full_module_name = ( runtime_task.task.runnable.pick_runner_module_from_entry_point_kind( runtime_task.task.runnable.kind ) ) if full_module_name is None: msg = f"Could not determine Python module name for runnable with kind {runtime_task.task.runnable.kind}" raise LXCSpawnerException(msg) # using the "python" symlink will result in the container default python version entry_point_args = ["python3", "-m", full_module_name, "task-run"] entry_point_args.extend(task.get_command_args()) dist = self.config.get("spawner.lxc.dist") release = self.config.get("spawner.lxc.release") arch = self.config.get("spawner.lxc.arch") create_hook = self.config.get("spawner.lxc.create_hook") container_id = runtime_task.spawner_handle container = lxc.Container(container_id) if not container.defined: # Create the container rootfs if not container.create( "download", lxc.LXC_CREATE_QUIET, {"dist": dist, "release": release, "arch": arch}, ): LOG.error("Failed to create the container rootfs") return False # Customize and deploy test data to the container if create_hook: customization_args = create_hook.split() exitcode, output, err = await LXCSpawner.run_container_cmd_async( container, customization_args ) LOG.debug(f"Customization command exited with code {exitcode}") if exitcode != 0: LOG.error(f"Error '{err}' on {container_id} with output:\n{output}") return False # Start the container if not container.running: if not container.start(): LOG.error("Failed to start the container") return False # Wait for connectivity # TODO: The current networking is not good enough to connect to the status server if not container.get_ips(timeout=30): LOG.error("Failed to connect to the container") return False # Query some information LOG.info(f"Container state: {container.state}") LOG.info(f"Container ID: {container_id} PID: {container.init_pid}") exitcode, output, err = await LXCSpawner.run_container_cmd_async( container, entry_point_args ) LOG.debug(f"Command exited with code {exitcode}") if exitcode != 0: LOG.error(f"Error '{err}' on {container_id} with output:\n{output}") return False return True def create_task_output_dir(self, runtime_task): output_dir_path = self.task_output_dir(runtime_task) output_lxc_path = "/tmp/.avocado_task_output_dir" os.makedirs(output_dir_path, exist_ok=True) runtime_task.task.setup_output_dir(output_lxc_path) async def wait_task(self, runtime_task): while True: if not LXCSpawner.is_task_alive(runtime_task): return await asyncio.sleep(0.1) async def terminate_task(self, runtime_task): container = lxc.Container(runtime_task.spawner_handle) # Stop the container if not container.shutdown(30): LOG.warning("Failed to cleanly shutdown the container, forcing.") if not container.stop(): LOG.error("Failed to kill the container") return False # TODO: we can provide extra options to not just stop but destroy the container # Destroy the container # if not container.destroy(): # LOG.error("Failed to destroy the container.") # return False return True @staticmethod async def check_task_requirements(runtime_task): """Check the runtime task requirements needed to be able to run""" # right now, limit the check to the LXC availability return LXC_AVAILABLE @staticmethod async def is_requirement_in_cache(runtime_task): return False @staticmethod async def save_requirement_in_cache(runtime_task): pass @staticmethod async def update_requirement_cache(runtime_task, result): pass avocado-113.0/avocado/plugins/spawners/podman.py000066400000000000000000000460111513561306000217230ustar00rootroot00000000000000import asyncio import json import logging import os import re import subprocess import time import uuid from avocado.core.dependencies.requirements import cache from avocado.core.plugin_interfaces import CLI, DeploymentSpawner, Init from avocado.core.resolver import ReferenceResolutionAssetType from avocado.core.settings import settings from avocado.core.spawners.common import SpawnCapabilities, SpawnerMixin, SpawnMethod from avocado.core.teststatus import STATUSES_NOT_OK from avocado.core.version import VERSION from avocado.utils import distro from avocado.utils.asset import Asset from avocado.utils.podman import AsyncPodman, PodmanException LOG = logging.getLogger(__name__) class PodmanSpawnerException(PodmanException): """Errors more closely related to the spawner functionality""" class PodmanSpawnerInit(Init): description = "Podman (container) based spawner initialization" def initialize(self): section = "spawner.podman" help_msg = "Path to the podman binary" settings.register_option( section=section, key="bin", help_msg=help_msg, default="/usr/bin/podman" ) this_distro = distro.detect() if this_distro != distro.UNKNOWN_DISTRO: default_distro = f"{this_distro.name}:{this_distro.version}" else: default_distro = "fedora:latest" help_msg = ( f"Image name to use when creating the container. " f"The first default choice is a container image " f"matching the current OS. If unable to detect, " f"default becomes the latest Fedora release. Default " f"on this system: {default_distro}" ) settings.register_option( section=section, key="image", help_msg=help_msg, default=default_distro ) help_msg = ( "Avocado egg path to be used during initial bootstrap " "of avocado inside the isolated environment. By default, " "Avocado will try to download (or get from cache) an " "egg from its repository. Please use a valid URL, including " 'the protocol (for local files, use the "file:///" prefix).' ) settings.register_option( section=section, key="avocado_spawner_egg", help_msg=help_msg, default=None ) help_msg = ( "Prefix to use when tagging images that are created by the " "Podman spawner." ) settings.register_option( section=section, key="image_tag_prefix", help_msg=help_msg, default="avocado_generated", ) class PodmanCLI(CLI): name = "podman" description = 'podman spawner command line options for "run"' def configure(self, parser): super().configure(parser) parser = parser.subcommands.choices.get("run", None) if parser is None: return parser = parser.add_argument_group("podman spawner specific options") settings.add_argparser_to_option( namespace="spawner.podman.bin", parser=parser, long_arg="--spawner-podman-bin", metavar="PODMAN_BIN", ) settings.add_argparser_to_option( namespace="spawner.podman.image", parser=parser, long_arg="--spawner-podman-image", metavar="CONTAINER_IMAGE", ) namespace = "spawner.podman.avocado_spawner_egg" long_arg = "--spawner-podman-avocado-egg" settings.add_argparser_to_option( namespace=namespace, parser=parser, long_arg=long_arg, metavar="AVOCADO_EGG" ) def run(self, config): pass class PodmanSpawner(DeploymentSpawner, SpawnerMixin): description = "Podman (container) based spawner" METHODS = [SpawnMethod.STANDALONE_EXECUTABLE] CAPABILITIES = [ SpawnCapabilities.AUTOMATIC_ENVIRONMENT_PROVISIONING, SpawnCapabilities.AVOCADO_DEPLOYMENT, SpawnCapabilities.ENVIRONMENT_PRESERVATION, SpawnCapabilities.FILESYSTEM_SHARING, ] _PYTHON_VERSIONS_CACHE = {} def __init__(self, config=None, job=None): # pylint: disable=W0231 SpawnerMixin.__init__(self, config, job) self.environment = f"podman:{self.config.get('spawner.podman.image')}" self._podman_version = (None, None, None) self._podman = None def _get_podman_version(self): podman_bin = self.config.get("spawner.podman.bin") try: cmd = [podman_bin, "--version"] process = subprocess.Popen( cmd, stdin=subprocess.DEVNULL, stdout=subprocess.PIPE, stderr=subprocess.DEVNULL, ) out, _ = process.communicate() except subprocess.SubprocessError as ex: raise PodmanException("Failed to get podman version information") from ex match = re.match(rb"podman version (\d+)\.(\d+).(\d+)\n", out) if match: major, minor, release = match.groups() return (int(major), int(minor), int(release)) raise PodmanException( f"Failed to get podman version information: " f'output received "{out}" does not match expected output' ) def is_operational(self): try: _ = self.podman_version except PodmanException as ex: LOG.error(ex) return False if self.podman_version[0] >= 3: return True LOG.error( f"The podman binary f{self.podman_bin} did not report a suitable version (>= 3.0)" ) return False @property def podman_version(self): if self._podman_version == (None, None, None): self._podman_version = self._get_podman_version() return self._podman_version @property def podman(self): if self._podman is None: podman_bin = self.config.get("spawner.podman.bin") try: self._podman = AsyncPodman(podman_bin) except PodmanException as ex: LOG.error(ex) return self._podman def _get_podman_state(self, runtime_task): if runtime_task.spawner_handle is None: return False podman_bin = self.config.get("spawner.podman.bin") cmd = [ podman_bin, "ps", "--all", "--format={{.State}}", f"--filter=id={runtime_task.spawner_handle}", ] process = subprocess.Popen( cmd, stdin=subprocess.DEVNULL, stdout=subprocess.PIPE, stderr=subprocess.DEVNULL, ) out, _ = process.communicate() return out def is_task_alive(self, runtime_task): # pylint: disable=W0221 out = self._get_podman_state(runtime_task) if self.podman_version[0] < 4: return out.startswith(b"Up ") if out == b"running\n": return True if out == b"created\n": # give the container a chance to transition to running time.sleep(0.1) out = self._get_podman_state(runtime_task) return out == b"running\n" return False def _fetch_asset(self, url): cachedirs = self.config.get("datadir.paths.cache_dirs") asset = Asset(url, cache_dirs=cachedirs) return asset.fetch() def get_eggs_paths(self, py_major, py_minor): """Return the basic eggs needed to bootstrap Avocado. This will return a tuple with the current location and where this should be deployed. """ result = [] # Setuptools # For now let's pin to setuptools 59.2. # TODO: Automatically get latest setuptools version. eggs = [ f"https://github.com/avocado-framework/setuptools/releases/download/v59.2.0/setuptools-59.2.0-py{py_major}.{py_minor}.egg" ] local_egg = self.config.get("spawner.podman.avocado_spawner_egg") if local_egg: eggs.append(local_egg) else: remote_egg = f"https://github.com/avocado-framework/avocado/releases/download/{VERSION}/avocado_framework-{VERSION}-py{py_major}.{py_minor}.egg" eggs.append(remote_egg) for url in eggs: path = self._fetch_asset(url) to = os.path.join("/tmp/", os.path.basename(path)) result.append((path, to)) return result @property async def python_version(self): image = self.config.get("spawner.podman.image") if image not in self._PYTHON_VERSIONS_CACHE: if not self.podman: msg = "Cannot get Python version: self.podman not defined." LOG.debug(msg) return None, None, None result = await self.podman.get_python_version(image) self._PYTHON_VERSIONS_CACHE[image] = result return self._PYTHON_VERSIONS_CACHE[image] async def deploy_artifacts(self): pass async def deploy_avocado(self, where): # Deploy all the eggs to container inside /tmp/ major, minor, _ = await self.python_version eggs = self.get_eggs_paths(major, minor) for egg, to in eggs: await self.podman.copy_to_container(where, egg, to) async def _create_container_for_task( self, runtime_task, env_args, test_output=None ): mount_status_server_socket = False mounted_status_server_socket = "/tmp/.status_server.sock" status_server_uri = runtime_task.task.status_services[0].uri if ":" not in status_server_uri: # a unix domain socket is being used mount_status_server_socket = True runtime_task.task.status_services[0].uri = mounted_status_server_socket _, _, python_binary = await self.python_version full_module_name = ( runtime_task.task.runnable.pick_runner_module_from_entry_point_kind( runtime_task.task.runnable.kind ) ) if full_module_name is None: msg = f"Could not determine Python module name for runnable with kind {runtime_task.task.runnable.kind}" raise PodmanSpawnerException(msg) entry_point_args = [python_binary, "-m", full_module_name, "task-run"] test_opts = [] if runtime_task.task.category == "test" and runtime_task.task.runnable.assets: for asset_type, asset in runtime_task.task.runnable.assets: if asset_type in ( ReferenceResolutionAssetType.TEST_FILE, ReferenceResolutionAssetType.DATA_FILE, ): if os.path.exists(asset): to = os.path.join("/tmp", asset) test_opts.append("-v") test_opts.append(f"{os.path.abspath(asset)}:{to}:ro") if asset_type == ReferenceResolutionAssetType.TEST_FILE: # The URI may contain a test specification within the file, # which is separated by a colon if runtime_task.task.runnable.uri.split(":")[0] == asset: runtime_task.task.runnable.uri = os.path.join( "/tmp", runtime_task.task.runnable.uri ) task = runtime_task.task entry_point_args.extend(task.get_command_args()) entry_point = json.dumps(entry_point_args) entry_point_arg = "--entrypoint=" + entry_point if mount_status_server_socket: status_server_opts = ( "--privileged", "-v", f"{status_server_uri}:{mounted_status_server_socket}", ) else: status_server_opts = ("--net=host",) output_opts = () if test_output: output_opts = ( "-v", f"{test_output}:{runtime_task.task.runnable.output_dir}", ) image, _ = self._get_image_from_cache(runtime_task) if not image: image = self.config.get("spawner.podman.image") envs = [f"-e={k}={v}" for k, v in env_args.items()] # pylint: disable=W0201 _, stdout, _ = await self.podman.execute( "create", *status_server_opts, *output_opts, *test_opts, entry_point_arg, *envs, image, ) return stdout.decode().strip() async def spawn_task(self, runtime_task): self.create_task_output_dir(runtime_task) major, minor, _ = await self.python_version # Return only the "to" location eggs = self.get_eggs_paths(major, minor) destination_eggs = ":".join(map(lambda egg: str(egg[1]), eggs)) env_args = {"PYTHONPATH": destination_eggs} output_dir_path = self.task_output_dir(runtime_task) try: container_id = await self._create_container_for_task( runtime_task, env_args, output_dir_path ) except PodmanException as ex: LOG.error("Could not create podman container: %s", ex) return False runtime_task.spawner_handle = container_id await self.deploy_avocado(container_id) try: # pylint: disable=W0201 returncode, _, _ = await self.podman.start(container_id) except PodmanException as ex: LOG.error("Could not start container: %s", ex) return False return returncode == 0 def create_task_output_dir(self, runtime_task): output_dir_path = self.task_output_dir(runtime_task) output_podman_path = "/tmp/.avocado_task_output_dir" os.makedirs(output_dir_path, exist_ok=True) runtime_task.task.setup_output_dir(output_podman_path) async def wait_task(self, runtime_task): while True: if not self.is_task_alive(runtime_task): return await asyncio.sleep(0.1) async def terminate_task(self, runtime_task): try: await self.podman.execute( "kill", "--signal=TERM", runtime_task.spawner_handle ) except PodmanException as ex: LOG.error("Could not signal termination to task on container: %s", ex) return False soft_interval = self.config.get( "runner.task.interval.from_soft_to_hard_termination" ) await asyncio.sleep(soft_interval) try: await self.podman.stop(runtime_task.spawner_handle) except PodmanException as ex: LOG.error("Could not stop container: %s", ex) return False hard_interval = self.config.get( "runner.task.interval.from_hard_termination_to_verification" ) await asyncio.sleep(hard_interval) info = await self.podman.get_container_info(runtime_task.spawner_handle) return info.get("Exited", False) @staticmethod async def check_task_requirements(runtime_task): """Check the runtime task requirements needed to be able to run""" # right now, limit the check to the runner availability. if runtime_task.task.runnable.runner_command() is None: return False return True async def _create_and_tag_image(self, runtime_task): _, stdout, _ = await self.podman.execute( "commit", "-q", runtime_task.spawner_handle ) image_id = stdout.decode().strip() tag_prefix = self.config.get("spawner.podman.image_tag_prefix") base_image = self.config.get("spawner.podman.image") kind = runtime_task.task.runnable.kind name = runtime_task.task.runnable.kwargs.get("name") tag = f"{tag_prefix}_{base_image}_{kind}_{name}".replace(":", "_") try: _, _, _ = await self.podman.execute("tag", image_id, tag) except PodmanException as ex: LOG.warning("Could not tag image %s with %s: %s", image_id, tag, ex) return image_id async def update_requirement_cache( self, runtime_task, result ): # pylint: disable=W0221 environment_id, _ = self._get_image_from_cache(runtime_task, True) if result in STATUSES_NOT_OK: cache.delete_environment(self.environment, environment_id) return image_id = await self._create_and_tag_image(runtime_task) cache.update_environment(self.environment, environment_id, image_id) cache.update_requirement_status( self.environment, image_id, runtime_task.task.runnable.kind, runtime_task.task.runnable.kwargs.get("name"), True, ) async def save_requirement_in_cache(self, runtime_task): # pylint: disable=W0221 image_id = str(uuid.uuid4()) _, requirements = self._get_image_from_cache(runtime_task) if requirements: for requirement_type, requirement in requirements: cache.set_requirement( self.environment, image_id, requirement_type, requirement ) cache.set_requirement( self.environment, image_id, runtime_task.task.runnable.kind, runtime_task.task.runnable.kwargs.get("name"), False, ) async def is_requirement_in_cache(self, runtime_task): # pylint: disable=W0221 environment, _ = self._get_image_from_cache(runtime_task, use_task=True) if not environment: return False if cache.is_environment_prepared(environment): return True return None def _get_image_from_cache(self, runtime_task, use_task=False): def _get_all_finished_requirements(requirement_tasks): all_finished_requirements = [] for requirement in requirement_tasks: all_finished_requirements.extend( _get_all_finished_requirements(requirement.dependencies) ) runnable = requirement.task.runnable all_finished_requirements.append( (runnable.kind, runnable.kwargs.get("name")) ) return all_finished_requirements finished_requirements = [] if use_task: finished_requirements.append( ( runtime_task.task.runnable.kind, runtime_task.task.runnable.kwargs.get("name"), ) ) finished_requirements.extend( _get_all_finished_requirements(runtime_task.dependencies) ) if not finished_requirements: return None, None runtime_task_kind, runtime_task_name = finished_requirements[0] cache_entries = cache.get_all_environments_with_requirement( self.environment, runtime_task_kind, runtime_task_name ) if not cache_entries: return None, None for image, requirements in cache_entries.items(): if len(finished_requirements) == len(requirements): if set(requirements) == set(finished_requirements): return image, requirements return None, None avocado-113.0/avocado/plugins/spawners/process.py000066400000000000000000000116411513561306000221240ustar00rootroot00000000000000import asyncio import os import socket from avocado.core.dependencies.requirements import cache from avocado.core.plugin_interfaces import Spawner from avocado.core.spawners.common import SpawnCapabilities, SpawnerMixin, SpawnMethod from avocado.core.teststatus import STATUSES_NOT_OK from avocado.core.utils.eggenv import get_python_path_env_if_egg ENVIRONMENT_TYPE = "local" ENVIRONMENT = socket.gethostname() class ProcessSpawnerHandle: def __init__(self, process): self.process = process self._wait_task = None def create_wait_task(self): if self._wait_task is None: loop = asyncio.get_event_loop() self._wait_task = loop.create_task(self.process.wait()) @property def wait_task(self): self.create_wait_task() return self._wait_task class ProcessSpawner(Spawner, SpawnerMixin): description = "Process based spawner" METHODS = [SpawnMethod.STANDALONE_EXECUTABLE] CAPABILITIES = [ SpawnCapabilities.AUTOMATIC_ENVIRONMENT_PROVISIONING, SpawnCapabilities.AVOCADO_DEPLOYMENT, SpawnCapabilities.FILESYSTEM_SHARING, ] def is_operational(self): return True @staticmethod def is_task_alive(runtime_task): if runtime_task.spawner_handle is None: return False if runtime_task.spawner_handle.wait_task.done(): return False return True async def spawn_task(self, runtime_task): self.create_task_output_dir(runtime_task) task = runtime_task.task runner = task.runnable.runner_command() args = runner[1:] + ["task-run"] + task.get_command_args() runner = runner[0] # pylint: disable=E1133 try: proc = await asyncio.create_subprocess_exec( runner, *args, stdout=asyncio.subprocess.DEVNULL, stderr=asyncio.subprocess.DEVNULL, env=get_python_path_env_if_egg(), ) except (FileNotFoundError, PermissionError): return False runtime_task.spawner_handle = ProcessSpawnerHandle(proc) return True def create_task_output_dir(self, runtime_task): output_dir_path = self.task_output_dir(runtime_task) os.makedirs(output_dir_path, exist_ok=True) with open(os.path.join(output_dir_path, "debug.log"), mode="ba"): pass runtime_task.task.setup_output_dir(output_dir_path) @staticmethod async def wait_task(runtime_task): # pylint: disable=W0221 await runtime_task.spawner_handle.wait_task async def terminate_task(self, runtime_task): try: runtime_task.spawner_handle.process.terminate() except ProcessLookupError: return True soft_interval = self.config.get( "runner.task.interval.from_soft_to_hard_termination" ) returncode = None try: returncode = await asyncio.wait_for( runtime_task.spawner_handle.process.wait(), soft_interval ) except asyncio.TimeoutError: try: runtime_task.spawner_handle.process.kill() except ProcessLookupError: return True hard_interval = self.config.get( "runner.task.interval.from_hard_termination_to_verification" ) try: returncode = await asyncio.wait_for( runtime_task.spawner_handle.process.wait(), hard_interval ) except asyncio.TimeoutError: pass return returncode is not None @staticmethod async def check_task_requirements(runtime_task): """Check the runtime task requirements needed to be able to run""" # right now, limit the check to the runner availability. if runtime_task.task.runnable.runner_command() is None: return False return True @staticmethod async def update_requirement_cache(runtime_task, result): kind = runtime_task.task.runnable.kind name = runtime_task.task.runnable.kwargs.get("name") cache.set_requirement(ENVIRONMENT_TYPE, ENVIRONMENT, kind, name) if result in STATUSES_NOT_OK: cache.delete_requirement(ENVIRONMENT_TYPE, ENVIRONMENT, kind, name) return cache.update_requirement_status(ENVIRONMENT_TYPE, ENVIRONMENT, kind, name, True) @staticmethod async def is_requirement_in_cache(runtime_task): kind = runtime_task.task.runnable.kind name = runtime_task.task.runnable.kwargs.get("name") return cache.is_requirement_in_cache(ENVIRONMENT_TYPE, ENVIRONMENT, kind, name) @staticmethod async def save_requirement_in_cache(runtime_task): kind = runtime_task.task.runnable.kind name = runtime_task.task.runnable.kwargs.get("name") cache.set_requirement(ENVIRONMENT_TYPE, ENVIRONMENT, kind, name, False) avocado-113.0/avocado/plugins/sysinfo.py000066400000000000000000000222471513561306000203020ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2013-2014 # Author: Lucas Meneghel Rodrigues """ System information plugin """ from avocado.core import sysinfo from avocado.core.nrunner.runnable import Runnable from avocado.core.plugin_interfaces import ( CLICmd, Init, JobPostTests, JobPreTests, PostTest, PreTest, ) from avocado.core.settings import settings from avocado.core.teststatus import STATUSES_NOT_OK from avocado.core.utils.path import prepend_base_path, system_wide_or_base_path from avocado.utils import path class SysinfoInit(Init): name = "sysinfo" description = "Initializes sysinfo settings" def initialize(self): help_msg = ( "Enable or disable sysinfo collection (like hardware " "details, profiles, etc.)" ) settings.register_option( section="sysinfo.collect", key="enabled", default=True, key_type=bool, help_msg=help_msg, ) help_msg = ( "Enable or disable sysinfo collection (like hardware " "details, profiles, etc.) for each test" ) settings.register_option( section="sysinfo.collect", key="per_test", default=False, key_type=bool, help_msg=help_msg, ) help_msg = ( "Overall timeout to collect commands, when <=0 no timeout is enforced" ) settings.register_option( section="sysinfo.collect", key="commands_timeout", key_type=int, default=-1, help_msg=help_msg, ) help_msg = ( "Whether to take a list of installed packages previous to avocado jobs" ) settings.register_option( section="sysinfo.collect", key="installed_packages", key_type=bool, default=False, help_msg=help_msg, ) help_msg = ( "Whether to run certain commands in bg to give extra job " "debug information" ) settings.register_option( section="sysinfo.collect", key="profiler", key_type=bool, default=False, help_msg=help_msg, ) help_msg = "Force LANG for sysinfo collection" settings.register_option( section="sysinfo.collect", key="locale", default="C", help_msg=help_msg ) help_msg = ( "Optimize sysinfo collected so that duplicates between pre " "and post re not stored in post" ) settings.register_option( section="sysinfo.collect", key="optimize", default=False, key_type=bool, help_msg=help_msg, ) help_msg = ( "File with list of commands that will be executed and " "have their output collected" ) default = system_wide_or_base_path("etc/avocado/sysinfo/commands") settings.register_option( section="sysinfo.collectibles", key="commands", key_type=prepend_base_path, default=default, help_msg=help_msg, ) help_msg = ( "File with list of commands that will be executed and " "have their output collected, in case of failed test" ) default = system_wide_or_base_path("etc/avocado/sysinfo/fail_commands") settings.register_option( section="sysinfo.collectibles", key="fail_commands", key_type=prepend_base_path, default=default, help_msg=help_msg, ) help_msg = "File with list of files that will be collected verbatim" default = system_wide_or_base_path("etc/avocado/sysinfo/files") settings.register_option( section="sysinfo.collectibles", key="files", key_type=prepend_base_path, default=default, help_msg=help_msg, ) help_msg = ( "File with list of files that will be collected verbatim" ", in case of failed test" ) default = system_wide_or_base_path("etc/avocado/sysinfo/fail_files") settings.register_option( section="sysinfo.collectibles", key="fail_files", key_type=prepend_base_path, default=default, help_msg=help_msg, ) help_msg = "File with list of commands that will run alongside the job/test" default = system_wide_or_base_path("etc/avocado/sysinfo/profilers") settings.register_option( section="sysinfo.collectibles", key="profilers", key_type=prepend_base_path, default=default, help_msg=help_msg, ) class SysInfoJob(JobPreTests, JobPostTests): name = "sysinfo" description = "Collects system information before/after the job is run" def __init__(self, config): # pylint: disable=W0231 self.sysinfo = None self.sysinfo_enabled = config.get("sysinfo.collect.enabled") def _init_sysinfo(self, job_logdir): if self.sysinfo is None: basedir = path.init_dir(job_logdir, "sysinfo") self.sysinfo = sysinfo.SysInfo(basedir=basedir) def pre_tests(self, job): if not self.sysinfo_enabled: return self._init_sysinfo(job.logdir) self.sysinfo.start() def post_tests(self, job): if not self.sysinfo_enabled: return self._init_sysinfo(job.logdir) self.sysinfo.end() class SysInfoTest(PreTest, PostTest): """Implements the sysinfo pre/post test plugin. It will create pre/post-test tasks for collecting system information. """ name = "sysinfo" description = "Collects system information before/after the test is run." def _is_sysinfo_enabled(self, config): if not ( config.get("sysinfo.collect.enabled") and config.get("sysinfo.collect.per_test") ): return False return True def pre_test_runnables(self, test_runnable, suite_config=None): suite_config = suite_config or {} if not self._is_sysinfo_enabled(suite_config): return [] sysinfo_config = sysinfo.gather_collectibles_config(suite_config) return [ Runnable( "sysinfo", "pre", config=suite_config, name="pre", sysinfo=sysinfo_config, output_dir=test_runnable.output_dir, ) ] def post_test_runnables(self, test_runnable, suite_config=None): suite_config = suite_config or {} if not self._is_sysinfo_enabled(suite_config): return [] sysinfo_config = sysinfo.gather_collectibles_config(suite_config) return [ ( Runnable( "sysinfo", "post", config=suite_config, name="post", sysinfo=sysinfo_config, test_fail=False, output_dir=test_runnable.output_dir, ), ["pass"], ), ( Runnable( "sysinfo", "post", config=suite_config, name="post", sysinfo=sysinfo_config, test_fail=True, output_dir=test_runnable.output_dir, ), [status.lower() for status in STATUSES_NOT_OK], ), ] class SysInfo(CLICmd): """ Collect system information """ name = "sysinfo" description = "Collect system information" def configure(self, parser): """ Add the subparser for the run action. :param parser: The Avocado command line application parser :type parser: :class:`avocado.core.parser.ArgumentParser` """ parser = super().configure(parser) help_msg = ( "Directory where Avocado will dump sysinfo data. If one " "is not given explicitly, it will default to a directory " 'named "sysinfo-" followed by a timestamp in the current ' "working directory." ) settings.register_option( section="sysinfo.collect", key="sysinfodir", default=None, help_msg=help_msg, parser=parser, positional_arg=True, nargs="?", ) def run(self, config): sysinfodir = config.get("sysinfo.collect.sysinfodir") sysinfo.collect_sysinfo(sysinfodir) avocado-113.0/avocado/plugins/tap.py000066400000000000000000000153531513561306000173740ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat, Inc. 2016 # Author: Lukas Doktor """ TAP output module. """ import os from avocado.core.output import LOG_UI from avocado.core.parser import FileOrStdoutAction from avocado.core.plugin_interfaces import CLI, Init, ResultEvents from avocado.core.settings import settings def file_log_factory(log_file): """ Generates a function which simulates writes to logger and outputs to file :param log_file: The output file """ def writeln(msg, *writeargs): """ Format msg and append '\n' """ if writeargs: try: msg %= writeargs except TypeError as details: raise TypeError(f"{details}: msg='{msg}' args='{writeargs}'") ret = log_file.write(msg + "\n") return ret return writeln class TAPResult(ResultEvents): """ TAP output class """ name = "tap" description = "TAP - Test Anything Protocol results" def __init__(self, config): # pylint: disable=W0613, W0231 self.__logs = [] self.__open_files = [] self.config = config output = self.config.get("job.run.result.tap.output") if output == "-": log = LOG_UI.debug self.__logs.append(log) elif output is not None: log = open(output, "w", 1, encoding="utf-8") self.__open_files.append(log) self.__logs.append(file_log_factory(log)) self.__include_logs = self.config.get("job.run.result.tap.include_logs") self.is_header_printed = False def __write(self, msg, *writeargs): """ Pass through the message to all registered log functions """ for log in self.__logs: log(msg, *writeargs) def __flush(self): """ Force-flush the output to opened files. """ for opened_file in self.__open_files: opened_file.flush() os.fsync(opened_file) def pre_tests(self, job): """ Log the test plan """ # Should we add default results.tap? if self.config.get("job.run.result.tap.enabled"): log = open( os.path.join(job.logdir, "results.tap"), "w", 1, encoding="utf-8" ) self.__open_files.append(log) self.__logs.append(file_log_factory(log)) def start_test(self, result, state): pass def end_test(self, result, state): """ Log the test status and details """ if not self.is_header_printed: self.__write("1..%d", result.tests_total) self.__flush() self.is_header_printed = True status = state.get("status", "ERROR") name = state.get("name") if not name: name = "Unknown" else: name = name.name + name.str_variant name.replace("#", "_") # Name must not contain # if name[0].isdigit(): # Name must not start with digit name = "_" + name # First log the system output if self.__include_logs: self.__write("# debug.log of %s:", name) with open(state.get("logfile"), "r", encoding="utf-8") as logfile_obj: for line in logfile_obj: self.__write("# %s", line.rstrip()) # Then the status if status == "PASS": self.__write("ok %s %s", result.tests_run, name) elif status == "WARN": self.__write( "ok %s %s # Warnings were printed into warn log", result.tests_run, name, ) elif status in ("SKIP", "CANCEL"): self.__write( "ok %s %s # SKIP %s", result.tests_run, name, state.get("fail_reason") ) else: self.__write("not ok %s %s", result.tests_run, name) self.__flush() def test_progress(self, progress=False): pass def post_tests(self, job): if job.interrupted_reason is not None: for pending_test in range( job.result.tests_run + 1, job.result.tests_total + 1 ): self.__write("ok %s # SKIP %s", pending_test, job.interrupted_reason) self.__flush() for open_file in self.__open_files: open_file.close() class TAPInit(Init): name = "TAP" description = "TAP - Test Anything Protocol - result plugin initialization" def initialize(self): section = "job.run.result.tap" help_msg = ( "Enable TAP result output and write it to FILE. Use " '"-" to redirect to standard output.' ) settings.register_option( section=section, key="output", help_msg=help_msg, default=None ) help_msg = ( "Enables default TAP result in the job results directory. " 'File will be named "results.tap"' ) settings.register_option( section=section, key="enabled", key_type=bool, default=True, help_msg=help_msg, ) help_msg = "Include test logs as comments in TAP output." settings.register_option( section=section, key="include_logs", default=False, key_type=bool, help_msg=help_msg, ) class TAP(CLI): """ TAP Test Anything Protocol output avocado plugin """ name = "TAP" description = "TAP - Test Anything Protocol results" def configure(self, parser): cmd_parser = parser.subcommands.choices.get("run", None) if cmd_parser is None: return settings.add_argparser_to_option( namespace="job.run.result.tap.output", metavar="FILE", action=FileOrStdoutAction, parser=cmd_parser, long_arg="--tap", ) settings.add_argparser_to_option( namespace="job.run.result.tap.enabled", parser=cmd_parser, long_arg="--disable-tap-job-result", ) settings.add_argparser_to_option( namespace="job.run.result.tap.include_logs", parser=cmd_parser, long_arg="--tap-include-logs", ) def run(self, config): pass avocado-113.0/avocado/plugins/testlogs.py000066400000000000000000000104771513561306000204560ustar00rootroot00000000000000import json import os from avocado.core.output import LOG_JOB, LOG_UI from avocado.core.plugin_interfaces import Init, JobPost, JobPre, ResultEvents from avocado.core.settings import settings from avocado.core.teststatus import STATUSES, STATUSES_NOT_OK class TestLogsUIInit(Init): description = "Initialize testlogs plugin settings" def initialize(self): help_msg = ( f"Status that will trigger the output of a test's logs " f"after the job ends. " f"Valid statuses: {', '.join(STATUSES)}" ) settings.register_option( section="job.output.testlogs", key="statuses", key_type=list, default=[], help_msg=help_msg, ) help_msg = ( "The specific log files that will be shown for tests " "whose exit status match the ones defined in the " '"job.output.testlogs.statuses" configuration. ' ) settings.register_option( section="job.output.testlogs", key="logfiles", key_type=list, default=["debug.log"], help_msg=help_msg, ) help_msg = ( f"Status that will trigger the output of a summary " f"after the job ends. This is useful to list failed " f"tests for instances at the end of a job run. " f"Valid statuses: {', '.join(STATUSES)}" ) settings.register_option( section="job.output.testlogs", key="summary_statuses", key_type=list, default=STATUSES_NOT_OK, help_msg=help_msg, ) class TestLogsUI(JobPre, JobPost): description = "Shows content from tests' logs" def pre(self, job): pass def post(self, job): statuses = job.config.get("job.output.testlogs.statuses") summary = job.config.get("job.output.testlogs.summary_statuses") if job.config.get("stdout_claimed_by") is not None: return try: with open( os.path.join(job.logdir, "results.json"), encoding="utf-8" ) as json_file: results = json.load(json_file) except FileNotFoundError: return logfiles = job.config.get("job.output.testlogs.logfiles") summary_tests = [] for test in results["tests"]: if test["status"] in summary: line = f"{test['id']}: {test['status']}" summary_tests.append(line) if not statuses or test["status"] not in statuses: continue for logfile in logfiles: path = os.path.join(test["logdir"], logfile) try: with open(path, encoding="utf-8") as log: LOG_UI.info( 'Log file "%s" content for test "%s" (%s):', logfile, test["id"], test["status"], ) LOG_UI.debug(log.read()) except (FileNotFoundError, PermissionError) as error: LOG_UI.error('Failure to access log file "%s": %s', path, error) if summary_tests: LOG_UI.info("") LOG_UI.info("Test summary:") for test in sorted(summary_tests): LOG_UI.error(test) class TestLogging(ResultEvents): """ TODO: The description should be changed when the legacy runner will be deprecated. """ description = "Nrunner specific Test logs for Job" def __init__(self, config): # pylint: disable=W0231 self.runner = config.get("run.test_runner") @staticmethod def _get_name(state): name = state.get("name") if name is None: return "" return name.name + name.str_variant def pre_tests(self, job): pass def post_tests(self, job): pass def start_test(self, result, state): LOG_JOB.info("%s: STARTED", self._get_name(state)) def test_progress(self, progress=False): pass def end_test(self, result, state): LOG_JOB.info("%s: %s", self._get_name(state), state.get("status", "ERROR")) LOG_JOB.info("More information in %s", state.get("task_path", "")) avocado-113.0/avocado/plugins/teststmpdir.py000066400000000000000000000027021513561306000211640ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat, Inc. 2016 # Author: Amador Pahim """ Tests temporary directory plugin """ import os import shutil import tempfile from avocado.core import test from avocado.core.plugin_interfaces import JobPost, JobPre class TestsTmpDir(JobPre, JobPost): name = "teststmpdir" description = "Creates a temporary directory for tests consumption" def __init__(self): # pylint: disable=W0231 self._varname = test.COMMON_TMPDIR_NAME self._dirname = None def pre(self, job): if os.environ.get(self._varname) is None: self._dirname = tempfile.mkdtemp(prefix="avocado_") os.environ[self._varname] = self._dirname def post(self, job): if self._dirname is not None and os.environ.get(self._varname) == self._dirname: try: shutil.rmtree(self._dirname) del os.environ[self._varname] except Exception: # pylint: disable=W0703 pass avocado-113.0/avocado/plugins/tmtresult.py000066400000000000000000000121501513561306000206430ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2025 # Authors: Lukas Kotek # Jan Richter """ TMT output module. """ import datetime import os import pathlib import time from avocado.core.output import LOG_UI from avocado.core.parser import FileOrStdoutAction from avocado.core.plugin_interfaces import CLI, Init, Result from avocado.core.settings import settings # This naming requires tmt version >= 1.29 yaml_template = """- name: /{name} result: {status} start-time: "{start}" end-time: "{end}" duration: {time} data-path: {data} {custom} """ class TMTResult(Result): name = "tmt" description = "TMT result support" @staticmethod def _allowed_status(status): """ Function matches avocado statuses with tmt ones """ status_match = { "SKIP": "skip", "ERROR": "error", "FAIL": "fail", "WARN": "warn", "PASS": "pass", "INTERRUPTED": "error", "CANCEL": "skip", "STARTED": "info", } if status in status_match: return status_match[status] else: return "error" @staticmethod def _render(result): content = "" data_path = os.path.dirname(result.logfile) for test in result.tests: tmt_result = {} tmt_result["name"] = test.get("name", "") # Avocado doesn't generate proper timestamps in json file tmt_result["start"] = datetime.datetime.fromtimestamp( test.get("actual_time_start", 0), tz=datetime.timezone.utc ).isoformat() tmt_result["end"] = datetime.datetime.fromtimestamp( test.get("actual_time_end", 0), tz=datetime.timezone.utc ).isoformat() tmt_result["time"] = time.strftime( "%H:%M:%S", time.gmtime(test.get("time_elapsed", 0)) ) tmt_result["status"] = TMTResult._allowed_status(test.get("status", {})) tmt_result["data"] = data_path tmt_result["custom"] = "" if test.get("logfile"): logfile_path = pathlib.Path(test.get("logfile")).relative_to(data_path) tmt_result["custom"] = f'log:\n{"- ":>4}{logfile_path}\n' content += f"{yaml_template.format(**tmt_result)}\n" return content def render(self, result, job): tmt_output = job.config.get("job.run.result.tmt.output") tmt_enabled = job.config.get("job.run.result.tmt.enabled") if not (tmt_enabled or tmt_output): return if not result.tests_total: return content = self._render(result) if tmt_enabled: tmt_path = os.path.join(job.logdir, "results.yaml") with open(tmt_path, "w", encoding="utf-8") as tmt_file: tmt_file.write(content) tmt_path = tmt_output if tmt_path is not None: if tmt_path == "-": LOG_UI.debug(content) else: with open(tmt_path, "w", encoding="utf-8") as tmt_file: tmt_file.write(content) class TMTInit(Init): name = "tmt" description = "TMT job result plugin initialization" def initialize(self): help_msg = ( "Enable TMT result format and write it to FILE. " 'Use "-" to redirect to the standard output.' ) settings.register_option( section="job.run.result.tmt", key="output", default=None, help_msg=help_msg ) help_msg = ( "Enables default TMT result in the job results " 'directory. File will be named "results.tmt".' ) settings.register_option( section="job.run.result.tmt", key="enabled", key_type=bool, default=True, help_msg=help_msg, ) class TMTCLI(CLI): """ TMT output """ name = "tmt" description = "TMT output options for 'run' command" def configure(self, parser): run_subcommand_parser = parser.subcommands.choices.get("run", None) if run_subcommand_parser is None: return settings.add_argparser_to_option( namespace="job.run.result.tmt.output", action=FileOrStdoutAction, metavar="FILE", parser=run_subcommand_parser, long_arg="--tmt", ) settings.add_argparser_to_option( namespace="job.run.result.tmt.enabled", parser=run_subcommand_parser, long_arg="--disable-tmt-job-result", ) def run(self, config): pass avocado-113.0/avocado/plugins/variants.py000066400000000000000000000141171513561306000204340ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2013-2014,2016 # Author: Lucas Meneghel Rodrigues # Author: Lukas Doktor import json import sys from avocado.core import exit_codes from avocado.core.output import LOG_UI from avocado.core.plugin_interfaces import CLICmd from avocado.core.settings import settings from avocado.core.varianter import Varianter _VERBOSITY_LEVELS = { "none": 0, "brief": 1, "normal": 2, "verbose": 3, "full": 4, "max": 99, } def map_verbosity_level(level): if level.isdigit(): return int(level) level = level.lower() if level in _VERBOSITY_LEVELS: return _VERBOSITY_LEVELS[level] else: raise ValueError class Variants(CLICmd): """ Implements "variants" command to visualize/debug test variants and params """ name = "variants" description = "Tool to analyze and visualize test variants and params" def configure(self, parser): parser = super().configure(parser) lvl = ", ".join(sorted(_VERBOSITY_LEVELS, key=lambda _: _VERBOSITY_LEVELS[_])) verbosity_levels = f"(positive integer - 0, 1, ... - or {lvl}" help_msg = "Verbosity of the variants summary. " + verbosity_levels settings.register_option( section="variants", key="summary", key_type=map_verbosity_level, default=0, help_msg=help_msg, parser=parser, long_arg="--summary", metavar="SUMMARY", ) help_msg = "Verbosity of the list of variants. " + verbosity_levels settings.register_option( section="variants", key="variants", key_type=map_verbosity_level, default=1, help_msg=help_msg, parser=parser, long_arg="--variants", metavar="VARIANTS", ) help_msg = "[obsoleted by --variants] Shows the node content (variables)" settings.register_option( section="variants", key="contents", key_type=bool, default=False, parser=parser, help_msg=help_msg, short_arg="-c", long_arg="--contents", ) help_msg = "Dump the Variants to a JSON serialized file" settings.register_option( section="variants", key="json_variants_dump", help_msg=help_msg, default=None, parser=parser, long_arg="--json-variants-dump", metavar="FILE", ) env_parser = parser.add_argument_group("environment view options") help_msg = "Use debug implementation to gather more information." settings.register_option( section="variants", key="debug", help_msg=help_msg, parser=env_parser, key_type=bool, default=False, long_arg="--debug", short_arg="-d", ) tree_parser = parser.add_argument_group("tree view options") help_msg = "[obsoleted by --summary] Shows the multiplex tree structure" settings.register_option( section="variants", key="tree", key_type=bool, default=False, help_msg=help_msg, parser=tree_parser, long_arg="--tree", short_arg="-t", ) help_msg = "[obsoleted by --summary] Show the inherited values" settings.register_option( section="variants", key="inherit", key_type=bool, help_msg=help_msg, parser=tree_parser, default=False, short_arg="-i", long_arg="--inherit", ) def run(self, config): tree = config.get("variants.tree") summary = config.get("variants.summary") variants = config.get("variants.variants") contents = config.get("variants.contents") inherit = config.get("variants.inherit") err = None if tree and config.get("variants.debug"): err = "Option --tree is incompatible with --debug." elif not tree and inherit: err = "Option --inherit can be only used with --tree" if err: LOG_UI.error(err) sys.exit(exit_codes.AVOCADO_FAIL) varianter = Varianter() try: varianter.parse(config) except (IOError, ValueError) as details: LOG_UI.error("Unable to parse varianter: %s", details) sys.exit(exit_codes.AVOCADO_FAIL) # Parse obsolete options (unsafe to combine them with new args) if tree: variants = 0 summary += 1 if contents: summary += 1 if inherit: summary += 2 else: if contents: variants += 2 json_variants_dump = config.get("variants.json_variants_dump") # Export the serialized variants if json_variants_dump is not None: try: with open(json_variants_dump, "w", encoding="utf-8") as variants_file: json.dump(varianter.dump(), variants_file) except IOError: LOG_UI.error("Cannot write %s", json_variants_dump) sys.exit(exit_codes.AVOCADO_FAIL) # Produce the output lines = varianter.to_str( summary=summary, variants=variants, use_utf8=sys.getdefaultencoding() == "utf-8", ) for line in lines.splitlines(): LOG_UI.debug(line) sys.exit(exit_codes.AVOCADO_ALL_OK) avocado-113.0/avocado/plugins/vmimage.py000066400000000000000000000146511513561306000202350ustar00rootroot00000000000000import json import os import re from avocado.core import exit_codes, output from avocado.core.output import LOG_UI, add_log_handler from avocado.core.plugin_interfaces import CLICmd from avocado.core.settings import settings from avocado.utils import astring, vmimage def list_downloaded_images(): """ List the available Image inside avocado cache :return: list with image's parameters :rtype: list of dicts """ images = [] for cache_dir in settings.as_dict().get("datadir.paths.cache_dirs"): for root, _, files in os.walk(cache_dir): if files: metadata_files = [ pos_json for pos_json in files if pos_json.endswith("_metadata.json") ] files = list(set(files) - set(metadata_files)) for metadata_file in metadata_files: with open( os.path.join(root, metadata_file), "r", encoding="utf-8" ) as data: metadata = json.loads(data.read()) if isinstance(metadata, dict): if metadata.get("type", None) == "vmimage": provider = None for p in vmimage.IMAGE_PROVIDERS: if p.name == metadata["name"]: provider = p( metadata["version"], metadata["build"], metadata["arch"], ) break if provider is not None: for image in files: if re.match(provider.file_name, image): data = { "name": provider.name, "version": provider.version, "arch": provider.arch, "file": os.path.join(root, image), } images.append(data) break return images def download_image(distro, version=None, arch=None): """ Downloads the vmimage to the cache directory if doesn't already exist :param distro: Name of image distribution :type distro: str :param version: Version of image :type version: str :param arch: Architecture of image :type arch: str :raise AttributeError: When image can't be downloaded :return: Information about downloaded image :rtype: dict """ cache_dir = settings.as_dict().get("datadir.paths.cache_dirs")[0] image_info = vmimage.Image.from_parameters( name=distro, version=version, arch=arch, cache_dir=cache_dir ) file_path = image_info.base_image image = { "name": distro, "version": image_info.version, "arch": image_info.arch, "file": file_path, } return image def display_images_list(images): """ Displays table with information about images :param images: list with image's parameters :type images: list of dicts """ image_matrix = [ [image["name"], image["version"], image["arch"], image["file"]] for image in images ] header = ( output.TERM_SUPPORT.header_str("Provider"), output.TERM_SUPPORT.header_str("Version"), output.TERM_SUPPORT.header_str("Architecture"), output.TERM_SUPPORT.header_str("File"), ) return astring.tabular_output(image_matrix, header=header, strip=True) class VMimage(CLICmd): """ Implements the avocado 'vmimage' subcommand """ name = "vmimage" description = "Provides VM images acquired from official repositories" def configure(self, parser): parser = super().configure(parser) subcommands = parser.add_subparsers(dest="vmimage_subcommand") subcommands.required = True subcommands.add_parser("list", help="List of all downloaded images") get_parser = subcommands.add_parser( "get", help="Downloads chosen VMimage if " "it's not already in the " "cache", ) help_msg = "Name of image distribution" settings.register_option( section="vmimage.get", key="distro", default=None, help_msg=help_msg, key_type=str, parser=get_parser, long_arg="--distro", required=True, ) help_msg = "Image version" settings.register_option( section="vmimage.get", key="version", default=None, help_msg=help_msg, key_type=str, parser=get_parser, long_arg="--distro-version", ) help_msg = "Image architecture" settings.register_option( section="vmimage.get", key="arch", default=None, help_msg=help_msg, key_type=str, parser=get_parser, long_arg="--arch", ) help_msg = "More information about failures" settings.register_option( section="vmimage.get", key="debug", default=False, help_msg=help_msg, key_type=bool, parser=get_parser, long_arg="--debug", ) def run(self, config): subcommand = config.get("vmimage_subcommand") if subcommand == "list": images = list_downloaded_images() LOG_UI.debug(display_images_list(images)) elif subcommand == "get": name = config.get("vmimage.get.distro") version = config.get("vmimage.get.version") arch = config.get("vmimage.get.arch") debug = config.get("vmimage.get.debug") if debug: add_log_handler(vmimage.LOG) try: image = download_image(name, version, arch) except AttributeError: LOG_UI.debug("The requested image could not be downloaded") return exit_codes.AVOCADO_FAIL LOG_UI.debug("The image was downloaded:") LOG_UI.debug(display_images_list([image])) return exit_codes.AVOCADO_ALL_OK avocado-113.0/avocado/plugins/xunit.py000066400000000000000000000236151513561306000177570ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2014 # Authors: Ruda Moura # Cleber Rosa """xUnit module.""" import datetime import os import string from xml.dom.minidom import Document from avocado.core.output import LOG_UI from avocado.core.parser import FileOrStdoutAction from avocado.core.plugin_interfaces import CLI, Init, Result from avocado.core.settings import settings from avocado.core.test_id import TestID from avocado.utils import astring from avocado.utils.data_structures import DataSize class XUnitResult(Result): name = "xunit" description = "XUnit result support" UNKNOWN = "" PRINTABLE = string.ascii_letters + string.digits + string.punctuation + "\n\r " def _escape_attr(self, attrib): attrib = "".join( _ if _ in self.PRINTABLE else f"\\x{ord(_):02x}" for _ in astring.to_text(attrib, encoding="utf-8") ) return attrib def _escape_cdata(self, cdata): cdata = "".join( _ if _ in self.PRINTABLE else f"\\x{ord(_):02x}" for _ in str(cdata) ) return cdata.replace("]]>", "]]>]]> avocado-113.0/avocado/utils/ar.py000066400000000000000000000072401513561306000166650ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2021 # Author: Cleber Rosa __doc__ = """ Module to read UNIX ar files """ import struct #: The first eight bytes of all AR archives MAGIC = b"!\n" #: The header for each file in the archive FILE_HEADER_FMT = "16s12s6s6s8s10s2c" FILE_HEADER_SIZE = struct.calcsize(FILE_HEADER_FMT) class ArMember: """A member of an UNIX ar archive.""" def __init__(self, identifier, size, offset): self.identifier = identifier self.size = size self.offset = offset def __repr__(self): return ( f'" ) class Ar: """An UNIX ar archive iterator. It reads ar archive file and iterates over the members in form of :class:`autils.archive.ar.ArMember` """ def __init__(self, path): self._path = path self._file = None self._position = None self._valid = False def __enter__(self): self._file = open(self._path, "rb") return self._file def __exit__(self, _exc_type, _exc_value, _traceback): self._file.close() def is_valid(self): """Checks if a file looks like an AR archive. :return: If file looks like an AR archive :rtype: bool """ with self as open_file: return open_file.read(8) == MAGIC def __iter__(self): self._position = 8 self._valid = self.is_valid() return self def __next__(self): if not self._valid: raise StopIteration with self as open_file: open_file.seek(self._position) try: member = struct.unpack( FILE_HEADER_FMT, open_file.read(FILE_HEADER_SIZE) ) except struct.error as exc: raise StopIteration from exc # No support for extended file names identifier = member[0].decode("ascii").strip() # from bytes containing a decimal to int size = int(member[5].decode("ascii").strip()) data_position = self._position + FILE_HEADER_SIZE # All data sections is aligned at 2 bytes data_position += data_position % 2 self._position += FILE_HEADER_SIZE + size return ArMember(identifier, size, data_position) def list(self): """List the members in the archive. :return: List the names of the members in the archive :rtype: list """ return [member.identifier for member in self] def read_member(self, identifier): """Reads the data for the given member identifier. :param identifier: Archive member name. :type identifier: str :return: data for the given member :rtype: bytes or None """ for member in self: if identifier == member.identifier: with self as open_file: open_file.seek(member.offset) return open_file.read(member.size) return None # pylint: disable=wrong-import-position from avocado.utils.deprecation import log_deprecation log_deprecation.warning("ar") avocado-113.0/avocado/utils/archive.py000066400000000000000000000434731513561306000177140ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2014 # Author: Ruda Moura """ Module to help extract and create compressed archives. """ import bz2 import gzip import logging import lzma import os import platform import shutil import stat import subprocess import tarfile import zipfile LOG = logging.getLogger(__name__) # Magic bytes for different archive formats MAGIC_BYTES = { "gzip": { "magic": b"\037\213", "description": "The first two bytes that all gzip files start with", }, "zstd": { "magic": b"\x28\xb5\x2f\xfd", "description": "The first bytes that all zstd files start with. See https://datatracker.ietf.org/doc/html/rfc8878#section-3.1.1-3.2", }, "bzip2": { "magic": b"BZh", "description": "The first three bytes that all bzip2 files start with", }, "xz": { "magic": b"\xfd\x37\x7a\x58\x5a\x00", "description": "The first six bytes that all xz files start with", }, } #: A valid zstd archive with "avocado\n" as content. Created with: #: echo "avocado" | zstd -c ZSTD_AVOCADO = ( MAGIC_BYTES["zstd"]["magic"] + b"\x04\x58\x41\x00\x00\x61\x76\x6f\x63\x61\x64\x6f\x0a\x3c\xfc\x9f\xb9" ) def _is_file_with_magic_bytes(path, magic_bytes): """ Checks if file given by path starts with the specified magic bytes :param path: Path to the file to check :param magic_bytes: The magic bytes to look for at the start of the file :return: True if the file starts with the magic bytes, False otherwise """ try: with open(path, "rb") as file_obj: return file_obj.read(len(magic_bytes)) == magic_bytes except (IOError, OSError): return False def is_gzip_file(path): """ Checks if file given by path has contents that suggests gzip file """ return _is_file_with_magic_bytes(path, MAGIC_BYTES["gzip"]["magic"]) def gzip_uncompress(path, output_path): """ Uncompress a gzipped file at path, to either a file or dir at output_path """ with gzip.GzipFile(filename=path, mode="rb") as input_file: if os.path.isdir(output_path): basename = os.path.basename(path) if basename.endswith(".gz"): basename = basename[:-3] output_path = os.path.join(output_path, basename) with open(output_path, "wb") as output_file: while True: chunk = input_file.read(4096) if not chunk: break output_file.write(chunk) return output_path def is_lzma_file(path): """ Checks if file given by path has contents that suggests lzma file """ # First try with magic bytes if _is_file_with_magic_bytes(path, MAGIC_BYTES["xz"]["magic"]): return True # If that fails, try the more thorough but slower method try: with lzma.LZMAFile(path, "rb") as lzma_file: try: _ = lzma_file.read(1) except EOFError: return False return True except (IOError, OSError, lzma.LZMAError): return False def _decide_on_path(path, suffix, output_path=None): if output_path is None: output_path = os.path.splitext(path)[0] elif os.path.isdir(output_path): basename = os.path.basename(path) if basename.endswith(suffix): basename = os.path.splitext(basename)[0] output_path = os.path.join(output_path, basename) return output_path def lzma_uncompress(path, output_path=None, force=False): """ Extracts a XZ compressed file to the same directory. """ output_path = _decide_on_path(path, ".xz", output_path) if not force and os.path.exists(output_path): return output_path with lzma.open(path, "rb") as file_obj: with open(output_path, "wb") as newfile_obj: newfile_obj.write(file_obj.read()) return output_path def is_zstd_file(path): """ Checks if file given by path has contents that suggests zstd file """ return _is_file_with_magic_bytes(path, MAGIC_BYTES["zstd"]["magic"]) def probe_zstd_cmd(): """ Attempts to find a suitable zstd tool that behaves as expected :rtype: str or None :returns: path to a suitable zstd executable or None if not found """ zstd_cmd = shutil.which("zstd") if zstd_cmd is not None: proc = subprocess.run( [zstd_cmd, "-d"], input=ZSTD_AVOCADO, stdout=subprocess.PIPE, stderr=subprocess.PIPE, check=False, ) if proc.returncode or proc.stdout != b"avocado\n": LOG.error("zstd command does not seem to be the Zstandard compression tool") return None return zstd_cmd return None def zstd_uncompress(path, output_path=None, force=False): """ Extracts a zstd compressed file. """ zstd_cmd = probe_zstd_cmd() if not zstd_cmd: raise ArchiveException("Unable to find a suitable zstd compression tool") output_path = _decide_on_path(path, ".zst", output_path) if not force and os.path.exists(output_path): return output_path proc = subprocess.run( [zstd_cmd, "-d", path, "-o", output_path], stdout=subprocess.PIPE, stderr=subprocess.PIPE, check=False, ) if proc.returncode: raise ArchiveException( f"Unable to decompress {path} into {output_path}: {proc.stderr}" ) return output_path def bzip2_uncompress(path, output_path=None, force=False): """ Extracts a bzip2 compressed file. """ output_path = _decide_on_path(path, ".bz2", output_path) if not force and os.path.exists(output_path): return output_path try: with bz2.open(path, "rb") as file_obj: with open(output_path, "wb") as newfile_obj: shutil.copyfileobj(file_obj, newfile_obj) except OSError as e: raise ArchiveException( f"Unable to decompress {path} into {output_path}: {e}" ) from e return output_path class ArchiveException(Exception): """ Base exception for all archive errors. """ class ArchiveFile: """ Class that represents an Archive file. Archives are ZIP files or Tarballs. """ # extension info: is_zip, is_tar, zipfile|tarfile, +mode _extension_table = { # ZIP archives ".zip": (True, False, zipfile.ZipFile, ""), # TAR archives (uncompressed) ".tar": (False, True, tarfile.open, ""), # TAR archives with gzip compression ".tar.gz": (False, True, tarfile.open, ":gz"), ".tgz": (False, True, tarfile.open, ":gz"), # TAR archives with bzip2 compression ".tar.bz2": (False, True, tarfile.open, ":bz2"), ".tbz2": (False, True, tarfile.open, ":bz2"), # TAR archives with xz compression ".tar.xz": (False, True, tarfile.open, ":xz"), ".txz": (False, True, tarfile.open, ":xz"), ".xz": (False, True, tarfile.open, ":xz"), # TAR archives with zstd compression ".tar.zst": (False, True, tarfile.open, ":zstd"), ".tzst": (False, True, tarfile.open, ":zstd"), # Standalone compressed files (not tar archives) ".gz": (False, False, None, "gz"), ".bz2": (False, False, None, "bz2"), ".zst": (False, False, None, "zst"), } def __init__(self, filename, mode="r"): """ Creates an instance of :class:`ArchiveFile`. :param filename: the archive file name. :param mode: file mode, `r` read, `w` write. """ self.filename = filename self.mode = mode engine = None self.is_zip = False self.is_tar = False self.is_compressed = False self.compression_type = None # First try to detect by extension for ext, value in ArchiveFile._extension_table.items(): if filename.endswith(ext): (self.is_zip, self.is_tar, engine, extra_mode) = value # Handle standalone compressed files if not self.is_zip and not self.is_tar and engine is None: self.is_compressed = True self.compression_type = extra_mode # For standalone compressed files, we'll use the appropriate # extraction function in the extract method self._engine = None return self.mode += extra_mode self._engine = engine(self.filename, self.mode) return # If extension detection fails, try content-based detection if mode == "r": # Only attempt content detection in read mode detected = self._detect_archive_type(filename) if detected: (self.is_zip, self.is_tar, engine, extra_mode) = detected self.mode += extra_mode self._engine = engine(self.filename, self.mode) return # If we get here, it's not a recognized archive raise ArchiveException("file is not an archive") def __repr__(self): return f"ArchiveFile('{self.filename}', '{self.mode}')" def __enter__(self): return self def __exit__(self, exc_type, exc_value, exc_traceback): if self._engine is not None: self.close() @classmethod def open(cls, filename, mode="r"): """ Creates an instance of :class:`ArchiveFile`. :param filename: the archive file name. :param mode: file mode, `r` read, `w` write. """ return cls(filename, mode) def add(self, filename, arcname=None): """ Add file to the archive. :param filename: file to archive. :param arcname: alternative name for the file in the archive. """ if self.is_zip: self._engine.write(filename, arcname, zipfile.ZIP_DEFLATED) else: self._engine.add(filename, arcname) def list(self): """ List files to the standard output. """ if self.is_zip: self._engine.printdir() else: self._engine.list() def extract(self, path="."): """ Extract all files from the archive. :param path: destination path. :return: the first member of the archive, a file or directory or None if the archive is empty """ # Handle standalone compressed files if self.is_compressed: if self.compression_type == "gz": return gzip_uncompress(self.filename, path) if self.compression_type == "bz2": return bzip2_uncompress(self.filename, path) if self.compression_type == "zst": return zstd_uncompress(self.filename, path) # Note: bz2 is handled by tarfile, xz by lzma_uncompress (called from top-level uncompress) # This check might be slightly redundant now but kept for safety. if self.compression_type not in ("gz", "zst"): raise ArchiveException( f"Unsupported standalone compression type for direct extraction: {self.compression_type}" ) # Handle regular archives (zip and tar) self._engine.extractall(path) if self.is_zip: self._update_zip_extra_attrs(path) files = self._engine.namelist() if files: return files[0].strip(os.sep) else: # is_tar files = self._engine.getnames() if files: return files[0] # If archive is empty (no files) return None @staticmethod def _detect_archive_type(filename): """ Detect archive type based on file content. :param filename: the archive file name. :return: tuple of (is_zip, is_tar, engine, extra_mode) or None if not an archive. """ result = None # Check for ZIP file if zipfile.is_zipfile(filename): result = (True, False, zipfile.ZipFile, "") # Check for TAR file elif tarfile.is_tarfile(filename): # Detect compression method for tar files if _is_file_with_magic_bytes(filename, MAGIC_BYTES["gzip"]["magic"]): result = (False, True, tarfile.open, ":gz") elif _is_file_with_magic_bytes(filename, MAGIC_BYTES["zstd"]["magic"]): result = (False, True, tarfile.open, ":zstd") elif _is_file_with_magic_bytes(filename, MAGIC_BYTES["xz"]["magic"]): result = (False, True, tarfile.open, ":xz") else: # Regular tar file result = (False, True, tarfile.open, "") # Check for standalone compressed files elif is_gzip_file(filename): result = (False, False, None, "gz") elif is_zstd_file(filename): result = (False, False, None, "zst") elif is_bzip2_file(filename): result = (False, False, None, "bz2") elif is_lzma_file(filename): result = (False, False, None, "xz") return result def _update_zip_extra_attrs(self, dst_dir): if platform.system() != "Linux": LOG.warning("Attr handling in zip files only supported on Linux.") return # Walk all files and re-create files as symlinks for path, info in self._engine.NameToInfo.items(): dst = os.path.join(dst_dir, path) if not os.path.exists(dst): LOG.warning( "One or more files in the ZIP archive '%s' could " "not be found after extraction. Their paths are " "probably stored in unsupported format and their " "attributes are not going to be updated", self.filename, ) return attr = info.external_attr >> 16 if attr & stat.S_IFLNK == stat.S_IFLNK: if not os.path.islink(dst): # Link created as an ordinary file containing the dst path with open(dst, "r") as dst_path: # pylint: disable=W1514 src = dst_path.read() else: # Link is already there and could be outdated. Let's read # the original destination from the zip file. src = self._engine.read(path) try: os.remove(dst) os.symlink(src, dst) except (OSError, FileNotFoundError) as e: LOG.warning("Failed to update symlink '%s': %s", dst, e) continue continue # Don't override any other attributes on links mode = attr & 511 # Mask only permissions if mode and mode != 436: # If mode is stored and is not default try: os.chmod(dst, mode) except (OSError, FileNotFoundError) as e: LOG.warning("Failed to update permissions for '%s'", e) def close(self): """ Close archive. """ self._engine.close() def is_bzip2_file(path): """ Checks if file given by path has contents that suggests bzip2 file """ return _is_file_with_magic_bytes(path, MAGIC_BYTES["bzip2"]["magic"]) def is_archive(filename): """ Test if a given file is an archive. :param filename: file to test. :return: `True` if it is an archive. """ return ( zipfile.is_zipfile(filename) or tarfile.is_tarfile(filename) or is_gzip_file(filename) or is_lzma_file(filename) or is_zstd_file(filename) or is_bzip2_file(filename) ) def compress(filename, path): """ Compress files in an archive. :param filename: archive file name. :param path: origin directory path to files to compress. No individual files allowed. """ with ArchiveFile.open(filename, "w") as x: if os.path.isdir(path): for root, _, files in os.walk(path): for name in files: newroot = root.replace(path, "") x.add(os.path.join(root, name), os.path.join(newroot, name)) elif os.path.isfile(path): x.add(path, os.path.basename(path)) def uncompress(filename, path): """ Extract files from an archive. :param filename: archive file name. :param path: destination path to extract to. """ is_tar = tarfile.is_tarfile(filename) if is_gzip_file(filename) and not is_tar: return gzip_uncompress(filename, path) if is_lzma_file(filename) and not is_tar: return lzma_uncompress(filename, path) if is_zstd_file(filename) and not is_tar: return zstd_uncompress(filename, path) if is_bzip2_file(filename) and not is_tar: return bzip2_uncompress(filename, path) try: with ArchiveFile.open(filename) as x: return x.extract(path) except ArchiveException as e: # If we get here but is_archive() returns True, we have an archive # that we can detect but not extract. Provide a more helpful error message. if is_archive(filename): raise ArchiveException( f"File '{filename}' is detected as an archive but its format " f"is not supported for extraction: {e}" ) from e raise # Some aliases create = compress extract = uncompress avocado-113.0/avocado/utils/asset.py000066400000000000000000000630371513561306000174100ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2016 # Author: Amador Pahim """ Asset fetcher from multiple locations """ import errno import hashlib import json import logging import operator import os import re import shutil import stat import sys import time import uuid from datetime import datetime from urllib.parse import urlparse from avocado.utils import astring, crypto from avocado.utils import path as utils_path from avocado.utils.download import url_download from avocado.utils.filelock import FileLock LOG = logging.getLogger(__name__) #: The default hash algorithm to use on asset cache operations DEFAULT_HASH_ALGORITHM = "sha1" #: The default timeout for the downloading of assets DOWNLOAD_TIMEOUT = 300 SUPPORTED_OPERATORS = { "==": operator.eq, "<": operator.lt, ">": operator.gt, "<=": operator.le, ">=": operator.ge, } class UnsupportedProtocolError(OSError): """ Signals that the protocol of the asset URL is not supported """ class Asset: """ Try to fetch/verify an asset file from multiple locations. """ # pylint: disable=R0913 def __init__( self, name=None, asset_hash=None, algorithm=None, locations=None, cache_dirs=None, expire=None, metadata=None, ): """Initialize the Asset() class. :param name: the asset filename. url is also supported. Default is ''. :param asset_hash: asset hash :param algorithm: hash algorithm :param locations: location(s) where the asset can be fetched from :param cache_dirs: list of cache directories :param expire: time in seconds for the asset to expire :param metadata: metadata which will be saved inside metadata file """ self.name = name or "" self.asset_hash = asset_hash if isinstance(locations, str): self.locations = [locations] else: self.locations = locations or [] if algorithm is None: self.algorithm = DEFAULT_HASH_ALGORITHM else: self.algorithm = algorithm self.cache_dirs = cache_dirs or [] self.expire = expire self.metadata = metadata def _create_hash_file(self, asset_path): """ Compute the hash of the asset file and add it to the CHECKSUM file. :param asset_path: full path of the asset file. """ result = crypto.hash_file(asset_path, algorithm=self.algorithm) hash_file = self._get_hash_file(asset_path) self._add_hash_to_hash_file(hash_file, result, self.algorithm) def _create_metadata_file(self, asset_file): """ Creates JSON file with metadata. The file will be saved as `asset_file`_metadata.json :param asset_file: The asset whose metadata will be saved :type asset_file: str """ if self.metadata is not None: basename = os.path.splitext(asset_file)[0] metadata_path = f"{basename}_metadata.json" with open(metadata_path, "w", encoding="utf-8") as metadata_file: json.dump(self.metadata, metadata_file) def _download(self, url_obj, asset_path, timeout=None): """ Download the asset from an uri. :param url_obj: object from urlparse. :param asset_path: full path of the asset file. :param timeout: timeout in seconds. Default is :data:`avocado.utils.asset.DOWNLOAD_TIMEOUT`. :returns: if the downloaded file matches the hash. :rtype: bool """ timeout = timeout or DOWNLOAD_TIMEOUT try: # Temporary unique name to use while downloading temp = f"{asset_path}.{str(uuid.uuid4())}" # To avoid parallel downloads of the same asset, and errors during # the write after download, let's get the lock before start the # download. with FileLock(asset_path, timeout): try: self.find_asset_file(create_metadata=True) return True except OSError: LOG.debug("Asset not in cache after lock, fetching it.") url_download(url_obj.geturl(), temp, timeout=timeout) shutil.copy(temp, asset_path) self._create_hash_file(asset_path) if not self._verify_hash(asset_path): msg = "Hash mismatch. Ignoring asset from the cache" raise OSError(msg) return True finally: try: os.remove(temp) except FileNotFoundError: LOG.info( "Temporary asset file unavailable due to failed" " download attempt." ) @staticmethod def _get_hash_file(asset_path): """ Returns the file name that contains the hash for a given asset file :param asset_path: full path of the asset file. :returns: the CHECKSUM path :rtype: str """ return f"{asset_path}-CHECKSUM" def _get_hash_from_file(self, asset_path): """ Read the CHECKSUM file from the asset and return the hash. :param asset_path: full path of the asset file. :returns: the hash, if it exists. :rtype: str """ hash_file = self._get_hash_file(asset_path) if not os.path.isfile(hash_file): self._create_hash_file(asset_path) return Asset.read_hash_from_file(hash_file, self.algorithm)[1] @classmethod def read_hash_from_file(cls, filename, algorithm=None): """Read the CHECKSUM file and return the hash. This method raises a FileNotFoundError if file is missing and assumes that filename is the CHECKSUM filename. :rtype: list with algorithm and hash """ try: with FileLock(filename, 30): with open(filename, "r", encoding="utf-8") as hash_file: algorithm = algorithm or DEFAULT_HASH_ALGORITHM for line in hash_file: # md5 is 32 chars big and sha512 is 128 chars big. # others supported algorithms are between those. if re.match(f"^{algorithm}\b.* [a-f0-9]{32,128}", line): return line.split() return [None, None] except Exception: # pylint: disable=W0703 exc_type, exc_value = sys.exc_info()[:2] LOG.error("%s: %s", exc_type.__name__, exc_value) return [None, None] @staticmethod def _add_hash_to_hash_file(hash_file, new_hash, algorithm): """ Adds new hash entry to the list inside hash file. :param hash_file: path to hash :param new_hash: hash value which will by added :param algorithm: algorithm which generated the new hash """ with FileLock(hash_file, 120): try: with open(hash_file, "r", encoding="utf-8") as fp: for line in fp: if algorithm == line.split()[0]: return except FileNotFoundError: pass with open(hash_file, "a", encoding="utf-8") as fp: fp.write(f"{algorithm} {new_hash}\n") def _get_local_file(self, url_obj, asset_path, _): """ Create a symlink for a local file into the cache. :param url_obj: object from urlparse. :param asset_path: full path of the asset file. :returns: if the local file matches the hash. :rtype: bool """ if os.path.isdir(url_obj.path): path = os.path.join(url_obj.path, self.name) else: path = url_obj.path with FileLock(asset_path, 1): try: os.symlink(path, asset_path) self._create_hash_file(asset_path) return self._verify_hash(asset_path) except OSError as detail: if detail.errno == errno.EEXIST: os.remove(asset_path) os.symlink(path, asset_path) self._create_hash_file(asset_path) return self._verify_hash(asset_path) return False def _get_relative_dir(self): """ When an asset name is not an URI, and: 1. it also has a hash; 2. or it has multiple locations; there's a clear intention for it to be unique *by name*, overwriting it if the file is corrupted or expired. These will be stored in the cache directory indexed by name. When an asset name is an URI, whether it has a hash or not, it will be saved according to their locations, so that multiple assets with the same file name, but completely unrelated to each other, will still coexist. :returns: target location of asset the file. :rtype: str """ if not self.name_scheme and (self.asset_hash or len(self.locations) > 1): return "by_name" # check if the URI is located on self.locations or self.parsed_name if self.locations: # if it is on self.locations, we need to check if it has the # asset name on it or a trailing '/' if (self.asset_name in self.locations[0]) or (self.locations[0][-1] == "/"): base_url = os.path.dirname(self.locations[0]) else: # here, self.locations is a pure conformant URI base_url = self.locations[0] else: # the URI is on self.parsed_name if self.parsed_name.query: base_url = ( f"{self.parsed_name.scheme}://" f"{self.parsed_name.netloc}" f"{self.parsed_name.path}" ) else: base_url = os.path.dirname(self.parsed_name.geturl()) base_url_hash = hashlib.new( DEFAULT_HASH_ALGORITHM, base_url.encode(astring.ENCODING) ) return os.path.join("by_location", base_url_hash.hexdigest()) def _get_writable_cache_dir(self): """ Returns the first available writable cache directory When a asset has to be downloaded, a writable cache directory is then needed. The first available writable cache directory will be used. :returns: the first writable cache dir :rtype: str :raises: OSError """ for cache_dir in self.cache_dirs: cache_dir = os.path.expanduser(cache_dir) if utils_path.usable_rw_dir(cache_dir): return cache_dir raise OSError("Can't find a writable cache directory.") @staticmethod def _is_expired(path, expire): """ Checks if a file is expired according to expired parameter. :param path: full path of the asset file. :returns: the expired status of an asset. :rtype: bool """ if expire is None: return False creation_time = os.lstat(path)[stat.ST_CTIME] expire_time = creation_time + expire if time.monotonic() > expire_time: return True return False @classmethod def _has_valid_hash(cls, asset_path, asset_hash=None, algorithm=None): """Checks if a file has a valid hash based on the hash parameter. If asset_hash is None then will consider a valid asset. """ if asset_hash is None: LOG.debug("No hash provided. Cannot check the asset file integrity.") return True if algorithm is None: algorithm = DEFAULT_HASH_ALGORITHM hash_path = cls._get_hash_file(asset_path) _, hash_from_file = cls.read_hash_from_file(hash_path, algorithm) if not hash_from_file: hash_from_file = crypto.hash_file(asset_path, algorithm=algorithm) if hash_from_file == asset_hash: cls._add_hash_to_hash_file(hash_path, asset_hash, algorithm) return True if hash_from_file == asset_hash: return True return False def _verify_hash(self, asset_path): """ Verify if the `asset_path` hash matches the hash in the hash file. :param asset_path: full path of the asset file. :returns: True when self.asset_hash is None or when it has the same value as the hash of the asset_file, otherwise return False. :rtype: bool """ return self._has_valid_hash(asset_path, self.asset_hash, self.algorithm) def fetch(self, timeout=None): """Try to fetch the current asset. First tries to find the asset on the provided cache_dirs list. Then tries to download the asset from the locations list provided. :param timeout: timeout in seconds. Default is :data:`avocado.utils.asset.DOWNLOAD_TIMEOUT`. :raise OSError: When it fails to fetch the asset :returns: The path for the file on the cache directory. :rtype: str """ # First let's search for the file in each one of the cache locations asset_file = None error = "Can't fetch: 'urls' is not defined." timeout = timeout or DOWNLOAD_TIMEOUT LOG.info("Fetching asset %s", self.name) try: return self.find_asset_file(create_metadata=True) except OSError: LOG.info("Asset not in cache, fetching it.") # If we get to this point, we have to download it from a location. # A writable cache directory is then needed. The first available # writable cache directory will be used. cache_dir = self._get_writable_cache_dir() # Now we have a writable cache_dir. Let's get the asset. for url in self.urls: if url is None: continue urlobj = urlparse(url) if urlobj.scheme in ["http", "https", "ftp"]: fetch = self._download elif urlobj.scheme == "file": fetch = self._get_local_file # We are assuming that everything starting with './' or '/' are a # file too. elif url.startswith(("/", "./")): fetch = self._get_local_file else: raise UnsupportedProtocolError( f"Unsupported protocol: " f"{urlobj.scheme}" ) asset_file = os.path.join(cache_dir, self.relative_dir) dirname = os.path.dirname(asset_file) if not os.path.isdir(dirname): os.makedirs(dirname, exist_ok=True) try: if fetch(urlobj, asset_file, timeout): LOG.info("Asset downloaded.") if self.metadata is not None: self._create_metadata_file(asset_file) return asset_file except Exception: # pylint: disable=W0703 exc_type, exc_value = sys.exc_info()[:2] LOG.error("%s: %s", exc_type.__name__, exc_value) error = exc_value raise OSError(f"Failed to fetch {self.asset_name} ({error}).") def find_asset_file(self, create_metadata=False): """ Search for the asset file in each one of the cache locations. It handles both normal asset names and direct file paths. :param bool create_metadata: Should this method create the metadata in case asset file found and metadata is not found? Default is False. :return: asset path, if it exists in the cache :rtype: str :raises: OSError """ # Try 1: Check if self.name is already a direct file path if self.name and os.path.isfile(self.name): return self.name # Try 2: Search in cache directories using relative_dir for cache_dir in self.cache_dirs: cache_dir = os.path.expanduser(cache_dir) asset_file = os.path.join(cache_dir, self.relative_dir) # Ignore non-files if not os.path.isfile(asset_file): continue # Ignore expired asset files if self._is_expired(asset_file, self.expire): continue # Ignore mismatch hash if not self._has_valid_hash(asset_file, self.asset_hash, self.algorithm): continue if create_metadata: self._create_metadata_file(asset_file) LOG.info("Asset already exists in cache.") return asset_file raise OSError(f"File {self.asset_name} not found in the cache.") def get_metadata(self): """ Returns metadata of the asset if it exists or None. It handles both normal asset names and direct file paths. :return: metadata :rtype: dict or None """ try: asset_file = self.find_asset_file() except OSError as exc: LOG.debug("Asset file not found, metadata not available for %s", self.name) raise OSError("Metadata not available.") from exc basename = os.path.splitext(asset_file)[0] metadata_file = f"{basename}_metadata.json" if os.path.isfile(metadata_file): try: with open(metadata_file, "r", encoding="utf-8") as f: metadata = json.load(f) return metadata except json.JSONDecodeError as e: LOG.warning("Invalid JSON in metadata file %s: %s", metadata_file, e) raise except OSError as e: LOG.warning("Cannot read metadata file %s: %s", metadata_file, e) raise return None @property def asset_name(self): if self.parsed_name.query: return self.parsed_name.query return os.path.basename(self.parsed_name.path) @classmethod def get_all_assets(cls, cache_dirs, sort=True): """Returns all assets stored in all cache dirs.""" assets = [] for cache_dir in cache_dirs: expanded = os.path.expanduser(cache_dir) for root, _, files in os.walk(expanded): for f in files: if not f.endswith("-CHECKSUM") and not f.endswith("_metadata.json"): assets.append(os.path.join(root, f)) if sort: assets = {a: os.stat(a).st_atime for a in assets} return [ a[0] for a in sorted(assets.items(), key=lambda x: x[1], reverse=True) ] return assets @classmethod def get_asset_by_name(cls, name, cache_dirs, expire=None, asset_hash=None): """This method will return a cached asset based on name if exists. You don't have to instantiate an object of Asset class. Just use this method. To be improved soon: cache_dirs should be not necessary. :param name: the asset filename used during registration. :param cache_dirs: list of directories to use during the search. :param expire: time in seconds for the asset to expire. Expired assets will not be returned. :param asset_hash: asset hash. :return: asset path, if it exists in the cache. :rtype: str :raises: OSError """ for cache_dir in cache_dirs: asset_file = os.path.join(os.path.expanduser(cache_dir), "by_name", name) # Ignore non-files if not os.path.isfile(asset_file): continue # Ignore expired asset files if cls._is_expired(asset_file, expire): continue # Ignore mismatch hash if not cls._has_valid_hash(asset_file, asset_hash): continue return asset_file raise OSError(f"File {name} not found in the cache.") @classmethod def get_assets_unused_for_days(cls, days, cache_dirs): """Return a list of all assets in cache based on the access time. This will check if the file's data wasn't modified N days ago. :param days: how many days ago will be the threshold. Ex: "10" will return the assets files that *was not* accessed during the last 10 days. :param cache_dirs: list of directories to use during the search. """ result = [] for file_path in cls.get_all_assets(cache_dirs): stats = os.stat(file_path) diff = datetime.now() - datetime.fromtimestamp(stats.st_atime) if diff.days >= days: result.append(file_path) return result @classmethod def get_assets_by_size(cls, size_filter, cache_dirs): """Return a list of all assets in cache based on its size in MB. :param size_filter: a string with a filter (comparison operator + value). Ex ">20", "<=200". Supported operators: ==, <, >, <=, >=. :param cache_dirs: list of directories to use during the search. """ try: op = re.match("^(\\D+)(\\d+)$", size_filter).group(1) value = int(re.match("^(\\D+)(\\d+)$", size_filter).group(2)) except (AttributeError, ValueError) as exc: msg = ( "Invalid syntax. You need to pass an comparison operatator", " and a value. Ex: '>=200'", ) raise OSError(msg) from exc try: method = SUPPORTED_OPERATORS[op] except KeyError as exc: msg = ( "Operator not supported. Currented valid values are: ", ", ".join(SUPPORTED_OPERATORS), ) raise OSError(msg) from exc result = [] for file_path in cls.get_all_assets(cache_dirs): file_size = os.path.getsize(file_path) if method(file_size, value): result.append(file_path) return result @classmethod def remove_assets_by_overall_limit(cls, limit, cache_dirs): """This will remove assets based on overall limit. We are going to sort the assets based on the access time first. For instance it may be the case that a GitLab cache limit is 4 GiB, in that case we can sort by last access, and remove all that exceeds 4 GiB (that is, keep the last accessed 4 GiB worth of cached files). Note: during the usage of this method, you should use bytes as limit. :param limit: a integer limit in bytes. :param cache_dirs: list of directories to use during the search. """ size_sum = 0 for asset in cls.get_all_assets(cache_dirs): size_sum += os.stat(asset).st_size if size_sum >= limit: cls.remove_asset_by_path(asset) @classmethod def remove_assets_by_size(cls, size_filter, cache_dirs): for file_path in cls.get_assets_by_size(size_filter, cache_dirs): cls.remove_asset_by_path(file_path) @classmethod def remove_assets_by_unused_for_days(cls, days, cache_dirs): for file_path in cls.get_assets_unused_for_days(days, cache_dirs): cls.remove_asset_by_path(file_path) @property def name_scheme(self): """This property will return the scheme part of the name if is an URL. Otherwise, will return None. """ parsed = self.parsed_name if parsed: return parsed.scheme return None @property def name_url(self): """This property will return the full url of the name if is an URL. Otherwise, will return None. """ if self.name_scheme: return self.parsed_name.geturl() return None @staticmethod def parse_name(name): """Returns a ParseResult object for the given name.""" return urlparse(name) @property def parsed_name(self): """Returns a ParseResult object for the currently set name.""" return self.parse_name(self.name) @property def relative_dir(self): return os.path.join(self._get_relative_dir(), self.asset_name) @classmethod def remove_asset_by_path(cls, asset_path): """Remove an asset and its checksum. To be fixed: Due the current implementation limitation, this method will not remove the metadata to avoid removing other asset metadata. :param asset_path: full path of the asset file. """ try: os.remove(asset_path) filename = f"{asset_path}-CHECKSUM" os.remove(filename) except FileNotFoundError: LOG.error("File not found: %s or its checksum file.", asset_path) except OSError as e: LOG.error("An error occurred while removing files: %s", e) @property def urls(self): """Complete list of locations including name if is an URL.""" urls = [] if self.name_scheme: urls.append(self.name_url) if self.locations: urls.extend(self.locations) return urls avocado-113.0/avocado/utils/astring.py000066400000000000000000000413471513561306000177400ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2013-2014 # Authors: Lucas Meneghel Rodrigues """Operations with strings (conversion and sanitation). The unusual name aims to avoid causing name clashes with the stdlib module string. Even with the dot notation, people may try to do things like import string ... from avocado.utils import string And not notice until their code starts failing. """ import itertools import locale import re from avocado.utils import path #: On import evaluated value representing the system encoding #: based on system locales using :func:`locale.getpreferredencoding`. #: Use this value wisely as some files are dumped in different #: encoding. ENCODING = locale.getpreferredencoding() #: String containing all fs-unfriendly chars (Windows-fat/Linux-ext3) FS_UNSAFE_CHARS = '<>:"/\\|?*;' # Translate table to replace fs-unfriendly chars _FS_TRANSLATE = bytes.maketrans(bytes(FS_UNSAFE_CHARS, "ascii"), b"__________") def bitlist_to_string(data): """Transform from bit list to ASCII string. Converts a list of bits to an ASCII string representation. Only complete bytes (8 bits) are processed; partial bytes are ignored. :param data: List of integers representing bits to be transformed :type data: list[int] :returns: ASCII string representation of the bit list :rtype: str :raises UnicodeDecodeError: If the resulting byte values are not valid ASCII .. note:: Only processes complete bytes. If the bit list length is not a multiple of 8, the remaining bits are ignored. .. rubric:: Example >>> bitlist_to_string([0, 1, 0, 0, 0, 0, 0, 1]) # 'A' = 65 'A' >>> bitlist_to_string([1, 0, 0, 0]) # Incomplete byte '' """ result = bytearray() c = 0 for pos, bit in enumerate(data): c |= bit << (7 - (pos % 8)) if (pos % 8) == 7: result.append(c) c = 0 return result.decode("ascii") def string_to_bitlist(data): """Transform from ASCII string to bit list. Converts each character in the string to its 8-bit binary representation and returns a flat list of all bits. :param data: ASCII string to be transformed to bit list :type data: str :returns: List of integers representing the bits of each character :rtype: list[int] .. note:: Each character produces exactly 8 bits, with the most significant bit first. .. rubric:: Example >>> string_to_bitlist('A') # 'A' = 65 = 01000001 [0, 1, 0, 0, 0, 0, 0, 1] >>> string_to_bitlist('AB') [0, 1, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 1, 0] """ ord_ = ord result = [] append = result.append for ch in data: ascii_value = ord_(ch) for i in range(7, -1, -1): append((ascii_value >> i) & 1) return result def shell_escape(command): """Escape special characters from a command so that it can be passed as a double quoted (" ") string in a (ba)sh command. Escapes backslashes, dollar signs, double quotes, and backticks that have special meaning in bash when inside double quotes. :param command: The command string to escape :type command: str :returns: The escaped command string safe for shell execution :rtype: str """ escape_chars = {"\\": "\\\\", "$": r"\$", '"': r"\"", "`": r"\`"} for char, escaped_char in escape_chars.items(): command = command.replace(char, escaped_char) return command def strip_console_codes(output, custom_codes=None): """Remove Linux console escape and control sequences from console output. Removes ANSI escape sequences and other console control codes to make the output readable and suitable for result checking. Handles common codes used during system boot and terminal color formatting. :param output: The console output string containing escape sequences :type output: str :param custom_codes: Additional regex patterns for codes not covered by the default patterns. Will be added to the built-in console codes regex. :type custom_codes: str or None :returns: Clean string with all console escape sequences removed :rtype: str :raises ValueError: If unknown console codes are encountered that don't match the known patterns .. note:: If the output doesn't contain ``\\x1b`` (ESC character), the original string is returned unchanged for performance. .. rubric:: Supported Console Codes * ANSI color codes: ``\\x1b[31m``, ``\\x1b[0m``, etc. * Cursor positioning: ``\\x1b[H``, ``\\x1b[2J``, etc. * Character set selection: ``\\x1b(B``, ``\\x1b(0``, etc. * Custom codes via the ``custom_codes`` parameter .. rubric:: Example >>> strip_console_codes('\\x1b[31mRed Text\\x1b[0m') 'Red Text' >>> strip_console_codes('Normal text') 'Normal text' """ if "\x1b" not in output: return output old_word = "" return_str = "" index = 0 output = f"\x1b[m{output}" console_codes = "%[G@8]|\\[[@A-HJ-MPXa-hl-nqrsu\\`]" console_codes += "|\\[[\\d;]+[HJKgqnrm]|#8|\\([B0UK]|\\)" if custom_codes is not None and custom_codes not in console_codes: console_codes += f"|{custom_codes}" while index < len(output): tmp_index = 0 tmp_word = "" while len(re.findall("\x1b", tmp_word)) < 2 and index + tmp_index < len(output): tmp_word += output[index + tmp_index] tmp_index += 1 tmp_word = re.sub("\x1b", "", tmp_word) index += len(tmp_word) + 1 if tmp_word == old_word: continue try: special_code = re.findall(console_codes, tmp_word)[0] except IndexError as exc: if index + tmp_index < len(output): raise ValueError( f"{tmp_word} is not included in the known " f"console codes list {console_codes}" ) from exc continue if special_code == tmp_word: continue old_word = tmp_word return_str += tmp_word[len(special_code) :] return return_str def iter_tabular_output(matrix, header=None, strip=False): """Generator for a pretty, aligned string representation of a nxm matrix. This representation can be used to print any tabular data, such as database results. It works by scanning the lengths of each element in each column, and determining the format string dynamically. :param matrix: Matrix representation (list with n rows of m elements). :type matrix: list :param header: Optional tuple or list with header elements to be displayed. :type header: tuple or list or None :param strip: Optionally remove trailing whitespace from each row. :type strip: bool :returns: Generator yielding each formatted row of the tabular output :rtype: generator of str """ lengths = [] len_matrix = [] str_matrix = [] if header: matrix = itertools.chain([header], matrix) for row in matrix: len_matrix.append([]) str_matrix.append([string_safe_encode(column) for column in row]) for i, column in enumerate(str_matrix[-1]): col_len = len(strip_console_codes(column)) len_matrix[-1].append(col_len) try: max_len = lengths[i] if col_len > max_len: lengths[i] = col_len except IndexError: lengths.append(col_len) # For different no cols we need to calculate `lengths` of the last item # but later in `yield` we don't want it in `len_matrix` len_matrix[-1] = len_matrix[-1][:-1] if strip: def str_out(x): """Join list elements and strip trailing whitespace. :param x: List of string elements to join :type x: list :returns: Joined string with trailing whitespace removed :rtype: str """ return " ".join(x).rstrip() else: def str_out(x): """Join list elements with spaces. :param x: List of string elements to join :type x: list :returns: Space-separated string of elements :rtype: str """ return " ".join(x) for row, row_lens in zip(str_matrix, len_matrix): out = [] padding = [" " * (lengths[i] - row_lens[i]) for i in range(len(row_lens))] out = ["%s%s" % line for line in zip(row, padding)] # pylint: disable=C0209 try: out.append(row[-1]) except IndexError: continue # Skip empty rows yield str_out(out) def tabular_output(matrix, header=None, strip=False): """Pretty, aligned string representation of a matrix. Creates a single formatted string with column-aligned tabular data, suitable for printing or logging. This is a convenience wrapper around :func:`iter_tabular_output` that joins all rows with newlines. :param matrix: Matrix representation as list of rows, where each row is a list of column elements. Rows may have different lengths. :type matrix: list[list] :param header: Optional header row elements to be displayed at the top. If provided, will be formatted with the same column alignment. :type header: list or tuple or None :param strip: If True, removes trailing whitespace from each output row :type strip: bool :returns: Complete formatted table as a single string with newline separators :rtype: str .. rubric:: Example >>> matrix = [['Alice', '25', 'Engineer'], ['Bob', '30', 'Designer']] >>> print(tabular_output(matrix, header=['Name', 'Age', 'Role'])) Name Age Role Alice 25 Engineer Bob 30 Designer .. seealso:: :func:`iter_tabular_output` for the underlying generator implementation """ return "\n".join(iter_tabular_output(matrix, header, strip)) def string_safe_encode(input_str): """Safely convert any input to a string representation. Handles mixed unicode and encoded strings by ensuring all input is converted to a proper string type. In Python 3, this primarily serves to convert non-string types (numbers, objects) to strings. :param input_str: Input value that needs to be converted to string. Can be string, numeric, or any object with __str__. :type input_str: Any :returns: String representation of the input :rtype: str .. note:: On Python 3, encoding/decoding is handled automatically by the language, so this function focuses on type conversion rather than encoding management. .. rubric:: Supported Input Types * Strings: returned as-is * Numbers: converted using ``str()`` * Objects: converted using their ``__str__`` method * None: converted to ``'None'`` .. rubric:: Example >>> string_safe_encode('hello') 'hello' >>> string_safe_encode(42) '42' >>> string_safe_encode([1, 2, 3]) '[1, 2, 3]' """ if not isinstance(input_str, str): input_str = str(input_str) return input_str def string_to_safe_path(input_str): """Convert string to a filesystem-safe filename or directory name. Sanitizes strings for use as filenames by replacing characters that are not allowed on common filesystems (FAT, NTFS, ext3/4) with underscores. Also handles length limits and hidden file conventions. :param input_str: String to be converted to a safe filename :type input_str: str :returns: Filesystem-safe string suitable for use as filename or directory name :rtype: str .. rubric:: Transformations Applied * Replaces unsafe characters with underscores: ``< > : " / \\ | ? * ;`` * Limits length to filesystem maximum (typically 255 characters) * Converts hidden files (starting with ``.``) to start with ``_`` * Handles Unicode characters that may cause encoding issues .. rubric:: Cross-Platform Compatibility The function ensures compatibility with: * **Windows**: FAT32, NTFS filesystems * **Linux**: ext3, ext4 filesystems * **macOS**: HFS+, APFS filesystems .. rubric:: Example >>> string_to_safe_path('my file: .txt') 'my file_ _test_.txt' >>> string_to_safe_path('.hidden_file') '_hidden_file' >>> string_to_safe_path('very_long_filename' * 20) # Too long 'very_long_filename...' # Truncated to max length .. seealso:: :data:`FS_UNSAFE_CHARS` for the complete list of replaced characters """ max_length = path.get_max_file_name_length(input_str) if input_str.startswith("."): input_str = "_" + input_str[1:max_length] elif len(input_str) > max_length: input_str = input_str[:max_length] try: return input_str.translate(_FS_TRANSLATE) except TypeError: # Deal with incorrect encoding for bad_chr in FS_UNSAFE_CHARS: input_str = input_str.replace(bad_chr, "_") return input_str def is_bytes(data): """Check if the given data is a bytes object. Determines whether the input is specifically a ``bytes`` type, as opposed to a text string or other sequence type. This is useful for encoding/decoding operations and type-specific processing. :param data: The data instance to check :type data: Any :returns: True if data is a bytes object, False otherwise :rtype: bool .. note:: This function specifically checks for the ``bytes`` type, not ``bytearray`` or other byte-like sequences. .. rubric:: Example >>> is_bytes(b'hello') True >>> is_bytes('hello') False >>> is_bytes(bytearray(b'hello')) False """ return isinstance(data, bytes) def is_text(data): """Check if the given data is a text string. Determines whether the input is a string type capable of holding Unicode text with multi-byte characters, as opposed to a bytes sequence or other data type. :param data: The data instance to check :type data: Any :returns: True if data is a text string, False otherwise :rtype: bool .. note:: In Python 3, this checks for the ``str`` type, which is Unicode-capable. .. rubric:: Example >>> is_text('hello') True >>> is_text(b'hello') False >>> is_text(42) False """ return isinstance(data, str) def to_text(data, encoding=ENCODING, errors="strict"): """Convert any input to a text string. Universal text conversion function that handles bytes, strings, and other object types. Ensures consistent text output regardless of input type while preserving encoding semantics. :param data: Data to be converted to text string :type data: bytes or str or Any :param encoding: Character encoding to use when decoding bytes. Uses system default if None. :type encoding: str or None :param errors: Error handling scheme for decoding failures. See Python's codec error handlers. :type errors: str :returns: Text representation of the input data :rtype: str :raises UnicodeDecodeError: When bytes cannot be decoded with the specified encoding and errors='strict' .. rubric:: Conversion Logic 1. **bytes input**: Decoded using specified encoding 2. **str input**: Returned unchanged 3. **Other types**: Converted using ``str()`` function .. rubric:: Error Handling Options * ``'strict'``: Raise exception on decode errors (default) * ``'ignore'``: Skip invalid characters * ``'replace'``: Replace invalid characters with ``\\ufffd`` * ``'backslashreplace'``: Replace with backslash escape sequences .. rubric:: Example >>> to_text(b'hello', 'utf-8') 'hello' >>> to_text('already text') 'already text' >>> to_text(42) '42' >>> to_text(b'\xff', 'utf-8', errors='ignore') '' .. seealso:: `Python Codec Error Handlers `_ """ if is_bytes(data): if encoding is None: encoding = ENCODING return data.decode(encoding, errors=errors) if not isinstance(data, str): return str(data) return data # pylint: disable=wrong-import-position from avocado.utils.deprecation import log_deprecation log_deprecation.warning("astring") avocado-113.0/avocado/utils/aurl.py000066400000000000000000000024371513561306000172310ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2013-2014 # Author: Lucas Meneghel Rodrigues """ URL related functions. The strange name is to avoid accidental naming collisions in code. """ import urllib.parse #: The most common schemes (protocols) used in URLs COMMON_SCHEMES = ("http", "https", "ftp", "git") def is_url(path): """Return `True` if path looks like an URL of common protocols. Refer to :data:`COMMON_SCHEMES` for the list of common protocols. :param path: path to check. :rtype: Boolean. """ url_parts = urllib.parse.urlparse(path) return url_parts[0] in COMMON_SCHEMES # pylint: disable=wrong-import-position from avocado.utils.deprecation import log_deprecation log_deprecation.warning( "aurl", "The aurl utility is deprecated and will be removed after the next LTS release.", ) avocado-113.0/avocado/utils/build.py000066400000000000000000000073571513561306000173730ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2013-2014 # Author: Lucas Meneghel Rodrigues import multiprocessing import os from avocado.utils import process def configure(path, configure=None): # pylint: disable=W0621 """ Configures the source tree for a subsequent build Most source directories coming from official released tarballs will have a "configure" script, but source code snapshots may have "autogen.sh" instead (which usually creates and runs a "configure" script itself). This function will attempt to run the first one found (if a configure script name not given explicitly). :param configure: the name of the configure script (None for trying to find one automatically) :type configure: str or None :returns: the configure script exit status, or None if no script was found and executed """ cwd = os.getcwd() try: os.chdir(path) if configure is not None: return process.run(os.path.join(path, configure)).exit_status candidates = ["autogen.sh", "configure"] for candidate in candidates: if os.access(candidate, os.R_OK | os.X_OK): return process.run(os.path.join(path, candidate)).exit_status finally: os.chdir(cwd) return None def run_make( path, make="make", extra_args="", process_kwargs=None ): # pylint: disable=W0621 """ Run make, adding MAKEOPTS to the list of options. :param path: directory from where to run make :param make: what make command name to use. :param extra_args: extra command line arguments to pass to make. :param process_kwargs: Additional key word arguments to the underlying process running the make. :returns: the make command result object """ cwd = os.getcwd() os.chdir(path) cmd = make if process_kwargs is None: process_kwargs = {} # Set default number of jobs as ncpus + 1 os_makeflags = os.environ.get("MAKEFLAGS", "") if process_kwargs.get("env") is None: process_kwargs["env"] = {} args_makeflags = process_kwargs["env"].get("MAKEFLAGS", "") if "-j" not in os_makeflags and "-j" not in args_makeflags: args_makeflags += f" -j{multiprocessing.cpu_count() + 1}" process_kwargs["env"].update({"MAKEFLAGS": args_makeflags}) makeopts = os.environ.get("MAKEOPTS", "") if makeopts: cmd += f" {makeopts}" if extra_args: cmd += f" {extra_args}" make_process = process.run(cmd, **process_kwargs) os.chdir(cwd) return make_process # pylint: disable=W0621, R0913 def make( path, make="make", env=None, extra_args="", ignore_status=None, process_kwargs=None, ): """ Run make, adding MAKEOPTS to the list of options. :param make: what make command name to use. :param env: dictionary with environment variables to be set before calling make (e.g.: CFLAGS). :param extra: extra command line arguments to pass to make. :returns: exit status of the make process """ kwargs = {"env": env, "ignore_status": ignore_status} if process_kwargs is not None: kwargs.update(process_kwargs) result = run_make(path, make, extra_args, kwargs) return result.exit_status avocado-113.0/avocado/utils/cloudinit.py000066400000000000000000000201011513561306000202440ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2018 # Author: Cleber Rosa """ cloudinit configuration support This module can be easily used with :mod:`avocado.utils.vmimage`, to configure operating system images via the cloudinit tooling. Please, keep in mind that if you would like to create/write in ISO images, you need pycdlib module installed in your environment. :see: http://cloudinit.readthedocs.io. """ import warnings from http.server import BaseHTTPRequestHandler, HTTPServer from avocado.utils import astring, iso9660 #: The meta-data file template #: #: Positional template variables are: instance-id, hostname METADATA_TEMPLATE = """instance-id: {0} hostname: {1} """ #: The header expected to be found at the beginning of the user-data file USERDATA_HEADER = "#cloud-config" #: A username configuration as per cloudinit/config/cc_set_passwords.py #: #: Positional template variables : username USERNAME_TEMPLATE = """ ssh_pwauth: True system_info: default_user: name: {0} """ #: A username configuration as per cloudinit/config/cc_set_passwords.py #: #: Positional template variables are: password PASSWORD_TEMPLATE = """ password: {0} chpasswd: expire: False """ #: An authorized key configuration for the default user #: #: Positional template variables are: ssh_authorized_keys AUTHORIZED_KEY_TEMPLATE = """ ssh_authorized_keys: - {0} """ #: A phone home configuration that will post just the instance id #: #: Positional template variables are: address, port PHONE_HOME_TEMPLATE = """ phone_home: url: http://{0}:{1}/$INSTANCE_ID/ post: [ instance_id ] """ # pylint: disable=R0913 def iso( output_path, instance_id, username=None, password=None, phone_home_host=None, phone_home_port=None, authorized_key=None, ): """ Generates an ISO image with cloudinit configuration The content always include the cloudinit metadata, and optionally the userdata content. On the userdata file, it may contain a username/password section (if both parameters are given) and/or a phone home section (if both host and port are given). :param output_path: the location of the resulting (to be created) ISO image containing the cloudinit configuration :type output_path: str :param instance_id: the ID of the cloud instance, a form of identification for the dynamically created executing instances :type instance_id: str :param username: the username to be used when logging interactively on the instance :type username: str :param password: the password to be used along with username when authenticating with the login services on the instance :type password: str :param phone_home_host: the address of the host the instance should contact once it has finished booting :type phone_home_host: str :param phone_home_port: the port acting as an HTTP phone home server that the instance should contact once it has finished booting :type phone_home_port: str :param authorized_key: a SSH public key to be added as an authorized key for the default user, similar to "ssh-rsa ..." :type authorized_key: str :raises: RuntimeError if the system can not create ISO images. On such a case, user is expected to install supporting packages, such as pycdlib. """ # The only supported method to create/write in an ISO today is via pycdlib out = iso9660.iso9660(output_path, ["create", "write"]) if out is None: msg = ( "The system lacks support for creating ISO images. ", "Please install pycdlib dependency and run again.", ) raise RuntimeError(msg) out.create(flags={"interchange_level": 3, "joliet": 3, "vol_ident": "cidata"}) metadata = METADATA_TEMPLATE.format(instance_id, instance_id).encode( astring.ENCODING ) out.write("/meta-data", metadata) userdata = USERDATA_HEADER if username: userdata += USERNAME_TEMPLATE.format(username) if username == "root": userdata += "\ndisable_root: False\n" if password: userdata += PASSWORD_TEMPLATE.format(password) if authorized_key: userdata += AUTHORIZED_KEY_TEMPLATE.format(authorized_key) if phone_home_host and phone_home_port: userdata += PHONE_HOME_TEMPLATE.format(phone_home_host, phone_home_port) out.write("/user-data", userdata.encode(astring.ENCODING)) out.close() class PhoneHomeServerHandler(BaseHTTPRequestHandler): """Handles HTTP requests to the phone home server.""" def do_POST(self): # pylint: disable=C0103 """Handles an HTTP POST request. Respond with status 200 if the instance phoned back. """ path = self.path[1:].rstrip("/") if path == self.server.instance_id: self.server.instance_phoned_back = True self.send_response(200) self.end_headers() def log_message(self, format_, *args): # pylint: disable=W0221 """Logs an arbitrary message. :note: It currently disables any message logging. """ class PhoneHomeServer(HTTPServer): """Implements the phone home HTTP server. Wait the phone home from a given instance. """ def __init__(self, address, instance_id): """Initialize the server. :param address: a hostname or IP address and port, in the same format given to socket and other servers :type address: tuple :param instance_id: the identification for the instance that should be calling back, and the condition for the wait to end :type instance_id: str """ HTTPServer.__init__(self, address, PhoneHomeServerHandler) self.instance_id = instance_id self.instance_phoned_back = False def wait_for_phone_home(self, new_call=False): """Waits for this instance to call. :param new_call: Default is False, so if this instance was called back already, this method will return immediately and will not wait for a new call. :type new_call: bool """ if new_call: self.instance_phoned_back = False while not self.instance_phoned_back: self.handle_request() @classmethod def set_up_and_wait_for_phone_home(cls, address, instance_id): """ Sets up a phone home server and waits for the given instance to call This is a shorthand for setting up a server that will keep handling requests, until it has heard from the specific instance requested. :param address: a hostname or IP address and port, in the same format given to socket and other servers :type address: tuple :param instance_id: the identification for the instance that should be calling back, and the condition for the wait to end :type instance_id: str """ s = cls(address, instance_id) s.wait_for_phone_home() def wait_for_phone_home(address, instance_id): """ This method is deprecated. Please use :meth:`.PhoneHomeServer.set_up_and_wait_for_phone_home`. """ warnings.warn( ( "wait_for_phone_home is deprecated. Please use " "PhoneHomeServer.set_up_and_wait_for_phone_home()" ), DeprecationWarning, ) PhoneHomeServer.set_up_and_wait_for_phone_home(address, instance_id) avocado-113.0/avocado/utils/cpu.py000066400000000000000000000523721513561306000170600ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # This code was inspired in the autotest project, # # client/base_utils.py # Original author: Martin J Bligh # Original author: John Admanski """ Get information from the current's machine CPU. """ import glob import logging import os import platform import random import re import warnings from avocado.utils import genio, process #: Map vendor's name with expected string in /proc/cpuinfo. VENDORS_MAP = { "intel": (b"GenuineIntel",), "amd": (b"AMD",), "ibm": ( rb"POWER\d", rb"IBM/S390", rb"Power\d", ), } LOG = logging.getLogger(__name__) class FamilyException(Exception): pass def _list_matches(content_list, pattern): """ Checks if any item in list matches the specified pattern. :param content_list: items to match :type content_list: list :param pattern: pattern to match from content_list :type pattern: str :return: if the pattern was found or not :rtype: bool """ for content in content_list: match = re.search(pattern, content) if match: return True return False def _get_info(): """ Returns info on the 1st CPU entry from /proc/cpuinfo as a list of lines. :return: `list` of lines 1st CPU entry from /proc/cpuinfo file :rtype: list """ cpuinfo = [] with open("/proc/cpuinfo", "rb") as proc_cpuinfo: # pylint: disable=W1514 for line in proc_cpuinfo: if line == b"\n" and len(cpuinfo) > 0: break cpuinfo.append(line) return cpuinfo def _get_status(cpu): """ Check if a CPU is online or offline. :param cpu: CPU number (e.g. 1, 2 or 39) :type cpu: int :return: `bool` True if online or False if not :rtype: bool """ with open( f"/sys/devices/system/cpu/cpu{cpu}/online", "rb" ) as cpu_online: # pylint: disable=W1514 if b"1" in cpu_online.read(): return True return False def cpu_has_flags(flags): """ Check if a list of flags are available on current CPU info. :param flags: A `list` of cpu flags that must exists on the current CPU. :type flags: list of str :return: True if all the flags were found or False if not :rtype: bool """ cpu_info = _get_info() if not isinstance(flags, list): flags = [flags] for flag in flags: if not any(flag.encode() in line for line in cpu_info): return False return True def get_version(): """ Get cpu version. :return: cpu version of given machine e.g.:- 'i5-5300U' for Intel and 'POWER9' for IBM machines in case of unknown/unsupported machines, return an empty string. :rtype: str """ version_pattern = { "x86_64": rb"\s([\S,\d]+)\sCPU", "i386": rb"\s([\S,\d]+)\sCPU", "powerpc": rb"revision\s+:\s+(\S+)", "s390": rb".*machine\s=\s(\d+)", } cpu_info = _get_info() arch = get_arch() pattern = version_pattern.get(arch) if not pattern: LOG.warning("No pattern string for arch: %s", arch) return "" for line in cpu_info: version_out = re.findall(pattern, line) if version_out: return version_out[0].decode("utf-8") return "" def get_vendor(): """ Get the current cpu vendor name. :return: a key of :data:`VENDORS_MAP` (e.g. 'intel') depending on the current CPU architecture. Return None if it was unable to determine the vendor name. :rtype: str or None """ cpu_info = _get_info() for vendor, identifiers in VENDORS_MAP.items(): for identifier in identifiers: if _list_matches(cpu_info, identifier): return vendor return None def get_revision(): """ Get revision from /proc/cpuinfo :return: revision entry from /proc/cpuinfo file e.g.:- '0080' for IBM POWER10 machine :rtype: str """ rev = None proc_cpuinfo = genio.read_file("/proc/cpuinfo") for line in proc_cpuinfo.splitlines(): if "revision" in line: rev = line.split(" ")[3].strip() return rev def get_va_bits(): """ Check for VA address bit size in /proc/cpuinfo :return: VA address bit size :rtype: str """ cpu_info = genio.read_file("/proc/cpuinfo") for line in cpu_info.splitlines(): if "address sizes" in line: return line.split()[-3].strip() return "" def get_arch(): """Work out which CPU architecture we're running on.""" cpu_table = [ (b"^cpu.*(RS64|Broadband Engine)", "powerpc"), (rb"^cpu.*POWER\d+", "powerpc"), (rb"^cpu.*Power\d+", "powerpc"), (b"^cpu.*PPC970", "powerpc"), ( b"(ARM|^CPU implementer|^CPU part|^CPU variant" b"|^Features|^BogoMIPS|^CPU revision)", "arm", ), ( b"(^cpu MHz dynamic|^cpu MHz static|^features" b"|^bogomips per cpu|^max thread id)", "s390", ), (b"^type", "sparc64"), (b"^flags.*:.* lm .*", "x86_64"), (b"^flags", "i386"), (b"^hart\\s*: 1$", "riscv"), ] cpuinfo = _get_info() for pattern, arch in cpu_table: if _list_matches(cpuinfo, pattern): if arch != "arm": return arch # ARM is a special situation, which matches both 32 bits # (v7) and 64 bits (v8). for line in cpuinfo: if line.startswith(b"CPU architecture"): version = int(line.split(b":", 1)[1]) if version >= 8: return "aarch64" # For historical reasons return arm return "arm" return platform.machine() def get_family(): """Get family name of the cpu like Broadwell, Haswell, power8, power9.""" family = None arch = get_arch() if arch in ("x86_64", "i386"): if get_vendor() == "amd": cpu_info = _get_info() pattern = r"cpu family\s*:" for line in cpu_info: line = line.decode("utf-8") if re.search(pattern, line): family = int(line.split(":")[1]) return family try: # refer below links for microarchitectures names # https://en.wikipedia.org/wiki/List_of_Intel_CPU_microarchitectures # https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/x86/events/intel/core.c#n4613 with open( "/sys/devices/cpu/caps/pmu_name", "rb" ) as mico_arch: # pylint: disable=W1514 family = mico_arch.read().decode("utf-8").strip("\n").lower() except FileNotFoundError as err: msg = f"Could not find micro-architecture/family, Error: {err}" LOG.warning(msg) raise FamilyException(msg) from err elif arch == "powerpc": res = [] try: for line in _get_info(): res = re.findall(rb"cpu\s+:\s+(POWER\d+|Power\d+)", line, re.IGNORECASE) if res: break family = res[0].decode("utf-8").lower() except IndexError as err: msg = f"Unable to parse cpu family {err}" LOG.warning(msg) raise FamilyException(msg) from err elif arch == "s390": zfamily_map = {"2964": "z13", "3906": "z14", "8561": "z15", "3931": "z16"} try: family = zfamily_map[get_version()].lower() except KeyError as err: msg = f"Could not find family for {get_version()}\nError: {err}" LOG.warning(msg) raise FamilyException(msg) from err else: raise NotImplementedError return family def get_model(): """ Get model of cpu """ arch = get_arch() if arch == "x86_64": cpu_info = _get_info() pattern = r"model\s*:" for line in cpu_info: line = line.decode("utf-8") if re.search(pattern, line): model = int(line.split(":")[1]) return model return None raise NotImplementedError def get_x86_amd_zen(family=None, model=None): """ Get the AMD Zen architecture's version of the x86_AMD CPU :param family: AMD family :type family: int :param model: AMD model :type model: int :return: AMD Zen :rtype: int """ x86_amd_zen = { 0x17: { 1: [(0x00, 0x2F), (0x50, 0x5F)], 2: [(0x30, 0x4F), (0x60, 0x7F), (0x90, 0x91), (0xA0, 0xAF)], }, 0x19: {3: [(0x00, 0x0F), (0x20, 0x5F)], 4: [(0x10, 0x1F), (0x60, 0xAF)]}, 0x1A: { 5: [(0x00, 0x0F), (0x20, 0x2F), (0x40, 0x4F), (0x60, 0x7F)], 6: [(0x50, 0x5F), (0x80, 0xAF), (0xC0, 0xCF)], }, } if not family: family = get_family() if not model: model = get_model() for _family, _zen_model in x86_amd_zen.items(): if _family == family: for _zen, _model in _zen_model.items(): if any(lower <= model <= upper for (lower, upper) in _model): return _zen return None def online_list(): """Reports a list of indexes of the online cpus.""" cpus = [] search_str = b"processor" index = 2 if platform.machine() == "s390x": search_str = b"cpu number" index = 3 with open("/proc/cpuinfo", "rb") as proc_cpuinfo: # pylint: disable=W1514 for line in proc_cpuinfo: if line.startswith(search_str): cpus.append(int(line.split()[index])) # grab cpu number return cpus def total_count(): """Return Number of Total cpus in the system including offline cpus.""" return os.sysconf("SC_NPROCESSORS_CONF") def online_count(): """Return Number of Online cpus in the system.""" return os.sysconf("SC_NPROCESSORS_ONLN") def is_hotpluggable(cpu): return os.path.exists(f"/sys/devices/system/cpu/cpu{cpu}/online") def online(cpu): """Online given CPU.""" if _get_status(cpu) is False: with open( f"/sys/devices/system/cpu/cpu{cpu}/online", "wb" ) as fd: # pylint: disable=W1514 fd.write(b"1") if _get_status(cpu) is False: return 0 return 1 def offline(cpu): """Offline given CPU.""" if _get_status(cpu): with open( f"/sys/devices/system/cpu/cpu{cpu}/online", "wb" ) as fd: # pylint: disable=W1514 fd.write(b"0") if _get_status(cpu): return 1 return 0 def get_idle_state(): """ Get current cpu idle values. :return: Dict of cpuidle states values for all cpus :rtype: dict """ cpus_list = online_list() states = len(glob.glob("/sys/devices/system/cpu/cpu0/cpuidle/state*")) cpu_idlestate = {} for cpu in cpus_list: cpu_idlestate[cpu] = {} for state_no in range(states): state_file = ( f"/sys/devices/system/cpu/cpu{cpu}/cpuidle/state{state_no}/disable" ) try: with open(state_file, "rb") as fl: # pylint: disable=W1514 cpu_idlestate[cpu][state_no] = bool(int(fl.read())) except IOError as err: LOG.warning( "Failed to read idle state on cpu %s for state %s:\n%s", cpu, state_no, err, ) return cpu_idlestate def _bool_to_binary(value): """ Turns a Python boolean value (True or False) into binary data. This function is suitable for writing to /proc/* and /sys/* files. """ if value is True: return b"1" if value is False: return b"0" raise TypeError(f"Value is not a boolean: {value}") def set_idle_state(state_number="all", disable=True, setstate=None): """ Set/Reset cpu idle states for all cpus. :param state_number: cpuidle state number, default: `all` all states :type state_number: str :param disable: whether to disable/enable given cpu idle state, default is to disable. :type disable: bool :param setstate: cpuidle state value, output of `get_idle_state()` :type setstate: dict """ cpus_list = online_list() if not setstate: states = [] if state_number == "all": states = list( range(len(glob.glob("/sys/devices/system/cpu/cpu0/cpuidle/state*"))) ) else: states.append(state_number) disable = _bool_to_binary(disable) for cpu in cpus_list: for state_no in states: state_file = ( f"/sys/devices/system/cpu/cpu{cpu}/cpuidle/state{state_no}/disable" ) try: with open(state_file, "wb") as fl: # pylint: disable=W1514 fl.write(disable) except IOError as err: LOG.warning( "Failed to set idle state on cpu %s for state %s:\n%s", cpu, state_no, err, ) else: for cpu, stateval in setstate.items(): for state_no, value in stateval.items(): state_file = ( f"/sys/devices/system/cpu/cpu{cpu}/cpuidle/state{state_no}/disable" ) disable = _bool_to_binary(value) try: with open(state_file, "wb") as fl: # pylint: disable=W1514 fl.write(disable) except IOError as err: LOG.warning( "Failed to set idle state on cpu %s for state %s:\n%s", cpu, state_no, err, ) def set_freq_governor(governor="random"): """ To change the given cpu frequency governor. :param governor: frequency governor profile name whereas `random` is default option to choose random profile among available ones. :type governor: str """ avl_gov_file = "/sys/devices/system/cpu/cpu0/cpufreq/scaling_available_governors" cur_gov_file = "/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor" cur_gov = get_freq_governor() if not cur_gov: return False if not (os.access(avl_gov_file, os.R_OK) and os.access(cur_gov_file, os.W_OK)): LOG.error( "Could not locate frequency governor sysfs entries or\n" " No proper permissions to read/write sysfs entries" ) return False cpus_list = total_count() with open(avl_gov_file, "r") as fl: # pylint: disable=W1514 avl_govs = fl.read().strip().split(" ") if governor == "random": avl_govs.remove(cur_gov) if not avl_govs: LOG.error("No other frequency governors to pick from...") return False governor = random.choice(avl_govs) if governor not in avl_govs: LOG.warning("Trying to change unknown frequency governor: %s", governor) for cpu in range(cpus_list): cur_gov_file = f"/sys/devices/system/cpu/cpu{cpu}/cpufreq/scaling_governor" try: with open(cur_gov_file, "w") as fl: # pylint: disable=W1514 fl.write(governor) except IOError as err: LOG.warning( "Unable to write a given frequency " "governor %s profile for cpu " "%s\n %s", governor, cpu, err, ) return True def get_freq_governor(): """Get current cpu frequency governor.""" cur_gov_file = "/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor" try: with open(cur_gov_file, "r") as fl: # pylint: disable=W1514 return fl.read().strip() except IOError as err: LOG.error("Unable to get the current governor\n %s", err) return "" def get_pid_cpus(pid): """ Get all the cpus being used by the process according to pid informed. :param pid: process id :type pid: str :return: A list include all cpus the process is using :rtype: list """ # processor id index is defined according proc documentation # the negative index is necessary because backward data # access has no misleading whitespaces processor_id_index = -14 cpus = set() proc_stat_files = glob.glob(f"/proc/{pid}/task/[123456789]*/stat") for proc_stat_file in proc_stat_files: try: with open(proc_stat_file) as proc_stat: # pylint: disable=W1514 cpus.add(proc_stat.read().split(" ")[processor_id_index]) except IOError: continue return list(cpus) def get_numa_node_has_cpus(): """ Get the list NUMA node numbers which has CPU's on the system, if there is no CPU associated to NUMA node,Those NUMA node number will not be appended to list. :return: A list where NUMA node numbers only which has CPU's - as elements of The list. :rtype: List """ cpu_path = "/sys/devices/system/node/has_cpu" delim = ",", "-" regex_pat = "|".join(map(re.escape, delim)) read_cpu_path = genio.read_file(cpu_path).rstrip("\n") nodes_with_cpus = re.split(regex_pat, read_cpu_path) return nodes_with_cpus def numa_nodes_with_assigned_cpus(): """ Get NUMA nodes with associated CPU's on the system. :return: A dictionary,in which "NUMA node numbers" as key and "NUMA node associated CPU's" as values. :rtype: dictionary """ numa_nodes_with_cpus = {} for path in glob.glob("/sys/devices/system/node/node[0-9]*"): node = int(re.search(r".node(\d+)$", path).group(1)) node_dir = os.path.join(path + "/") pinned_cpus = [] for cpu_numbers in glob.glob(node_dir + "cpu[0-9]*"): cpu = int(re.search(r".cpu(\d+)$", cpu_numbers).group(1)) if pinned_cpus is not None: pinned_cpus.append(cpu) numa_nodes_with_cpus[node] = sorted(pinned_cpus) return numa_nodes_with_cpus def _deprecated(newfunc, oldfuncname): """ Print a warning to user and return the new function. :param newfunc: new function to be assigned :param oldfunctionname: Old function name string :rtype: `function` """ def wrap(*args, **kwargs): fmt_str = f"avocado.utils.cpu.{oldfuncname}() it is getting deprecat" fmt_str += f"ed, Use avocado.utils.cpu.{newfunc.__name__}() instead" warnings.warn((fmt_str), DeprecationWarning, stacklevel=2) return newfunc(*args, **kwargs) return wrap def lscpu(): """ Get Cores per socket, Physical sockets and Physical chips by executing 'lscpu' command. :rtype: `dict_obj` with the following details :cores per socket: :physical sockets: :physical chips: :threads per core: :sockets: :chips: physical sockets * physical chips """ output = process.run("LANG=en_US.UTF-8;lscpu", shell=True) res = {} for line in output.stdout.decode("utf-8").split("\n"): if "Physical cores/chip:" in line: res["cores_per_chip"] = int(line.split(":")[1].strip()) if "Core(s) per socket:" in line: res["virtual_cores"] = int(line.split(":")[1].strip()) if "Physical sockets:" in line: res["physical_sockets"] = int(line.split(":")[1].strip()) if "Physical chips:" in line: res["physical_chips"] = int(line.split(":")[1].strip()) if "Thread(s) per core:" in line: res["threads_per_core"] = int(line.split(":")[1].strip()) if "Socket(s):" in line: res["sockets"] = int(line.split(":")[1].strip()) if "physical_sockets" in res and "physical_chips" in res: res["chips"] = res["physical_sockets"] * res["physical_chips"] if ( "physical_sockets" in res and "physical_chips" in res and "cores_per_chip" in res ): res["physical_cores"] = ( res["physical_sockets"] * res["physical_chips"] * res["cores_per_chip"] ) return res total_cpus_count = _deprecated(total_count, "total_cpus_count") _get_cpu_info = _deprecated(_get_info, "_get_cpu_info") _get_cpu_status = _deprecated(_get_status, "_get_cpu_status") get_cpu_vendor_name = _deprecated(get_vendor, "get_cpu_vendor_name") get_cpu_arch = _deprecated(get_arch, "get_cpu_arch") cpu_online_list = _deprecated(online_list, "cpu_online_list") online_cpus_count = _deprecated(online_count, "online_cpus_count") get_cpuidle_state = _deprecated(get_idle_state, "get_cpuidle_state") set_cpuidle_state = _deprecated(set_idle_state, "set_cpuidle_state") set_cpufreq_governor = _deprecated(set_freq_governor, "set_cpufreq_governor") get_cpufreq_governor = _deprecated(get_freq_governor, "get_cpufreq_governor") avocado-113.0/avocado/utils/crypto.py000066400000000000000000000037521513561306000176070ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2013-2014 # Author: Lucas Meneghel Rodrigues import hashlib import io import logging import os LOG = logging.getLogger(__name__) def hash_file(filename, size=None, algorithm="md5"): """ Calculate the hash value of filename. If size is not None, limit to first size bytes. Throw exception if something is wrong with filename. Can be also implemented with bash one-liner (assuming ``size%1024==0``):: dd if=filename bs=1024 count=size/1024 | sha1sum - :param filename: Path of the file that will have its hash calculated. :param algorithm: Method used to calculate the hash (default is md5). :param size: If provided, hash only the first size bytes of the file. :return: Hash of the file, if something goes wrong, return None. """ chunksize = io.DEFAULT_BUFFER_SIZE fsize = os.path.getsize(filename) if not size or size > fsize: size = fsize try: hash_obj = hashlib.new(algorithm) except ValueError as detail: LOG.error( 'Returning "None" due to inability to create hash object: "%s"', detail ) return None with open(filename, "rb") as file_to_hash: while size > 0: chunksize = min(chunksize, size) data = file_to_hash.read(chunksize) if not data: LOG.debug("Nothing left to read but size=%d", size) break hash_obj.update(data) size -= len(data) return hash_obj.hexdigest() avocado-113.0/avocado/utils/data_factory.py000066400000000000000000000050571513561306000207270ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2013-2014 # Author: Lucas Meneghel Rodrigues """ Generate data useful for the avocado framework and tests themselves. """ import logging import os import random import string import tempfile _RAND_POOL = random.SystemRandom() LOG = logging.getLogger(__name__) def generate_random_string(length, ignore=string.punctuation, convert=""): """ Generate a random string using alphanumeric characters. :param length: Length of the string that will be generated. :type length: int :param ignore: Characters that will not include in generated string. :type ignore: str :param convert: Characters that need to be escaped (prepend "\\"). :type convert: str :return: The generated random string. """ result = "" chars = string.ascii_letters + string.digits + string.punctuation if not ignore: ignore = "" for i in ignore: chars = chars.replace(i, "") while length > 0: tmp = _RAND_POOL.choice(chars) if convert and (tmp in convert): tmp = f"\\{tmp}" result += tmp length -= 1 return result def make_dir_and_populate(basedir="/tmp"): """ Create a directory in basedir and populate with a number of files. The files just have random text contents. :param basedir: Base directory where directory should be generated. :type basedir: str :return: Path of the dir created and populated. :rtype: str """ try: path = tempfile.mkdtemp(prefix="avocado_" + __name__, dir=basedir) n_files = _RAND_POOL.randint(100, 150) for _ in range(n_files): fd, _ = tempfile.mkstemp(dir=path, text=True) str_length = _RAND_POOL.randint(30, 50) n_lines = _RAND_POOL.randint(5, 7) for _ in range(n_lines): os.write(fd, generate_random_string(str_length).encode()) os.close(fd) except OSError as details: log_msg = "Failed to generate dir in '%s' and populate: %s" LOG.error(log_msg, basedir, details) return None return path avocado-113.0/avocado/utils/data_structures.py000066400000000000000000000447721513561306000215120ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2014 # IBM, 2023 # # Authors: Ruda Moura # Lucas Meneghel Rodrigues # Harish S # Maram Srimannarayana Murthy # """This module contains handy classes that can be used inside avocado core code or plugins. """ import math import re import sys class InvalidDataSize(ValueError): """Signals that the value given to :class:`DataSize` is not valid. This exception is raised when an invalid data size string is provided to the DataSize class constructor. """ def ordered_list_unique(object_list): """Returns an unique list of objects, with their original order preserved. This function removes duplicates from a list while maintaining the original order of the first occurrence of each element. :param object_list: List of objects that may contain duplicates :type object_list: list :returns: List with duplicates removed, order preserved :rtype: list Example:: >>> ordered_list_unique([1, 2, 2, 3, 1, 4]) [1, 2, 3, 4] """ seen = set() seen_add = seen.add return [x for x in object_list if not (x in seen or seen_add(x))] def geometric_mean(values): """Evaluates the geometric mean for a list of numeric values. This implementation is slower but allows unlimited number of values. The geometric mean is calculated as the nth root of the product of n numbers. :param values: List with numeric values :type values: list :returns: Single value representing the geometric mean for the list values, or None if the list is empty :rtype: float or None :raises ValueError: If any value in the list cannot be converted to int :see: http://en.wikipedia.org/wiki/Geometric_mean Example:: >>> geometric_mean([1, 2, 4, 8]) 2.8284271247461903 """ try: values = [int(value) for value in values] except ValueError as exc: raise ValueError(f"Invalid inputs {values}. Provide valid inputs") from exc no_values = len(values) if not no_values: return None return math.exp(sum(math.log(number) for number in values) / no_values) def compare_matrices(matrix1, matrix2, threshold=0.05): # pylint: disable=R0912 """Compare 2 matrices nxm and return a matrix nxm with comparison data and stats. When the first columns match, they are considered as header and included in the results intact. This function is useful for comparing performance data between different test runs. :param matrix1: Reference Matrix of floats; first column could be header :type matrix1: list of lists :param matrix2: Matrix that will be compared; first column could be header :type matrix2: list of lists :param threshold: Any difference greater than this percent threshold will be reported (default: 0.05 = 5%) :type threshold: float :returns: Tuple containing: - Matrix with the difference in comparison - Number of improvements - Number of regressions - Total number of comparisons :rtype: tuple(list, int, int, int) Example:: >>> matrix1 = [['test1', 10.0, 20.0]] >>> matrix2 = [['test1', 12.0, 18.0]] >>> result = compare_matrices(matrix1, matrix2) >>> # Returns comparison matrix and statistics """ improvements = 0 regressions = 0 same = 0 new_matrix = [] for line1, line2 in zip(matrix1, matrix2): new_line = [] elements = iter(zip(line1, line2)) try: element1, element2 = next(elements) except StopIteration: # no data in this row new_matrix.append(new_line) continue if element1 == element2: # this column contains header new_line.append(element1) try: element1, element2 = next(elements) except StopIteration: new_matrix.append(new_line) continue while True: try: ratio = float(element2) / float(element1) except ZeroDivisionError: # For 0s, allow exact match or error if not float(element2): new_line.append(".") same += 1 else: new_line.append(f"error_{element2}/{element1}") improvements += 1 try: element1, element2 = next(elements) except StopIteration: break continue if ratio < (1 - threshold): # handling regression regressions += 1 new_line.append(100 * ratio - 100) elif ratio > (1 + threshold): # handling improvements improvements += 1 new_line.append(f"+{100 * ratio - 100:.6g}") else: same += 1 new_line.append(".") try: element1, element2 = next(elements) except StopIteration: break new_matrix.append(new_line) total = improvements + regressions + same return (new_matrix, improvements, regressions, total) def comma_separated_ranges_to_list(string): """Provides a list from comma separated ranges. Converts a string containing comma-separated ranges into a list of integers. Ranges can be specified as single numbers or as ranges using hyphens. :param string: String of comma separated range (e.g., "1,3-5,7") :type string: str :returns: List of integer values in comma separated range :rtype: list of int :raises ValueError: If the string contains invalid range format Example:: >>> comma_separated_ranges_to_list("1,3-5,7") [1, 3, 4, 5, 7] >>> comma_separated_ranges_to_list("10-12") [10, 11, 12] """ values = [] for range_str in string.split(","): if "-" in range_str: start, end = range_str.split("-") values.extend(range(int(start), int(end) + 1)) else: values.append(int(range_str)) return values def recursive_compare_dict(dict1, dict2, level="DictKey", diff_btw_dict=None): """Finds difference between two dictionaries. Recursively compares two dictionaries and returns a list of differences. The function handles nested structures and provides detailed difference information. :param dict1: First dictionary to compare :type dict1: dict, list, or any :param dict2: Second dictionary to compare :type dict2: dict, list, or any :param level: Current level identifier for nested comparison :type level: str :param diff_btw_dict: List to store differences (used internally for recursion) :type diff_btw_dict: list or None :returns: List of differences between the two dictionaries, or None for recursive calls :rtype: list or None Example:: >>> dict1 = {'a': 1, 'b': {'c': 2}} >>> dict2 = {'a': 2, 'b': {'c': 2}} >>> differences = recursive_compare_dict(dict1, dict2) >>> # Returns list of differences """ if diff_btw_dict is None: diff_btw_dict = [] if isinstance(dict1, dict) and isinstance(dict2, dict): if dict1.keys() != dict2.keys(): set1 = set(dict1.keys()) set2 = set(dict2.keys()) diff_btw_dict.append(f"{level} + {set1-set2} - {set2-set1}") common_keys = set1 & set2 else: common_keys = set(dict1.keys()) for k in common_keys: recursive_compare_dict( dict1[k], dict2[k], level=f"{level}.{k}", diff_btw_dict=diff_btw_dict ) return diff_btw_dict if isinstance(dict1, list) and isinstance(dict2, list): if len(dict1) != len(dict2): diff_btw_dict.append(f"{level} + {len(dict1)} - {len(dict2)}") common_len = min(len(dict1), len(dict2)) for i in range(common_len): recursive_compare_dict( dict1[i], dict2[i], level=f"{level}.{dict1[i]}", diff_btw_dict=diff_btw_dict, ) else: if dict1 != dict2: diff_btw_dict.append(f"{level} - dict1 value:{dict1}, dict2 value:{dict2}") return None class Borg: """Multiple instances of this class will share the same state. This is considered a better design pattern in Python than more popular patterns, such as the Singleton. The Borg pattern allows multiple instances to exist but they all share the same state, making them effectively equivalent. Inspired by Alex Martelli's article mentioned below. All instances of this class will have the same ``__dict__``, so any changes to instance variables will be reflected across all instances. :see: http://www.aleax.it/5ep.html Example:: >>> b1 = Borg() >>> b2 = Borg() >>> b1.value = 42 >>> b2.value # Will be 42, state is shared 42 """ __shared_state = {} def __init__(self): """Initialize a new Borg instance with shared state. Sets the instance's ``__dict__`` to the shared state dictionary, ensuring all instances share the same state. """ self.__dict__ = self.__shared_state class LazyProperty: """Lazily instantiated property. Use this decorator when you want to set a property that will only be evaluated the first time it's accessed. This is useful for expensive computations that should be deferred until actually needed. Once computed, the value is stored as a regular attribute on the instance, avoiding repeated computation. Inspired by the discussion in the Stack Overflow thread below. :see: http://stackoverflow.com/questions/15226721/ """ def __init__(self, f_get): """Initialize the lazy property with a getter function. :param f_get: Function that computes the property value :type f_get: callable """ self.f_get = f_get self.func_name = f_get.__name__ def __get__(self, obj, cls): """Descriptor method to get the property value. :param obj: Instance the property is being accessed on :type obj: object or None :param cls: Class the property is defined on :type cls: type :returns: The computed property value :rtype: any """ if obj is None: return None value = self.f_get(obj) setattr(obj, self.func_name, value) return value class CallbackRegister: """Registers pickable functions to be executed later. This class maintains a registry of functions with their arguments that can be called at a later time, typically for cleanup operations. All registered functions must be pickable (serializable). """ def __init__(self, name, log): """Initialize the callback register. :param name: Human readable identifier of this register :type name: str :param log: Logger instance for error reporting :type log: logging.Logger """ self._name = name self._items = [] self._log = log def register(self, func, args, kwargs, once=False): """Register function/args to be called on self.run(). :param func: Pickable function to be called later :type func: callable :param args: Pickable positional arguments for the function :type args: tuple :param kwargs: Pickable keyword arguments for the function :type kwargs: dict :param once: Add unique (func,args,kwargs) combination only once :type once: bool """ item = (func, args, kwargs) if not once or item not in self._items: self._items.append(item) def unregister(self, func, args, kwargs): """Unregister (func,args,kwargs) combination. :param func: Pickable function to unregister :type func: callable :param args: Pickable positional arguments :type args: tuple :param kwargs: Pickable keyword arguments :type kwargs: dict """ item = (func, args, kwargs) if item in self._items: self._items.remove(item) def run(self): """Call all registered functions. Executes all registered functions with their associated arguments. If any function raises an exception, it is logged and execution continues with the remaining functions. Functions are called in LIFO order (last registered, first executed). """ while self._items: item = self._items.pop() try: func, args, kwargs = item func(*args, **kwargs) except: # Ignore all exceptions pylint: disable=W0702 self._log.error( "%s failed to destroy %s:\n%s", self._name, item, sys.exc_info()[1] ) def __del__(self): """Destructor that runs all registered callbacks. .. warning:: Always call self.run() manually, this is not guaranteed to be executed! """ self.run() def time_to_seconds(time): """Convert time in minutes, hours and days to seconds. Converts a time string with optional unit suffix to seconds. Supported units are 's' (seconds), 'm' (minutes), 'h' (hours), and 'd' (days). If no unit is specified, the value is assumed to be in seconds. :param time: Time, optionally including the unit (i.e. '10d', '5m', '30') :type time: str, int, or None :returns: Time converted to seconds :rtype: int :raises ValueError: If the time format is invalid Example:: >>> time_to_seconds('10m') 600 >>> time_to_seconds('2h') 7200 >>> time_to_seconds('30') 30 >>> time_to_seconds(None) 0 """ units = {"s": 1, "m": 60, "h": 3600, "d": 86400} if time is not None: try: unit = time[-1].lower() if unit in units: mult = units[unit] seconds = int(time[:-1]) * mult else: seconds = int(time) except (ValueError, TypeError) as exc: raise ValueError( f"Invalid value '{time}' for time. Use a string " f"with the number and optionally the time unit " f"(s, m, h or d)." ) from exc else: seconds = 0 return seconds class DataSize: """Data Size object with builtin unit-converted attributes. Represents a data size with automatic unit conversion capabilities. Supports bytes (b), kibibytes (k), mebibytes (m), gibibytes (g), and tebibytes (t). All conversions use binary multipliers (1024-based). :param data: Data size plus optional unit string. i.e. '10m'. No unit string means the data size is in bytes. :type data: str, int, or float :raises InvalidDataSize: If the data format is invalid Example:: >>> size = DataSize('10m') >>> size.b # bytes 10485760 >>> size.k # kibibytes 10240 >>> size.g # gibibytes 0 """ __slots__ = ["_value", "_unit"] MULTIPLIERS = { "b": 1, # bytes (2**0) "k": 1024, # kibibytes (2**10) "m": 1048576, # mebibytes (2**20) "g": 1073741824, # gibibytes (2**30) "t": 1099511627776, # tebibytes (2**40) } def __init__(self, data): """Initialize a DataSize object. :param data: Data size with optional binary unit (e.g., '10M', '2.5G', '100') :type data: str, int, or float :raises InvalidDataSize: If the data format is invalid """ try: norm = str(data).strip().lower() match = re.match(r"^(\d+(\.\d+)?)(?:\s*([bkmgt]))?$", norm) if not match: raise ValueError self._value = float(match.group(1)) self._unit = match.group(3) or "b" if self._unit not in self.MULTIPLIERS or self._value < 0: raise ValueError except ValueError as exc: raise InvalidDataSize( f"Invalid data size '{data}'. Use binary unit formats like '10M', '2.5G', or '100'." ) from exc @property def value(self): """The numeric value of the data size. :returns: The original numeric value without unit conversion :rtype: float """ return self._value @property def unit(self): """The unit of the data size. :returns: Single character representing the unit ('b', 'k', 'm', 'g', 't') :rtype: str """ return self._unit @property def b(self): """Data size in bytes. :returns: Size converted to bytes :rtype: float """ return self._value * self.MULTIPLIERS[self._unit] @property def k(self): """Data size in kibibytes. :returns: Size converted to kibibytes (truncated to integer) :rtype: int """ return int(self._value * self.MULTIPLIERS[self._unit] / self.MULTIPLIERS["k"]) @property def m(self): """Data size in mebibytes. :returns: Size converted to mebibytes (truncated to integer) :rtype: int """ return int(self._value * self.MULTIPLIERS[self._unit] / self.MULTIPLIERS["m"]) @property def g(self): """Data size in gibibytes. :returns: Size converted to gibibytes (truncated to integer) :rtype: int """ return int(self._value * self.MULTIPLIERS[self._unit] / self.MULTIPLIERS["g"]) @property def t(self): """Data size in tebibytes. :returns: Size converted to tebibytes (truncated to integer) :rtype: int """ return int(self._value * self.MULTIPLIERS[self._unit] / self.MULTIPLIERS["t"]) # pylint: disable=wrong-import-position from avocado.utils.deprecation import log_deprecation log_deprecation.warning("data_structures") avocado-113.0/avocado/utils/datadrainer.py000066400000000000000000000127641513561306000205500ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2019 # Author: Cleber Rosa """ data drainer This module provides utility classes for draining data and dispatching it to different destinations. This is intended to be used concurrently with other code, usually test code producing the output to be drained/processed. A thread is started and maintained on behalf of the user. """ import abc import io import os import select import threading class BaseDrainer(abc.ABC): """ Base drainer, doesn't provide complete functionality to be useful. """ name = "avocado.utils.datadrainer.BaseDrainer" def __init__(self, source, stop_check=None, name=None): """ :param source: where to read data from, this is intentionally abstract :param stop_check: callable that should determine if the drainer should quit. If None is given, it will never stop. :type stop_check: function :param name: instance name of the drainer, used for describing the name of the thread maintained by this instance :type name: str """ self._source = source if stop_check is None: def stop_check(): # pylint: disable=E0102 return False self._stop_check = stop_check if name is not None: self.name = name # internal state flag, used to stop processing because of a # condition that may have happened in between the loop cycles self._internal_quit = False @staticmethod def data_available(): """ Checks if source appears to have data to be drained """ return False @abc.abstractmethod def read(self): """ Abstract method supposed to read from the data source """ @abc.abstractmethod def write(self, data): """ Abstract method supposed to write the read data to its destination """ def _loop(self): """ Basic implementation of the thread target This loops until either an internal quit flag is set, or when the stop_check function evaluates to True. """ while True: if self._internal_quit: break if self.data_available(): self.write(self.read()) if self._stop_check(): break def start(self): """ Starts a thread to do the data draining """ self._thread = threading.Thread( # pylint: disable=W0201 target=self._loop, name=self.name ) self._thread.daemon = True self._thread.start() def wait(self): """ Waits on the thread completion """ self._thread.join() class FDDrainer(BaseDrainer): """ Drainer whose source is a file descriptor This drainer uses select to efficiently wait for data to be available on a file descriptor. If the file descriptor is closed, the drainer responds by shutting itself down. This drainer doesn't provide a write() implementation, and is consequently not a complete implementation users can pick and use. """ name = "avocado.utils.datadrainer.FDDrainer" def data_available(self): # pylint: disable=W0221 try: return select.select([self._source], [], [], 1)[0] except OSError: return False def read(self): data = b"" try: data = os.read(self._source, io.DEFAULT_BUFFER_SIZE) except OSError as exc: if exc.errno == 9: self._internal_quit = True return data def write(self, data): # necessary to avoid pylint W0223 raise NotImplementedError class BufferFDDrainer(FDDrainer): """ Drains data from a file descriptor and stores it in an internal buffer """ name = "avocado.utils.datadrainer.BufferFDDrainer" def __init__(self, source, stop_check=None, name=None): super().__init__(source, stop_check, name) self._data = io.BytesIO() def write(self, data): self._data.write(data) @property def data(self): """ Returns the buffer data, as bytes """ return self._data.getvalue() class LineLogger(FDDrainer): name = "avocado.utils.datadrainer.LineLogger" def __init__(self, source, stop_check=None, name=None, logger=None): super().__init__(source, stop_check, name) self._logger = logger self._buffer = io.BytesIO() def write(self, data): if b"\n" not in data: self._buffer.write(data) return data = self._buffer.getvalue() + data lines = data.split(b"\n") if not lines[-1].endswith(b"\n"): self._buffer.close() self._buffer = io.BytesIO() self._buffer.write(lines[-1]) for line in lines: line = line.decode(errors="replace").rstrip("\n") if line: self._logger.debug(line) avocado-113.0/avocado/utils/debug.py000066400000000000000000000055721513561306000173570ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2014 # Author: Lucas Meneghel Rodrigues # Author: Lukas Doktor """ This file contains tools for (not only) Avocado developers. """ import logging import os import time # Use this for debug logging LOGGER = logging.getLogger(__name__) # measure_duration global storage __MEASURE_DURATION = {} def measure_duration(func): """ Use this as decorator to measure duration of the function execution. The output is "Function $name: ($current_duration, $accumulated_duration)" """ def wrapper(*args, **kwargs): """Wrapper function""" start = time.monotonic() try: return func(*args, **kwargs) finally: duration = time.monotonic() - start __MEASURE_DURATION[func] = __MEASURE_DURATION.get(func, 0) + duration LOGGER.debug( "PERF: %s: (%.9fs, %.9fs)", func, duration, __MEASURE_DURATION[func] ) return wrapper def log_calls_class(length=None): """ Use this as decorator to log the function methods' calls. :param length: Max message length """ def wrap(orig_cls): for key, attr in orig_cls.__dict__.items(): if callable(attr): setattr(orig_cls, key, _log_calls(attr, length, orig_cls.__name__)) return orig_cls return wrap def _log_calls(func, length=None, cls_name=None): """ log_calls wrapper function """ def wrapper(*args, **kwargs): """Wrapper function""" msg = "CALL: %s:%s%s(%s, %s)" % ( # pylint: disable=C0209 os.path.relpath(func.__code__.co_filename), cls_name, func.__name__, ", ".join([str(_) for _ in args]), ", ".join( [ f"{key}={value}" # pylint: disable=C0209 for key, value in kwargs.items() ] ), ) if length: msg = msg[:length] LOGGER.debug(msg) return func(*args, **kwargs) if cls_name: cls_name = cls_name + "." return wrapper def log_calls(length=None, cls_name=None): """ Use this as decorator to log the function call altogether with arguments. :param length: Max message length :param cls_name: Optional class name prefix """ def wrap(func): return _log_calls(func, length, cls_name) return wrap avocado-113.0/avocado/utils/deprecation.py000066400000000000000000000023751513561306000205640ustar00rootroot00000000000000# This module was introduced only for AAutils migration purposes and will be removed # after the AAutils migration is complete. No need to migrate this module to AAutils. import logging LOGGER = logging.getLogger(__name__) class LogDeprecation: """ A class that ensures a message is logged only once. """ def __init__(self): self._logged = set() self._logged_messages = set() def warning(self, utility_name, msg=None): """ Logs a warning message only once. """ if msg is None: self._logged.add(utility_name) else: self._logged_messages.add(msg) def flush(self): for msg in self._logged_messages: LOGGER.info(msg) for utility_name in self._logged: msg = ( f"The {utility_name} utility is deprecated. " "It has been moved to the AAutils project " "and will be removed after the next LTS release. " "For more information, please see the following " "link: https://github.com/avocado-framework/aautils" ) LOGGER.info(msg) self._logged_messages.clear() self._logged.clear() log_deprecation = LogDeprecation() avocado-113.0/avocado/utils/diff_validator.py000066400000000000000000000250701513561306000212410ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; specifically version 2 of the License. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # This code was inspired in the autotest project, # client/shared/utils.py # Authors: Plamen Dimitrov , Kristof Katus """ Diff validator: Utility for testing file changes Some typical use of this utility would be: >>> import diff_validator >>> change = diff_validator.Change() >>> change.add_validated_files(["/etc/somerc"]) >>> change.append_expected_add("/etc/somerc", "this is a new line") >>> change.append_expected_remove("/etc/somerc", "this line is removed") >>> diff_validator.make_temp_file_copies(change.get_target_files()) After making changes through some in-test operation: >>> changes = diff_validator.extract_changes(change.get_target_files()) >>> change_success = diff_validator.assert_change(changes, change.files_dict) If test fails due to invalid change on the system: >>> if not change_success: >>> changes = diff_validator.assert_change_dict(changes, change.files_dict) >>> raise DiffValidationError("Change is different than expected:\n%s" % diff_validator.create_diff_report(changes)) >>> else: >>> logging.info("Change made successfully") >>> diff_validator.del_temp_file_copies(change.get_target_files()) """ import difflib import os import shutil class DiffValidationError(Exception): pass def get_temp_file_path(file_path): """ Generates a temporary filename. :param str file_path: file path prefix :returns: appended file path :rtype: str """ return file_path + ".tmp" def make_temp_file_copies(file_paths): """ Creates temporary copies of the provided files. :param file_paths: file paths to be copied :type file_paths: [str] """ for file_path in file_paths: temp_file_path = get_temp_file_path(file_path) shutil.copyfile(file_path, temp_file_path) def del_temp_file_copies(file_paths): """ Deletes all the provided files. :param file_paths: deleted file paths (their temporary versions) :type file_paths: [str] """ for file_path in file_paths: temp_file_path = get_temp_file_path(file_path) os.remove(temp_file_path) def parse_unified_diff_output(lines): """ Parses the unified diff output of two files. :param lines: diff lines :type lines: [str] :returns: pair of adds and removes, where each is a list of trimmed lines :rtype: ([str], [str]) """ adds = [] removes = [] for line in lines: # ignore filepaths in the output if len(line) > 2 and (line[:3] == "+++" or line[:3] == "---"): continue # ignore line range information in the output if len(line) > 1 and line[:2] == "@@": continue # gather adds if len(line) > 0 and line[0] == "+": added_line = line[1:].lstrip().rstrip() if not added_line: continue adds = adds + [added_line] # gather removes elif len(line) > 0 and line[0] == "-": removed_line = line[1:].lstrip().rstrip() if not removed_line: continue removes = removes + [removed_line] return (adds, removes) def extract_changes(file_paths, compared_file_paths=None): """ Extracts diff information based on the new and temporarily saved old files. :param file_paths: original file paths (whose temporary versions will be retrieved) :type file_paths: [str] :param compared_file_paths: custom file paths to use instead of the temporary versions :type compared_file_paths: [str] or None :returns: file paths with corresponding diff information key-value pairs :rtype: {str, ([str], [str])} """ changes = {} if compared_file_paths is None: compared_file_paths = [] for i, file_path in enumerate(file_paths): temp_file_path = get_temp_file_path(file_path) if len(compared_file_paths) > i: file1, file2 = compared_file_paths[i], file_path else: file1, file2 = temp_file_path, file_path with open(file1, encoding="utf-8") as f1: lines1 = f1.readlines() with open(file2, encoding="utf-8") as f2: lines2 = f2.readlines() lines = difflib.unified_diff(lines1, lines2, fromfile=file1, tofile=file2, n=0) changes[file_path] = parse_unified_diff_output(lines) return changes def assert_change_dict(actual_result, expected_result): """ Calculates unexpected line changes. :param actual_result: actual added and removed lines :type actual_result: {file_path, ([added_line, ...], [removed_line, ...])} :param expected_result: expected added and removed lines :type expected_result: {file_path, ([added_line, ...], [removed_line, ...])} :returns: detected differences as groups of lines with filepath keys and a tuple of (unexpected_adds, not_present_adds, unexpected_removes, not_present_removes) :rtype: {str, (str, str, str, str)} """ change_diffs = {} for file_path, actual_changes in actual_result.items(): expected_changes = expected_result[file_path] actual_adds = actual_changes[0] actual_removes = actual_changes[1] expected_adds = expected_changes[0] expected_removes = expected_changes[1] # Additional unexpected adds -- they should have been not added unexpected_adds = sorted(set(actual_adds) - set(expected_adds)) # Not present expected adds -- they should have been added not_present_adds = sorted(set(expected_adds) - set(actual_adds)) # Additional unexpected removes - they should have been not removed unexpected_removes = sorted(set(actual_removes) - set(expected_removes)) # Not present expected removes - they should have been removed not_present_removes = sorted(set(expected_removes) - set(actual_removes)) change_diffs[file_path] = ( unexpected_adds, not_present_adds, unexpected_removes, not_present_removes, ) return change_diffs def assert_change(actual_result, expected_result): """ Condition wrapper of the upper method. :param actual_result: actual added and removed lines with filepath keys and a tuple of ([added_line, ...], [removed_line, ...]) :type actual_result: {str, ([str], [str])} :param expected_result: expected added and removed lines of type as the actual result :type expected_result: {str, ([str], [str])} :returns: whether changes were detected :rtype: bool """ change_diffs = assert_change_dict(actual_result, expected_result) for file_change in change_diffs.values(): for line_change in file_change: if len(line_change): return False return True def create_diff_report(change_diffs): """ Pretty prints the output of the `change_diffs` variable. :param change_diffs: detected differences as groups of lines with filepath keys and a tuple of (unexpected_adds, not_present_adds, unexpected_removes, not_present_removes) :type: {str, (str, str, str, str)} :returns: print string of the line differences :rtype: str """ diff_strings = [] for file_path, change_diff in change_diffs.items(): if not (change_diff[0] or change_diff[1] or change_diff[2] or change_diff[3]): continue diff_strings.append(f"--- {get_temp_file_path(file_path)}") diff_strings.append(f"+++ {file_path}") for iter_category in range(4): change_category = change_diff[iter_category] if not iter_category and change_category: diff_strings.append("*++ Additional unexpected adds") elif iter_category == 1 and change_category: diff_strings.append("/++ Not present expected adds") elif iter_category == 2 and change_category: diff_strings.append("*-- Additional unexpected removes") elif iter_category == 3 and change_category: diff_strings.append("/-- Not present expected removes") for line_change in change_category: diff_strings.append(str(line_change).encode("unicode_escape").decode()) return "\n".join(diff_strings) class Change: """Class for tracking and validating file changes""" def __init__(self): """Creates a change object.""" self.files_dict = {} def get_target_files(self): """Get added files for change.""" return list(self.files_dict.keys()) def add_validated_files(self, filenames): """ Add file to change object. :param filenames: files to validate :type filenames: [str] """ for filename in filenames: self.files_dict[filename] = ([], []) def append_expected_add(self, filename, line): """ Append expected added line to a file. :param str filename: file to append to :param str line: line to append to as an expected addition """ try: self.files_dict[filename][0].append(line) except KeyError: self.files_dict[filename] = ([], []) self.files_dict[filename][0].append(line) def append_expected_remove(self, filename, line): """ Append removed added line to a file. :param str filename: file to append to :param str line: line to append to as an expected removal """ try: self.files_dict[filename][1].append(line) except KeyError: self.files_dict[filename] = ([], []) self.files_dict[filename][1].append(line) def get_all_adds(self): """Return a list of the added lines for all validated files.""" all_adds = [] for f in list(self.files_dict.keys()): for add in self.files_dict[f][0]: all_adds.append(add) return all_adds def get_all_removes(self): """Return a list of the removed lines for all validated files.""" all_removes = [] for f in list(self.files_dict.keys()): for add in self.files_dict[f][1]: all_removes.append(add) return all_removes avocado-113.0/avocado/utils/disk.py000066400000000000000000000337241513561306000172230ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # This code was inspired in the autotest project, # # client/base_utils.py # # Copyright: 2018 IBM # Authors : Praveen K Pandey # : Narasimhan V # : Naresh Bannoth """ Disk utilities """ import json import logging import os import re from avocado.utils import genio, multipath, process LOGGER = logging.getLogger(__name__) class DiskError(Exception): """ Generic DiskError """ def freespace(path): fs_stats = os.statvfs(path) return fs_stats.f_bsize * fs_stats.f_bavail def get_disk_blocksize(path): """Return the disk block size, in bytes""" fs_stats = os.statvfs(path) return fs_stats.f_bsize def create_loop_device(size, blocksize=4096, directory="./"): """ Creates a loop device of size and blocksize specified. :param size: Size of loop device, in bytes :type size: int :param blocksize: block size of loop device, in bytes. Defaults to 4096 :type blocksize: int :param directory: Directory where the backing file will be created. Defaults to current directory. :type directory: str :return: loop device name :rtype: str """ cmd = "losetup --find" loop = process.run(cmd, ignore_status=True, sudo=True).stdout_text.strip("\n") loop_file = os.path.join(directory, f"tmp_{loop.split('/')[-1]}.img") cmd = ( f"dd if=/dev/zero of={loop_file} bs={blocksize} " f"count={int(size / blocksize)}" ) if process.system(cmd, ignore_status=True, sudo=True): raise DiskError("Unable to create backing file for loop device") cmd = f"losetup {loop} {loop_file} -P" if process.system(cmd, ignore_status=True, sudo=True): raise DiskError("Unable to create the loop device") return loop def delete_loop_device(device): """ Deletes the specified loop device. :param device: device to be deleted :type device: str :return: True if deleted. :rtype: bool """ cmd = "losetup -aJl" loop_dic = json.loads(process.run(cmd, ignore_status=True, sudo=True).stdout_text) loop_file = "" for loop_dev in loop_dic["loopdevices"]: if device == loop_dev["name"]: loop_file = loop_dev["back-file"] if not loop_file: raise DiskError("Unable to find backing file for loop device") cmd = f"losetup -d {device}" if process.system(cmd, ignore_status=True, sudo=True): raise DiskError("Unable to delete the loop device") os.remove(loop_file) return True def get_disks(): """ Returns the physical "hard drives" available on this system This is a simple wrapper around `lsblk` and will return all the top level physical (non-virtual) devices return by it. TODO: this is currently Linux specific. Support for other platforms is desirable and may be implemented in the future. :returns: a list of paths to the physical disks on the system :rtype: list of str """ try: json_result = process.run("lsblk --json --paths --inverse") except process.CmdError as ce: raise DiskError(f"Error occurred while executing lsblk command: {ce}") from ce try: json_data = json.loads(json_result.stdout_text) except json.JSONDecodeError as je: raise DiskError(f"Error occurred while parsing JSON data: {je}") from je disks = [] to_process = json_data.get("blockdevices", []) for device in to_process: disks.append(device.get("name")) if "children" in device: to_process.extend(device["children"]) return disks def get_all_disk_paths(): """ Returns all available disk names and alias on this system This will get all the sysfs disks name entries by its device node name, by-uuid, by-id and by-path, irrespective of any platform and device type :returns: a list of all disk path names :rtype: list of str """ disk_list = abs_path = [] for path in [ "/dev", "/dev/mapper", "/dev/disk/by-id", "/dev/disk/by-path", "/dev/disk/by-uuid", "/dev/disk/by-partuuid", "/dev/disk/by-partlabel", ]: if os.path.exists(path): for device in os.listdir(path): abs_path.append(os.path.join(path, device)) disk_list.extend(abs_path) return disk_list def get_absolute_disk_path(device): """ Returns absolute device path of given disk This will get actual disks path of given device, it can take node name, by-uuid, by-id and by-path, irrespective of any platform and device type :param device: disk name or disk alias names sda or scsi-xxx :type device: str :returns: the device absolute path name :rtype: bool """ if not os.path.exists(device): for dev_path in get_all_disk_paths(): if device == os.path.basename(dev_path): return dev_path return device def get_available_filesystems(): """ Return a list of all available filesystem types :returns: a list of filesystem types :rtype: list of str """ filesystems = set() with open("/proc/filesystems") as proc_fs: # pylint: disable=W1514 for proc_fs_line in proc_fs.readlines(): filesystems.add(re.sub(r"(nodev)?\s*", "", proc_fs_line)) return list(filesystems) def get_filesystem_type(mount_point="/"): """ Returns the type of the filesystem of mount point informed. The default mount point considered when none is informed is the root "/" mount point. :param str mount_point: mount point to asses the filesystem type. Default "/" :returns: filesystem type :rtype: str """ with open("/proc/mounts") as mounts: # pylint: disable=W1514 for mount_line in mounts.readlines(): _, fs_file, fs_vfstype, _, _, _ = mount_line.split() if fs_file == mount_point: return fs_vfstype return None def is_root_device(device): """ check for root disk :param device: device to check :returns: True or False, True if given device is root disk otherwise will return False. """ cmd = "lsblk --j -o MOUNTPOINT,PKNAME" output = process.run(cmd) result = json.loads(output.stdout_text) for item in result["blockdevices"]: if item["mountpoint"] == "/" and device == str(item["pkname"]): return True return False def is_disk_mounted(device): """ check if given disk is mounted or not :param device: disk/device name :type device: str :returns: True if the device/disk is mounted else False :rtype: bool """ with open("/proc/mounts") as mounts: # pylint: disable=W1514 for mount_line in mounts.readlines(): dev, _, _, _, _, _ = mount_line.split() if dev == device: return True return False def is_dir_mounted(dir_path): """ check if given directory is mounted or not :param dir_path: directory path :type dir_path: str :returns: True if the given director is mounted else False :rtype: bool """ with open("/proc/mounts") as mounts: # pylint: disable=W1514 for mount_line in mounts.readlines(): _, fs_dir, _, _, _, _ = mount_line.split() if fs_dir == dir_path: return True return False def fs_exists(device): """ check if filesystem exists on give disk/device :param device: disk/device name :type device: str :returns: returns True if filesystem exists on the give disk else False :rtype: bool """ cmd = f"blkid -o value -s TYPE {device}" out = process.system_output(cmd, shell=True, ignore_status=True).decode("utf-8") fs_list = ["ext2", "ext3", "ext4", "xfs", "btrfs"] if out in fs_list: return True return False def get_dir_mountpoint(dir_path): """ get mounted disk name that is mounted on given dir_path :param dir_path: absolute directory path :type dir_path: str :returns: returns disk name which mounted on given dir_path :rtype: str """ with open("/proc/mounts") as mounts: # pylint: disable=W1514 for mount_line in mounts.readlines(): dev, fs_dir, _, _, _, _ = mount_line.split() if fs_dir == dir_path: return dev return None def get_disk_mountpoint(device): """ get mountpoint on which given disk is mounted :param device: disk/device name :type device: str :return: return directory name on which disk is mounted :rtype: str """ with open("/proc/mounts") as mounts: # pylint: disable=W1514 for mount_line in mounts.readlines(): dev, fs_dir, _, _, _, _ = mount_line.split() if dev == device: return fs_dir return None def create_linux_raw_partition(disk_name, size=None, num_of_par=1): """ Creates partitions using sfdisk command :param disk_name: disk/device name :type disk_name: str :param size: size of partition :type size: str :param num_of_par: Number of partitions to be created :type num_of_par: int Returns list of created partitions """ if not size: size = get_size_of_disk(disk_name) / 1073741824 size = size / num_of_par size = str(size) + "G" partitions = [ "size= +" + size if val != 3 else "type=5" for val in range(0, num_of_par + 1) ] disk_partition_file = ( "/tmp/creat_partition" + process.run("date '+%d-%m-%y_%T'").stdout_text.strip() ) if not os.path.isfile(disk_partition_file): process.run("touch " + disk_partition_file) for line in partitions: genio.append_one_line(disk_partition_file, line) try: part_output = process.getoutput( "sfdisk " + disk_name + " < " + disk_partition_file ) except Exception as exc: msg = f"sfdisk partition creation command failed on disk {disk_name}" LOGGER.warning(msg) raise DiskError(msg) from exc rescan_disk(disk_name) if "The partition table has been altered" in part_output: return get_disk_partitions(disk_name) return None def get_size_of_disk(disk): """ Returns size of disk in bytes :param disk: disk/device name :type disk: str Return Type: int """ return int(process.getoutput("lsblk -b --output SIZE -n -d " + disk)) def delete_partition(partition_name): """ Deletes mentioned partition from disk :param partition_name: partition absolute path :type partition_name: str """ disk_index = re.search(r"\d+", partition_name).start() try: process.run( "sfdisk --delete " + partition_name[:disk_index] + " " + partition_name[disk_index:] ) except Exception as exc: msg = f"sfdisk --delete command failed on disk {partition_name}" LOGGER.warning(msg) raise DiskError(msg) from exc def clean_disk(disk_name): """ Cleans partitions table of a disk :param disk_name: disk name :type disk_name: str """ output = process.getoutput("sfdisk --delete " + disk_name) rescan_disk(disk_name) if not get_disk_partitions(disk_name): if "The partition table has been altered" in output: process.run("wipefs -af " + disk_name) def rescan_disk(disk_name): """ Re-scans disk :param disk_name: disk name :type disk_name: str """ disk_name = os.path.realpath(disk_name) if re.search(r"dm-\d+", disk_name): mpath_dict = multipath.get_multipath_details() for _ in range(len(mpath_dict["maps"])): if mpath_dict["maps"][_]["sysfs"] == disk_name.split("/")[-1]: disk_name = ( "/dev/" + mpath_dict["maps"][_]["path_groups"][0]["paths"][0]["dev"] ) break process.run(f"echo 1 > /sys/block/{disk_name}/device/rescan") def get_disk_partitions(disk): """ Returns partitions of a disk excluding extended partition :param disk: disk name :type disk: str Returns array with all partitions of disk """ rescan_disk(disk) partitions_op = process.getoutput("sfdisk -l " + disk) return [ line.split(" ")[0] for line in partitions_op.split("\n") if line.startswith(disk) and "Extended" not in line ] def get_io_scheduler_list(device_name): """ Returns io scheduler available for the IO Device :param device_name: Device name example like sda :return: list of IO scheduler """ with open(__sched_path(device_name), "r", encoding="utf-8") as fl: return fl.read().translate(str.maketrans("[]", " ")).split() def get_io_scheduler(device_name): """ Return io scheduler name which is set currently for device :param device_name: Device name example like sda :return: IO scheduler :rtype : str """ return re.split( r"[\[\]]", open(__sched_path(device_name), "r", encoding="utf-8").read() )[1] def __sched_path(device_name): file_path = f"/sys/block/{device_name}/queue/scheduler" return file_path def set_io_scheduler(device_name, name): """ Set io scheduler to a device :param device_name: Device name example like sda :param name: io scheduler name """ if name not in get_io_scheduler_list(device_name): raise DiskError(f"No such IO scheduler: {name}") with open(__sched_path(device_name), "w", encoding="utf-8") as fp: fp.write(name) avocado-113.0/avocado/utils/distro.py000066400000000000000000000401011513561306000175600ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2013-2014 # Author: Lucas Meneghel Rodrigues """ This module provides the client facilities to detect the Linux Distribution it's running under. """ import logging import os import platform import re LOGGER = logging.getLogger(__name__) __all__ = [ "LinuxDistro", "UNKNOWN_DISTRO_NAME", "UNKNOWN_DISTRO_VERSION", "UNKNOWN_DISTRO_RELEASE", "UNKNOWN_DISTRO_ARCH", "Probe", "register_probe", "detect", ] # pylint: disable=R0903 class LinuxDistro: """ Simple collection of information for a Linux Distribution """ def __init__(self, name, version, release, arch): """ Initializes a new Linux Distro :param name: a short name that precisely distinguishes this Linux Distribution among all others. :type name: str :param version: the major version of the distribution. Usually this is a single number that denotes a large development cycle and support file. :type version: str :param release: the release or minor version of the distribution. Usually this is also a single number, that is often omitted or starts with a 0 when the major version is initially release. It's often associated with a shorter development cycle that contains incremental a collection of improvements and fixes. :type release: str :param arch: the main target for this Linux Distribution. It's common for some architectures to ship with packages for previous and still compatible architectures, such as it's the case with Intel/AMD 64 bit architecture that support 32 bit code. In cases like this, this should be set to the 64 bit architecture name. :type arch: str """ self.name = name self.version = version self.release = release self.arch = arch def __repr__(self): return ( f"" ) UNKNOWN_DISTRO_NAME = "unknown" UNKNOWN_DISTRO_VERSION = 0 UNKNOWN_DISTRO_RELEASE = 0 UNKNOWN_DISTRO_ARCH = "unknown" #: The distribution that is used when the exact one could not be found UNKNOWN_DISTRO = LinuxDistro( UNKNOWN_DISTRO_NAME, UNKNOWN_DISTRO_VERSION, UNKNOWN_DISTRO_RELEASE, UNKNOWN_DISTRO_ARCH, ) class Probe: """ Probes the machine and does it best to confirm it's the right distro. If given an avocado.utils.ssh.Session object representing another machine, Probe will attempt to detect another machine's distro via an ssh connection. """ #: Points to a file that can determine if this machine is running a given #: Linux Distribution. This servers a first check that enables the extra #: checks to carry on. CHECK_FILE = None #: Sets the content that should be checked on the file pointed to by #: :attr:`CHECK_FILE_EXISTS`. Leave it set to `None` (its default) #: to check only if the file exists, and not check its contents CHECK_FILE_CONTAINS = None #: The name of the Linux Distribution to be returned if the file defined #: by :attr:`CHECK_FILE_EXISTS` exist. CHECK_FILE_DISTRO_NAME = None #: A regular expression that will be run on the file pointed to by #: :attr:`CHECK_FILE_EXISTS` CHECK_VERSION_REGEX = None def __init__(self, session=None): self.score = 0 self.session = session def check_for_remote_file(self, file_name): """ Checks if provided file exists in remote machine :param file_name: name of file :type file_name: str :returns: whether the file exists in remote machine or not :rtype: bool """ if self.session and not self.session.cmd(f"test -f {file_name}").exit_status: return True return False def check_name_for_file(self): """ Checks if this class will look for a file and return a distro The conditions that must be true include the file that identifies the distro file being set (:attr:`CHECK_FILE`) and the name of the distro to be returned (:attr:`CHECK_FILE_DISTRO_NAME`) """ if self.CHECK_FILE is None: return False if self.CHECK_FILE_DISTRO_NAME is None: return False return True def name_for_file(self): """ Get the distro name if the :attr:`CHECK_FILE` is set and exists """ if self.check_name_for_file(): if self.check_for_remote_file(self.CHECK_FILE): return self.CHECK_FILE_DISTRO_NAME if os.path.exists(self.CHECK_FILE): return self.CHECK_FILE_DISTRO_NAME return None def check_name_for_file_contains(self): """ Checks if this class will look for text on a file and return a distro The conditions that must be true include the file that identifies the distro file being set (:attr:`CHECK_FILE`), the text to look for inside the distro file (:attr:`CHECK_FILE_CONTAINS`) and the name of the distro to be returned (:attr:`CHECK_FILE_DISTRO_NAME`) """ if self.CHECK_FILE is None: return False if self.CHECK_FILE_CONTAINS is None: return False if self.CHECK_FILE_DISTRO_NAME is None: return False return True def name_for_file_contains(self): """ Get the distro if the :attr:`CHECK_FILE` is set and has content """ if self.check_name_for_file_contains(): check_file = None if self.check_for_remote_file(self.CHECK_FILE): check_file = self.session.cmd( f"cat {self.CHECK_FILE}" ).stdout_text.split("/n") elif os.path.exists(self.CHECK_FILE): try: # pylint: disable=R1732 check_file = open(self.CHECK_FILE, encoding="utf-8") except IOError as err: LOGGER.debug("Could not open %s", self.CHECK_FILE) LOGGER.debug("Exception: %s", str(err)) return None if not check_file: return None for line in check_file: if self.CHECK_FILE_CONTAINS in line: return self.CHECK_FILE_DISTRO_NAME return None def check_version(self): """ Checks if this class will look for a regex in file and return a distro """ if self.CHECK_FILE is None: return False if self.CHECK_VERSION_REGEX is None: return False return True def _get_version_match(self): """ Returns the match result for the version regex on the file content """ if self.check_version(): if self.session: if self.session.cmd(f"test -f {self.CHECK_FILE}").exit_status: return None elif not os.path.exists(self.CHECK_FILE): return None version_file_content = None if self.session: version_file_content = self.session.cmd( f"cat {self.CHECK_FILE}" ).stdout_text else: try: with open(self.CHECK_FILE, encoding="utf-8") as check_file: version_file_content = check_file.read() except IOError as err: LOGGER.debug("Could not open %s", self.CHECK_FILE) LOGGER.debug("Exception: %s", str(err)) return None return self.CHECK_VERSION_REGEX.match(version_file_content) return None def version(self): """ Returns the version of the distro """ version = UNKNOWN_DISTRO_VERSION match = self._get_version_match() if match is not None: if len(match.groups()) > 0: version = match.groups()[0] return version def check_release(self): """ Checks if this has the conditions met to look for the release number """ return self.check_version() and self.CHECK_VERSION_REGEX.groups > 1 def release(self): """ Returns the release of the distro """ release = UNKNOWN_DISTRO_RELEASE match = self._get_version_match() if match is not None: if len(match.groups()) > 1: if match.groups()[1]: release = match.groups()[1] return release def get_distro(self): """ :param session: ssh connection between another machine Returns the :class:`LinuxDistro` this probe detected """ name = None version = UNKNOWN_DISTRO_VERSION release = UNKNOWN_DISTRO_RELEASE distro = None if self.check_name_for_file(): name = self.name_for_file() self.score += 1 if self.check_name_for_file_contains(): name = self.name_for_file_contains() self.score += 1 if self.check_version(): version = self.version() self.score += 1 if self.check_release(): release = self.release() self.score += 1 if self.session: arch = self.session.cmd("uname -m").stdout_text else: arch = platform.machine() # name is the first thing that should be identified. If we don't know # the distro name, we don't bother checking for versions if name is not None: distro = LinuxDistro(name, version, release, arch) else: distro = UNKNOWN_DISTRO return distro class RedHatProbe(Probe): """ Probe with version checks for Red Hat Enterprise Linux systems """ CHECK_FILE = "/etc/redhat-release" CHECK_FILE_CONTAINS = "Red Hat Enterprise Linux" CHECK_FILE_DISTRO_NAME = "rhel" CHECK_VERSION_REGEX = re.compile( r"Red Hat Enterprise Linux\s+\w*\s*release\s+(\d{1,2})\.(\d{1,2}).*" ) class CentosProbe(RedHatProbe): """ Probe with version checks for CentOS systems """ CHECK_FILE = "/etc/redhat-release" CHECK_FILE_CONTAINS = "CentOS Linux" CHECK_FILE_DISTRO_NAME = "centos" CHECK_VERSION_REGEX = re.compile(r"CentOS Linux release (\d{1,2})\.(\d{1,2}).*") class CentosStreamProbe(RedHatProbe): """ Probe with version checks for CentOS Stream systems """ CHECK_FILE = "/etc/redhat-release" CHECK_FILE_CONTAINS = "CentOS Stream" CHECK_FILE_DISTRO_NAME = "centos-stream" CHECK_VERSION_REGEX = re.compile(r"CentOS Stream release (\d{1,2})") class FedoraProbe(RedHatProbe): """ Probe with version checks for Fedora systems """ CHECK_FILE = "/etc/fedora-release" CHECK_FILE_CONTAINS = "Fedora" CHECK_FILE_DISTRO_NAME = "fedora" CHECK_VERSION_REGEX = re.compile(r"Fedora release (\d{1,2}).*") class AmazonLinuxProbe(Probe): """ Probe for Amazon Linux systems """ CHECK_FILE = "/etc/os-release" CHECK_FILE_CONTAINS = "Amazon Linux" CHECK_FILE_DISTRO_NAME = "amzn" CHECK_VERSION_REGEX = re.compile( r".*VERSION=\"(\d+)\.?(\d+)?\".*", re.MULTILINE | re.DOTALL ) class DebianProbe(Probe): """ Simple probe with file checks for Debian systems """ CHECK_FILE = "/etc/debian_version" CHECK_FILE_DISTRO_NAME = "debian" CHECK_VERSION_REGEX = re.compile(r".+/(.+)|(\d+)\.\d+") class UbuntuProbe(Probe): """ Simple probe for Ubuntu systems in general """ CHECK_FILE = "/etc/os-release" CHECK_FILE_CONTAINS = "ubuntu" CHECK_FILE_DISTRO_NAME = "Ubuntu" CHECK_VERSION_REGEX = re.compile( r".*VERSION_ID=\"(\d+\.\d+)\".*", re.MULTILINE | re.DOTALL ) class SUSEProbe(Probe): """ Simple probe for SUSE systems in general """ CHECK_FILE = "/etc/os-release" CHECK_FILE_CONTAINS = "SUSE" # this is the (incorrect) spelling used in python's platform # and tests are looking for it in distro.name. So keep using it CHECK_FILE_DISTRO_NAME = "SuSE" def get_distro(self): distro = super().get_distro() # if the default methods find SUSE, detect version if not distro.name == self.CHECK_FILE_DISTRO_NAME: return distro # we need to check VERSION_ID, which is number - VERSION can # be a string # for openSUSE Tumbleweed this will be e.g. 20161225 # for openSUSE Leap this will be e.g. 42.2 # for SUSE Linux Enterprise this will be e.g. 12 or 12.2 (for SP2) version_id_re = re.compile(r'VERSION_ID="([\d\.]*)"') version_id = None with open(self.CHECK_FILE, encoding="utf-8") as check_file: for line in check_file: match = version_id_re.match(line) if match: version_id = match.group(1) if version_id: version_parts = version_id.split(".") distro.version = int(version_parts[0]) if len(version_parts) > 1: distro.release = int(version_parts[1]) return distro class OpenEulerProbe(Probe): """ Simple probe for openEuler systems in general """ CHECK_FILE = "/etc/openEuler-release" CHECK_FILE_CONTAINS = "openEuler release" CHECK_FILE_DISTRO_NAME = "openEuler" CHECK_VERSION_REGEX = re.compile(r"openEuler release (\d+)\.(\d+).*") class UnionTechProbe(Probe): """ Simple probe for UnionTech systems in general """ CHECK_FILE = "/etc/os-version" CHECK_FILE_CONTAINS = "UnionTech OS" CHECK_FILE_DISTRO_NAME = "uos" CHECK_VERSION_REGEX = re.compile(r"MinorVersion=(\d+)") #: the complete list of probes that have been registered REGISTERED_PROBES = [] def register_probe(probe_class): """ Register a probe to be run during autodetection """ if probe_class not in REGISTERED_PROBES: REGISTERED_PROBES.append(probe_class) register_probe(RedHatProbe) register_probe(CentosProbe) register_probe(CentosStreamProbe) register_probe(FedoraProbe) register_probe(AmazonLinuxProbe) register_probe(DebianProbe) register_probe(SUSEProbe) register_probe(UbuntuProbe) register_probe(OpenEulerProbe) register_probe(UnionTechProbe) def detect(session=None): """ Attempts to detect the Linux Distribution running on this machine. If given an avocado.utils.ssh.Session object, it will attempt to detect the distro of another machine via an ssh connection. :param session: ssh connection between another machine :type session: avocado.utils.ssh.Session :returns: the detected :class:`LinuxDistro` or :data:`UNKNOWN_DISTRO` :rtype: :class:`LinuxDistro` """ results = [] for probe_class in REGISTERED_PROBES: probe_instance = probe_class(session) distro_result = probe_instance.get_distro() if distro_result is not UNKNOWN_DISTRO: results.append((distro_result, probe_instance)) results.sort(key=lambda t: t[1].score) if len(results) > 0: distro = results[-1][0] else: distro = UNKNOWN_DISTRO return distro class Spec: """ Describes a distro, usually for setting minimum distro requirements """ def __init__(self, name, min_version=None, min_release=None, arch=None): self.name = name self.min_version = min_version self.min_release = min_release self.arch = arch avocado-113.0/avocado/utils/dmesg.py000066400000000000000000000135271513561306000173670ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify. # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: 2020 IBM # Author: Praveen K Pandey # """Module for manipulate dmesg while running test.""" import logging import os import tempfile import time from functools import wraps from avocado.utils import genio, process LOGGER = logging.getLogger(__name__) class TestFail(AssertionError, Exception): """ Indicates that the test failed. This is here, just because of an impossible circular import. """ status = "FAIL" class DmesgError(Exception): """Base Exception Class for all dmesg utils exceptions.""" def fail_on_dmesg(level=5): """Dmesg fail method decorator Returns a class decorator used to signal the test when DmesgError exception is raised. :param level: Dmesg Level based on which the test failure should be raised :type level: int :return: Class decorator :rtype: class """ def dmesg_fail(cls): def raise_dmesg_fail(func): @wraps(func) def wrapper(*args, **kwargs): try: data = func(*args, **kwargs) collect_errors_by_level(level_check=level) return data except DmesgError as details: raise TestFail(repr(details)) from details return wrapper for key, value in list(vars(cls).items()): if callable(value) and value.__name__.startswith("test"): setattr(cls, key, raise_dmesg_fail(value)) return cls return dmesg_fail def clear_dmesg(): """function clear dmesg. The dmesg operation is a privileged user task. This function needs sudo permissions enabled on the target host """ cmd = "dmesg -c" status = process.system( cmd, timeout=30, ignore_status=True, verbose=False, shell=True, sudo=True ) if status: raise DmesgError("Unable to clear dmesg as some issue while clearing") def collect_dmesg(output_file=None): """Function collect dmesg and save in file. The dmesg operation is a privileged user task. This function needs sudo permissions enabled on the target host :param output_file: File use for save dmesg output if not provided it use tmp file which located in system /tmp path :type output_file: str :returns: file which contain dmesg :rtype: str """ if output_file is None: _, output_file = tempfile.mkstemp( suffix=f".-{time.strftime('%Y-%m-%d:%H:%M:%S')}", dir=tempfile.gettempdir() ) dmesg = process.system_output("dmesg", ignore_status=True, sudo=True).decode() genio.write_file(output_file, dmesg) if not os.path.isfile(output_file): raise DmesgError(f"{output_file} is not a valid file.") return output_file def collect_errors_dmesg(patterns): """Check patterns in dmesg. :param patterns : list variable to search in dmesg :returns: error log in form of list :rtype: list of str """ error = [] dmesg_log_file = collect_dmesg(None) for fail_pattern in patterns: for log in genio.read_file(dmesg_log_file).splitlines(): if fail_pattern in log: error.append(log) return error def collect_errors_by_level(output_file=None, level_check=5, skip_errors=None): """Verify dmesg having severity level of OS issue(s). :param output_file: The file used to save dmesg :type output_file: str :param level_check: level of severity of issues to be checked 1 - emerg 2 - emerg,alert 3 - emerg,alert,crit 4 - emerg,alert,crit,err 5 - emerg,alert,crit,err,warn :type level_check: int :skip_errors: list of dmesg error messages which want skip :type skip_errors: list """ if not isinstance(level_check, int): raise DmesgError("level_check param should be integer") dmsg_log = "" cmd = f"dmesg -T -l {','.join(map(str, range(0, int(level_check))))}" f"|grep ." out = process.run(cmd, timeout=30, ignore_status=True, verbose=False, shell=True) if not out.exit_status: err = "Found failures in dmesg" if skip_errors: dmsg_log = skip_dmesg_messages(out.stdout_text, skip_errors) else: dmsg_log = f"dmesg log:\n{out.stdout_text}" if dmsg_log: if output_file: with open(output_file, "w+", encoding="utf-8") as log_f: log_f.write(dmsg_log) err += f" Please check dmesg log {output_file}." else: err += " Please check dmesg log in debug log." LOGGER.debug(dmsg_log) raise DmesgError(f"Test is failed {err}") def skip_dmesg_messages(dmesg_stdout, skip_messages): """Remove some messages from a dmesg buffer. This method will remove some lines in a dmesg buffer if some strings are present. Returning the same buffer, but with less lines (in case of match). :dmesg_stdout: dmesg messages from which filter should be applied. This must be a decoded output buffer with new lines. :type dmesg_stdout: str :skip_messages: list of strings to be removed :type skip_messages: list """ def filter_strings(line): return not any(string in line for string in skip_messages) return "\n".join(filter(None, filter(filter_strings, dmesg_stdout.splitlines()))) avocado-113.0/avocado/utils/download.py000066400000000000000000000154231513561306000200740ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; specifically version 2 of the License. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # This code was inspired in the autotest project, # client/shared/utils.py # Authors: Martin J Bligh , Andy Whitcroft """ Methods to download URLs and regular files. """ import logging import os import shutil import socket import sys import urllib.parse from multiprocessing import Process from urllib.error import HTTPError from urllib.request import urlopen from avocado.utils import crypto, output log = logging.getLogger(__name__) def url_open(url, data=None, timeout=5): """ Wrapper to :func:`urllib2.urlopen` with timeout addition. :param url: URL to open. :param data: (optional) data to post. :param timeout: (optional) default timeout in seconds. Please, be aware that timeout here is just for blocking operations during the connection setup, since this method doesn't read the file from the url. :return: file-like object. :raises: `URLError`. """ try: result = urlopen(url, data=data, timeout=timeout) # pylint: disable=R1732 except (socket.timeout, HTTPError) as ex: msg = f"Failed downloading file: {str(ex)}" log.error(msg) return None msg = ( 'Opened URL "%s" and received response with headers including: ' 'content-length %s, date: "%s", last-modified: "%s"' ) log.debug( msg, url, result.headers.get("Content-Length", "UNKNOWN"), result.headers.get("Date", "UNKNOWN"), result.headers.get("Last-Modified", "UNKNOWN"), ) return result def _url_download(url, filename, data): src_file = url_open(url, data=data) if not src_file: msg = ( "Failed to get file. Probably timeout was reached when " "connecting to the server.\n" ) sys.stderr.write(msg) sys.exit(1) try: with open(filename, "wb") as dest_file: shutil.copyfileobj(src_file, dest_file) finally: src_file.close() def url_download(url, filename, data=None, timeout=300): """ Retrieve a file from given url. :param url: source URL. :param filename: destination path. :param data: (optional) data to post. :param timeout: (optional) default timeout in seconds. :return: `None`. """ process = Process(target=_url_download, args=(url, filename, data)) log.info("Fetching %s -> %s", url, filename) process.start() process.join(timeout) if process.is_alive(): process.terminate() process.join() raise OSError("Aborting downloading. Timeout was reached.") def url_download_interactive(url, output_file, title="", chunk_size=102400): """ Interactively downloads a given file url to a given output file. :type url: string :param url: URL for the file to be download :type output_file: string :param output_file: file name or absolute path on which to save the file to :type title: string :param title: optional title to go along the progress bar :type chunk_size: integer :param chunk_size: amount of data to read at a time """ output_dir = os.path.dirname(output_file) with open(output_file, "w+b") as open_output_file: with urlopen(url) as input_file: try: file_size = int(input_file.headers["Content-Length"]) except KeyError as exc: raise ValueError("Could not find file size in HTTP headers") from exc log.info( "Downloading %s, %s to %s", os.path.basename(url), output.display_data_size(file_size), output_dir, ) progress_bar = output.ProgressBar(maximum=file_size, title=title) # Download the file, while interactively updating the progress progress_bar.draw() while True: data = input_file.read(chunk_size) if data: progress_bar.append_amount(len(data)) open_output_file.write(data) else: progress_bar.update_amount(file_size) break def _get_file(src, dst, permissions=None): if src == dst: return None if urllib.parse.urlparse(src)[0] in ["http", "https"]: url_download(src, dst) else: shutil.copyfile(src, dst) if permissions: os.chmod(dst, permissions) return dst # pylint: disable=R0913 def get_file( src, dst, permissions=None, hash_expected=None, hash_algorithm="md5", download_retries=1, ): """ Gets a file from a source location, optionally using caching. If no hash_expected is provided, simply download the file. Else, keep trying to download the file until download_failures exceeds download_retries or the hashes match. If the hashes match, return dst. If download_failures exceeds download_retries, raise an EnvironmentError. :param src: source path or URL. May be local or a remote file. :param dst: destination path. :param permissions: (optional) set access permissions. :param hash_expected: Hash string that we expect the file downloaded to have. :param hash_algorithm: Algorithm used to calculate the hash string (md5, sha1). :param download_retries: Number of times we are going to retry a failed download. :raise: EnvironmentError. :return: destination path. """ def _verify_hash(filename): if os.path.isfile(filename): return crypto.hash_file(filename, algorithm=hash_algorithm) return None if hash_expected is None: return _get_file(src, dst, permissions) download_failures = 0 hash_file = _verify_hash(dst) while not hash_file == hash_expected: hash_file = _verify_hash(_get_file(src, dst, permissions)) if hash_file != hash_expected: log.error("It seems that dst %s is corrupted", dst) download_failures += 1 if download_failures > download_retries: raise EnvironmentError( f"Failed to retrieve {src}. " f"Possible reasons - Network connectivity " f"problems or incorrect hash_expected " f"provided -> '{hash_expected}'" ) log.error("Retrying download of src %s", src) return dst avocado-113.0/avocado/utils/exit_codes.py000066400000000000000000000014331513561306000204070ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. """ Avocado Utilities exit codes. These codes are returned on the command-line and may be used by the Avocado command-line utilities. """ #: The utility finished successfully UTILITY_OK = 0x0000 #: The utility ran, but needs to signalize a fail. UTILITY_FAIL = 0x0001 #: Utility generic crash UTILITY_GENERIC_CRASH = -1 avocado-113.0/avocado/utils/external/000077500000000000000000000000001513561306000175305ustar00rootroot00000000000000avocado-113.0/avocado/utils/external/__init__.py000066400000000000000000000010411513561306000216350ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2015 # Author: Cleber Rosa avocado-113.0/avocado/utils/external/gdbmi_parser.py000066400000000000000000000310341513561306000225410ustar00rootroot00000000000000# # Copyright (c) 2008 Michael Eddington # # Permission is hereby granted, free of charge, to any person obtaining a copy # of this software and associated documentation files (the "Software"), to deal # in the Software without restriction, including without limitation the rights # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell # copies of the Software, and to permit persons to whom the Software is # furnished to do so, subject to the following conditions: # # The above copyright notice and this permission notice shall be included in # all copies or substantial portions of the Software. # # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE # SOFTWARE. # # Authors: # Frank Laub (frank.laub@gmail.com) # Michael Eddington (mike@phed.org) import pprint import re from avocado.utils.external import spark class GdbMiError(Exception): """ Exception raised when there is an error parsing GDB/MI output. """ class Token: def __init__(self, token_type, value=None): self.type = token_type self.value = value def __lt__(self, o): return self.type < o def __gt__(self, o): return self.type > o def __le__(self, o): return self.type <= o def __ge__(self, o): return self.type >= o def __eq__(self, o): return self.type == o def __ne__(self, o): return self.type != o def __repr__(self): return self.value or self.type class AST: def __init__(self, ast_type): self.type = ast_type self._kids = [] def __getitem__(self, i): return self._kids[i] def __setitem__(self, i, k): self._kids[i] = k def __len__(self): return len(self._kids) def __lt__(self, o): return self.type < o def __gt__(self, o): return self.type > o def __le__(self, o): return self.type <= o def __ge__(self, o): return self.type >= o def __eq__(self, o): return self.type == o def __ne__(self, o): return self.type != o class GdbMiScannerBase(spark.GenericScanner): def tokenize(self, s): # pylint: disable=W0221 self.rv = [] # pylint: disable=W0201 spark.GenericScanner.tokenize(self, s) return self.rv def t_nl(self, s): # pylint: disable=W0613 r"\n|\r\n" self.rv.append(Token("nl")) def t_whitespace(self, s): # pylint: disable=W0613 r"[ \t\f\v]+" def t_symbol(self, s): r",|\{|\}|\[|\]|\=" self.rv.append(Token(s, s)) def t_result_type(self, s): r"\*|\+|\^" self.rv.append(Token("result_type", s)) def t_stream_type(self, s): r"\@|\&|\~" self.rv.append(Token("stream_type", s)) def t_string(self, s): r"[\w-]+" self.rv.append(Token("string", s)) def t_c_string(self, s): r"\".*?(? 2: print(f"{tokens[i - 3]} {tokens[i - 2]} " f"{tokens[i - 1]} {tokens[i]}") raise GdbMiError(f"Syntax error at or near {int(i)}:'{token}' token") class GdbMiInterpreter(spark.GenericASTTraversal): def __init__(self, ast): spark.GenericASTTraversal.__init__(self, ast) self.postorder() @staticmethod def __translate_type(token_type): table = { "^": "result", "=": "notify", "+": "status", "*": "exec", "~": "console", "@": "target", "&": "log", } return table[token_type] @staticmethod def n_result(node): # result ::= variable = value node.value = {node[0].value: node[2].value} # print 'result: %s' % node.value @staticmethod def n_tuple(node): if len(node) == 2: # tuple ::= {} node.value = {} elif len(node) == 3: # tuple ::= { result } node.value = node[1].value elif len(node) == 4: # tuple ::= { result result_list } node.value = node[1].value for result in node[2].value: for n, v in list(result.items()): if n in node.value: # print '**********list conversion: [%s] %s -> %s' % (n, node.value[n], v) old = node.value[n] if not isinstance(old, list): node.value[n] = [node.value[n]] node.value[n].append(v) else: node.value[n] = v else: raise GdbMiError("Invalid tuple") # print 'tuple: %s' % node.value @staticmethod def n_list(node): if len(node) == 2: # list ::= [] node.value = [] elif len(node) == 3: # list ::= [ value ] node.value = [node[1].value] elif len(node) == 4: # list ::= [ value value_list ] node.value = [node[1].value] + node[2].value # list ::= [ result ] # list ::= [ result result_list ] # list ::= { value } # list ::= { value value_list } # print 'list %s' % node.value @staticmethod def n_value_list(node): if len(node) == 2: # value_list ::= , value node.value = [node[1].value] elif len(node) == 3: # value_list ::= , value value_list node.value = [node[1].value] + node[2].value @staticmethod def n_result_list(node): if len(node) == 2: # result_list ::= , result node.value = [node[1].value] else: # result_list ::= , result result_list node.value = [node[1].value] + node[2].value # print 'result_list: %s' % node.value @staticmethod def n_result_record(node): node.value = node[0].value if len(node) == 3: # result_record ::= result_header result_list nl node.value["results"] = node[1].value elif len(node) == 2: # result_record ::= result_header nl pass # print 'result_record: %s' % (node.value) def n_result_header(self, node): if len(node) == 3: # result_header ::= token result_type class node.value = { "token": node[0].value, "type": self.__translate_type(node[1].value), "class_": node[2].value, "record_type": "result", } elif len(node) == 2: # result_header ::= result_type class node.value = { "token": None, "type": self.__translate_type(node[0].value), "class_": node[1].value, "record_type": "result", } def n_stream_record(self, node): # stream_record ::= stream_type c_string nl node.value = { "type": self.__translate_type(node[0].value), "value": node[1].value, "record_type": "stream", } # print 'stream_record: %s' % node.value @staticmethod def n_record_list(node): if len(node) == 1: # record_list ::= generic_record node.value = [node[0].value] elif len(node) == 2: # record_list ::= generic_record record_list node.value = [node[0].value] + node[1].value # print 'record_list: %s' % node.value # def default(self, node): # print 'default: ' + node.type class GdbDynamicObject: def __init__(self, dict_): self.graft(dict_) def __repr__(self): return pprint.pformat(self.__dict__) def __bool__(self): return len(self.__dict__) > 0 def __getitem__(self, i): if not i and len(self.__dict__) > 0: return self raise IndexError def __getattr__(self, name): if name.startswith("__"): raise AttributeError def graft(self, dict_): for name, value in list(dict_.items()): name = name.replace("-", "_") if isinstance(value, dict): value = GdbDynamicObject(value) elif isinstance(value, list): x = value value = [] for item in x: if isinstance(item, dict): item = GdbDynamicObject(item) value.append(item) setattr(self, name, value) class GdbMiRecord: def __init__(self, record): self.result = None for name, value in list(record[0].items()): name = name.replace("-", "_") if name == "results": for result in value: if not self.result: self.result = GdbDynamicObject(result) else: # graft this result to self.results self.result.graft(result) else: setattr(self, name, value) def __repr__(self): return pprint.pformat(self.__dict__) class session: # pylint: disable=C0103 def __init__(self): self.the_scanner = GdbMiScanner() self.the_parser = GdbMiParser() self.the_interpreter = GdbMiInterpreter self.the_output = GdbMiRecord def scan(self, data_input): return self.the_scanner.tokenize(data_input) def parse(self, tokens): return self.the_parser.parse(tokens) def process(self, data_input): tokens = self.scan(data_input) ast = self.parse(tokens) self.the_interpreter(ast) return self.the_output(ast.value) # pylint: disable=wrong-import-position from avocado.utils.deprecation import log_deprecation log_deprecation.warning("external.gdbmi_parser") avocado-113.0/avocado/utils/external/spark.py000066400000000000000000000662561513561306000212410ustar00rootroot00000000000000# Copyright (c) 1998-2002 John Aycock # # Permission is hereby granted, free of charge, to any person obtaining # a copy of this software and associated documentation files (the # "Software"), to deal in the Software without restriction, including # without limitation the rights to use, copy, modify, merge, publish, # distribute, sublicense, and/or sell copies of the Software, and to # permit persons to whom the Software is furnished to do so, subject to # the following conditions: # # The above copyright notice and this permission notice shall be # included in all copies or substantial portions of the Software. # # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, # EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF # MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. # IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY # CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, # TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE # SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. __version__ = "SPARK-0.7 (pre-alpha-7)" import re def _namelist(instance): namelist, namedict, classlist = [], {}, [instance.__class__] for c in classlist: for b in c.__bases__: classlist.append(b) # pylint: disable=W4701 for name in c.__dict__.keys(): if name not in namedict: namelist.append(name) namedict[name] = 1 return namelist class GenericScanner: def __init__(self, flags=0): pattern = self.reflect() self.re = re.compile(pattern, re.VERBOSE | flags) self.index2func = {} for name, number in self.re.groupindex.items(): self.index2func[number - 1] = getattr(self, "t_" + name) def makeRE(self, name): # pylint: disable=C0103 doc = getattr(self, name).__doc__ rv = f"(?P<{name[2:]}>{doc})" return rv def reflect(self): rv = [] for name in _namelist(self): if name[:2] == "t_" and name != "t_default": rv.append(self.makeRE(name)) rv.append(self.makeRE("t_default")) return "|".join(rv) @staticmethod def error(s, pos): # pylint: disable=W0613 print(f"Lexical error at position {pos}") raise SystemExit def tokenize(self, s): pos = 0 n = len(s) while pos < n: m = self.re.match(s, pos) if m is None: self.error(s, pos) groups = m.groups() for i, group in enumerate(groups): if group and i in self.index2func: self.index2func[i](group) pos = m.end() @staticmethod def t_default(s): # pylint: disable=W0613 r"( . | \n )+" print("Specification error: unmatched input") raise SystemExit # # Extracted from GenericParser and made global so that [un]picking works. # class _State: def __init__(self, stateno, items): self.t, self.complete, self.items = [], [], items self.stateno = stateno # pylint: disable=R0902,R0904 class GenericParser: # # An Earley parser, as per J. Earley, "An Efficient Context-Free # Parsing Algorithm", CACM 13(2), pp. 94-102. Also J. C. Earley, # "An Efficient Context-Free Parsing Algorithm", Ph.D. thesis, # Carnegie-Mellon University, August 1968. New formulation of # the parser according to J. Aycock, "Practical Earley Parsing # and the SPARK Toolkit", Ph.D. thesis, University of Victoria, # 2001, and J. Aycock and R. N. Horspool, "Practical Earley # Parsing", unpublished paper, 2001. # def __init__(self, start): self.rules = {} self.rule2func = {} self.rule2name = {} self.collectRules() self.augment(start) self.ruleschanged = 1 _NULLABLE = r"\e_" _START = "START" _BOF = "|-" # # When pickling, take the time to generate the full state machine; # some information is then extraneous, too. Unfortunately we # can't save the rule2func map. # def __getstate__(self): if self.ruleschanged: # # FIX ME - duplicated from parse() # self.computeNull() self.newrules = {} # pylint: disable=W0201 self.new2old = {} # pylint: disable=W0201 self.makeNewRules() self.ruleschanged = 0 self.edges, self.cores = {}, {} # pylint: disable=W0201 self.states = {0: self.makeState0()} # pylint: disable=W0201 self.makeState(0, self._BOF) # # FIX ME - should find a better way to do this.. # changes = 1 while changes: changes = 0 for k, v in self.edges.items(): if v is None: state, sym = k if state in self.states: self.goto(state, sym) changes = 1 rv = self.__dict__.copy() for s in self.states.values(): del s.items del rv["rule2func"] del rv["nullable"] del rv["cores"] return rv def __setstate__(self, d): self.rules = {} self.rule2func = {} self.rule2name = {} self.collectRules() start = d["rules"][self._START][0][1][1] # Blech. self.augment(start) d["rule2func"] = self.rule2func d["makeSet"] = self.makeSet_fast self.__dict__ = d # pylint: disable=W0201 # # A hook for GenericASTBuilder and GenericASTMatcher. Mess # thee not with this; nor shall thee toucheth the _preprocess # argument to addRule. # @staticmethod def preprocess(rule, func): return rule, func def addRule(self, doc, func, _preprocess=1): # pylint: disable=C0103 fn = func rules = doc.split() index = [] for i, rule in enumerate(rules): if rule == "::=": index.append(i - 1) index.append(len(rules)) for i in range(len(index) - 1): lhs = rules[index[i]] rhs = rules[index[i] + 2 : index[i + 1]] rule = (lhs, tuple(rhs)) if _preprocess: rule, fn = self.preprocess(rule, func) if lhs in self.rules: self.rules[lhs].append(rule) else: self.rules[lhs] = [rule] self.rule2func[rule] = fn self.rule2name[rule] = func.__name__[2:] self.ruleschanged = 1 def collectRules(self): # pylint: disable=C0103 for name in _namelist(self): if name[:2] == "p_": func = getattr(self, name) doc = func.__doc__ self.addRule(doc, func) def augment(self, start): rule = f"{self._START} ::= {self._BOF} {start}" self.addRule(rule, lambda args: args[1], 0) def computeNull(self): # pylint: disable=C0103 self.nullable = {} # pylint: disable=W0201 tbd = [] for rulelist in self.rules.values(): lhs = rulelist[0][0] self.nullable[lhs] = 0 for rule in rulelist: rhs = rule[1] if not rhs: self.nullable[lhs] = 1 continue # # We only need to consider rules which # consist entirely of nonterminal symbols. # This should be a savings on typical # grammars. # for sym in rhs: if sym not in self.rules: break else: tbd.append(rule) changes = 1 while changes: changes = 0 for lhs, rhs in tbd: if self.nullable[lhs]: continue for sym in rhs: if not self.nullable[sym]: break else: self.nullable[lhs] = 1 changes = 1 def makeState0(self): # pylint: disable=C0103 s0 = _State(0, []) for rule in self.newrules[self._START]: s0.items.append((rule, 0)) return s0 def finalState(self, tokens): # pylint: disable=C0103 # # Yuck. # if len(self.newrules[self._START]) == 2 and not tokens: return 1 start = self.rules[self._START][0][1][1] return self.goto(1, start) def makeNewRules(self): # pylint: disable=C0103 worklist = [] for rulelist in self.rules.values(): for rule in rulelist: worklist.append((rule, 0, 1, rule)) for rule, i, candidate, oldrule in worklist: lhs, rhs = rule n = len(rhs) while i < n: sym = rhs[i] if sym not in self.rules or not self.nullable[sym]: candidate = 0 i += 1 continue newrhs = list(rhs) newrhs[i] = self._NULLABLE + sym newrule = (lhs, tuple(newrhs)) # pylint: disable=W4701 worklist.append((newrule, i + 1, candidate, oldrule)) candidate = 0 i += 1 else: # pylint: disable=W0120 if candidate: lhs = self._NULLABLE + lhs rule = (lhs, rhs) if lhs in self.newrules: self.newrules[lhs].append(rule) else: self.newrules[lhs] = [rule] self.new2old[rule] = oldrule @staticmethod def typestring(token): # pylint: disable=W0613 return None @staticmethod def error(token): print(f"Syntax error at or near `{token}' token") raise SystemExit def parse(self, tokens): sets = [[(1, 0), (2, 0)]] self.links = {} # pylint: disable=W0201 if self.ruleschanged: self.computeNull() self.newrules = {} # pylint: disable=W0201 self.new2old = {} # pylint: disable=W0201 self.makeNewRules() self.ruleschanged = 0 self.edges, self.cores = {}, {} # pylint: disable=W0201 self.states = {0: self.makeState0()} # pylint: disable=W0201 self.makeState(0, self._BOF) for i, token in enumerate(tokens): sets.append([]) if sets[i] == []: break self.makeSet(token, sets, i) else: sets.append([]) self.makeSet(None, sets, len(tokens)) finalitem = (self.finalState(tokens), 0) if finalitem not in sets[-2]: if len(tokens) > 0: self.error(tokens[i - 1]) # pylint: disable=W0631 else: self.error(None) return self.buildTree(self._START, finalitem, tokens, len(sets) - 2) def isnullable(self, sym): # # For symbols in G_e only. If we weren't supporting 1.5, # could just use sym.startswith(). # return self._NULLABLE == sym[0 : len(self._NULLABLE)] def skip(self, hs, pos=0): n = len(hs[1]) while pos < n: if not self.isnullable(hs[1][pos]): break pos += 1 return pos def makeState(self, state, sym): # pylint: disable=R0914, R0912, C0103 assert sym is not None # # Compute \epsilon-kernel state's core and see if # it exists already. # kitems = [] for rule, pos in self.states[state].items: _, rhs = rule if rhs[pos : pos + 1] == (sym,): kitems.append((rule, self.skip(rule, pos + 1))) core = kitems core.sort() tcore = tuple(core) if tcore in self.cores: return self.cores[tcore] # # Nope, doesn't exist. Compute it and the associated # \epsilon-nonkernel state together; we'll need it right away. # k = self.cores[tcore] = len(self.states) ks, nk = _State(k, kitems), _State(k + 1, []) self.states[k] = ks predicted = {} edges = self.edges rules = self.newrules for x in ks, nk: worklist = x.items for item in worklist: rule, pos = item _, rhs = rule if pos == len(rhs): x.complete.append(rule) continue next_sym = rhs[pos] key = (x.stateno, next_sym) if next_sym not in rules: if key not in edges: edges[key] = None x.t.append(next_sym) else: edges[key] = None if next_sym not in predicted: predicted[next_sym] = 1 for prule in rules[next_sym]: ppos = self.skip(prule) new = (prule, ppos) nk.items.append(new) # # Problem: we know K needs generating, but we # don't yet know about NK. Can't commit anything # regarding NK to self.edges until we're sure. Should # we delay committing on both K and NK to avoid this # hacky code? This creates other problems.. # if x is ks: edges = {} if nk.items == []: return k # # Check for \epsilon-nonkernel's core. Unfortunately we # need to know the entire set of predicted nonterminals # to do this without accidentally duplicating states. # core = sorted(predicted.keys()) tcore = tuple(core) if tcore in self.cores: self.edges[(k, None)] = self.cores[tcore] return k nk_ = self.cores[tcore] = self.edges[(k, None)] = nk.stateno self.edges.update(edges) self.states[nk_] = nk return k def goto(self, state, sym): key = (state, sym) if key not in self.edges: # # No transitions from state on sym. # return None rv = self.edges[key] if rv is None: # # Target state isn't generated yet. Remedy this. # rv = self.makeState(state, sym) self.edges[key] = rv return rv def gotoT(self, state, t): # pylint: disable=C0103 return [self.goto(state, t)] def gotoST(self, state, st): # pylint: disable=C0103 rv = [] for t in self.states[state].t: if st == t: rv.append(self.goto(state, t)) return rv # pylint: disable=R0913 def add(self, input_set, item, i=None, predecessor=None, causal=None): if predecessor is None: if item not in input_set: input_set.append(item) else: key = (item, i) if item not in input_set: self.links[key] = [] input_set.append(item) self.links[key].append((predecessor, causal)) def makeSet(self, token, sets, i): # pylint: disable=R0914,C0103 cur, next_item = sets[i], sets[i + 1] ttype = ( # pylint: disable=R1709 token is not None and self.typestring(token) or None ) if ttype is not None: fn, arg = self.gotoT, ttype else: fn, arg = self.gotoST, token for item in cur: ptr = (item, i) state, parent = item add = fn(state, arg) for k in add: if k is not None: self.add(next_item, (k, parent), i + 1, ptr) nk = self.goto(k, None) if nk is not None: self.add(next_item, (nk, i + 1)) if parent == i: continue for rule in self.states[state].complete: lhs, _ = rule for pitem in sets[parent]: pstate, pparent = pitem k = self.goto(pstate, lhs) if k is not None: why = (item, i, rule) pptr = (pitem, parent) self.add(cur, (k, pparent), i, pptr, why) nk = self.goto(k, None) if nk is not None: self.add(cur, (nk, i)) def makeSet_fast(self, token, sets, i): # pylint: disable=R0914, R0912, C0103 # # Call *only* when the entire state machine has been built! # It relies on self.edges being filled in completely, and # then duplicates and inlines code to boost speed at the # cost of extreme ugliness. # cur, next_item = sets[i], sets[i + 1] ttype = ( # pylint: disable=R1709 token is not None and self.typestring(token) or None ) for item in cur: # pylint: disable=R1702 ptr = (item, i) state, parent = item if ttype is not None: k = self.edges.get((state, ttype), None) if k is not None: # self.add(next_item, (k, parent), i + 1, ptr) # INLINED --v new = (k, parent) key = (new, i + 1) if new not in next_item: self.links[key] = [] next_item.append(new) self.links[key].append((ptr, None)) # INLINED --^ # nk = self.goto(k, None) nk = self.edges.get((k, None), None) if nk is not None: # self.add(next_item, (nk, i + 1)) # INLINED --v new = (nk, i + 1) if new not in next_item: next_item.append(new) # INLINED --^ else: add = self.gotoST(state, token) for k in add: if k is not None: self.add(next_item, (k, parent), i + 1, ptr) # nk = self.goto(k, None) nk = self.edges.get((k, None), None) if nk is not None: self.add(next_item, (nk, i + 1)) if parent == i: continue for rule in self.states[state].complete: lhs, _ = rule for pitem in sets[parent]: pstate, pparent = pitem # k = self.goto(pstate, lhs) k = self.edges.get((pstate, lhs), None) if k is not None: why = (item, i, rule) pptr = (pitem, parent) # self.add(cur, (k, pparent), # i, pptr, why) # INLINED --v new = (k, pparent) key = (new, i) if new not in cur: self.links[key] = [] cur.append(new) self.links[key].append((pptr, why)) # INLINED --^ # nk = self.goto(k, None) nk = self.edges.get((k, None), None) if nk is not None: # self.add(cur, (nk, i)) # INLINED --v new = (nk, i) if new not in cur: cur.append(new) # INLINED --^ def predecessor(self, key, causal): for p, c in self.links[key]: if c == causal: return p assert 0 def causal(self, key): links = self.links[key] if len(links) == 1: return links[0][1] choices = [] rule2cause = {} for _, c in links: rule = c[2] choices.append(rule) rule2cause[rule] = c return rule2cause[self.ambiguity(choices)] def deriveEpsilon(self, nt): # pylint: disable=C0103 if len(self.newrules[nt]) > 1: rule = self.ambiguity(self.newrules[nt]) else: rule = self.newrules[nt][0] rhs = rule[1] attr = [None] * len(rhs) for i in range(len(rhs) - 1, -1, -1): attr[i] = self.deriveEpsilon(rhs[i]) return self.rule2func[self.new2old[rule]](attr) def buildTree(self, nt, item, tokens, k): # pylint: disable=C0103 state, _ = item choices = [] for rule in self.states[state].complete: if rule[0] == nt: choices.append(rule) rule = choices[0] if len(choices) > 1: rule = self.ambiguity(choices) rhs = rule[1] attr = [None] * len(rhs) for i in range(len(rhs) - 1, -1, -1): sym = rhs[i] if sym not in self.newrules: if sym != self._BOF: attr[i] = tokens[k - 1] key = (item, k) item, k = self.predecessor(key, None) # elif self.isnullable(sym): elif self._NULLABLE == sym[0 : len(self._NULLABLE)]: attr[i] = self.deriveEpsilon(sym) else: key = (item, k) why = self.causal(key) attr[i] = self.buildTree(sym, why[0], tokens, why[1]) item, k = self.predecessor(key, why) return self.rule2func[self.new2old[rule]](attr) def ambiguity(self, rules): # # FIX ME - problem here and in collectRules() if the same rule # appears in >1 method. Also undefined results if rules # causing the ambiguity appear in the same method. # sortlist = [] name2index = {} for i, rule in enumerate(rules): _, rhs = rule = rule name = self.rule2name[self.new2old[rule]] sortlist.append((len(rhs), name)) name2index[name] = i sortlist.sort() result_list = [name for _, name in sortlist] return rules[name2index[self.resolve(result_list)]] @staticmethod def resolve(input_list): # # Resolve ambiguity in favor of the shortest RHS. # Since we walk the tree from the top down, this # should effectively resolve in favor of a "shift". # return input_list[0] # # GenericASTBuilder automagically constructs a concrete/abstract syntax tree # for a given input. The extra argument is a class (not an instance!) # which supports the "__setslice__" and "__len__" methods. # # FIX ME - silently overrides any user code in methods. # class GenericASTBuilder(GenericParser): def __init__(self, AST, start): GenericParser.__init__(self, start) self.ast = AST def preprocess(self, rule, func): # pylint: disable=W0221 # pylint: disable=C3001 rebind = ( lambda lhs, self=self: lambda args, lhs=lhs, self=self: self.buildASTNode( args, lhs ) ) lhs, _ = rule return rule, rebind(lhs) def buildASTNode(self, args, lhs): # pylint: disable=C0103 children = [] for arg in args: if isinstance(arg, self.ast): children.append(arg) else: children.append(self.terminal(arg)) return self.nonterminal(lhs, children) @staticmethod def terminal(token): return token def nonterminal(self, token_type, args): rv = self.ast(token_type) rv[: len(args)] = args return rv # # GenericASTTraversal is a Visitor pattern according to Design Patterns. For # each node it attempts to invoke the method n_, falling # back onto the default() method if the n_* can't be found. The preorder # traversal also looks for an exit hook named n__exit (no default # routine is called if it's not found). To prematurely halt traversal # of a subtree, call the prune() method -- this only makes sense for a # preorder traversal. Node type is determined via the typestring() method. # class GenericASTTraversalPruningException(Exception): pass class GenericASTTraversal: def __init__(self, ast): self.ast = ast @staticmethod def typestring(node): return node.type @staticmethod def prune(): raise GenericASTTraversalPruningException def preorder(self, node=None): if node is None: node = self.ast try: name = "n_" + self.typestring(node) if hasattr(self, name): func = getattr(self, name) func(node) else: self.default(node) except GenericASTTraversalPruningException: return for kid in node: self.preorder(kid) name = name + "_exit" if hasattr(self, name): func = getattr(self, name) func(node) def postorder(self, node=None): if node is None: node = self.ast for kid in node: self.postorder(kid) name = "n_" + self.typestring(node) if hasattr(self, name): func = getattr(self, name) func(node) else: self.default(node) def default(self, node): pass # # GenericASTMatcher. AST nodes must have "__getitem__" and "__cmp__" # implemented. # # FIX ME - makes assumptions about how GenericParser walks the parse tree. # class GenericASTMatcher(GenericParser): def __init__(self, start, ast): GenericParser.__init__(self, start) self.ast = ast def preprocess(self, rule, func): # pylint: disable=W0221 # pylint: disable=C3001 rebind = ( lambda func, self=self: lambda args, func=func, self=self: self.foundMatch( args, func ) ) lhs, rhs = rule rhslist = list(rhs) rhslist.reverse() return (lhs, tuple(rhslist)), rebind(func) @staticmethod def foundMatch(args, func): # pylint: disable=C0103 func(args[-1]) return args[-1] def match_r(self, node): self.input.insert(0, node) children = 0 for child in node: if not children: self.input.insert(0, "(") children += 1 self.match_r(child) if children > 0: self.input.insert(0, ")") def match(self, ast=None): if ast is None: ast = self.ast self.input = [] # pylint: disable=W0201 self.match_r(ast) self.parse(self.input) def resolve(self, input_list): # pylint: disable=W0221 # # Resolve ambiguity in favor of the longest RHS. # return input_list[-1] # pylint: disable=wrong-import-position from avocado.utils.deprecation import log_deprecation log_deprecation.warning("external.spark") avocado-113.0/avocado/utils/file_utils.py000066400000000000000000000056671513561306000204350ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; specifically version 2 of the License. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # This code was inspired in the autotest project, # client/shared/utils.py # Authors: Christian Herdtweck """ SUMMARY ------------------------------------------------------ Utilities for file tests. INTERFACE ------------------------------------------------------ """ import logging import os from glob import iglob from grp import getgrgid, getgrnam from pwd import getpwnam, getpwuid from stat import S_IMODE LOG = logging.getLogger(__name__) def check_owner(owner, group, file_name_pattern, check_recursive=False): """ Verifies that given file belongs to given owner and group. :param str owner: user that owns of the file :param str group: group of the owner of the file :param str file_name_pattern: can be a glob :param check_recursive: if file_name_pattern matches a directory, recurse into that subdir or not :type check_recursive: bool :raises: :py:class:`RuntimeError` if file has wrong owner or group """ for file_name in iglob(file_name_pattern): actual_id = os.stat(file_name).st_uid if actual_id != getpwnam(owner).pw_uid: raise RuntimeError( f"file {file_name} has wrong owner " f"{getpwuid(actual_id).pw_name} (should be {owner})" ) actual_id = os.stat(file_name).st_gid if actual_id != getgrnam(group).gr_gid: raise RuntimeError( f"file {file_name} has wrong group " f"{getgrgid(actual_id).gr_name} (should be {group})" ) LOG.debug("checked owner %s:%s of file %s", owner, group, file_name) if check_recursive and os.path.isdir(file_name): new_pattern = os.path.join(file_name, os.path.basename(file_name_pattern)) check_owner(owner, group, new_pattern, True) def check_permissions(perms, file_name_pattern): """ Verify that a given file has a given numeric permission. :param int perms: best given in octal form, e.g. 0o755 :param str file_name_pattern: can be a glob :raises: :py:class:`RuntimeError` if file has wrong permissions """ for file_name in iglob(file_name_pattern): actual_perms = S_IMODE(os.stat(file_name).st_mode) if perms != actual_perms: raise RuntimeError( f"file {file_name} has permissions {oct(actual_perms)} " f"(should be {oct(perms)})!" ) LOG.debug("checked permissions %s of file %s", oct(perms), file_name) avocado-113.0/avocado/utils/filelock.py000066400000000000000000000062711513561306000200560ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2016 # Author: Amador Pahim """ Utility for individual file access control implemented via PID lock files. """ import os import time from avocado.utils.process import pid_exists class AlreadyLocked(Exception): pass class LockFailed(Exception): pass class FileLock: """ Creates an exclusive advisory lock for a file. All processes should use and honor the advisory locking scheme, but uncooperative processes are free to ignore the lock and access the file in any way they choose. """ def __init__(self, filename, timeout=0): self.filename = f"{filename}.lock" self.pid = f"{os.getpid()}".encode() self.locked = False self.timeout = timeout def __enter__(self): flags = os.O_CREAT | os.O_EXCL | os.O_WRONLY | os.O_SYNC timelimit = time.monotonic() + self.timeout while True: try: fd = os.open(self.filename, flags) os.write(fd, self.pid) os.close(fd) self.locked = True return self except Exception as exc: # pylint: disable=W0703 try: # Read the file to realize what's happening. with open(self.filename, "r", encoding="utf-8") as f: content = f.read() existing_lock_pid = int(content) if existing_lock_pid != self.pid: # If there's no process with the PID in lockfile, # let's try to remove the lockfile to acquire the # lock in the next iteration. if not pid_exists(existing_lock_pid): os.remove(self.filename) continue except Exception: # pylint: disable=W0703 # If we cannot read the lock file, let's just # go on. Maybe in next iteration (if we have time) # we have a better luck. pass # If we get to this point, the lock file is there, it belongs # to a running process and we are just waiting for the lock # to be released. if self.timeout <= 0: raise AlreadyLocked("File is already locked.") from exc if time.monotonic() > timelimit: raise AlreadyLocked("Timeout waiting for the lock.") from exc time.sleep(0.1) def __exit__(self, *args): if self.locked: try: os.remove(self.filename) self.locked = False except OSError: pass avocado-113.0/avocado/utils/gdb.py000066400000000000000000000736671513561306000170370ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2014 # Authors: Cleber Rosa __doc__ = """ GDB Communication and Debugging Utilities This module provides comprehensive functionality for interacting with the GNU Debugger (GDB) through multiple interfaces and protocols. It supports both local debugging sessions and remote debugging scenarios. Key Features: - GDB/MI (Machine Interface) communication for programmatic control - GDB Server management for remote debugging sessions - GDB Remote Protocol client implementation - Command execution with structured result parsing - Breakpoint management and program flow control - Support for both CLI and MI command interfaces Main Classes: GDB: Wraps a local GDB subprocess with MI interface communication GDBServer: Manages a gdbserver instance for remote debugging GDBRemote: Implements GDB remote protocol client for direct communication CommandResult: Encapsulates command execution results and metadata Common Usage Patterns: - Automated debugging workflows in test environments - Remote debugging of embedded or containerized applications - Programmatic analysis of application crashes and core dumps - Integration with continuous integration and testing frameworks The module handles low-level protocol details, message parsing, and connection management, providing a high-level Python interface for GDB operations. """ __all__ = ["GDB", "GDBServer", "GDBRemote"] import fcntl import os import socket import subprocess import tempfile import time from avocado.utils.external import gdbmi_parser from avocado.utils.network import ports from avocado.utils.path import find_command GDB_PROMPT = b"(gdb)" GDB_EXIT = b"^exit" GDB_BREAK_CONDITIONS = [GDB_PROMPT, GDB_EXIT] #: How the remote protocol signals a transmission success (in ACK mode) REMOTE_TRANSMISSION_SUCCESS = "+" #: How the remote protocol signals a transmission failure (in ACK mode) REMOTE_TRANSMISSION_FAILURE = "-" #: How the remote protocol flags the start of a packet REMOTE_PREFIX = b"$" #: How the remote protocol flags the end of the packet payload, and that the #: two digits checksum follow REMOTE_DELIMITER = b"#" #: Rather conservative default maximum packet size for clients using the #: remote protocol. Individual connections can ask (and do so by default) #: the server about the maximum packet size they can handle. REMOTE_MAX_PACKET_SIZE = 1024 class UnexpectedResponseError(Exception): """A response different from the one expected was received from GDB""" class ServerInitTimeoutError(Exception): """Server took longer than expected to initialize itself properly""" class InvalidPacketError(Exception): """Packet received has invalid format""" class NotConnectedError(Exception): """GDBRemote is not connected to a remote GDB server""" class RetransmissionRequestedError(Exception): """Message integrity was not validated and retransmission is being requested""" def parse_mi(line): """Parse a GDB/MI line :param line: a string supposedly coming from GDB using MI language :type line: str :returns: a parsed GDB/MI response :rtype: gdbmi_parser.GdbMiRecord """ if not line.endswith("\n"): line = f"{line}\n" return gdbmi_parser.session().process(line) def encode_mi_cli(command): """Encodes a regular (CLI) command into the proper MI form :param command: the regular cli command to send :type command: str :returns: the encoded (escaped) MI command :rtype: str """ return f'-interpreter-exec console "{command}"' def is_stopped_exit(parsed_mi_msg): """Check if a parsed GDB MI message indicates the program exited normally. :param parsed_mi_msg: a parsed GDB MI message object :type parsed_mi_msg: gdbmi_parser.GdbMiRecord :returns: True if the message indicates normal program exit, False otherwise :rtype: bool """ return ( hasattr(parsed_mi_msg, "class_") and (parsed_mi_msg.class_ == "stopped") and hasattr(parsed_mi_msg, "result") and hasattr(parsed_mi_msg.result, "reason") and (parsed_mi_msg.result.reason == "exited") ) def is_thread_group_exit(parsed_mi_msg): """Check if a parsed GDB MI message indicates a thread group has exited. :param parsed_mi_msg: a parsed GDB MI message object :type parsed_mi_msg: gdbmi_parser.GdbMiRecord :returns: True if the message indicates thread group exit, False otherwise :rtype: bool """ return hasattr(parsed_mi_msg, "class_") and ( parsed_mi_msg.class_ == "thread-group-exited" ) def is_exit(parsed_mi_msg): """Check if a parsed GDB MI message indicates any type of program exit. This function combines checks for both normal program exit and thread group exit. :param parsed_mi_msg: a parsed GDB MI message object :type parsed_mi_msg: gdbmi_parser.GdbMiRecord :returns: True if the message indicates any form of program exit, False otherwise :rtype: bool """ return is_stopped_exit(parsed_mi_msg) or is_thread_group_exit(parsed_mi_msg) def is_break_hit(parsed_mi_msg): """Check if a parsed GDB MI message indicates a breakpoint was hit. :param parsed_mi_msg: a parsed GDB MI message object :type parsed_mi_msg: gdbmi_parser.GdbMiRecord :returns: True if the message indicates a breakpoint hit, False otherwise :rtype: bool """ return ( hasattr(parsed_mi_msg, "class_") and (parsed_mi_msg.class_ == "stopped") and hasattr(parsed_mi_msg, "result") and hasattr(parsed_mi_msg.result, "reason") and (parsed_mi_msg.result.reason == "breakpoint-hit") ) def is_sigsegv(parsed_mi_msg): """Check if a parsed GDB MI message indicates a segmentation fault (SIGSEGV). :param parsed_mi_msg: a parsed GDB MI message object :type parsed_mi_msg: gdbmi_parser.GdbMiRecord :returns: True if the message indicates SIGSEGV signal, False otherwise :rtype: bool """ return ( hasattr(parsed_mi_msg, "class_") and (parsed_mi_msg.class_ == "stopped") and hasattr(parsed_mi_msg, "result") and hasattr(parsed_mi_msg.result, "signal_name") and (parsed_mi_msg.result.reason == "SIGSEGV") ) def is_sigabrt_stopped(parsed_mi_msg): """Check if a parsed GDB MI message indicates a SIGABRT signal with stopped status. :param parsed_mi_msg: a parsed GDB MI message object :type parsed_mi_msg: gdbmi_parser.GdbMiRecord :returns: True if the message indicates SIGABRT in stopped state, False otherwise :rtype: bool """ return ( hasattr(parsed_mi_msg, "class_") and (parsed_mi_msg.class_ == "stopped") and hasattr(parsed_mi_msg, "record_type") and (parsed_mi_msg.record_type == "result") and (parsed_mi_msg.result.reason == "signal-received") and (parsed_mi_msg.result.signal_name == "SIGABRT") ) def is_sigabrt_console(parsed_mi_msg): """Check if a parsed GDB MI message indicates a SIGABRT signal from console output. :param parsed_mi_msg: a parsed GDB MI message object :type parsed_mi_msg: gdbmi_parser.GdbMiRecord :returns: True if the message indicates SIGABRT from console, False otherwise :rtype: bool """ return ( hasattr(parsed_mi_msg, "record_type") and (parsed_mi_msg.record_type == "stream") and hasattr(parsed_mi_msg, "type") and (parsed_mi_msg.type == "console") and hasattr(parsed_mi_msg, "value") and parsed_mi_msg.value == "SIGABRT, Aborted.\n" ) def is_sigabrt(parsed_mi_msg): """Check if a parsed GDB MI message indicates a SIGABRT signal from any source. This function combines checks for SIGABRT from both stopped state and console output. :param parsed_mi_msg: a parsed GDB MI message object :type parsed_mi_msg: gdbmi_parser.GdbMiRecord :returns: True if the message indicates SIGABRT from any source, False otherwise :rtype: bool """ return is_sigabrt_stopped(parsed_mi_msg) or is_sigabrt_console(parsed_mi_msg) def is_fatal_signal(parsed_mi_msg): """Check if a parsed GDB MI message indicates a fatal signal (SIGSEGV or SIGABRT). This function identifies signals that typically indicate serious program errors that would cause the program to terminate abnormally. :param parsed_mi_msg: a parsed GDB MI message object :type parsed_mi_msg: gdbmi_parser.GdbMiRecord :returns: True if the message indicates a fatal signal, False otherwise :rtype: bool """ return is_sigsegv(parsed_mi_msg) or is_sigabrt(parsed_mi_msg) def format_as_hex(char): """Formats a single ascii character as a lower case hex string :param char: a single ascii character :type char: str :returns: the character formatted as a lower case hex string :rtype: str """ return f"{ord(char):2x}" def string_to_hex(text): """Formats a string of text into an hex representation :param text: a multi character string :type text: str :returns: the string converted to an hex representation :rtype: str """ return "".join(map(format_as_hex, text)) class CommandResult: """A GDB command, its result, and other possible messages""" def __init__(self, command): self.command = command self.timestamp = time.monotonic() self.stream_messages = [] self.application_output = [] self.result = None def get_application_output(self): """Return all application output concatenated as a single string :returns: application output concatenated :rtype: str """ return "".join(self.application_output) def get_stream_messages_text(self): """Return all stream messages text concatenated as a single string :returns: stream messages text concatenated :rtype: str """ return "".join([m.value for m in self.stream_messages]) def __repr__(self): return f"{self.command} at {self.timestamp:.9f}" # pylint: disable=E1101 class GDB: """Wraps a GDB subprocess for easier manipulation""" REQUIRED_ARGS = ["--interpreter=mi", "--quiet"] DEFAULT_BREAK = "main" def __init__(self, path=None, *extra_args): # pylint: disable=W1113 if path is None: path = find_command("gdb", default="/usr/bin/gdb") self.path = path args = [self.path] args += self.REQUIRED_ARGS args += extra_args try: self.process = subprocess.Popen( # pylint: disable=R1732 args, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, close_fds=True, ) except OSError as details: if details.errno == 2: exc = OSError(f"File '{args[0]}' not found") exc.errno = 2 raise exc from details raise fcntl.fcntl(self.process.stdout.fileno(), fcntl.F_SETFL, os.O_NONBLOCK) self.read_until_break() # If this instance is connected to another target. If so, what # tcp port it's connected to self.connected_to = None # any GDB MI async messages self.async_messages = [] self.commands_history = [] # whatever comes from the app that is not a GDB MI message self.output_messages = [] self.output_messages_queue = [] def read_gdb_response(self, timeout=0.01, max_tries=100): """Read raw responses from GDB :param timeout: the amount of time to way between read attempts :type timeout: float :param max_tries: the maximum number of cycles to try to read until a response is obtained :type max_tries: int :returns: a string containing a raw response from GDB :rtype: str :raises ValueError: if can't read GDB response """ current_try = 0 while current_try < max_tries: try: line = self.process.stdout.readline() line = line.strip() if line: return line except IOError: current_try += 1 if current_try >= max_tries: raise ValueError("Could not read GDB response") time.sleep(timeout) return None def read_until_break(self, max_lines=100): """Read lines from GDB until a break condition is reached :param max_lines: the maximum number of lines to read :type max_lines: int :returns: a list of messages read :rtype: list of str """ result = [] while True: line = self.read_gdb_response() if line in GDB_BREAK_CONDITIONS: break if len(result) >= max_lines: break result.append(line) return result def send_gdb_command(self, command): """Send a raw command to the GNU debugger input :param command: the GDB command, hopefully in MI language :type command: str """ if not command.endswith("\n"): command = f"{command}\n" self.process.stdin.write(command.encode()) self.process.stdin.flush() def cmd(self, command): """Sends a command and parses all lines until prompt is received :param command: the GDB command, hopefully in MI language :type command: str :returns: a :class:`CommandResult` instance :rtype: :class:`CommandResult` :raises gdbmi_parser.GdbMiError: if there are many result responses to a single cmd """ cmd = CommandResult(command) self.send_gdb_command(command) responses = self.read_until_break() result_response_received = False for line in responses: # If the line can not be properly parsed, it is *most likely* # generated by the application being run inside the debugger try: parsed_response = parse_mi(line.decode()) except Exception: # pylint: disable=W0703 cmd.application_output.append(line) continue if ( parsed_response.type == "console" and parsed_response.record_type == "stream" ): cmd.stream_messages.append(parsed_response) elif parsed_response.type == "result": if result_response_received: # raise an exception here, because no two result # responses should come from a single command AFAIK raise gdbmi_parser.GdbMiError( "Many result responses to a single cmd" ) result_response_received = True cmd.result = parsed_response else: self.async_messages.append(parsed_response) return cmd def cli_cmd(self, command): """Sends a cli command encoded as an MI command :param command: a regular GDB cli command :type command: str :returns: a :class:`CommandResult` instance :rtype: :class:`CommandResult` """ cmd = encode_mi_cli(command) return self.cmd(cmd) def cmd_exists(self, command): """Checks if a given command exists :param command: a GDB MI command, including the dash (-) prefix :type command: str :returns: either True or False :rtype: bool """ gdb_info_command = f"-info-gdb-mi-command {command[1:]}" r = self.cmd(gdb_info_command) return r.result.result.command.exists == "true" def set_file(self, path): """Sets the file that will be executed :param path: the path of the binary that will be executed :type path: str :returns: a :class:`CommandResult` instance :rtype: :class:`CommandResult` :raises UnexpectedResponseError: if response is unexpected """ cmd = f"-file-exec-and-symbols {path}" r = self.cmd(cmd) if not r.result.class_ == "done": raise UnexpectedResponseError if self.connected_to is not None: cmd = f"set remote exec-file {path}" r = self.cmd(cmd) if not r.result.class_ == "done": raise UnexpectedResponseError return r def set_break(self, location, ignore_error=False): """Sets a new breakpoint on the binary currently being debugged :param location: a breakpoint location expression as accepted by GDB :type location: str :param ignore_error: if set, won't raise exceptions :type ignore_error: bool :returns: a :class:`CommandResult` instance :rtype: :class:`CommandResult` :raises UnexpectedResponseError: if response is unexpected """ cmd = f"-break-insert {location}" r = self.cmd(cmd) if not r.result.class_ == "done": if not ignore_error: raise UnexpectedResponseError return r def del_break(self, number): """Deletes a breakpoint by its number :param number: the breakpoint number :type number: int :returns: a :class:`CommandResult` instance :rtype: :class:`CommandResult` :raises UnexpectedResponseError: if response is unexpected """ cmd = f"-break-delete {number}" r = self.cmd(cmd) if not r.result.class_ == "done": raise UnexpectedResponseError return r def run(self, args=None): """Runs the application inside the debugger :param args: the arguments to be passed to the binary as command line arguments :type args: builtin.list :returns: a :class:`CommandResult` instance :rtype: :class:`CommandResult` :raises UnexpectedResponseError: if response is unexpected """ if args: args_text = " ".join(args) cmd = f"-exec-arguments {args_text}" r = self.cmd(cmd) if not r.result.class_ == "done": raise UnexpectedResponseError r = self.cmd("-exec-run") if not r.result.class_ == "running": raise UnexpectedResponseError return r def connect(self, port): """Connects to a remote debugger (a gdbserver) at the given TCP port This uses the "extended-remote" target type only :param port: the TCP port number :type port: int :returns: a :class:`CommandResult` instance :rtype: :class:`CommandResult` :raises UnexpectedResponseError: if response is unexpected """ cmd = f"-target-select extended-remote :{port}" r = self.cmd(cmd) if not r.result.class_ == "connected": raise UnexpectedResponseError self.connected_to = port return r def disconnect(self): """Disconnects from a remote debugger :returns: a :class:`CommandResult` instance :rtype: :class:`CommandResult` :raises UnexpectedResponseError: if response is unexpected """ cmd = "-target-disconnect" r = self.cmd(cmd) if not r.result.class_ == "done": raise UnexpectedResponseError self.connected_to = None return r def exit(self): """Exits the GDB application gracefully :returns: the result of :meth:`subprocess.POpen.wait`, that is, a :attr:`subprocess.POpen.returncode` :rtype: int or None """ self.cmd("-gdb-exit") return self.process.wait() class GDBServer: """Wraps a gdbserver instance""" #: The default arguments used when starting the GDB server process REQUIRED_ARGS = ["--multi"] #: The range from which a port to GDB server will try to be allocated from PORT_RANGE = (20000, 20999) #: The time to optionally wait for the server to initialize itself and be #: ready to accept new connections INIT_TIMEOUT = 5.0 # pylint: disable=W0613, W1113 def __init__( self, path=None, port=None, wait_until_running=True, *extra_args, ): """Initializes a new gdbserver instance :param path: location of the gdbserver binary :type path: str :param port: tcp port number to listen on for incoming connections :type port: int :param wait_until_running: wait until the gdbserver is running and accepting connections. It may take a little after the process is started and it is actually bound to the allocated port :type wait_until_running: bool :param extra_args: optional extra arguments to be passed to gdbserver """ if path is None: path = find_command("gdbserver", default="/usr/bin/gdbserver") self.path = path args = [self.path] args += self.REQUIRED_ARGS if port is None: self.port = ports.find_free_port(*self.PORT_RANGE) else: self.port = port args.append(f":{self.port}") prefix = f"avocado_gdbserver_{self.port}_" _, self.stdout_path = tempfile.mkstemp(prefix=prefix + "stdout_") # pylint: disable=R1732 self.stdout = open(self.stdout_path, "w", encoding="utf-8") _, self.stderr_path = tempfile.mkstemp(prefix=prefix + "stderr_") # pylint: disable=R1732 self.stderr = open(self.stderr_path, "w", encoding="utf-8") try: # pylint: disable=R1732 self.process = subprocess.Popen( args, stdin=subprocess.PIPE, stdout=self.stdout, stderr=self.stderr, close_fds=True, ) except OSError as details: if details.errno == 2: exc = OSError(f"File '{args[0]}' not found") exc.errno = 2 raise exc from details raise if wait_until_running: self._wait_until_running() def _wait_until_running(self): connection_ok = False c = GDB() end_time = time.monotonic() + self.INIT_TIMEOUT while time.monotonic() < end_time: try: c.connect(self.port) connection_ok = True break except UnexpectedResponseError: time.sleep(0.1) c.disconnect() c.exit() if not connection_ok: raise ServerInitTimeoutError def exit(self, force=True): """Quits the gdb_server process Most correct way of quitting the GDB server is by sending it a command. If no GDB client is connected, then we can try to connect to it and send a quit command. If this is not possible, we send it a signal and wait for it to finish. :param force: if a forced exit (sending SIGTERM) should be attempted :type force: bool """ temp_client = GDB() try: temp_client.connect(self.port) temp_client.cli_cmd("monitor exit") except (UnexpectedResponseError, ValueError): if force: self.process.kill() finally: try: temp_client.disconnect() temp_client.exit() except (UnexpectedResponseError, ValueError): if force: temp_client.process.kill() temp_client.process.wait() self.process.wait() self.stdout.close() self.stderr.close() class GDBRemote: """A GDBRemote acts like a client that speaks the GDB remote protocol, documented at: https://sourceware.org/gdb/current/onlinedocs/gdb/Remote-Protocol.html Caveat: we currently do not support communicating with devices, only with TCP sockets. This limitation is basically due to the lack of use cases that justify an implementation, but not due to any technical shortcoming. """ def __init__(self, host, port, no_ack_mode=True, extended_mode=True): """Initializes a new GDBRemote object. :param host: the IP address or host name :type host: str :param port: the port number where the the remote GDB is listening on :type port: int :param no_ack_mode: if the packet transmission confirmation mode should be disabled :type no_ack_mode: bool :param extended_mode: if the remote extended mode should be enabled :type param extended_mode: bool """ self.host = host self.port = port # Temporary holder for the class init attributes self._no_ack_mode = no_ack_mode self.no_ack_mode = False self._extended_mode = extended_mode self.extended_mode = False self._socket = None @staticmethod def checksum(input_message): """Calculates a remote message checksum. More details are available at: https://sourceware.org/gdb/current/onlinedocs/gdb/Overview.html :param input_message: the message input payload, without the start and end markers :type input_message: bytes :returns: two byte checksum :rtype: bytes """ total = 0 for i in input_message: total += i result = total % 256 return b"%02x" % result @staticmethod def encode(data): """Encodes a command. That is, add prefix, suffix and checksum. More details are available at: https://sourceware.org/gdb/current/onlinedocs/gdb/Overview.html :param data: the command data payload :type data: bytes :returns: the encoded command, ready to be sent to a remote GDB :rtype: bytes """ return b"$%b#%b" % (data, GDBRemote.checksum(data)) @staticmethod def decode(data): """Decodes a packet and returns its payload. More details are available at: https://sourceware.org/gdb/current/onlinedocs/gdb/Overview.html :param data: the command data payload :type data: bytes :returns: the encoded command, ready to be sent to a remote GDB :rtype: bytes :raises InvalidPacketError: if the packet is not well constructed, like in checksum mismatches """ if data[0:1] != REMOTE_PREFIX: raise InvalidPacketError if data[-3:-2] != REMOTE_DELIMITER: raise InvalidPacketError payload = data[1:-3] checksum = data[-2:] if payload == b"": expected_checksum = b"00" else: expected_checksum = GDBRemote.checksum(payload) if checksum != expected_checksum: raise InvalidPacketError return payload def cmd(self, command_data, expected_response=None): """Sends a command data to a remote gdb server Limitations: the current version does not deal with retransmissions. :param command_data: the remote command to send the the remote stub :type command_data: str :param expected_response: the (optional) response that is expected as a response for the command sent :type expected_response: str :returns: raw data read from from the remote server :rtype: str :raises NotConnectedError: if the socket is not initialized :raises RetransmissionRequestedError: if there was a failure while reading the result of the command :raises UnexpectedResponseError: if response is unexpected """ if self._socket is None: raise NotConnectedError data = self.encode(command_data) self._socket.send(data) if not self.no_ack_mode: transmission_result = self._socket.recv(1) if transmission_result == REMOTE_TRANSMISSION_FAILURE: raise RetransmissionRequestedError result = self._socket.recv(REMOTE_MAX_PACKET_SIZE) response_payload = self.decode(result) if expected_response is not None: if expected_response != response_payload: raise UnexpectedResponseError return response_payload def set_extended_mode(self): """Enable extended mode. In extended mode, the remote server is made persistent. The 'R' packet is used to restart the program being debugged. Original documentation at: https://sourceware.org/gdb/current/onlinedocs/gdb/Packets.html#extended-mode """ self.cmd(b"!", b"OK") self.extended_mode = True def start_no_ack_mode(self): """Request that the remote stub disable the normal +/- protocol acknowledgments. Original documentation at: https://sourceware.org/gdb/current/onlinedocs/gdb/General-Query-Packets.html#QStartNoAckMode """ self.cmd(b"QStartNoAckMode", b"OK") self.no_ack_mode = True def connect(self): """Connects to the remote target and initializes the chosen modes""" self._socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM) self._socket.connect((self.host, self.port)) if self._no_ack_mode: self.start_no_ack_mode() if self._extended_mode: self.set_extended_mode() # pylint: disable=wrong-import-position from avocado.utils.deprecation import log_deprecation log_deprecation.warning("gdb") avocado-113.0/avocado/utils/genio.py000066400000000000000000000142121513561306000173610ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; specifically version 2 of the License. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # This code was inspired in the autotest project, # client/shared/utils.py # Authors: Martin J Bligh # Andy Whitcroft # # Modification History: # - Added read_line_with_matching_pattern() """ Avocado generic IO related functions. """ import logging import os import re from avocado.utils import crypto LOG = logging.getLogger(__name__) class GenIOError(Exception): """ Base Exception Class for all IO exceptions """ def ask(question, auto=False): """ Prompt the user with a (y/n) question. :param question: Question to be asked :type question: str :param auto: Whether to return "y" instead of asking the question :type auto: bool :return: User answer :rtype: str """ if auto: LOG.info("%s (y/n) y", question) return "y" return input(f"{question} (y/n) ") def read_file(filename): """ Read the entire contents of file. :param filename: Path to the file. :type filename: str :return: File contents :rtype: str """ with open(filename, "r", encoding="utf-8") as file_obj: contents = file_obj.read() return contents def read_one_line(filename): """ Read the first line of filename. :param filename: Path to the file. :type filename: str :return: First line contents :rtype: str """ with open(filename, "r", encoding="utf-8") as file_obj: line = file_obj.readline().rstrip("\n") return line def read_all_lines(filename): """ Return all lines of a given file This utility method returns an empty list in any error scenario, that is, it doesn't attempt to identify error paths and raise appropriate exceptions. It does exactly the opposite to that. This should be used when it's fine or desirable to have an empty set of lines if a file is missing or is unreadable. :param filename: Path to the file. :type filename: str :return: all lines of the file as list :rtype: builtin.list """ contents = [] try: with open(filename, "r", encoding="utf-8") as file_obj: contents = [line.rstrip("\n") for line in file_obj.readlines()] except Exception: # pylint: disable=W0703 pass return contents def read_line_with_matching_pattern(filename, pattern): """ Return the line/lines found with a given matching pattern. This method returns the line/lines wherever the occurrence of the pattern, that is passed as an input, is found in the file. All the occurrences are stored in the list and the list is returned as an output. :param filename: Path to the file to be read. :type filename: str :param pattern: List of all the patterns, intended to be searched :type pattern: list :return: All those lines from the file, matching the pattern. :rtype: list """ contents = [] with open(filename, "r", encoding="utf-8") as file_obj: for line in file_obj.readlines(): if pattern in line: contents.append(line.rstrip("\n")) return contents def write_file(filename, data): """ Write data to a file. :param filename: Path to the file. :type filename: str :param line: Line to be written. :type line: str """ with open(filename, "w", encoding="utf-8") as file_obj: file_obj.write(data) def write_one_line(filename, line): """ Write one line of text to filename. :param filename: Path to the file. :type filename: str :param line: Line to be written. :type line: str """ write_file(filename, line.rstrip("\n") + "\n") def write_file_or_fail(filename, data): """ Write to a file and raise exception on write failure :param filename: Path to file :type filename: str :param data: Data to be written to file :type data: str :raises GenIOError: On write Failure """ try: with open(filename, "w", encoding="utf-8") as file_obj: file_obj.write(data) except OSError as details: raise GenIOError(f"The write to {filename} failed: {details}") from details def append_file(filename, data): """ Append data to a file. :param filename: Path to the file. :type filename: str :param line: Line to be written. :type line: str """ with open(filename, "a+", encoding="utf-8") as file_obj: file_obj.write(data) def append_one_line(filename, line): """ Append one line of text to filename. :param filename: Path to the file. :type filename: str :param line: Line to be written. :type line: str """ append_file(filename, line.rstrip("\n") + "\n") def is_pattern_in_file(filename, pattern): """ Check if a pattern matches in a specified file. If a non regular file be informed a GenIOError will be raised. :param filename: Path to file :type filename: str :param pattern: Pattern that need to match in file :type pattern: str :return: True when pattern matches in file if not return False :rtype: boolean """ if not os.path.isfile(filename): raise GenIOError(f"invalid file {filename} " f"to match pattern {pattern}") with open(filename, "r", encoding="utf-8") as content_file: if re.search(pattern, content_file.read(), re.MULTILINE): return True return False def are_files_equal(filename, other): """ Comparison of two files line by line :param filename: path to the first file :type filename: str :param other: path to the second file :type other: str :return: equality of file :rtype: boolean """ hash_1 = crypto.hash_file(filename) hash_2 = crypto.hash_file(other) return hash_1 == hash_2 avocado-113.0/avocado/utils/git.py000066400000000000000000000162741513561306000170550ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2015 # Author: Lucas Meneghel Rodrigues """ APIs to download/update git repositories from inside python scripts. """ import logging import os from avocado.utils import astring, path, process __all__ = ["GitRepoHelper", "get_repo"] LOG = logging.getLogger(__name__) class GitRepoHelper: """ Helps to deal with git repos, mostly fetching content from a repo """ # pylint: disable=R0913, R0902 def __init__( self, uri, branch="master", lbranch=None, commit=None, destination_dir=None, base_uri=None, submodule=False, ): """ Instantiates a new GitRepoHelper :type uri: string :param uri: git repository url :type branch: string :param branch: git remote branch :type lbranch: string :param lbranch: git local branch name, if different from remote :type commit: string :param commit: specific commit to download :type destination_dir: string :param destination_dir: path of a dir where to save downloaded code :type base_uri: string :param base_uri: a closer, usually local, git repository url from where to fetch content first from :type submodule: Boolean :param submodule: to download submodules recursively """ self.uri = uri self.base_uri = base_uri self.branch = branch self.commit = commit self.submodule = submodule if destination_dir is None: uri_basename = uri.split("/")[-1] self.destination_dir = os.path.join("/tmp", uri_basename) else: self.destination_dir = destination_dir if lbranch is None: self.lbranch = branch else: self.lbranch = lbranch self.cmd = path.find_command("git") def init(self): """ Initializes a directory for receiving a verbatim copy of git repo This creates a directory if necessary, and either resets or inits the repo """ if not os.path.exists(self.destination_dir): LOG.debug( "Creating directory %s for git repo %s", self.destination_dir, self.uri ) os.makedirs(self.destination_dir) os.chdir(self.destination_dir) if os.path.exists(".git"): LOG.debug( "Resetting previously existing git repo at %s for " "receiving git repo %s", self.destination_dir, self.uri, ) self.git_cmd("reset --hard") else: LOG.debug( "Initializing new git repo at %s for receiving git repo %s", self.destination_dir, self.uri, ) self.git_cmd("init") def git_cmd(self, cmd, ignore_status=False): """ Wraps git commands. :param cmd: Command to be executed. :param ignore_status: Whether we should suppress error.CmdError exceptions if the command did return exit code !=0 (True), or not suppress them (False). """ os.chdir(self.destination_dir) return process.run( rf"{self.cmd} {astring.shell_escape(cmd)}", ignore_status=ignore_status ) def fetch(self, uri): """ Performs a git fetch from the remote repo """ LOG.info( "Fetching git [REP '%s' BRANCH '%s'] -> %s", uri, self.branch, self.destination_dir, ) self.git_cmd(f"fetch -q -f -u -t {uri} {self.branch}:{self.lbranch}") def get_top_commit(self): """ Returns the topmost commit id for the current branch. :return: Commit id. """ return self.git_cmd("log --pretty=format:%H -1").stdout.strip() def get_top_tag(self): """ Returns the topmost tag for the current branch. :return: Tag. """ try: return self.git_cmd("describe").stdout.strip() except process.CmdError: return None def checkout(self, branch=None, commit=None): """ Performs a git checkout for a given branch and start point (commit) :param branch: Remote branch name. :param commit: Specific commit hash. """ if branch is None: branch = self.branch LOG.debug("Checking out branch %s", branch) self.git_cmd(f"checkout {branch}") if commit is None: commit = self.commit if commit is not None: LOG.debug("Checking out commit %s", self.commit) self.git_cmd(f"checkout {self.commit}") else: LOG.debug("Specific commit not specified") top_commit = self.get_top_commit() top_tag = self.get_top_tag() if top_tag is None: top_tag_desc = "no tag found" else: top_tag_desc = f"tag {top_tag}" LOG.info("git commit ID is %s (%s)", top_commit, top_tag_desc) def submodule_checkout(self): """ Performs a git checkout of submodules recursively """ LOG.debug("Checking out submodules") self.git_cmd("submodule update --init --recursive") def execute(self): """ Performs all steps necessary to initialize and download a git repo. This includes the init, fetch and checkout steps in one single utility method. """ self.init() if self.base_uri is not None: self.fetch(self.base_uri) self.fetch(self.uri) self.checkout() if self.submodule: self.submodule_checkout() # pylint: disable=R0913 def get_repo( uri, branch="master", lbranch=None, commit=None, destination_dir=None, base_uri=None, submodule=False, ): """ Utility function that retrieves a given git code repository. :type uri: string :param uri: git repository url :type branch: string :param branch: git remote branch :type lbranch: string :param lbranch: git local branch name, if different from remote :type commit: string :param commit: specific commit to download :type destination_dir: string :param destination_dir: path of a dir where to save downloaded code :type base_uri: string :param base_uri: a closer, usually local, git repository url from where to fetch content first from :type submodule: Boolean :param submodule: to download submodules recursively """ repo = GitRepoHelper( uri, branch, lbranch, commit, destination_dir, base_uri, submodule ) repo.execute() return repo.destination_dir avocado-113.0/avocado/utils/iso9660.py000066400000000000000000000327601513561306000174070ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2015 # Author: Cleber Rosa """ Basic ISO9660 file-system support. This code does not attempt (so far) to implement code that knows about ISO9660 internal structure. Instead, it uses commonly available support either in userspace tools or on the Linux kernel itself (via mount). """ __all__ = [ "iso9660", "Iso9660IsoInfo", "Iso9660IsoRead", "Iso9660Mount", "ISO9660PyCDLib", ] import io import logging import os import re import shutil import string import sys import tempfile from avocado.utils import process try: import pycdlib except ImportError: pycdlib = None LOG = logging.getLogger(__name__) def has_userland_tool(executable): """ Returns whether the system has a given executable :param executable: the name of the executable :type executable: str :rtype: bool """ if os.path.isabs(executable): return os.path.isfile(executable) for path in os.environ["PATH"].split(os.pathsep): if os.path.isfile(os.path.join(path, executable)): return True return False def has_isoinfo(): """ Returns whether the system has the isoinfo executable Maybe more checks could be added to see if isoinfo supports the needed features :rtype: bool """ return has_userland_tool("isoinfo") def has_isoread(): """ Returns whether the system has the iso-read executable Maybe more checks could be added to see if iso-read supports the needed features :rtype: bool """ return has_userland_tool("iso-read") def has_pycdlib(): """ Returns whether the system has the Python "pycdlib" library :rtype: bool """ return pycdlib is not None def can_mount(): """ Test whether the current user can perform a loop mount AFAIK, this means being root, having mount and iso9660 kernel support :rtype: bool """ if not process.can_sudo(): LOG.debug( 'Can not use mount: current user is not "root" and ' "sudo is not configured." ) return False if not has_userland_tool("mount"): LOG.debug('Can not use mount: missing "mount" tool') return False with open("/proc/filesystems") as proc_filesystems: # pylint: disable=W1514 if "iso9660" not in proc_filesystems.read(): process.system("modprobe iso9660", ignore_status=True, sudo=True) with open("/proc/filesystems") as proc_filesystems: # pylint: disable=W1514 if "iso9660" not in proc_filesystems.read(): LOG.debug("Can not use mount: lack of iso9660 kernel support") return False return True class BaseIso9660: """ Represents a ISO9660 filesystem This class holds common functionality and has many abstract methods """ def __init__(self, path): self.path = path def read(self, path): """ Abstract method to read data from path :param path: path to the file :returns: data content from the file :rtype: str """ raise NotImplementedError def copy(self, src, dst): """ Simplistic version of copy that relies on read() :param src: source path :type src: str :param dst: destination path :type dst: str :rtype: None """ content = self.read(src) with open(dst, "w+b") as output: output.write(content) @property def mnt_dir(self): """ Returns a path to the browsable content of the iso """ raise NotImplementedError def close(self): """ Cleanup and free any resources being used :rtype: None """ class MixInMntDirMount: """ Mix in class which defines `mnt_dir` property and instantiates the Iso9660Mount class to provide one. It requires `self.path` to store path to the target iso file. """ _mount_instance = None path = None @property def mnt_dir(self): """ Returns a path to the browsable content of the iso """ if self._mount_instance is None: if not self.path: raise RuntimeError(f"Path to iso image not available: " f"{self.path}") self._mount_instance = Iso9660Mount(self.path) return self._mount_instance.mnt_dir def close(self): """ Cleanups and frees any resources being used """ if self._mount_instance: self._mount_instance.close() self._mount_instance = None class Iso9660IsoInfo(MixInMntDirMount, BaseIso9660): """ Represents a ISO9660 filesystem This implementation is based on the cdrkit's isoinfo tool """ def __init__(self, path): super().__init__(path) self.joliet = False self.rock_ridge = False self.el_torito = False self._get_extensions(path) def _get_extensions(self, path): """ Get and store the image's extensions """ cmd = f"isoinfo -i {path} -d" output = process.system_output(cmd) if b"\nJoliet" in output: self.joliet = True if b"\nRock Ridge signatures" in output: self.rock_ridge = True if b"\nEl Torito" in output: self.el_torito = True @staticmethod def _normalize_path(path): """ Normalize the path to match isoinfo notation """ if not os.path.isabs(path): path = os.path.join("/", path) return path def _get_filename_in_iso(self, path): """ Locate the path in the list of files inside the iso image """ cmd = f"isoinfo -i {self.path} -f" flist = process.system_output(cmd) fname = re.findall(f"({self._normalize_path(path)}.*)", flist, re.I) if fname: return fname[0] return None def read(self, path): cmd = ["isoinfo", f"-i {self.path}"] fname = self._normalize_path(path) if self.joliet: cmd.append("-J") elif self.rock_ridge: cmd.append("-R") else: fname = self._get_filename_in_iso(path) if not fname: LOG.warning("Could not find '%s' in iso '%s'", path, self.path) return "" cmd.append(f"-x {fname}") result = process.run(" ".join(cmd), verbose=False) return result.stdout class Iso9660IsoRead(MixInMntDirMount, BaseIso9660): """ Represents a ISO9660 filesystem This implementation is based on the libcdio's iso-read tool """ def __init__(self, path): super().__init__(path) self.temp_dir = tempfile.mkdtemp(prefix="avocado_" + __name__) def read(self, path): temp_path = os.path.join(self.temp_dir, path) cmd = f"iso-read -i {self.path} -e {path} -o {temp_path}" process.run(cmd) with open(temp_path, "rb") as temp_file: return bytes(temp_file.read()) def copy(self, src, dst): cmd = f"iso-read -i {self.path} -e {src} -o {dst}" process.run(cmd) def close(self): super().close() shutil.rmtree(self.temp_dir, True) class Iso9660Mount(BaseIso9660): """ Represents a mounted ISO9660 filesystem. """ def __init__(self, path): """ initializes a mounted ISO9660 filesystem :param path: path to the ISO9660 file :type path: str """ super().__init__(path) self._mnt_dir = tempfile.mkdtemp(prefix="avocado_" + __name__) if sys.platform.startswith("darwin"): fs_type = "cd9660" else: fs_type = "iso9660" process.run( f"mount -t {fs_type} -v -o loop,ro {path} {self.mnt_dir}", sudo=True ) def read(self, path): """ Read data from path :param path: path to read data :type path: str :return: data content :rtype: str """ full_path = os.path.join(self.mnt_dir, path) with open(full_path, "rb") as file_to_read: return bytes(file_to_read.read()) def copy(self, src, dst): """ :param src: source :type src: str :param dst: destination :type dst: str :rtype: None """ full_path = os.path.join(self.mnt_dir, src) shutil.copy(full_path, dst) def close(self): """ Perform umount operation on the temporary dir :rtype: None """ if self._mnt_dir: if os.path.ismount(self._mnt_dir): process.run(f"fuser -k {self.mnt_dir}", ignore_status=True, sudo=True) process.run(f"umount {self.mnt_dir}", sudo=True) shutil.rmtree(self._mnt_dir) self._mnt_dir = None @property def mnt_dir(self): if not self._mnt_dir: raise RuntimeError( f"Trying to get mnt_dir of already closed " f"iso {self.path}" ) return self._mnt_dir class ISO9660PyCDLib(MixInMntDirMount, BaseIso9660): """ Represents a ISO9660 filesystem This implementation is based on the pycdlib library """ #: Default flags used when creating a new ISO image DEFAULT_CREATE_FLAGS = {"interchange_level": 3, "joliet": 3} def __init__(self, path): if not has_pycdlib(): raise RuntimeError("This class requires the pycdlib library") super().__init__(path) self._iso = None self._iso_opened_for_create = False def _open_for_read(self): if self._iso is None: self._iso = pycdlib.PyCdlib() self._iso.open(self.path) def create(self, flags=None): """ Creates a new ISO image :param flags: the flags used when creating a new image :type flags: dict """ if self._iso is None: self._iso = pycdlib.PyCdlib() if flags is None: flags = self.DEFAULT_CREATE_FLAGS self._iso.new(**flags) self._iso_opened_for_create = True @staticmethod def _get_iso_path(path): iso_path = "".join( [c for c in path if c in (string.ascii_letters + string.digits)] ) iso_path = iso_path[:7].upper() + ";" if not os.path.isabs(iso_path): iso_path = "/" + iso_path[:6] + ";" return iso_path @staticmethod def _get_abs_path(path): if not os.path.isabs(path): path = "/" + path return path def write(self, path, content): """ Writes a new file into the ISO image :param path: the path of the new file inside the ISO image :type path: str :param content: the content of the new file :type path: bytes """ self.create() self._iso.add_fp( io.BytesIO(content), len(content), iso_path=self._get_iso_path(path), joliet_path=self._get_abs_path(path), ) def read(self, path): self._open_for_read() if not os.path.isabs(path): path = "/" + path data = io.BytesIO() self._iso.get_file_from_iso_fp(data, joliet_path=path) return data.getvalue() def copy(self, src, dst): self._open_for_read() if not os.path.isabs(src): src = "/" + src self._iso.get_file_from_iso(dst, joliet_path=src) def close(self): super().close() if self._iso: if self._iso_opened_for_create: self._iso.write(self.path) self._iso.close() self._iso = None def iso9660(path, capabilities=None): """ Checks the available tools on a system and chooses class accordingly This is a convenience function, that will pick the first available iso9660 capable tool. :param path: path to an iso9660 image file :type path: str :param capabilities: list of specific capabilities that are required for the selected implementation, such as "read", "copy" and "mnt_dir". :type capabilities: list :return: an instance of any iso9660 capable tool :rtype: :class:`Iso9660IsoInfo`, :class:`Iso9660IsoRead`, :class:`Iso9660Mount`, :class:`ISO9660PyCDLib` or None """ # all implementations so far have these base capabilities common_capabilities = ["read", "copy", "mnt_dir"] implementations = [ ( "pycdlib", has_pycdlib, ISO9660PyCDLib, common_capabilities + ["create", "write"], ), ("isoinfo", has_isoinfo, Iso9660IsoInfo, common_capabilities), ("iso-read", has_isoread, Iso9660IsoRead, common_capabilities), ("mount", can_mount, Iso9660Mount, common_capabilities), ] for name, check, klass, cap in implementations: if capabilities is not None and not set(capabilities).issubset(cap): continue if check(): LOG.debug("Automatically chosen class for iso9660: %s", name) return klass(path) return None avocado-113.0/avocado/utils/kernel.py000066400000000000000000000165061513561306000175500ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2014 # Author: Ruda Moura # Author: Santhosh G """ Provides utilities for the Linux kernel. """ import logging import multiprocessing import os import re import shutil import tempfile from avocado.utils import archive, asset, build, distro, process LOG = logging.getLogger(__name__) class KernelBuildError(Exception): """ Exception raised when there is an error building the kernel. """ class KernelBuild: """ Build the Linux Kernel from official tarballs. """ URL = "https://www.kernel.org/pub/linux/kernel/v{major}.x/" SOURCE = "linux-{version}.tar.gz" def __init__(self, version, config_path=None, work_dir=None, data_dirs=None): """ Creates an instance of :class:`KernelBuild`. :param version: kernel version ("3.19.8"). :param config_path: path to config file. :param work_dir: work directory. :param data_dirs: list of directories to keep the downloaded kernel :return: None. """ self.asset_path = None self.version = version self.config_path = config_path self.distro = distro.detect() if work_dir is None: work_dir = tempfile.mkdtemp(prefix="avocado_" + __name__) self.work_dir = work_dir if data_dirs is not None: self.data_dirs = data_dirs else: self.data_dirs = [self.work_dir] self._build_dir = os.path.join(self.work_dir, f"linux-{self.version}") def __repr__(self): return f"KernelBuild('{self.version}, {self.config_path}, " f"{self.work_dir}')" @property def vmlinux(self): """ Return the vmlinux path if the file exists """ if not self.build_dir: return None vmlinux_path = os.path.join(self.build_dir, "vmlinux") if os.path.isfile(vmlinux_path): return vmlinux_path return None @property def build_dir(self): """ Return the build path if the directory exists """ if os.path.isdir(self._build_dir): return self._build_dir return None def _build_kernel_url(self, base_url=None): kernel_file = self.SOURCE.format(version=self.version) if base_url is None: base_url = self.URL.format(major=self.version.split(".", 1)[0]) return base_url + kernel_file def download(self, url=None): """ Download kernel source. :param url: override the url from where to fetch the kernel source tarball :type url: str or None """ full_url = self._build_kernel_url(base_url=url) self.asset_path = asset.Asset( full_url, asset_hash=None, algorithm=None, locations=None, cache_dirs=self.data_dirs, ).fetch() def uncompress(self): """ Uncompress kernel source. :raises: Exception in case the tarball is not downloaded """ if self.asset_path: LOG.info("Uncompressing tarball") archive.extract(self.asset_path, self.work_dir) else: raise KernelBuildError("Unable to find the tarball") def configure(self, targets=("defconfig"), extra_configs=None): """ Configure/prepare kernel source to build. :param targets: configuration targets. Default is 'defconfig'. :type targets: list of str :param extra_configs: additional configurations in the form of CONFIG_NAME=VALUE. :type extra_configs: list of str """ build.make(self._build_dir, extra_args=f"-C {self._build_dir} mrproper") if self.config_path is not None: dotconfig = os.path.join(self._build_dir, ".config") shutil.copy(self.config_path, dotconfig) build.make(self._build_dir, extra_args=f"-C {self._build_dir} olddefconfig") else: if isinstance(targets, list): _targets = " ".join(targets) else: _targets = targets build.make(self.build_dir, extra_args=f"-C {self.build_dir} {_targets}") if extra_configs: with tempfile.NamedTemporaryFile( mode="w+t", prefix="avocado_" ) as config_file: config_file.write("\n".join(extra_configs)) config_file.flush() cmd = [ "cd", self._build_dir, "&&", "./scripts/kconfig/merge_config.sh", ".config", config_file.name, ] process.run(" ".join(cmd), shell=True) def build(self, binary_package=False, njobs=multiprocessing.cpu_count()): """ Build kernel from source. :param binary_package: when True, the appropriate platform package is built for install() to use :type binary_pacakge: bool :param njobs: number of jobs. It is mapped to the -j option from make. If njobs is None then do not limit the number of jobs (e.g. uses -j without value). The -j is omitted if a value equal or less than zero is passed. Default value is set to `multiprocessing.cpu_count()`. :type njobs: int or None """ make_args = [] LOG.info("Starting build the kernel") if njobs is not None and njobs > 0: make_args.append("-j" + str(njobs)) make_args.extend(["-C", self._build_dir]) if binary_package and self.distro.name == "Ubuntu": make_args.append("deb-pkg") build.make(self._build_dir, extra_args=" ".join(make_args)) def install(self): """ Install built kernel. """ LOG.info("Starting kernel install") if self.distro.name == "Ubuntu": process.run(f"dpkg -i {self.work_dir}/*.deb", shell=True, sudo=True) else: LOG.info("Skipping kernel install") def __del__(self): shutil.rmtree(self.work_dir) def _parse_kernel_version(version): match = re.match(r"(\d+)\.(\d+)\.(\d+)-(\d+).*", version) if match: return tuple(map(int, match.groups())) raise AssertionError(f'Malformed kernel version "{version}"') def check_version(version): """ This utility function compares the current kernel version with the version parameter and gives assertion error if the version parameter is greater. :type version: string :param version: version to be compared with current kernel version """ os_version = _parse_kernel_version(os.uname()[2]) version = _parse_kernel_version(version) assert os_version > version, "Old kernel" avocado-113.0/avocado/utils/linux.py000066400000000000000000000061451513561306000174250ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # This code was inspired in the autotest project, # # client/base_utils.py # # Copyright: IBM, 2019 # Red Hat Inc. 2019 # Authors : Praveen K Pandey # Cleber Rosa """ Linux OS utilities """ import os from avocado.utils import genio, process class UnsupportedMachineError(Exception): """ Exception class for unsupported hardware """ def get_proc_sys(key): """ Read values from /proc/sys :param key: A location under /proc/sys :return: The single-line sysctl value as a string. """ path = os.path.join("/proc/sys/%s", key) return genio.read_one_line(path) def set_proc_sys(key, value): """ Set values on /proc/sys :param key: A location under /proc/sys :param value: If not None, a value to write into the sysctl. :return: The single-line sysctl value as a string. """ path = os.path.join("/proc/sys/%s", key) genio.write_one_line(path, value) return get_proc_sys(key) def is_selinux_enforcing(): """ Returns True if SELinux is in enforcing mode, False if permissive/disabled. """ if "1" in genio.read_one_line("/sys/fs/selinux/enforce"): return True return False def enable_selinux_enforcing(): """ Enable SELinux Enforcing in system :return: True if SELinux enable in enforcing mode, False if not enabled """ genio.write_one_line("/sys/fs/selinux/enforce", "1") if is_selinux_enforcing(): return True return False def is_os_secureboot_enabled(): """ Check whether the secure-boot is enabled at os level. Check for "00000002" in "/proc/device-tree/ibm,secure-boot" file If found, then secure-boot is enabled. :return: True if secureboot is enabled, False if otherwise """ try: cmd = "lsprop /proc/device-tree/ibm,secure-boot" for line in process.system_output(cmd).decode("utf-8").splitlines(): if "00000002" in line: return True except FileNotFoundError as exc: raise UnsupportedMachineError("lsprop not a supported command") from exc return False def is_sched_schedstats_enabled(): """ Returns True if sched_schedstats is in enabled mode, False if no. """ if "1" in genio.read_one_line("/proc/sys/kernel/sched_schedstats"): return True return False def enable_sched_schedstats(): """ Enable SELinux Enforcing in system :return: True if sched_schedstats enabled , False if not enabled """ genio.write_one_line("/proc/sys/kernel/sched_schedstats", "1") if is_sched_schedstats_enabled(): return True return False avocado-113.0/avocado/utils/linux_modules.py000066400000000000000000000214361513561306000211550ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # This code was inspired in the autotest project, # # client/base_utils.py # Original author: Ross Brattain # # Copyright: 2016 IBM # Authors : Praveen K Pandey """ Linux kernel modules APIs """ import gzip import logging import os import platform import re from enum import Enum from pathlib import Path from avocado.utils import astring, data_structures, process LOG = logging.getLogger(__name__) class ModuleConfig(Enum): #: Config commented out or not set NOT_SET = object() #: Config compiled as loadable module (`=m`) MODULE = object() #: Config built-in to kernel (`=y`) BUILTIN = object() def load_module(module_name): """ Checks if a module has already been loaded. :param module_name: Name of module to check :return: True if module is loaded, False otherwise :rtype: Bool """ if module_is_loaded(module_name): return True if process.system("/sbin/modprobe " + module_name, ignore_status=True): return False return True def parse_lsmod_for_module(l_raw, module_name, escape=True): """ Use a regex to parse raw lsmod output and get module information :param l_raw: raw output of lsmod :type l_raw: str :param module_name: Name of module to search for :type module_name: str :param escape: Escape regex tokens in module_name, default True :type escape: bool :return: Dictionary of module info, name, size, submodules if present :rtype: dict """ # re.escape the module name for safety if escape: module_search = re.escape(module_name) else: module_search = module_name # ^module_name spaces size spaces used optional spaces optional submodules # use multiline regex to scan the entire output as one string without # having to splitlines use named matches so we can extract the dictionary # with groupdict pattern = r"^(?P%s)\s+(?P\d+)\s+(?P\d+)\s*(?P\S+)?$" lsmod = re.search(pattern % module_search, l_raw, re.M) if lsmod: # default to empty list if no submodules module_info = lsmod.groupdict([]) # convert size to integer because it is an integer module_info["size"] = int(module_info["size"]) module_info["used"] = int(module_info["used"]) if module_info["submodules"]: module_info["submodules"] = module_info["submodules"].split(",") return module_info # return empty dict to be consistent return {} def loaded_module_info(module_name): """ Get loaded module details: Size and Submodules. :param module_name: Name of module to search for :type module_name: str :return: Dictionary of module name, size, submodules if present, filename, version, number of modules using it, list of modules it is dependent on, list of dictionary of param name and type :rtype: dict """ l_raw = process.system_output("/sbin/lsmod").decode("utf-8") modinfo_dic = parse_lsmod_for_module(l_raw, module_name) output = process.system_output(f"/sbin/modinfo {module_name}").decode("utf-8") if output: param_list = [] for line in output.splitlines(): items = line.split() if not items: continue key = items[0].rstrip(":") value = None if len(items) > 1: if key in ("filename", "version"): value = str(items[-1]) elif key == "depends": value = items[1].split(",") elif key == "parm": param_dic = {"type": None} param_dic["name"] = items[1].split(":")[0] param_type = re.search(r"\((\w+)\)", items[-1]) if param_type is not None: param_dic["type"] = param_type.group(1) param_list.append(param_dic) if value: modinfo_dic[key] = value if param_list: modinfo_dic["params"] = param_list return modinfo_dic def get_submodules(module_name): """ Get all submodules of the module. :param module_name: Name of module to search for :type module_name: str :return: List of the submodules :rtype: builtin.list """ module_info = loaded_module_info(module_name) module_list = [] try: submodules = module_info["submodules"] except KeyError: LOG.info("Module %s is not loaded", module_name) else: module_list = submodules for module in submodules: module_list += get_submodules(module) return data_structures.ordered_list_unique(module_list) def unload_module(module_name): """ Removes a module. Handles dependencies. If even then it's not possible to remove one of the modules, it will throw an error.CmdError exception. :param module_name: Name of the module we want to remove. :type module_name: str """ module_info = loaded_module_info(module_name) try: submodules = module_info["submodules"] except KeyError: LOG.info("Module %s is already unloaded", module_name) else: for module in submodules: unload_module(module) module_info = loaded_module_info(module_name) try: module_used = module_info["used"] except KeyError: LOG.info("Module %s is already unloaded", module_name) return if module_used: raise RuntimeError( f"Module {module_name} is still in use. " f"Can not unload it." ) process.system(f"/sbin/modprobe -r {module_name}") LOG.info("Module %s unloaded", module_name) def module_is_loaded(module_name): """ Is module loaded :param module_name: Name of module to search for :type module_name: str :return: True if module is loaded :rtype: bool """ module_name = module_name.replace("-", "_") return module_name in get_loaded_modules() def get_loaded_modules(): """ Gets list of loaded modules. :return: List of loaded modules. """ modules_path = "/proc/modules" if not os.path.exists(modules_path): LOG.info("Modules file can not be found at %s", modules_path) return [] with open(modules_path, "rb") as proc_modules: # pylint: disable=W1514 return [astring.to_text(_.split(b" ", 1)[0]) for _ in proc_modules] def check_kernel_config(config_name): """ Reports the configuration of $config_name of the current kernel :param config_name: Name of kernel config to search :type config_name: str :return: Config status in running kernel (NOT_SET, BUILTIN, MODULE) :rtype: :class:`ModuleConfig` """ def parse_kernel_config(kernel_config): for line in kernel_config: line = line.split("=") if len(line) != 2: continue config = line[0].strip() if config == config_name: option = line[1].strip() if option == "m": return ModuleConfig.MODULE return ModuleConfig.BUILTIN return ModuleConfig.NOT_SET kernel_version = platform.uname()[2] # NOTE: If other locations for config files are known, they should be added here config_locations = [ Path("/proc/config.gz"), Path(f"/boot/config-{kernel_version}"), Path(f"/lib/modules/{kernel_version}/build/.config"), Path(f"/usr/lib/modules/{kernel_version}/config"), ] config_file = None for loc in config_locations: if loc.exists(): config_file = loc break if not config_file: raise FileNotFoundError( f"No kernel configuration file found for version {kernel_version}" ) if config_file.name.endswith(".gz"): with gzip.open(config_file, "rt") as kernel_config: return parse_kernel_config(kernel_config) else: with open(config_file, "r") as kernel_config: # pylint: disable=W1514 return parse_kernel_config(kernel_config) def get_modules_dir(): """ Return the modules dir for the running kernel version :return: path of module directory :rtype: String """ kernel_version = platform.uname()[2] return f"/lib/modules/{kernel_version}/kernel" avocado-113.0/avocado/utils/lv_utils.py000066400000000000000000000621301513561306000201230ustar00rootroot00000000000000# Copyright (C) IBM 2016 - Harish # Copyright (C) Red Hat 2016 - Lukas Doktor # Copyright (C) Intra2net AG 2018 - Plamen Dimitrov # # This program is free software; you can redistribute it and/or modify it # under the terms of the GNU General Public License as published by the # Free Software Foundation; version 2. # # This program is distributed in the hope that it will be useful, but # WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY # or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License # for more details. # You should have received a copy of the GNU General Public License along # with this program; if not, write to the Free Software Foundation, Inc., # 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. # # Source: https://github.com/autotest/autotest/blob/master/client/lv_utils.py import logging import os import re import shutil import time import warnings from avocado.utils import process LOGGER = logging.getLogger(__name__) class LVException(Exception): """ Base Exception Class for all exceptions """ def get_diskspace(disk): """ Get the entire disk space of a given disk. :param str disk: name of the disk to find the free space of :returns: size in bytes :rtype: str :raises: :py:class:`LVException` on failure to find disk space """ warnings.warn("deprecated, use get_device_total_space instead", DeprecationWarning) result = process.run(f"fdisk -l {disk}", env={"LANG": "C"}, sudo=True).stdout_text results = result.splitlines() for line in results: if line.startswith("Disk " + disk): return re.findall(r", (.*?) bytes", line)[0] raise LVException("Error in finding disk space") def get_device_total_space(disk): """Get the total device size. :param str device: name of the device/disk to find the total size :returns: size in bytes :rtype: int :raises: :py:class:`LVException` on failure to find disk space """ result = process.run(f"fdisk -l {disk}", env={"LANG": "C"}, sudo=True).stdout_text results = result.splitlines() for line in results: if line.startswith("Disk " + disk): return int(re.findall(r", (.*?) bytes", line)[0]) raise LVException("Error in finding disk space") def get_devices_total_space(devices): """Get the total size of given device(s)/disk(s). :param list devices: list with the names of devices separated with space. :returns: sizes in bytes :rtype: int :raises: :py:class:`LVException` on failure to find disk space """ size = 0 for device in devices: size = size + get_device_total_space(device) if not size: raise LVException("failed to get disks size") return size # pylint: disable=R0913 def vg_ramdisk( disk, vg_name, ramdisk_vg_size, ramdisk_basedir, ramdisk_sparse_filename, use_tmpfs=True, ): """ Create volume group on top of ram memory to speed up LV performance. When disk is specified the size of the physical volume is taken from existing disk space. :param str disk: name of the disk in which volume groups are created :param str vg_name: name of the volume group :param str ramdisk_vg_size: size of the ramdisk virtual group (MB) :param str ramdisk_basedir: base directory for the ramdisk sparse file :param str ramdisk_sparse_filename: name of the ramdisk sparse file :param bool use_tmpfs: whether to use RAM or slower storage :returns: ramdisk_filename, vg_ramdisk_dir, vg_name, loop_device :rtype: (str, str, str, str) :raises: :py:class:`LVException` on failure at any stage Sample ramdisk params: - ramdisk_vg_size = "40000" - ramdisk_basedir = "/tmp" - ramdisk_sparse_filename = "virtual_hdd" Sample general params: - vg_name='autotest_vg', - lv_name='autotest_lv', - lv_size='1G', - lv_snapshot_name='autotest_sn', - lv_snapshot_size='1G' The ramdisk volume group size is in MB. """ warnings.warn( "deprecated, use existing methods: vg_create, lv_create", DeprecationWarning ) vg_size = ramdisk_vg_size vg_ramdisk_dir = os.path.join(ramdisk_basedir, vg_name) ramdisk_filename = os.path.join(vg_ramdisk_dir, ramdisk_sparse_filename) # Try to cleanup the ramdisk before defining it try: vg_ramdisk_cleanup(ramdisk_filename, vg_ramdisk_dir, vg_name, use_tmpfs) except LVException: pass if not os.path.exists(vg_ramdisk_dir): os.makedirs(vg_ramdisk_dir) try: if use_tmpfs: LOGGER.debug("Mounting tmpfs") process.run(f"mount -t tmpfs tmpfs {vg_ramdisk_dir}", sudo=True) LOGGER.debug("Converting and copying /dev/zero") if disk: vg_size = get_diskspace(disk) # Initializing sparse file with extra few bytes cmd = f"dd if=/dev/zero of={ramdisk_filename} bs=1M count=1 " f"seek={vg_size}" process.run(cmd) if not disk: LOGGER.debug("Finding free loop device") result = process.run("losetup --find", sudo=True) except process.CmdError as ex: LOGGER.error(ex) vg_ramdisk_cleanup(ramdisk_filename, vg_ramdisk_dir, vg_name, use_tmpfs) raise LVException(f"Fail to create vg_ramdisk: {ex}") from ex if not disk: loop_device = result.stdout_text.rstrip() else: loop_device = disk try: if not disk: LOGGER.debug("Creating loop device") process.run(f"losetup {loop_device} {ramdisk_filename}", sudo=True) LOGGER.debug("Creating physical volume %s", loop_device) process.run(f"pvcreate -y {loop_device}", sudo=True) LOGGER.debug("Creating volume group %s", vg_name) process.run(f"vgcreate {vg_name} {loop_device}", sudo=True) except process.CmdError as ex: LOGGER.error(ex) vg_ramdisk_cleanup( ramdisk_filename, vg_ramdisk_dir, vg_name, loop_device, use_tmpfs ) raise LVException(f"Fail to create vg_ramdisk: {ex}") from ex return ramdisk_filename, vg_ramdisk_dir, vg_name, loop_device def vg_ramdisk_cleanup( ramdisk_filename=None, vg_ramdisk_dir=None, vg_name=None, loop_device=None, use_tmpfs=True, ): """ Clean up any stage of the VG ramdisk setup in case of test error. This detects whether the components were initialized and if so tries to remove them. In case of failure it raises summary exception. :param str ramdisk_filename: name of the ramdisk sparse file :param str vg_ramdisk_dir: location of the ramdisk file :param str vg_name: name of the volume group :param str loop_device: name of the disk or loop device :param bool use_tmpfs: whether to use RAM or slower storage :returns: ramdisk_filename, vg_ramdisk_dir, vg_name, loop_device :rtype: (str, str, str, str) :raises: :py:class:`LVException` on intolerable failure at any stage """ def pvremove(loop_device): result = process.run(f"pvremove {loop_device}", ignore_status=True, sudo=True) if result.exit_status: errs.append("wipe pv") LOGGER.error("Failed to wipe pv from %s: %s", loop_device, result) losetup_all = process.run("losetup --all", sudo=True).stdout_text if loop_device in losetup_all: ramdisk_filename = re.search( r"%s: \[\d+\]:\d+ \(([/\w]+)\)" % loop_device, # pylint: disable=C0209 losetup_all, ) if ramdisk_filename is not None: ramdisk_filename = ramdisk_filename.group(1) for _ in range(10): result = process.run( f"losetup -d {loop_device}", ignore_status=True, sudo=True ) if b"resource busy" not in result.stderr: if result.exit_status: errs.append("remove loop device") LOGGER.error( "Unexpected failure when removing loop" "device %s, check the log", loop_device, ) break time.sleep(0.1) def unmount(): if use_tmpfs and not process.system( f"mountpoint {vg_ramdisk_dir}", ignore_status=True ): for _ in range(10): result = process.run( f"umount {vg_ramdisk_dir}", ignore_status=True, sudo=True ) time.sleep(0.1) if not result.exit_status: break else: errs.append("umount") LOGGER.error( 'Unexpected failure unmounting %s, check the "log"', vg_ramdisk_dir ) if os.path.exists(vg_ramdisk_dir): try: shutil.rmtree(vg_ramdisk_dir) LOGGER.debug("Ramdisk directory %s deleted", vg_ramdisk_dir) except OSError as details: errs.append("rm-ramdisk-dir") LOGGER.error("Failed to remove ramdisk_dir: %s", details) warnings.warn( "deprecated, use existing methods: vg_remove, lv_remove", DeprecationWarning ) errs = [] if vg_name is not None: loop_device = re.search( rf"([/\w-]+) +{vg_name} +lvm2", process.run("pvs", sudo=True).stdout_text ) if loop_device is not None: loop_device = loop_device.group(1) process.run(f"vgremove -f {vg_name}", ignore_status=True, sudo=True) if loop_device is not None: pvremove(loop_device) if ramdisk_filename is not None: if os.path.exists(ramdisk_filename): os.unlink(ramdisk_filename) LOGGER.debug("Ramdisk filename %s deleted", ramdisk_filename) vg_ramdisk_dir = os.path.dirname(ramdisk_filename) if vg_ramdisk_dir is not None: unmount() if errs: raise LVException(f"vg_ramdisk_cleanup failed: {', '.join(errs)}") def vg_check(vg_name): """ Check whether provided volume group exists. :param str vg_name: name of the volume group :returns: whether the volume group was found :rtype: bool """ cmd = f"vgdisplay {vg_name}" try: process.run(cmd, sudo=True) LOGGER.debug("Provided volume group exists: %s", vg_name) return True except process.CmdError as exception: LOGGER.error(exception) return False def vg_list(vg_name=None): """ List all info about available volume groups. :param vg_name: name of the volume group to list or or None to list all :type vg_name: str or None :returns: list of available volume groups :rtype: {str, {str, str}} """ cmd = "vgs --all" cmd += f" {vg_name}" if vg_name is not None else "" vgroups = {} result = process.run(cmd, sudo=True) lines = result.stdout_text.strip().splitlines() if len(lines) > 1: columns = lines[0].split() lines = lines[1:] else: return vgroups for line in lines: details = line.split() details_dict = {} index = 0 for column in columns: if "VG" in column: vg_name = details[index] else: details_dict[column] = details[index] index += 1 vgroups[vg_name] = details_dict return vgroups def vg_create(vg_name, pv_list, force=False): """ Create a volume group from a list of physical volumes. :param str vg_name: name of the volume group :param pv_list: list of physical volumes to use :type pv_list: str or [str] :param bool force: create volume group with a force flag :raises: :py:class:`LVException` if volume group already exists """ if vg_check(vg_name): raise LVException(f"Volume group '{vg_name}' already exist") if isinstance(pv_list, list): pv_list = " ".join(pv_list) else: pv_list = str(pv_list) cmd = f"vgcreate {vg_name} {pv_list}" if force: cmd = f"{cmd} -f -y" process.run(cmd, sudo=True) def vg_remove(vg_name): """ Remove a volume group. :param str vg_name: name of the volume group :raises: :py:class:`LVException` if volume group cannot be found """ if not vg_check(vg_name): raise LVException(f"Volume group '{vg_name}' could not be found") cmd = f"vgremove -f {vg_name}" process.run(cmd, sudo=True) def lv_check(vg_name, lv_name): """ Check whether provided logical volume exists. :param str vg_name: name of the volume group :param str lv_name: name of the logical volume :return: whether the logical volume was found :rtype: bool """ cmd = f"lvdisplay {vg_name}" result = process.run(cmd, ignore_status=True, sudo=True) lvpattern = rf"LV Name\s+{lv_name}\s+" match = re.search(lvpattern, result.stdout_text.rstrip()) if match: LOGGER.debug("Provided Logical volume %s exists in %s", lv_name, vg_name) return True return False def lv_list(vg_name=None): """ List all info about available logical volumes. :param str vg_name: name of the volume group or None to list all :returns: list of available logical volumes :rtype: {str, {str, str}} """ cmd = "lvs --all" cmd += f" {vg_name}" if vg_name is not None else "" volumes = {} result = process.run(cmd, sudo=True) lines = result.stdout_text.strip().splitlines() if len(lines) > 1: lines = lines[1:] else: return volumes for line in lines: details = line.split() length = len(details) details_dict = {} lv_name = details[0] details_dict["VG"] = details[1] details_dict["Attr"] = details[2] details_dict["LSize"] = details[3] if length == 5: details_dict["Origin_Data"] = details[4] elif length > 5: details_dict["Origin_Data"] = details[5] details_dict["Pool"] = details[4] volumes[lv_name] = details_dict return volumes # pylint: disable=R0913 def lv_create( vg_name, lv_name, lv_size, force_flag=True, pool_name=None, pool_size="1G" ): """ Create a (possibly thin) logical volume in a volume group. The volume group must already exist. A thin pool will be created if pool parameters are provided and the thin pool doesn't already exist. The volume group must already exist. :param str vg_name: name of the volume group :param str lv_name: name of the logical volume :param str lv_size: size for the logical volume to be created :param bool force_flag: whether to abort if volume already exists or remove and recreate it :param str pool_name: name of thin pool or None for a regular volume :param str pool_size: size of thin pool if it will be created :raises: :py:class:`LVException` if preconditions or execution fails """ if not vg_check(vg_name): raise LVException("Volume group could not be found") if lv_check(vg_name, lv_name) and not force_flag: raise LVException("Logical volume already exists") if lv_check(vg_name, lv_name) and force_flag: lv_remove(vg_name, lv_name) lv_cmd = f"lvcreate --name {lv_name}" if pool_name is not None: if not lv_check(vg_name, pool_name): tp_cmd = ( f"lvcreate --thinpool {pool_name} " f"--size {pool_size} {vg_name} -y" ) try: process.run(tp_cmd, sudo=True) except process.CmdError as detail: LOGGER.debug(detail) raise LVException("Create thin volume pool failed.") from detail LOGGER.debug("Created thin volume pool: %s", pool_name) lv_cmd += f" --virtualsize {lv_size}" lv_cmd += f" --thin {vg_name}/{pool_name} -y" else: lv_cmd += f" --size {lv_size}" lv_cmd += f" {vg_name} -y" try: process.run(lv_cmd, sudo=True) except process.CmdError as detail: LOGGER.error(detail) raise LVException("Create thin volume failed.") from detail LOGGER.debug("Created thin volume:%s", lv_name) def lv_remove(vg_name, lv_name): """ Remove a logical volume. :param str vg_name: name of the volume group :param str lv_name: name of the logical volume :raises: :py:class:`LVException` if volume group or logical volume cannot be found """ if not vg_check(vg_name): raise LVException("Volume group could not be found") if not lv_check(vg_name, lv_name): raise LVException("Logical volume could not be found") cmd = f"lvremove -f /dev/{vg_name}/{lv_name}" process.run(cmd, sudo=True) def lv_take_snapshot( vg_name, lv_name, lv_snapshot_name, lv_snapshot_size=None, pool_name=None ): """ Take a (possibly thin) snapshot of a regular (or thin) logical volume. :param str vg_name: name of the volume group :param str lv_name: name of the logical volume :param str lv_snapshot_name: name of the snapshot be to created :param str lv_snapshot_size: size of the snapshot or None for thin snapshot of an already thin volume :param pool_name: name of thin pool or None for regular snapshot or snapshot in the same thin pool like the volume :raises: :py:class:`process.CmdError` on failure to create snapshot :raises: :py:class:`LVException` if preconditions fail """ if not vg_check(vg_name): raise LVException("Volume group could not be found") if pool_name is not None and not lv_check(vg_name, pool_name): raise LVException("Snapshot's thin pool could not be found") if lv_check(vg_name, lv_snapshot_name): raise LVException("Snapshot already exists") if not lv_check(vg_name, lv_name): raise LVException("Snapshot's origin could not be found") # thin snapshot extensions (from thin or external volume) cmd = ( f"lvcreate --snapshot --name {lv_snapshot_name} " f"/dev/{vg_name}/{lv_name} --ignoreactivationskip" ) if lv_snapshot_size is not None: cmd += f" --size {lv_snapshot_size}" if pool_name is not None: cmd += f" --thinpool {vg_name}/{pool_name}" try: process.run(cmd, sudo=True) except process.CmdError as ex: lv = ( f'Logical volume "{lv_snapshot_name}" already exists ' f'in volume group "{vg_name}"' ) if lv in ex.result.stderr_text: active = ( lv_snapshot_name + " [active]" in process.run("lvdisplay", sudo=True).stdout_text ) if active: # the above conditions detect if merge of snapshot was postponed log_msg = ( "Logical volume %s is still active! Attempting to deactivate..." ) LOGGER.debug(log_msg, lv_name) lv_reactivate(vg_name, lv_name) process.run(cmd, sudo=True) else: raise ex def lv_revert(vg_name, lv_name, lv_snapshot_name): """ Revert the origin logical volume to a snapshot. :param str vg_name: name of the volume group :param str lv_name: name of the logical volume :param str lv_snapshot_name: name of the snapshot to be reverted :raises: :py:class:`process.CmdError` on failure to revert snapshot :raises: :py:class:`LVException` if preconditions or execution fails """ try: if not vg_check(vg_name): raise LVException("Volume group could not be found") if not lv_check(vg_name, lv_snapshot_name): raise LVException("Snapshot could not be found") if not lv_check(vg_name, lv_snapshot_name) and not lv_check(vg_name, lv_name): raise LVException("Snapshot and its origin could not be found") if lv_check(vg_name, lv_snapshot_name) and not lv_check(vg_name, lv_name): raise LVException("Snapshot origin could not be found") cmd = f"lvconvert --merge --interval 1 " f"/dev/{vg_name}/{lv_snapshot_name}" result = process.run(cmd, sudo=True) if ( f"Merging of snapshot {lv_snapshot_name} will start " f"next activation." ) in result.stdout_text: raise LVException(f"The Logical volume {lv_name} is still active") except process.CmdError as ex: # detect if merge of snapshot was postponed # and attempt to reactivate the volume. active_lv_pattern = re.escape(f"{lv_snapshot_name} [active]") lvdisplay_output = process.run("lvdisplay", sudo=True).stdout_text if ( "Snapshot could not be found" in ex.result.stderr_text and re.search(active_lv_pattern, lvdisplay_output) or f"The Logical volume {lv_name} is still active" in ex.result.stderr_text ): log_msg = "Logical volume %s is still active! Attempting to deactivate..." LOGGER.debug(log_msg, lv_name) lv_reactivate(vg_name, lv_name) LOGGER.error("Continuing after reactivation") elif "Snapshot could not be found" in ex.result.stderr_text: LOGGER.error("Could not revert to snapshot:") LOGGER.error(ex.result) else: raise ex def lv_revert_with_snapshot(vg_name, lv_name, lv_snapshot_name, lv_snapshot_size): """ Perform logical volume merge with snapshot and take a new snapshot. :param str vg_name: name of the volume group :param str lv_name: name of the logical volume :param str lv_snapshot_name: name of the snapshot to be reverted :param str lv_snapshot_size: size of the snapshot """ lv_revert(vg_name, lv_name, lv_snapshot_name) lv_take_snapshot(vg_name, lv_name, lv_snapshot_name, lv_snapshot_size) def lv_reactivate(vg_name, lv_name, timeout=10): """ In case of unclean shutdowns some of the lvs is still active and merging is postponed. Use this function to attempt to deactivate and reactivate all of them to cause the merge to happen. :param str vg_name: name of the volume group :param str lv_name: name of the logical volume :param int timeout: timeout between operations :raises: :py:class:`LVException` if the logical volume is still active """ try: process.run(f"lvchange -an /dev/{vg_name}/{lv_name}", sudo=True) time.sleep(timeout) process.run(f"lvchange -ay /dev/{vg_name}/{lv_name}", sudo=True) time.sleep(timeout) except process.CmdError as exc: log_msg = ( "Failed to reactivate %s - please, nuke the process that uses it first." ) LOGGER.error(log_msg, lv_name) raise LVException(f"The Logical volume {lv_name} is still active") from exc def lv_mount(vg_name, lv_name, mount_loc, create_filesystem=""): """ Mount a logical volume to a mount location. :param str vg_name: name of the volume group :param str lv_name: name of the logical volume :param str mount_loc: location to mount the logical volume to :param str create_filesystem: can be one of ext2, ext3, ext4, vfat or empty if the filesystem was already created and the mkfs process is skipped :raises: :py:class:`LVException` if the logical volume could not be mounted """ try: if create_filesystem: process.run(f"mkfs.{create_filesystem} /dev/{vg_name}/{lv_name}", sudo=True) process.run(f"mount /dev/{vg_name}/{lv_name} {mount_loc}", sudo=True) except process.CmdError as ex: raise LVException(f"Fail to mount logical volume: {ex}") from ex def vg_reactivate(vg_name, timeout=10, export=False): """ In case of unclean shutdowns some of the vgs is still active and merging is postponed. Use this function to attempt to deactivate and reactivate all of them to cause the merge to happen. :param str vg_name: name of the volume group :param int timeout: timeout between operations :raises: :py:class:`LVException` if the logical volume is still active """ try: process.run(f"vgchange -an {vg_name}", sudo=True) time.sleep(timeout) if export: process.run(f"vgexport {vg_name}", sudo=True) time.sleep(timeout) process.run(f"vgimport {vg_name}", sudo=True) time.sleep(timeout) process.run(f"vgchange -ay {vg_name}", sudo=True) time.sleep(timeout) except process.CmdError as exc: log_msg = ( "Failed to reactivate %s - please, nuke the process that uses it first." ) LOGGER.error(log_msg, vg_name) raise LVException(f"The Volume group {vg_name} is still active") from exc def lv_umount(vg_name, lv_name): """ Unmount a Logical volume from a mount location. :param str vg_name: name of the volume group :param str lv_name: name of the logical volume :raises: :py:class:`LVException` if the logical volume could not be unmounted """ try: process.run(f"umount /dev/{vg_name}/{lv_name}", sudo=True) except process.CmdError as ex: raise LVException(f"Fail to unmount logical volume: {ex}") from ex avocado-113.0/avocado/utils/memory.py000066400000000000000000000360021513561306000175710ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2013-2014 # Author: Yiqiao Pu # # This code was inspired in the autotest project, # client/shared/utils.py # Authors: Yiqiao Pu import glob import logging import math import os import re from avocado.utils import data_structures, genio, process, wait from avocado.utils.data_structures import DataSize LOG = logging.getLogger(__name__) class MemError(Exception): """ called when memory operations fails """ def get_blk_string(block): """ Format the given block id to string :param block: memory block id or block string. :type string: like 198 or memory198 :return: returns string memory198 if id 198 is given :rtype: string """ if not block.startswith("memory"): return f"memory{block}" return block def _check_memory_state(block): """ Check the given memory block is online or offline :param block: memory block id. :type string: like 198 or memory198 :return: 'True' if online or 'False' if offline :rtype: bool """ def _is_online(): path = f"/sys/devices/system/memory/{get_blk_string(block)}/state" if genio.read_file(path) == "online\n": return True return False return wait.wait_for(_is_online, timeout=10, step=0.2) or False def check_hotplug(): """ Check kernel support for memory hotplug :return: True if hotplug supported, else False :rtype: 'bool' """ if glob.glob("/sys/devices/system/memory/memory*"): return True return False def is_hot_pluggable(block): """ Check if the given memory block is hotpluggable :param block: memory block id. :type string: like 198 or memory198 :return: True if hotpluggable, else False :rtype: 'bool' """ path = f"/sys/devices/system/memory/{get_blk_string(block)}/removable" return bool(int(genio.read_file(path))) def hotplug(block): """ Online the memory for the given block id. :param block: memory block id or or memory198 :type string: like 198 """ block = get_blk_string(block) # pylint: disable=W1514 with open(f"/sys/devices/system/memory/{block}/state", "w") as state_file: state_file.write("online") if not _check_memory_state(block): raise MemError(f"unable to hot-plug {block} block, not supported ?") def hotunplug(block): """ Offline the memory for the given block id. :param block: memory block id. :type string: like 198 or memory198 """ block = get_blk_string(block) # pylint: disable=W1514 with open(f"/sys/devices/system/memory/{block}/state", "w") as state_file: state_file.write("offline") if _check_memory_state(block): raise MemError(f"unable to hot-unplug {block} block. Device busy?") def read_from_meminfo(key): """ Retrieve key from meminfo. :param key: Key name, such as ``MemTotal``. """ for line in genio.read_file("/proc/meminfo").splitlines(): if key in line: return int(re.search(r"(\d+)\s*(?:kB)?$", line).group(1)) return None def memtotal(): """ Read ``Memtotal`` from meminfo. """ return read_from_meminfo("MemTotal") def memtotal_sys(): """ Reports actual memory size according to online-memory blocks available via "/sys" :return: system memory in Kb as float """ sys_mempath = "/sys/devices/system/memory" no_memblocks = 0 for directory in os.listdir(sys_mempath): if directory.startswith("memory"): path = os.path.join(sys_mempath, directory, "online") if genio.read_file(path).strip() == "1": no_memblocks += 1 path = os.path.join(sys_mempath, "block_size_bytes") block_size = int(genio.read_file(path).strip(), 16) return (no_memblocks * block_size) / 1024.0 def freememtotal(): """ Read ``MemFree`` from meminfo. """ return read_from_meminfo("MemFree") def rounded_memtotal(): """ Get memtotal, properly rounded. :return: Total memory, KB. """ # Get total of all physical mem, in kbytes usable_kbytes = memtotal() # usable_kbytes is system's usable DRAM in kbytes, # as reported by memtotal() from device /proc/meminfo memtotal # after Linux deducts 1.5% to 5.1% for system table overhead # Undo the unknown actual deduction by rounding up # to next small multiple of a big power-of-two # eg 12GB - 5.1% gets rounded back up to 12GB mindeduct = 0.015 # 1.5 percent maxdeduct = 0.055 # 5.5 percent # deduction range 1.5% .. 5.5% supports physical mem sizes # 6GB .. 12GB in steps of .5GB # 12GB .. 24GB in steps of 1 GB # 24GB .. 48GB in steps of 2 GB ... # Finer granularity in physical mem sizes would require # tighter spread between min and max possible deductions # increase mem size by at least min deduction, without rounding min_kbytes = int(usable_kbytes / (1.0 - mindeduct)) # increase mem size further by 2**n rounding, by 0..roundKb or more round_kbytes = int(usable_kbytes / (1.0 - maxdeduct)) - min_kbytes # find least binary roundup 2**n that covers worst-cast roundKb mod2n = 1 << int(math.ceil(math.log(round_kbytes, 2))) # have round_kbytes <= mod2n < round_kbytes*2 # round min_kbytes up to next multiple of mod2n phys_kbytes = min_kbytes + mod2n - 1 phys_kbytes -= phys_kbytes % mod2n # clear low bits return phys_kbytes def numa_nodes(): """ Get a list of NUMA nodes present on the system. :return: List with nodes. """ node_paths = glob.glob("/sys/devices/system/node/node*") nodes = [int(re.sub(r".*node(\d+)", r"\1", x)) for x in node_paths] return sorted(nodes) def numa_nodes_with_memory(): """ Get a list of NUMA nodes present with memory on the system. :return: List with nodes which has memory. """ mem_path = "/sys/devices/system/node/has_normal_memory" if not os.path.exists(mem_path): mem_path = "/sys/devices/system/node/has_memory" if not os.path.exists(mem_path): raise MemError("No NUMA nodes have memory") node_list = str(genio.read_file(mem_path).rstrip("\n")) return data_structures.comma_separated_ranges_to_list(node_list) def node_size(): """ Return node size. :return: Node size. """ nodes = max(len(numa_nodes()), 1) return (memtotal() * 1024) / nodes def get_page_size(): """ Get linux page size for this system. :return Kernel page size (Bytes). """ output = process.system_output("getconf PAGESIZE") return int(output) def get_supported_huge_pages_size(): """ Get all supported huge page sizes for this system. :return: list of Huge pages size (kB). """ output = os.listdir("/sys/kernel/mm/hugepages/") # Given the items in this directory are in the format hugepages-kB, # the will always start from index 10. output = [int(each[10:].rstrip("kB")) for each in output] if ( os.uname()[4] in ["ppc64", "ppc64le"] and b"PowerVM" in process.system_output("pseries_platform", ignore_status=True) and "MMU\t\t: Hash" in genio.read_file("/proc/cpuinfo").rstrip("\t\r\n\0") ): output.remove(16777216) return output def get_huge_page_size(): """ Get size of the huge pages for this system. :return: Huge pages size (KB). """ output = process.system_output("grep Hugepagesize /proc/meminfo") return int(output.split()[1]) # Assumes units always in kB. :( def get_num_huge_pages(): """ Get number of huge pages for this system. :return: Number of huge pages. """ raw_hugepages = process.system_output("/sbin/sysctl vm.nr_hugepages") return int(raw_hugepages.split()[2]) def get_num_huge_pages_meminfo(): """ Get number of huge pages for this system from /proc/meminfo file. :return: Number of huge pages. """ output = process.system_output("grep HugePages_Total /proc/meminfo") return int(output.split()[1]) def set_num_huge_pages(num): """ Set number of huge pages. :param num: Target number of huge pages. """ process.system(f"/sbin/sysctl vm.nr_hugepages={int(num)}") def drop_caches(): """ Writes back all dirty pages to disk and clears all the caches. """ process.run("sync", verbose=False) # We ignore failures here as this will fail on 2.6.11 kernels. process.run( "/bin/sh -c 'echo 3 > /proc/sys/vm/drop_caches'", ignore_status=True, verbose=False, sudo=True, ) def read_from_vmstat(key): """ Get specific item value from vmstat :param key: The item you want to check from vmstat :type key: String :return: The value of the item :rtype: int """ with open("/proc/vmstat") as vmstat: # pylint: disable=W1514 vmstat_info = vmstat.read() return int(re.findall(rf"{key}\s+(\d+)", vmstat_info)[0]) def read_from_smaps(pid, key): """ Get specific item value from the smaps of a process include all sections. :param pid: Process id :type pid: String :param key: The item you want to check from smaps :type key: String :return: The value of the item in kb :rtype: int """ with open(f"/proc/{pid}/smaps") as smaps: # pylint: disable=W1514 smaps_info = smaps.read() memory_size = 0 for each_number in re.findall(rf"{key}:\s+(\d+)", smaps_info): memory_size += int(each_number) return memory_size def read_from_numa_maps(pid, key): """ Get the process numa related info from numa_maps. This function only use to get the numbers like anon=1. :param pid: Process id :type pid: String :param key: The item you want to check from numa_maps :type key: String :return: A dict using the address as the keys :rtype: dict """ with open(f"/proc/{pid}/numa_maps") as numa_maps: # pylint: disable=W1514 numa_map_info = numa_maps.read() numa_maps_dict = {} numa_pattern = rf"(^[\dabcdfe]+)\s+.*{key}[=:](\d+)" for address, number in re.findall(numa_pattern, numa_map_info, re.M): numa_maps_dict[address] = number return numa_maps_dict def _get_buddy_info_content(): buddy_info_content = "" with open("/proc/buddyinfo") as buddy_info: # pylint: disable=W1514 buddy_info_content = buddy_info.read() return buddy_info_content def get_buddy_info(chunk_sizes, nodes="all", zones="all"): """ Get the fragment status of the host. It uses the same method to get the page size in buddyinfo. The expression to evaluate it is:: 2^chunk_size * page_size The chunk_sizes can be string make up by all orders that you want to check split with blank or a mathematical expression with ``>``, ``<`` or ``=``. For example: * The input of chunk_size could be: ``0 2 4``, and the return will be ``{'0': 3, '2': 286, '4': 687}`` * If you are using expression: ``>=9`` the return will be ``{'9': 63, '10': 225}`` :param chunk_size: The order number shows in buddyinfo. This is not the real page size. :type chunk_size: string :param nodes: The numa node that you want to check. Default value is all :type nodes: string :param zones: The memory zone that you want to check. Default value is all :type zones: string :return: A dict using the chunk_size as the keys :rtype: dict """ buddy_info_content = _get_buddy_info_content() re_buddyinfo = r"Node\s+" if nodes == "all": re_buddyinfo += r"(\d+)" else: re_buddyinfo += f"({'|'.join(nodes.split())})" if not re.findall(re_buddyinfo, buddy_info_content): LOG.warning("Can not find Nodes %s", nodes) return None re_buddyinfo += r".*?zone\s+" if zones == "all": re_buddyinfo += r"(\w+)" else: re_buddyinfo += f"({'|'.join(zones.split())})" if not re.findall(re_buddyinfo, buddy_info_content): LOG.warning("Can not find zones %s", zones) return None re_buddyinfo += r"\s+([\s\d]+)" buddy_list = re.findall(re_buddyinfo, buddy_info_content) if re.findall("[<>=]", chunk_sizes) and buddy_list: size_list = len(buddy_list[-1][-1].strip().split()) chunk_sizes = [ # pylint: disable=W0123 str(_) for _ in range(size_list) if eval(f"{_} {chunk_sizes}") ] chunk_sizes = " ".join(chunk_sizes) buddyinfo_dict = {} for chunk_size in chunk_sizes.split(): buddyinfo_dict[chunk_size] = 0 for _, _, chunk_info in buddy_list: chunk_info = chunk_info.strip().split()[int(chunk_size)] buddyinfo_dict[chunk_size] += int(chunk_info) return buddyinfo_dict def set_thp_value(feature, value): """ Sets THP feature to a given value :param feature: Thp feature to set :type feature: str :param value: Value to be set to feature :type value: str """ thp_path = "/sys/kernel/mm/transparent_hugepage/" thp_feature_to_set = os.path.join(thp_path, feature) genio.write_file_or_fail(thp_feature_to_set, value) def get_thp_value(feature): """ Gets the value of the thp feature arg passed :Param feature: Thp feature to get value :type feature: str """ thp_path = "/sys/kernel/mm/transparent_hugepage/" thp_feature_to_get = os.path.join(thp_path, feature) value = genio.read_file(thp_feature_to_get) if feature in ("enabled", "defrag", "shmem_enabled"): return (re.search(r"\[(\w+)\]", value)).group(1) return value class _MemInfoItem: """ Representation of one item from /proc/meminfo """ def __init__(self, name): self.name = name def __getattr__(self, attr): datasize = DataSize(f"{read_from_meminfo(self.name)}k") return getattr(datasize, attr) class MemInfo: """ Representation of /proc/meminfo There will not be memory information on systems that do not have a /proc/meminfo file accessible. """ def __init__(self): try: with open("/proc/meminfo", "r") as meminfo_file: # pylint: disable=W1514 for line in meminfo_file.readlines(): name = line.strip().split()[0].strip(":") safe_name = name.replace("(", "_").replace(")", "_") setattr(self, safe_name, _MemInfoItem(name)) except FileNotFoundError: pass def __iter__(self): for item in self.__dict__.items(): yield item meminfo = MemInfo() avocado-113.0/avocado/utils/multipath.py000066400000000000000000000311241513561306000202700ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: 2016 IBM # Author: Narasimhan V """ Module with multipath related utility functions. It needs root access. """ import ast import logging import time from avocado.utils import distro, process, service, wait LOG = logging.getLogger(__name__) class MPException(Exception): """ Base Exception Class for all exceptions """ def get_svc_name(): """ Gets the multipath service name based on distro. """ if distro.detect().name == "Ubuntu": return "multipath-tools" return "multipathd" def form_conf_mpath_file(blacklist="", defaults_extra=""): """ Form a multipath configuration file, and restart multipath service. :param blacklist: Entry in conf file to indicate blacklist section. :param defaults_extra: Extra entry in conf file in defaults section. """ conf_file = "/etc/multipath.conf" with open(conf_file, "w", encoding="utf-8") as mpath_fp: mpath_fp.write("defaults {\n") mpath_fp.write(" find_multipaths yes\n") mpath_fp.write(" user_friendly_names yes\n") if defaults_extra: mpath_fp.write(f" {defaults_extra}\n") mpath_fp.write("}\n") if blacklist: mpath_fp.write("blacklist {\n") mpath_fp.write(f" {blacklist}\n") mpath_fp.write("}\n") with open(conf_file, "r", encoding="utf-8") as fl: LOG.debug(fl.read()) # The reason for sleep here is to give some time for change in # multipath.conf file to take effect. time.sleep(5) mpath_svc = service.SpecificServiceManager(get_svc_name()) mpath_svc.restart() # pylint: disable=E1101 wait.wait_for(mpath_svc.status, timeout=10) # pylint: disable=E1101 def device_exists(mpath): """ Checks if a given mpath exists. :param mpath: The multipath path :return: True if path exists, False if does not exist. :rtype: bool """ cmd = "multipath -ll" out = process.run(cmd, ignore_status=True, sudo=True, shell=True).stdout_text if mpath in out: return True return False def get_mpath_name(wwid): """ Get multipath name for a given wwid. :param wwid: wwid of multipath device. :return: Name of multipath device. :rtype: str """ if device_exists(wwid): cmd = f"multipath -l {wwid}" return process.run(cmd, sudo=True).stdout_text.split()[0] return None def get_mpath_from_dm(dm_id): """ Get the mpath name for given device mapper id :param dev_mapper: Input device mapper dm-x :return: mpath name like mpathx :rtype: str """ cmd = "multipathd show maps format '%d %n'" try: mpaths = process.run(cmd, ignore_status=True, sudo=True, shell=True).stdout_text except process.CmdError as ex: raise MPException(f"Multipathd Command Failed : {ex} ") from ex for mpath in mpaths.splitlines(): if dm_id in mpath: return mpath.split()[1] return None def get_multipath_wwids(): """ Get list of multipath wwids. :return: List of multipath wwids. :rtype: list of str """ cmd = "egrep -v '^($|#)' /etc/multipath/wwids" wwids = process.run(cmd, ignore_status=True, sudo=True, shell=True).stdout_text wwids = wwids.strip("\n").replace("/", "").split("\n") return wwids def get_multipath_wwid(mpath): """ Get the wwid binding for given mpath name :return: Multipath wwid :rtype: str """ cmd = "multipathd show maps format '%n %w'" try: wwids = process.run(cmd, ignore_status=True, sudo=True, shell=True).stdout_text except process.CmdError as ex: raise MPException(f"Multipathd Command Failed : {ex} ") from ex for wwid in wwids.splitlines(): if mpath in wwid: return wwid.split()[1] return None def is_mpath_dev(mpath): """ Check the give name is a multipath device name or not. :return: True if device is multipath or False :rtype: Boolean """ cmd = "multipath -l -v 1" try: mpaths = process.run(cmd, ignore_status=True, sudo=True, shell=True).stdout_text except process.CmdError as ex: raise MPException(f"Multipath Command Failed : {ex} ") from ex if mpath in mpaths.strip("\n").split("\n"): return True return False def get_paths(wwid): """ Get list of paths, given a multipath wwid. :return: List of paths. :rtype: list of str """ if not device_exists(wwid): return None cmd = f"multipath -ll {wwid}" lines = process.run(cmd, sudo=True).stdout_text.strip("\n") paths = [] for line in lines.split("\n"): if not (("size" in line) or ("policy" in line) or (wwid in line)): paths.append(line.split()[-5]) return paths def get_multipath_details(): """ Get multipath details as a dictionary. This is the output of the following command: $ multipathd show maps json :return: Dictionary of multipath output in json format :rtype: dict """ mpath_op = process.run( "multipathd show maps json", sudo=True, verbose=False ).stdout_text if "multipath-tools v" in mpath_op: return "" mpath_op = ast.literal_eval(mpath_op.replace("\n", "").replace(" ", "")) return mpath_op def is_path_a_multipath(disk_path): """ Check if given disk path is part of a multipath. :param disk_path: disk path. Example: sda, sdb. :return: True if part of multipath, else False. """ if not process.system( f"multipath -c /dev/{disk_path}", sudo=True, ignore_status=True, shell=True ): return True return False def get_path_status(disk_path): """ Return the status of a path in multipath. :param disk_path: disk path. Example: sda, sdb. :return: Tuple in the format of (dm status, dev status, checker status) """ mpath_op = get_multipath_details() if not mpath_op: return ("", "", "") for maps in mpath_op["maps"]: for path_groups in maps["path_groups"]: for paths in path_groups["paths"]: if paths["dev"] == disk_path: return (paths["dm_st"], paths["dev_st"], paths["chk_st"]) return None def get_mpath_paths_status(wwid): """ Return the status of all paths of mpath device. :param wwid: wwid or user friendly name of mpath. Example: mpatha or 360050768108001b3a800000000000296 :return: Dict in the format of {path: (dm status, dev status, checker status)} """ mpath_op = get_multipath_details() if not mpath_op: return None wwid_paths = {} for maps in mpath_op["maps"]: if wwid in (maps["name"], maps["uuid"]): for path_groups in maps["path_groups"]: for paths in path_groups["paths"]: wwid_paths[paths["dev"]] = ( paths["dm_st"], paths["dev_st"], paths["chk_st"], ) if len(wwid_paths): return wwid_paths return None def fail_path(path): """ Fail the individual paths. :param str path: disk path. Example: sda, sdb. :return: True if succeeded, False otherwise :rtype: bool """ def is_failed(): path_stat = get_path_status(path) return ( path_stat is not None and path_stat[0] == "failed" and path_stat[2] == "faulty" ) cmd = f'multipathd -k"fail path {path}"' if not process.system(cmd): return wait.wait_for(is_failed, timeout=10) or False return False def reinstate_path(path): """ Reinstate the individual paths. :param str path: disk path. Example: sda, sdb. :return: True if succeeded, False otherwise """ def is_reinstated(): path_stat = get_path_status(path) return ( path_stat is not None and path_stat[0] == "active" and path_stat[2] == "ready" ) cmd = f'multipathd -k"reinstate path {path}"' if not process.system(cmd): return wait.wait_for(is_reinstated, timeout=10) or False return False def get_policy(wwid): """ Gets path_checker policy, given a multipath wwid. :return: path checker policy. :rtype: str """ if device_exists(wwid): cmd = f"multipath -ll {wwid}" lines = process.run(cmd, sudo=True).stdout_text.strip("\n") for line in lines.split("\n"): if "policy" in line: return line.split("'")[1].split()[0] return None def get_size(wwid): """ Gets size of device, given a multipath wwid. :return: size of multipath device. :rtype: str """ if device_exists(wwid): cmd = f"multipath -ll {wwid}" lines = process.run(cmd, sudo=True).stdout_text.strip("\n") for line in lines.split("\n"): if "size" in line: return line.split("=")[1].split()[0] return None def flush_path(path_name): """ Flushes the given multipath. :return: Returns False if command fails, True otherwise. """ cmd = f"multipath -f {path_name}" if process.system(cmd, ignore_status=True, sudo=True, shell=True): return False return True def get_mpath_status(mpath): """ Get the status of mpathX of multipaths. :param mpath: mpath names. Example: mpatha, mpathb. :return: state of mpathX eg: Active, Suspend, None """ cmd = f'multipathd -k"show maps status" | grep -i {mpath}' mpath_status = process.getoutput(cmd).split()[-2] return mpath_status def suspend_mpath(mpath): """ Suspend the given mpathX of multipaths. :param mpath: mpath names. Example: mpatha, mpathb. :return: True or False """ def is_mpath_suspended(): if get_mpath_status(mpath) == "suspend": return True return False cmd = f'multipathd -k"suspend map {mpath}"' if not process.system(cmd): return wait.wait_for(is_mpath_suspended, timeout=10) or False return False def resume_mpath(mpath): """ Resume the suspended mpathX of multipaths. :param mpath_name: mpath names. Example: mpatha, mpathb. :return: True or False """ def is_mpath_resumed(): if get_mpath_status(mpath) == "active": return True return False cmd = f'multipathd -k"resume map {mpath}"' if not process.system(cmd): return wait.wait_for(is_mpath_resumed, timeout=10) or False return False def remove_mpath(mpath): """ Remove the mpathX of multipaths. :param mpath_name: mpath names. Example: mpatha, mpathb. :return: True or False """ def is_mpath_removed(): if device_exists(mpath): return False return True cmd = f'multipathd -k"remove map {mpath}"' if not process.system(cmd): return wait.wait_for(is_mpath_removed, timeout=10) or False return False def add_mpath(mpath): """ Add back the removed mpathX of multipath. :param mpath_name: mpath names. Example: mpatha, mpathb. :return: True or False """ def is_mpath_added(): if device_exists(mpath): return True return False cmd = f'multipathd -k"add map {mpath}"' if not process.system(cmd): return wait.wait_for(is_mpath_added, timeout=10) or False return False def remove_path(path): """ Remove the individual paths. :param disk_path: disk path. Example: sda, sdb. :return: True or False """ def is_path_removed(): if get_path_status(path) is None: return True return False cmd = f'multipathd -k"remove path {path}"' if not process.system(cmd): return wait.wait_for(is_path_removed, timeout=10) or False return False def add_path(path): """ Add back the removed individual paths. :param str path: disk path. Example: sda, sdb. :return: True or False """ def is_path_added(): if get_path_status(path) is None: return False return True cmd = f'multipathd -k"add path {path}"' if not process.system(cmd): return wait.wait_for(is_path_added, timeout=10) or False return False avocado-113.0/avocado/utils/network/000077500000000000000000000000001513561306000173775ustar00rootroot00000000000000avocado-113.0/avocado/utils/network/__init__.py000066400000000000000000000000001513561306000214760ustar00rootroot00000000000000avocado-113.0/avocado/utils/network/common.py000066400000000000000000000031251513561306000212420ustar00rootroot00000000000000"""Utility helpers for executing shell commands on local or remote hosts. This module centralizes the logic used by network utilities to run commands either on the local machine or through a remote session, with optional privilege escalation via 'sudo'. """ from avocado.utils import process def run_command(command, host, sudo=False): """Executes a given command on a specified host, either locally or remotely. This function determines if the command should be run on the local machine or a remote host based on the type of the 'host' object. It can also execute commands with superuser privileges if specified. :param command: The command string to be executed. :type command: str :param host: The host object where the command will be run. If host.__class__.__name__ is "LocalHost", the command is run locally. Otherwise, it's treated as a remote host with a 'remote_session' attribute supporting a 'cmd' method. :type host: object :param sudo: If True, the command will be executed with 'sudo'. Defaults to False. :type sudo: bool :return: The standard output of the executed command, decoded as UTF-8. :rtype: str :raises AttributeError: If 'host' is not "LocalHost" and does not have a 'remote_session.cmd' method. """ if host.__class__.__name__ == "LocalHost": return process.system_output(command, sudo=sudo).decode("utf-8") if sudo: command = f"sudo {command}" return host.remote_session.cmd(command).stdout.decode("utf-8") avocado-113.0/avocado/utils/network/exceptions.py000066400000000000000000000005361513561306000221360ustar00rootroot00000000000000"""Exception hierarchy for network utilities. This module defines custom exception classes used throughout network utilities. All specific network errors should inherit from :class:`NWException` so they can be handled collectively when required. """ class NWException(Exception): """Base Exception Class for all network utility exceptions.""" avocado-113.0/avocado/utils/network/hosts.py000066400000000000000000000271441513561306000211210ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # # Copyright: 2019-2020 Red Hat Inc. # Authors : Beraldo Leal """Provides a high-level API for interacting with network hosts. This module offers classes to represent and manage local and remote hosts, allowing users to query network interface information, validate MAC addresses, and retrieve routing details. It abstracts the underlying command execution for common network-related tasks. """ import json import re from avocado.utils.network.common import run_command from avocado.utils.network.exceptions import NWException from avocado.utils.network.interfaces import NetworkInterface from avocado.utils.ssh import Session class Host: """This class represents a base Host and shouldn't be instantiated directly. Use one of the child classes (LocalHost or RemoteHost). During the initialization of a child, network interfaces can be detected and made available via the `interfaces` attribute. This attribute can be accessed on both LocalHost and RemoteHost instances. So, for instance, you could have a local and a remote host:: remote = RemoteHost(host='foo', port=22, username='foo', password='bar') local = LocalHost() You can iterate over the network interfaces of any host:: for i in remote.interfaces: print(f"Interface: {i.name}, Link Up: {i.is_link_up()}") """ def __init__(self, host): """Initializes the base Host class. This constructor is intended to be called by sub-classes. Direct initialization of `Host` will raise a TypeError. :param host: The hostname or IP address of the host. For `LocalHost`, this typically defaults to 'localhost'. :type host: str :raises TypeError: If the `Host` class is instantiated directly. """ if type(self) == Host: # pylint: disable=C0123 raise TypeError("Host class should not be instantiated") self.host = host @property def interfaces(self): """Retrieves the network interfaces for the host. Detects network interface names by listing contents of '/sys/class/net'. Excludes 'bonding_masters' from the list. :return: A list of NetworkInterface objects for all network interfaces on the host. :rtype: list[NetworkInterface] :raises avocado.utils.network.exceptions.NWException: If the command to list interfaces fails or the output cannot be processed. """ cmd = "ls /sys/class/net" try: names = run_command(cmd, self).split() except Exception as ex: raise NWException(f"Failed to get interfaces on {self.host}: {ex}") from ex if "bonding_masters" in names: names.remove("bonding_masters") return [NetworkInterface(if_name=name, host=self) for name in names] def get_interface_by_ipaddr(self, ipaddr): """Retrieves a network interface that has a specific IP address. :param ipaddr: The IP address to search for among the host's network interfaces. :type ipaddr: str :return: The NetworkInterface object matching the given IP address, or None if no interface is found with that IP. :rtype: NetworkInterface or None """ for interface in self.interfaces: if ipaddr in interface.get_ipaddrs(): return interface return None def get_interface_by_hwaddr(self, mac): """Retrieves a network interface that has a specific MAC address. :param mac: The MAC address to search for. The MAC address format should be consistent with the output of `interface.get_hwaddr()`. :type mac: str :return: The NetworkInterface object matching the given MAC address, or None if no interface is found with that MAC address. :rtype: NetworkInterface or None """ for interface in self.interfaces: if mac in interface.get_hwaddr(): return interface return None def get_all_hwaddr(self): """Gets a list of all MAC addresses on the host. Executes 'ip -j address' and parses the JSON output to extract MAC addresses. :return: A list of strings, where each string is a MAC address found on the host. :rtype: list[str] :raises avocado.utils.network.exceptions.NWException: If the command execution fails or the JSON output cannot be parsed. """ cmd = "ip -j address" output = run_command(cmd, self) try: result = json.loads(output) return [str(item["address"]) for item in result] except Exception as ex: raise NWException( f"Could not get MAC addresses from {self.host}: {ex}" ) from ex @staticmethod def validate_mac_addr(mac_id): """Checks if a given MAC address string is in a valid format. A valid MAC address is defined as a 12-digit hexadecimal number, with pairs of digits separated by colons (e.g., '36:84:37:5a:ea:02'). :param mac_id: The MAC address string to validate. :type mac_id: str or None :return: True if `mac_id` is a valid MAC address string, False otherwise. Also returns False if `mac_id` is None. :rtype: bool """ exp = "^[0-9a-f]{2}([:])[0-9a-f]{2}(\\1[0-9a-f]{2}){4}$" check = re.compile(exp) if mac_id is None: return False if re.search(check, mac_id): return True return False def get_default_route_interface(self): """Gets the name(s) of the interface(s) used for the default route. Executes 'ip -j route list default' and parses the JSON output to extract the device name(s) associated with the default route(s). :return: A list of strings, where each string is the name of an interface used for a default route. Can be empty if no default route is found or if the 'dev' field is missing. :rtype: list :raises avocado.utils.network.exceptions.NWException: If the command execution fails or the JSON output cannot be parsed. """ cmd = "ip -j route list default" output = run_command(cmd, self) try: result = json.loads(output) return [str(item["dev"]) for item in result] except Exception as ex: raise NWException( f"Could not get default route interface name from {self.host}: {ex}" ) from ex class LocalHost(Host): """Represents the local machine, inheriting from `Host`. Use this class to get information about the network configuration of the machine where the script is running. Example:: local = LocalHost() print(f"Local hostname: {local.host}") for iface in local.interfaces: print(f"Interface: {iface.name}") """ def __init__(self, host="localhost"): """Initializes a LocalHost instance. :param str host: The name to assign to the local host. Defaults to "localhost". """ super().__init__(host) class RemoteHost(Host): """Represents a remote machine, inheriting from `Host`. This class manages an SSH connection to a remote host to execute commands and retrieve network information. It requires credentials (username and either password or SSH key) to establish the connection. The class can be used as a context manager to ensure the SSH session is properly closed. Example with password:: remote_details = { 'host': '192.168.0.1', 'port': 22, 'username': 'user', 'password': 'secure-password' } with RemoteHost(**remote_details) as remote: print(f"Remote interfaces on {remote.host}:") for iface in remote.interfaces: print(f"- {iface.name}") Example with SSH key:: remote_key_details = { 'host': 'server.example.com', 'username': 'admin', 'key': '/path/to/private/key' } with RemoteHost(**remote_key_details) as remote_server: # ... interact with remote_server ... pass # Session closes automatically on exit """ # pylint: disable=R0913 def __init__(self, host, username, port=22, key=None, password=None): """Initializes a RemoteHost instance and establishes an SSH connection. :param host: The hostname or IP address of the remote host. :type host: str :param username: The username for SSH authentication. :type username: str :param port: The SSH port on the remote host. Defaults to 22. :type port: int :param key: The file path to the private SSH key for key-based authentication. Defaults to None. :type key: str :param str password: The password for password-based authentication. Defaults to None. :raises avocado.utils.network.exceptions.NWException: If the SSH connection to the remote host fails. """ super().__init__(host) self.port = port self.username = username self.key = key self.password = password self.remote_session = self._connect() def __enter__(self): """Enters the runtime context related to this object. Ensures the SSH session is active. If the session was previously closed or not established, it attempts to reconnect. :return: The instance itself. :rtype: RemoteHost """ if not self.remote_session: self._connect() return self def __exit__(self, _type, _exc_value, _traceback): """Exits the runtime context, ensuring the SSH session is closed. :param _type: The exception type if an exception was raised in the `with` block. :param _exc_value: The exception value if an exception was raised. :param _traceback: The traceback object if an exception was raised. """ if self.remote_session: self.remote_session.quit() def _connect(self): """Establishes an SSH session to the remote host. Uses the host, port, username, and key/password provided during initialization. :return: The established SSH session object. :rtype: avocado.utils.ssh.Session :raises avocado.utils.network.exceptions.NWException: If the SSH connection attempt fails. """ session = Session( host=self.host, port=self.port, user=self.username, key=self.key, password=self.password, ) if session.connect(): return session msg = ( f"Failed connecting to {self.username}@{self.host}:{self.port}. " f"Ensure credentials are correct and host is reachable." ) raise NWException(msg) avocado-113.0/avocado/utils/network/interfaces.py000066400000000000000000001234121513561306000220770ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # # Copyright: 2019-2020 IBM # Copyright: 2019-2020 Red Hat Inc. # Authors : Beraldo Leal # : Praveen K Pandey # : Vaishnavi Bhat """Network interface management utilities. This module defines the `NetworkInterface` class, which provides a high-level, distribution-aware wrapper for performing common network configuration tasks on both local and remote hosts. Functionality includes, but is not limited to: * Querying and manipulating interface attributes (MAC, MTU, IP addresses, VLANs, bonding information). * Persisting interface settings to the appropriate configuration files for RHEL/Fedora and SuSE families, with support for NetworkManager connection profiles on modern releases (RHEL â‰Ĩ 9, SuSE â‰Ĩ 16). * Executing connectivity checks such as ICMP ping and detecting packet loss or interface link status. * Gracefully handling errors through custom exceptions defined in `avocado.utils.network.exceptions`. All shell commands are executed via `avocado.utils.network.common.run_command`, allowing seamless operation on either a local `LocalHost` instance or a remote `RemoteHost` session. """ # pylint: disable=too-many-lines import json import logging import os import re import shutil import subprocess from ipaddress import AddressValueError, IPv4Address, ip_interface from avocado.utils import process from avocado.utils.distro import detect as distro_detect from avocado.utils.network.common import run_command from avocado.utils.network.exceptions import NWException from avocado.utils.wait import wait_for LOG = logging.getLogger(__name__) # pylint: disable=R0904 class NetworkInterface: """Represents a network card interface (NIC). An "NetworkInterface" is attached to some host. This could be an instance of LocalHost or RemoteHost. If a RemoteHost then all commands will be executed on a remote_session (host.remote_session). Otherwise will be executed locally. Here you will find a few methods to perform basic operations on a NIC. """ def __init__(self, if_name, host, if_type="Ethernet"): """Initializes a NetworkInterface object. :param if_name: The name of the network interface (e.g., 'eth0'). :type if_name: str :param host: The host object (LocalHost or RemoteHost) to which this interface is attached. :type host: object :param if_type: The type of the interface (e.g., 'Ethernet', 'Bond'). Defaults to "Ethernet". :type if_type: str """ self.name = if_name self.if_type = if_type self.host = host self.distro_is_rhel9_or_later = False self.distro_is_suse16_or_later = False @property def config_filename(self): """Gets the full path to the network interface configuration file. The path is determined based on the detected Linux distribution. Supports RHEL/Fedora and SuSE distributions, with different paths for newer versions (RHEL 9+, SuSE 16+). :return: The absolute path to the configuration file. :rtype: str :raises avocado.utils.network.exceptions.NWException: If the distribution is not supported. """ current_distro = distro_detect() if current_distro.name in ["rhel", "fedora"]: if self.distro_is_rhel9_or_later: path = "/etc/NetworkManager/system-connections" else: path = "/etc/sysconfig/network-scripts" elif current_distro.name == "SuSE": if self.distro_is_suse16_or_later: path = "/etc/NetworkManager/system-connections" else: path = "/etc/sysconfig/network" else: msg = "Distro not supported by API. Could not get interface filename." raise NWException(msg) if self.distro_is_rhel9_or_later or self.distro_is_suse16_or_later: return f"{path}/{self.name}.nmconnection" return f"{path}/ifcfg-{self.name}" @property def config_file_path(self): """Gets the directory path for network interface configuration files. The path is determined based on the detected Linux distribution. Supports RHEL/Fedora and SuSE distributions. :return: The absolute path to the configuration directory, or None if the distribution is not supported. :rtype: str or None """ current_distro = distro_detect() if current_distro.name in ["rhel", "fedora"]: if self.distro_is_rhel9_or_later: return "/etc/NetworkManager/system-connections" return "/etc/sysconfig/network-scripts" if current_distro.name == "SuSE": if self.distro_is_suse16_or_later: return "/etc/NetworkManager/system-connections" return "/etc/sysconfig/network" msg = "Distro not supported by API. Could not get interface filename." LOG.error(msg) return None @property def slave_config_filename(self): """Gets the configuration filenames for all slave interfaces of a bond. This is only applicable for 'Bond' type interfaces. :return: A list of absolute paths to the slave interface configuration files, or None if not a bond interface or if slaves cannot be determined. :rtype: list[str] or None """ try: slave_dict = self._get_bondinterface_details() if self.distro_is_rhel9_or_later or self.distro_is_suse16_or_later: return [ f"{self.config_file_path}/{slave}.nmconnection" for slave in slave_dict["slaves"] ] return [ f"{self.config_file_path}/ifcfg-{slave}" for slave in slave_dict["slaves"] ] except NWException: msg = "Slave config filename not available" LOG.debug(msg) return None def _get_interface_details(self, version=None): """Retrieves detailed information for the network interface. Uses the 'ip' command to fetch details and parses the JSON output. :param version: The IP version (4 or 6) to query for address information. If None, link layer information is fetched. Defaults to None. :type version: int :return: A dictionary containing interface details. :rtype: dict :raises avocado.utils.network.exceptions.NWException: If the command fails or the interface is not found. """ cmd = f"ip -j link show {self.name}" if version: cmd = f"ip -{version} -j address show {self.name}" output = run_command(cmd, self.host) try: result = json.loads(output) for item in result: if item.get("ifname") == self.name: return item raise NWException("Interface not found") except (NWException, json.JSONDecodeError) as exc: msg = f"Unable to get the details of interface {self.name}" LOG.error(msg) raise NWException(msg) from exc def _get_bondinterface_details(self): """Retrieves bonding mode and slave interfaces for a bond interface. Reads bonding information from '/sys/class/net//bonding/'. :return: A dictionary with 'mode' and 'slaves' of the bond. :rtype: dict :raises avocado.utils.network.exceptions.NWException: If the interface is not a bond or information cannot be retrieved. """ cmd = ( f"cat /sys/class/net/{self.name}/bonding/mode " f"/sys/class/net/{self.name}/bonding/slaves" ) try: mode, slaves = run_command(cmd, self.host).splitlines() return {"mode": mode.split(), "slaves": slaves.split()} except Exception as exc: raise NWException( f"Slave interface not found for " f"the bond {self.name}" ) from exc def _move_file_to_backup(self, filename, ignore_missing=True): """Moves a file to a backup location. Appends '.backup' to the original filename. :param filename: The path to the file to be backed up. :type filename: str :param ignore_missing: If True, do not raise an exception if the file does not exist. Defaults to True. :type ignore_missing: bool :raises avocado.utils.network.exceptions.NWException: If the file does not exist and `ignore_missing` is False. """ destination = f"{filename}.backup" if os.path.exists(filename): shutil.move(filename, destination) else: if not ignore_missing: raise NWException(f"{self.name} interface not available") def _write_to_file(self, filename, values): """Writes key-value pairs to a configuration file. First, it backs up the existing file using `_move_file_to_backup`. Then, it writes the new content in 'KEY=value' format. :param filename: The path to the file to write to. :type filename: str :param values: A dictionary of key-value pairs to write to the file. :type values: dict """ self._move_file_to_backup(filename) with open(filename, "w+") as fp: # pylint: disable=W1514 for key, value in values.items(): fp.write(f"{key}={value}\n") def set_hwaddr(self, hwaddr): """Sets a Hardware Address (MAC Address) to the interface. This method will try to set a new hwaddr to this interface, if fails it will raise a NWException. You must have sudo permissions to run this method on a host. :param hwaddr: Hardware Address (Mac Address). :type hwaddr: str :raises avocado.utils.network.exceptions.NWException: If the command to set the MAC address fails. """ cmd = f"ip link set dev {self.name} address {hwaddr}" try: run_command(cmd, self.host, sudo=True) except Exception as ex: raise NWException(f"Adding hw address fails: {ex}") from ex def add_ipaddr(self, ipaddr, netmask): """Add an IP Address (with netmask) to the interface. This method will try to add a new ipaddr/netmask this interface, if fails it will raise a NWException. You must have sudo permissions to run this method on a host. :param ipaddr: IP Address. :type ipaddr: str :param netmask: Network mask. :type netmask: str :raises avocado.utils.network.exceptions.NWException: If adding the IP address fails. """ ip = ip_interface(f"{ipaddr}/{netmask}") cmd = f"ip addr add {ip.compressed} dev {self.name}" try: run_command(cmd, self.host, sudo=True) except Exception as ex: raise NWException(f"Failed to add address {ex}") from ex @property def vlans(self): """Return all interface's VLAN. This is a dict were key is the VLAN number and the value is the name of the VLAN interface. :return: A dictionary where the key is the VLAN number and the value is the name of the VLAN interface. :rtype: dict """ vlans = {} if not os.path.exists("/proc/net/vlan/config"): return vlans with open("/proc/net/vlan/config", encoding="utf-8") as vlan_config_file: for line in vlan_config_file: # entry is formatted as "vlan_name | vlan_id | parent_device" line = "".join(line.split()) if line.endswith(self.name): line = line.split("|") vlans[line[1]] = line[0] return vlans def add_vlan_tag(self, vlan_num, vlan_name=None): """Configure 802.1Q VLAN tagging to the interface. This method will attempt to add a VLAN tag to this interface. If it fails, the method will raise a NWException. :param vlan_num: VLAN ID. :type vlan_num: int :param vlan_name: option to name VLAN interface, by default it is named . :type vlan_name: str :raises avocado.utils.network.exceptions.NWException: If adding the VLAN tag fails. """ vlan_name = vlan_name or f"{self.name}.{vlan_num}" cmd = ( f"ip link add link {self.name} name {vlan_name} " f"type vlan id {vlan_num}" ) try: run_command(cmd, self.host, sudo=True) except Exception as ex: raise NWException(f"Failed to add VLAN tag: {ex}") from ex def remove_vlan_by_tag(self, vlan_num): """Remove the VLAN of the interface by tag number. This method will try to remove the VLAN tag of this interface. If it fails, the method will raise a NWException. :param vlan_num: VLAN ID :type vlan_num: int :return: True or False, True if it found the VLAN interface and removed it successfully, otherwise it will return False. :rtype: bool :raises avocado.utils.network.exceptions.NWException: If removing the VLAN tag fails. """ if str(vlan_num) in self.vlans: vlan_name = self.vlans[str(vlan_num)] else: return False cmd = f"ip link delete {vlan_name}" try: run_command(cmd, self.host, sudo=True) return True except Exception as ex: raise NWException(f"Failed to remove VLAN interface: {ex}") from ex def remove_all_vlans(self): """Remove all VLANs of this interface. This method will remove all the VLAN interfaces associated by the interface. :raises avocado.utils.network.exceptions.NWException: If removing the VLAN interfaces fails. """ try: for v in self.vlans.values(): cmd = f"ip link delete {v}" run_command(cmd, self.host, sudo=True) except Exception as ex: raise NWException(f"Failed to remove VLAN interface: {ex}") from ex def bring_down(self): """Shutdown the interface. This will shutdown the interface link. Be careful, you might lost connection to the host. You must have sudo permissions to run this method on a host. :raises avocado.utils.network.exceptions.NWException: If the command to bring down the interface fails. """ cmd = f"ip link set {self.name} down" try: run_command(cmd, self.host, sudo=True) except Exception as ex: raise NWException(f"Failed to bring down: {ex}") from ex def bring_up(self): """Wake-up the interface. This will wake-up the interface link. You must have sudo permissions to run this method on a host. :raises avocado.utils.network.exceptions.NWException: If the command to bring up the interface fails. """ cmd = f"ip link set {self.name} up" try: run_command(cmd, self.host, sudo=True) except Exception as ex: raise NWException(f"Failed to bring up: {ex}") from ex def is_admin_link_up(self): """Check the admin link state is up or not. :return: True or False, True if network interface state is 'UP' otherwise will return False. :rtype: bool :raises avocado.utils.network.exceptions.NWException: If the link state cannot be retrieved. """ try: if "UP" in self._get_interface_details().get("flags"): return True except (NWException, IndexError) as exc: raise NWException("Could not get Administrative link state.") from exc return False def is_operational_link_up(self): """Check Operational link state is up or not. :return: True or False. True if operational link state is LOWER_UP, otherwise will return False. :rtype: bool :raises avocado.utils.network.exceptions.NWException: If the link state cannot be retrieved. """ try: if "LOWER_UP" in self._get_interface_details().get("flags"): return True except (NWException, IndexError) as exc: raise NWException("Could not get operational link state.") from exc return False def is_link_up(self): """Check if the interface is up or not. :return: True or False. True if admin link state and operational link state is up otherwise will return False. :rtype: bool :raises avocado.utils.network.exceptions.NWException: If the link state cannot be retrieved. """ return self.is_admin_link_up() and self.is_operational_link_up() def get_ipaddrs(self, version=4): """Get the IP addresses from a network interface. Interfaces can hold multiple IP addresses. This method will return a list with all addresses on this interface. :param version: Address Family Version (4 or 6). This must be a integer and default is 4. :type version: int :return: IP address as string. :rtype: list[str] :raises avocado.utils.network.exceptions.NWException: If an unsupported IP version is provided. """ if version not in [4, 6]: raise NWException(f"Version {version} not supported") try: details = self._get_interface_details(version) addr_info = details.get("addr_info") if addr_info: return [x.get("local") for x in addr_info] except (NWException, IndexError): msg = f"Could not get ip addresses for {self.name}" LOG.debug(msg) return [] def get_hwaddr(self): """Get the Hardware Address (MAC) of this interface. This method will try to get the address and if fails it will raise a NWException. :return: The hardware address (MAC address). :rtype: str :raises avocado.utils.network.exceptions.NWException: If fetching the hardware address fails. """ cmd = f"cat /sys/class/net/{self.name}/address" try: return run_command(cmd, self.host) except Exception as ex: raise NWException(f"Failed to get hw address: {ex}") from ex def get_mtu(self): """Return the current MTU value of this interface. This method will try to get the current MTU value, if fails will raise a NWException. :return: The MTU value of the interface. :rtype: int :raises avocado.utils.network.exceptions.NWException: If the MTU value cannot be retrieved. """ try: return self._get_interface_details().get("mtu") except (NWException, IndexError) as exc: raise NWException("Could not get MUT value.") from exc def ping_check(self, peer_ip, count=2, options=None): """This method will try to ping a peer address (IPv4 or IPv6). You should provide a IPv4 or IPV6 that would like to ping. This method will try to ping the peer and if fails it will raise a NWException. :param peer_ip: Peer IP address (IPv4 or IPv6). :type peer_ip: str :param count: How many packets to send. Default is 2. :type count: int :param options: ping command options. Default is None. :type options: str :raises avocado.utils.network.exceptions.NWException: If the ping command fails. """ cmd = f"ping -I {self.name} {peer_ip} -c {count}" if options is not None: cmd = f"{cmd} {options}" try: run_command(cmd, self.host) except Exception as ex: raise NWException(f"Failed to ping: {ex}") from ex # pylint: disable=R0915 def save(self, ipaddr, netmask): """Save current interface IP Address to the system configuration file. If the ipaddr is valid (currently being used by the interface) this will try to save the current settings into /etc/. This check is necessary to avoid inconsistency. Before save, you should add_ipaddr, first. Currently, only RHEL, Fedora and SuSE are supported. And this will create a backup file of your current configuration if found. :param ipaddr : IP Address which need to configure for interface :type ipaddr: str :param netmask: Network mask which is associated to the provided IP :type netmask: str :raises avocado.utils.network.exceptions.NWException: If *ipaddr* is not configured on the interface or the distribution is not supported. """ def save_distro_rhel9_and_suse16_or_later(): filename = f"{self.name}.nmconnection" prefix = self.netmask_to_cidr(netmask) path = "/etc/NetworkManager/system-connections" if self.if_type == "Ethernet": if not os.path.exists(f"{path}/{filename}"): run_command( f"nmcli connection add con-name {self.name} ifname {self.name} type ethernet ipv4.address {ipaddr}/{prefix}", self.host, ) self._move_file_to_backup(f"{path}/{filename}") if os.path.exists(f"{path}/{filename}.backup"): destination = f"{path}/{filename}" shutil.copy(f"{path}/{filename}.backup", destination) run_command( f"nmcli c mod id {self.name} ipv4.method manual ipv4.address {ipaddr}/{prefix}", self.host, ) run_command(f"nmcli connection up {self.name}", self.host) if self.if_type == "Bond": bond_dict = self._get_bondinterface_details() mode = bond_dict["mode"][0] if os.path.exists(f"{path}/{filename}") is False: run_command( f"nmcli connection add con-name {self.name} ifname {self.name} type bond ipv4.address {ipaddr}/{prefix}" f' bond.options "mode={mode},miimon=100"', self.host, ) self._move_file_to_backup(f"{path}/{filename}") if os.path.exists(f"{path}/{filename}.backup"): destination = f"{path}/{filename}" shutil.copy(f"{path}/{filename}.backup", destination) run_command( f"nmcli c mod id {self.name} ipv4.method manual ipv4.address {ipaddr}/{prefix}", self.host, ) run_command(f"nmcli connection up {self.name}", self.host) def save_distro_rhel8_or_older(): filename = f"ifcfg-{self.name}" if current_distro.name in ["rhel", "fedora"]: path = "/etc/sysconfig/network-scripts" ifcfg_dict = { "TYPE": self.if_type, "BOOTPROTO": "static", "NAME": self.name, "DEVICE": self.name, "ONBOOT": "yes", "IPADDR": ipaddr, "NETMASK": netmask, "IPV6INIT": "yes", "IPV6_AUTOCONF": "yes", "IPV6_DEFROUTE": "yes", } self._write_to_file(f"{path}/{filename}", ifcfg_dict) if self.if_type == "Bond": bond_dict = self._get_bondinterface_details() ifcfg_dict["BONDING_MASTER"] = "yes" ifcfg_slave_dict = { "SLAVE": "yes", "ONBOOT": "yes", "MASTER": self.name, } ifcfg_dict["BONDING_OPTS"] = "mode=" + bond_dict["mode"][0] for _, slave in enumerate(bond_dict["slaves"]): ifcfg_slave_dict.update( {"NAME": slave, "DEVICE": slave, "TYPE": "Ethernet"} ) self._write_to_file(f"{path}/ifcfg-{slave}", ifcfg_slave_dict) def save_suse(): filename = f"ifcfg-{self.name}" prefix = self.netmask_to_cidr(netmask) path = "/etc/sysconfig/network" ifcfg_dict = { "IPADDR": f"{ipaddr}/{prefix}", "BOOTPROTO": "static", "STARTMODE": "auto", } self._write_to_file(f"{path}/{filename}", ifcfg_dict) if self.if_type == "Bond": bond_dict = self._get_bondinterface_details() ifcfg_dict["BONDING_MASTER"] = "yes" ifcfg_slave_dict = { "SLAVE": "yes", "ONBOOT": "yes", "MASTER": self.name, } ifcfg_dict["BONDING_MODULE_OPTS"] = "mode=" + bond_dict["mode"][0] for index, slave in enumerate(bond_dict["slaves"]): bonding_slave = f"BONDING_SLAVE{index}" ifcfg_dict[bonding_slave] = slave ifcfg_slave_dict.update({"NAME": slave, "DEVICE": slave}) self._write_to_file(f"{path}/ifcfg-{slave}", ifcfg_slave_dict) if ipaddr not in self.get_ipaddrs(): msg = ( "ipaddr not configured on interface. To avoid " "inconsistency, please add the ipaddr first." ) raise NWException(msg) current_distro = distro_detect() if current_distro.name == "rhel" and int(current_distro.version) >= 9: self.distro_is_rhel9_or_later = True save_distro_rhel9_and_suse16_or_later() elif current_distro.name == "rhel" and int(current_distro.version) < 9: save_distro_rhel8_or_older() elif current_distro.name == "SuSE" and int(current_distro.version) >= 16: self.distro_is_suse16_or_later = True save_distro_rhel9_and_suse16_or_later() elif current_distro.name == "SuSE": save_suse() else: msg = "Distro not supported by API. Could not save ipaddr." raise NWException(msg) def set_mtu(self, mtu, timeout=30): """Sets a new MTU value to this interface. This method will try to set a new MTU value to this interface, if fails it will raise a NWException. Also it will wait until the Interface is up before returning or until timeout be reached. You must have sudo permissions to run this method on a host. :param mtu: mtu size that need to be set. This must be an int. :type mtu: int :param timeout: how many seconds to wait until the interface is up again. Default is 30. :type timeout: int :raises avocado.utils.network.exceptions.NWException: If setting the MTU fails. """ cmd = f"ip link set {self.name} mtu {mtu}" run_command(cmd, self.host, sudo=True) wait_for(self.is_link_up, timeout=timeout) if int(mtu) != self.get_mtu(): raise NWException("Failed to set MTU.") def remove_ipaddr(self, ipaddr, netmask): """Removes an IP address from this interface. This method will try to remove the address from this interface and if fails it will raise a NWException. Be careful, you can lost connection. You must have sudo permissions to run this method on a host. :param ipaddr: The IP address to remove. :type ipaddr: str :param netmask: The netmask of the IP address to remove. :type netmask: str :raises avocado.utils.network.exceptions.NWException: If removing the IP address fails. """ ip = ip_interface(f"{ipaddr}/{netmask}") cmd = f"ip addr del {ip.compressed} dev {self.name}" try: run_command(cmd, self.host, sudo=True) except Exception as ex: msg = f"Failed to remove ipaddr. {ex}" raise NWException(msg) from ex def flush_ipaddr(self): """Flush all the IP address for this interface. This method will try to flush the ip address from this interface and if fails it will raise a NWException. Be careful, you can lost connection. You must have sudo permissions to run this method on a host. :raises avocado.utils.network.exceptions.NWException: If flushing the IP addresses fails. """ cmd = f"ip addr flush dev {self.name}" try: run_command(cmd, self.host, sudo=True) except Exception as ex: msg = f"Failed to flush ipaddr. {ex}" raise NWException(msg) from ex def nm_flush_ipaddr(self): """Remove all the IPv4 address for this interface by using IP tool. This method will explicitly identifies all available IP addresses with netmask of interfaces and deletes them. You must have sudo permissions to run this method on a host. :raises avocado.utils.network.exceptions.NWException: If flushing the IP addresses fails. """ cmd = f"nmcli -g ip4.ADDRESS device show {self.name}" ipaddresses_str = run_command(cmd, self.host, sudo=True).strip() if not ipaddresses_str: return ipaddresses = ipaddresses_str.split(" | ") if ipaddresses: try: for ipaddr in ipaddresses: cmd = f"ip addr delete {ipaddr} dev {self.name}" run_command(cmd, self.host, sudo=True) except Exception as ex: msg = f"Failed to flush ipaddr. {ex}" raise NWException(msg) from ex def remove_link(self): """Deletes virtual interface link. This method will try to delete the virtual device link and the interface will no more be listed with 'ip a' and if fails it will raise a NWException. Be careful, you can lost connection. You must have sudo permissions to run this method on a host. :raises avocado.utils.network.exceptions.NWException: If deleting the link fails. """ cmd = f"ip link del dev {self.name}" try: run_command(cmd, self.host, sudo=True) except Exception as ex: msg = f"Failed to delete link. {ex}" raise NWException(msg) from ex def restore_from_backup(self): """Revert interface file from backup. This method checks if a backup version is available for given interface then it copies backup file to interface file in /sysfs path. :raises avocado.utils.network.exceptions.NWException: If the backup file is not available. """ backup_file = f"{self.config_filename}.backup" if os.path.exists(backup_file): shutil.move(backup_file, self.config_filename) else: raise NWException("Backup file not available, could not restore file.") def is_available(self): """Check if interface is available. This method checks if the interface is available. :return: True if the interface exists, False otherwise. :rtype: bool """ cmd = f"ip link show dev {self.name}" try: run_command(cmd, self.host) return True except Exception as ex: # pylint: disable=W0718 msg = f"Interface {self.name} is not available. {ex}" LOG.debug(msg) return False def is_bond(self): """Check if interface is a bonding device. This method checks if the interface is a bonding device or not. :return: True if the interface is a bonding device, False otherwise. :rtype: bool """ cmd = f"cat /proc/net/bonding/{self.name}" try: run_command(cmd, self.host) return True except Exception as ex: # pylint: disable=W0718 msg = f"{self.name} is not a bond device. {ex}" LOG.debug(msg) return False def is_veth(self): """Check if interface is a Virtual Ethernet. This method checks if the interface is a Virtual Ethernet or not. :return: True if the interface is a veth device, False otherwise. :rtype: bool :raises avocado.utils.network.exceptions.NWException: If the sysfs file for the device does not exist. """ if not os.path.isfile(f"/sys/class/net/{self.name}/device/devspec"): raise NWException("Network interface sysfs file does not exists") cmd = f"cat /sys/class/net/{self.name}/device/devspec" output = run_command(cmd, self.host) if "l-lan" in output: return True return False def is_vnic(self): """Check if interface is a virtual network. This method checks if the interface is a virtual NIC or not. :return: True if the interface is a virtual NIC, False otherwise. :rtype: bool :raises avocado.utils.network.exceptions.NWException: If the sysfs file for the device does not exist. """ if not os.path.isfile(f"/sys/class/net/{self.name}/device/devspec"): raise NWException("Network interface sysfs file does not exists") cmd = f"cat /sys/class/net/{self.name}/device/devspec" output = run_command(cmd, self.host) if "vnic" in output: return True return False def is_sriov(self): """Check if interface is a SRIOV virtual interface. This method checks if the interface is SRIOV logical interface or not. :return: True if the interface is an SRIOV VF, False otherwise. :rtype: bool :raises avocado.utils.network.exceptions.NWException: If the sysfs file for the device does not exist. """ if not os.path.isfile(f"/sys/class/net/{self.name}/device/vpd"): raise NWException("Network interface sysfs file does not exists") cmd = f"cat /sys/class/net/{self.name}/device/vpd" output = run_command(cmd, self.host) for vpd in output.split(): if "VF" in vpd and vpd.endswith("SN"): return True return False def remove_cfg_file(self): """Remove any config files that is created as a part of the test.""" if os.path.isfile(self.config_filename): os.remove(self.config_filename) def restore_slave_cfg_file(self): """Restore or delete slave config files. :raises avocado.utils.network.exceptions.NWException: If the config file cannot be restored. """ if self.if_type != "Bond": return for slave_config in self.slave_config_filename: backup_slave_config = f"{slave_config}.backup" try: if os.path.exists(backup_slave_config): shutil.move(backup_slave_config, slave_config) else: os.remove(slave_config) except Exception as ex: raise NWException(f"Could not restore the config file {ex}") from ex def are_packets_lost(self, peer_ip, options=None, sudo=False): """Check packet loss that occurs during ping. Function returns True for 0% packet loss and False if packet loss occurs. :param peer_ip: Peer IP address (IPv4 or IPv6) :type peer_ip: str :param options: Type is List. Options such as -c, -f. Default is None :type options: list :param sudo: If sudo permissions are needed. Default is False :type sudo: bool :return: True for 0% packet loss, False otherwise. :rtype: bool :raises avocado.utils.network.exceptions.NWException: If the ping command fails. """ cmd = f"ping -I {self.name} {peer_ip}" cmd = f"{cmd} " if options is not None: for elem in options: cmd += f"{elem} " try: output = run_command(cmd, self.host, sudo=sudo) match = re.search(r"(\d+(?:\.\d+)?)% packet loss", output) if match: return float(match.group(1)) > 0 return True except Exception as ex: msg = f"Failed to ping. {ex}" raise NWException(msg) from ex @staticmethod def netmask_to_cidr(netmask): """Function is used to check the netmask value and convert it into short form (mask) of netmask values Example : 255.255.255.0 = 24 255.255.252.0 = 22 :param netmask: Netmask value example 255.255.255.0 :type netmask: str :return: Returns mask value of given netmask :rtype: int """ return sum(bin(int(bits)).count("1") for bits in netmask.split(".")) @staticmethod def validate_ipv4_format(ip): """Validates IPv4 address with following format set. 1. A string in decimal-dot notation, consisting of four decimal integers in the inclusive range 0-255,separated by dots (e.g- 192.168.0.1).Each integer represents an octet in the address. 2. An integer that fits into 32 bits. 3. An integer packed into a bytes object of length 4. And for IP address which are not met above conditions, raises AddressValueError and returns False. :param ip: IP address :type ip: str :return: True when IP address pattern/format matches if not return False :rtype: bool """ try: IPv4Address(ip) except AddressValueError as ex: LOG.debug("Failed to validate IP format %s", ex) return False return True @staticmethod def validate_ipv4_netmask_format(netmask): """Validates IPv4 Netmask address with following format set. 1. A string in decimal-dot notation,consisting of four decimal integers starting from 255 and octets separated by dots (e.g 255.255.255.0) 2. An integer packed into a bytes object of length 4 And for Netmask which are not met above conditions, [ eg : 255.0.255.0, 255.255.255, 255.255.255.256, 255.255.255.255.0] returns False. :param netmask: netmask address :type netmask: str :return: True when netmask address pattern/format matches if not return False :rtype: bool """ netmask_list = netmask.split(".") if len(netmask_list) != 4: return False for octet in netmask_list: num = int(octet) if not 0 <= num <= 255: return False octet_bin = [format(int(i), "08b") for i in netmask_list] binary_netmask = ("").join(octet_bin) accept_zero_only = False first_bit = True for symbol in binary_netmask: if accept_zero_only and symbol == "1": return False if symbol == "0": accept_zero_only = True if first_bit and symbol == "0": return False first_bit = False return True @staticmethod def ping_flood(int_name, peer_ip, ping_count): """Start ping to remote machine with "-f" [ flood ] option, on given interface. Also this function enables to track the live data to determine the ping flood failure, in case of failure the program will exit. :param int_name: source interface name. :type int_name: str :param peer_ip: Peer IP address (IPv4 or IPv6) :type peer_ip: str :param ping_count: How many ICMP echo packets to send. :type ping_count: int :return: returns True on successful ping flood. returns False on ping flood failure. :rtype: bool """ cmd = f"ping -I {int_name} {peer_ip} -c {ping_count} -f " with subprocess.Popen( cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, universal_newlines=True, ) as ping_process: pattern = r"\.{10}" while True: char = ping_process.stdout.read(100) match = re.search(pattern, char) if match: ping_process.terminate() msg = "ping flood failed to remote machine, Please check the logs" LOG.debug(msg) return False return True ping_process.stdout.close() ping_process.wait() def get_device_IPI_name(self): # pylint: disable=C0103 """Convert IO device name to device_ipi names according to "/proc/interrupts" context. Ex: vnic@30000009 to vnic-30000009 :return: A converted Network device according to device_ipi name. :rtype: str or None """ if self.is_vnic(): cmd = ( f"cat /sys/class/net/{self.name}/device/devspec | " f"awk -F/ '{{print $3}}'" ) interface_type = process.system_output( cmd, shell=True, ignore_status=True ).decode("utf-8") cmd = f"echo {interface_type} | sed 's/@/-/' " interface_type = process.system_output( cmd, shell=True, ignore_status=True ).decode("utf-8") return interface_type if self.is_veth(): return self.name return None avocado-113.0/avocado/utils/network/ports.py000066400000000000000000000165351513561306000211320ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2013-2014 # Author: Lucas Meneghel Rodrigues """Module with network related utility functions.""" import random import socket import warnings from avocado.utils.data_structures import Borg #: Families taken into account in this class FAMILIES = (socket.AF_INET, socket.AF_INET6) #: Protocols taken into account in this class PROTOCOLS = (socket.SOCK_STREAM, socket.SOCK_DGRAM) def is_port_available( port, address, family=socket.AF_INET, protocol=socket.SOCK_STREAM ): """Return True if the given port is available for use. :param port: Port value to check. :type port: int :param address: Address to use this port. :type address: str :param family: Default is socket.AF_INET. Accepted values are: socket.AF_INET or socket.AF_INET6. :type family: socket.AddressFamily.AF_* :param protocol: Protocol type. Default is socket.SOCK_STREAM (TCP). Accepted values are: socket.SOCK_STREAM or socket.SOCK_DGRAM. :type protocol: socket.AddressFamily.SOCK_* :return: True if the port is available, False otherwise. :rtype: bool """ try: with socket.socket(family, protocol) as sock: sock.bind((address, port)) except PermissionError: # Permission denied, can't be sure return False except OSError: # Address already in use or cannot assign requested address return False return True def is_port_free(port, address): """This method is deprecated. Please use is_port_available(). :param port: Port value to check. :type port: int :param address: Address to use this port. :type address: str :return: True if the port is available, False otherwise. :rtype: bool """ warnings.warn("deprecated, use is_port_available() instead.", DeprecationWarning) return is_port_available(port, address) # pylint: disable=R0913 def find_free_port( start_port=1024, end_port=65535, address="localhost", sequent=False, family=socket.AF_INET, protocol=socket.SOCK_STREAM, ): """Return a host free port in the range [start_port, end_port]. :param start_port: header of candidate port range, defaults to 1024 :type start_port: int :param end_port: ender of candidate port range, defaults to 65535 :type end_port: int :param address: Socket address to bind or connect :type address: str :param sequent: Find port sequentially, random order if it's False :type sequent: bool :param family: Default is socket.AF_INET. Accepted values are: socket.AF_INET or socket.AF_INET6. :type family: socket.AddressFamily.AF_* :param protocol: Protocol type. Default is socket.SOCK_STREAM (TCP). Accepted values are: socket.SOCK_STREAM or socket.SOCK_DGRAM. :type protocol: socket.AddressFamily.SOCK_* :return: A free port number, or None if no free port is found. :rtype: int or None if no free port found """ ports = find_free_ports(start_port, end_port, 1, address, sequent, family, protocol) if ports: return ports[0] return None # pylint: disable=R0913 def find_free_ports( start_port, end_port, count, address="localhost", sequent=False, family=socket.AF_INET, protocol=socket.SOCK_STREAM, ): """Return a number of host free ports in the range [start_port, end_port]. :param start_port: header of candidate port range :type start_port: int :param end_port: ender of candidate port range :type end_port: int :param count: Initial number of ports known to be free in the range. :type count: int :param address: Socket address to bind or connect :type address: str :param sequent: Find port sequentially, random order if it's False :type sequent: bool :param family: Default is socket.AF_INET. Accepted values are: socket.AF_INET or socket.AF_INET6. :type family: socket.AddressFamily.AF_* :param protocol: Protocol type. Default is socket.SOCK_STREAM (TCP). Accepted values are: socket.SOCK_STREAM or socket.SOCK_DGRAM. :type protocol: socket.AddressFamily.SOCK_* :return: A list of free port numbers. :rtype: list[int] """ ports = [] port_range = list(range(start_port, end_port)) if not sequent: random.shuffle(port_range) for i in port_range: if is_port_available(i, address, family, protocol): ports.append(i) if len(ports) >= count: break return ports class PortTracker(Borg): """Tracks ports used in the host machine.""" def __init__(self): """Initializes the PortTracker instance.""" Borg.__init__(self) self.address = "localhost" self.start_port = 5000 if not hasattr(self, "retained_ports"): self._reset_retained_ports() def __str__(self): """Returns a string representation of the tracked ports. :return: A string showing the list of retained ports. :rtype: str """ return f"Ports tracked: {self.retained_ports!r}" def _reset_retained_ports(self): """Resets the list of retained ports to an empty list.""" self.retained_ports = [] def register_port(self, port): """Registers a port as being in use. :param port: The port number to register. :type port: int :return: The registered port number if successful. :rtype: int :raises ValueError: If the port is already in use or cannot be registered. """ if (port not in self.retained_ports) and is_port_free(port, self.address): self.retained_ports.append(port) else: raise ValueError(f"Port {int(port)} in use") return port def find_free_port(self, start_port=None): """Finds and registers a free port. It starts searching from `start_port` if provided, otherwise it uses the default `self.start_port`. :param start_port: The port number to start searching from. :type start_port: int or None :return: The first free port number found. :rtype: int """ if start_port is None: start_port = self.start_port port = start_port while (port in self.retained_ports) or (not is_port_free(port, self.address)): port += 1 self.retained_ports.append(port) return port def release_port(self, port): """Releases a previously registered port. :param port: The port number to release. :type port: int """ if port in self.retained_ports: self.retained_ports.remove(port) # pylint: disable=wrong-import-position from avocado.utils.deprecation import log_deprecation log_deprecation.warning("network.ports") avocado-113.0/avocado/utils/nvme.py000066400000000000000000000423571513561306000172400ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # This code was inspired in the autotest project, # # client/base_utils.py # # Copyright: 2022 IBM # Authors : Naresh Bannoth """ Nvme utilities """ import json import logging import os import re import time from avocado.utils import pci, process LOGGER = logging.getLogger(__name__) class NvmeException(Exception): """ Base Exception Class for all exceptions """ def get_controller_name(pci_addr): """ Returns the controller/Adapter name with the help of pci_address :param pci_addr: pci_address of the adapter :rtype: string :raises: :py:class:`NvmeException` on failure to find pci_address in OS """ if pci_addr in pci.get_pci_addresses(): path = f"/sys/bus/pci/devices/{pci_addr}/nvme/" return "".join(os.listdir(path)) raise NvmeException("Unable to list as wrong pci_addr") def get_max_ns_supported(controller_name): """ Returns the number of namespaces supported for the nvme adapter :param controller_name: Name of the controller eg: nvme0 :rtype: integer """ cmd = f"nvme id-ctrl /dev/{controller_name}" out = process.run(cmd, ignore_status=True, sudo=True, shell=True).stdout_text for line in out.splitlines(): if line.split(":")[0].strip() == "nn": return int(line.split(":")[-1].strip()) return "" def get_total_capacity(controller_name): """ Returns the total capacity of the nvme adapter :param controller_name: Name of the controller eg: nvme0 :rtype: integer """ cmd = f"nvme id-ctrl /dev/{controller_name}" out = process.run(cmd, ignore_status=True, sudo=True, shell=True).stdout_text for line in out.splitlines(): if line.split(":")[0].strip() == "tnvmcap": return int(line.split(":")[-1].strip()) return "" def get_controller_id(controll_name): """ Returns the nvme controller id :param controller_name: Name of the controller eg: nvme0 :rtype: string """ cmd = f"nvme id-ctrl /dev/{controll_name}" output = process.run(cmd, shell=True, sudo=True, ignore_status=True).stdout_text for line in output.splitlines(): if "cntlid" in line: return line.split(":")[-1].strip() return "" def get_current_ns_ids(controller_name): """ Returns the list of namespaces in the nvme controller :param controller_name: Name of the nvme controller like nvme0, nvme1 :rtype: list """ cmd = f"nvme list-ns /dev/{controller_name}" namespaces = [] output = process.run(cmd, shell=True, sudo=True, ignore_status=True).stdout_text for line in output.splitlines(): if line.startswith("["): namespaces.append(int(line.split()[1].split("]")[0]) + 1) return namespaces def get_current_ns_list(controller_name, shared_ns=False): """ Returns the list of namespaces in the nvme controller :param controller_name: Name of the nvme controller like nvme0, nvme1 :rtype: list """ namespace_list = [] namespaces_ids = get_current_ns_ids(controller_name) if shared_ns: subsys = get_subsystem_using_ctrl_name(controller_name) controller_name = f"nvme{subsys[len('nvme-subsy'):]}" for ns_id in namespaces_ids: namespace_list.append(f"/dev/{controller_name}n{ns_id}") return namespace_list def get_block_size(controller_name, shared_ns=False): """ Returns the block size of the namespace. If not found, return defaults to 4k. :param namespace: Name of the namespace like /dev/nvme0n1 etc.. :rtype: Integer """ namespaces = get_current_ns_list(controller_name, shared_ns=shared_ns) if namespaces: namespace = get_namespace_absolute_path(namespaces[0]) if shared_ns: subsys = get_subsystem_using_ctrl_name(controller_name) controller_name = f"nvme{subsys[len('nvme-subsy'):]}" ns_match = re.search(r"nvme\d+n(\d+)", namespace) if ns_match: namespace = f"{controller_name}n{ns_match.group(1)}" namespace = get_namespace_absolute_path(namespace) cmd = f"nvme id-ns {namespace}" out = process.run(cmd, shell=True, ignore_status=True).stdout_text for line in out.splitlines(): if "in use" in line: return pow(2, int(line.split()[4].split(":")[-1])) return 4096 def get_namespace_absolute_path(namespace): """ Returns absolute path for nvme namespace :rtype: String """ if "dev" not in namespace: return f"/dev/{namespace}" return namespace def delete_ns(controller_name, ns_id, shared_ns=False): """ Deletes the specified namespace on the controller :param controller_name: Nvme controller name to which namespace belongs :param ns_id: namespace id to be deleted """ cont_id = get_controller_id(controller_name) if shared_ns: ctrls = get_alternate_controller_name(controller_name) for ctrl in ctrls: cont_id = f"{cont_id},{get_controller_id(ctrl)}" detach_ns(controller_name, ns_id, cont_id) cmd = f"nvme delete-ns /dev/{controller_name} -n {ns_id}" if process.system(cmd, shell=True, ignore_status=True): raise NvmeException(f"/dev/{controller_name}n{ns_id} delete failed") if is_ns_exists(controller_name, ns_id): raise NvmeException("namespace still listed even after deleted") def delete_all_ns(controller_name, shared_ns=False): """ Deletes all the name spaces available on the given nvme controller :param controller_name: Nvme controller name eg : nvme0, nvme1 etc.. """ namespaces_ids = get_current_ns_ids(controller_name) for ns_id in namespaces_ids[::-1]: delete_ns(controller_name, ns_id, shared_ns=shared_ns) time.sleep(5) def is_ns_exists(controller_name, ns_id): """ Returns if that particular namespace exists on the controller or not :param controller_name: name of the controller on which we want to check ns existence :returns: True if exists else False :rtype: boolean """ ns_list = get_current_ns_ids(controller_name) if ns_id in ns_list: return True for ctrl in get_alternate_controller_name(controller_name): ns_list = get_current_ns_ids(ctrl) if ns_id in ns_list: return True return False def get_lba(namespace, shared_ns=False): """ Returns LBA of the namespace. If not found, return defaults to 0. :param namespace: nvme namespace like /dev/nvme0n1, /dev/nvme0n2 etc.. :rtype: Integer """ if namespace: if shared_ns: ns_match = re.search(r"(/dev/)?(nvme\d+)n\d+", namespace) if ns_match: ctrl_name = ns_match.group(2) subsys = get_subsystem_using_ctrl_name(ctrl_name) controller_name = f"nvme{subsys[len('nvme-subsy'):]}" ns_id_match = re.search(r"nvme\d+n(\d+)", namespace) if ns_id_match: namespace = f"{controller_name}n{ns_id_match.group(1)}" namespace = get_namespace_absolute_path(namespace) cmd = f"nvme id-ns {namespace}" out = process.run(cmd, shell=True, ignore_status=True).stdout_text for line in out.splitlines(): if "in use" in line: return int(line.split()[1]) return 0 def ns_rescan(controller_name): """ re-scans all the names spaces on the given controller :param controller_name: controller name on which re-scan is applied """ cmd = f"nvme ns-rescan /dev/{controller_name}" try: process.run(cmd, shell=True, ignore_status=True) except process.CmdError as detail: LOGGER.debug(detail) def detach_ns(controller_name, ns_id, cont_id): """ detach the namespace_id to specified controller :param ns_id: namespace ID :param controller_name: controller name :param cont_id: controller_ID """ cmd = f"nvme detach-ns /dev/{controller_name} --namespace-id={ns_id} --controllers={cont_id}" if not process.run(cmd, shell=True, ignore_status=True): raise NvmeException("detach command failed") ns_rescan(controller_name) time.sleep(5) if is_ns_exists(controller_name, ns_id): raise NvmeException("namespace dettached but still listing") def attach_ns(ns_id, controller_name, cont_id): """ attach the namespace_id to specified controller :param ns_id: namespace ID :param controller_name: controller name :param cont_id: controller_ID """ cmd = f"nvme attach-ns /dev/{controller_name} --namespace-id={ns_id} -controllers={cont_id}" if not process.run(cmd, shell=True, ignore_status=True): raise NvmeException("namespaces attach command failed") ns_rescan(controller_name) if not is_ns_exists(controller_name, ns_id): raise NvmeException("namespaces attached but not listing") def create_full_capacity_ns(controller_name, shared_ns=False): """ Creates one namespace with full capacity :param controller_name: name of the controller like nvme0/nvme1 etc.. """ ns_size = get_total_capacity(controller_name) // get_block_size(controller_name) if get_current_ns_list(controller_name, shared_ns=shared_ns): raise NvmeException("ns already exist, delete it before creating ") create_one_ns("1", controller_name, ns_size, shared_ns=shared_ns) def create_one_ns(ns_id, controller_name, ns_size, shared_ns=False): """ creates a single namespaces with given size and controller_id :param ns_id: Namespace ID :param controller_name: name of the controller like nvme0/nvme1 etc.. :param ns_size: Size of the namespace that is going to be created """ cmd = f"nvme create-ns /dev/{controller_name} --nsze={ns_size} --ncap={ns_size} --flbas=0 --dps=0" if shared_ns: cmd = f"{cmd} -m 1" if process.system(cmd, shell=True, ignore_status=True): raise NvmeException(f"namespace create command failed {cmd}") cont_id = get_controller_id(controller_name) if shared_ns: ctrls = get_alternate_controller_name(controller_name) for ctrl in ctrls: cont_id = f"{cont_id},{get_controller_id(ctrl)}" attach_ns(ns_id, controller_name, cont_id) def create_max_ns(controller_name, force, shared_ns=False): """ Creates maximum number of namespaces, with equal capacity :param controller_name: name of the controller like nvme0/nvme1 etc.. :param force: if wants to create the namespace force, then pass force=True """ if get_current_ns_list(controller_name, shared_ns=shared_ns) and not force: raise NvmeException("ns already exist, cannot create max_ns") max_ns = int(get_max_ns_supported(controller_name)) ns_size = get_equal_ns_size(controller_name, max_ns) for ns_id in range(1, (max_ns + 1)): create_one_ns(str(ns_id), controller_name, ns_size) def get_equal_ns_size(controller_name, ns_count): """ It calculate and return the size of a namespace when want to create more than one namespace with equal sizes :param controller_name: name of the controller like nvme0/nvme1 etc... :param ns_count: Number of namespaces you want to create with equal sizes it should be less than or equal to max ns supported on the controller :rtype: integer """ existing_ns_list = len(get_current_ns_ids(controller_name)) max_ns = get_max_ns_supported(controller_name) if ns_count > (max_ns - existing_ns_list): raise NvmeException("required ns count is greater than max supported") free_space = get_free_space(controller_name) if free_space < 1000: raise NvmeException("available free space is less than 1GB") return int(((60 * (free_space // 4096)) // 100) // ns_count) def get_free_space(controller_name): """ Returns the total capacity of the nvme adapter :param controller_name: Name of the controller eg: nvme0 :rtype: integer """ cmd = f"nvme id-ctrl /dev/{controller_name}" out = process.run(cmd, ignore_status=True, sudo=True, shell=True).stdout_text for line in out.splitlines(): if line.split(":")[0].strip() == "unvmcap": return int(line.split(":")[-1].strip()) return 0 def create_namespaces(controller_name, ns_count, shared_ns=False): """ creates equal n number of namespaces on the specified controller :param controller_name: name of the controller like nvme0 :param ns_count: number of namespaces to be created """ namespaces = get_current_ns_ids(controller_name) if namespaces: delete_all_ns(controller_name) blk_size = get_total_capacity(controller_name) // get_block_size(controller_name) ns_size = blk_size // (ns_count + 1) for ns_id in range(1, ns_count + 1): create_one_ns(ns_id, controller_name, ns_size, shared_ns=shared_ns) def get_ns_status(controller_name, ns_id): """ Returns the status of namespaces on the specified controller :param controller_name: name of the controller like nvme0 :param ns_id: ID of namespace for which we need the status :rtype: list """ stat = [] cmd = f"nvme show-topology /dev/{controller_name} -o json" data = process.run(cmd, ignore_status=True, sudo=True, shell=True).stdout_text json_data = json.loads(data) for data in json_data: for subsystem in data["Subsystems"]: for namespace in subsystem["Namespaces"]: nsid = namespace["NSID"] for paths in namespace["Paths"]: if nsid == ns_id and paths["Name"] == controller_name: stat.extend([paths["State"], paths["ANAState"]]) return stat def get_nslist_with_pci(pci_address): """ Fetches and returns list of namespaces for specified pci_address :param pci_address: pci_address of any nvme adapter :rtype: list """ ns_list = [] cmd = "nvme show-topology -o json" data = process.run(cmd, ignore_status=True, sudo=True, shell=True).stdout_text json_data = json.loads(data) for data in json_data: for subsystem in data["Subsystems"]: for namespace in subsystem["Namespaces"]: for paths in namespace["Paths"]: if paths["Address"] == pci_address: ns_list.append(namespace["NSID"]) return ns_list def get_nvme_subsystem(): """ Fetches subsystem data and returns dictionary of all subsystems :rtype: dict """ cmd = "nvme list-subsys -o json" data = process.run(cmd, ignore_status=True, sudo=True, shell=True).stdout_text json_data = json.loads(data) subsystems_dict = {} for host in json_data: for subsystem in host.get("Subsystems", []): nqn = subsystem.get("NQN") if nqn: subsystem_data = { "Name": subsystem.get("Name"), "IOPolicy": subsystem.get("IOPolicy"), "Type": subsystem.get("Type"), "Paths": subsystem.get("Paths", []), } subsystems_dict[nqn] = subsystem_data return subsystems_dict def get_controllers_with_nqn(nqn): """ Fetches controllers from subsystem based on input Non-Volatile Memory Express Qualified Name :rtype: list """ subsys_dict = get_nvme_subsystem().get(nqn) if not subsys_dict: return "" return [path["Name"] for path in subsys_dict["Paths"]] def get_subsys_name_with_nqn(nqn): """ Fetches subsystem name based on input Non-Volatile Memory Express Qualified Name :rtype: string """ subsys_dict = get_nvme_subsystem().get(nqn) if not subsys_dict: return "" return subsys_dict["Name"] def get_controllers_with_subsys(subsys): """ Fetches controllers from nvme subsystem with input as subsystem name :rtype: list """ subsys_dict = get_nvme_subsystem() subsys_arr = [ sub_sys_val for sub_sys, sub_sys_val in subsys_dict.items() if sub_sys_val.get("Name") == subsys ] if not subsys_arr: return [] return [path["Name"] for path in subsys_arr[0]["Paths"]] def get_alternate_controller_name(ctrl): """ Fetches other controller in a subsystem based on input controller :rtype: list """ subsys_dict = get_nvme_subsystem() for device_nqn in subsys_dict: ctrls = get_controllers_with_nqn(device_nqn) if ctrl in ctrls: return ctrls.remove(ctrl) or ctrls return [] def get_subsystem_using_ctrl_name(ctrl): """ Fetches subsystem name with controller name as input :rtype: string """ subsys_dict = get_nvme_subsystem() for device_nqn in subsys_dict: ctrls = get_controllers_with_nqn(device_nqn) if ctrl in ctrls: return get_subsys_name_with_nqn(device_nqn) return "" avocado-113.0/avocado/utils/output.py000066400000000000000000000125131513561306000176220ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # This code was inspired in the autotest project, # # client/shared/utils.py # Original author: Cleber Rosa # # Copyright: Red Hat Inc. 2015 # Author: Lucas Meneghel Rodrigues """Utility functions for user friendly display of information.""" import sys def display_data_size(size): """Display data size in human readable units (SI). :param size: Data size, in Bytes. :type size: int :return: Human readable string with data size, using SI prefixes. :rtype: str """ prefixes = ["B", "KB", "MB", "GB", "TB", "PB"] factor = float(1000) i = 0 while size >= factor: if i >= len(prefixes) - 1: break size /= factor i += 1 return f"{size:.2f} {prefixes[i]}" class ProgressBar: """Displays interactively the progress of a given task. Inspired/adapted from https://gist.github.com/t0xicCode/3306295 """ def __init__(self, minimum=0, maximum=100, width=75, title=""): """Initializes a new progress bar. :param minimum: Minimum (initial) value on the progress bar :type minimum: int :param maximum: Maximum (final) value on the progress bar :type maximum: int :param width: Number of columns, that is screen width :type width: int :param title: Optional title to display with the progress bar :type title: str :raises AssertionError: When maximum is not greater than minimum """ assert maximum > minimum self.prog_bar = "" self.old_prog_bar = "" if title: width -= len(title) self.minimum = minimum self.maximum = maximum self.width = width self.title = title self.current_amount = minimum self.update_amount(minimum) def append_amount(self, amount): """Increments the current amount value by the specified amount. :param amount: The value to add to the current amount :type amount: int or float """ self.update_amount(self.current_amount + amount) def update_percentage(self, percentage): """Updates the progress bar to the specified percentage value. :param percentage: The percentage value to set (0-100) :type percentage: int or float """ self.update_amount((percentage * float(self.maximum)) / 100.0) def update_amount(self, amount): """Performs sanity checks and updates the current amount value. The amount is clamped between the minimum and maximum values set during initialization. After updating the amount, the progress bar is refreshed and redrawn. :param amount: The new amount value to set :type amount: int or float """ if amount < self.minimum: amount = self.minimum if amount > self.maximum: amount = self.maximum self.current_amount = amount self._update_progress_bar() self.draw() def _update_progress_bar(self): """Builds the actual progress bar text representation. This internal method calculates the percentage completion, creates the visual bar with '=' characters and '>' indicator, and formats the display string including the percentage and optional title. """ diff = float(self.current_amount - self.minimum) done = (diff / float(self.maximum - self.minimum)) * 100.0 done = int(round(done)) all_full = self.width - 2 hashes = (done / 100.0) * all_full hashes = int(round(hashes)) if not hashes: screen_text = f"[>{' ' * (all_full - 1)}]" elif hashes == all_full: screen_text = f"[{'=' * all_full}]" else: screen_text = f"[{'=' * (hashes - 1)}>{' ' * (all_full - hashes)}]" percent_string = str(done) + "%" # slice the percentage into the bar screen_text = " ".join([screen_text, percent_string]) if self.title: screen_text = f"{self.title}: {screen_text}" self.prog_bar = screen_text def draw(self): """Prints the updated progress bar text to the screen. Only prints when the progress bar has changed from the previous state to avoid unnecessary screen updates. Uses carriage return to overwrite the previous progress bar on the same line. """ if self.prog_bar != self.old_prog_bar: self.old_prog_bar = self.prog_bar sys.stdout.write("\r" + self.prog_bar) sys.stdout.flush() def __str__(self): """Returns the current progress bar as a string. :return: The formatted progress bar string :rtype: str """ return str(self.prog_bar) # pylint: disable=wrong-import-position from avocado.utils.deprecation import log_deprecation log_deprecation.warning("output") avocado-113.0/avocado/utils/partition.py000066400000000000000000000276031513561306000203010ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: 2016 IBM. # Author: Rajashree Rajendran # Copyright: 2016 Red Hat, Inc. # Author: Lukas Doktor # # Based on code by: Martin Bligh (mbligh@google.com) # Copyright: Google 2006-2008 # https://github.com/autotest/autotest/blob/master/client/partition.py """ Utility for handling partitions. """ import hashlib import logging import os import tempfile from avocado.utils import filelock, process LOG = logging.getLogger(__name__) class PartitionError(Exception): """ Generic PartitionError """ def __init__(self, partition, reason, details=None): msg = reason + ": " + str(details) if details else reason super().__init__(msg) self.partition = partition def __str__(self): return f"Partition({self.partition.device}): {super().__str__()}" class MtabLock: device = "/etc/mtab" def __init__(self, timeout=60): self.timeout = timeout self.mtab = None device_hash = hashlib.sha1(self.device.encode("utf-8")).hexdigest() lock_filename = os.path.join(tempfile.gettempdir(), device_hash) self.lock = filelock.FileLock(lock_filename, timeout=self.timeout) def __enter__(self): try: self.lock.__enter__() except (filelock.LockFailed, filelock.AlreadyLocked) as e: reason = ( f"Unable to obtain '{self.device}' " f"lock in {int(self.timeout)}s" ) raise PartitionError(self, reason, e) from e self.mtab = open(self.device) # pylint: disable=W1514 return self def __exit__(self, exc_type, exc_value, exc_traceback): if self.mtab: self.mtab.close() self.lock.__exit__(exc_type, exc_value, exc_traceback) class Partition: """ Class for handling partitions and filesystems """ # pylint: disable=R0913 def __init__( self, device, loop_size=0, mountpoint=None, mkfs_flags="", mount_options=None ): """ :param device: The device in question (e.g."/dev/hda2"). If device is a file it will be mounted as loopback. :param loop_size: Size of loopback device (in MB). Defaults to 0. :param mountpoint: Where the partition to be mounted to. :param mkfs_flags: Optional flags for mkfs :param mount_options: Add mount options optionally """ self.device = device self.loop = loop_size self.fstype = None self.mountpoint = mountpoint self.mkfs_flags = mkfs_flags self.mount_options = mount_options if self.loop: process.run(f"dd if=/dev/zero of={device} bs=1M " f"count={int(self.loop)}") def __repr__(self): return f"" @staticmethod def list_mount_devices(): """ Lists mounted file systems and swap on devices. """ # list mounted file systems devices = [line.split()[0] for line in process.getoutput("mount").splitlines()] # list mounted swap devices swaps = process.getoutput("swapon -s").splitlines() devices.extend([line.split()[0] for line in swaps if line.startswith("/")]) return devices @staticmethod def list_mount_points(): """ Lists the mount points. """ return [line.split()[2] for line in process.getoutput("mount").splitlines()] def get_mountpoint(self, filename=None): """ Find the mount point of this partition object. :param filename: where to look for the mounted partitions information (default None which means it will search /proc/mounts and/or /etc/mtab) :return: a string with the mount point of the partition or None if not mounted """ # Try to match this device/mountpoint if filename: with open(filename) as open_file: # pylint: disable=W1514 for line in open_file: parts = line.split() if parts[0] == self.device or os.path.realpath( parts[0] ) == os.path.realpath(self.device): if parts[1] == self.mountpoint: return parts[1] # The mountpoint where it's mounted return None # no specific file given, look in /proc/mounts res = self.get_mountpoint(filename="/proc/mounts") if not res: # sometimes the root partition is reported as /dev/root in # /proc/mounts in this case, try /etc/mtab res = self.get_mountpoint(filename="/etc/mtab") if res != "/": res = None return res def mkfs(self, fstype=None, args=""): """ Format a partition to filesystem type :param fstype: the filesystem type, such as "ext3", "ext2". Defaults to previously set type or "ext2" if none has set. :param args: arguments to be passed to mkfs command. """ if self.device in self.list_mount_devices(): raise PartitionError(self, "Unable to format mounted device") if not fstype: if self.fstype: fstype = self.fstype else: fstype = "ext2" if self.mkfs_flags: args += " " + self.mkfs_flags if fstype in ["xfs", "btrfs"]: args += " -f" if self.loop: if fstype.startswith("ext"): args += " -F" elif fstype == "reiserfs": args += " -f" # If there isn't already a '-t ' argument, add one. if "-t" not in args: args = f"-t {fstype} {args}" args = args.strip() mkfs_cmd = f"mkfs {args} {self.device}" try: process.system_output(f"yes | {mkfs_cmd}", shell=True) except process.CmdError as error: raise PartitionError(self, "Failed to mkfs", error) from error self.fstype = fstype def mount(self, mountpoint=None, fstype=None, args="", mnt_check=True): """ Mount this partition to a mount point :param mountpoint: If you have not provided a mountpoint to partition object or want to use a different one, you may specify it here. :param fstype: Filesystem type. If not provided partition object value will be used. :param args: Arguments to be passed to "mount" command. :param mnt_check: Flag to check/avoid checking existing device/mountpoint """ if not mountpoint: mountpoint = self.mountpoint if not mountpoint: raise PartitionError( self, "No mountpoint specified and no " "default provided to this partition object", ) if fstype is None: fstype = self.fstype else: self.fstype = fstype if self.mount_options: args += " -o " + self.mount_options if fstype: args += " -t " + fstype args = args.lstrip() with MtabLock(): if mnt_check: if self.device in self.list_mount_devices(): raise PartitionError(self, "Attempted to mount mounted device") if mountpoint in self.list_mount_points(): raise PartitionError(self, "Attempted to mount busy directory") if not os.path.isdir(mountpoint): os.makedirs(mountpoint) try: if self.loop: losetup_cmd = f"losetup --find --show -f {self.device}" self.device = process.run( losetup_cmd, sudo=True ).stdout_text.strip() process.system(f"mount {args} {self.device} {mountpoint}", sudo=True) except process.CmdError as details: raise PartitionError(self, "Mount failed", details) from details # Update the fstype as the mount command passed self.fstype = fstype def _get_pids_on_mountpoint(self, mnt): """ Returns a list of processes using a given mountpoint """ try: cmd = "lsof " + mnt out = process.system_output(cmd, sudo=True) return [int(line.split()[1]) for line in out.splitlines()[1:]] except OSError as details: msg = f'Could not run lsof to identify processes using "{mnt}"' LOG.error(msg) raise PartitionError(self, msg, details) from details except process.CmdError as details: msg = f'Failure executing "{cmd}"' LOG.error(msg) raise PartitionError(self, msg, details) from details def _unmount_force(self, mountpoint): """ Kill all other jobs accessing this partition and force unmount it. :return: None :raise PartitionError: On critical failure """ for pid in self._get_pids_on_mountpoint(mountpoint): try: process.system(f"kill -9 {int(pid)}", ignore_status=True, sudo=True) except process.CmdError as kill_details: raise PartitionError( self, "Failed to kill processes", kill_details ) from kill_details # Unmount try: process.run(f"umount -f {mountpoint}", sudo=True) except process.CmdError: try: process.run(f"umount -l {mountpoint}", sudo=True) except process.CmdError as umount_details: raise PartitionError( self, "Force unmount failed", umount_details ) from umount_details def unmount(self, force=True, mountpoint=None): """ Umount this partition. It's easier said than done to umount a partition. We need to lock the mtab file to make sure we don't have any locking problems if we are umounting in parallel. When the unmount fails and force==True we unmount the partition ungracefully. :param force: Whether to force unmount if the standard unmount fails. :param mountpoint: Optional mountpoint to unmount. If not provided, the current mountpoint of the partition object will be used. :return: 1 on success, 2 on force umount success :raise PartitionError: On failure """ with MtabLock(): if not mountpoint: mountpoint = self.get_mountpoint() result = 1 if not mountpoint: LOG.debug("%s not mounted", self.device) return result try: process.run("umount " + mountpoint, sudo=True) result = 1 except process.CmdError as details: if force: LOG.debug("Standard umount failed on %s, forcing", mountpoint) self._unmount_force(mountpoint) result = 2 else: raise PartitionError( self, "Unable to unmount gracefully", details ) from details if self.loop: try: process.run(f"losetup -d {self.device}", sudo=True) except process.CmdError as details: raise PartitionError( self, "Unable to cleanup loop device", details ) from details return result avocado-113.0/avocado/utils/path.py000066400000000000000000000231161513561306000172170ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2013-2014 # Author: Yiqiao Pu __doc__ = """ Module to handle file and directory paths. It provides functions to manipulate file and directory paths, check permissions, and inspect files. """ import os import stat import tempfile import urllib #: The indicator for a script file, usually the first line of the file. SHEBANG = "#!" class CmdNotFoundError(Exception): """Indicates that the command was not found in the system after a search. :param cmd: String with the command. :param paths: List of paths where we looked after. """ def __init__(self, cmd, paths): # pylint: disable=W0231 super() self.cmd = cmd self.paths = paths def __str__(self): """String representation of the exception. :return: A string describing the missing command and the paths searched. :rtype: str """ return ( f"Command '{self.cmd}' could not be found in any " f"of the PATH dirs: {self.paths}" ) def get_path(base_path, user_path): """Translate a user specified path to a real path. If user_path is relative, append it to base_path. If user_path is absolute, return it as is. :param base_path: The base path of relative user specified paths. :param type base_path: str :param user_path: The user specified path. :type user_path: str :return: The resolved path. :rtype: str """ if os.path.isabs(user_path) or urllib.parse.urlparse(user_path)[0] in [ "http", "https", "ftp", "file", ]: return user_path return os.path.join(base_path, user_path) def init_dir(*args): """Wrapper around os.path.join that creates dirs based on the final path. :param args: List of dir arguments that will be os.path.joined. :return: directory. :rtype: str """ directory = os.path.join(*args) if not os.path.isdir(directory): os.makedirs(directory, exist_ok=True) return directory def find_command(cmd, default=None, check_exec=True): """Try to find a command in the PATH, paranoid version. :param cmd: Command to be found. :type cmd: str :param default: Command path to use as a fallback if not found in the standard directories. :type default: str or None :param check_exec: if a check for permissions that render the command executable by the current user should be performed. :type check_exec: bool :raise avocado.utils.path.CmdNotFoundError: in case the command was not found and no default was given. :return: Returns an absolute path to the command or the default value if the command is not found :rtype: str """ common_bin_paths = [ "/usr/libexec", "/usr/local/sbin", "/usr/local/bin", "/usr/sbin", "/usr/bin", "/sbin", "/bin", ] try: path_paths = os.environ["PATH"].split(":") except KeyError: path_paths = [] path_paths = list(set(common_bin_paths + path_paths)) for dir_path in path_paths: cmd_path = os.path.join(dir_path, cmd) if os.path.isfile(cmd_path): if check_exec: if not os.access(cmd_path, os.R_OK | os.X_OK): continue return os.path.abspath(cmd_path) if default is not None: return default path_paths.sort() raise CmdNotFoundError(cmd, path_paths) class PathInspector: """Inspects paths to provide information about them. :param path: The path to inspect. :type path: str """ def __init__(self, path): self.path = path def get_first_line(self): """Reads and returns the first line of the file from path. :return: The first line of the file or an empty string if the file does not exist or is empty. :rtype: str """ first_line = "" if os.path.isfile(self.path): with open(self.path, "r", encoding="utf-8") as open_file: first_line = open_file.readline() return first_line def has_exec_permission(self): """Checks if the file from path has execute permissions for the user. :return: True if the file has execute permissions, False otherwise. :rtype: bool """ if os.path.exists(self.path): mode = os.stat(self.path)[stat.ST_MODE] return mode & stat.S_IXUSR return False def is_empty(self): """Checks if the file in path is empty. :return: True if the file is empty, False otherwise. :rtype: bool """ if os.path.exists(self.path): size = os.stat(self.path)[stat.ST_SIZE] return not size return False def is_script(self, language=None): """Checks if the file in the path is a script, optionally checking for a specific language. :param language: The scripting language to check for (e.g., "python"). If None, checks for any shebang. :type language: str, optional :return: True if the file is a script (and matches the language, if provided), False otherwise. :rtype: bool """ first_line = self.get_first_line() if first_line: if first_line.startswith(SHEBANG): if language is None: return True if language in first_line: return True return False def is_python(self): """Checks if the file in path is a Python script. :return: True if the file is a Python script, False otherwise. :rtype: bool """ if self.path.endswith(".py"): return True return self.is_script(language="python") def usable_rw_dir(directory, create=True): """Verify whether we can use this dir (read/write). Checks for appropriate permissions, and creates missing dirs as needed. :param directory: Directory to check. :type directory: str :param create: whether to create the directory if it does not exist. :type create: bool :return: True if the directory is usable for rw, False otherwise. :rtype: bool """ if os.path.isdir(directory): try: fd, path = tempfile.mkstemp(dir=directory) os.close(fd) os.unlink(path) return True except OSError: pass elif create: try: init_dir(directory) return True except OSError: pass return False def usable_ro_dir(directory): """Verify whether dir exists and we can access its contents. Check if a usable RO directory is there. :param directory: Directory to check. :type directory: str :return: True if the directory is accessible, False otherwise. :rtype: bool """ try: cwd = os.getcwd() except FileNotFoundError: return False if os.path.isdir(directory): try: os.chdir(directory) os.chdir(cwd) return True except OSError: pass return False def check_readable(path): """Verify that the given path exists and is readable. This should be used where an assertion makes sense, and is useful because it can provide a better message in the exception it raises. :param path: the path to test :type path: str :raise OSError: path does not exist or path could not be read """ if not os.path.exists(path): raise OSError(f'File "{path}" does not exist') if not os.access(path, os.R_OK): raise OSError(f'File "{path}" can not be read') def get_path_mount_point(path): """Returns the mount point for a given file path. :param path: the complete filename path. if a non-absolute path is given, it's transformed into an absolute path first. :type path: str :returns: the mount point for a given file path :rtype: str """ path = os.path.abspath(path) while not os.path.ismount(path): path = os.path.dirname(path) return path def get_max_file_name_length(path): """Returns the maximum length of a file name in the underlying file system. :param path: the complete filename path. if a non-absolute path is given, it's transformed into an absolute path first. :type path: str :returns: the maximum length of a file name :rtype: int """ if hasattr(os, "pathconf"): mount_point = get_path_mount_point(path) return os.pathconf(mount_point, "PC_NAME_MAX") # Given the unavailability of os.pathconf(), always available # under Unix, it should be safe to assume this is Windows. # About Windows, versions and configurations can yield different # file name length limits. The value hardcoded here (248) is # calculated from the 260 MAX_PATH limit, plus the provision # for directories names allowing a 8.3 filename inside it. return 248 # pylint: disable=wrong-import-position from avocado.utils.deprecation import log_deprecation log_deprecation.warning("path") avocado-113.0/avocado/utils/pci.py000066400000000000000000000451301513561306000170360ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: 2016 IBM # Author: Narasimhan V # # Author: Kleber Sacilotto de Souza # Author: Daniel Kreling # for get_memory_address() and get_mask() """ Module for all PCI devices related functions. """ import os import re from avocado.utils import genio, process, wait def get_domains(): """ Gets all PCI domains. Example, it returns ['0000', '0001', ...] :return: List of PCI domains. :rtype: list of str """ cmd = "lspci -D" output = process.run(cmd, ignore_status=True, shell=True).stdout_text if output: domains = [] for line in output.splitlines(): domains.append(line.split(":")[0]) return list(set(domains)) return [] def get_pci_addresses(): """ Gets list of PCI addresses in the system. Does not return the PCI Bridges/Switches. :return: list of full PCI addresses including domain (0000:00:14.0) :rtype: list of str """ addresses = [] cmd = "lspci -D" for line in process.run(cmd).stdout_text.splitlines(): if not get_pci_prop(line.split()[0], "Class").startswith("06"): addresses.append(line.split()[0]) if addresses: return addresses return [] def get_num_interfaces_in_pci(dom_pci_address): """ Gets number of interfaces of a given partial PCI address starting with full domain address. :param dom_pci_address: Partial PCI address including domain address (0000, 0000:00:1f, 0000:00:1f.2, etc) :return: number of devices in a PCI domain. :rtype: int """ cmd = "ls -l /sys/class/*/ -1" output = process.run(cmd, ignore_status=True, shell=True).stdout_text if output: filt = f"/{dom_pci_address}" count = 0 for line in output.splitlines(): if filt in line: count += 1 return count return 0 def get_disks_in_pci_address(pci_address): """ Gets disks in a PCI address. :param pci_address: Any segment of a PCI address (1f, 0000:00:1f, ...) :return: list of disks in a PCI address. """ disk_list = [] for device in os.listdir("/sys/block"): if pci_address in os.path.realpath(os.path.join("/sys/block", device)): disk_list.append(f"/dev/{device}") return disk_list def get_nics_in_pci_address(pci_address): """ Gets network interface(nic) in a PCI address. :param pci_address: Any segment of a PCI address (1f, 0000:00:1f, ...) :return: list of network interfaces in a PCI address. """ return get_interfaces_in_pci_address(pci_address, "net") def get_interfaces_in_pci_address(pci_address, pci_class): """ Gets interface in a PCI address. e.g: host = pci.get_interfaces_in_pci_address("0001:01:00.0", "net") ['enP1p1s0f0'] host = pci.get_interfaces_in_pci_address("0004:01:00.0", "fc_host") ['host6'] :param pci_address: Any segment of a PCI address (1f, 0000:00:1f, ...) :param class: Adapter type (FC(fc_host), FCoE(net), NIC(net), SCSI(scsi)..) :return: list of generic interfaces in a PCI address. """ pci_class_path = f"/sys/class/{pci_class}/" ifaces = [] if not os.path.isdir(pci_class_path): raise ValueError("Please provide valid class name") for interface in os.listdir(pci_class_path): pci_path = os.path.join(pci_class_path, interface) if os.path.isdir(pci_path) and pci_address in os.readlink(pci_path): ifaces.append(interface) return ifaces def get_pci_class_name(pci_address): """ Gets pci class name for given pci bus address e.g: >>> pci.get_pci_class_name("0000:01:00.0") 'scsi_host' :param pci_address: Any segment of a PCI address(1f, 0000:00:if, ...) :return: class name for corresponding pci bus address """ pci_class_dic = { "0104": "scsi_host", "0c04": "fc_host", "0200": "net", "0108": "nvme", "0280": "net", "0207": "net", "0c03": "usb", } pci_class_id = get_pci_prop(pci_address, "Class") if pci_class_id not in pci_class_dic: if pci_class_id is None: raise ValueError( f"Unable to get 'Class' property of given pci " f"address {pci_address}" ) raise ValueError( f"Class ID {pci_class_id} is not defined " f"in this library please send an update" ) return pci_class_dic.get(pci_class_id) def get_pci_fun_list(pci_address): """ Gets list of functions in the given PCI address. Example: in address 0000:03:00, functions are 0000:03:00.0 and 0000:03:00.1 :param pci_address: Any segment of a PCI address (1f, 0000:00:1f, ...) :return: list of functions in a PCI address. """ return list(dev for dev in get_pci_addresses() if pci_address in dev) def get_slot_from_sysfs(full_pci_address): """ Gets the PCI slot of given address. :note: Specific for ppc64 processor. :param full_pci_address: Full PCI address including domain (0000:03:00.0) :return: Removed port related details using re, only returns till physical slot of the adapter. """ if not os.path.isfile(f"/sys/bus/pci/devices/{full_pci_address}/devspec"): return None devspec = genio.read_file( f"/sys/bus/pci/devices/" f"{full_pci_address}/devspec" ).strip() if not os.path.isfile(f"/proc/device-tree/{devspec}/ibm,loc-code"): return None slot = genio.read_file(f"/proc/device-tree/{devspec}/ibm,loc-code") slot_ibm = re.match(r"((\w+)[.])+(\w+)-[PC(\d+)-]*C(\d+)", slot) if slot_ibm: return slot_ibm.group() slot_openpower = re.match(r"(\w+)[\s]*(\w+)(\d*)", slot) if slot_openpower: return slot_openpower.group() raise ValueError(f"Failed to get slot from: '{slot}'") def get_slot_list(): """ Gets list of PCI slots in the system. :note: Specific for ppc64 processor. :return: list of slots in the system. """ return list(set(get_slot_from_sysfs(dev) for dev in get_pci_addresses())) def get_pci_id_from_sysfs(full_pci_address): """ Gets the PCI ID from sysfs of given PCI address. :param full_pci_address: Full PCI address including domain (0000:03:00.0) :return: PCI ID of a PCI address from sysfs. """ path = f"/sys/bus/pci/devices/{full_pci_address}" if os.path.isdir(path): path = "%s/%%s" % path # pylint: disable=C0209 return ":".join( [ f"{int(open(path % param).read(), 16):04x}" # pylint: disable=W1514 for param in [ "vendor", "device", "subsystem_vendor", "subsystem_device", ] ] ) return None def get_pci_prop(pci_address, prop): """ Gets specific PCI ID of given PCI address. (first match only) :param pci_address: Any segment of a PCI address (1f, 0000:00:1f, ...) :param part: prop of PCI ID. :return: specific PCI ID of a PCI address. :rtype: str """ cmd = f"lspci -Dnvmm -s {pci_address}" output = process.run(cmd, ignore_status=True, shell=True).stdout_text if output: for line in output.splitlines(): if prop == line.split(":")[0]: return line.split()[-1] return None def get_pci_id(pci_address): """ Gets PCI id of given address. (first match only) :param pci_address: Any segment of a PCI address (1f, 0000:00:1f, ...) :return: PCI ID of a PCI address. """ pci_id = [] for params in ["Vendor", "Device", "SVendor", "SDevice"]: output = get_pci_prop(pci_address, params) if not output: return None pci_id.append(output) if pci_id: return ":".join(pci_id) return None def get_pci_info(pci_address): """ Gets PCI info of given PCI address. :param pci_address: Any segment of a PCI address (1f, 0000:00:1f, ...) :return: Dictionary attribute name as key and attribute value as value. :rtype: Dict """ cmd = f"lspci -Dnvmm -s {pci_address}" output = process.run(cmd, ignore_status=True, shell=True).stdout_text pci_info = {} if not output: return None for line in output.splitlines(): if line.strip(): pci_info[line.split(":")[0].strip()] = ( ":".join(line.split(":")[1:]) ).strip() return pci_info def get_driver(pci_address): """ Gets the kernel driver in use of given PCI address. (first match only) :param pci_address: Any segment of a PCI address (1f, 0000:00:1f, ...) :return: driver of a PCI address. :rtype: str """ cmd = f"lspci -ks {pci_address}" output = process.run(cmd, ignore_status=True, shell=True).stdout_text if output: for line in output.splitlines(): if "Kernel driver in use:" in line: return line.rsplit(None, 1)[-1] return None def unbind(driver, full_pci_address): """ Unbind the pci device(full_pci_address) from driver :param driver: driver of the PCI address (full_pci_address) :param full_pci_address: Full PCI address including domain (0000:03:00.0) :return: None """ genio.write_file_or_fail(f"/sys/bus/pci/drivers/{driver}/unbind", full_pci_address) if wait.wait_for( lambda: os.path.exists( f"/sys/bus/pci/drivers/\ {driver}/{full_pci_address}" ), timeout=5, ): raise ValueError(f"Not able to unbind {full_pci_address} from {driver}") def bind(driver, full_pci_address): """ Bind the pci device(full_pci_address) to driver :param driver: driver of the PCI address (full_pci_address) :param full_pci_address: Full PCI address including domain (0000:03:00.0) :return: None """ genio.write_file_or_fail(f"/sys/bus/pci/drivers/{driver}/bind", full_pci_address) if not wait.wait_for( lambda: os.path.exists( f"/sys/bus/pci/drivers/\ {driver}/{full_pci_address}" ), timeout=5, ): raise ValueError(f"Not able to bind {full_pci_address} to {driver}") def get_vendor_id(full_pci_address): """ Get vendor id of a PCI address :param full_pci_address: Full PCI address including domain (0000:03:00.0) :return: vendor id of PCI address :rtype: str """ cmd = f"lspci -n -s {full_pci_address}" out = process.run(cmd, ignore_status=True, shell=True).stdout_text if not out: raise ValueError(f"Not able to get {full_pci_address} vendor id") return out.split(" ")[2].strip() def reset_check(full_pci_address): """ Check if reset for "full_pci_address" is successful :param full_pci_address: Full PCI address including domain (0000:03:00.0) :return: whether reset for "full_pci_address" is successful :rtype: bool """ cmd = f"lspci -vvs {full_pci_address}" output = process.run(cmd, ignore_status=True, shell=True).stdout_text if not output: return False return True def rescan_check(full_pci_address): """ Check if rescan for full_pci_address is successful :param full_pci_address: Full PCI address including domain (0000:03:00.0) :return: whether rescan for full_pci_address is successful :rtype: bool """ cmd = f"lspci -vvs {full_pci_address}" output = process.run(cmd, ignore_status=True, shell=True).stdout_text if not output: return False return True def change_domain_check(dom, full_pci_address, def_dom): """ Check if the domain changed successfully to "dom" for "full_pci_address" :param dom: domain type :param def_dom: default domain of pci device(full_pci_address) :param full_pci_address: Full PCI address including domain (0000:03:00.0) :return: whether domain changed successfully to "dom" :rtype: bool """ try: output = genio.read_one_line( f"/sys/bus/pci/devices/{full_pci_address}/iommu_group/type" ) out = output.rsplit(None, 1)[-1] if (dom not in ("auto", out)) or (dom == "auto" and out != def_dom): return False return True except OSError as details: raise ValueError(f"Change domain check failed: {details}") from details def reset(full_pci_address): """ Remove the full_pci_address :param full_pci_address: Full PCI address including domain (0000:03:00.0) :return: None """ genio.write_file_or_fail(f"/sys/bus/pci/devices/{full_pci_address}/remove", "1") if not wait.wait_for(lambda: reset_check(full_pci_address), timeout=5): raise ValueError(f"Unsuccessful to remove {full_pci_address}") def rescan(full_pci_address): """ Rescan the system and check for full_pci_address :param full_pci_address: Full PCI address including domain (0000:03:00.0) :return: None """ genio.write_file_or_fail("/sys/bus/pci/rescan", "1") if not wait.wait_for(lambda: rescan_check(full_pci_address), timeout=5): raise ValueError(f"Unsuccessful to rescan for {full_pci_address}") def get_iommu_group(full_pci_address): """ Return the iommu group of full_pci_address :param full_pci_address: Full PCI address including domain (0000:03:00.0) :return: iommu group of full_pci_address :rtype: string """ cmd = f"lspci -vvvv -s {full_pci_address}" out = process.run(cmd, ignore_status=True, shell=True) for line in out.stdout_text.split("\n"): if "IOMMU group" in line: return line.strip().split(" ")[2] raise ValueError(f"{full_pci_address} group not found") def change_domain(dom, def_dom, full_pci_address): """ Change the domain of pci device(full_pci_address) to dom :param dom: domain type :param def_dom: default domain of pci device(full_pci_address) :param full_pci_address: Full PCI address including domain (0000:03:00.0) :return: None """ genio.write_file_or_fail( f"/sys/bus/pci/devices/{full_pci_address}/iommu_group/type", dom ) if not wait.wait_for( lambda: change_domain_check(dom, full_pci_address, def_dom), timeout=5 ): raise ValueError(f"Domain type change failed for {full_pci_address}") def get_memory_address(pci_address): """ Gets the memory address of a PCI address. (first match only) :note: There may be multiple memory address for a PCI address. :note: This function returns only the first such address. :param pci_address: Any segment of a PCI address (1f, 0000:00:1f, ...) :return: memory address of a pci_address. :rtype: str """ cmd = f"lspci -bv -s {pci_address}" output = process.run(cmd, ignore_status=True, shell=True).stdout_text if output: for line in output.splitlines(): if "Memory at" in line: return f"0x{line.split()[2]}" return None def get_mask(pci_address): """ Gets the mask of PCI address. (first match only) :note: There may be multiple memory entries for a PCI address. :note: This mask is calculated only with the first such entry. :param pci_address: Any segment of a PCI address (1f, 0000:00:1f, ...) :return: mask of a PCI address. :rtype: str """ cmd = f"lspci -vv -s {pci_address}" output = process.run(cmd, ignore_status=True, shell=True).stdout_text if not output: return None dic = {"K": 1024, "M": 1048576, "G": 1073741824} for line in output.splitlines(): if "Region" in line and "Memory at" in line: val = line.split("=")[-1].split("]")[0] memory_size = int(val[:-1]) * dic[val[-1]] break # int("0xffffffff", 16) = 4294967295 mask = hex((memory_size - 1) ^ 4294967295) return mask def get_vpd(dom_pci_address): """ Gets the VPD (Virtual Product Data) of the given PCI address. :note: Specific for ppc64 processor. :param dom_pci_address: Partial PCI address including domain addr and at least bus addr (0003:00, 0003:00:1f.2, ...) :return: dictionary of VPD of a PCI address. :rtype: dict """ cmd = f"lsvpd -l {dom_pci_address}" vpd = process.run(cmd).stdout_text vpd_dic = {} dev_list = [] for line in vpd.splitlines(): if len(line) < 5: continue if "*YL" in line: vpd_dic["slot"] = line[4:] elif "*DS" in line: vpd_dic["pci_id"] = line[4:] elif "*FC" in line: vpd_dic["feature_code"] = line[4:] elif "*AX" in line: if not (dom_pci_address in line or vpd_dic["pci_id"].split()[0] in line): dev_list.append(line[4:]) elif "*CD" in line: vpd_dic["pci_id"] = line[4:] vpd_dic["devices"] = dev_list return vpd_dic def get_cfg(dom_pci_address): """ Gets the hardware configuration data of the given PCI address. :note: Specific for ppc64 processor. :param dom_pci_address: Partial PCI address including domain addr and at least bus addr (0003:00, 0003:00:1f.2, ...) :return: dictionary of configuration data of a PCI address. :rtype: dict """ cmd = f"lscfg -vl {dom_pci_address}" cfg = process.run(cmd).stdout_text cfg_dic = {} desc = re.match( r" (%s)( [-\w+,\.]+)+([ \n])+([-\w+, \(\)])+" # pylint: disable=C0209 % dom_pci_address, cfg, ).group() cfg_dic["Description"] = desc for line in cfg.splitlines(): if "Manufacturer Name" in line: cfg_dic["Mfg"] = line.split(".")[-1] if "Machine Type-Model" in line: cfg_dic["Model"] = line.split(".")[-1] if "Device Specific" in line: cfg_dic["YC"] = line.split(".")[-1] if "Location Code" in line: cfg_dic["YL"] = line.split("..")[-1].strip(".") if "Description" in cfg_dic: pcid = re.search( r"[0-9a-e]{4}\:[0-9a-e]{2}\:[0-9a-e]{2}\.[0-9a-e]{1}", cfg_dic["Description"], ) cfg_dic["pci_id"] = pcid.group() device_subvendor = re.search(r"([0-9a-e]{8})", cfg_dic["Description"]) cfg_dic["subvendor_device"] = device_subvendor.group() return cfg_dic avocado-113.0/avocado/utils/pmem.py000066400000000000000000000437121513561306000172250ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright (C) IBM 2020 - Harish # # Author: Harish # import glob import json import re from avocado.utils import genio, path, process class PMemException(Exception): """ Error raised for all PMem failures """ def __init__(self, additional_text=None): # pylint: disable=W0231 self.additional_text = additional_text def __str__(self): return f"Command failed.\ninfo: {self.additional_text}" class PMem: """ PMem class which provides function to perform ndctl and daxctl operations This class can be used only if ndctl binaries are provided before hand """ def __init__(self, ndctl="ndctl", daxctl="daxctl"): """ Initialize PMem object :param ndctl: path to ndctl binary, defaults to ndctl :param daxctl: path to daxctl binary, defaults to ndctl """ abs_ndctl = path.find_command(ndctl, False) if not abs_ndctl: raise PMemException("Cannot use library without proper ndctl binary") self.ndctl = abs_ndctl abs_daxctl = path.find_command(daxctl, False) if not abs_daxctl: raise PMemException("Cannot use library without proper daxctl binary") self.daxctl = abs_daxctl @staticmethod def check_subcmd(binary, command): """Check if given sub command is supported by binary :param command: sub command of ndctl to check for existence :return: True if sub command is available :rtype: bool """ cmd = f"{binary} --list-cmds" out = process.system_output(cmd).decode().splitlines() if command in out: return True return False def check_ndctl_subcmd(self, command): """Check if given sub command is supported by ndctl""" return self.check_subcmd(self.ndctl, command) def check_daxctl_subcmd(self, command): """Check if given sub command is supported by daxctl""" return self.check_subcmd(self.daxctl, command) def run_ndctl_list(self, option=""): """ Get the json of each provided options :param option: optional arguments to ndctl list command :return: By default returns entire list of json objects :rtype: list of json objects """ try: cmd = f"{self.ndctl} list {option}" json_op = json.loads(process.system_output(cmd)) except ValueError: json_op = [] return json_op @staticmethod def run_ndctl_list_val(json_op, field): """ Get the value of a field in given json :param json_op: Input Json object :param field: Field to find the value from json_op object :rtype: Found value type, None if not found """ for key, value in json_op.items(): if key == field: return value return None def run_daxctl_list(self, options=""): """ Get the json of each provided options :param options: optional arguments to daxctl list command :return: By default returns entire list of json objects :rtype: list of json objects """ cmd = f"{self.daxctl} list {options}" return json.loads(process.system_output(cmd)) def set_dax_memory_online(self, device, region=None, no_movable=False): """Set memory from a given devdax device online :param device: Device from which memory is to be online :param region: Optionally filter device by region :param no_movable: Optionally make the memory non-movable :return: True if command succeeds :rtype: bool :raise: :class:`PMemException`, if command fails. """ cmd = f"{self.daxctl} online-memory {device}" if region: cmd += f" -r {region}" if no_movable: cmd += " --no-movable" if process.system(cmd, shell=True, ignore_status=True): raise PMemException(f"Failed to online memory with {device}") return True def set_dax_memory_offline(self, device, region=None): """Set memory from a given devdax device offline :param device: Device from which memory is to be offline :param region: Optionally filter device by region :return: True if command succeeds :rtype: bool :raise: :class:`PMemException`, if command fails. """ cmd = f"{self.daxctl} offline-memory {device}" if region: cmd += f" -r {region}" if process.system(cmd, shell=True, ignore_status=True): raise PMemException(f"Failed to offline memory with {device}") return True # pylint: disable=R0913 def reconfigure_dax_device( self, device, mode="devdax", region=None, no_online=False, no_movable=False ): """Reconfigure devdax device into devdax or system-ram mode :param device: Device from which memory is to be online :param mode: Mode with which device is to be configured, default:devdax :param region: Optionally filter device by region :param no_online: Optionally don't online the memory(only system-ram) :param no_movable: Optionally mark memory non-movable(only system-ram) :return: Property of configured device :rtype: str :raise: :class:`PMemException`, if command fails. """ cmd = f"{self.daxctl} reconfigure-device {device} -m {mode}" if region: cmd += f" -r {region}" if no_online: cmd += " -N" if no_movable: cmd += " --no-movable" device_property = process.run(cmd, shell=True, ignore_status=True) if device_property.exit_status: raise PMemException(f"Failed to reconfigure device {device}") return device_property.stdout_text def get_slot_count(self, region): """ Get max slot count in the index area for a dimm backing a region We use region0 - > nmem0 :param region: Region for which slot count is found :return: Number of slots for given region 0 in case region is not available/command fails :rtype: int """ nmem = "nmem%s" % re.findall(r"\d+", region)[0] try: json_op = json.loads( process.system_output( f"{self.ndctl} read-labels -j {nmem} ", shell=True ) ) except ValueError: return [] first_dict = json_op[0] index_dict = self.run_ndctl_list_val(first_dict, "index")[0] return self.run_ndctl_list_val(index_dict, "nslot") - 2 @staticmethod def is_region_legacy(region): """ Check whether we have label index namespace. If legacy we can't create new namespaces. :param region: Region for which legacy check is made :return: True if given region is legacy, else False """ reg = f"/sys/bus/nd/devices/{region}/nstype" nstype = genio.read_file(reg).rstrip("\n") if nstype == "4": return True return False @staticmethod def check_buses(): """ Get buses from sys subsystem to verify persistent devices exist :return: List of buses available :rtype: list """ return glob.glob("/sys/bus/nd/drivers/nd_bus/ndbus*") def disable_region(self, name="all"): """ Disable given region :param name: name of the region to be disabled :return: True on success :raise: :class:`PMemException`, if command fails. """ if process.system( f"{self.ndctl} disable-region {name}", shell=True, ignore_status=True ): raise PMemException(f"Failed to disable {name} region(s)") return True def enable_region(self, name="all"): """ Enable given region :param name: name of the region to be enabled :return: True on success :raise: :class:`PMemException`, if command fails. """ if process.system( f"{self.ndctl} enable-region {name}", shell=True, ignore_status=True ): raise PMemException(f"Failed to enable {name} region(s)") return True def disable_namespace(self, namespace="all", region="", bus="", verbose=False): """ Disable namespaces :param namespace: name of the namespace to be disabled :param region: Filter namespace by region :param bus: Filter namespace by bus :param verbose: Enable True command with debug information :return: True on success :raise: :class:`PMemException`, if command fails. """ args = namespace if region: args = f"{args} -r {region}" if bus: args = f"{args} -b {bus}" if verbose: args = f"{args} -v" if process.system( f"{self.ndctl} disable-namespace {args}", shell=True, ignore_status=True ): raise PMemException(f"Namespace disable failed for {namespace}") return True def enable_namespace(self, namespace="all", region="", bus="", verbose=False): """ Enable namespaces :param namespace: name of the namespace to be enabled :param region: Filter namespace by region :param bus: Filter namespace by bus :param verbose: Enable True command with debug information return: True on success :raise: :class:`PMemException`, if command fails. """ args = namespace if region: args = f"{args} -r {region}" if bus: args = f"{args} -b {bus}" if verbose: args = f"{args} -v" if process.system( f"{self.ndctl} enable-namespace {args}", shell=True, ignore_status=True ): raise PMemException(f'Namespace enable failed for "{namespace}"') return True # pylint: disable=R0914, R0913 def create_namespace( self, region="", bus="", n_type="pmem", mode="fsdax", memmap="dev", name="", size="", uuid="", sector_size="", align="", reconfig="", force=False, autolabel=False, ): """ Creates namespace with specified options :param region: Region on which namespace has to be created :param bus: Bus with which namespace has to be created :param n_type: Type of namespace to be created [pmem/blk] :param mode: Mode of namespace to be created, defaults to fsdax :param memmap: Metadata mapping for created namespace :param name: Optional name provided for namespace :param size: Size with which namespace has to be created :param uuid: Optional uuid provided for namespace :param sector_size: Sector size with which namespace has to be created :param align: Alignment with which namespace has to be created :param reconfig: Optionally reconfigure namespace providing existing namespace/region name :param force: Force creation of namespace :param autolabel: Optionally autolabel the namespace :return: True on success :raise: :class:`PMemException`, if command fails. """ args_dict = { region: "-r", bus: "-b", name: "-n", size: "-s", uuid: "-u", sector_size: "-l", align: "-a", reconfig: "-e", } minor_dict = {force: "-f", autolabel: "-L"} args = f"-t {n_type} -m {mode} " if mode in ["fsdax", "devdax"]: args += f" -M {memmap}" for option in list(args_dict.keys()): if option: args += f" {args_dict[option]} {option}" for option in list(minor_dict.keys()): if option: args += f" {minor_dict[option]}" if self.is_region_legacy(region) and not reconfig: namespace = "namespace%s.0" % re.findall(r"\d+", region)[0] args += " -f -e " + namespace if process.system( f"{self.ndctl} create-namespace {args}", shell=True, ignore_status=True ): raise PMemException("Namespace create command failed") return True def destroy_namespace(self, namespace="all", region="", bus="", force=False): """ Destroy namespaces, skipped in case of legacy namespace :param namespace: name of the namespace to be destroyed :param region: Filter namespace by region :param bus: Filter namespace by bus :param force: Force a namespace to be destroyed :return: True on Success :raise: :class:`PMemException`, if command fails. """ if region and self.is_region_legacy(region): return True args = namespace args_dict = {region: "-r", bus: "-b"} for option in list(args_dict.keys()): if option: args += f" {args_dict[option]} {option}" if force: args += " -f" if process.system( f"{self.ndctl} destroy-namespace {args}", shell=True, ignore_status=True ): raise PMemException("Namespace destroy command failed") return True @staticmethod def _check_arg(key, kwargs): if key in kwargs and kwargs[key]: return True return False @staticmethod def _check_add_arg(args_dict, key, kwargs, pop=False): if PMem._check_arg(key, kwargs): if pop: return f" {args_dict[key]}" % kwargs.pop(key) return f" {args_dict[key]}" % kwargs.get(key) return "" @staticmethod def _filter_ns_infoblock(namespace, args_dict, kwargs): args = "" if namespace == "all": for key in ["region", "bus"]: args += PMem._check_add_arg(args_dict, key, kwargs, pop=True) return args def write_infoblock(self, namespace="", stdout=False, output=None, **kwargs): """ Write an infoblock to the specified medium. :param namespace: Write the infoblock to given namespace :param stdout: Write the infoblock to stdout if True :param output: Write the infoblock to the file path specified :param kwargs: Example: pmem.write_infoblock(namespace=ns_name, align=align, size=size, mode='devdax') :return: True if command succeeds :rtype: bool :raise: :class:`PMemException`, if command fails. """ if not (namespace or stdout or output): raise PMemException("Specify at least one output medium") args_dict = { "region": "-r %s", "bus": "-b %s", "mode": "-m %s", "memmap": "-M %s", "size": "-s %s", "align": "-a %s", "uuid": "-u %s", "parent_uuid": "-p %", "offset": "-O %s", } if namespace: args = namespace elif stdout: args = "-c" elif output: args = f"-o {output}" args += self._filter_ns_infoblock(namespace, args_dict, kwargs) args += f" {args_dict['mode']}" % kwargs.pop("mode", "fsdax") args += f" {args_dict['memmap']}" % kwargs.pop("memmap", "dev") for key, value in kwargs.items(): if not value: continue if key in args_dict: args += f" {args_dict[key]}" % value else: raise PMemException("Input not supported for write-infoblock") write_cmd = f"{self.ndctl} write-infoblock {args}" if process.system(write_cmd, shell=True, ignore_status=True): raise PMemException("write-infoblock command failed") return True def read_infoblock(self, namespace="", inp_file="", **kwargs): """ Read an infoblock from the specified medium :param namespace: Read the infoblock from given namespace :param inp_file: Input file to read the infoblock from :param kwargs: Example: self.plib.read_infoblock(namespace=ns_name, json_form=True) :return: By default return list of json objects, if json_form is True Return as raw data, if json_form is False Return file path if op_file is specified :raise: :class:`PMemException`, if command fails. """ if not (namespace or inp_file): raise PMemException("Namespace or input file must be specified") args_dict = {"region": "-r %s", "bus": "-b %s", "op_file": "-o %s"} if namespace: args = namespace elif inp_file: args = f"-i {inp_file}" args += self._filter_ns_infoblock(namespace, args_dict, kwargs) args += self._check_add_arg(args_dict, "op_file", kwargs) json_form = kwargs.pop("json_form", True) verify = kwargs.pop("verify", False) if verify: args += " -V" if json_form: args += " -j" read_cmd = f"{self.ndctl} read-infoblock {args}" ret = process.run(read_cmd, shell=True, ignore_status=True) if ret.exit_status: raise PMemException("read-infoblock command failed") if self._check_arg("op_file", kwargs): return kwargs.get("op_file") read_op = ret.stdout.decode() if json_form: read_op = json.loads(read_op) return read_op avocado-113.0/avocado/utils/podman.py000066400000000000000000000252101513561306000175360ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # # Copyright: 2021 Red Hat Inc. # Authors : Beraldo Leal """ This module provides an basic API for interacting with podman. This module it was designed to be executed in async mode. Remember this when consuming this API. """ import json import logging import subprocess from asyncio import create_subprocess_exec from asyncio import subprocess as asyncio_subprocess from shutil import which LOG = logging.getLogger(__name__) class PodmanException(Exception): pass class _Podman: PYTHON_VERSION_COMMAND = json.dumps( [ "/usr/bin/env", "python3", "-c", ( "import sys; print(sys.version_info.major, " "sys.version_info.minor, sys.executable)" ), ] ) def __init__(self, podman_bin=None): path = which(podman_bin or "podman") if not path: msg = f"Podman binary {podman_bin} is not available on the system." raise PodmanException(msg) self.podman_bin = path class Podman(_Podman): def execute(self, *args): """Execute a command and return the returncode, stdout and stderr. :param args: Variable length argument list to be used as argument during execution. :rtype: tuple with returncode, stdout and stderr. """ try: LOG.debug("Executing %s", args) cmd = [self.podman_bin, *args] with subprocess.Popen( cmd, stdin=subprocess.DEVNULL, stdout=subprocess.PIPE, stderr=subprocess.PIPE, ) as proc: stdout, stderr = proc.communicate() LOG.debug("Return code: %s", proc.returncode) LOG.debug("Stdout: %s", stdout.decode("utf-8", "replace")) LOG.debug("Stderr: %s", stderr.decode("utf-8", "replace")) if proc.returncode: command_args = " ".join(args) msg = f'Failure from command "{self.podman_bin} {command_args}": returned code "{proc.returncode}" stderr: "{stderr}"' LOG.error(msg) raise PodmanException(msg) return proc.returncode, stdout, stderr except (FileNotFoundError, PermissionError) as ex: # Since this method is also used by other methods, let's # log here as well. msg = "Could not execute the command." LOG.error("%s: %s", msg, str(ex)) raise PodmanException(msg) from ex def copy_to_container(self, container_id, src, dst): """Copy artifacts from src to container:dst. This method allows copying the contents of src to the dst. Files will be copied from the local machine to the container. The "src" argument can be a file or a directory. :param str container_id: string with the container identification. :param str src: what file or directory you are trying to copy. :param str dst: the destination inside the container. :rtype: tuple with returncode, stdout and stderr. """ try: return self.execute("cp", src, f"{container_id}:{dst}") except PodmanException as ex: error = f"Failed copying data to container {container_id}" LOG.error(error) raise PodmanException(error) from ex def get_python_version(self, image): """Return the current Python version installed in an image. :param str image: Image name. i.e: 'fedora:33'. :rtype: tuple with both: major, minor numbers and executable path. """ try: _, stdout, _ = self.execute( "run", "--rm", f"--entrypoint={self.PYTHON_VERSION_COMMAND}", image ) except PodmanException as ex: raise PodmanException("Failed getting Python version.") from ex if stdout: output = stdout.decode().strip().split() return int(output[0]), int(output[1]), output[2] return None def get_container_info(self, container_id): """Return all information about specific container. :param container_id: identifier of container :type container_id: str :rtype: dict """ try: _, stdout, _ = self.execute( "ps", "--all", "--format=json", "--filter", f"id={container_id}" ) except PodmanException as ex: raise PodmanException( f"Failed getting information about container: {container_id}." ) from ex containers = json.loads(stdout.decode()) for container in containers: if container.get("Id") == container_id: return container return {} def start(self, container_id): """Starts a container and return the returncode, stdout and stderr. :param str container_id: Container identification string to start. :rtype: tuple with returncode, stdout and stderr. """ try: return self.execute("start", container_id) except PodmanException as ex: raise PodmanException("Failed to start the container.") from ex def stop(self, container_id): """Stops a container and return the returncode, stdout and stderr. :param str container_id: Container identification string to stop. :rtype: tuple with returncode, stdout and stderr. """ try: return self.execute("stop", "-t=0", container_id) except PodmanException as ex: raise PodmanException("Failed to stop the container.") from ex class AsyncPodman(_Podman): async def execute(self, *args): """Execute a command and return the returncode, stdout and stderr. :param args: Variable length argument list to be used as argument during execution. :rtype: tuple with returncode, stdout and stderr. """ try: LOG.debug("Executing %s", args) proc = await create_subprocess_exec( self.podman_bin, *args, stdout=asyncio_subprocess.PIPE, stderr=asyncio_subprocess.PIPE, ) stdout, stderr = await proc.communicate() LOG.debug("Return code: %s", proc.returncode) LOG.debug("Stdout: %s", stdout.decode("utf-8", "replace")) LOG.debug("Stderr: %s", stderr.decode("utf-8", "replace")) except (FileNotFoundError, PermissionError) as ex: # Since this method is also used by other methods, let's # log here as well. msg = "Could not execute the command." LOG.error("%s: %s", msg, str(ex)) raise PodmanException(msg) from ex if proc.returncode: command_args = " ".join(args) msg = f'Failure from command "{self.podman_bin} {command_args}": returned code "{proc.returncode}" stderr: "{stderr}"' LOG.error(msg) raise PodmanException(msg) return proc.returncode, stdout, stderr async def copy_to_container(self, container_id, src, dst): """Copy artifacts from src to container:dst. This method allows copying the contents of src to the dst. Files will be copied from the local machine to the container. The "src" argument can be a file or a directory. :param str container_id: string with the container identification. :param str src: what file or directory you are trying to copy. :param str dst: the destination inside the container. :rtype: tuple with returncode, stdout and stderr. """ try: return await self.execute("cp", src, f"{container_id}:{dst}") except PodmanException as ex: error = f"Failed copying data to container {container_id}" LOG.error(error) raise PodmanException(error) from ex async def get_python_version(self, image): """Return the current Python version installed in an image. :param str image: Image name. i.e: 'fedora:33'. :rtype: tuple with both: major, minor numbers and executable path. """ try: _, stdout, _ = await self.execute( "run", "--rm", f"--entrypoint={self.PYTHON_VERSION_COMMAND}", image ) except PodmanException as ex: raise PodmanException("Failed getting Python version.") from ex if stdout: output = stdout.decode().strip().split() return int(output[0]), int(output[1]), output[2] async def get_container_info(self, container_id): """Return all information about specific container. :param container_id: identifier of container :type container_id: str :rtype: dict """ try: _, stdout, _ = await self.execute( "ps", "--all", "--format=json", "--filter", f"id={container_id}" ) except PodmanException as ex: raise PodmanException( f"Failed getting information about container:" f" {container_id}." ) from ex containers = json.loads(stdout.decode()) for container in containers: if container["Id"] == container_id: return container return {} async def start(self, container_id): """Starts a container and return the returncode, stdout and stderr. :param str container_id: Container identification string to start. :rtype: tuple with returncode, stdout and stderr. """ try: return await self.execute("start", container_id) except PodmanException as ex: raise PodmanException("Failed to start the container.") from ex async def stop(self, container_id): """Stops a container and return the returncode, stdout and stderr. :param str container_id: Container identification string to stop. :rtype: tuple with returncode, stdout and stderr. """ try: return await self.execute("stop", "-t=0", container_id) except PodmanException as ex: raise PodmanException("Failed to stop the container.") from ex avocado-113.0/avocado/utils/process.py000066400000000000000000001413701513561306000177440ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2013-2014 # Author: Lucas Meneghel Rodrigues # pylint: disable=C0302 """Functions dedicated to find and run external commands.""" import contextlib import errno import glob import logging import os import re import select import shlex import signal import subprocess import threading import time from io import BytesIO, UnsupportedOperation from avocado.utils import astring, path from avocado.utils.wait import wait_for LOG = logging.getLogger(__name__) # variable=value bash assignment _RE_BASH_SET_VARIABLE = re.compile(r"[a-zA-Z]\w*=.*") class CmdError(Exception): """Exception raised when a command fails execution.""" def __init__( self, command=None, result=None, additional_text=None ): # pylint: disable=W0231 self.command = command self.result = result self.additional_text = additional_text def __str__(self): return ( f"Command '{self.command}' failed.\nstdout: " f"{self.result.stdout!r}\nstderr: " f"{self.result.stderr!r}\nadditional_info: " f"{self.additional_text}" ) class CmdInputError(Exception): """Raised when the command given is invalid, such as an empty command.""" def can_sudo(cmd=None): """Check whether sudo is available or if running as root. This function checks if the current process has the ability to run commands with elevated privileges. It first checks if the process is running as root (UID 0), then checks if sudo is installed and functional. :param cmd: Optional command to test sudo capabilities with. If provided, tests whether this specific command can be run with sudo. If not provided, tests basic sudo functionality. :type cmd: str or None :return: True if sudo is available or running as root, False otherwise. :rtype: bool Example:: >>> can_sudo() True >>> can_sudo("ls /root") True """ if not os.getuid(): # Root return True try: # Does sudo binary exists? path.find_command("sudo") except path.CmdNotFoundError: return False try: if cmd: # Am I able to run the cmd or plain sudo id? return not system(cmd, ignore_status=True, sudo=True) if getoutput("id -u", sudo=True).strip() == "0": return True return False except OSError: # Broken sudo binary return False def get_capabilities(pid=None): """Gets a list of all capabilities for a process. In case the getpcaps command is not available, and empty list will be returned. It supports getpcaps' two different formats, the current and the so called legacy/ugly. :param pid: the process ID (PID), if one is not given, the current PID is used (given by :func:`os.getpid`) :type pid: int :returns: all capabilities :rtype: list """ if pid is None: pid = os.getpid() try: result = run(f"getpcaps {int(pid)}", ignore_status=True) except FileNotFoundError: return [] if result.exit_status: return [] if result.stderr_text.startswith("Capabilities "): info = result.stderr_text separator = "=" else: info = result.stdout_text separator = ":" return info.split(separator, 1)[1].strip().split(",") def has_capability(capability, pid=None): """Check if a process has a given Linux capability. This is a simple wrapper around getpcaps, part of the libcap package. In case the getpcaps command is not available, the capability will be considered *not* to be available. :param capability: The name of the capability (e.g., "cap_sys_admin"). Refer to capabilities(7) man page for more information. Note: capability names are UPPERCASE in capabilities(7) (e.g., CAP_SYS_ADMIN) but must be lowercase in Python (e.g., "cap_sys_admin"). :type capability: str :param pid: The process ID to check. If None, checks the current process. :type pid: int or None :return: True if the capability is available, False otherwise. :rtype: bool Example:: >>> has_capability("cap_chown") True >>> has_capability("cap_sys_admin", pid=1234) False """ return capability in get_capabilities(pid) def pid_exists(pid): """Check if a process with the given PID exists. This function uses os.kill with signal 0 to check if a process exists without actually sending a signal to it. :param pid: The process ID number to check. :type pid: int :return: True if the process exists, False otherwise. :rtype: bool Example:: >>> pid_exists(1) True >>> pid_exists(999999) False """ try: os.kill(pid, 0) except OSError as detail: if detail.errno == errno.ESRCH: return False return True def safe_kill(pid, signal): # pylint: disable=W0621 """Attempt to send a signal to a process that may or may not exist. This function safely sends a signal to a process, handling cases where the process might not exist or require elevated privileges. :param pid: The process ID to send the signal to. :type pid: int :param signal: The signal number to send (e.g., signal.SIGTERM). :type signal: int :return: True if signal was sent successfully, False otherwise. :rtype: bool Example:: >>> safe_kill(1234, signal.SIGTERM) True """ if not get_owner_id(int(pid)): kill_cmd = f"kill -{int(signal)} {int(pid)}" try: run(kill_cmd, sudo=True) return True except CmdError: return False try: os.kill(pid, signal) return True except Exception: # pylint: disable=W0703 return False def get_parent_pid(pid): """Get the parent process ID for a given process. This function reads the /proc filesystem to determine the parent PID. .. note:: This is currently Linux specific. :param pid: The PID of the child process. :type pid: int :return: The parent process ID. :rtype: int :raises IOError: If the /proc entry cannot be read. Example:: >>> get_parent_pid(1234) 1 """ with open(f"/proc/{int(pid)}/stat", "rb") as proc_stat: parent_pid = proc_stat.read().split(b" ")[-49] return int(parent_pid) def _get_pid_from_proc_pid_stat(proc_path): match = re.match(r"\/proc\/([0-9]+)\/.*", proc_path) if match is not None: return int(match.group(1)) return None def get_children_pids(parent_pid, recursive=False): """Get the list of child process IDs for a given parent process. This function scans the /proc filesystem to find all child processes of the specified parent PID. .. note:: This is currently Linux specific. :param parent_pid: The PID of the parent process. :type parent_pid: int :param recursive: If True, also returns grandchildren and all descendants. If False, only returns direct children. :type recursive: bool :return: List of child process IDs. :rtype: list of int Example:: >>> get_children_pids(1) [234, 456, 789] >>> get_children_pids(1, recursive=True) [234, 456, 789, 1011, 1213] """ proc_stats = glob.glob("/proc/[123456789]*/stat") children = [] for proc_stat in proc_stats: try: with open(proc_stat, "rb") as proc_stat_fp: this_parent_pid = int(proc_stat_fp.read().split(b" ")[-49]) except IOError: continue if this_parent_pid == parent_pid: children.append(_get_pid_from_proc_pid_stat(proc_stat)) if recursive: for child in children: children.extend(get_children_pids(child)) return children def kill_process_tree(pid, sig=None, send_sigcont=True, timeout=0): """Signal a process and all of its children. If the process does not exist -- return. :param pid: The pid of the process to signal. :type pid: int :param sig: The signal to send to the processes, defaults to :data:`signal.SIGKILL` :type sig: int or None :param send_sigcont: Send SIGCONT to allow killing stopped processes. :type send_sigcont: bool :param timeout: How long to wait for the pid(s) to die (negative=infinity, 0=don't wait, positive=number_of_seconds). :type timeout: int or float :return: List of all PIDs we sent signal to. :rtype: list :raises RuntimeError: If timeout is reached waiting for processes to die. """ def _all_pids_dead(killed_pids): for pid in killed_pids: if pid_exists(pid): return False return True if sig is None: sig = signal.SIGKILL if timeout > 0: start = time.monotonic() if not safe_kill(pid, signal.SIGSTOP): return [pid] killed_pids = [pid] for child in get_children_pids(pid): killed_pids.extend(kill_process_tree(int(child), sig, False)) safe_kill(pid, sig) if send_sigcont: for killed_pid in killed_pids: safe_kill(killed_pid, signal.SIGCONT) if not timeout: return killed_pids if timeout > 0: if not wait_for( _all_pids_dead, timeout + start - time.monotonic(), step=0.01, args=(killed_pids[::-1],), ): raise RuntimeError( f"Timeout reached when waiting for pid {pid} " f"and children to die ({timeout})" ) else: while not _all_pids_dead(killed_pids[::-1]): time.sleep(0.01) return killed_pids def kill_process_by_pattern(pattern): """Send SIGTERM signal to processes matching a pattern. This function uses the pkill command to terminate processes whose command line matches the given pattern. :param pattern: Pattern to match against process command lines. This is matched using pkill's -f flag, which matches against the full command line. :type pattern: str Example:: >>> kill_process_by_pattern("firefox") >>> kill_process_by_pattern("python.*test_script") """ cmd = f"pkill -f {pattern}" result = run(cmd, ignore_status=True) if result.exit_status: LOG.error("Failed to run '%s': %s", cmd, result) else: LOG.info("Succeed to run '%s'.", cmd) def process_in_ptree_is_defunct(ppid): """Verify if any processes deriving from PPID are in the defunct state. Attempt to verify if parent process and any children from PPID is defunct (zombie) or not. This relies on the GNU version of "ps" and is not guaranteed to work in MacOS. :param ppid: The parent PID of the process to verify. :type ppid: int :return: True if any process in the tree is defunct, False otherwise. :rtype: bool """ defunct = False try: pids = get_children_pids(ppid) except CmdError: # Process doesn't exist return True for pid in pids: cmd = f"ps --no-headers -o cmd {int(pid)}" proc_name = system_output(cmd, ignore_status=True, verbose=False) if "" in proc_name: defunct = True break return defunct def binary_from_shell_cmd(cmd): """Extract the first binary path from a shell-like command string. This function parses a shell command and returns the first binary/executable found, skipping environment variable assignments. .. note:: This is a naive implementation that handles common patterns like environment variable assignments before the binary name. :param cmd: A shell-like command string to parse. :type cmd: str :return: The first binary/executable found in the command. :rtype: str :raises ValueError: If no binary can be extracted from the command. Example:: >>> binary_from_shell_cmd("binary") 'binary' >>> binary_from_shell_cmd("VAR=VAL binary -args") 'binary' >>> binary_from_shell_cmd("FOO=bar ./script.py") './script.py' """ cmds = shlex.split(cmd) for item in cmds: if not _RE_BASH_SET_VARIABLE.match(item): return item raise ValueError(f"Unable to parse first binary from '{cmd}'") #: This is kept for compatibility purposes, but is now deprecated and #: will be removed in later versions. Please use :func:`shlex.split` #: instead. cmd_split = shlex.split class CmdResult: """Command execution result. :param command: the command line itself :type command: str :param exit_status: exit code of the process :type exit_status: int :param stdout: content of the process stdout :type stdout: bytes :param stderr: content of the process stderr :type stderr: bytes :param duration: elapsed wall clock time running the process :type duration: float :param pid: ID of the process :type pid: int :param encoding: the encoding to use for the text version of stdout and stderr, by default :data:`avocado.utils.astring.ENCODING` :type encoding: str """ # pylint: disable=R0913, R0902 def __init__( self, command="", stdout=b"", stderr=b"", exit_status=None, duration=0, pid=None, encoding=None, ): self.command = command self.exit_status = exit_status #: The raw stdout (bytes) self.stdout = stdout #: The raw stderr (bytes) self.stderr = stderr self.duration = duration self.interrupted = False self.pid = pid if encoding is None: encoding = astring.ENCODING self.encoding = encoding def __str__(self): return "\n".join( f"{key}: {getattr(self, key, 'MISSING')!r}" for key in ( "command", "exit_status", "duration", "interrupted", "pid", "encoding", "stdout", "stderr", ) ) @property def stdout_text(self): """Return stdout decoded as text. :return: The stdout content as a string. :rtype: str :raises TypeError: If stdout cannot be decoded. """ if hasattr(self.stdout, "decode"): return self.stdout.decode(self.encoding) if isinstance(self.stdout, str): return self.stdout raise TypeError("Unable to decode stdout into a string-like type") @property def stderr_text(self): """Return stderr decoded as text. :return: The stderr content as a string. :rtype: str :raises TypeError: If stderr cannot be decoded. """ if hasattr(self.stderr, "decode"): return self.stderr.decode(self.encoding) if isinstance(self.stderr, str): return self.stderr raise TypeError("Unable to decode stderr into a string-like type") class FDDrainer: """Reads data from a file descriptor in a thread, storing locally.""" # pylint: disable=R0913, R0902 def __init__( self, fd, result, name=None, logger=None, logger_prefix="%s", stream_logger=None, ignore_bg_processes=False, verbose=False, ): """Initialize FDDrainer to read from a file descriptor in a thread. Stores data locally in a file-like :attr:`data` object. :param fd: a file descriptor that will be read (drained) from :type fd: int :param result: a :class:`CmdResult` instance associated with the process used to detect if the process is still running and if there's still data to be read. :type result: CmdResult :param name: a descriptive name that will be passed to the Thread name :type name: str :param logger: the logger that will be used to (interactively) write the content from the file descriptor :type logger: logging.Logger :param logger_prefix: the prefix used when logging the data :type logger_prefix: str :param stream_logger: a logger for streaming output :type stream_logger: logging.Logger :param ignore_bg_processes: When True the process does not wait for child processes which keep opened stdout/stderr streams after the main process finishes (eg. forked daemon which did not closed the stdout/stderr). Note this might result in missing output produced by those daemons after the main thread finishes and also it allows those daemons to be running after the process finishes. :type ignore_bg_processes: bool :param verbose: whether to log in both the logger and stream_logger :type verbose: bool """ self.fd = fd self.name = name self.data = BytesIO() self._result = result self._thread = None self._logger = logger self._logger_prefix = logger_prefix self._stream_logger = stream_logger self._ignore_bg_processes = ignore_bg_processes self._verbose = verbose def _log_line(self, line, newline_for_stream="\n"): line = astring.to_text(line, self._result.encoding, "replace") if self._logger is not None: self._logger.debug(self._logger_prefix, line) if self._stream_logger is not None: self._stream_logger.debug(line + newline_for_stream) def _drainer(self): """Read from fd, storing and optionally logging the output.""" bfr = b"" while True: if self._ignore_bg_processes: has_io = select.select([self.fd], [], [], 1)[0] if not has_io and self._result.exit_status is not None: # Exit if no new data and main process has finished break if not has_io: # Don't read unless there are new data available continue try: tmp = os.read(self.fd, 8192) except OSError: break if not tmp: break self.data.write(tmp) if self._verbose: bfr += tmp lines = bfr.splitlines() for line in lines[:-1]: self._log_line(line) if bfr.endswith(b"\n"): self._log_line(lines[-1]) else: self._log_line(lines[-1], "") bfr = b"" def start(self): """Start the drainer thread to read from the file descriptor.""" self._thread = threading.Thread(target=self._drainer, name=self.name) self._thread.daemon = True self._thread.start() def flush(self): """Wait for drainer thread to complete and flush stream handlers.""" self._thread.join() if self._stream_logger is not None: for handler in self._stream_logger.handlers: # FileHandler has a close() method, which we expect will # flush the file on disk. SocketHandler, MemoryHandler # and other logging handlers (custom ones?) also have # the same interface, so let's try to use it if available stream = getattr(handler, "stream", None) if (stream is not None) and (not stream.closed): if hasattr(stream, "fileno"): try: fileno = stream.fileno() os.fsync(fileno) except UnsupportedOperation: pass if hasattr(handler, "close"): handler.close() class SubProcess: """Run a subprocess in the background, collecting stdout/stderr streams.""" # pylint: disable=R0913, R0902 def __init__( self, cmd, verbose=True, shell=False, env=None, sudo=False, ignore_bg_processes=False, encoding=None, logger=None, ): """Create the subprocess object, stdout/err, reader threads and locks. :param cmd: Command line to run. :type cmd: str :param verbose: Whether to log the command run and stdout/stderr. :type verbose: bool :param shell: Whether to run the subprocess in a subshell. :type shell: bool :param env: Use extra environment variables. :type env: dict :param sudo: Whether the command requires admin privileges to run, so that sudo will be prepended to the command. The assumption here is that the user running the command has a sudo configuration such that a password won't be prompted. If that's not the case, the command will straight out fail. :type sudo: bool :param ignore_bg_processes: When True the process does not wait for child processes which keep opened stdout/stderr streams after the main process finishes (eg. forked daemon which did not closed the stdout/stderr). Note this might result in missing output produced by those daemons after the main thread finishes and also it allows those daemons to be running after the process finishes. :type ignore_bg_processes: bool :param encoding: the encoding to use for the text representation of the command result stdout and stderr, by default :data:`avocado.utils.astring.ENCODING` :type encoding: str :param logger: User's custom logger, which will be logging the subprocess outputs. When this parameter is not set, the `avocado.utils.process` logger will be used. :type logger: logging.Logger :raises ValueError: If incorrect values are given to parameters. """ if encoding is None: encoding = astring.ENCODING if sudo: self.cmd = self._prepend_sudo(cmd, shell) else: self.cmd = cmd self.verbose = verbose self.result = CmdResult(self.cmd, encoding=encoding) self.shell = shell if env: self.env = os.environ.copy() self.env.update(env) else: self.env = None self._popen = None self.logger = logger or LOG self.stdout_logger = self.logger.getChild("stdout") self.stderr_logger = self.logger.getChild("stderr") self.output_logger = self.logger.getChild("output") # Drainers used when reading from the PIPEs and writing to # files and logs self._stdout_drainer = None self._stderr_drainer = None self._ignore_bg_processes = ignore_bg_processes def __repr__(self): if self._popen is None: rc = "(not started)" elif self.result.exit_status is None: rc = "(running)" else: rc = self.result.exit_status return f"{self.__class__.__name__}(cmd={self.cmd!r}, rc={rc!r})" def __str__(self): if self._popen is None: rc = "(not started)" elif self.result.exit_status is None: rc = "(running)" else: rc = f"(finished with exit status={int(self.result.exit_status)})" return f"{self.cmd} {rc}" @staticmethod def _prepend_sudo(cmd, shell): if os.getuid(): try: sudo_cmd = f"{path.find_command('sudo', check_exec=False)} -n" except path.CmdNotFoundError as details: LOG.error(details) LOG.error( "Parameter sudo=True provided, but sudo was " "not found. Please consider adding sudo to " "your OS image" ) return cmd if shell: if " -s" not in sudo_cmd: sudo_cmd = f"{sudo_cmd} -s" cmd = f"{sudo_cmd} {cmd}" return cmd def _init_subprocess(self): def signal_handler(*args): self.result.interrupted = "signal/ctrl+c" self.wait() signal.default_int_handler(*args) if self._popen is not None: return if self.verbose: LOG.info("Running '%s'", self.cmd) if self.shell is False: cmd = shlex.split(self.cmd) else: cmd = self.cmd try: self._popen = subprocess.Popen( # pylint: disable=R1732 cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=self.shell, env=self.env, ) except OSError as details: details.strerror += f" ({self.cmd})" raise details self.start_time = time.monotonic() # pylint: disable=W0201 # prepare fd drainers self._stdout_drainer = FDDrainer( self._popen.stdout.fileno(), self.result, name=f"{self.cmd}-stdout", logger=self.logger, logger_prefix="[stdout] %s", stream_logger=None, ignore_bg_processes=self._ignore_bg_processes, verbose=self.verbose, ) self._stderr_drainer = FDDrainer( self._popen.stderr.fileno(), self.result, name=f"{self.cmd}-stderr", logger=self.logger, logger_prefix="[stderr] %s", stream_logger=None, ignore_bg_processes=self._ignore_bg_processes, verbose=self.verbose, ) # start stdout/stderr threads self._stdout_drainer.start() self._stderr_drainer.start() try: signal.signal(signal.SIGINT, signal_handler) except ValueError: if self.verbose: LOG.info("Command %s running on a thread", self.cmd) def _fill_results(self, rc): self._init_subprocess() self.result.exit_status = rc if not self.result.duration: self.result.duration = time.monotonic() - self.start_time if self.verbose: LOG.info( "Command '%s' finished with %s after %.9fs", self.cmd, rc, self.result.duration, ) self.result.pid = self._popen.pid self._fill_streams() def _fill_streams(self): """Close subprocess stdout and stderr, and put values into result obj.""" # Cleaning up threads if self._stdout_drainer is not None: self._stdout_drainer.flush() if self._stderr_drainer is not None: self._stderr_drainer.flush() # Clean subprocess pipes and populate stdout/err self.result.stdout = self.get_stdout() self.result.stderr = self.get_stderr() def start(self): """Start running the subprocess. This method is particularly useful for background processes, since you can start the subprocess and not block your test flow. :return: Subprocess PID. :rtype: int """ self._init_subprocess() return self._popen.pid def get_stdout(self): """Get the full stdout of the subprocess so far. :return: Standard output of the process. :rtype: bytes """ self._init_subprocess() return self._stdout_drainer.data.getvalue() def get_stderr(self): """Get the full stderr of the subprocess so far. :return: Standard error of the process. :rtype: bytes """ self._init_subprocess() return self._stderr_drainer.data.getvalue() def terminate(self): """Send a :attr:`signal.SIGTERM` to the process. Please consider using :meth:`stop` instead if you want to do all that's possible to finalize the process and wait for it to finish. """ self._init_subprocess() self.send_signal(signal.SIGTERM) def kill(self): """Send a :attr:`signal.SIGKILL` to the process. Please consider using :meth:`stop` instead if you want to do all that's possible to finalize the process and wait for it to finish. """ self._init_subprocess() self.send_signal(signal.SIGKILL) def send_signal(self, sig): """Send the specified signal to the process. :param sig: Signal to send. :type sig: int """ self._init_subprocess() if self.is_sudo_enabled(): pids = get_children_pids(self.get_pid()) pids.append(self.get_pid()) for pid in pids: kill_cmd = f"kill -{int(sig)} {int(pid)}" with contextlib.suppress(Exception): run(kill_cmd, sudo=True) else: self._popen.send_signal(sig) def poll(self): """Call the subprocess poll() method, fill results if rc is not None. :return: Return code if process has finished, None otherwise. :rtype: int or None """ self._init_subprocess() rc = self._popen.poll() if rc is not None: self._fill_results(rc) return rc def wait(self, timeout=None, sig=signal.SIGTERM): """Wait for subprocess to complete, fill results when done. :param timeout: Time (seconds) we'll wait until the process is finished. If it's not, we'll try to terminate it and it's children using ``sig`` and get a status. When the process refuses to die within 1s we use SIGKILL and report the status (be it exit_code or zombie). :type timeout: float or None :param sig: Signal to send to the process in case it did not end after the specified timeout. :type sig: int :return: Exit status of the process. :rtype: int :raises AssertionError: If the process becomes a zombie. """ def nuke_myself(): timeout = time.monotonic() - self.start_time self.result.interrupted = f"timeout after {timeout:.9f}s" try: kill_process_tree(self.get_pid(), sig, timeout=1) except RuntimeError: try: kill_process_tree(self.get_pid(), signal.SIGKILL, timeout=1) LOG.warning( "Process '%s' refused to die in 1s after " "sending %s to, destroyed it successfully " "using SIGKILL.", self.cmd, sig, ) except RuntimeError: LOG.error( "Process '%s' refused to die in 1s after " "sending %s, followed by SIGKILL, probably " "dealing with a zombie process.", self.cmd, sig, ) self._init_subprocess() rc = None if timeout is None: rc = self._popen.wait() elif timeout > 0.0: timer = threading.Timer(timeout, nuke_myself) try: timer.start() rc = self._popen.wait() finally: timer.cancel() if rc is None: stop_time = time.monotonic() + 1 while time.monotonic() < stop_time: rc = self._popen.poll() if rc is not None: break else: nuke_myself() rc = self._popen.poll() if rc is None: # If all this work fails, we're dealing with a zombie process. raise AssertionError(f"Zombie Process {self._popen.pid}") self._fill_results(rc) return rc def stop(self, timeout=None): """Stop background subprocess. Call this method to terminate the background subprocess and wait for its results. :param timeout: Time (seconds) we'll wait until the process is finished. If it's not, we'll try to terminate it and its children using ``sig`` and get a status. When the process refuses to die within 1s we use SIGKILL and report the status (be it exit_code or zombie). :type timeout: float or None :return: Exit status of the process. :rtype: int """ self._init_subprocess() if self.result.exit_status is None: self.terminate() return self.wait(timeout) def get_pid(self): """Report PID of this process. :return: Process ID. :rtype: int """ self._init_subprocess() return self._popen.pid def get_user_id(self): """Report user id of this process. :return: User ID of the process owner. :rtype: int or None """ self._init_subprocess() return get_owner_id(self.get_pid()) def is_sudo_enabled(self): """Return whether the subprocess is running with sudo enabled. :return: True if running as root (UID 0), False otherwise. :rtype: bool """ self._init_subprocess() return not self.get_user_id() def run(self, timeout=None, sig=signal.SIGTERM): """Start a process and wait for it to end, returning the result attr. If the process was already started using .start(), this will simply wait for it to end. :param timeout: Time (seconds) we'll wait until the process is finished. If it's not, we'll try to terminate it and its children using ``sig`` and get a status. When the process refuses to die within 1s we use SIGKILL and report the status (be it exit_code or zombie). :type timeout: float or None :param sig: Signal to send to the process in case it did not end after the specified timeout. :type sig: int :return: The command result object. :rtype: CmdResult """ self._init_subprocess() self.wait(timeout, sig) return self.result # pylint: disable=R0913 def run( cmd, timeout=None, verbose=True, ignore_status=False, shell=False, env=None, sudo=False, ignore_bg_processes=False, encoding=None, logger=None, ): """Run a subprocess, returning a CmdResult object. :param cmd: Command line to run. :type cmd: str :param timeout: Time limit in seconds before attempting to kill the running process. This function will take a few seconds longer than 'timeout' to complete if it has to kill the process. :type timeout: float or None :param verbose: Whether to log the command run and stdout/stderr. :type verbose: bool :param ignore_status: Whether to raise an exception when command returns =! 0 (False), or not (True). :type ignore_status: bool :param shell: Whether to run the command on a subshell. :type shell: bool :param env: Use extra environment variables. :type env: dict :param sudo: Whether the command requires admin privileges to run, so that sudo will be prepended to the command. The assumption here is that the user running the command has a sudo configuration such that a password won't be prompted. If that's not the case, the command will straight out fail. :type sudo: bool :param ignore_bg_processes: Whether to ignore background processes. :type ignore_bg_processes: bool :param encoding: the encoding to use for the text representation of the command result stdout and stderr, by default :data:`avocado.utils.astring.ENCODING` :type encoding: str :param logger: User's custom logger, which will be logging the subprocess outputs. When this parameter is not set, the `avocado.utils.process` logger will be used. :type logger: logging.Logger :return: A CmdResult object. :rtype: CmdResult :raises CmdInputError: If the command is empty. :raises CmdError: If ``ignore_status=False`` and command fails. """ if not cmd: raise CmdInputError("Invalid empty command") if encoding is None: encoding = astring.ENCODING sp = SubProcess( cmd=cmd, verbose=verbose, shell=shell, env=env, sudo=sudo, ignore_bg_processes=ignore_bg_processes, encoding=encoding, logger=logger, ) cmd_result = sp.run(timeout=timeout) fail_condition = cmd_result.exit_status or cmd_result.interrupted if fail_condition and not ignore_status: raise CmdError(cmd, sp.result) return cmd_result # pylint: disable=R0913 def system( cmd, timeout=None, verbose=True, ignore_status=False, shell=False, env=None, sudo=False, ignore_bg_processes=False, encoding=None, logger=None, ): """Run a subprocess, returning its exit code. :param cmd: Command line to run. :type cmd: str :param timeout: Time limit in seconds before attempting to kill the running process. This function will take a few seconds longer than 'timeout' to complete if it has to kill the process. :type timeout: float or None :param verbose: Whether to log the command run and stdout/stderr. :type verbose: bool :param ignore_status: Whether to raise an exception when command returns =! 0 (False), or not (True). :type ignore_status: bool :param shell: Whether to run the command on a subshell. :type shell: bool :param env: Use extra environment variables. :type env: dict :param sudo: Whether the command requires admin privileges to run, so that sudo will be prepended to the command. The assumption here is that the user running the command has a sudo configuration such that a password won't be prompted. If that's not the case, the command will straight out fail. :type sudo: bool :param ignore_bg_processes: Whether to ignore background processes. :type ignore_bg_processes: bool :param encoding: the encoding to use for the text representation of the command result stdout and stderr, by default :data:`avocado.utils.astring.ENCODING` :type encoding: str :param logger: User's custom logger, which will be logging the subprocess outputs. When this parameter is not set, the `avocado.utils.process` logger will be used. :type logger: logging.Logger :return: Exit code. :rtype: int :raises CmdError: If ``ignore_status=False`` and command fails. """ cmd_result = run( cmd=cmd, timeout=timeout, verbose=verbose, ignore_status=ignore_status, shell=shell, env=env, sudo=sudo, ignore_bg_processes=ignore_bg_processes, encoding=encoding, logger=logger, ) return cmd_result.exit_status # pylint: disable=R0913 def system_output( cmd, timeout=None, verbose=True, ignore_status=False, shell=False, env=None, sudo=False, ignore_bg_processes=False, strip_trail_nl=True, encoding=None, logger=None, ): """Run a subprocess, returning its output. :param cmd: Command line to run. :type cmd: str :param timeout: Time limit in seconds before attempting to kill the running process. This function will take a few seconds longer than 'timeout' to complete if it has to kill the process. :type timeout: float or None :param verbose: Whether to log the command run and stdout/stderr. :type verbose: bool :param ignore_status: Whether to raise an exception when command returns =! 0 (False), or not (True). :type ignore_status: bool :param shell: Whether to run the command on a subshell. :type shell: bool :param env: Use extra environment variables. :type env: dict :param sudo: Whether the command requires admin privileges to run, so that sudo will be prepended to the command. The assumption here is that the user running the command has a sudo configuration such that a password won't be prompted. If that's not the case, the command will straight out fail. :type sudo: bool :param ignore_bg_processes: Whether to ignore background processes. :type ignore_bg_processes: bool :param strip_trail_nl: Whether to strip the trailing newline. :type strip_trail_nl: bool :param encoding: the encoding to use for the text representation of the command result stdout and stderr, by default :data:`avocado.utils.astring.ENCODING` :type encoding: str :param logger: User's custom logger, which will be logging the subprocess outputs. When this parameter is not set, the `avocado.utils.process` logger will be used. :type logger: logging.Logger :return: Command output. :rtype: bytes :raises CmdError: If ``ignore_status=False`` and command fails. """ cmd_result = run( cmd=cmd, timeout=timeout, verbose=verbose, ignore_status=ignore_status, shell=shell, env=env, sudo=sudo, ignore_bg_processes=ignore_bg_processes, encoding=encoding, logger=logger, ) if strip_trail_nl: return cmd_result.stdout.rstrip(b"\n\r") return cmd_result.stdout # pylint: disable=R0913 def getoutput( cmd, timeout=None, verbose=False, ignore_status=True, shell=True, env=None, sudo=False, ignore_bg_processes=False, logger=None, ): """Return output (stdout or stderr) of executing cmd in a shell. Because commands module is removed in Python3 and it redirect stderr to stdout, we port commands.getoutput to make code compatible. :param cmd: Command line to run. :type cmd: str :param timeout: Time limit in seconds before attempting to kill the running process. This function will take a few seconds longer than 'timeout' to complete if it has to kill the process. :type timeout: float or None :param verbose: Whether to log the command run and stdout/stderr. :type verbose: bool :param ignore_status: Whether to raise an exception when command returns =! 0 (False), or not (True). :type ignore_status: bool :param shell: Whether to run the command on a subshell. :type shell: bool :param env: Use extra environment variables. :type env: dict :param sudo: Whether the command requires admin privileges to run, so that sudo will be prepended to the command. The assumption here is that the user running the command has a sudo configuration such that a password won't be prompted. If that's not the case, the command will straight out fail. :type sudo: bool :param ignore_bg_processes: Whether to ignore background processes. :type ignore_bg_processes: bool :param logger: User's custom logger, which will be logging the subprocess outputs. When this parameter is not set, the `avocado.utils.process` logger will be used. :type logger: logging.Logger :return: Command output (stdout or stderr). :rtype: str """ return getstatusoutput( cmd=cmd, timeout=timeout, verbose=verbose, ignore_status=ignore_status, shell=shell, env=env, sudo=sudo, ignore_bg_processes=ignore_bg_processes, logger=logger, )[1] # pylint: disable=R0913 def getstatusoutput( cmd, timeout=None, verbose=False, ignore_status=True, shell=True, env=None, sudo=False, ignore_bg_processes=False, logger=None, ): """Return (status, output) of executing cmd in a shell. Because commands module is removed in Python3 and it redirect stderr to stdout, we port commands.getstatusoutput to make code compatible. :param cmd: Command line to run. :type cmd: str :param timeout: Time limit in seconds before attempting to kill the running process. This function will take a few seconds longer than 'timeout' to complete if it has to kill the process. :type timeout: float or None :param verbose: Whether to log the command run and stdout/stderr. :type verbose: bool :param ignore_status: Whether to raise an exception when command returns =! 0 (False), or not (True). :type ignore_status: bool :param shell: Whether to run the command on a subshell. :type shell: bool :param env: Use extra environment variables. :type env: dict :param sudo: Whether the command requires admin privileges to run, so that sudo will be prepended to the command. The assumption here is that the user running the command has a sudo configuration such that a password won't be prompted. If that's not the case, the command will straight out fail. :type sudo: bool :param ignore_bg_processes: Whether to ignore background processes. :type ignore_bg_processes: bool :param logger: User's custom logger, which will be logging the subprocess outputs. When this parameter is not set, the `avocado.utils.process` logger will be used. :type logger: logging.Logger :return: Exit status and command output (stdout and stderr). :rtype: tuple """ cmd_result = run( cmd=cmd, timeout=timeout, verbose=verbose, ignore_status=ignore_status, shell=shell, env=env, sudo=sudo, ignore_bg_processes=ignore_bg_processes, logger=logger, ) text = cmd_result.stdout_text sts = cmd_result.exit_status if text[-1:] == "\n": text = text[:-1] return (sts, text) def get_owner_id(pid): """Get the user ID of the process owner. This function reads the /proc filesystem to determine the user ID that owns the specified process. .. note:: This is currently Linux specific. :param pid: The process ID to query. :type pid: int :return: The user ID of the process owner, or None if not found. :rtype: int or None Example:: >>> get_owner_id(1) 0 >>> get_owner_id(999999) None """ try: return os.stat(f"/proc/{int(pid)}/").st_uid except OSError: return None def get_command_output_matching(command, pattern): """Run a command and return lines matching a pattern. This function executes a command and searches its output for lines containing the specified pattern, returning all matching lines. :param command: The command to execute. :type command: str :param pattern: Pattern to search for in the output. Matching is done on a line-by-line basis using substring matching. :type pattern: str :return: List of lines from the command output that contain the pattern. :rtype: list of str Example:: >>> get_command_output_matching("ls -la", "txt") ['file1.txt', 'file2.txt'] """ return [line for line in run(command).stdout_text.splitlines() if pattern in line] # pylint: disable=wrong-import-position from avocado.utils.deprecation import log_deprecation log_deprecation.warning("process") avocado-113.0/avocado/utils/script.py000066400000000000000000000151231513561306000175660ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2014 # Author: Ruda Moura """Module to handle script file creation and management. This module provides utilities for creating executable script files, both permanent and temporary, with proper file permissions and automatic cleanup capabilities. """ import os import shutil import stat import tempfile from avocado.utils import path as utils_path #: What is commonly known as "0775" or "u=rwx,g=rwx,o=rx" DEFAULT_MODE = ( stat.S_IRUSR | stat.S_IWUSR | stat.S_IXUSR | stat.S_IRGRP | stat.S_IWGRP | stat.S_IXGRP | stat.S_IROTH | stat.S_IXOTH ) #: What is commonly known as "0444" or "u=r,g=r,o=r" READ_ONLY_MODE = stat.S_IRUSR | stat.S_IRGRP | stat.S_IROTH class Script: """Class that represents a script file. This class provides methods to create, save, and remove script files with configurable permissions. It supports context manager protocol for automatic cleanup. """ def __init__(self, path, content, mode=DEFAULT_MODE, open_mode="w"): """Creates an instance of :class:`Script`. When used as a context manager, the script is automatically saved on entry and removed on exit. :param path: The path where the script file will be created. :type path: str :param content: The content to write to the script file. :type content: str or bytes :param mode: File permissions mode. Defaults to 0775 (rwxrwxr-x). :type mode: int :param open_mode: File open mode ('w' for text, 'wb' for binary). :type open_mode: str """ self.path = path self.content = content self.mode = mode self.stored = False self.open_mode = open_mode def __repr__(self): return ( f'{self.__class__.__name__}(path="{self.path}", ' f"stored={self.stored})" ) def __str__(self): return self.path def __enter__(self): self.save() return self def __exit__(self, exc_type, exc_value, exc_traceback): self.remove() def save(self): """Store script content to the file system. Creates parent directories if they do not exist and sets the specified file permissions. :return: True if script has been stored successfully. :rtype: bool """ dirname = os.path.dirname(self.path) utils_path.init_dir(dirname) with open(self.path, self.open_mode) as fd: # pylint: disable=W1514 fd.write(self.content) os.chmod(self.path, self.mode) self.stored = True return self.stored def remove(self): """Remove script file from the file system. :return: True if the script file was removed, False if it did not exist. :rtype: bool """ if os.path.exists(self.path): os.remove(self.path) self.stored = False return True return False class TemporaryScript(Script): """Class that represents a temporary script in an auto-managed directory. The script is created in a temporary directory that is automatically cleaned up when the instance is garbage collected or when used as a context manager. """ # pylint: disable=R0913 def __init__( self, name, content, prefix="avocado_script", mode=DEFAULT_MODE, open_mode="w" ): """Creates an instance of :class:`TemporaryScript`. The script is created in a newly created temporary directory. When used as a context manager, both the script and its directory are automatically removed on exit. The directory is also removed when the object is garbage collected. :param name: The script filename (not the full path). :type name: str :param content: The content to write to the script file. :type content: str or bytes :param prefix: Prefix for the temporary directory name. :type prefix: str :param mode: File permissions mode. Defaults to 0775 (rwxrwxr-x). :type mode: int :param open_mode: File open mode ('w' for text, 'wb' for binary). :type open_mode: str """ tmpdir = tempfile.mkdtemp(prefix=prefix) super().__init__(os.path.join(tmpdir, name), content, mode, open_mode) def __del__(self): self.remove() def remove(self): if os.path.exists(os.path.dirname(self.path)): shutil.rmtree(os.path.dirname(self.path)) self.stored = False def make_script(path, content, mode=DEFAULT_MODE): """Creates and saves a new script file to the file system. This is a convenience function that creates a Script instance, saves it, and returns the path. :param path: The path where the script file will be created. :type path: str :param content: The content to write to the script file. :type content: str or bytes :param mode: File permissions mode. Defaults to 0775 (rwxrwxr-x). :type mode: int :return: The path to the created script file. :rtype: str """ scpt = Script(path, content, mode=mode) scpt.save() return scpt.path def make_temp_script(name, content, prefix="avocado_script", mode=DEFAULT_MODE): """Creates and saves a new temporary script in a temporary directory. This is a convenience function that creates a TemporaryScript instance and saves it. Note that the script's temporary directory will be removed when the TemporaryScript object is garbage collected. :param name: The script filename (not the full path). :type name: str :param content: The content to write to the script file. :type content: str or bytes :param prefix: Prefix for the temporary directory name. :type prefix: str :param mode: File permissions mode. Defaults to 0775 (rwxrwxr-x). :type mode: int :return: The full path to the created script file. :rtype: str """ scpt = TemporaryScript(name, content, prefix=prefix, mode=mode) scpt.save() return scpt.path # pylint: disable=wrong-import-position from avocado.utils.deprecation import log_deprecation log_deprecation.warning("script") avocado-113.0/avocado/utils/service.py000066400000000000000000000656611513561306000177360ustar00rootroot00000000000000# Copyright(c) 2013 Intel Corporation. # # This program is free software; you can redistribute it and/or modify it # under the terms and conditions of the GNU General Public License, # version 2, as published by the Free Software Foundation. # # This program is distributed in the hope it will be useful, but WITHOUT # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or # FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for # more details. # # You should have received a copy of the GNU General Public License along with # this program; if not, write to the Free Software Foundation, Inc., # 51 Franklin St - Fifth Floor, Boston, MA 02110-1301 USA. # # The full GNU General Public License is included in this distribution in # the file called "COPYING". # # This code was inspired in the autotest project, # # client/shared/service.py # Original author: Ross Brattain import logging import os import re from tempfile import mkstemp from avocado.utils import process LOG = logging.getLogger(__name__) _COMMAND_TABLE_DOC = """ Taken from http://fedoraproject.org/wiki/SysVinit_to_Systemd_Cheatsheet service frobozz start systemctl start frobozz.service Used to start a service (not reboot persistent) service frobozz stop systemctl stop frobozz.service Used to stop a service (not reboot persistent) service frobozz restart systemctl restart frobozz.service Used to stop and then start a service service frobozz reload systemctl reload frobozz.service When supported, reloads the config file without interrupting pending operations. service frobozz condrestart systemctl condrestart frobozz.service Restarts if the service is already running. service frobozz status systemctl status frobozz.service Tells whether a service is currently running. ls /etc/rc.d/init.d/ systemctl list-unit-files --type=service (preferred) Used to list the services that can be started or stopped ls /lib/systemd/system/*.service /etc/systemd/system/*.service Used to list all the services and other units chkconfig frobozz on systemctl enable frobozz.service Turn the service on, for start at next boot, or other trigger. chkconfig frobozz off systemctl disable frobozz.service Turn the service off for the next reboot, or any other trigger. chkconfig frobozz systemctl is-enabled frobozz.service Used to check whether a service is configured to start or not in the current environment. chkconfig --list systemctl list-unit-files --type=service(preferred) ls /etc/systemd/system/*.wants/ Print a table of services that lists which runlevels each is configured on or off chkconfig frobozz --list ls /etc/systemd/system/*.wants/frobozz.service Used to list what levels this service is configured on or off chkconfig frobozz --add systemctl daemon-reload Used when you create a new service file or modify any configuration """ def sys_v_init_result_parser(command): """ Parse results from sys_v style commands. command status: return true if service is running. command is_enabled: return true if service is enabled. command list: return a dict from service name to status. command others: return true if operate success. :param command: command. :type command: str. :return: different from the command. """ if command == "status": def status_method(cmd_result): """ Parse method for `service $name status`. Returns True if running. Returns False if stopped. Returns None if unrecognized. """ # If service is stopped, exit_status is also not zero. # So, we can't use exit_status to check result. # Returns None if unrecognized. if re.search(b"unrecognized", cmd_result.stderr.lower()): return None # Returns False if stopped. output = cmd_result.stdout.lower() dead_flags = [b"stopped", b"not running", b"dead"] for flag in dead_flags: if re.search(flag, output): return False # If output does not contain a dead flag, check it with "running". return bool(re.search(b"running", output)) return status_method if command == "list": def list_method(cmd_result): """ Parse method for `service $name list`. Return dict from service name to status. >>> {"sshd": {0: 'off', 1: 'off', 2: 'off', 3: 'off', 4: 'off', >>> 5: 'off', 6: 'off'}, >>> "vsftpd": {0: 'off', 1: 'off', 2: 'off', 3: 'off', 4: 'off', >>> 5: 'off', 6: 'off'}, >>> "xinetd": {'discard-dgram:': 'off', >>> 'rsync:': 'off'...'chargen-stream:': 'off'}, >>> ... >>> } """ if cmd_result.exit_status: raise process.CmdError(cmd_result.command, cmd_result) # The final dict to return. services_statuses_dict = {} # Dict to store status on every target for each service. status_per_target_dict = {} # Dict to store the status for service based on xinetd. xinet_services_dict = {} lines = cmd_result.stdout_text.strip().splitlines() for line in lines: sublines = line.strip().split() if len(sublines) == 8: # Service and status on each target. service_name = sublines[0] # Store the status of each target in status_per_target_dict for target in range(7): status = sublines[target + 1].split(":")[-1] status_per_target_dict[target] = status services_statuses_dict[service_name] = status_per_target_dict.copy() elif len(sublines) == 2: # Service based on xinetd. service_name = sublines[0].strip(":") status = sublines[-1] xinet_services_dict[service_name] = status else: # Header or some lines useless. continue # Add xinetd based service in the main dict. services_statuses_dict["xinetd"] = xinet_services_dict return services_statuses_dict return list_method return _ServiceResultParser.default_method def systemd_result_parser(command): """ Parse results from systemd style commands. command status: return true if service is running. command is_enabled: return true if service is enabled. command list: return a dict from service name to status. command others: return true if operate success. :param command: command. :type command: str. :return: different from the command. """ if command == "status": def status_method(cmd_result): """ Parse method for `systemctl status $name.service`. Returns True if running. Returns False if stopped. Returns None if not loaded. """ # If service is stopped, exit_status is also not zero. # So, we can't use exit_status to check result. output = cmd_result.stdout # Returns None if not loaded. if not re.search(b"Loaded: loaded", output): return None # Check it with Active status. return output.count(b"Active: active") > 0 return status_method if command == "list": def list_method(cmd_result): """ Parse method for `systemctl list $name.service`. Return a dict from service name to status. e.g: {"sshd": "enabled", "vsftpd": "disabled", "systemd-sysctl": "static", ... } """ if cmd_result.exit_status: raise process.CmdError(cmd_result.command, cmd_result) # Dict to store service name to status. _service2status_dict = {} lines = cmd_result.stdout_text.strip().splitlines() for line in lines: sublines = line.strip().split() if (not len(sublines) == 2) or (not sublines[0].endswith("service")): # Some lines useless. continue service_name = sublines[0].rstrip(".service") status = sublines[-1] _service2status_dict[service_name] = status return _service2status_dict return list_method return _ServiceResultParser.default_method def sys_v_init_command_generator(command): """ Generate lists of command arguments for sys_v style inits. :param command: start,stop,restart, etc. :type command: str :return: list of commands to pass to process.run or similar function :rtype: builtin.list """ command_name = "service" if command == "is_enabled": command_name = "chkconfig" command = "" elif command == "enable": command_name = "chkconfig" command = "on" elif command == "disable": command_name = "chkconfig" command = "off" elif command == "list": # noinspection PyUnusedLocal def _list_command(_): return ["chkconfig", "--list"] return _list_command elif command == "set_target": def _set_target_command(target): target = convert_systemd_target_to_runlevel(target) return ["telinit", target] return _set_target_command # Do not need reset failed, mask and unmask in sys_v style system. elif command in ["reset_failed", "mask", "unmask"]: def _true_command(_): return ["true"] return _true_command def _method(service_name): return [command_name, service_name, command] return _method def systemd_command_generator(command): """ Generate list of command line argument strings for systemctl. One argument per string for compatibility Popen WARNING: If systemctl detects that it is running on a tty it will use color, pipe to $PAGER, change column sizes and not truncate unit names. Use --no-pager to suppress pager output, or set PAGER=cat in the environment. You may need to take other steps to suppress color output. See https://bugzilla.redhat.com/show_bug.cgi?id=713567 :param command: start,stop,restart, etc. :type command: str :return: List of command and arguments to pass to process.run or similar functions :rtype: builtin.list """ command_name = "systemctl" if command == "is_enabled": command = "is-enabled" elif command == "list": # noinspection PyUnusedLocal def _list_command(_): # systemctl pipes to `less` or $PAGER by default. Workaround this # add '--full' to avoid systemctl truncates service names. return [ command_name, "list-unit-files", "--type=service", "--no-pager", "--full", ] return _list_command elif command == "set_target": def _set_target_command(target): return [command_name, "isolate", target] return _set_target_command elif command == "reset_failed": command = "reset-failed" def _method(service_name): return [command_name, command, f"{service_name}.service"] return _method # mapping command/whether it requires root COMMANDS = ( ("start", True), ("stop", True), ("reload", True), ("restart", True), ("condrestart", True), ("status", False), ("enable", True), ("disable", True), ("is_enabled", False), ("list", False), ("set_target", True), ("reset_failed", True), ("mask", True), ("unmask", True), ) class _ServiceResultParser: # pylint: disable=too-few-public-methods """ A class that contains staticmethods to parse the result of service command. """ def __init__(self, result_parser, command_list=COMMANDS): """ Create staticmethods for each command in command_list using setattr. :param result_parser: function that generates functions that parse the result of command. :type result_parser: function :param command_list: list of all the commands, e.g. start, stop, restart, etc. :type command_list: list """ self.commands = command_list for command, _ in self.commands: setattr(self, command, result_parser(command)) @staticmethod def default_method(cmd_result): """ Default method to parse result from command which is not 'list'/'status' Returns True if command was executed successfully. """ if cmd_result.exit_status: LOG.debug(cmd_result) return False return True class _ServiceCommandGenerator: # pylint: disable=too-few-public-methods """ Generate command lists for starting/stopping services. """ def __init__(self, command_generator, command_list=COMMANDS): """ Create staticmethods for each command in command_list. :param command_generator: function that generates functions that generate lists of command strings :type command_generator: function :param command_list: list of all the commands, e.g. start, stop, restart, etc. :type command_list: list """ self.commands = command_list for command, _ in self.commands: setattr(self, command, command_generator(command)) def get_name_of_init(run=process.run): """ Internal function to determine what executable is PID 1 It does that by checking /proc/1/exe. Fall back to checking /proc/1/cmdline (local execution). :return: executable name for PID 1, aka init :rtype: str """ if run is process.run: # On a local run, there are better ways to check # our PID 1 executable name. try: return os.path.basename(os.readlink("/proc/1/exe")) except OSError: with open("/proc/1/cmdline", "r") as cmdline: # pylint: disable=W1514 init = cmdline.read().split(chr(0))[0] try: init = os.readlink(init) except OSError: pass return os.path.basename(init) else: output = run("readlink /proc/1/exe").stdout.strip() if isinstance(output, bytes): output = output.decode(encoding="utf-8") return os.path.basename(output) class _SpecificServiceManager: # pylint: disable=too-few-public-methods def __init__( self, service_name, service_command_generator, service_result_parser, run=process.run, ): """ Create staticmethods that call process.run with the given service_name >>> my_generator = auto_create_specific_service_command_generator >>> lldpad = SpecificServiceManager("lldpad", my_generator()) >>> lldpad.start() >>> lldpad.stop() :param service_name: init service name or systemd unit name :type service_name: str :param service_command_generator: a sys_v_init or systemd command generator :type service_command_generator: _ServiceCommandGenerator :param run: function that executes the commands and return CmdResult object, default process.run :type run: function """ for cmd, requires_root in service_command_generator.commands: run_func = run parse_func = getattr(service_result_parser, cmd) command = getattr(service_command_generator, cmd) setattr( self, cmd, self.generate_run_function( run_func=run_func, parse_func=parse_func, command=command, service_name=service_name, requires_root=requires_root, ), ) @staticmethod def generate_run_function( run_func, parse_func, command, service_name, requires_root ): """ Generate the wrapped call to process.run for the given service_name. :param run_func: function to execute command and return CmdResult object. :type run_func: function :param parse_func: function to parse the result from run. :type parse_func: function :param command: partial function that generates the command list :type command: function :param service_name: init service name or systemd unit name :type service_name: str :param requires_root: whether command needs superuser privileges :type requires_root: bool :return: wrapped process.run function. :rtype: function """ def run(**kwargs): """ Wrapped process.run invocation that will start/stop/etc a service. :param kwargs: extra arguments to process.run, .e.g. timeout. But not for ignore_status. We need a CmdResult to parse and raise an exception.TestError if command failed. We will not let the CmdError out. :return: result of parse_func. """ # If run_func is process.run by default, we need to set # ignore_status = True. Otherwise, skip this setting. if run_func is process.run: LOG.debug("Setting ignore_status to True.") kwargs["ignore_status"] = True if requires_root: LOG.debug("Setting sudo to True.") kwargs["sudo"] = True cmd = " ".join(command(service_name)) result = run_func(cmd, **kwargs) return parse_func(result) return run class _GenericServiceManager: # pylint: disable=too-few-public-methods """ Base class for SysVInitServiceManager and SystemdServiceManager. """ def __init__( self, service_command_generator, service_result_parser, run=process.run ): """ Create staticmethods for each service command, e.g. start, stop These staticmethods take as an argument the service to be started or stopped. >>> my_generator = auto_create_specific_service_command_generator >>> systemd = SpecificServiceManager(my_generator()) >>> systemd.start("lldpad") >>> systemd.stop("lldpad") :param service_command_generator: a sys_v_init or systemd command generator :type service_command_generator: _ServiceCommandGenerator :param run: function to call the run the commands, default process.run :type run: function """ for cmd, requires_root in service_command_generator.commands: parse_func = getattr(service_result_parser, cmd) command = getattr(service_command_generator, cmd) setattr( self, cmd, self.generate_run_function( run_func=run, parse_func=parse_func, command=command, requires_root=requires_root, ), ) @staticmethod def generate_run_function(run_func, parse_func, command, requires_root): """ Generate the wrapped call to process.run for the service command. :param run_func: process.run :type run_func: function :param command: partial function that generates the command list :type command: function :param requires_root: whether command needs superuser privileges :type requires_root: bool :return: wrapped process.run function. :rtype: function """ def run(service="", **kwargs): """ Wrapped process.run invocation that will start/stop/etc. a service. :param service: service name, e.g. crond, dbus, etc. :param kwargs: extra arguments to process.run, .e.g. timeout. But not for ignore_status. We need a CmdResult to parse and raise a exception.TestError if command failed. We will not let the CmdError out. :return: result of parse_func. """ # If run_func is process.run by default, we need to set # ignore_status = True. Otherwise, skip this setting. if run_func is process.run: LOG.debug("Setting ignore_status to True.") kwargs["ignore_status"] = True if requires_root: LOG.debug("Setting sudo to True.") kwargs["sudo"] = True cmd = " ".join(command(service)) result = run_func(cmd, **kwargs) return parse_func(result) return run class _SysVInitServiceManager( _GenericServiceManager ): # pylint: disable=too-few-public-methods """ Concrete class that implements the SysVInitServiceManager """ def convert_sysv_runlevel(level): """ Convert runlevel to systemd target. :param level: sys_v runlevel :type level: str or int :return: systemd target :rtype: str :raise ValueError: when runlevel is unknown """ runlevel = str(level) if runlevel == "0": target = "poweroff.target" elif runlevel in ["1", "s", "single"]: target = "rescue.target" elif runlevel in ["2", "3", "4"]: target = "multi-user.target" elif runlevel == "5": target = "graphical.target" elif runlevel == "6": target = "reboot.target" else: raise ValueError(f"unknown runlevel {level}") return target def convert_systemd_target_to_runlevel(target): """ Convert systemd target to runlevel. :param target: systemd target :type target: str :return: sys_v runlevel :rtype: str :raise ValueError: when systemd target is unknown """ if target == "poweroff.target": runlevel = "0" elif target == "rescue.target": runlevel = "s" elif target == "multi-user.target": runlevel = "3" elif target == "graphical.target": runlevel = "5" elif target == "reboot.target": runlevel = "6" else: raise ValueError(f"unknown target {target}") return runlevel class _SystemdServiceManager(_GenericServiceManager): """ Concrete class that implements the SystemdServiceManager """ @staticmethod def change_default_runlevel(runlevel="multi-user.target"): # atomic symlinking, symlink and then rename """ Set the default systemd target. Create the symlink in a temp directory and then use atomic rename to move the symlink into place. :param runlevel: default systemd target :type runlevel: str """ tmp_symlink = mkstemp(dir="/etc/systemd/system") os.symlink(f"/usr/lib/systemd/system/{runlevel}", tmp_symlink) os.rename(tmp_symlink, "/etc/systemd/system/default.target") _COMMAND_GENERATORS = { "init": sys_v_init_command_generator, "systemd": systemd_command_generator, } _RESULT_PARSERS = {"init": sys_v_init_result_parser, "systemd": systemd_result_parser} _SERVICE_MANAGERS = {"init": _SysVInitServiceManager, "systemd": _SystemdServiceManager} def service_manager(run=process.run): """ Detect which init program is being used, init or systemd and return a class has methods to start/stop services. Example of use: .. code-block:: python # Get the system service manager service_manager = ServiceManager() # Stating service/unit "sshd" service_manager.start("sshd") # Getting a list of available units units = service_manager.list() # Disabling and stopping a list of services services_to_disable = ['ntpd', 'httpd'] for s in services_to_disable: service_manager.disable(s) service_manager.stop(s) :return: SysVInitServiceManager or SystemdServiceManager :rtype: _GenericServiceManager """ init = get_name_of_init(run) internal_service_manager = _SERVICE_MANAGERS[init] internal_command_generator = _COMMAND_GENERATORS[init] internal_result_parser = _RESULT_PARSERS[init] internal_generator = _ServiceCommandGenerator(internal_command_generator) internal_parser = _ServiceResultParser(internal_result_parser) _service_manager = internal_service_manager( internal_generator, internal_parser, run=run ) return _service_manager ServiceManager = service_manager def _auto_create_specific_service_result_parser(run=process.run): """ Create a class that will create partial functions that generate result_parser for the current init command. :return: A ServiceResultParser for the auto-detected init command. :rtype: _ServiceResultParser """ result_parser = _RESULT_PARSERS[get_name_of_init(run)] # remove list method command_list = [(c, r) for (c, r) in COMMANDS if c not in ["list", "set_target"]] return _ServiceResultParser(result_parser, command_list) def _auto_create_specific_service_command_generator(run=process.run): """ Create a class that will create partial functions that generate commands for the current init command. >>> my_generator = auto_create_specific_service_command_generator >>> lldpad = SpecificServiceManager("lldpad", my_generator()) >>> lldpad.start() >>> lldpad.stop() :return: A ServiceCommandGenerator for the auto-detected init command. :rtype: _ServiceCommandGenerator """ command_generator = _COMMAND_GENERATORS[get_name_of_init(run)] # remove list method command_list = [(c, r) for (c, r) in COMMANDS if c not in ["list", "set_target"]] return _ServiceCommandGenerator(command_generator, command_list) def specific_service_manager(service_name, run=process.run): """Get the service manager for a specific service. Example of use: .. code-block:: python # Get the specific service manager for sshd sshd = SpecificServiceManager("sshd") sshd.start() sshd.stop() sshd.reload() sshd.restart() sshd.condrestart() sshd.status() sshd.enable() sshd.disable() sshd.is_enabled() :param service_name: systemd unit or init.d service to manager :type service_name: str :return: SpecificServiceManager that has start/stop methods :rtype: _SpecificServiceManager """ init = get_name_of_init(run) result_parser = _RESULT_PARSERS[init] specific_generator = _auto_create_specific_service_command_generator return _SpecificServiceManager( service_name, specific_generator(run), _ServiceResultParser(result_parser), run ) SpecificServiceManager = specific_service_manager avocado-113.0/avocado/utils/software_manager/000077500000000000000000000000001513561306000212325ustar00rootroot00000000000000avocado-113.0/avocado/utils/software_manager/__init__.py000066400000000000000000000000001513561306000233310ustar00rootroot00000000000000avocado-113.0/avocado/utils/software_manager/backends/000077500000000000000000000000001513561306000230045ustar00rootroot00000000000000avocado-113.0/avocado/utils/software_manager/backends/__init__.py000066400000000000000000000000001513561306000251030ustar00rootroot00000000000000avocado-113.0/avocado/utils/software_manager/backends/apt.py000066400000000000000000000221231513561306000241420ustar00rootroot00000000000000import logging import os import re import tempfile from avocado.utils import path as utils_path from avocado.utils import process from avocado.utils.software_manager.backends.dpkg import DpkgBackend log = logging.getLogger("avocado.utils.software_manager") class AptBackend(DpkgBackend): """ Implements the apt backend for software manager. Set of operations for the apt package manager, commonly found on Debian and Debian based distributions, such as Ubuntu Linux. """ def __init__(self): """ Initializes the base command and the debian package repository. """ super().__init__() executable = utils_path.find_command("apt-get") self.base_command = executable + " --yes --allow-unauthenticated" self.repo_file_path = "/etc/apt/sources.list.d/avocado.list" self.dpkg_force_confdef = ( '-o Dpkg::Options::="--force-confdef" ' '-o Dpkg::Options::="--force-confold"' ) cmd_result = process.run( "apt-get -v | head -1", ignore_status=True, verbose=False, shell=True ) out = cmd_result.stdout_text.strip() try: ver = re.findall(r"\d\S*", out)[0] except IndexError: ver = out self.pm_version = ver log.debug("apt-get version: %s", self.pm_version) # gdebi-core is necessary for local installation with dependency # handling if not self.check_installed("gdebi-core"): if not self.install("gdebi-core"): log.info( "SoftwareManager (AptBackend) can't install packages " "from local .deb files with dependency resolution: " "Package 'gdebi-core' could not be installed" ) os.environ["DEBIAN_FRONTEND"] = "noninteractive" def install(self, name): """ Installs package [name]. :param name: Package name. """ if os.path.isfile(name): i_cmd = utils_path.find_command("gdebi") + " -n -q " + name else: command = "install" i_cmd = " ".join( [self.base_command, self.dpkg_force_confdef, command, name] ) try: process.system(i_cmd, shell=True, sudo=True) return True except process.CmdError: return False def remove(self, name): """ Remove package [name]. :param name: Package name. """ command = "remove" flag = "--purge" r_cmd = self.base_command + " " + command + " " + flag + " " + name return self._run_cmd(r_cmd) def add_repo(self, repo): """ Add an apt repository. :param repo: Repository string. Example: 'deb http://archive.ubuntu.com/ubuntu/ maverick universe' """ def _add_repo_file(): add_cmd = f"bash -c \"echo '{repo}' > {self.repo_file_path}\"" process.system(add_cmd, shell=True, sudo=True) def _get_repo_file_contents(): with open(self.repo_file_path, "r", encoding="utf-8") as repo_file: return repo_file.read() if not os.path.isfile(self.repo_file_path): _add_repo_file() return True repo_file_contents = _get_repo_file_contents() if repo not in repo_file_contents: try: _add_repo_file() return True except process.CmdError: return False def remove_repo(self, repo): """ Remove an apt repository. :param repo: Repository string. Example: 'deb http://archive.ubuntu.com/ubuntu/ maverick universe' """ try: new_file_contents = [] with open(self.repo_file_path, "r", encoding="utf-8") as repo_file: for line in repo_file.readlines(): if line != repo: new_file_contents.append(line) new_file_contents = "\n".join(new_file_contents) prefix = "avocado_software_manager" with tempfile.NamedTemporaryFile("w", prefix=prefix) as tmp_file: tmp_file.write(new_file_contents) tmp_file.flush() # Sync the content process.system(f"cp {tmp_file.name} {self.repo_file_path}", sudo=True) return True except (OSError, process.CmdError) as details: log.error(details) return False def upgrade(self, name=None): """ Upgrade all packages of the system with eventual new versions. Optionally, upgrade individual packages. :param name: optional parameter wildcard spec to upgrade :type name: str """ ud_command = "update" ud_cmd = self.base_command + " " + ud_command try: process.system(ud_cmd, sudo=True) except process.CmdError: log.error("Apt package update failed") if name: up_command = "install --only-upgrade" up_cmd = " ".join( [self.base_command, self.dpkg_force_confdef, up_command, name] ) else: up_command = "upgrade" up_cmd = " ".join([self.base_command, self.dpkg_force_confdef, up_command]) try: process.system(up_cmd, shell=True, sudo=True) return True except process.CmdError: return False def provides(self, name): """ Return a list of packages that provide [name of package/file]. :param name: File name. """ try: command = utils_path.find_command("apt-file") except utils_path.CmdNotFoundError: self.install("apt-file") command = utils_path.find_command("apt-file") try: process.run(command + " update") except process.CmdError: log.error("Apt file cache update failed") fu_cmd = command + " search " + name try: paths = list(filter(None, os.environ["PATH"].split(":"))) provides = list(filter(None, process.run(fu_cmd).stdout_text.split("\n"))) list_provides = [] for each_path in paths: for line in provides: try: line = line.split(":") package = line[0].strip() lpath = line[1].strip() if ( lpath == os.path.join(each_path, name) and package not in list_provides ): list_provides.append(package) except IndexError: pass if len(list_provides) > 1: log.warning("More than one package found, opting by the first result") if list_provides: log.info("Package %s provides %s", list_provides[0], name) return list_provides[0] return None except process.CmdError: return None def get_source(self, name, path): """ Download source for provided package. Returns the path with source placed. :param name: parameter wildcard package to get the source for :return path: path of ready-to-build source """ if not self.check_installed("dpkg-dev"): if not self.install("dpkg-dev"): log.info( "SoftwareManager (AptBackend) can't install packages " "from local .deb files with dependency resolution: " "Package 'dpkg-dev' could not be installed" ) src_cmd = f"{self.base_command} source {name}" try: if self.build_dep(name): if not os.path.exists(path): os.makedirs(path) os.chdir(path) process.system_output(src_cmd) for subdir in os.listdir(path): if subdir.startswith(name) and os.path.isdir(subdir): return os.path.join(path, subdir) except process.CmdError as details: log.error("Apt package source failed %s", details) return "" def build_dep(self, name): """ Installed build-dependencies of a given package [name]. :param name: parameter package to install build-dependencies for. :return True: If packages are installed properly """ if not self.check_installed("dpkg-dev"): if not self.install("dpkg-dev"): log.info( "SoftwareManager (AptBackend) can't install packages " "from local .deb files with dependency resolution: " "Package 'dpkg-dev' could not be installed" ) src_cmd = f"{self.base_command} build-dep {name}" try: process.system_output(src_cmd) return True except process.CmdError as details: log.error("Apt package build-dep failed %s", details) return False avocado-113.0/avocado/utils/software_manager/backends/base.py000066400000000000000000000033171513561306000242740ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: IBM 2008-2009 # Copyright: Red Hat Inc. 2009-2014 # Author: Lucas Meneghel Rodrigues # Author: Higor Vieira Alves # Author: Ramon de Carvalho Valle # # This code was adapted from the autotest project, # client/shared/software_manager.py import logging from avocado.utils import process log = logging.getLogger("avocado.utils.software_manager") class BaseBackend: """ This class implements all common methods among backends. """ def install_what_provides(self, path): """ Installs package that provides [path]. :param path: Path to file. """ provides = self.provides(path) # pylint: disable=E1101 if provides is not None: return self.install(provides) # pylint: disable=E1101 log.warning("No package seems to provide %s", path) return False @staticmethod def _run_cmd(cmd): """ Helper for running a backend command. :param cmd: Command to run. """ try: process.system(cmd, sudo=True) return True except process.CmdError as details: log.error("Failed to run command: %s", details) return False avocado-113.0/avocado/utils/software_manager/backends/dnf.py000066400000000000000000000016561513561306000241350ustar00rootroot00000000000000import logging from avocado.utils import process from avocado.utils.software_manager.backends.yum import YumBackend log = logging.getLogger("avocado.utils.software_manager") class DnfBackend(YumBackend): """ Implements the dnf backend for software manager. DNF is the successor to yum in recent Fedora. """ def __init__(self): """ Initializes the base command and the DNF package repository. """ super().__init__(cmd="dnf") def build_dep(self, name): # pylint: disable=W0221 """ Install build-dependencies for package [name] :param name: name of the package :return True: If build dependencies are installed properly """ try: process.system(f"{self.base_command} builddep {name}", sudo=True) return True except process.CmdError as details: log.error(details) return False avocado-113.0/avocado/utils/software_manager/backends/dpkg.py000066400000000000000000000072721513561306000243130ustar00rootroot00000000000000import io import logging import os import re import tarfile from avocado.utils import ar from avocado.utils import path as utils_path from avocado.utils import process from avocado.utils.software_manager.backends.base import BaseBackend log = logging.getLogger("avocado.utils.software_manager") class DpkgBackend(BaseBackend): """ This class implements operations executed with the dpkg package manager. dpkg is a lower level package manager, used by higher level managers such as apt and aptitude. """ PACKAGE_TYPE = "deb" INSTALLED_OUTPUT = "install ok installed" def __init__(self): self.lowlevel_base_cmd = utils_path.find_command("dpkg") def check_installed(self, name): if os.path.isfile(name): n_cmd = self.lowlevel_base_cmd + " -f " + name + " Package" name = process.system_output(n_cmd) i_cmd = self.lowlevel_base_cmd + " -s " + name # Checking if package is installed package_status = process.run(i_cmd, ignore_status=True).stdout_text dpkg_installed = self.INSTALLED_OUTPUT in package_status if dpkg_installed: return True return False @staticmethod def list_all(): """ List all packages available in the system. """ log.debug("Listing all system packages (may take a while)") installed_packages = [] cmd_result = process.run("dpkg -l", verbose=False) out = cmd_result.stdout_text.strip() raw_list = out.splitlines()[5:] for line in raw_list: parts = line.split() if parts[0] == "ii": # only grab "installed" packages installed_packages.append(f"{parts[1]}-{parts[2]}") return installed_packages @staticmethod def is_valid(package_path): """Verifies if a package is a valid deb file. :param str package_path: .deb package path. :returns: True if valid, otherwise false. :rtype: bool """ abs_path = os.path.abspath(os.path.expanduser((package_path))) member_regexes = [r"debian-binary", r"control\.tar\..*", r"data\.tar\..*"] members = ar.Ar(abs_path).list() if len(members) != len(member_regexes): return False for regex, member in zip(member_regexes, members): if not re.match(regex, member): return False return True @staticmethod def extract_from_package(package_path, dest_path=None): """Extracts the package content to a specific destination path. :param str package_path: path to the deb package. :param dest_path: destination path to extract the files. Default is the current directory. :returns: path of the extracted file :returns: the path of the extracted files. :rtype: str """ abs_path = os.path.abspath(os.path.expanduser(package_path)) dest = dest_path or os.path.curdir os.makedirs(dest, exist_ok=True) archive = ar.Ar(abs_path) data_tarball_name = archive.list()[2] member_data = archive.read_member(data_tarball_name) with tarfile.open(fileobj=io.BytesIO(member_data)) as tarball: tarball.extractall(dest) return dest def list_files(self, package): """ List files installed by package [package]. :param package: Package name. :return: List of paths installed by package. """ if os.path.isfile(package): l_cmd = self.lowlevel_base_cmd + " -c " + package else: l_cmd = self.lowlevel_base_cmd + " -l " + package return process.system_output(l_cmd).decode().split("\n") avocado-113.0/avocado/utils/software_manager/backends/rpm.py000066400000000000000000000235561513561306000241670ustar00rootroot00000000000000import logging import os import re from avocado.utils import path as utils_path from avocado.utils import process from avocado.utils.software_manager.backends.base import BaseBackend LOG = logging.getLogger(__name__) class RpmBackend(BaseBackend): """ This class implements operations executed with the rpm package manager. rpm is a lower level package manager, used by higher level managers such as yum and zypper. """ PACKAGE_TYPE = "rpm" SOFTWARE_COMPONENT_QRY = ( PACKAGE_TYPE + " " + "%{NAME} %{VERSION} %{RELEASE} %{SIGMD5} %{ARCH}" ) def __init__(self): self.lowlevel_base_cmd = utils_path.find_command("rpm") def _check_installed_version(self, name, version): """ Helper for the check_installed public method. :param name: Package name. :param version: Package version. """ cmd = self.lowlevel_base_cmd + " -q --qf %{VERSION} " + name inst_version = process.run(cmd, ignore_status=True).stdout_text if "not installed" in inst_version: return False return bool(inst_version >= version) def check_installed(self, name, version=None, arch=None): """ Check if package [name] is installed. :param name: Package name. :param version: Package version. :param arch: Package architecture. """ if arch: cmd = self.lowlevel_base_cmd + " -q --qf %{ARCH} " + name inst_archs = process.system_output(cmd, ignore_status=True) inst_archs = inst_archs.split("\n") for inst_arch in inst_archs: if inst_arch == arch: return self._check_installed_version(name, version) return False if version: return self._check_installed_version(name, version) cmd = "rpm -q " + name try: process.system(cmd) return True except process.CmdError: return False def list_all(self, software_components=True): """ List all installed packages. :param software_components: log in a format suitable for the SoftwareComponent schema """ LOG.debug("Listing all system packages (may take a while)") if software_components: cmd_format = "rpm -qa --qf '%s' | sort" query_format = f"{self.SOFTWARE_COMPONENT_QRY}\n" cmd_format %= query_format cmd_result = process.run(cmd_format, verbose=False, shell=True) else: cmd_result = process.run("rpm -qa | sort", verbose=False, shell=True) out = cmd_result.stdout_text.strip() installed_packages = out.splitlines() return installed_packages @staticmethod def list_files(name): """ List files installed on the system by package [name]. :param name: Package name. """ path = os.path.abspath(name) if os.path.isfile(path): option = "-qlp" name = path else: option = "-ql" l_cmd = "rpm" + " " + option + " " + name try: result = process.system_output(l_cmd) list_files = result.split("\n") return list_files except process.CmdError: return [] @staticmethod def rpm_install(file_path, no_dependencies=False, replace=False): """ Install the rpm file [file_path] provided. :param str file_path: file path of the installed package :param bool no_dependencies: whether to add "nodeps" flag :param bool replace: whether to replace existing package :returns: whether file is installed properly :rtype: bool """ if not os.path.isfile(file_path): LOG.warning("Please provide proper rpm path") return False nodeps = "--nodeps " if no_dependencies else "" update = "-U" if replace else "-i" cmd = f"rpm {update} {nodeps}{file_path}" try: process.system(cmd) return True except process.CmdError as details: LOG.error(details) return False @staticmethod def rpm_verify(package_name): """ Verify an RPM package with an installed one. :param str package_name: name of the verified package :returns: whether the verification was successful :rtype: bool """ LOG.info("Verifying package information.") cmd = "rpm -V " + package_name result = process.run(cmd, ignore_status=True) # unstable approach but currently works # installed_pattern = r"\s" + package_name + r" is installed\s+" # match = re.search(installed_pattern, result) if not result.exit_status: LOG.info("Verification successful.") return True LOG.info(result.stdout_text.rstrip()) return False @staticmethod def rpm_erase(package_name): """ Erase an RPM package. :param str package_name: name of the erased package :returns: whether file is erased properly :rtype: bool """ LOG.warning("Erasing rpm package %s", package_name) cmd = "rpm -e " + package_name result = process.run(cmd, ignore_status=True) if result.exit_status: return False return True @staticmethod def prepare_source(spec_file, dest_path=None, build_option=None): """ Rpmbuild the spec path and return build dir :param spec_path: spec path to install :param build_option: rpmbuild option :return path: build directory """ if build_option is None: build_option = "-bc" if dest_path is not None: build_option += f" --define '_builddir {dest_path}'" else: LOG.error("Please provide a valid path") return "" try: process.system(f"rpmbuild {build_option} {spec_file}") return os.path.join(dest_path, os.listdir(dest_path)[0]) except process.CmdError as details: LOG.error(details) return "" def find_rpm_packages(self, rpm_dir): """ Extract product dependencies from a defined RPM directory and all its subdirectories. :param str rpm_dir: directory to search in :returns: found RPM packages :rtype: [str] """ subpaths = os.listdir(rpm_dir) subpacks = [] for subpath in subpaths: if subpath in (".", ".."): continue new_filepath = rpm_dir + "/" + subpath LOG.debug("Checking path for rpm %s", new_filepath) # if path is file validate name and inject if os.path.isfile(new_filepath) and re.search( r"\s*.rpm$", os.path.basename(new_filepath) ): LOG.info("Marking package %s for setup", new_filepath) subpacks.append(new_filepath) elif os.path.isdir(new_filepath): subpacks += self.find_rpm_packages(new_filepath) return subpacks @staticmethod def is_valid(package_path): """Verifies if a package is a valid rpm file. :param str package_path: .rpm package path. :returns: True if valid, otherwise false. :rtype: bool """ abs_path = os.path.abspath(os.path.expanduser((package_path))) try: result = process.run(f"rpm -qp {abs_path}") except process.CmdError: return False if not result.exit_status: return True return False @staticmethod def extract_from_package(package_path, dest_path=None): """Extracts the package content to a specific destination path. :param str package_path: path to the rpm package. :param dest_path: destination path to extract the files. Default it will be the current directory. :returns: path of the extracted file :returns: the path of the extracted files. :rtype: str """ abs_path = os.path.abspath(os.path.expanduser(package_path)) dest = dest_path or os.path.curdir # If something goes wrong process.run will raise a CmdError exception process.run(f"rpm2cpio {abs_path} | cpio -dium -D {dest}", shell=True) return dest def perform_setup(self, packages, no_dependencies=False): """ General RPM setup with automatic handling of dependencies based on install attempts. :param packages: the RPM packages to install in dependency-friendly order :type packages: [str] :returns: whether setup completed successfully :rtype: bool """ while len(packages) > 0: LOG.debug("Trying to install: %s", packages) failed_packages = [] for package_path in packages: package_file = os.path.basename(package_path) package_name = "-".join(package_file.split("-")[0:-2]) LOG.debug("%s -> %s", package_file, package_name) installed = self.check_installed(package_name) verified = self.rpm_verify(package_name) if installed else False if installed and not verified: self.rpm_erase(package_name) if not installed or not verified: success = self.rpm_install(package_path, no_dependencies) if not success: failed_packages.append(package_path) if len(packages) == len(failed_packages) > 0: LOG.warning( "Some of the rpm packages could not be installed: %s", ", ".join(failed_packages), ) return False packages = failed_packages return True avocado-113.0/avocado/utils/software_manager/backends/yum.py000066400000000000000000000235611513561306000241770ustar00rootroot00000000000000import configparser import logging import os import re import shutil import tempfile from avocado.utils import data_factory from avocado.utils import path as utils_path from avocado.utils import process from avocado.utils.software_manager.backends.rpm import RpmBackend try: import yum except ImportError: HAS_YUM_MODULE = False else: HAS_YUM_MODULE = True log = logging.getLogger("avocado.utils.software_manager") class YumBackend(RpmBackend): """ Implements the yum backend for software manager. Set of operations for the yum package manager, commonly found on Yellow Dog Linux and Red Hat based distributions, such as Fedora and Red Hat Enterprise Linux. """ #: Path to the repository managed by Avocado REPO_FILE_PATH = "/etc/yum.repos.d/avocado-managed.repo" def __init__(self, cmd="yum"): """ Initializes the base command and the yum package repository. """ super().__init__() self.cmd = cmd self.base_command = f"{utils_path.find_command(cmd)} -y " self._cfgparser = None self._set_version(cmd) self._yum_base = None self.transient_opts = "" # Inspect the image mode of RHEL/CentOS/Fedora if os.path.isdir("/ostree"): self.transient_opts += " --transient " @property def repo_config_parser(self): if self._cfgparser is None: self._cfgparser = configparser.ConfigParser() self._cfgparser.read(self.REPO_FILE_PATH) return self._cfgparser @property def yum_base(self): if self._yum_base is None: if HAS_YUM_MODULE: self._yum_base = yum.YumBase() else: log.debug( "%s module for Python is required to use the " "'provides' command. Using the basic support " "from rpm and %s commands", self.cmd, self.cmd, ) return self._yum_base @staticmethod def _cleanup(): """ Clean up the yum cache so new package information can be downloaded. """ process.system("yum clean all", sudo=True) def _set_version(self, cmd): result = process.run( self.base_command + "--version", verbose=False, ignore_status=True ) first_line = result.stdout_text.splitlines()[0].strip() try: ver = re.findall(r"\d*.\d*.\d*", first_line)[0] except IndexError: ver = first_line self.pm_version = ver log.debug("%s version: %s", cmd, self.pm_version) def install(self, name): """ Installs package [name]. Handles local installs. """ i_cmd = self.base_command + "install" + " " + name + self.transient_opts return self._run_cmd(i_cmd) def remove(self, name): """ Removes package [name]. :param name: Package name (eg. 'ipython'). """ r_cmd = self.base_command + "erase" + " " + name + self.transient_opts return self._run_cmd(r_cmd) def add_repo(self, url, **opt_params): """ Adds package repository located on [url]. :param url: Universal Resource Locator of the repository. :param opt_params: Dict for optional repo options, eg. {"priority": "1"}. """ # Set default values for dnf repo options repo_options = {"enabled": "1", "gpgcheck": "0"} repo_options.update(opt_params) # Check if we URL is already set for section in self.repo_config_parser.sections(): for option, value in self.repo_config_parser.items(section): if option == "baseurl" and value == url: return True # Didn't find it, let's set it up while True: section_name = "software_manager" + "_" section_name += data_factory.generate_random_string(4) if not self.repo_config_parser.has_section(section_name): break try: self.repo_config_parser.add_section(section_name) self.repo_config_parser.set( section_name, "name", "Avocado managed repository" ) self.repo_config_parser.set(section_name, "baseurl", url) for opt_key, opt_value in repo_options.items(): self.repo_config_parser.set(section_name, opt_key, opt_value) prefix = "avocado_software_manager" with tempfile.NamedTemporaryFile("w", prefix=prefix) as tmp_file: self.repo_config_parser.write(tmp_file, space_around_delimiters=False) tmp_file.flush() # Sync the content process.system(f"cp {tmp_file.name} {self.REPO_FILE_PATH}", sudo=True) return True except (OSError, process.CmdError) as details: log.error(details) return False def remove_repo(self, url): """ Removes package repository located on [url]. :param url: Universal Resource Locator of the repository. """ try: prefix = "avocado_software_manager" with tempfile.NamedTemporaryFile("w", prefix=prefix) as tmp_file: for section in self.repo_config_parser.sections(): for option, value in self.repo_config_parser.items(section): if option == "baseurl" and value == url: self.repo_config_parser.remove_section(section) self.repo_config_parser.write( tmp_file.file, space_around_delimiters=False ) tmp_file.flush() # Sync the content process.system(f"cp {tmp_file.name} {self.REPO_FILE_PATH}", sudo=True) return True except (OSError, process.CmdError) as details: log.error(details) return False def upgrade(self, name=None): """ Upgrade all available packages. Optionally, upgrade individual packages. :param name: optional parameter wildcard spec to upgrade :type name: str """ if not name: r_cmd = self.base_command + "update" + self.transient_opts else: r_cmd = self.base_command + "update" + " " + name + self.transient_opts return self._run_cmd(r_cmd) def provides(self, name): """ Returns a list of packages that provides a given capability. :param name: Capability name (eg, 'foo'). """ if self.yum_base is None: log.error( "The method 'provides' is disabled, " "%s module is required for this operation", self.cmd, ) return None try: # Python API need to be passed globs along with name for searching # all possible occurrences of pattern 'name' d_provides = self.yum_base.searchPackageProvides(args=["*/" + name]) except Exception as exc: # pylint: disable=W0703 log.error("Error searching for package that provides %s: %s", name, exc) d_provides = [] provides_list = list(d_provides) if provides_list: return str(provides_list[0]) return None @staticmethod def build_dep(name): """ Install build-dependencies for package [name] :param name: name of the package :return True: If build dependencies are installed properly """ try: process.system(f"yum-builddep -y --tolerant {name}", sudo=True) return True except process.CmdError as details: log.error(details) return False def get_source(self, name, dest_path, build_option=None): """ Downloads the source package and prepares it in the given dest_path to be ready to build. :param name: name of the package :param dest_path: destination_path :param build_option: rpmbuild option :return final_dir: path of ready-to-build directory """ path = tempfile.mkdtemp(prefix="avocado_software_manager") try: if dest_path is None: log.error("Please provide a valid path") return "" for pkg in ["rpm-build", "yum-utils"]: if not self.check_installed(pkg): if not self.install(pkg): log.error( "SoftwareManager (YumBackend) can't get " "packageswith dependency resolution: Package" " '%s' could not be installed", pkg, ) return "" try: process.run( f"yumdownloader --assumeyes --verbose " f"--source {name} --destdir {path}" ) src_rpms = [_ for _ in next(os.walk(path))[2] if _.endswith(".src.rpm")] if len(src_rpms) != 1: log.error( "Failed to get downloaded src.rpm from %s:\n%s", path, next(os.walk(path))[2], ) elif self.rpm_install(os.path.join(path, src_rpms[-1])): spec_path = os.path.join( os.environ["HOME"], "rpmbuild", "SPECS", f"{name}.spec" ) if self.build_dep(spec_path): return self.prepare_source(spec_path, dest_path, build_option) log.error("Installing build dependencies failed") else: log.error("Installing source rpm failed") except process.CmdError as details: log.error(details) return "" finally: shutil.rmtree(path) avocado-113.0/avocado/utils/software_manager/backends/zypper.py000066400000000000000000000126401513561306000247120ustar00rootroot00000000000000import logging import re from avocado.utils import path as utils_path from avocado.utils import process from avocado.utils.software_manager.backends.rpm import RpmBackend log = logging.getLogger("avocado.utils.software_manager") class ZypperBackend(RpmBackend): """ Implements the zypper backend for software manager. Set of operations for the zypper package manager, found on SUSE Linux. """ def __init__(self): """ Initializes the base command and the zypper package repository. """ super().__init__() self.base_command = utils_path.find_command("zypper") + " -n" z_cmd = self.base_command + " --version" cmd_result = process.run(z_cmd, ignore_status=True, verbose=False) out = cmd_result.stdout_text.strip() try: ver = re.findall(r"\d.\d*.\d*", out)[0] except IndexError: ver = out self.pm_version = ver log.debug("Zypper version: %s", self.pm_version) def install(self, name): """ Installs package [name]. Handles local installs. :param name: Package Name. """ i_cmd = self.base_command + " install -l " + name return self._run_cmd(i_cmd) def add_repo(self, url): """ Adds repository [url]. :param url: URL for the package repository. """ ar_cmd = self.base_command + " addrepo " + url return self._run_cmd(ar_cmd) def remove_repo(self, url): """ Removes repository [url]. :param url: URL for the package repository. """ rr_cmd = self.base_command + " removerepo " + url self._run_cmd(rr_cmd) def remove(self, name): """ Removes package [name]. """ r_cmd = self.base_command + " " + "remove" + " " + name return self._run_cmd(r_cmd) def upgrade(self, name=None): """ Upgrades all packages of the system. Optionally, upgrade individual packages. :param name: Optional parameter wildcard spec to upgrade :type name: str """ if not name: u_cmd = self.base_command + " update -l" else: u_cmd = self.base_command + " " + "update" + " " + name return self._run_cmd(u_cmd) # pylint: disable=R0801 def provides(self, name): """ Searches for what provides a given file. :param name: File path. """ p_cmd = self.base_command + " what-provides " + name list_provides = [] try: p_output = process.system_output(p_cmd).split("\n")[4:] for line in p_output: line = [a.strip() for a in line.split("|")] try: # state, pname, type, version, arch, repository = line pname = line[1] if pname not in list_provides: list_provides.append(pname) except IndexError: pass if len(list_provides) > 1: log.warning( "More than one package found, opting by the first queue result" ) if list_provides: log.info("Package %s provides %s", list_provides[0], name) return list_provides[0] return None except process.CmdError: return None def build_dep(self, name): """Return True if build-dependencies are installed for provided package Keyword argument: name -- name of the package """ s_cmd = f"{self.base_command} source-install -d {name}" try: process.system(s_cmd, sudo=True) return True except process.CmdError: log.error("Installing dependencies failed") return False def _source_install(self, name): """ Source install the given package [name] Returns the SPEC file of the package :param name: name of the package :return path: path of the spec file """ s_cmd = f"{self.base_command} source-install {name}" try: process.system(s_cmd, sudo=True) if self.build_dep(name): return f"/usr/src/packages/SPECS/{name}.spec" except process.CmdError: log.error("Installing source failed") return "" def get_source(self, name, dest_path, build_option=None): """ Downloads the source package and prepares it in the given dest_path to be ready to build :param name: name of the package :param dest_path: destination_path :param build_option: rpmbuild option :return final_dir: path of ready-to-build directory """ if not self.check_installed("rpm-build"): if not self.install("rpm-build"): log.error( "SoftwareManager (RpmBackend) can't get packages" "with dependency resolution: Package 'rpm-build'" "could not be installed" ) return "" try: spec_path = self._source_install(name) if spec_path: return self.prepare_source(spec_path, dest_path, build_option) log.error("Source not installed properly") return "" except process.CmdError as details: log.error(details) return "" avocado-113.0/avocado/utils/software_manager/distro_packages.py000066400000000000000000000051131513561306000247460ustar00rootroot00000000000000import logging import os from avocado.utils import distro from avocado.utils.software_manager.manager import SoftwareManager log = logging.getLogger("avocado.utils.software_manager") def install_distro_packages(distro_pkg_map, interactive=False): """ Installs packages for the currently running distribution This utility function checks if the currently running distro is a key in the distro_pkg_map dictionary, and if there is a list of packages set as its value. If these conditions match, the packages will be installed using the software manager interface, thus the native packaging system if the currently running distro. :type distro_pkg_map: dict :param distro_pkg_map: mapping of distro name, as returned by utils.get_os_vendor(), to a list of package names :return: True if any packages were actually installed, False otherwise """ if not interactive: os.environ["DEBIAN_FRONTEND"] = "noninteractive" result = False pkgs = [] detected_distro = distro.detect() distro_specs = [spec for spec in distro_pkg_map if isinstance(spec, distro.Spec)] for distro_spec in distro_specs: if ( distro_spec.name == detected_distro.name and distro_spec.arch == detected_distro.arch and int(detected_distro.version) >= distro_spec.min_version and int(detected_distro.release) >= distro_spec.min_release ): pkgs = distro_pkg_map[distro_spec] break if not pkgs: log.info("No specific distro release package list") # when comparing distro names only, fallback to a lowercase version # of the distro name is it's more common than the case as detected pkgs = distro_pkg_map.get(detected_distro.name, None) if not pkgs: pkgs = distro_pkg_map.get(detected_distro.name.lower(), None) if not pkgs: log.error("No generic distro package list") log.error( "No packages found for %s %s %s %s", detected_distro.name, detected_distro.arch, detected_distro.version, detected_distro.release, ) if pkgs: needed_pkgs = [] software_manager = SoftwareManager() for pkg in pkgs: if not software_manager.check_installed(pkg): needed_pkgs.append(pkg) if needed_pkgs: text = " ".join(needed_pkgs) log.info('Installing packages "%s"', text) result = software_manager.install(text) return result avocado-113.0/avocado/utils/software_manager/inspector.py000066400000000000000000000056331513561306000236210ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: IBM 2008-2009 # Copyright: Red Hat Inc. 2009-2014 # Author: Lucas Meneghel Rodrigues # Author: Higor Vieira Alves # Author: Ramon de Carvalho Valle # # This code was adapted from the autotest project, # client/shared/software_manager.py from avocado.utils import distro from avocado.utils import path as utils_path from avocado.utils.software_manager.backends.apt import AptBackend from avocado.utils.software_manager.backends.dnf import DnfBackend from avocado.utils.software_manager.backends.yum import YumBackend from avocado.utils.software_manager.backends.zypper import ZypperBackend #: Mapping of package manager name to implementation class. SUPPORTED_PACKAGE_MANAGERS = { "apt-get": AptBackend, "yum": YumBackend, "dnf": DnfBackend, "zypper": ZypperBackend, } class SystemInspector: """ System inspector class. This may grow up to include more complete reports of operating system and machine properties. """ def __init__(self): """ Probe system, and save information for future reference. """ self.distro = distro.detect().name def get_package_management(self): """ Determine the supported package management systems present on the system. If more than one package management system installed, try to find the best supported system. """ list_supported = [] for high_level_pm in SUPPORTED_PACKAGE_MANAGERS: try: utils_path.find_command(high_level_pm) list_supported.append(high_level_pm) except utils_path.CmdNotFoundError: pass pm_supported = None if not list_supported: pm_supported = None if len(list_supported) == 1: pm_supported = list_supported[0] elif len(list_supported) > 1: if "apt-get" in list_supported and self.distro in ("debian", "ubuntu"): pm_supported = "apt-get" elif "dnf" in list_supported and self.distro in ("rhel", "fedora"): pm_supported = "dnf" elif "yum" in list_supported and self.distro in ("rhel", "fedora"): pm_supported = "yum" elif "zypper" in list_supported and self.distro == "SuSE": pm_supported = "zypper" else: pm_supported = list_supported[0] return pm_supported avocado-113.0/avocado/utils/software_manager/main.py000066400000000000000000000133721513561306000225360ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: IBM 2008-2009 # Copyright: Red Hat Inc. 2009-2014 # Author: Lucas Meneghel Rodrigues # Author: Higor Vieira Alves # Author: Ramon de Carvalho Valle # # This code was adapted from the autotest project, # client/shared/software_manager.py """ Software package management library. This is an abstraction layer on top of the existing distributions high level package managers. It supports package operations useful for testing purposes, and multiple high level package managers (here called backends). """ import argparse import logging from avocado.utils import exit_codes from avocado.utils.software_manager.manager import SoftwareManager log = logging.getLogger("avocado.utils.software_manager") MESSAGES = { "install": { "success": "Package(s) %s installed successfully", "fail": "Failed to install %s. Check the package(s) name and if sudo" " permission is granted.", }, "remove": { "success": "Package(s) %s removed successfully", "fail": "Failed to remove %s. Check the package(s) name and if sudo" " permission is granted.", }, "check-installed": { "success": "Package %s already installed", "fail": "Package %s not installed", }, "add-repo": { "success": "Repo %s added successfully", "fail": "Failed add repo %s. Check the repo name and if sudo" " permission is granted.", }, "remove-repo": { "success": "Repo %s removed successfully", "fail": "Failed to remove repo %s. Check the repo name and if sudo" " permission is granted.", }, "upgrade": "Package manager upgrade successful", "what-provides": "Package %s provides %s", "install-what-provides": "Installed successfully what provides %s", } # pylint: disable=R0914 def main(): def install(): if software_manager.install(args): log.info(MESSAGES[action]["success"], args) return exit_codes.UTILITY_OK log.error(MESSAGES[action]["fail"], args) return exit_codes.UTILITY_FAIL def remove(): if software_manager.remove(args): log.info(MESSAGES[action]["success"], args) return exit_codes.UTILITY_OK log.error(MESSAGES[action]["fail"], args) return exit_codes.UTILITY_FAIL def check_installed(): if software_manager.check_installed(args): log.info(MESSAGES[action]["success"], args) return exit_codes.UTILITY_OK log.info(MESSAGES[action]["fail"], args) return exit_codes.UTILITY_FAIL def list_all(): for pkg in software_manager.list_all(): log.info(pkg) return exit_codes.UTILITY_OK def list_files(): for f in software_manager.list_files(args): log.info(f) return exit_codes.UTILITY_OK def add_repo(): if software_manager.add_repo(args): log.info(MESSAGES[action]["success"], args) return exit_codes.UTILITY_OK log.error(MESSAGES[action]["fail"], args) return exit_codes.UTILITY_FAIL def remove_repo(): if software_manager.remove_repo(args): log.info(MESSAGES[action]["success"], args) return exit_codes.UTILITY_OK log.error(MESSAGES[action]["fail"], args) return exit_codes.UTILITY_FAIL def upgrade(): if software_manager.upgrade(): log.info(MESSAGES[action]) return exit_codes.UTILITY_OK def what_provides(): provides = software_manager.provides(args) if provides is not None: log.info(MESSAGES[action], provides, args) return exit_codes.UTILITY_OK def install_what_provides(): if software_manager.install_what_provides(args): log.info(MESSAGES[action], args) return exit_codes.UTILITY_OK def show_help(): parser.print_help() return exit_codes.UTILITY_OK parser = argparse.ArgumentParser( "install|remove|check-installed|list-all|list-files|add-repo|" "remove-repo|upgrade|what-provides|install-what-provides arguments" ) parser.add_argument( "--verbose", dest="debug", action="store_true", help="include debug messages in console output", ) namespace, args = parser.parse_known_args() if namespace.debug: logging.basicConfig(level=logging.DEBUG, format="%(message)s") else: logging.basicConfig(level=logging.INFO, format="%(message)s") software_manager = SoftwareManager() if not software_manager.is_capable(): log.error( "ERROR: There is no backend implementation for this system. No " "action will be performed." ) return exit_codes.UTILITY_FAIL options = { "install": install, "remove": remove, "check-installed": check_installed, "list-all": list_all, "list-files": list_files, "add-repo": add_repo, "remove-repo": remove_repo, "upgrade": upgrade, "what-provides": what_provides, "install-what-provides": install_what_provides, "show-help": show_help, } if args: action = args[0] args = " ".join(args[1:]) else: action = "show-help" return options[action]() avocado-113.0/avocado/utils/software_manager/manager.py000066400000000000000000000051641513561306000232240ustar00rootroot00000000000000from avocado.utils.software_manager.inspector import ( SUPPORTED_PACKAGE_MANAGERS, SystemInspector, ) class SoftwareManager: """ Package management abstraction layer. It supports a set of common package operations for testing purposes, and it uses the concept of a backend, a helper class that implements the set of operations of a given package management tool. """ def __init__(self): """ Lazily instantiate the object """ self.initialized = False self.backend = None self.lowlevel_base_command = None self.base_command = None self.pm_version = None def _init_on_demand(self): """ Determines the best supported package management system for the given operating system running and initializes the appropriate backend. """ if not self.initialized: inspector = SystemInspector() backend_type = inspector.get_package_management() if backend_type not in SUPPORTED_PACKAGE_MANAGERS: raise NotImplementedError( f"Unimplemented package management " f"system: {backend_type}." ) backend = SUPPORTED_PACKAGE_MANAGERS[backend_type] self.backend = backend() self.initialized = True def __getattr__(self, name): self._init_on_demand() return self.backend.__getattribute__(name) def is_capable(self): """Checks if environment is capable by initializing the backend.""" try: self._init_on_demand() except NotImplementedError: pass return self.initialized @staticmethod def extract_from_package(package_path, dest_path=None): """Try to extract a package content into a destination directory. It will try to see if the package is valid against all supported package managers and if any is found, then extracts its content into the extract_path. Raises NotImplementedError when a non-supported package is used. :param str package_path: package file path. :param str dest_path: destination path to extract. Default is the current directory. :returns: destination path were the package it was extracted. """ for backend in SUPPORTED_PACKAGE_MANAGERS.values(): if backend.is_valid(package_path): return backend.extract_from_package(package_path, dest_path) raise NotImplementedError( f"No package manager supported was found " f"for package {package_path}." ) avocado-113.0/avocado/utils/softwareraid.py000066400000000000000000000123221513561306000207520ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: 2020 IBM # Author: Narasimhan V # Author: Venkat Rao B """ This module provides APIs to work with software raid. """ import logging import time from avocado.utils import process LOG = logging.getLogger(__name__) class SoftwareRaid: """ Perform software raid related operations. :param name: Name of the software raid to be created :type name: str :param level: Level of software raid to be created :type name: str :param disks: List of disks for software raid :type disks: list :param metadata: Metadata level for software raid :type metadata: str :param spare_disks: List of spare disks for software raid :type spare_disks: list """ # pylint: disable=R0913 def __init__(self, name, level, disks, metadata, spare_disks=None): self.name = name self.level = level self.metadata = metadata self.disks = disks self.remadd = "" self.spare_disks = spare_disks or [] def _run_command(self, cmd, log_details=True, check_recovery=False): if process.system(cmd, ignore_status=True, shell=True): if log_details: self.get_detail() return False if check_recovery: while self.is_recovering(): time.sleep(30) if log_details: self.get_detail() return True def add_disk(self, disk): """ Adds disk specified to software raid. :param disk: disk to be added. :type disk: str :return: True if add is successful, False otherwise. :rtype: bool """ cmd = f"mdadm {self.name} --add {disk}" return self._run_command(cmd) def assemble(self): """ Assembles software raid. :return: True if assembled, False otherwise. :rtype: bool """ cmd = f"mdadm --assemble {self.name} {' '.join(self.disks)}" if self.spare_disks: cmd += f" {' '.join(self.spare_disks)}" return self._run_command(cmd, check_recovery=True) def clear_superblock(self): """ Zeroes superblocks in member devices of raid. :return: True if zeroed, False otherwise. :rtype: bool """ cmd = f"mdadm --zero-superblock {' '.join(self.disks)}" if self.spare_disks: cmd += f" {' '.join(self.spare_disks)}" return self._run_command(cmd, log_details=False) def create(self): """ Creates software raid. :return: True if raid is created. False otherwise. :rtype: bool """ cmd = f"yes | mdadm --create --assume-clean {self.name}" cmd += f" --level={self.level}" cmd += f" --raid-devices={len(self.disks)} {' '.join(self.disks)}" cmd += f" --metadata={self.metadata}" if self.spare_disks: cmd += ( f" --spare-devices={len(self.spare_disks)} " f"{' '.join(self.spare_disks)}" ) cmd += " --verbose --force" return self._run_command(cmd) def get_detail(self): """ Returns mdadm details. :return: mdadm --detail output :rtype: str """ cmd = f"mdadm --detail {self.name}" output = process.run(cmd, ignore_status=True, shell=True) return output.stdout_text def is_recovering(self): """ Checks if raid is recovering. :return: True if recovering, False otherwise. :rtype: bool """ LOG.debug("RECOVERY") for line in self.get_detail().splitlines(): if "State" in line and "recovering" in line: return True return False def remove_disk(self, disk): """ Removes disk specified from software raid. :param disk: disk to be removed. :type disk: str :return: True if remove is successful, False otherwise. :rtype: bool """ cmd = f"mdadm {self.name} --fail {disk} --remove {disk}" return self._run_command(cmd) def stop(self): """ Stops software raid. :return: True if stopped, False otherwise. :rtype: bool """ cmd = f"mdadm --manage {self.name} --stop" return self._run_command(cmd, log_details=False) def exists(self): """ checks if softwareraid exists or not :mdadm: must be super-user(root) to perform this action :return: True if exists, False otherwise. :rtype: bool """ cmd = f"mdadm --detail --test {self.name}" result = process.run(cmd, shell=True, sudo=True, ignore_status=True) if result.exit_status == 4: return False return True avocado-113.0/avocado/utils/ssh.py000066400000000000000000000230641513561306000170620ustar00rootroot00000000000000""" Provides utilities to carry out an SSH session. Example of use: .. code-block:: python from avocado.utils import ssh with ssh.Session(host, user='root', key='/path/to/file') as session: result = session.cmd('ls') if result.exit_status == 0: print(result.stdout_text) """ import os import shlex import stat import subprocess import sys import tempfile from avocado.utils import path as path_utils from avocado.utils import process try: #: The SSH client binary to use, if one is found in the system SSH_CLIENT_BINARY = path_utils.find_command("ssh") except path_utils.CmdNotFoundError: SSH_CLIENT_BINARY = None class NWException(Exception): """ Base Exception Class for all exceptions """ class Session: """ Represents an SSH session to a remote system, for the purpose of executing commands remotely. :class:`Session` is also a context manager. On entering the context it tries to establish the connection, therefore on exiting that connection is closed. """ DEFAULT_OPTIONS = ( ("StrictHostKeyChecking", "no"), ("UpdateHostKeys", "no"), ("ControlPath", "~/.ssh/avocado-master-%r@%h:%p"), ) MASTER_OPTIONS = (("ControlMaster", "yes"), ("ControlPersist", "yes")) # pylint: disable=R0913 def __init__(self, host, port=None, user=None, key=None, password=None): """ :param host: a host name or IP address :type host: str :param port: port number :type port: int :param user: the name of the remote user :type user: str :param key: path to a key for authentication purpose :type key: str :param password: password for authentication purpose :type password: str """ self.host = host self.port = port self.user = user self.key = key self.password = password self._connection = None def __enter__(self): self.connect() return self def __exit__(self, _exc_type, _exc_value, _traceback): self.quit() @staticmethod def _dash_o_opts_to_str(opts): """ Transforms tuples into options that should be given by "-o Key=Val" """ return " ".join([f"-o '{_[0]}={_[1]}'" for _ in opts]) def _ssh_cmd(self, dash_o_opts=(), opts=(), command=""): cmd = self._dash_o_opts_to_str(dash_o_opts) if self.user is not None: cmd += f" -l {self.user}" if self.key is not None: cmd += f" -i {self.key}" if self.port is not None: cmd += f" -p {self.port}" cmd = f"{SSH_CLIENT_BINARY} {cmd} {' '.join(opts)} " f"{self.host} '{command}'" return cmd def _master_connection(self): options = self.DEFAULT_OPTIONS + self.MASTER_OPTIONS options += (("PubkeyAuthentication", "yes" if self.key else "no"),) if self.password is None: options += (("PasswordAuthentication", "no"),) else: options += ( ("PasswordAuthentication", "yes"), ("NumberOfPasswordPrompts", "1"), ) return self._ssh_cmd(options, ("-T", "-n")) def _create_ssh_askpass(self): """ Writes a simple program that complies with SSH_ASKPASS This basically writes to stdout the password given """ script = f"#!{sys.executable}\nprint('{self.password}')" fd, path = tempfile.mkstemp() os.write(fd, script.encode()) os.fchmod(fd, stat.S_IRUSR | stat.S_IXUSR) os.close(fd) return path def _master_command(self, command): cmd = self._ssh_cmd(self.DEFAULT_OPTIONS, ("-O", command)) result = process.run(cmd, ignore_status=True) return not result.exit_status def _check(self): return self._master_command("check") def cleanup_master(self): """Removes master file if exists.""" if self.control_master: os.unlink(self.control_master) def connect(self): """ Establishes the connection to the remote endpoint On this implementation, it means creating the master connection, which is a process that will live while and be used for subsequent commands. :returns: whether the connection is successfully established :rtype: bool """ if SSH_CLIENT_BINARY is None: return False if not self._check(): cmd = shlex.split(self._master_connection()) if self.password is not None: ssh_askpass_path = self._create_ssh_askpass() env = { "DISPLAY": "FAKE_VALUE_TO_SATISFY_SSH", "SSH_ASKPASS": ssh_askpass_path, } # pylint: disable=W1509 master = subprocess.Popen( cmd, stdin=subprocess.DEVNULL, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, env=env, preexec_fn=os.setsid, ) else: # pylint: disable=R1732 master = subprocess.Popen( cmd, stdin=subprocess.DEVNULL, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, ) master.wait() if self.password is not None: os.unlink(ssh_askpass_path) if master.returncode: return False self._connection = master return self._check() @property def control_master(self): control = f"~/.ssh/avocado-master-{self.user}@{self.host}:{self.port}" control = os.path.expanduser(control) if os.path.exists(control): return control return None def get_raw_ssh_command(self, command): """ Returns the raw command that will be executed locally This should only be used if you need to interact with the ssh subprocess, and most users will *NOT* need to. Try to use the :meth:`cmd` method instead. :param command: the command to execute over the SSH session :type command: str :returns: The raw SSH command, that can be executed locally for the execution of a remote command. :rtype: str """ return self._ssh_cmd(self.DEFAULT_OPTIONS, ("-q",), command) def cmd(self, command, ignore_status=True, timeout=None): """ Runs a command over the SSH session :param command: the command to execute over the SSH session :type command: str :param ignore_status: Whether to check the operation failed or not. If set to False then it raises an :class:`avocado.utils.process.CmdError` exception in case of either the command or ssh connection returned with exit status other than zero. :type ignore_status: bool :param timeout: Limit the command execution time, if you want the command to end within a few seconds, you can set a specific time. :type timeout: float :returns: The command result object. :rtype: A :class:`avocado.utils.process.CmdResult` instance. """ if timeout: command_argument = f"timeout {timeout} {command}" else: command_argument = command try: return process.run( self.get_raw_ssh_command(command_argument), ignore_status=ignore_status, ) except process.CmdError as exc: if exc.result.exit_status == 255: exc.additional_text = "SSH connection failed" else: exc.additional_text = f"Command '{command}' failed" exc.stderr = exc.result.stderr exc.stdout = exc.result.stdout raise exc def quit(self): """ Attempts to gracefully end the session, by finishing the master process :returns: if closing the session was successful or not :rtype: bool """ return self._master_command("exit") def copy_files(self, source, destination, recursive=False): """ Copy Files to and from remote through scp session. :param source: Source file :type: str :param destination: Destination file location :type: str :param recursive: Scp option for copy file. if set to True copy files inside directory recursively. :type: bool :returns: True if success and an exception if not. :rtype: bool """ path = destination.split(":")[-1] try: cmd = path_utils.find_command("scp") except path_utils.CmdNotFoundError as exc: raise exc options = list(self.DEFAULT_OPTIONS) if self.user is not None: # Prevent failure from unspecified user options.append(("User", self.user)) options = self._dash_o_opts_to_str(options) if recursive: options += " -r" options += f" {source} {destination}" try: if self.cmd(f"test -d {path}").exit_status: self.cmd(f"mkdir -p {path}") result = process.run(f"{cmd} {options}", ignore_status=True) return not result.exit_status except process.CmdError as exc: raise NWException(f"failed to copy file {exc}") from exc avocado-113.0/avocado/utils/stacktrace.py000066400000000000000000000066651513561306000204210ustar00rootroot00000000000000""" Traceback standard module plus some additional APIs. """ import inspect import logging import pickle from pprint import pformat from traceback import format_exception def tb_info(exc_info): """ Prepare traceback info. :param exc_info: Exception info produced by sys.exc_info() """ exc_type, exc_value, exc_traceback = exc_info return format_exception(exc_type, exc_value, exc_traceback.tb_next) def prepare_exc_info(exc_info): """ Prepare traceback info. :param exc_info: Exception info produced by sys.exc_info() """ return "".join(tb_info(exc_info)) def log_exc_info(exc_info, logger=None): """ Log exception info to logger_name. :param exc_info: Exception info produced by sys.exc_info() :param logger: Name or logger instance (defaults to '') """ logger = logger or __name__ if isinstance(logger, str): logger = logging.getLogger(logger) logger.error("") called_from = inspect.currentframe().f_back logger.error( "Reproduced traceback from: %s:%s", called_from.f_code.co_filename, called_from.f_lineno, ) for trace in tb_info(exc_info): for line in trace.splitlines(): logger.error(line) logger.error("") def log_message(message, logger=None): """ Log message to logger. :param message: Message :param logger: Name or logger instance (defaults to '') """ logger = logger or __name__ if isinstance(logger, str): logger = logging.getLogger(logger) for line in message.splitlines(): logger.error(line) def analyze_unpickable_item(path_prefix, obj): """ Recursive method to obtain unpickable objects along with location :param path_prefix: Path to this object :param obj: The sub-object under introspection :return: [($path_to_the_object, $value), ...] """ _path_prefix = path_prefix try: if hasattr(obj, "items"): subitems = obj.items() path_prefix += "[%s]" elif isinstance(obj, list): subitems = enumerate(obj) path_prefix += "[%s]" elif hasattr(obj, "__iter__"): subitems = enumerate(iter(obj)) path_prefix += "<%s>" elif hasattr(obj, "__dict__"): subitems = obj.__dict__.items() path_prefix += ".%s" else: return [(path_prefix, obj)] except Exception: # pylint: disable=W0703 return [(path_prefix, obj)] unpickables = [] for key, value in subitems: try: pickle.dumps(value) except pickle.PickleError: ret = analyze_unpickable_item(path_prefix % key, value) if ret: unpickables.extend(ret) if not unpickables: return [(_path_prefix, obj)] return unpickables def str_unpickable_object(obj): """ Return human readable string identifying the unpickable objects :param obj: The object for analysis :raise ValueError: In case the object is pickable """ try: pickle.dumps(obj) except pickle.PickleError: pass else: raise ValueError(f"This object is pickable:\n{pformat(obj)}") unpickables = analyze_unpickable_item("this", obj) return "Unpickable object in:\n %s\nItems causing troubles:\n " % "\n ".join( pformat(obj).splitlines() ) + "\n ".join( "%s => %s" % obj for obj in unpickables # pylint: disable=C0209 ) avocado-113.0/avocado/utils/sysinfo.py000066400000000000000000000274571513561306000177710ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2021 # Authors: Jan Richter # John Admanski import json import os import shlex import subprocess import tempfile from abc import ABC, abstractmethod from avocado.utils import astring, process DATA_SIZE = 200000 class CollectibleException(Exception): """ Base exception for all collectible errors. """ class Collectible(ABC): """ Abstract class for representing sysinfo collectibles. """ def __init__(self, log_path): self.log_path = astring.string_to_safe_path(log_path) self._name = os.path.basename(log_path) @abstractmethod def collect(self): pass @property def name(self): return self._name @staticmethod def _read_file(path, bytes_to_skip=0): """Method for lazy reading of file""" with open(path, "rb") as in_messages: in_messages.seek(bytes_to_skip) while True: # Read data in manageable chunks rather than # all at once. in_data = in_messages.read(DATA_SIZE) if not in_data: break yield in_data def __eq__(self, other): if hash(self) == hash(other): return True if isinstance(other, Collectible): return False return NotImplemented def __ne__(self, other): result = self.__eq__(other) if result is NotImplemented: return result return not result def __hash__(self): return hash((self.log_path, Collectible)) class Logfile(Collectible): """ Collectible system file. :param path: Path to the log file. :param log_path: Basename of the file where output is logged (optional). """ def __init__(self, path, log_path=None): if not log_path: log_path = os.path.basename(path) super().__init__(log_path) self.path = path def __repr__(self): r = "Logfile(%r, %r)" r %= (self.path, self.log_path) return r def __eq__(self, other): if isinstance(other, Logfile): return (self.path, self.log_path) == (other.path, other.log_path) if isinstance(other, Collectible): return False return NotImplemented def __hash__(self): return hash((self.path, self.log_path, Logfile)) def collect(self): """ Reads the log file. :raise CollectibleException """ if os.path.exists(self.path): try: yield from self._read_file(self.path) except IOError as exc: raise CollectibleException( f"Not logging {self.path} " f"(lack of permissions)" ) from exc else: raise CollectibleException(f"Not logging {self.path} " f"(file not found)") class Command(Collectible): """ Collectible command. :param cmd: String with the command. :param timeout: Timeout for command execution. :param locale: Force LANG for sysinfo collection """ def __init__(self, cmd, timeout=-1, locale="C"): super().__init__(cmd) self._name = self.log_path self.cmd = cmd self.timeout = timeout self.locale = locale def __repr__(self): r = "Command(%r, %r)" r %= (self.cmd, self.log_path) return r def __eq__(self, other): if isinstance(other, Command): return (self.cmd, self.log_path) == (other.cmd, other.log_path) if isinstance(other, Collectible): return False return NotImplemented def __hash__(self): return hash((self.cmd, self.log_path, Command)) def collect(self): """ Execute the command as a subprocess and returns it's output. :raise CollectibleException """ env = os.environ.copy() if "PATH" not in env: env["PATH"] = "/usr/bin:/bin" if self.locale: env["LC_ALL"] = self.locale # the sysinfo configuration supports negative or zero integer values # but the avocado.utils.process APIs define no timeouts as "None" if int(self.timeout) <= 0: self.timeout = None try: result = process.run( self.cmd, timeout=self.timeout, verbose=False, ignore_status=True, shell=True, env=env, ) yield result.stdout except FileNotFoundError as exc_fnf: raise CollectibleException( f"Not logging '{self.cmd}' " f"(command '{exc_fnf.filename}' " f"was not found)" ) from exc_fnf except Exception as exc: # pylint: disable=W0703, W0612 raise CollectibleException( f'Could not execute "{self.cmd}": ' "{exc}" ) from exc class Daemon(Command): """ Collectible daemon. :param cmd: String with the command. :param timeout: Timeout for command execution. :param locale: Force LANG for sysinfo collection """ def __init__(self, *args, **kwargs): super().__init__(*args, **kwargs) self.daemon_process = None self.temp_file = tempfile.NamedTemporaryFile() # pylint: disable=R1732 def __repr__(self): r = "Daemon(%r, %r)" r %= (self.cmd, self.log_path) return r def __eq__(self, other): if isinstance(other, Daemon): return (self.cmd, self.log_path) == (other.cmd, other.log_path) if isinstance(other, Collectible): return False return NotImplemented def __hash__(self): return hash((self.cmd, self.log_path, Daemon)) def __del__(self): self.temp_file.close() def run(self): """ Start running the daemon as a subprocess. :raise CollectibleException """ env = os.environ.copy() if "PATH" not in env: env["PATH"] = "/usr/bin:/bin" if self.locale: env["LC_ALL"] = self.locale logf_path = self.temp_file.name stdin = open(os.devnull, "r") # pylint: disable=W1514, R1732 stdout = open(logf_path, "w") # pylint: disable=W1514, R1732 try: # pylint: disable=R1732 self.daemon_process = subprocess.Popen( shlex.split(self.cmd), stdin=stdin, stdout=stdout, stderr=subprocess.STDOUT, shell=False, env=env, ) except OSError as os_err: raise CollectibleException( f'Could not execute "{self.cmd}": ' f"{os_err}" ) from os_err finally: stdin.close() stdout.close() def collect(self): """ Stop daemon execution and returns it's logs. :raise OSError """ if self.daemon_process is not None: retcode = self.daemon_process.poll() if retcode is None: process.kill_process_tree(self.daemon_process.pid) self.daemon_process.wait() for line in self.temp_file.readlines(): yield line else: raise OSError( f"Daemon process '{self.cmd}' " f"(pid {int(self.daemon_process.pid)}) " f"terminated abnormally (code {int(retcode)})" ) class JournalctlWatcher(Collectible): """ Track the content of systemd journal. :param log_path: Basename of the file where output is logged (optional). """ def __init__(self, log_path=None): if not log_path: log_path = "journalctl.gz" super().__init__(log_path) self.cursor = self._get_cursor() def __repr__(self): r = "JournalctlWatcher(%r)" r %= self.log_path return r def __eq__(self, other): if isinstance(other, JournalctlWatcher): return self.log_path == other.log_path if isinstance(other, Collectible): return False return NotImplemented def __hash__(self): return hash((self.log_path, JournalctlWatcher)) @staticmethod def _get_cursor(): try: cmd = "journalctl --quiet --lines 1 --output json" result = process.system_output(cmd, verbose=False) last_record = json.loads(astring.to_text(result, "utf-8")) return last_record["__CURSOR"] except Exception as detail: # pylint: disable=W0703 raise CollectibleException( f"Journalctl collection failed: " f"{detail}" ) from detail def collect(self): """ Returns the content of systemd journal :raise CollectibleException """ if self.cursor: try: cmd = f"journalctl --quiet --after-cursor {self.cursor}" log_diff = process.system_output(cmd, verbose=False) yield log_diff except Exception as detail: # pylint: disable=W0703 raise CollectibleException( f"Journalctl collection failed: " f"{detail}" ) from detail class LogWatcher(Collectible): """ Keep track of the contents of a log file in another compressed file. This object is normally used to track contents of the system log (/var/log/messages), and the outputs are gzipped since they can be potentially large, helping to save space. :param path: Path to the log file. :param log_path: Basename of the file where output is logged (optional). """ def __init__(self, path, log_path=None): if not log_path: log_path = os.path.basename(path) + ".gz" else: log_path += ".gz" super().__init__(log_path) self.path = path self.size = 0 self.inode = 0 try: stat = os.stat(path) self.size = stat.st_size self.inode = stat.st_ino except (IOError, OSError) as exc: raise CollectibleException( f"Not logging {self.path} " f"(lack of permissions)" ) from exc def __repr__(self): r = "LogWatcher(%r, %r)" r %= (self.path, self.log_path) return r def __eq__(self, other): if isinstance(other, LogWatcher): return (self.path, self.log_path) == (other.path, other.log_path) if isinstance(other, Collectible): return False return NotImplemented def __hash__(self): return hash((self.path, self.log_path, LogWatcher)) def collect(self): """ Collect all of the new data present in the log file. :raise CollectibleException """ bytes_to_skip = 0 current_stat = os.stat(self.path) current_inode = current_stat.st_ino current_size = current_stat.st_size if current_inode == self.inode: bytes_to_skip = self.size self.inode = current_inode self.size = current_size try: yield from self._read_file(self.path, bytes_to_skip) except (IOError, OSError) as exc: raise CollectibleException( f"Not logging {self.path} " f"(lack of permissions)" ) from exc avocado-113.0/avocado/utils/vmimage.py000066400000000000000000000700101513561306000177030ustar00rootroot00000000000000# This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2017 # Author: Amador Pahim """ Provides VM images acquired from official repositories """ import logging import os import re import tempfile import uuid import warnings from html.parser import HTMLParser from urllib.error import HTTPError from urllib.request import urlopen from avocado.utils import archive, asset, astring from avocado.utils import path as utils_path from avocado.utils import process LOG = logging.getLogger(__name__) # pylint: disable=C0401 #: The "qemu-img" binary used when creating the snapshot images. If #: set to None (the default), it will attempt to find a suitable binary #: with :func:`avocado.utils.path.find_command`, which uses the the #: system's PATH environment variable QEMU_IMG = None #: Default image architecture, which defaults to the current machine #: architecture, if that's can be retrieved via :func:`os.uname`. If #: not, it defaults to "x86_64" simply because it's the most commonly #: used architecture and lack of a better default if hasattr(os, "uname"): DEFAULT_ARCH = os.uname()[4] else: DEFAULT_ARCH = "x86_64" class ImageProviderError(Exception): """ Generic error class for ImageProvider """ class VMImageHtmlParser(HTMLParser): # pylint: disable=W0223 """ Custom HTML parser to extract the href items that match a given pattern """ def __init__(self, pattern): HTMLParser.__init__(self) self.items = [] self.pattern = pattern def handle_starttag(self, tag, attrs): if tag != "a": return for attr in attrs: if attr[0] == "href" and re.match(self.pattern, attr[1]): match = attr[1].strip("/") if match not in self.items: self.items.append(match) # pylint: disable=R0902 class ImageProviderBase: """ Base class to define the common methods and attributes of an image. Intended to be sub-classed by the specific image providers. """ HTML_ENCODING = "utf-8" def __init__(self, version, build, arch): self.url_versions = None self.url_images = None self.image_pattern = None self._file_name = None self._version = version self._best_version = None self.build = build self.arch = arch @property def version(self): return self._best_version or self.get_version() @property def version_pattern(self): return f"^{self._version}/$" @property def file_name(self): if self._file_name is None: self._file_name = self.image_pattern.format( version=self.version, build=self.build, arch=self.arch ) return self._file_name def _feed_html_parser(self, url, parser): try: with urlopen(url) as u: data = u.read() parser.feed(astring.to_text(data, self.HTML_ENCODING)) except HTTPError as exc: raise ImageProviderError(f"Network error: Cannot open {url}") from exc @staticmethod def get_best_version(versions): return max(versions) def get_versions(self): """Return all available versions for the current parameters.""" parser = VMImageHtmlParser(self.version_pattern) self._feed_html_parser(self.url_versions, parser) resulting_versions = [] if parser.items: for version in parser.items: # Trying to convert version to int or float so max() # can compare numerical values. try: # Can it be converted to integer? resulting_versions.append(int(version)) except ValueError: try: # Can it be converted to float? resulting_versions.append(float(version)) except ValueError: # So it's just a string resulting_versions.append(version) return resulting_versions def get_version(self): """Probes the higher version available for the current parameters.""" resulting_versions = self.get_versions() if resulting_versions: self._best_version = self.get_best_version(resulting_versions) return self._best_version raise ImageProviderError(f"Version not available at " f"{self.url_versions}") def get_image_url(self): """ Probes the higher image available for the current parameters. """ if not self.url_images or not self.image_pattern: raise ImageProviderError( "url_images and image_pattern attributes are required to get image url" ) url_images = self.url_images.format( version=self.version, build=self.build, arch=self.arch ) image = self.image_pattern.format( version=self.version, build=self.build, arch=self.arch ) parser = VMImageHtmlParser(image) self._feed_html_parser(url_images, parser) if parser.items: return url_images + max(parser.items) raise ImageProviderError( f"No images matching '{image}' " f"at '{url_images}'. Wrong arch?" ) def get_image_parameters(self, image_file_name): """ Computation of image parameters from image_pattern :param image_file_name: pattern with parameters :type image_file_name: str :return: dict with parameters :rtype: dict or None """ keywords = re.split(r"\{(.*?)\}", self.image_pattern)[1::2] matches = re.match(self.file_name, image_file_name) if not matches: return None return {x: matches.group(x) for x in keywords} class FedoraImageProviderBase(ImageProviderBase): """ Base Fedora Image Provider """ HTML_ENCODING = "iso-8859-1" url_old_images = None def get_image_url(self): if int(self.version) >= 28: cloud = "Cloud" else: cloud = "CloudImages" if self.url_old_images and int(self.version) <= 40: self.url_versions = self.url_old_images self.url_images = self.url_versions + "{version}/" + cloud + "/{arch}/images/" return super().get_image_url() class FedoraImageProvider(FedoraImageProviderBase): """ Fedora Image Provider """ name = "Fedora" def __init__(self, version="[0-9]+", build="[0-9]+.[0-9]+", arch=DEFAULT_ARCH): super().__init__(version, build, arch) self.url_versions = "https://dl.fedoraproject.org/pub/fedora/linux/releases/" self.url_old_images = ( "https://archives.fedoraproject.org/pub/archive/fedora/linux/releases/" ) if int(self.version) == 40: self.image_pattern = "Fedora-Cloud-Base-Generic.(?P{arch})-(?P{version})-(?P{build}).qcow2$" elif int(self.version) >= 41: self.image_pattern = "Fedora-Cloud-Base-Generic-(?P{version})-(?P{build}).(?P{arch}).qcow2$" else: self.image_pattern = "Fedora-Cloud-Base-(?P{version})-(?P{build}).(?P{arch}).qcow2$" class FedoraSecondaryImageProvider(FedoraImageProviderBase): """ Fedora Secondary Image Provider """ name = "FedoraSecondary" def __init__(self, version="[0-9]+", build="[0-9]+.[0-9]+", arch=DEFAULT_ARCH): super().__init__(version, build, arch) self.url_versions = ( "https://dl.fedoraproject.org/pub/fedora-secondary/releases/" ) self.url_old_images = ( "https://archives.fedoraproject.org/pub/archive/fedora-secondary/releases/" ) if int(self.version) == 40: self.image_pattern = "Fedora-Cloud-Base-Generic.(?P{arch})-(?P{version})-(?P{build}).qcow2$" elif int(self.version) >= 41: self.image_pattern = "Fedora-Cloud-Base-Generic-(?P{version})-(?P{build}).(?P{arch}).qcow2$" else: self.image_pattern = "Fedora-Cloud-Base-(?P{version})-(?P{build}).(?P{arch}).qcow2$" class CentOSImageProvider(ImageProviderBase): """ CentOS Image Provider """ name = "CentOS" def __init__(self, version="[0-9]+", build="[0-9]{4}", arch=DEFAULT_ARCH): super().__init__(version, build, arch) self.url_versions = "https://cloud.centos.org/centos/" self.url_images = self.url_versions + "{version}/images/" self.image_pattern = "CentOS-(?P{version})-(?P{arch})-GenericCloud-(?P{build}).qcow2.xz$" def _set_patterns(self): if int(self.version) >= 8: self.build = r"[\d\.\-]+" self.url_images = self.url_versions + "{version}/{arch}/images/" self.image_pattern = "CentOS-(?P{version})-GenericCloud-(?P{build}).(?P{arch}).qcow2$" @property def file_name(self): self._set_patterns() return super().file_name def get_image_url(self): self._set_patterns() return super().get_image_url() class UbuntuImageProvider(ImageProviderBase): """ Ubuntu Image Provider """ name = "Ubuntu" def __init__(self, version="[0-9]+.[0-9]+", build=None, arch=DEFAULT_ARCH): # Ubuntu uses 'amd64' instead of 'x86_64' if arch == "x86_64": arch = "amd64" # and 'arm64' instead of 'aarch64' elif arch == "aarch64": arch = "arm64" super().__init__(version, build, arch) self.url_versions = "http://cloud-images.ubuntu.com/releases/" self.url_images = self.url_versions + "releases/{version}/release/" self.image_pattern = ( "ubuntu-(?P{version})-server-cloudimg-(?P{arch}).img" ) def get_best_version(self, versions): # pylint: disable=W0221 """Return best (more recent) version""" max_float = max(float(item) for item in versions) return str(f"{max_float:2.2f}") def get_versions(self): """Return all available versions for the current parameters.""" parser = VMImageHtmlParser(self.version_pattern) self._feed_html_parser(self.url_versions, parser) resulting_versions = [] if parser.items: for version in parser.items: max_float = float(version) resulting_versions.append(str(f"{max_float:2.2f}")) return resulting_versions class DebianImageProvider(ImageProviderBase): """ Debian Image Provider """ name = "Debian" def __init__(self, version=None, build=r"[\d{8}\-\d{3}]", arch=DEFAULT_ARCH): # Debian uses 'amd64' instead of 'x86_64' if arch == "x86_64": arch = "amd64" # and 'arm64' instead of 'aarch64' elif arch == "aarch64": arch = "arm64" table_version = { "buster": "10", "bullseye": "11", } table_codename = { "10": "buster", "11": "bullseye", } # Default version if none was selected, should work at least until 2023 Q3 if version is None: version = "bullseye" # User provided a numerical version version = table_codename.get(version, version) # If version is not a codename by now, it's wrong or unknown, # so let's fail early if version not in table_version: raise ImageProviderError("Unknown version", version) super().__init__(version, build, arch) self.url_versions = "https://cloud.debian.org/images/cloud/" self.url_images = self.url_versions + version + "/{build}/" self.image_pattern = ( "debian-" + table_version[version] + "-generic-(?P{arch})-{build}.qcow2$" ) def get_image_url(self): # Find out the build first parserbuild = VMImageHtmlParser(self.build) self._feed_html_parser(self.url_versions + self._version + "/", parserbuild) self.build = max(parserbuild.items) return super().get_image_url() class JeosImageProvider(ImageProviderBase): """ JeOS Image Provider """ name = "JeOS" def __init__(self, version="[0-9]+", build=None, arch=DEFAULT_ARCH): # JeOS uses '64' instead of 'x86_64' if arch == "x86_64": arch = "64" super().__init__(version, build, arch) self.url_versions = "https://avocado-project.org/data/assets/jeos/" self.url_images = self.url_versions + "{version}/" self.image_pattern = "jeos-(?P{version})-(?P{arch}).qcow2.xz$" class OpenSUSEImageProvider(ImageProviderBase): """ OpenSUSE Image Provider """ HTML_ENCODING = "iso-8859-1" name = "OpenSUSE" def __init__(self, version="[0-9]{2}.[0-9]{1}", build=None, arch=DEFAULT_ARCH): super().__init__(version, build, arch) self.url_versions = ( "https://download.opensuse.org/pub/opensuse/distribution/leap/" ) self.url_images = self.url_versions + "{version}/appliances/" if not build: self.image_pattern = "openSUSE-Leap-(?P{version})-(JeOS|Minimal-VM).(?P{arch})(-OpenStack)?-Cloud.qcow2$" else: self.image_pattern = ( "openSUSE-Leap-(?P{version})-(JeOS|Minimal-VM).(?P{arch})-{version}" "(-OpenStack)?-Cloud-Build(?P{build}).qcow2$" ) @staticmethod def _convert_version_numbers(versions): """ Return float instead of strings """ return [float(v) for v in versions] def get_versions(self): versions = super().get_versions() return self._convert_version_numbers(versions) def get_best_version(self, versions): # pylint: disable=W0221 version_numbers = self._convert_version_numbers(versions) if str(self._version).startswith("4"): version_numbers = [v for v in version_numbers if v >= 40.0] else: version_numbers = [v for v in version_numbers if v < 40.0] return max(version_numbers) class CirrOSImageProvider(ImageProviderBase): """ CirrOS Image Provider CirrOS is a Tiny OS that specializes in running on a cloud. """ name = "CirrOS" def __init__( self, version=r"[0-9]+\.[0-9]+\.[0-9]+", build=None, arch=DEFAULT_ARCH ): super().__init__(version=version, build=build, arch=arch) self.url_versions = "https://download.cirros-cloud.net/" self.url_images = self.url_versions + "{version}/" self.image_pattern = "cirros-{version}-{arch}-disk.img$" class FreeBSDImageProvider(ImageProviderBase): """ FreeBSD Image Provider """ name = "FreeBSD" def __init__(self, version="[0-9]+.[0-9]", build=None, arch=DEFAULT_ARCH): # FreeBSD uses 'amd64' instead of 'x86_64' if arch == "x86_64": arch = "amd64" super().__init__(version + "-RELEASE", build, arch) self.url_versions = ( "http://ftp-archive.freebsd.org/pub/FreeBSD-Archive/old-releases/VM-IMAGES/" ) self.url_images = self.url_versions + "{version}-RELEASE/{arch}/Latest/" arch2 = "" if arch == "aarch64": arch2 = "arm64-" elif arch == "riscv64": arch2 = "riscv-" self.image_pattern = ( "FreeBSD-(?P{version})-RELEASE-" + arch2 + "(?P{arch}).qcow2.xz" ) def get_best_version(self, versions): # pylint: disable=W0221 """Return best (more recent) version""" max_float = max(float(item) for item in versions) return str(f"{max_float:2.1f}") def get_versions(self): """Return all available versions for the current parameters.""" parser = VMImageHtmlParser(self.version_pattern) self._feed_html_parser(self.url_versions, parser) resulting_versions = [] if parser.items: for version in parser.items: max_float = float(version.split("-")[0]) resulting_versions.append(str(f"{max_float:2.1f}")) return resulting_versions class Image: # pylint: disable=R0913, R0902 def __init__( self, name, url, version, arch, build, checksum, algorithm, cache_dir, snapshot_dir=None, ): """ Creates an instance of Image class. :param name: Name of image. :type name: str :param url: The url where the image can be fetched from. :type url: str :param version: Version of image. :type version: int :param arch: Architecture of the system image. :type arch: str :param build: Build of the system image. :type build: str :param checksum: Hash of the system image to match after download. :type checksum: str :param algorithm: Hash type, used when the checksum is provided. :type algorithm: str :param cache_dir: Local system path where the base images will be held. :type cache_dir: str or iterable :param snapshot_dir: Local system path where the snapshot images will be held. Defaults to cache_dir if none is given. :type snapshot_dir: str """ self.name = name self.url = url self.version = version self.arch = arch self.build = build self.checksum = checksum self.algorithm = algorithm self.cache_dir = cache_dir self.snapshot_dir = snapshot_dir self._path = None self._base_image = None @property def path(self): return self._path or self.get() @property def base_image(self): return self._base_image or self.download() def __repr__(self): return ( f"<{self.__class__.__name__} name={self.name} " f"version={self.version} arch={self.arch}>" ) def download(self): metadata = { "type": "vmimage", "name": self.name, "version": self.version, "arch": self.arch, "build": self.build, } if isinstance(self.cache_dir, str): cache_dirs = [self.cache_dir] else: cache_dirs = self.cache_dir LOG.debug("Attempting to download image from URL: %s", self.url) asset_path = asset.Asset( name=self.url, asset_hash=self.checksum, algorithm=self.algorithm, locations=None, cache_dirs=cache_dirs, expire=None, metadata=metadata, ).fetch(timeout=asset.DOWNLOAD_TIMEOUT * 3) if archive.is_archive(asset_path): uncompressed_path = os.path.splitext(asset_path)[0] asset_path = archive.uncompress(asset_path, uncompressed_path) self._base_image = asset_path return self._base_image def get(self): self.download() self._path = self._take_snapshot() return self._path def _take_snapshot(self): """ Takes a snapshot from the current image. """ if QEMU_IMG is None: qemu_img = utils_path.find_command("qemu-img") else: qemu_img = QEMU_IMG name, extension = os.path.splitext(self.base_image) new_image = f"{name}-{str(uuid.uuid4()).split('-', maxsplit=1)[0]}{extension}" if self.snapshot_dir is not None: new_image = os.path.join(self.snapshot_dir, os.path.basename(new_image)) cmd = ( f"{qemu_img} create -f qcow2 -b {self.base_image} -F " f"qcow2 {new_image}" ) process.run(cmd) return new_image @classmethod def _find_cached_image( # pylint: disable=too-many-locals cls, cache_dirs, name=None, version=None, build=None, arch=None, checksum=None, algorithm=None, snapshot_dir=None, ): """ Find a cached image using asset.py enhanced built-in functionality. This version uses the enhanced Asset.get_metadata() method which supports both normal asset names and direct file paths, maximizing utilization of existing asset.py caching features. """ # pylint: disable-next=invalid-name ARCH_COMPATIBILITY = { "x86_64": ["x86_64", "amd64", "64"], "amd64": ["x86_64", "amd64", "64"], "aarch64": ["aarch64", "arm64"], "arm64": ["aarch64", "arm64"], } compatible_arches = ARCH_COMPATIBILITY.get(arch, [arch]) if arch else [] def matches_image_criteria(metadata, name, version, build, compatible_arches): return ( metadata.get("type") == "vmimage" and (not name or metadata.get("name", "").lower() == name.lower()) and (not version or str(metadata.get("version", "")) == str(version)) and (not build or str(metadata.get("build", "")) == str(build)) and (not compatible_arches or metadata.get("arch") in compatible_arches) ) # Use Asset.get_all_assets() to find cached assets for asset_path in asset.Asset.get_all_assets(cache_dirs, sort=False): try: temp_asset = asset.Asset( name=asset_path, asset_hash=checksum, algorithm=algorithm, cache_dirs=cache_dirs, ) try: metadata = temp_asset.get_metadata() if not metadata: continue except OSError: continue if matches_image_criteria( metadata, name, version, build, compatible_arches ): # pylint: disable-next=W0212 if checksum and not temp_asset._verify_hash(asset_path): LOG.debug("Hash mismatch for cached image: %s", asset_path) continue LOG.info("Found matching cached image: %s", asset_path) return cls( name=metadata.get("name", name), url=asset_path, version=metadata.get("version", version), arch=metadata.get("arch", arch), build=metadata.get("build", build), checksum=checksum, algorithm=algorithm, cache_dir=cache_dirs[0], snapshot_dir=snapshot_dir, ) except (OSError, ValueError, KeyError, AttributeError) as e: # Skip assets that can't be processed (common errors during metadata processing) LOG.debug("Skipping asset %s due to error: %s", asset_path, e) continue return None @classmethod # pylint: disable=R0913 def from_parameters( cls, name=None, version=None, build=None, arch=None, checksum=None, algorithm=None, cache_dir=None, snapshot_dir=None, ): """ Returns an Image, according to the parameters provided. :param name: (optional) Name of the Image Provider, usually matches the distro name. :param version: (optional) Version of the system image. :param build: (optional) Build number of the system image. :param arch: (optional) Architecture of the system image. :param checksum: (optional) Hash of the system image to match after download. :param algorithm: (optional) Hash type, used when the checksum is provided. :param cache_dir: (optional) Local system path where the base images will be held. :param snapshot_dir: (optional) Local system path where the snapshot images will be held. Defaults to cache_dir if none is given. :returns: Image instance that can provide the image according to the parameters. """ # Use the current system architecture if arch is not provided if arch is None: arch = DEFAULT_ARCH provider = get_best_provider(name, version, build, arch) if cache_dir is None: cache_dir = tempfile.gettempdir() try: return cls( name=provider.name, url=provider.get_image_url(), version=provider.version, arch=provider.arch, checksum=checksum, algorithm=algorithm, build=provider.build, cache_dir=cache_dir, snapshot_dir=snapshot_dir, ) except ImageProviderError as e: LOG.debug(e) raise AttributeError("Provider not available") # pylint: disable=R0913 def get( name=None, version=None, build=None, arch=None, checksum=None, algorithm=None, cache_dir=None, snapshot_dir=None, ): """This method is deprecated. Use Image.from_parameters().""" warnings.warn( "deprecated, use Image.from_parameters() instead.", DeprecationWarning ) return Image.from_parameters( name, version, build, arch, checksum, algorithm, cache_dir, snapshot_dir ) def get_best_provider(name=None, version=None, build=None, arch=None): """ Wrapper to get parameters of the best Image Provider, according to the parameters provided. :param name: (optional) Name of the Image Provider, usually matches the distro name. :param version: (optional) Version of the system image. :param build: (optional) Build number of the system image. :param arch: (optional) Architecture of the system image. :returns: Image Provider """ if name is not None: name = name.lower() provider_args = {} if version is not None: provider_args["version"] = version if build is not None: provider_args["build"] = build if arch is not None: provider_args["arch"] = arch if name == "fedora" and arch in ("ppc64", "ppc64le", "s390x"): name = "fedorasecondary" provider_errors = [] for provider in IMAGE_PROVIDERS: if name is None or name == provider.name.lower(): try: return provider(**provider_args) except ImageProviderError as e: LOG.debug(e) provider_errors.append(f"{provider.name}: {e}") except (HTTPError, OSError) as e: LOG.debug( "Network error while creating provider %s: %s", provider.name, e ) # Network errors should raise immediately raise ImageProviderError(f"Network error: {e}") from e LOG.debug("Provider for %s not available", name) if provider_errors: error_details = "; ".join(provider_errors) raise ImageProviderError(f"Providers failed - {error_details}") raise AttributeError("Provider not available") def list_providers(): """ List the available Image Providers """ return set( _ for _ in globals().values() if ( isinstance(_, type) and issubclass(_, ImageProviderBase) and hasattr(_, "name") ) ) #: List of available providers classes IMAGE_PROVIDERS = list_providers() avocado-113.0/avocado/utils/wait.py000066400000000000000000000107501513561306000172270ustar00rootroot00000000000000"""Utilities for waiting for conditions to be met. This module provides utilities for polling functions until they return a truthy value or a timeout expires, useful for testing and development scenarios where you need to wait for system state changes. """ import logging import time LOG = logging.getLogger(__name__) # pylint: disable=R0913 def wait_for(func, timeout, first=0.0, step=1.0, text=None, args=None, kwargs=None): """Wait until a function returns a truthy value or timeout expires. This function repeatedly calls a given function with optional arguments until it returns a truthy value (anything that evaluates to True in a boolean context) or until the specified timeout expires. It provides configurable delays before the first attempt and between subsequent attempts, making it useful for polling operations in testing and development scenarios. The function uses time.monotonic() for reliable timeout calculation that is not affected by system clock adjustments. Note that the step sleep duration is not interrupted when timeout expires, so actual elapsed time may exceed the specified timeout by up to one step duration. :param func: Callable to be executed repeatedly until it returns a truthy value. Can be any callable object (function, lambda, method, callable class instance). :type func: callable :param timeout: Maximum time in seconds to wait for func to return a truthy value. Must be a non-negative number. If timeout expires before func returns truthy, None is returned. :type timeout: float or int :param first: Time in seconds to sleep before the first attempt to call func. Useful when you know the condition won't be met immediately. Defaults to 0.0 (no initial delay). :type first: float or int :param step: Time in seconds to sleep between successive calls to func. The actual sleep happens after each failed attempt. Defaults to 1.0 second. Note that this sleep is not interrupted when timeout expires. :type step: float or int :param text: Optional debug message to log before each attempt. When provided, logs at DEBUG level with elapsed time since start. If None, no logging occurs. Useful for debugging wait operations. :type text: str or None :param args: Optional list or tuple of positional arguments to pass to func on each call. If None, defaults to empty list. :type args: list, tuple, or None :param kwargs: Optional dictionary of keyword arguments to pass to func on each call. If None, defaults to empty dict. :type kwargs: dict or None :return: The truthy return value from func if it succeeds within timeout, or None if timeout expires without func returning a truthy value. The actual return value from func is preserved (e.g., strings, numbers, lists, objects). :rtype: Any (return type of func) or None :raises: Any exception raised by func will be propagated to the caller. No exception handling is performed on func calls. Example:: >>> import os >>> # Wait for a file to exist >>> wait_for(lambda: os.path.exists("/tmp/myfile"), timeout=30, step=1) True >>> # Wait for a counter to reach threshold >>> counter = [0] >>> def check(): counter[0] += 1; return counter[0] >= 5 >>> wait_for(check, timeout=10, step=0.5) True >>> # Wait with custom function and arguments >>> def check_value(expected, current): ... return current >= expected >>> wait_for(check_value, timeout=5, step=0.1, args=[10, 15]) True >>> # Wait with debug logging >>> wait_for(lambda: False, timeout=2, step=0.5, text="Waiting for condition") None """ args = args or [] kwargs = kwargs or {} start_time = time.monotonic() end_time = start_time + timeout time.sleep(first) while time.monotonic() < end_time: if text: LOG.debug("%s (%.9f secs)", text, (time.monotonic() - start_time)) output = func(*args, **kwargs) if output: return output time.sleep(step) return None # pylint: disable=wrong-import-position from avocado.utils.deprecation import log_deprecation log_deprecation.warning("wait") avocado-113.0/codecov.yml000066400000000000000000000003741513561306000153030ustar00rootroot00000000000000coverage: status: project: default: informational: true patch: default: informational: true # When modifying this file, please validate using # curl -X POST --data-binary @codecov.yml https://codecov.io/validate avocado-113.0/contrib/000077500000000000000000000000001513561306000145725ustar00rootroot00000000000000avocado-113.0/contrib/containers/000077500000000000000000000000001513561306000167375ustar00rootroot00000000000000avocado-113.0/contrib/containers/ci/000077500000000000000000000000001513561306000173325ustar00rootroot00000000000000avocado-113.0/contrib/containers/ci/selftests/000077500000000000000000000000001513561306000213465ustar00rootroot00000000000000avocado-113.0/contrib/containers/ci/selftests/check-copr-rpm-version.docker000066400000000000000000000004141513561306000270330ustar00rootroot00000000000000# This container is used in selftests/pre_release/tests/check-copr-rpm-version.sh FROM fedora:42 LABEL description "Fedora image used on COPR RPM version check" RUN dnf -y install 'dnf-command(copr)' RUN dnf -y copr enable @avocado/avocado-latest RUN dnf -y clean all avocado-113.0/contrib/containers/ci/selftests/fedora-40.docker000066400000000000000000000005661513561306000242270ustar00rootroot00000000000000FROM fedora:40 LABEL description "Fedora image used on integration checks" RUN dnf -y install dnf-plugins-core RUN dnf -y copr enable @avocado/avocado-latest-release RUN dnf -y install dnf-plugins-core git findutils make which RUN dnf -y install diffutils python3-isort python3-enchant python3-pylint python3-autopep8 RUN dnf -y builddep python-avocado RUN dnf -y clean all avocado-113.0/contrib/containers/ci/selftests/magic.docker000066400000000000000000000007771513561306000236320ustar00rootroot00000000000000# This dockerfile copies and installs the magic example plugin, and # thus needs access to the "magic" directory. Make sure the context # (directory) is correct before building this image. One example # is by running: # $ cd examples/plugins/tests # $ buildah bud -f ../../../contrib/containers/ci/selftests/magic.docker FROM fedora:40 LABEL description "Image that contains the example magic plugin" RUN dnf -y install python3-setuptools COPY magic /tmp/magic WORKDIR /tmp/magic RUN python3 setup.py install avocado-113.0/contrib/containers/fedora-40-latest-copr.docker000066400000000000000000000010171513561306000240430ustar00rootroot00000000000000FROM fedora:40 LABEL description "Fedora image with the latest Avocado COPR build" RUN dnf -y install dnf-plugins-core RUN dnf -y copr enable @avocado/avocado-latest-release RUN dnf -y install python3-avocado python3-avocado-bash python3-avocado-common python3-avocado-examples python3-avocado-plugins-golang python3-avocado-plugins-output-html python3-avocado-plugins-result-upload python3-avocado-plugins-varianter-cit python3-avocado-plugins-varianter-pict python3-avocado-plugins-varianter-yaml-to-mux RUN dnf -y clean all avocado-113.0/contrib/docker/000077500000000000000000000000001513561306000160415ustar00rootroot00000000000000avocado-113.0/contrib/docker/Dockerfile.debian000066400000000000000000000036101513561306000212540ustar00rootroot00000000000000# This Dockerfile creates an debian image with avocado framework installed # from source # # VERSION 0.1 ################ ## Usage example: # git clone github.com/avocado-framework/avocado.git avocado.git # cd avocado.git ## Make some changes # patch -p1 < MY_PATCH ## Finally build a docker image # docker build --force-rm -t debian-avocado-custom -f contrib/docker/Dockerfile.debian . ## Run test inside the docker image # avocado run --docker fedora-avocado-custom passtest.py # FROM debian MAINTAINER Dmitry Monakhov dmonakhov@openvz.org # Install and clean in one step to decrease image size RUN apt-get update && \ echo install avocado def packages && \ apt-get install -y --no-install-recommends \ git \ rsync \ make \ gdebi-core \ pkg-config \ libvirt-dev \ python-dev \ python-lzma \ python-pip \ python-pystache \ python-setuptools \ python-yaml && \ echo install extra avocado packages && \ apt-get install -y --no-install-recommends \ ansible \ emacs-nox \ pigz \ libzip2 \ pxz && \ ln -f /usr/bin/pigz /bin/gzip && \ ln -f /usr/bin/pigz /usr/bin/gzip && \ echo install kernel-devel packages && \ apt-get install -y --no-install-recommends \ build-essential \ guilt \ bc \ flex \ bison \ libc6-dev \ libelf-dev \ libnuma-dev \ liblzma-dev && \ echo "Cleanup" && \ apt-get clean && \ rm -rf \ /usr/share/doc /usr/share/doc-base \ /usr/share/man /usr/share/locale /usr/share/zoneinfo COPY . /devel/avocado-framework/avocado RUN cd /devel/avocado-framework/avocado && \ make requirements && \ make install && \ mkdir -p /usr/share/avocado/data/cache && \ git config --global user.email "avocado@localhost" && \ git config --global user.name "Avocado tool" && \ rm -rf /devel/avocado-framework/avocado avocado-113.0/contrib/docker/Dockerfile.fedora000066400000000000000000000016711513561306000212770ustar00rootroot00000000000000# This Dockerfile creates an fedora image with avocado framework installed # from source # # VERSION 0.1 ################ ## Usage example: ## Checkout avocado source # git clone github.com/avocado-framework/avocado.git avocado.git # cd avocado.git ## Make some changes # patch -p1 < MY_PATCH ## Finally build a docker image # docker build --force-rm -t fedora-avocado-custom -f contrib/docker/Dockerfile.fedora . ## Run test inside the docker image # avocado run --docker fedora-avocado-custom passtest.py # FROM fedora MAINTAINER Dmitry Monakhov dmonakhov@openvz.org COPY . /devel/avocado-framework/avocado # Install and clean in one step to decrease image size RUN dnf install -y \ gcc \ make \ python-devel \ python-pip \ libvirt-devel \ libyaml-devel \ redhat-rpm-config \ xz-devel \ which && \ cd /devel/avocado-framework/avocado && \ make requirements && \ make install && \ dnf clean all avocado-113.0/contrib/job-scripts/000077500000000000000000000000001513561306000170315ustar00rootroot00000000000000avocado-113.0/contrib/job-scripts/coredump-handling/000077500000000000000000000000001513561306000224315ustar00rootroot00000000000000avocado-113.0/contrib/job-scripts/coredump-handling/README000066400000000000000000000024231513561306000233120ustar00rootroot00000000000000These are a set of scripts that will: * Setup the core pattern to the "coredumps" sub directory of the current job and sets the core file size limit to "unlimited" * Restore the original settings once the job finishes running tests * Links the core files (if any) on the job level "coredumps" subdir into the test result specific "coredumps" dir based on the time stamps of the test directories and the coredump files. If your system lacks the prlimit utility, then you have to set core file size limits yourself. Options to do so include running: $ ulimit -c unlimited Or setting it permanently with an entry in /etc/security/limits.conf, or adding a file to /etc/security/limits.d. To enable these scripts permanently, copy them to the Avocado systemwide job scripts directories, located by default at /etc/avocado/scripts/job, respecting the pre and post directories. To enable these scripts during a development session, set the following configuration content: [plugins.jobscripts] pre = /pre.d/ post = /post.d/ warn_non_existing_dir = True warn_non_zero_status = True WARNING: Since the core pattern is a system wide setting, if you run multiple parallel jobs with these scripts, the result is pretty much unpredictable and prone to errors. avocado-113.0/contrib/job-scripts/coredump-handling/post.d/000077500000000000000000000000001513561306000236405ustar00rootroot00000000000000avocado-113.0/contrib/job-scripts/coredump-handling/post.d/001-restore-core-pattern000077500000000000000000000016001513561306000301450ustar00rootroot00000000000000#!/bin/sh -e # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2016 # # This script restores the original settings of the systemwide kernel # core pattern once the job finishes running tests # if [ -z $AVOCADO_JOB_LOGDIR ]; then exit 1 fi BACKUP_CORE_PATTERN="$AVOCADO_JOB_LOGDIR/coredumps/.backup_core_pattern" if [ -f $BACKUP_CORE_PATTERN ]; then cat $BACKUP_CORE_PATTERN > /proc/sys/kernel/core_pattern rm $BACKUP_CORE_PATTERN fi avocado-113.0/contrib/job-scripts/coredump-handling/post.d/002-link-core-to-tests000077500000000000000000000047111513561306000275330ustar00rootroot00000000000000#!/usr/bin/env python3 # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2016 # # This script links the core files (if any) on the job level # "coredumps" subdir into the test result specific "coredumps" dir # based on the time stamps of the test directories and the timestamps # of coredump files. # import glob import os import sys if 'AVOCADO_JOB_LOGDIR' not in os.environ: sys.exit(1) AVOCADO_JOB_LOGDIR = os.environ['AVOCADO_JOB_LOGDIR'] if not os.path.isdir(AVOCADO_JOB_LOGDIR): sys.exit(2) COREDUMPS_DIR = os.path.join(AVOCADO_JOB_LOGDIR, 'coredumps') COREDUMPS = glob.glob(os.path.join(COREDUMPS_DIR, 'core.*')) if not COREDUMPS: sys.exit(0) # If we reached this far, there are core dumps, so let's attempt to # link them to the test results directories. # # This pattern list can be expanded, and if a match occurs, it should # return as the first group member the PID (an unsigned integer) of # the process which may have an associated coredump file. TESTS_DIR = os.path.join(AVOCADO_JOB_LOGDIR, 'test-results') def symlink_coredumps(): for test_dir in (os.path.join(TESTS_DIR, _) for _ in os.listdir(TESTS_DIR)): try: debug_log = os.path.join(test_dir, "debug.log") start = os.path.getctime(os.path.join(test_dir, "sysinfo")) stop = os.path.getmtime(debug_log) dst_dir = None for coredump in COREDUMPS: ctime = os.path.getctime(coredump) if ctime <= stop and ctime >= start: if not dst_dir: dst_dir = os.path.join(test_dir, "coredumps") os.makedirs(dst_dir) try: dst = os.path.join(dst_dir, os.path.basename(coredump)) os.symlink(os.path.relpath(coredump, dst_dir), dst) except Exception: # pylint: disable=W0703 pass except Exception: # pylint: disable=W0703 pass if __name__ == "__main__": symlink_coredumps() avocado-113.0/contrib/job-scripts/coredump-handling/pre.d/000077500000000000000000000000001513561306000234415ustar00rootroot00000000000000avocado-113.0/contrib/job-scripts/coredump-handling/pre.d/001-set-core-pattern000077500000000000000000000023541513561306000270650ustar00rootroot00000000000000#!/bin/sh -e # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; either version 2 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2016 # # This script set up the core pattern to the "coredumps" sub directory # of the current job and sets the core file size limit to "unlimited" if [ -z $AVOCADO_JOB_LOGDIR ]; then exit 1 fi # Skip this when not running as root if [ $UID -ne 0 ]; then (>&2 echo "Skipping coredump support because not running as root") exit 0 fi # Set the core ulimit for Avocado main process ppid=$$ prlimit --core=unlimited --pid=$(cat /proc/$ppid/status | grep PPid | cut -f2) COREDUMPS_DIR="$AVOCADO_JOB_LOGDIR/coredumps" mkdir $COREDUMPS_DIR # Save a backup of the core pattern setting cat /proc/sys/kernel/core_pattern > $COREDUMPS_DIR/.backup_core_pattern # Set core pattern echo "$COREDUMPS_DIR/core.%p" > /proc/sys/kernel/core_pattern avocado-113.0/contrib/packages/000077500000000000000000000000001513561306000163505ustar00rootroot00000000000000avocado-113.0/contrib/packages/debian/000077500000000000000000000000001513561306000175725ustar00rootroot00000000000000avocado-113.0/contrib/packages/debian/Makefile000066400000000000000000000015731513561306000212400ustar00rootroot00000000000000BUILDIR=$(CURDIR)/debian/avocado PROJECT=avocado all: @echo @echo "Debian related targets:" @echo "deb: Generate both source and binary debian packages" @echo "deb-src: Generate a source debian package" @echo "deb-bin: Generate a binary debian package" @echo deb-prepare-source: # build the source package in the parent directory # then rename it to project_version.orig.tar.gz dch -D "vivid" -M -v "$(VERSION)" "Automated (make builddeb) build." $(PYTHON) setup.py sdist $(COMPILE) --dist-dir=../ rename -f 's/$(PROJECT)-(.*)\.tar\.gz/$(PROJECT)_$$1\.orig\.tar\.gz/' ../* deb-src: deb-prepare-source # build the source package dpkg-buildpackage -S -elookkas@gmail.com -rfakeroot deb-bin: deb-prepare-source # build binary package dpkg-buildpackage -b -rfakeroot deb: deb-prepare-source # build both source and binary packages dpkg-buildpackage -i -I -rfakeroot avocado-113.0/contrib/packages/debian/README000066400000000000000000000006541513561306000204570ustar00rootroot00000000000000This is a collection of files that are useful to generate packages for Debian(-like) systems. Their "contrib" status mean that they should work, but there's no support for them. If there are users out there interested in keeping Debian(-like) packages update, please step up. The basic usage is to run the various targets defined in the Makefile, such as running `make deb` to have both source and binary packages generated. avocado-113.0/contrib/packages/debian/changelog000066400000000000000000000040571513561306000214520ustar00rootroot00000000000000avocado (91.0) buster; urgency=medium * contrib/packages/debian/control: Fix debian package dependencies * debian: Increment compat version * debian: Update package version -- Anton Mikanovich Thu, 02 Sep 2021 10:09:44 +0000 avocado (85.0) buster; urgency=medium * Non-maintainer upload. * Update version. -- Baurzhan Ismagulov Sun, 14 Feb 2021 15:54:38 +0100 avocado (0.29.0) vivid; urgency=medium * Automated (make builddeb) build. -- Lucas Meneghel Rodrigues (lmr) Fri, 09 Oct 2015 09:32:33 -0300 avocado (0.21.0) utopic; urgency=medium * Automated (make builddeb) build. -- Lucas Meneghel Rodrigues (lmr) Thu, 19 Mar 2015 15:48:56 -0300 avocado (0.20.1) trusty; urgency=medium * Automated (make builddeb) build. -- Lucas Meneghel Rodrigues (lmr) Thu, 12 Feb 2015 11:40:36 -0200 avocado (0.14.0) trusty; urgency=medium * Add new upstream version 0.14.0. -- Lucas Meneghel Rodrigues (lmr) Wed, 10 Dec 2014 18:13:44 -0200 avocado (0.11.1) trusty; urgency=medium * Add new upstream version 0.11.1. -- Lucas Meneghel Rodrigues (lmr) Mon, 25 Aug 2014 10:14:34 -0300 avocado (0.0.1ubuntu4) saucy; urgency=medium * Automated (make builddeb) build. -- Lucas Meneghel Rodrigues (lmr) Wed, 02 Apr 2014 13:39:36 -0300 avocado (0.0.1ubuntu3) saucy; urgency=medium * Automated (make builddeb) build. -- Lucas Meneghel Rodrigues (lmr) Wed, 02 Apr 2014 13:39:07 -0300 avocado (0.0.1ubuntu2) saucy; urgency=medium * Automated (make builddeb) build. -- Lucas Meneghel Rodrigues (lmr) Wed, 02 Apr 2014 13:35:43 -0300 avocado (0.0.1ubuntu1) saucy; urgency=medium * Automated (make builddeb) build. -- Lucas Meneghel Rodrigues (lmr) Wed, 02 Apr 2014 13:35:31 -0300 avocado (0.0.1) saucy; urgency=low * Initial release. -- Lucas Meneghel Rodrigues (lmr) Wed, 02 Apr 2014 14:12:43 -0200 avocado-113.0/contrib/packages/debian/compat000066400000000000000000000000031513561306000207710ustar00rootroot0000000000000010 avocado-113.0/contrib/packages/debian/control000066400000000000000000000007171513561306000212020ustar00rootroot00000000000000Source: avocado Section: python Priority: optional Maintainer: Lucas Meneghel Rodrigues (lmr) Build-Depends: debhelper (>=7.0.50~), cdbs (>= 0.4.49), python3, python3-setuptools, dh-python Standards-Version: 3.8.4 Package: avocado Architecture: all Homepage: http://avocado-framework.github.io/ X-Python-Version: ${python:Versions} Depends: ${misc:Depends}, ${python3:Depends} Description: Avocado is a framework for fully automated testing. avocado-113.0/contrib/packages/debian/copyright000066400000000000000000000016401513561306000215260ustar00rootroot00000000000000Upstream Author: Lucas Meneghel Rodrigues Files: * Copyright: 2013-2014, Lucas Meneghel Rodrigues, RudÃĄ Moura License: GPL Files: debian/* Copyright: 2012, Lucas Meneghel Rodrigues (lmr) License: GPL License: GPL This package is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; version 2 only. This package is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this package; if not, see . On Debian systems, the complete text of the GNU General Public License 2 can be found in `/usr/share/common-licenses/GPL-2'. avocado-113.0/contrib/packages/debian/rules000077500000000000000000000002431513561306000206510ustar00rootroot00000000000000#!/usr/bin/make -f # -*- makefile -*- clean:: rm -rf build build-stamp configure-stamp build/ MANIFEST dh_clean %: dh $@ --with python3 --buildsystem=pybuild avocado-113.0/contrib/packages/debian/source/000077500000000000000000000000001513561306000210725ustar00rootroot00000000000000avocado-113.0/contrib/packages/debian/source/format000066400000000000000000000000151513561306000223010ustar00rootroot000000000000003.0 (native) avocado-113.0/contrib/packages/debian/source/options000066400000000000000000000000261513561306000225060ustar00rootroot00000000000000tar-ignore = ".git/*" avocado-113.0/contrib/packages/debian/watch000066400000000000000000000001251513561306000206210ustar00rootroot00000000000000version=3 https://github.com/avocado-framework/avocado/tags .*/(\d[\d\.]+)\.tar\.gz avocado-113.0/contrib/scripts/000077500000000000000000000000001513561306000162615ustar00rootroot00000000000000avocado-113.0/contrib/scripts/avocado-bisect.sh000077500000000000000000000037371513561306000215150ustar00rootroot00000000000000#!/bin/sh # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; specifically version 2 of the License. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: 2017 Red Hat, Inc. # Author: Lukas Doktor # # This script helps with bisecting avocado sources by doing the prep # and cleanup work needed to successfully bisect avocado execution # Use it by running: # $ git bisect start $BAD $GOOD # $ git bisect run ./contrib/scripts/avocado-bisect.sh $YOUR_CMD # ERR=() INTERACTIVE=0 case $1 in "-h") echo "$0 [-h] [-i] [CHECK_CMD [...]]" echo " -h Show this help" echo " -i On failure ask whether to ignore this failure" echo " CHECK_CMD Check cmd(s), when not supplied 'make check' is used" exit 1 ;; "-i") shift INTERACTIVE=1 ;; esac run() { echo -e "\n\e[32mRunning '$*'\e[0m" eval $* if [ $? != 0 ]; then if [ $INTERACTIVE -eq 1 ]; then echo -ne "\e[33mUse 'y' to ignore this failure: \e[0m" read RES [ "$RES" == "y" ] && echo -e "\e[33mStatus of $* overridden to PASS\[0m" && return 0 fi echo -e "\e[31m$* FAILED\e[0m" ERR+=("$1") else echo -e "\e[32m$* PASSED\e[0m\n" fi } run "git log -1 --oneline" run "make develop" if [ "$*" ]; then run $* else run "make check" fi run "make clean" if [ "$ERR" ]; then echo -e "\e[31m" echo "Failed checks of commit $(git log -1 --oneline):" for CMD in "${ERR[@]}"; do echo -e " * $CMD FAILED" done echo -ne "\e[0m" else echo -e "\e[32mAll checks PASSED\e[0m" fi if [ "$ERR" ]; then exit 1 fi exit 0 avocado-113.0/contrib/scripts/avocado-check-pr.sh000077500000000000000000000213571513561306000217360ustar00rootroot00000000000000#!/bin/bash # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; specifically version 2 of the License. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: 2017 Red Hat, Inc. # Author: Lukas Doktor usage() { cat << EOF Usage: $0 [-i] [-v] [-h] [-u] [-t] [-r] [-b] [-d] [-D] -i: Interactive mode (on failure allow to override result) -v: Verbose mode -h: Help -u: Override user name (from cfg) -t: Override token (from cfg) -r: Override remote origin (origin) -b: Override the target branch (\$current-branch) -d: Enable debug mode -D: Dry run (no checking/publishing) Verifies that all commits between the oldest new commit compare to origin/master pass the "make check" and publishes the results to github. Example workflow: git checkout master git merge ... $0 git push origin master In order to be able to push results one has to create ~/.config/github_checker.ini with user and token in it: user = my_github_user token = 123456789012345678901234567890 # remote, interactive EOF } cleanup() { [ "$tmp" ] && rm "$tmp" [ "$BRANCH" ] && git checkout $BRANCH &>/dev/null } set_status() { # Sets status to given commit on github local base_url="$1"; shift # base github repo url local commit="$1"; shift # commit-sha local status="$1"; shift # status (success, failure, pending, ...) local result_url="$1"; shift # url to detailed results (log) local description="$*" # long(er) description data="\"state\": \"$status\", \"description\": \"$description\", \"context\": \"manual/$GITHUB_USER\"" [ "$result_url" ] && data+=", \"target_url\": \"$result_url\"" if [ "$DEBUG" ]; then echo curl -u $GITHUB_USER:$GITHUB_TOKEN --data "{$data}" -H "Accept: application/vnd.github.v3+json" "$base_url/statuses/$commit" [ "$DRY" ] || curl -u $GITHUB_USER:$GITHUB_TOKEN --data "{$data}" -H "Accept: application/vnd.github.v3+json" "$base_url/statuses/$commit" else [ "$DRY" ] || curl -u $GITHUB_USER:$GITHUB_TOKEN --data "{$data}" -H "Accept: application/vnd.github.v3+json" "$base_url/statuses/$commit" &>/dev/null fi } check() { # Checks the current commit local outfile="$1" local status=-1 local description="Checked by $GITHUB_USER" local tmp=$(mktemp) make check &>> "$tmp" if [ $? -eq 0 ]; then status=0 elif [ $INTERACTIVE ]; then cat $tmp echo -ne "\e[33mUse 'y' to ignore this failure or 'a' to abort execution: \e[0m" read RES case $RES in "y") echo -e "\e[33mForce-updating results to PASS\e[0m" status=1 ;; "a") echo -e "\e[33mAborting execution\e[0m" cleanup exit -1 ;; esac fi if [ $status -eq -1 ]; then echo >> "$outfile" echo >> "$outfile" echo "------------------------------< BEGIN >------------------------------" >> "$outfile" cat "$tmp" >> "$outfile" echo "------------------------------< END >------------------------------" >> "$outfile" fi rm $tmp return $status } publish_results() { # Processes results, publishes log and updates status on github local commit="$1"; shift local passed="$1"; shift local forced="$1"; shift local failed="$1"; shift local all_failures="$1" local description="" [ "$failed" ] && status="failure" || status="success" [ "$all_failures" ] && [ "$all_failures" -ne 0 ] && status="failure" && description+="#failures: $all_failures; " [ "$passed" ] && description+="passed: ${passed:0:-1}; " [ "$forced" ] && description+="forced: ${forced:0:-1}; " [ "$failed" ] && description+="failed: ${failed:0:-1}; " description=${description:0:-2} if [ ${#description} -gt 140 ]; then # Description can only be 140 chars long if [ ${#failed} -ne 0 ]; then description="FAIL: ${failed[@]}" [ ${#description} -gt 140 ] && description="${description:0:130}..." else description="ALL PASS" fi fi [ "$DRY" ] || url=$(fpaste -x 604800 $tmp | tail -n 1) set_status "$BASE_URL" "$commit" "$status" "${url%%/raw/}" "$description" } # Pre-exec check [ "$(git diff)" ] && { usage; echo; echo "Uncommitted changes present!"; exit -1; } which fpaste &>/dev/null || { usage; echo; echo "fpaste is necessary to run this script!"; exit -1; } # Global defaults GIT_REMOTE="origin" # Read config file if [ -e "$HOME/.config/github_checker.ini" ]; then exec {fd}<"$HOME/.config/github_checker.ini" while read -u $fd line; do if [[ "$line" =~ ^user\ *=\ *(.*)$ ]]; then GITHUB_USER=${BASH_REMATCH[1]} elif [[ "$line" =~ ^token\ *=\ *(.*)$ ]]; then GITHUB_TOKEN=${BASH_REMATCH[1]} elif [[ "$line" =~ ^remote\ *=\ *(.*)$ ]]; then GIT_REMOTE=${BASH_REMATCH[1]} elif [[ "$line" =~ ^interactive\ *=\ *(.*)$ ]]; then INTERACTIVE=${BASH_REMATCH[1]} fi done fi # Parse arguments while getopts "ivu:t:r:b:dhD" opt; do case $opt in i) INTERACTIVE=1 ;; v) VERBOSE=1 ;; u) GITHUB_USER="$OPTARG" ;; t) GITHUB_TOKEN="$OPTARG" ;; r) GIT_REMOTE="$OPTARG" ;; b) REMOTE_BRANCH="$OPTARG" ;; d) DEBUG=1 ;; D) DRY=1 ;; *) usage exit 1 ;; esac done BRANCH=$(git rev-parse --abbrev-ref HEAD) [ "$REMOTE_BRANCH" ] || REMOTE_BRANCH="$BRANCH" if [ "$VERBOSE" ]; then echo "GITHUB_USER: $GITHUB_USER" echo "GITHUB_TOKEN_LENGTH: ${#GITHUB_TOKEN}" echo "GIT_REMOTE: $GIT_REMOTE" echo "LOCAL_BRANCH: $BRANCH" echo "REMOTE_BRANCH: $REMOTE_BRANCH" fi [ ! "$GITHUB_USER" ] || [ ! "$GITHUB_TOKEN" ] && { usage; echo; echo "Github user or token not specified" exit -1; } # Get commit range GIT_URL="$(git remote get-url $GIT_REMOTE)" BASE_URL="https://api.github.com/repos/${GIT_URL:19}" # Commits that should be part of the PR GIT_PR_COMMITS=$(git cherry $GIT_REMOTE/$REMOTE_BRANCH | sed -n 's/+ \(.*\)/\1/p') # All commits between first-1 commit and HEAD GIT_RANGE=$(git rev-list --reverse $(echo "$GIT_PR_COMMITS" | head -n 1)~1..HEAD) [ "$?" -ne 0 ] && { echo "Error parsing git range using $GIT_REMOTE/$REMOTE_BRANCH"; exit -1; } echo "Checking $BRANCH..$GIT_REMOTE/$REMOTE_BRANCH ($(echo "$GIT_PR_COMMITS" | wc -l | xargs)/$(echo "$GIT_RANGE" | wc -l | xargs))" if [ "$VERBOSE" ]; then echo "PR commits:" echo "$GIT_PR_COMMITS" echo fi if [ "$DEBUG" ]; then echo "All commits:" for commit in $GIT_RANGE; do echo "$commit" "$(echo "$GIT_PR_COMMITS" | grep -q "$commit" && echo "*")" done echo fi for commit in $GIT_PR_COMMITS; do set_status "$BASE_URL" "$commit" "pending" "" "Starting manual check" done [ "$VERBOSE" ] && { echo; echo "Starting incremental check"; } all_failures=0 passed="" forced="" failed="" tmp=$(mktemp /tmp/check-XXXXXX) for commit in $GIT_RANGE; do if echo "$GIT_PR_COMMITS" | grep -q "$commit"; then if [ "$previous_pr_commit" ]; then # This commit belongs to PR, publish results of previous commit publish_results "$previous_pr_commit" "$passed" "$forced" "$failed" passed="" forced="" failed="" rm "$tmp" tmp="$(mktemp /tmp/check-XXXXXX)" fi previous_pr_commit=$commit fi git checkout $commit &>/dev/null [ "$DEBUG" ] && echo "Checking $commit" [ "$DRY" ] || check "$tmp" case $? in 0) passed+="${commit:0:6}," echo -n "." echo "PASS: $commit" >> "$tmp" ;; 1) forced+="${commit:0:6}," echo -n "+" echo "FORCE PASS: $commit" >> "$tmp" ;; *) all_failures=$(expr $all_failures + 1) failed+="${commit:0:6}," [ "$VERBOSE" ] && echo -e "\nFAIL: $commit" || echo -n "F" echo "FAIL|ERROR: $commit" >> "$tmp" ;; esac done if [ "$previous_pr_commit" ]; then publish_results "$previous_pr_commit" "$passed" "$forced" "$failed" "$all_failures" echo else echo "No commits found/checked!" exit -1 fi cleanup avocado-113.0/contrib/scripts/avocado-fetch-eggs.py000077500000000000000000000050351513561306000222670ustar00rootroot00000000000000#!/bin/env python3 # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; specifically version 2 of the License. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: 2021 Red Hat, Inc. # Author: Beraldo Leal import logging import sys from avocado.core.settings import settings from avocado.core.version import VERSION from avocado.utils.asset import Asset CACHE_DIRS = settings.as_dict().get("datadir.paths.cache_dirs") # Avocado asset lib already has its logger. Let's use it. LOG = logging.getLogger("avocado.utils.asset") def configure_logging_settings(): LOG.setLevel(logging.INFO) logger_handler = logging.StreamHandler() LOG.addHandler(logger_handler) formatter = logging.Formatter("%(levelname)s: %(message)s") logger_handler.setFormatter(formatter) def get_setuptools_egg_url(python_version=None): if python_version is None: version = sys.version_info python_version = f"{version.major}.{version.minor}" return f"https://github.com/avocado-framework/setuptools/releases/download/v59.2.0/setuptools-59.2.0-py{python_version}.egg" def get_avocado_egg_url(avocado_version=None, python_version=None): if avocado_version is None: avocado_version = VERSION if python_version is None: version = sys.version_info python_version = f"{version.major}.{version.minor}" asset = f"avocado_framework-{avocado_version}-py{python_version}.egg" return f"https://github.com/avocado-framework/avocado/releases/download/{avocado_version}/{asset}" def main(): configure_logging_settings() for version in ["3.9", "3.10", "3.11", "3.12", "3.13"]: url = get_avocado_egg_url(python_version=version) try: asset = Asset(url, cache_dirs=CACHE_DIRS) asset.fetch() except OSError: LOG.error("Failed to fetch Avocado egg for Python version %s", version) return 1 url = get_setuptools_egg_url(python_version=version) try: asset = Asset(url, cache_dirs=CACHE_DIRS) asset.fetch() except OSError: LOG.error("Failed to fetch setuptools egg for Python version %s", version) return 1 return 0 if __name__ == "__main__": sys.exit(main()) avocado-113.0/contrib/scripts/avocado-get-job-results-dir.py000077500000000000000000000021031513561306000240460ustar00rootroot00000000000000#!/usr/bin/env python3 # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; specifically version 2 of the License. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2016 # Author: Amador Pahim # # Simple script that, given the Job (partial) ID, returns the job # results directory. # # $ python avocado-get-job-results-dir.py # import sys from avocado.core import data_dir if __name__ == "__main__": if len(sys.argv) < 2: sys.stderr.write("Please inform the Job ID.\n") sys.exit(-1) resultsdir = data_dir.get_job_results_dir(sys.argv[1]) if resultsdir is None: sys.stderr.write(f"Can't find job results directory for '{sys.argv[1]}'\n") sys.exit(-1) sys.stdout.write(f"{resultsdir}\n") avocado-113.0/contrib/scripts/avocado-safeloader-find-avocado-instrumented000077500000000000000000000027611513561306000270030ustar00rootroot00000000000000#!/usr/bin/env python3 # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; specifically version 2 of the License. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2021 # Author: Cleber Rosa # # Simple script that, given Python file containing unittests, returns the canonical # location of each test, each one on its own line. The idea is to be able # to filter the tests you want to run by doing something like: # # $ avocado run `avocado-safeloader-find-avocado-instrumented | | xargs` # import os import sys from avocado.core.safeloader import find_avocado_tests if __name__ == '__main__': test_module_paths = sys.argv[1:] result = [] for test_module_path in test_module_paths: try: test_class_methods, _ = find_avocado_tests(test_module_path) except IOError as error: continue for klass, methods in test_class_methods.items(): test_module_name = os.path.relpath(test_module_path) result += ["%s:%s.%s" % (test_module_name, klass, method_name) for (method_name, _, _) in methods] if result: print("\n".join(result)) avocado-113.0/contrib/scripts/avocado-safeloader-find-python-unittest000077500000000000000000000035621513561306000260460ustar00rootroot00000000000000#!/usr/bin/env python3 # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; specifically version 2 of the License. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2016 # Author: Cleber Rosa # # Simple script that, given Python file containing unittests, returns the canonical # location of each test, each one on its own line. The idea is to be able # to filter the tests you want to run by doing something like: # # $ avocado run `avocado-safeloader-find-python-unittests | | xargs` # import os import sys from functools import cmp_to_key from unittest.util import three_way_cmp from avocado.core.safeloader import find_python_unittests if __name__ == '__main__': test_module_paths = sys.argv[1:] result = [] for test_module_path in test_module_paths: try: test_class_methods = find_python_unittests(test_module_path) except IOError as error: continue for klass, methods in test_class_methods.items(): if test_module_path.endswith(".py"): test_module_path = test_module_path[:-3] test_module_name = os.path.relpath(test_module_path) test_module_name = test_module_name.replace(os.path.sep, ".") result += ["%s.%s.%s" % (test_module_name, klass, method_name) for (method_name, _, _) in methods] if result: if os.environ.get('AVOCADO_SAFELOADER_UNITTEST_ORDER_COMPAT', False): result.sort(key=cmp_to_key(three_way_cmp)) print("\n".join(result)) avocado-113.0/contrib/scripts/docs-change-home.sh000077500000000000000000000003241513561306000217200ustar00rootroot00000000000000#!/bin/sh -e # Script to keep the references to a home dir (in docs) without specific names git grep /home/ -- docs/source/ man/ | cut -d ':' -f 1 | sort -u | xargs sed -i -E 's/\/home\/[a-z]+\//\/home\/user\//' avocado-113.0/contrib/scripts/find-python-unittest000077500000000000000000000035461513561306000223330ustar00rootroot00000000000000#!/usr/bin/env python3 # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; specifically version 2 of the License. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: Red Hat Inc. 2021 # Author: Cleber Rosa # # Simple script that, given Python file containing unittests, returns the canonical # location of each test, each one on its own line. # # Warning: this script uses Python's unittest TestLoader.discover() # and will load (and thus eval) the Python files given in the command # as arguments. # import os import sys import unittest def get_tests(suite): result = [] if hasattr(suite, '__iter__'): for suite_or_test in suite: result.extend(get_tests(suite_or_test)) else: result.append(suite) return result if __name__ == '__main__': if sys.version_info < (3, 8, 0) or sys.version_info >= (3, 9, 6): abs_path = os.path.abspath(__file__) top_path = os.path.dirname(os.path.dirname(os.path.dirname(abs_path))) sys.path.insert(0, top_path) test_module_paths = sys.argv[1:] result = [] loader = unittest.TestLoader() for test_module_path in test_module_paths: start_dir, pattern = os.path.split(test_module_path) suite = loader.discover(start_dir, pattern) tests = get_tests(suite) module_name = start_dir.replace(os.path.sep, ".") result.extend(["%s.%s" % (module_name, test.id()) for test in tests]) if result: print("\n".join(result)) for err in loader.errors: print("ERROR: ", err, file=sys.stderr) avocado-113.0/contrib/scripts/summarize-job-failures.py000077500000000000000000000042221513561306000232320ustar00rootroot00000000000000#!/usr/bin/env python3 """ Script supposed to be given paths to "results.json" files with same set of tests. It points out which tests have *not* passed in the complete set of jobs. Useful for running say a 100 jobs, to validate the stability of jobs/tests and the lack of intermittent failures, race conditions, etc. """ import collections import json import sys try: from avocado.core.teststatus import STATUSES_NOT_OK except ImportError: STATUSES_NOT_OK = ["ERROR", "FAIL", "INTERRUPTED"] def get_one_job_results(path, results): with open(path, encoding="utf-8") as result_json: js = json.load(result_json) job = {} for test in js["tests"]: test_id = test["id"] if results: # we have already parsed at least one job, so that will be # used as the baseline for the set of tests that should be # present on every job if test_id not in results: print(f'Test ID "{test_id}" is not present in a previous job, aborting') sys.exit(1) status = test["status"] job[test_id] = status return job def merge_results(results, result_json_paths): for result_json_path in result_json_paths: job = get_one_job_results(result_json_path, results) for test_id_key, test_id_value in job.items(): if test_id_key not in results: results[test_id_key] = {} status = test_id_value if status not in results[test_id_key]: results[test_id_key][status] = 0 results[test_id_key][status] += 1 def main(): all_job_results = collections.OrderedDict() merge_results(all_job_results, sys.argv[1:]) number_of_jobs = len(sys.argv[1:]) any_failure = False for test in all_job_results: if set(all_job_results[test].keys()).intersection(STATUSES_NOT_OK): any_failure = True print(test) for status, count in all_job_results[test].items(): print(f" - {status}: {count}") print("Jobs analyzed:", number_of_jobs, "(all PASSed)" if not any_failure else "") if __name__ == "__main__": main() avocado-113.0/contrib/scripts/task-state-machine.py000066400000000000000000000161111513561306000223150ustar00rootroot00000000000000import asyncio import itertools import random import time from avocado.utils.astring import tabular_output DEBUG = False def debug(msg): if DEBUG: print(msg) async def sleep_random(): await asyncio.sleep(random.random()) def true_or_false(handicap=3): """Returns a random positive or negative outcome, with some bias.""" if handicap > 1: choices = [True] + ([False] * handicap) else: choices = [False] + ([True] * abs(handicap)) return random.choice(choices) def mock_check_task_requirement(): # More success than failures, please return true_or_false(-8) def mock_check_task_start(): # More success than failures, please return true_or_false(-6) def mock_monitor_task_finished(): # More failures than successes, please return true_or_false(5) class Task: """Used here as a placeholder for an avocado.core.nrunner.Task.""" def __init__(self, identification): self._identification = identification class TaskInfo(Task): """Task with extra status information on its life-cycle. The equivalent of a StatusServer will contain this information in the real implementation.""" def __init__(self, identification): super().__init__(identification) self._status = None self._timeout = None @property def status(self): return self._status @status.setter def status(self, status): self._status = status @property def timeout(self): return self._timeout @timeout.setter def timeout(self, timeout): self._timeout = timeout def __repr__(self): if self._status is None: return f"{self._identification}" else: return f"{self._identification} ({self.status})" class TaskStateMachine: """Represents all phases that a task can go through its life.""" def __init__(self, tasks): self._requested = tasks self._triaging = [] self._ready = [] self._started = [] self._finished = [] self._lock = asyncio.Lock() @property def requested(self): return self._requested @property def triaging(self): return self._triaging @property def ready(self): return self._ready @property def started(self): return self._started @property def finished(self): return self._finished @property def lock(self): return self._lock @property async def complete(self): async with self._lock: pending = any([self._requested, self._triaging, self._ready, self._started]) return not pending def __str__(self): headers = ( "|_REQUESTED_|", "|_TRIAGING__|", "|___READY___|", "|__STARTED__|", "|______FINISHED_______|", ) data = itertools.zip_longest( self._requested, self._triaging, self._ready, self._started, self._finished, fillvalue="", ) matrix = [_ for _ in data] return tabular_output(matrix, headers) async def bootstrap(lc): """Reads from requested, moves into triaging.""" # fake some rate limiting if true_or_false(10): return try: async with lc.lock: task = lc.requested.pop() lc.triaging.append(task) debug(f"Moved Task {task}: REQUESTED => TRIAGING") except IndexError: debug("BOOTSTRAP: nothing to do") return async def triage(lc): """Reads from triaging, moves into either: ready or finished.""" await sleep_random() try: async with lc.lock: task = lc.triaging.pop() except IndexError: debug("TRIAGE done") return if mock_check_task_requirement(): async with lc.lock: lc.ready.append(task) debug(f"Moving Task {task}: TRIAGING => READY") else: async with lc.lock: lc.finished.append(task) task.status = "FAILED ON TRIAGE" debug(f"Moving Task {task}: TRIAGING => FINISHED") async def start(lc): """Reads from ready, moves into either: started or finished.""" await sleep_random() try: async with lc.lock: task = lc.ready.pop() except IndexError: debug("START: nothing to do") return # enforce a rate limit on the number of started (currently running) tasks. # this is a global limit, but the spawners can also be queried with regards # to their capacity to handle new tasks MAX_RUNNING_TASKS = 8 async with lc.lock: if len(lc.started) >= MAX_RUNNING_TASKS: lc.ready.insert(0, task) task.status = "WAITING" return # suppose we're starting the tasks if mock_check_task_start(): async with lc.lock: task.status = None # Let's give each task 15 seconds from start time task.timeout = time.monotonic() + 15 lc.started.append(task) debug(f"Moving Task {task}: READY => STARTED") else: async with lc.lock: lc.finished.append(task) task.status = "FAILED ON START" debug(f"Moving Task {task}: READY => FINISHED (ERRORED ON START)") async def monitor(lc): """Reads from started, moves into finished.""" await sleep_random() try: async with lc.lock: task = lc.started.pop() except IndexError: debug("MONITOR: nothing to do") return if time.monotonic() > task.timeout: async with lc.lock: task.status = "FAILED W/ TIMEOUT" lc.finished.append(task) debug(f"Moving Task {task}: STARTED => FINISHED (FAILED ON TIMEOUT)") elif mock_monitor_task_finished(): async with lc.lock: lc.finished.append(task) debug(f"Moving Task {task}: STARTED => FINISHED (COMPLETED AFTER STARTED)") else: async with lc.lock: lc.started.insert(0, task) debug(f"Task {task}: has not finished yet") def print_lc_status(lc): print("\033c", end="") print(str(lc)) async def worker(lc): """Pushes Tasks forward and makes them do something with their lives.""" while True: complete = await lc.complete debug(f"Complete? {complete}") if complete: break await bootstrap(lc) print_lc_status(lc) await triage(lc) print_lc_status(lc) await start(lc) print_lc_status(lc) await monitor(lc) print_lc_status(lc) if __name__ == "__main__": NUMBER_OF_TASKS = 40 NUMBER_OF_LIFECYCLE_WORKERS = 4 tasks_info = [ # pylint: disable=C0209 TaskInfo("%03i" % _) for _ in range(1, NUMBER_OF_TASKS - 1) ] state_machine = TaskStateMachine(tasks_info) loop = asyncio.get_event_loop() workers = [ loop.create_task(worker(state_machine)) for _ in range(NUMBER_OF_LIFECYCLE_WORKERS) ] loop.run_until_complete(asyncio.gather(*workers)) print("JOB COMPLETED") avocado-113.0/contrib/scripts/vmimage-populate-cache.py000077500000000000000000000020751513561306000231570ustar00rootroot00000000000000#!/usr/bin/env python3 """ Script that downloads cloud images via avocado.utils.vmimage """ from avocado.core.settings import settings from avocado.utils import vmimage KNOWN_IMAGES = ( # from https://download.cirros-cloud.net/0.4.0/MD5SUMS ("cirros", "0.4.0", "aarch64", "ecc9a5132e7a0f11a4c585f513cd0873", "md5"), ("cirros", "0.4.0", "arm", "7e9cfcb763e83573a4b9d9315f56cc5f", "md5"), ("cirros", "0.4.0", "i386", "b7d8ac291c698c3f1dc0705ce52a3b64", "md5"), ("cirros", "0.4.0", "x86_64", "443b7623e27ecf03dc9e01ee93f67afe", "md5"), ) def main(): for image in KNOWN_IMAGES: name, version, arch, checksum, algorithm = image print(f"{name} version {version} ({arch}): ", end="") cache_dir = settings.as_dict().get("datadir.paths.cache_dirs")[0] download = vmimage.get( name=name, version=version, arch=arch, checksum=checksum, algorithm=algorithm, cache_dir=cache_dir, ) print(download.base_image) if __name__ == "__main__": main() avocado-113.0/contrib/testsuites/000077500000000000000000000000001513561306000170065ustar00rootroot00000000000000avocado-113.0/contrib/testsuites/run-kvm-unit-test.sh000077500000000000000000000061631513561306000227040ustar00rootroot00000000000000#!/bin/sh # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; specifically version 2 of the License. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # See LICENSE for more details. # # Copyright: 2016 Red Hat, Inc. # Author: Lukas Doktor # # This script runs kvm-unit-tests as individual tests inside avocado. # Optionally it downloads the kvm-unit-tests from git. # # Parse arguments WILDCARD="*" while [ true ]; do case $1 in "--endian") shift ENDIAN="--endian=$1" ;; "--configure-args") shift CONFIGURE_ARGS=$1 ;; "--branch") shift BRANCH=$1 ;; "--path") shift KVM_UNIT_TEST=$1 ;; "--wildcard") shift WILDCARD="$1" ;; "-h"|"--help") echo "Usage: $0 [-h] [--endian ENDIAN] [--configure-args KVM_UNIT_TEST_CONFIGURE_ARGS] [--branch KVM_UNIT_TEST_BRANCH] [--path PATH] [--wildcard WILDCARD] [avocado arguments ...]" echo echo " -h Show this help" echo " --endian Endian flag to kvm-unit-test configure" echo " --configure-args Arguments given to configure kvm-unit-tests" echo " --branch Branch/tag of kvm-unit-tests to clone (default is master)" echo " --path Path to kvm-unit-test suite (default is tmp)" echo " --wildcard BASH Wildcard to select tests (by default all)" echo echo "Note: You might need to set ACCEL and/or QEMU env variables." exit 1 ;; *) break esac shift done CONF_FILE=~/.config/avocado/avocado.conf restore_config() { rm $CONF_FILE 2>/dev/null mv "$CONF_FILE".kvm-unit-tests "$CONF_FILE" 2>/dev/null } setup_skip_exitcode() { mkdir -p $(dirname $CONF_FILE) [ -f $CONF_FILE ] && cp "$CONF_FILE" "$CONF_FILE".kvm-unit-tests trap restore_config EXIT echo "[runner.exectest.exitcodes]" >>$CONF_FILE echo "skip = [2, 77]" >>$CONF_FILE } # Initialize directory and download kvm-unit-test if necessary [ "$KVM_UNIT_TEST" ] || { KVM_UNIT_TEST="$(mktemp -d)"; CLEAN_DIR=true; } [ -n "$BRANCH" ] || BRANCH="master" [ -d "$KVM_UNIT_TEST" ] || mkdir -p "$KVM_UNIT_TEST" cd "$KVM_UNIT_TEST" [ -f "configure" ] || git clone --depth 1 -b $BRANCH -q https://gitlab.com/kvm-unit-tests/kvm-unit-tests.git . || exit # Compile kvm-unit-test as standalone to get tests as separate files ./configure $ENDIAN $CONFIGURE_ARGS || { echo Fail to configure kvm-unit-test; exit -1; } make standalone >/dev/null || { echo Fail to "make standalone" kvm-unit-test; exit -1; } setup_skip_exitcode cd tests eval "avocado run ./$WILDCARD $*" RET=$? # Cleanup and exit [ "$CLEAN_DIR" ] && rm -Rf "$KVM_UNIT_TEST" exit $RET avocado-113.0/docs/000077500000000000000000000000001513561306000140625ustar00rootroot00000000000000avocado-113.0/docs/Makefile000066400000000000000000000130571513561306000155300ustar00rootroot00000000000000# Makefile for Sphinx documentation # ifndef PYTHON PYTHON=$(shell which python3 2>/dev/null || which python 2>/dev/null) endif # You can set these variables from the command line. SPHINXOPTS = -W SPHINXBUILD = $(PYTHON) -m sphinx PAPER = BUILDDIR = build # Internal variables. PAPEROPT_a4 = -D latex_paper_size=a4 PAPEROPT_letter = -D latex_paper_size=letter ALLSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) source # the i18n builder cannot share the environment and doctrees with the others I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) source .PHONY: help clean html dirhtml singlehtml pickle json htmlhelp qthelp devhelp epub latex latexpdf text man changes linkcheck doctest gettext help: @echo "Please use \`make ' where is one of" @echo " html to make standalone HTML files" @echo " dirhtml to make HTML files named index.html in directories" @echo " singlehtml to make a single large HTML file" @echo " pickle to make pickle files" @echo " json to make JSON files" @echo " htmlhelp to make HTML files and a HTML help project" @echo " qthelp to make HTML files and a qthelp project" @echo " devhelp to make HTML files and a Devhelp project" @echo " epub to make an epub" @echo " latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter" @echo " latexpdf to make LaTeX files and run them through pdflatex" @echo " text to make text files" @echo " man to make manual pages" @echo " texinfo to make Texinfo files" @echo " info to make Texinfo files and run them through makeinfo" @echo " gettext to make PO message catalogs" @echo " changes to make an overview of all changed/added/deprecated items" @echo " linkcheck to check all external links for integrity" @echo " doctest to run all doctests embedded in the documentation (if enabled)" clean: -rm -rf $(BUILDDIR) html: $(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html @echo @echo "Build finished. The HTML pages are in $(BUILDDIR)/html." dirhtml: $(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml @echo @echo "Build finished. The HTML pages are in $(BUILDDIR)/dirhtml." singlehtml: $(SPHINXBUILD) -b singlehtml $(ALLSPHINXOPTS) $(BUILDDIR)/singlehtml @echo @echo "Build finished. The HTML page is in $(BUILDDIR)/singlehtml." pickle: $(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) $(BUILDDIR)/pickle @echo @echo "Build finished; now you can process the pickle files." json: $(SPHINXBUILD) -b json $(ALLSPHINXOPTS) $(BUILDDIR)/json @echo @echo "Build finished; now you can process the JSON files." htmlhelp: $(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) $(BUILDDIR)/htmlhelp @echo @echo "Build finished; now you can run HTML Help Workshop with the" \ ".hhp project file in $(BUILDDIR)/htmlhelp." qthelp: $(SPHINXBUILD) -b qthelp $(ALLSPHINXOPTS) $(BUILDDIR)/qthelp @echo @echo "Build finished; now you can run "qcollectiongenerator" with the" \ ".qhcp project file in $(BUILDDIR)/qthelp, like this:" @echo "# qcollectiongenerator $(BUILDDIR)/qthelp/avocado.qhcp" @echo "To view the help file:" @echo "# assistant -collectionFile $(BUILDDIR)/qthelp/avocado.qhc" devhelp: $(SPHINXBUILD) -b devhelp $(ALLSPHINXOPTS) $(BUILDDIR)/devhelp @echo @echo "Build finished." @echo "To view the help file:" @echo "# mkdir -p $$HOME/.local/share/devhelp/books" @echo "# cp -r $(BUILDDIR)/devhelp $$HOME/.local/share/devhelp/books/Avocado" @echo "# devhelp" epub: $(SPHINXBUILD) -b epub $(ALLSPHINXOPTS) $(BUILDDIR)/epub @echo @echo "Build finished. The epub file is in $(BUILDDIR)/epub." latex: $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex @echo @echo "Build finished; the LaTeX files are in $(BUILDDIR)/latex." @echo "Run \`make' in that directory to run these through (pdf)latex" \ "(use \`make latexpdf' here to do that automatically)." latexpdf: $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex @echo "Running LaTeX files through pdflatex..." $(MAKE) -C $(BUILDDIR)/latex all-pdf @echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex." text: $(SPHINXBUILD) -b text $(ALLSPHINXOPTS) $(BUILDDIR)/text @echo @echo "Build finished. The text files are in $(BUILDDIR)/text." man: $(SPHINXBUILD) -b man $(ALLSPHINXOPTS) $(BUILDDIR)/man @echo @echo "Build finished. The manual pages are in $(BUILDDIR)/man." texinfo: $(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo @echo @echo "Build finished. The Texinfo files are in $(BUILDDIR)/texinfo." @echo "Run \`make' in that directory to run these through makeinfo" \ "(use \`make info' here to do that automatically)." info: $(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo @echo "Running Texinfo files through makeinfo..." make -C $(BUILDDIR)/texinfo info @echo "makeinfo finished; the Info files are in $(BUILDDIR)/texinfo." gettext: $(SPHINXBUILD) -b gettext $(I18NSPHINXOPTS) $(BUILDDIR)/locale @echo @echo "Build finished. The message catalogs are in $(BUILDDIR)/locale." changes: $(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) $(BUILDDIR)/changes @echo @echo "The overview file is in $(BUILDDIR)/changes." linkcheck: $(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck @echo @echo "Link check complete; look for any errors in the above output " \ "or in $(BUILDDIR)/linkcheck/output.txt." doctest: $(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) $(BUILDDIR)/doctest @echo "Testing of doctests in the sources finished, look at the " \ "results in $(BUILDDIR)/doctest/output.txt." avocado-113.0/docs/README.rst000066400000000000000000000011041513561306000155450ustar00rootroot00000000000000Avocado Documentation ===================== If you want to build the documentation yourself: 1) Make sure you have the package ``python-sphinx`` installed. For Fedora:: $ sudo yum install python-sphinx 2) For Mint/Ubuntu/Debian:: $ sudo apt-get install python-sphinx 3) Optionally, you can install the read the docs theme, that will make your in-tree documentation look just like the online version:: $ sudo pip install sphinx_rtd_theme 4) Build the docs:: $ make html 5) Once done, point your browser to:: $ [your-browser] build/html/index.html avocado-113.0/docs/source/000077500000000000000000000000001513561306000153625ustar00rootroot00000000000000avocado-113.0/docs/source/_static/000077500000000000000000000000001513561306000170105ustar00rootroot00000000000000avocado-113.0/docs/source/_static/.gitignore000066400000000000000000000000001513561306000207660ustar00rootroot00000000000000avocado-113.0/docs/source/_templates/000077500000000000000000000000001513561306000175175ustar00rootroot00000000000000avocado-113.0/docs/source/_templates/.gitignore000066400000000000000000000000001513561306000214750ustar00rootroot00000000000000avocado-113.0/docs/source/api/000077500000000000000000000000001513561306000161335ustar00rootroot00000000000000avocado-113.0/docs/source/api/headers/000077500000000000000000000000001513561306000175465ustar00rootroot00000000000000avocado-113.0/docs/source/api/headers/core000066400000000000000000000004231513561306000204200ustar00rootroot00000000000000Internal APIs that may be of interest to Avocado hackers. Everything under :mod:`avocado.core` is part of the application's infrastructure and should not be used by tests. Extensions and Plugins can use the core libraries, but API stability is not guaranteed at any level. avocado-113.0/docs/source/api/headers/optional000066400000000000000000000001131513561306000213110ustar00rootroot00000000000000The following pages document the private APIs of optional Avocado plugins. avocado-113.0/docs/source/api/headers/plugins000066400000000000000000000000721513561306000211510ustar00rootroot00000000000000Extension APIs that may be of interest to plugin writers. avocado-113.0/docs/source/api/headers/test000066400000000000000000000013241513561306000204500ustar00rootroot00000000000000At the most basic level, there's the Test APIs which you should use when writing tests in Python and planning to make use of any other utility library. The Test APIs can be found in the :mod:`avocado` main module and its most important member is the :class:`avocado.Test` class. By conforming to the :class:`avocado.Test` API, that is, by inheriting from it, you can use the full set of utility libraries. The Test APIs are guaranteed to be stable across a single major version of Avocado. That means that a test written for a given version of Avocado should not break on later minor versions because of Test API changes. This is the bare minimum set of APIs that users should use, and can rely on, while writing tests. avocado-113.0/docs/source/api/headers/utils000066400000000000000000000016561513561306000206410ustar00rootroot00000000000000Avocado gives to you more than 40 python utility libraries (so far), that can be found under the :mod:`avocado.utils`. You can use these libraries to avoid having to write necessary routines for your tests. These are very general in nature and can help you speed up your test development. The utility libraries may receive incompatible changes across minor versions, but these will be done in a staged fashion. If a given change to an utility library can cause test breakage, it will first be documented and/or deprecated, and only on the next subsequent minor version, it will actually be changed. What this means is that upon updating to later minor versions of Avocado, you should look at the Avocado Release Notes for changes that may impact your tests. This is a set of utility APIs that Avocado provides as added value to test writers. It's suppose to be generic, without any knowledge of Avocado and reusable in different projects. avocado-113.0/docs/source/blueprints/000077500000000000000000000000001513561306000175515ustar00rootroot00000000000000avocado-113.0/docs/source/blueprints/BP000.rst000066400000000000000000000331011513561306000210220ustar00rootroot00000000000000BP000 ##### :Number: BP000 :Title: Blueprint specification and requirements :Author: Beraldo Leal :Reviewers: Cleber Rosa , Jan Richter , Plamen Dimitrov , Willian Rampazzo :Created: 29-Sep-2020 :Type: Meta Blueprint :Status: Approved .. contents:: Table of Contents TL;DR ***** Having better records of architectural decisions in our repository is a good way to socialize our technical decisions, improve the code review process, avoid unnecessary code, and balance the workload among other developers. We are already using the “blueprint” documents for a while and still learning how to write them. Although we have a basic idea of what is a blueprint, some may not understand the concept because we are missing one meta-document that describes the blueprint's basic notions. This document is a blueprint to specify how we should write blueprints from now on. Motivation ********** Depending on the project size, having very well defined and structured documents about the architecture decisions seems like an overkill, but could help projects of any size, including Avocado, save time, make better decisions and improve the way we socialize those decisions. Today in the Avocado project we have the good practice to submit an RFC to our mailing list or to use/open a GitHub issue when we have a new idea. RFCs are a widespread way to disclose the architecture decisions, but they are just one part of a longer process. During the RFC phase, we argue in favor of a proposal, and we are mostly concerned about collecting feedback. After this phase, we could go one step forward and consolidate the discussion in Blueprints (sometimes called ADRs - Architecture Decision Records). This could be the next step so we could better socialize our decisions for future readers. A very well defined and structured document has some advantages over an RFC, but it is not intended to replace it, just be a later stage to follow from it. With blueprints, we could not only, but mainly: * Create better documents for future members and future reference, when we are trying to answer the following questions: a) *why* are you doing this? (the “problem” or the “motivation”); b) *what* are you proposing to solve the problem? (your "solution")? c) And *how* are you going to implement the proposed solution? Depending on the type of your blueprint, the answer for the last question (c) could be written in pseudocode, general text or might even not be necessary (although desired) — more details on the section named Specification. When using RFCs as email threads, there are no sections or headers, each contributor will try to send the RFC without following formal sections and headers. RFCs, as we use them today are just thread discussions and are not focused on future review/reading. * Make sure that another peer will be able to implement a feature/requirement. Blueprints are not for you; they are for the community. If you know that a problem exists and know how to fix it, the most natural course of action would be to start coding this fix and submitting a Pull Request. While this is still valid for most of the cases, some important architectural changes must be discussed first to explain the “why”, “what” and “how” to keep everyone on the same page, avoid unnecessary coding, and most importantly: allow others to implement it in case you are not available. * Improve the code review quality and time. Having a better understanding of the problem and the big picture is better for code review. It is harder to capture that from the Pull Request (PR) description and PR changes. Developers that are aware of the problem tend to review your changes with the problem in mind and hence more quickly. * Reduce onboarding time for new members. Having a history of how we made an architectural decision and why we are implementing it this way will give new members reading material to understand our project, avoiding unnecessary discussions and meetings to explain something. * Create a common standard that will make it easier for readers. With an open RFC, authors tend to organize the ideas in different ways with different sections. Having a very well structured document with common sections will make it easier for readers to understand the problem and the solution. * Track the progress of a significant implementation. We could use the blueprints header “status” line to track the progress of some features. We could even have a page parsing and listing all the blueprints with the title, author, status, and target milestone for that feature. * Find the middle ground between “overthinking” and “auto-pilot.” Last but not least: we are *not trying* to overthink here and/or slow down our development processes. The idea is to have a lightweight document, very objective that will save us time in the medium and long run. We don’t have to overthink by trying to handle any possible scenario outside of ones we actually have a use case for. But we should also avoid the “auto-pilot” mode in our contributions where we are fixing issues as quick as possible without thinking about the big picture, it is not healthy for the project. Specification ************* One blueprint per topic ----------------------- Try to follow the minimalist approach and be concise with content relevant to one particular topic. If you have a more general topic to discuss, you should set the type as “Epic Blueprint” (more below) but still try to be concise and focused on the subject. File format and location ------------------------ Our current documentation already uses ReStructuredText (.rst) format, so we will adopt .rst format here too. All blueprints will be located inside `docs/source/blueprints` with the filename `BPXXX.rst`, where XXX is the number of the blueprint. Just pick the next number available for your blueprint. It's recommended that you use ``docs/source/blueprints/template.rst`` as a starting point. Write for your audience ----------------------- As mentioned before, your blueprint will be read by your peers, future members, and future yourself. Keep in mind that your audience is developers with a minimal understanding of the Avocado internals and be kind providing any necessary context to understand the problem. Blueprints types ---------------- Currently, we have the following blueprint types: * Architectural Blueprint: Any blueprint changing or introducing a new core feature or architectural change to Avocado. * Process Blueprint: Any blueprint that is not implementing a new core feature, but changing how the project works. This could be, for instance, related to the repositories or processes. * Meta Blueprint: A blueprint about blueprints. Like this one and any future blueprint that changes our blueprint's styles and methods. * Epic Blueprint: A blueprint that is touching on multiple areas and is too big to have all the documentation in one single blueprint. We could split epic blueprints into smaller blueprints or issues (if they are small and easy to understand). Epic Blueprints are not a merge of all sub-blueprints. Like an epic issue, epic blueprints don't need to detail "how" (or provide details) that the sub-blueprints could have. * Component Blueprint: A blueprint with the intent to describe a new utility module or a new plugin. Headers ------- Python PEPs (Python Enhancement Proposals) uses RFC822 for describing the headers. This could be useful here too, especially when parsing those headers to display our list of blueprints with the current status. The current list of items of our blueprint headers is below: * Number: Usually, the blueprint number in the format BPXXX * Title: A short descriptive title, limited to 80 characters * Author: The author or authors of blueprint. Following the format: `[FIRST NAME] [LAST NAME] - ` * Reviewers: All reviewers that approved and helped during the review process * Created: Date string when the blueprint first draft was submitted. Please use the following format: DD-MMM-YYYY. * Type: One of the types described during the previous section * Status: One of the types described during the next section Here is an example of a header:: :Number: BP001 :Title: Configuration by convention :Author: Beraldo Leal :Reviewers: Cleber Rosa, LukÃĄÅĄ Doktor and Plamen Dimitrov :Created: 06-Dec-2019 :Type: Epic Blueprint :Status: WIP Blueprint statuses ------------------ * Draft: All blueprints should be created in this state. This means the blueprint is accepting comments, and probably there is a discussion happening. Blueprints in draft mode can be part of our repository. * Approved: Blueprint was approved after discussions, and all suggestions are already incorporated on the document. Nobody has started working on this yet. * Assigned: This status is not about the blueprint itself, but about the proposal that is the subject of the BP. This means that the blueprint was approved, and someone is already working on implementing it. A BP status can change from Draft to Assigned if the work has started already. * WIP: Blueprint was approved and someone is working on it. Work in Progress. * Implemented: This means the BP is already implemented and delivered to the Avocado’s master branch. * Rejected: Rejected status means the idea was not implemented because it wasn't approved by everyone or has some technical limitations. * Deprecated: Deprecated means it was approved, implemented, and at some point, makes no more sense to have it. For example, anything related to the legacy runner. Usually, Deprecated means that it was replaced by something else. As you can see, there is no status to accommodate any future change in a blueprint. Blueprints should not be “voided.” Any improvement on an old blueprint should be presented as a new blueprint, changing the status of the original to “deprecated”. Sections -------- In order to facilitate the reading and understanding of the problem, all blueprints must have the following sections: * TL;DR * Motivation * Specification * Backwards Compatibility * Security Implications * How to Teach This * Related Issues * References Below you can find a brief description of what you should write in each section: * TL;DR: Should be a short description of your blueprint. Like an abstract. We recommend writing this at the end of your first draft. This will give you a better overview of it. * Motivation: This should be the motivation of your proposed solution, not the motivation of the blueprint itself. It describes the problem. Here, you should answer "why" your solution is needed. * Specification: In this section, you should describe how you are going to solve the problem. You can create subsections here to organize your ideas better. Please keep in mind that it is useful to mention the details, with code snippets, examples, and/or references. This will save you time, making sure that everyone is in agreement with the proposed solution. * Backwards Compatibility: How is your proposal going to affect older versions of Avocado? Should we deprecate some modules, classes, or methods? Are we going to keep backwards compatibility or not? * Security Implications: Do you have any concerns about security with your proposed solution and what are they? If there's functionality that is insecure but highly convenient, consider how to make it "opt-in", disabled by default. * How to Teach This: What is the best way to inform our devs and users about your new feature/solution? Consider both "how-to" and reference style documentation, and if appropriate, examples (under ``examples/``) using the feature. * Related Issues: Here, you should mention Github links for both: a) current open issues that are blocking while waiting for your BP and b) all open issues that will render this BP as “implemented” when closed. 1. Issues to address this BP Would be nice, if possible, to open issues on GH that covers all aspects of your Blueprint. 2. Issues this BP will solve What are the issues already existent on Avocado project that your proposal will solve? * References: Any external reference for helping understand the problem and your solution. Backwards Compatibility *********************** So far, we are on our 3rth blueprint (BP003 was the last one). This BP000 should have been released before those blueprints. So probably those three blueprints are not 100% compliaant with this meta blueprint, and that is fine. We were learning on the fly. We don’t need to change any of those blueprints after BP000 gets approved. Security Implications ********************* No security implications found so far. How to Teach This ***************** Getting used to writing blueprints is not an easy task. And probably we are going to find unplanned issues with this process on the way. The general rule of thumb is to use common sense. To make this more public, we could consider the following: * If approved, BP000 should be on top of our blueprints lists for reference. * We could also have a template inside the `blueprints` directory to help people when submitting their own blueprints. * Also, we could include pointers and instructions in our development guide for this BP. * Another good practice would be to make comments in Avocado’s source code with some pointers to specific blueprints. Related Issues ************** None. References ********** None. avocado-113.0/docs/source/blueprints/BP001.rst000066400000000000000000000401631513561306000210310ustar00rootroot00000000000000BP001 ##### :Number: BP001 :Title: Configuration by convention :Author: Beraldo Leal :Discussions-To: avocado-devel@redhat.com :Reviewers: Cleber Rosa, LukÃĄÅĄ Doktor and Plamen Dimitrov :Created: 06-Dec-2019 :Type: Epic Blueprint :Status: Approved .. contents:: Table of Contents TL;DR ***** The number of plugins made by many people and the lack of some name, config options, and argument type conventions may turn Avocado's usability difficult. This also makes it challenging to create a future API for executing more complex jobs. Even without plugins the lack of convention (or another type or order setting mechanism) can induce growth pains. After an initial discussion on avocado-devel, we came up with this "blueprint" to change some config file settings and argparse options in Avocado. This document has the intention to list the requirements before coding. And note that, since this is a relatively big change, this blueprint will be broken into small cards/issues. At the end of this document you can find a list of all issues that we should solve in order to solve this big epic Blueprint. Motivation ********** An Avocado Job is primarily executed through the `avocado run` command line. The behavior of such an Avocado Job is determined by parsing the following settings (listed in parsed order): 1) Default values in source code 2) Configuration file contents 3) Command-line options Currently, the Avocado config file is an .ini file that is parsed by Python's `configparser` library and this config is broken into sections. Each Avocado plugin has its dedicated section. Today, the parsing of the command line options is made by `argparse` library and produces a dictionary that is given to the `avocado.core.job.Job()` class as its `config` parameter. There is a lack of convention/order in the item 1. For instance, we have "avocado/core/defaults.py" with some defaults, but there are other such defaults scattered around the project, with ad-hoc names. There is also no convention on the naming pattern used either on configuration files or on command-line options. Besides the name convention, there is also a lack of convention for some argument types. For instance:: $ avocado run -d and:: $ avocado run --sysinfo on Both are boolean variables, but with different "execution model" (the former doesn't need arguments and the latter needs `on` or `off` as argument). Since the Avocado trend is to have more and more plugins, we need to design a name convention on command-line arguments and settings to avoid chaos. But, most important: It would be valuable for our users if Avocado provides a Python API in such a way that developers could write more complex jobs programmatically and advanced users that know the configuration entries used on jobs, could do a quick one-off execution on command-line. Example:: import sys from avocado.core.job import Job config = {'references': ['tests/passtest.py:PassTest.test']} with Job(config) as j: sys.exit(j.run()) Before we address this API use-case, it is important to create this convention so we can have an intuitive use of Avocado config options. We understand that, plugin developers have the flexibility to configure they options as desired but inside Avocado core and plugin, settings should have a good naming convention. Specification ************* Basics on Defaults ================== The Oxford dictionary lists the following as one of the meanings of the word "default" (noum): *"a preselected option adopted by a computer program or other mechanism when no alternative is specified by the user or programmer."* The basic behavior on defaults values vs config files vs command line arguments should be: 1. Avocado has all default values inside the source code; 2. Avocado parses the config files and override the defined values; 3. Avocado parses the command-line options and override the defined values; If the config files or configuration options are missing, Avocado should still be able to use the default values. Users can only change 2 and 3. .. note:: New Issue: Converte all "currently configured settings" into a default value. Mapping between configuration options ===================================== Currently, Avocado has the following options to configure it: 1. Default values; 2. Configuration files; 3. Command-line options; Soon, we will have a fourth option: 4. Job API config argument; Although we should keep an eye on item 4 while implementing this blueprint, it is not intended to address the API at this time. The default values (within the source code) should have an 1:1 mapping to the configuration file options. Must follow the same naming convention and sections. Example:: #avocado.conf: [core] foo = bar [core.sysinfo] foo = bar [pluginx] foo = bar Should generate a dictionary or object in memory with a 1:1 mapping, respecting chained sections:: {'core': {'foo': 'bar', 'sysinfo': {'foo': 'bar'}}, 'pluginx': {'foo': 'bar'}} Again, if the config file is missing or some option is missing the result should be the same, but with the default values. Since the command-line options are only the most used and basic ones, there is no need to have a 1:1 mapping between item 2 and item 3. When naming subcommands options you don’t have to worry about name conflicts outside the subcommand scope, just keep them short, simple and intuitive. When naming a command-line option on the core functionality we should remove the "core" word section and replace "_" by "-". For instance:: [core] execution_timeout = 30 Should be:: avocado --execution-timeout 30 When naming plugin options, we should try to use the following standard:: [pluginx] foo = bar Becomes:: avocado --pluginx-foo bar This only makes sense if the plugins' names are short. .. warning:: Maybe I have to get more used with all the Avocado options to understand better. Or someone could help here. Standards for Command Line Interface ==================================== When it comes to the command line interface, a very interesting recommendation is the POSIX Standard's recommendation for arguments[1]. Avocado should try to follow this standard and its recommendations. This pattern does not cover long options (starting with --). For this, we should also embrace the GNU extension[2]. One of the goals of this extension, by introducing long options, was to make command-line utilities user-friendly. Also, another aim was to try to create a norm among different command-line utilities. Thus, --verbose, --debug, --version (with other options) would have the same behavior in many programs. Avocado should try to, where applicable, use the GNU long options table[3] as reference. .. note:: New Issue: Review the command line options to see if we can use the GNU long options table. Many of these recommendations are obvious and already used by Avocado or enforced by default, thanks to libraries like `argparse`. However, those libraries do not force the developer to follow all recommendations. Besides the basic ones, there is a particular case to pay attention: "option-arguments". Option-arguments should not be optional (Guideline 7, from POSIX). So we should avoid this:: avocado run --loaders [LOADERS [LOADERS ...]] or:: avocado run --store-logging-stream [STREAM[:LEVEL] [STREAM[:LEVEL] ...]] As discussed we should try to have this:: avocado run --loaders LOADERS [LOADERS ...] .. note:: New Issue: Make the option-arguments not optional. Argument Types -------------- Basic types, like strings and integers, are clear how to use. But here is a list of what should expect when using other types: 1. **Booleans**: Boolean options should be expressed as "flags" args (without the "option-argument"). Flags, when present, should represent a True/Active value. This will reduce the command line size. We should avoid using this:: avocado run --json-job-result {on,off} So, if the default it is enabled, we should have only one option on the command-line:: avocado run --disable-json-job-result This is just an example, the name and syntax may be different. .. note:: New Issue: Fix boolean command line options 2. **Lists**: When an option argument has multiple values we should use the space as the separator. .. note:: New Issue: Review if we have any command line list using non space as separator. Presentation ------------ Finding options easily, either in the manual or in the help, favor usability and avoids chaos. We can arrange the display of these options in alphabetical order within each section. Standards for Config File Interface =================================== Many other config file options could be used here, but since that this is another discussion, we are assuming that we are going to keep using `configparser` for a while. As one of the main motivations of this Blueprint is to create a convention to avoid chaos and make the job execution API use as straightforward as possible, We believe that the config file should be as close as possible to the dictionary that will be passed to this API. For this reason, this may be the most critical point of this blueprint. We should create a pattern that is intuitive for the developer to convert from one format to another without much juggling. Nested Sections --------------- While the current `configparser` library does not support nested sections, Avocado can use the dot character as a convention for that. i.e: `[runner.output]`. This convention will be important soon, when converting a dictionary into a config file and vice-versa. And since almost everything in Avocado is a plugin, each plugin section should **not** use the "plugins" prefix and **must** respect the reserved sections mentioned before. Currently, we have a mix of sections that start with "plugins" and sections that don't. .. note:: New Issue: Remove "plugins" from the configuration section names. Plugin section name ------------------- Most plugins currently have the same name as the python module. Example: human, diff, tap, nrun, run, journal, replay, sysinfo, etc. These are examples of "good" names. However, some other plugins do not follow this convention. Ex: runnable_run, runnable_run_recipe, task_run, task_run_recipe, archive, etc. We believe that having a convention here helps when writing more complex tests, configfiles, as well as easily finding plugins in various parts of the project, either on a manual page or during the installation procedure. We understand that the name of the plugin is different from the module name in python, but in any case we should try to follow the PEP8: From PEP8: *Modules should have short, all-lowercase names. Underscores can be used in the module name if it improves readability. Python packages should also have short, all-lowercase names, although the use of underscores is discouraged.* Let's get the `human` example: * Python module name: human * Plugin name: human Let's get the `task_run_recipe` example: * Python module name: task_run_recipe * Plugin name: task-run-recipe Let's get another example: * Python module name: archive * Plugin name: zip_archive One suggestion should be to have a namespace like `resolvers.tests.exec`, `resolvers.tests.unit.python`. And all the duplicated code could be imported from a common module inside the plugin. But yes, it is a "delicate issue". .. note:: New Issue: Rename the plugins modules and names. This might be tricky. Reserved Sections ----------------- We should have one reserved section, the `core` section for the Avocado's core functionalities. All plugin code that it is considered "core" should be inside core as a "nested section". Example:: [core] foo = bar [core.sysinfo] collect_enabled = True .. note:: New Issue: Move all 'core' related settings to the core section. Config Types ------------ `configparser` do not guess datatypes of values in configuration files, always storing them internally as strings. This means that if you need other datatypes, you should convert on your own There are few methods on this library to help us: `getboolean()`, `getint()` and `getfloat()`. Basic types here, are also straightforward. Regarding boolean values, `getboolean()` can accept `yes/no`, `on/off`, `true/false` or `1/0`. But we should adopt one style and stick with it. .. note:: New Issue: Create a simple but effective type system for configuration files and argument options. Presentation ============ As the avocado trend is to have more and more plugins, We believe that to make it easier for the user to find where each configuration is, we should split the file into smaller files, leaving one file for each plugin. Avocado already supports that with the conf.d directory. What do you think? .. note:: New Issue: Split config files into small ones (if necessary). Backwards Compatibility *********************** In order to keep a good naming convention, this set of changes probably will rename some args and/or config file options. While some changes proposed here are simple and do not affect Avocado's behavior, others are critical and may break Avocado jobs. Command line syntax changes =========================== These command-line conversions will lead to a "syntax error". We should have a transition period with a "deprecated message". Plugin name changes =================== Changing the modules names and/or the 'name' attribute of plugins will require to change the config files inside Avocado as well. This will not break unless the user is using an old config file. In that case, we should also have a "deprecated message" and accept the old config file option for some time. Security Implications ********************* Avocado users should have the warranty that their jobs are running on isolated environment. We should consider this and keep in mind that any moves here should continue with this assumption. How to Teach This ***************** We should provide a complete configuration reference guide section in our User's Documentation. .. note:: New Issue: Create a complete configuration reference. In the future, the Job API should also be very well detailed so sphinx could generate good documentation on our Test Writer's Guide. Besides a good documentation, there is no better way to learn than by example. If our plugins, options and settings follow a good convention it will serve as template to new plugins. If these changes are accepted by the community and implemented, this RFC could be adapted to become a section on one of our guides, maybe something like the a Python PEP that should be followed when developing new plugins. .. note:: New Issue: Create a new section in our Contributor's Guide describing all the conventions on this blueprint. Related Issues ************** Here a list of all issues related to this blueprint: #. Create a new section in our Contributor's Guide describing all the conventions on this blueprint. #. Create a complete configuration reference. #. Split config files into small ones (if necessary). #. Create a simple but effective type system for configuration files and argument options. #. Move all 'core' related settings to the core section. #. Rename the plugins modules and names. This might be tricky. #. Remove "plugins" from the configuration section names. #. Review if we have any command line list using non space as separator. #. Fix boolean command line options. #. Make the option-arguments not optional. #. Review the command line options to see if we can use the GNU long options table. #. Converte all "currently configured settings" into a default value. .. warning:: After this blueprint get approved, I will open all issues on GH, add links here and remove all the notes. References ********** [1] - https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap12.html [2] - https://www.gnu.org/prep/standards/html_node/Command_002dLine-Interfaces.html [3] - https://www.gnu.org/prep/standards/html_node/Option-Table.html#Option-Table avocado-113.0/docs/source/blueprints/BP002.rst000066400000000000000000000276171513561306000210430ustar00rootroot00000000000000BP002 ##### :Number: BP002 :Title: Requirements resolver :Author: Willian Rampazzo :Discussions-To: https://github.com/avocado-framework/avocado/issues/3455 :Reviewers: Beraldo Leal, Cleber Rosa :Created: 27-Jan-2020 :Type: Architecture Blueprint :Status: WIP .. contents:: Table of Contents TL;DR ***** The current management of test assets is handled manually by the test developer. It is usual to have a set of repetitive code blocks to define the name, location, and other attributes of an asset, download it or signal an error condition if a problem occurred and the download failed. Based on use cases compiled from the discussion on qemu-devel mailing-list [1] and discussions during Avocado meetings, this blueprint describes the architecture of a requirements resolver aiming the extensibility and flexibility when handling different types of assets, like a file, a cloud image, a package, a Git repository, source codes or Operating System parameters. Motivation ********** Implementing a test that gathers its requirements while executing may lead to a wrong interpretation of the test results if a requirement is not satisfied. The failure of a test because of a missing requirement does not mean the test itself failed. During its execution, the test has never reached the core test code; still, it may be considered a failing test. Fulfilling all the test requirements beforehand can be an efficient way to handle requirements problems and can improve the trustworthiness of the test result. It means that if a test ran and failed, the code responsible for the failure is related to the core test and not with one of its requirements. Regardless of how the test defines a requirement, an architecture capable of identifying them is beneficial. Storing its references and delegating to the code responsible for handling each different type of requirement makes the overall architecture of Avocado and the requirement definition of a test more flexible. A requirements resolver can bring the necessary flexibility to the Avocado architecture, as well as managing support for different types of requirements. This blueprint discusses the architecture of a requirements resolver responsible for handling the different requirements types. Specification ************* Basics ====== The strict meaning of a resolver is related to something responsible for creating resolutions from a given representation. When there is a well-defined way to declare something, a resolver can translate this representation to another well-defined representation. The classic example is a Domain Name Server (DNS), which resolves the hostname into an Internet Protocol (IP) address. The use of the word `resolver` in this text means a code responsible for gathering and fulfilling well-know representations with little or no transformation. The definition of requirements resolver in this blueprint is a code responsible for gathering well-known formats of requirements, possibly from different sources, and centralizing in one place, or fulfilling them. The requirements fulfillment can take place starting from the centralized collection of requirements as input to one of several modules responsible for handling each specific type of requirement, like, for example, files, images, packages, git repositories, source code or operating system parameters. The following diagram shows the underlying architecture of a requirements resolver proposed in this blueprint. The next sessions describes, in detail, each part of the resolver, its inputs, and outputs. .. image:: images/requirements-resolver.png :width: 800 :alt: Requirements resolver diagram Requirements representations ============================ Define how to represent a requirement is the first step to define the architecture of a resolver. This blueprint defines the following ways to represent a requirement: 1. JavaScript Object Notation (JSON) file; 2. Python executable that produces a JSON file; 3. Metadata included in the test docstring. Requirements representation as JSON files ----------------------------------------- JSON is a lightweight data-interchange format [2] supported by the Python standard library. Using it to represent requirements is flexible and straightforward. The standard proposed way to represent requirements with JSON is defining one requirement per entry. Each entry should start with the requirement type, followed by other keyword arguments related to that type. Example:: [ {"type": "file", "uri": "https://cabort.com/cabort.c", "hash": "deadbeefdeadbeef"}, {"type": "vmimage", "distro": "fedora", "version": 31, "arch": "x86_64"}, {"type": "package", "package": "lvm"} ] The requirement `type` should match the module responsible for that type of requirement. Requirements representation as Python executable ------------------------------------------------ Another way to create the requirements representation as JSON files is by writing a Python executable. This approach makes the requirements representation flexible, by allowing the use of Python variables and code that may change the parameters values for the requirements, depending on the environment the Python code runs. The following example shows a requirement that depends on the architecture the test is running:: #!/usr/bin/python3 import os import json requirements = [ {"type": "file", "uri": "https://cabort.com/cabort.c", "hash": "deadbeefdeadbeef"}, {"type": "vmimage", "distro": "fedora", "version": 31, "arch": os.uname()[4]}, {"type": "package", "package": "lvm"} ] print(json.dumps(requirements)) Requirements representation as Metadata on test docstring --------------------------------------------------------- Test writers may want to add the requirements of a test into de test code. The option proposed here allows the use of metadata on test docstrings to represent the requirements list. Below is an example of how to define requirements as metadata on docstrings:: def test_something(self): ''' :avocado: requirement={"type": "file", "uri": "https://cabort.com/cabort.c", "hash": "deadbeefdeadbeef"} :avocado: requirement={"type": "vmimage", "distro": "fedora", "version": 31, "arch": "x86_64"} :avocado requirement={"type": "package", "package": "lvm"} ''' Requirements files location =========================== It may be useful for test writers to define a standard source location for the requirements JSON files and the requirements Python executable. This blueprint defines the default location for a job-wide requirements file in the same directory of the test files or test-specific requirements files into a requirements directory preceded by the test file name. It is also possible to use sub-directories with the name of a specific test to define requirements for that test. The following file tree is an example of possible use for requirements directories for a test:: requirements.json cabort.py cabort.py.requirements/ ├── CAbort.test_2 │   └── requirements.py └── requirements.json In this case, all the tests on `cabort.py`, except for `CAbort.test_2`, use the `requirements.json` file located at `cabort.py.requirements`. The `CAbort.test_2` test uses its own `requirements.py` located at `CAbort.test_2` directory inside the requirements directory. The tests located at the same directory of `cabort.py` use the `requirements.json` in the root directory. Requirements files command-line parameter ========================================= It is also possible to use a command-line parameter to define the location of the requirements file. The command-line parameter supersedes all the other possible uses of requirements files. For that, this blueprint defines the parameter `--requirements-file` followed by the location of the requirements file. As a command-line example, we have:: avocado --requirements-file requirements.json run passtest.py .. note:: New Issue: Add the support for `--requirements-file` command-line parameter. Backward Compatibility ********************** The implementation of the requirements resolver, proposed here, affects Avocado's behavior related to the tasks executed before a test execution starts. To make the requirements resolver as flexible as possible, the implementation of this blueprint may change the utility APIs related to a requirement type. Security Implications ********************* Avocado users should have the warranty that their jobs are running in an isolated environment, but Avocado can, conservatively, create mechanisms to protect the users from running unintended code. The use of a Python executable to build the requirements file is subject to security considerations. A malicious code distributed as a Python executable to build the requirements file can lead to security implications. This blueprint proposes a security flag in a general Avocado configuration file to avoid Python executable code to run by default. Users can change this flag anytime to be able to use the ability to run Python executable codes to generate the requirements JSON file. Following is an example of how this flag can look like:: [resolver.requirements] # Whether to run Python executables to build the requirements file unsafe = False .. note:: New Issue: Add the unsafe flag support for the requirements resolver. How to Teach This ***************** We should provide a complete and detailed explanation of how to handle test requirements in the User's Documentation. .. note:: New Issue: Create a complete section in the User's Guide on how to handle test requirements. Also, we should address how to create utility modules to handle new types of requirements in the Contributor's Guide. .. note:: New Issue: Create a new section in the Contributor's Guide on how to develop modules to handle new types of requirements. Related Issues ************** Here a list of all issues related to this blueprint: #. `[DONE] `__ Implement the initial architecture for a requirements resolver based on BP002 and add support for packages #. `[DONE] `__ Create a complete section in the User's Guide on how to handle test requirements. #. `[DONE] `__ BP002: implement the file runner to support file requirements #. `[OPEN] `__ BP002: implement the image runner to support image requirements #. `[OPEN] `__ BP002: implement the git repository runner to support Git repository requirements #. `[OPEN] `__ BP002: add support to JSON representation for requirements #. `[OPEN] `__ BP002: add support to Python executable that produces JSON representation for requirements #. `[OPEN] `__ BP002: add command-line parameter to support requirements file #. `[OPEN] `__ BP002: add support to auto-detect requirements files based on file location #. `[OPEN] `__ BP002 add support to run unsafe Python code that produces JSON representation for requirements #. `[OPEN] `__ BP002: Create a new section in the Contributor's Guide on how to implement requirements runners References ********** [1] - https://lists.gnu.org/archive/html/qemu-devel/2019-11/msg04074.html [2] - https://docs.python.org/3/library/json.html avocado-113.0/docs/source/blueprints/BP003.rst000066400000000000000000000533131513561306000210340ustar00rootroot00000000000000BP003 ##### :Number: BP003 :Title: N(ext)Runner Task Life-Cycle :Author: Cleber Rosa :Discussions-To: avocado-devel@redhat.com :Reviewers: Beraldo Leal , Willian Rampazzo :Created: 20-July-2020 :Type: Architecture Blueprint :Status: WIP .. contents:: Table of Contents TL;DR ***** The N(ext) Runner has been used as Avocado's runner for selftests for over a year. The implementation used is based on the ``avocado nrun`` command, that is, outside of the Avocado's traditional ``avocado run`` entrypoint. Under the hood, it means that the N(ext) Runner is not integrated well enough with an Avocado Job. A partial implementation of the N(ext) Runner integration with an Avocado Job is available at ``avocado/plugins/runner_nrunner.py`` but it has a number limitations. The N(ext) Runner executes tests as :class:`Tasks `. This blueprint describes the phases that a Task can be in throughout its life-cycle, and how the handling of these phases or states, will power the tests execution mechanism within the context of an Avocado Job. Motivations *********** Propose an architecture for integrating the N(ext) Runner concepts and features into an Avocado Job. Because the N(ext) Runner contains distinguishing features that the original Avocado Job did not anticipate, a proxy layer is necessary. The current runner (and Job) is built on the premises that there's a "currently executing test", and thus, does not need to keep track of various running tests states at once. The N(ext) Runner, on the other hand, support for running tests in parallel, and thus needs supporting code for keeping track of their state and forward their relevant information to an Avocado Job. Goals of this BluePrint *********************** 1. Propose an architecture based on the life-cycle phases that an N(ext) Runner Task can go through while running under an Avocado Job. 2. Describe how the proposed architecture can power an implementation suitable for the next Avocado LTS release (82.0), having feature completeness when compared to the current runner, while still making its distinguishing features available to users who opt in. This also means that the current Avocado Job interface will continue to support the current runner implementation. 3. Prove that the current runner can be removed without significant user impact after the LTS release (within the 83.0 development cycle), based on the feature completeness of the N(ext) Runner with regards to its integration with an Avocado Job. 4. Allow for future extension of the Task life-cycle phases architecture, such as into a more capable and further reaching scheduler for Tasks. This means that this BluePrint is focused on short term integration issues, as describe in the motivation, but at the same tries to not impose future limitations to have new features implemented for other use cases. Requirements ************ This section describes the requirements to manage the Task's life-cycle. It also describes the phases of a Task life-cycle and includes an example. Task Execution Requirements Verification ---------------------------------------- For a Task to actually be executed, there needs to be a minimal number of requirements present. For instance, it's pointless to attempt to execute a Task of kind "custom" without either: 1. An ``avocado-runner-custom`` runner script that is compatible with the Avocado interface, OR 2. A ``CustomRunner`` runner class that is compatible with the :class:`avocado.core.nrunner.BaseRunner` interface Other types of Task Execution Requirements checks may be added in the future, but the core concept that a Task can not always be executed remains. Currently, as per the ``avocado nrun`` implementation, this verification is done in a synchronous way, and it's of limited visibility to the user. Requirements: 1. The verification of one Task's requirement should not block other Tasks from progressing to other phases. 2. The user interface should provide more information on tasks that either failed the verification or that still going through the verification process. Parallelization and Result Events --------------------------------- The N(ext) Runner allows for the parallel execution of tasks. When integrated into a Job, it means there can be more than one test running at a given time. Currently, plugins that implement the :class:`avocado.core.plugin_interfaces.ResultEvents` interface may contain logic that assumes that the same test will have ``start_test``, ``test_progress`` and then ``end_test`` methods called in that particular order, and only then another test will have any of those called on its behalf. For instance, the Human UI plugin will currently: 1. Print a line such as ``(1/1) /bin/true:`` when a test starts, that is, when :meth:`avocado.core.plugin_interfaces.ResultEvents.start_test` is called. 2. Add a throbber and/or change its state whenever a progress update is received, that is, when :meth:`avocado.core.plugin_interfaces.ResultEvents.test_progress` is called. 3. Add a test result such as ``PASS (0.01 s)`` when the test finishes, that is, when :meth:`avocado.core.plugin_interfaces.ResultEvents.end_test` is called. Other implementations, such as the TAP result plugin, will only print a line when the final test result is known. Requirement: have no conflicts of test information when more than one is running in parallel. Requirement example: provide the test progress notification and the final test result information "in line" with the correct test indication (if given earlier). .. note:: Ideally, this shouldn't require a change to the interface, but only within the implementation so that the presentation of coherent test result events is achieved. Non-blocking Parallelization ---------------------------- As stated earlier, the N(ext) Runner allows for the parallel execution of tasks. A given Task should be allowed to be executed as early as possible, provided: 1. Its requirements (such as its specific test runners) are available. 2. A limit for concurrently running tasks has not been reached. Requirement: there should be no artificial and unnecessary blocking of the parallelization level. Requirement example: if an hypothetical Result Events plugin interacts with a high latency server, and such interaction takes 2 minutes, the execution of new tasks should not be affected by it. .. note:: There are a number of strategies for concurrent programming in Python these days, and the "avocado nrun" command currently makes use of asyncio to have coroutines that spawn tasks and collect results concurrently (in a preemptive cooperative model). The actual tools/libraries used in the implementation shall be discussed later. Passive Task Status Collection ------------------------------ The N(ext) Runner architecture allows tests to run in a much more decoupled way, because of a number of its characteristics, including the fact that Tasks communicate their status by sending asynchronous messages. .. note:: The current implementation uses network sockets as the transport for these messages, in part for its universal aspect, and in part to enforce this decoupling. Future implementations may provide alternate transports, such as file descriptors, serial connections, etc. There currently is a component used for a similar role used in ``avocado nrun``: :class:`avocado.core.nrunner.StatusServer`, but it exceeds what's needed here in some aspects, and lacks in others aspects. Requirement: have a mechanism that can receive and collect in an organized manner, all the state messages coming from tasks that are part of an Avocado Job. Requirement example: the Avocado Job should be able to use the collection of task status information to ask questions such as the following. 1. When was the last time that task "123-foobar" gave an status update? Such information would be useful to determine if the task should be abandoned or destroyed as part of a timeout handling, as described in the later section about Task Monitoring and Termination. 2. Has the task "123-foobar" given a final status update? That is, can we conclude that, as a Task, regardless of the success or failure of what it ran, it finished its execution? Such information would be useful to post the final test result to the Job results and ``ResultEvent`` plugins, as described in the next section. Proxy from Task Status To Job Result ------------------------------------ An Avocado Job contains an :class:`avocado.core.result.Result` which tallies the overall job results. But, the state messages coming from Tasks are not suitable to being given directly to methods such as :meth:`avocado.core.result.Result.check_state`. A mechanism is needed to proxy and convert the relevant message and events to the current Avocado job result and ``ResultEvents`` plugins. Requirements: 1. Proxy Task Status messages and convert them into the appropriate information suitable for :class:`avocado.core.result.Result`. 2. Allow :class:`ResultEvents ` plugins to act as soon as possible on relevant status messages; Task Monitoring and Termination ------------------------------- The N(ext) Runner architecture, as stated before, can have tasks running without much, if any, contact with an Avocado Job. But, an Avocado Job must have a beginning and end, and with that it's necessary to monitor tasks, and if their situation is not clear, decide their fate. For instance, a Task started as part of an Avocado Job may communicate the following messages:: {'status': 'started', 'time': 1596680574.8790667, 'output_dir': '/tmp/.avocado-task-d8w0k9s1', 'id': '1-/bin/sleep'} {'status': 'running', 'time': 1596680574.889258, 'id': '1-/bin/sleep'} Then it may go offline for eternity. The possible reasons are varied, and despite them, the Job will eventually have to deal the non-final, unknown state of tasks and given them a resolution. .. Note:: The Spanwer may be able to provide additional information that will help to decide the handling given to such as Task (or its recorded final status). For instance, if a Task running on a container is not communicating its status, and its verified that the container has finished its execution, it may be wise to not wait for the timeout. Requirements: 1. Monitor the execution of a task (from an external PoV). 2. Unless it proves to be, say because of complexity or impossibilities when interacting with the spawners, tasks that are unresponsive should attempt to be terminated. 3. Notify the user if stray tasks should be clean up manually. This may be, for instance, necessary if a Task on a container seems to be stuck, and the container can not be destroyed. The same applies to a process in an uninterruptible sleep state. 4. Update Job result with the information about monitored tasks. .. Note:: Tasks going through the usual phases will end up having their final state in the going through the task status collection described earlier, and from there have them proxied/converted into the Job result and plugins. At first sight, it seems that the task monitoring should use the same repository of status and update it in a similar way, but on behalf of the "lost/exterminated task". Suggested Terminology for the Task Phases ***************************************** Task execution has been requested --------------------------------- A Task whose execution was requested by the user. All of the tasks on a Job's ``test_suite`` attribute are requested tasks. If a software component deals with this type of task, it's advisable that it refers to ``TASK_REQUESTED`` or ``requested_tasks`` or a similar name that links to this definition. Task is being triaged --------------------- The details of the task are being analyzed, including, and most importantly, the ability of the system to run it. A task that leaves triage, and it's either considered ``FINISHED`` because it can not be executed, or is ``READY`` and waits to be executed. If a software component deals with this type of task, for instance, if a "task scheduler" is looking for runners matching the Task's kind, it should keep it under a ``tasks_under_triage`` or mark the tasks as ``TASK_UNDER_TRIAGE`` or ``TASK_TRIAGING`` a similar name that links to this definition. Task is ready to be started --------------------------- Task has been triaged, and as much as the system knows, it's ready to be executed. A task may be in this phase for any amount of time, given that the capacity to have an additional task started is dynamic and may be enforced here. If a software component deals with this type of task, it should keep it under a ``tasks_ready`` or mark the tasks as ``TASK_READY`` or a similar name that links to this definition. Task has been started --------------------- A task was successfully started by a spawner. Note that it does *not* mean that the test that the task runner (say, an ``avocado-runner-$kind task-run`` command) will run has already started. This will be signaled by a runner, say ``avocado-runner-$kind`` producing an ``status: started`` kind of status message. If a software component deals with this type of task, it should keep it under a ``tasks_started`` or mark the tasks as ``TASK_STARTED`` or a similar name that links to this definition. Task is finished ---------------- This means that there's no longer any activity or a new phase for this task to move to. It's expected that extra information will be available explaining how/why the task arrived in this phase. For instance, it may have come from the ``TASK_TRIAGING`` phase and never gone through the ``TASK_STARTED`` phase. Alternatively, it may been in the ``TASK_STARTED`` phase and finished without any errors. It should be kept under a ``tasks_finished`` structure or be marked as ``TASK_FINISHED`` or a similar name that links to this definition. .. note:: There's no associated meaning here about the pass/fail output of the test payload executed by the task. Task life-cycle example *********************** A task will usually be created from a Runnable. A Runnable will, in turn, almost always be created as part of the "avocado.core.resolver" module. Let's consider the following output of a resolution: .. image:: images/BP003/Resolution.png :width: 200 :alt: Test resolution Two Runnables here will be transformed into Tasks. The process usually includes adding an identification and a status URI: .. image:: images/BP003/Resolution-to-task.png :width: 400 :alt: Test resolution to task In the end, a job will contain a ``test_suite`` with "Task #1" and "Task #2". It means that the execution of both tasks was requested by the Job owner. These tasks will now be triaged. A suitable implementation will move those tasks to a ``tasks_under_triage`` queue, mark them as ``TASK_UNDER_TRIAGE`` or some other strategy to differentiate the tasks at this stage. .. image:: images/BP003/Task-scheduler-flow.png :width: 800 :alt: Task scheduler flow Iteration I ----------- Task #1 is selected on the first iteration, and it's found that: 1. A suitable runner for tasks of kind ``python-unittest`` exists; 2. The ``mylib.py`` requirement is already present on the current environment; 3. The ``gcc`` and ``libc-devel`` packages are not installed in the current environment; Task #1 is not ready to be executed, so it's moved to ``TASK_FINISHED`` and it's reason is recorded. No further action is performed on the first iteration because no other relevant state exists (Task #2, the only other requested task, has not progressed beyond its initial stage). Iteration II ------------ On the second iteration, Task #2 is selected, and it's found that: 1. A suitable runner for tasks of kind ``python-unittest`` exists; 2. The ``mylib.py`` requirement is already present on the current environment. Task #2 is now ready to be started. As a reminder, Task #1 has not passed triaging and is ``TASK_FINISHED``. Iteration III ------------- On the third iteration, there are no tasks left under triage, so the action is now limited to tasks being prepared and ready to be started. .. Note:: As an optimization, supposing that the "status uri" 127.0.0.1:8080, was set by the job, as its internal status server, it must be started before any task, to avoid any status message being lost. Without such an optimization, the status server could be started earlier. At this stage, Task #2 has been started. Final Iteration --------------- On the fifth iteration, the spawner reports that Task #2 is not alive anymore, and the status server has received a message about it. Because of that, Task #2 is now considered ``TASK_FINISHED``. Tallying results ---------------- The nrunner plugin should be able to provide meaningful results to the Job, and consequently, to the user, based on the resulting information on the final iteration. Notice that some information, such as the ``PASS`` for the second test, will come from the "result" given in a status message from the task itself. Some other status, such as the ``CANCEL`` status for the first test will not come from a status message received, but from a realization of the actual management of the task execution. It's expected to other information will also have to be inferred, and "filled in" by the nrunner plugin implementation. In the end, it's expected that results similar to this would be presented:: JOB ID : f59bd40b8ac905864c4558dc02b6177d4f422ca3 JOB LOG : /home/user/avocado/job-results/job-2020-05-20T17.58-f59bd40/job.log (1/2) tests.py:Test.test_1: CANCEL (0 s) (1/2) tests.py:Test.test_2: PASS (2.56 s) RESULTS : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 1 JOB TIME : 0.19 s JOB HTML : /home/user/avocado/job-results/job-2020-05-20T17.58-f59bd40/results.html Notice that Task #2 may show up before Task #1. There may be issues associated with the current UI to deal with regarding out of order task status updates. Implementation Example ********************** The following implementation example uses random sleeps and random (but biased) results from operations expected to happen on different phases, to simulate the behavior of real tasks. The enforcement of some artificial limits (such as the number of tasks ``TASK_STARTED``) is also exemplified. As a general rule, all tasks are attempted to be moved further into their life-cycle and a number of "workers" doing that should not conflict with each other. This implementation uses Python's :mod:`asyncio` library very crudely. The final implementation may use other tools, such as a :class:`asyncio.Queue` instead of plain lists with a :class:`asyncio.Lock`. It may also use individual :class:`Tasks ` for each work in each phase. .. literalinclude:: ../../../contrib/scripts/task-state-machine.py Backwards Compatibility *********************** The compatibility of the resulting Job compatible runner implementation with the current runner is to be verified by running the same set of "Job API feature tests", but with this runner selected instead. There are no compatibility issues with the previous versions of itself, or with the non-Job compatible ``nrun`` implementation. Security Implications ********************* None that we can determine at this point. How to Teach This ***************** The distinctive features that the N(ext) Runner provides should be properly documented. Users should not be required to learn about the N(ext) Runner features to use it just as an alternative to the current runner implementation. Related Issues ************** Current issues that are expected to be solved when this blueprint is implemented: #. Have a passive Task Status collection server implementation. #. Have a Task Life Cycle / State Machine implementation. #. Have Spawner features to check the status (alive or not) for Tasks. This is intended to be used in place or in addition of the status messages from Tasks, when they failed to be generated by tasks or received by the Task Status collection server. #. Have Spawner features to destroy (best effort) stray Tasks. #. Fully integrate the N(ext) Runner into the Avocado Job and command line app, that is, as a general rule all features of the current runner should be present when the N(ext) Runner is used in a job. Future work *********** These are possible future improvements to the Task phases, and may be a partial list of addition towards a more comprehensive "Task scheduler". They are provided for discussion only and do not constitute hard requirements for this or future work. Tasks' requirements fulfillment ------------------------------- 1. Prepare for the execution of a task, such as the fulfillment of extra task requirements. The requirements resolver is one, if not the only way, component that should be given a chance to act here; 2. Executes a task in a prepared environment; Active Task Status Collection ----------------------------- Some environments and use cases may require disconnected execution of tasks. In such cases, a Job will have to activelly poll for tasks' statuses, which may be: 1. an operation that happens along the task execution. 2. only at the end of the task execution, as signalled by the termination of the environment in which a task is running on. References ********** * RFC: https://www.redhat.com/archives/avocado-devel/2020-May/msg00015.html * Early implementation: https://github.com/avocado-framework/avocado/pull/3765 * Requirement check prototype: https://github.com/avocado-framework/avocado/pull/4015 avocado-113.0/docs/source/blueprints/BP004.rst000066400000000000000000000342611513561306000210360ustar00rootroot00000000000000BP004 ##### :Number: BP004 :Title: Dependency graph :Author: Willian Rampazzo :Discussions-To: https://github.com/avocado-framework/avocado/pull/5125 :Reviewers: Ana Guerrero Lopez , Beraldo Leal , Cleber Rosa , Jan Richter :Created: 21-Oct-2021 :Type: Architecture Blueprint :Status: Approved .. contents:: Table of Contents TL;DR ***** The current implementation of the Requirement Resolver handles requirements individually for each test. When two different tests depend on the same requirement, each is resolved separately, creating two different tasks. Each duplicated requirement task runs before each of the tests depending on them. This is not desired as the requirements tasks may conflict. This blueprint proposes to expand the Requirements Resolver to a Dependency Graph architecture, where the resolution of the same requirement for multiple tests, in the same environment, creates one requirement task and makes all the tests tasks depend on it. This mechanism also allows complex dependency structures, like pre and post dependency tasks. Motivation ********** The current implementation of the Requirements Resolver (see BP002) handles each test with its requirements separately. When multiple tests have the same requirement, the Requirements Resolver creates one requirement task for each test. This behavior is not desired and can cause problems. For example, two requirement tasks running the same package installation in the same environment waste one task cycle in the state machine, checking if the package installation happened correctly. Another example, two requirement tasks downloading the same remote file in the same environment may download the file twice. The diagram below illustrates the current behavior. .. image:: images/BP004/current_requirements_resolver.png :width: 400 :alt: Current requirement resolver behavior The straightforward solution to this problem is to identify the same requirement for multiple tests in the same environment and create one single task for this requirement. This solution is limited to tests and their requirements. Another idea, and the proposal of this blueprint, is to expand the Requirements Resolver to a Dependency Graph. A dependency graph is a directed acyclic graph describing the dependencies between the nodes. In a dependency graph, each arc of a node connects to the node on which it depends. The diagram below illustrates a dependency graph: .. image:: images/BP004/dependency_graph.png :width: 400 :alt: A dependency graph In the example above, node A depends on nodes B and C; node B depends on nodes D and E; node C depends on nodes E and F. Nodes D, E and F are the independent nodes and take the initial positions. A dependency graph may have multiple topological orders. Some valid topological orders for the graph above are: 1. A, C, B, F, E, D; 2. A, C, F, B, E, D; 3. A, B, D, C, F, E. The inverse topological order represents the execution order for the tasks, where independent nodes can execute in parallel. The algorithm to create the topological order removes the source nodes (those nodes with in-degree equal to zero) while enumerating them. With the removal of a source node, new source nodes may be created. The algorithm ends when there are no more nodes to remove, meaning the graph is empty or the remaining nodes are not sources. In the latter case, the graph has cycles and cannot be handled as a dependency graph. A dependency graph allows the construction and control of more elaborated structures representing the dependencies between the tests, requirements, and others. One example covered by the dependency graph is a test depending on a plugin, like SysInfo, which can run before and after a test or before and after a job. What is not covered by the Dependency Graphs ============================================ The main objective of the dependency graph is to solve the problem where two tasks depending on the same requirement, in the same environment, create two different requirements tasks that may conflict. What this blueprint does not solve is how the dependencies, after fulfilled, are cached in each kind of environment. For example, when two different tests have the same dependency and run inside a container, it is the caching system responsibility to cache that state of the container and pass it to the tests before they run. Specification ************* On the default runner, the requirements resolution happens during the creation of the test task. The nrunner runner plugin creates a requirement task when a test has a requirement listed on its structure and sets the new test task to depend on this requirement task. As mentioned, different tests with the same requirement will depend on different requirements tasks that perform the same activity. See `runner_nrunner.py#L207 `__ for more details on how it happens. Dependency representation ========================= This section describes how to represent a dependency graph in a way that Avocado can use. Current representations from BP002 ---------------------------------- The current implementation for representing the requirements of a test uses the test docstring. There are other representations for requirements specified in the Requirements Resolver blueprint (`BP002 `__), like JSON and Python executable that produces a JSON file, but those are still not implemented. Adjusting the terminology ------------------------- When a test has a requirement, the test task depends on the requirement task. When the execution of the requirement task fails, the test is skipped. On a dependency graph, a test task may depend on a requirement task, meaning the requirement needs to be fulfilled and the task complete with success before the test task starts. Also, the test task may require another task to run after it finishes. In this case, a dependency task running after the test task may not impact the status of the test run. By definition, a requirement is "something essential to the existence or occurrence of something else". Having a requirement that should be fulfilled after a test task may confuse. To adjust the terminology related to representations, the proposal is renaming the "requirement" tag to "dependency". The "dependency" word has a high-level meaning and is less misinterpreted when compared to "requirement". The idea is not to use a strict math definition of a dependency relation, but a soft meaning where after one task runs, another should run, and the second cannot start if the first has not finished. Extending representations from BP002 ------------------------------------ The proposal is to extend the representations defined at BP002, adding a parameter to state when a dependency should be satisfied. This parameter is expressed as a list of stages supported by the dependency graph. Initially, the 'pre' and 'post' stages are supported, meaning a task should run before or after the given test. The 'stage' parameter is optional and has the 'pre' value as default to remove the extra work on the user side. The user overwrites the parameter by adding it to the dependency definition. The behavior when a 'pre' stage task fails is skipping the test depending on it. If a task fails while running in the 'post' stage, it doesn't make sense to skip a test that already ran or mark it as failed. The proposed behavior is to keep the success state of the test and log the failure of the 'post' stage task. This may need a mechanism to allow failure of a task. The following example represents the use of the additional parameter 'stage' to the JSON representation:: [ {"type": "file", "uri": "https://cabort.com/cabort.c", "hash": "deadbeefdeadbeef", "stage": "pre"}, {"type": "sysinfo", "stage": "['pre', 'post']"}, {"type": "ansible", "stage": "['post']"}, ] The representation above should be considered a didactic example, as plugins, like 'sysinfo' may have the 'pre' and 'post' stages set as default to reduce the user's extra work of adding it every time the plugin is used. The same approach of adding a new parameter 'stage' to the dependency specification applies to other representation formats. One limitation of this representation is to restrict the definition of dependencies to the test definitions. Represent the dependency graph using DOT files ---------------------------------------------- DOT is a powerful graph description language. It is possible to use its digraph description to represent dependencies between tasks. With the DOT digraph description on Avocado, the nodes represent runnables, like test references or requirements, and arcs represent the dependencies. The support for DOT on Python is available thru third-party libraries, like `Graphviz `__ and `pydot `__. Avocado can benefit from those libraries while handling DOT files. Still, as it adds a package dependency to the project, the proposal adds the support for DOT files as an optional plugin. Following is the DOT description of the same graph shown in section "Motivation":: digraph G { A -> B; B -> D; B -> E; A -> C; C -> E; C -> F; } In a DOT description, the nodes are defined with any representation, like strings, for example. Avocado can translate the node information into a task and set the dependencies. Following is an example of a valid DOT description.:: digraph G { "ansible": "{'type': 'ansible', 'stage': 'pre'}"; "package": "{'type': 'package', 'name': 'lvm'}"; "test1": "{'type': 'test', 'reference': 'examples/tests/passtest.py:PassTest.test'}"; "test2": "{'type': 'test', 'reference': 'examples/tests/sleeptest.py:SleepTest.test'}"; "sysinfo_pre": "{'type': 'sysinfo', 'stage': 'pre'}"; "sysinfo_post": "{'type': 'sysinfo', 'stage': 'post'}"; sysinfo_post -> test2; test2 -> test1; test1 -> sysinfo_pre; test1 -> package; test1 -> ansible; } The Avocado optional plugin can use this description to represent the graph dependency of tasks. The image below depicts the DOT description above. .. image:: images/BP004/dot_representation.png :width: 400 :alt: Representation of the DOT file High-level workflow =================== The current workflow needs some adjustments to support a workflow using the dependency graph. In the current implementation, the dependency of tasks is set during the test task creation process. During the test task creation, the code inspects for test requirements and starts a new process of requirements tasks creation. The output of this process is a list of tasks that serves as the input for the state machine. In the current workflow, there is a direct and restricted link between a test and its requirements, meaning that the tests are not aware of the requirements of other tests. This is one of the causes of multiple requirements tasks related to different tests trying to fulfill the same requirement. In a workflow considering a dependency graph, ideally, data structures related to all types of runnable should be available before creating the dependency graph. This way, the duplication of tasks for the same activity (like a requirement fulfillment) is avoided. One key factor in avoiding unneeded duplication of tasks is making the structure representing the runnable comparable. After all the data structures related to the runnables are available, the dependency graph can be built and also the topological order. The state machine will handle the same list of tasks, but this time in topological order. Internal representation ======================= There are two ways of representing the dependency graph: implicit, using the structures already available, or explicit, creating a new graph structure. In an implicit representation, the graph arcs are defined in the runnable overlay as a doubly linked list. It is necessary to evaluate whether the doubly linked list should be part of the runnable structure, the task structure, or the runtime task structure. The advantage of this approach is that the dependency information is within the object (the runnable, task, or runtime task object). The disadvantage of this approach is that all the objects should be consulted to discover the graph structure and change it. In an explicit representation, the graph is represented in a new data structure. It can be an adjacency matrix or an adjacency list. Considering the sparsity of graphs dealt with here, the adjacency list structure should be a better choice. The advantage of an explicit representation is that the information about the graph structure is straightforward. One more data structure to handle may be considered a disadvantage, but the benefits outweigh the disadvantages. Backward Compatibility *********************** This blueprint proposes a new feature. When it is not used, it will not affect the current behavior of the execution of the tests. As this blueprint extends the Requirements Resolver, the current behavior of a requirement is not changed. Security Implications ********************* As far as analyzed, this feature does not introduce new security implications. It is important to note that the security implication described on BP002 for Python executable to build the requirements file is also valid in this blueprint. How to Teach This ***************** One of the steps in developing this feature, and as soon as it replaces the current Requirements Resolver, should be to extend the `documentation `__. The addition of more complex examples is another way to spread the new feature. Related Issues ************** Here is a list of all issues related to this blueprint: #. `[OPEN] `__ [Dependency Graph] Track the development of the dependency graph #. `[OPEN] `__ Functional test for requirements resolver fails easily References ********** avocado-113.0/docs/source/blueprints/BP005.rst000066400000000000000000000304731513561306000210400ustar00rootroot00000000000000BP005 ##### :Number: BP005 :Title: autils project plan: An unified standalone repository for utility libraries :Author: Beraldo Leal , Cleber Rosa :Discussions-To: avocado-devel@redhat.com or PR thread :Reviewers: Plamen Dimitrov , Xu Han , Chunfu Wen , Dan Zheng :Created: 14-June-2022 :Type: Process Blueprint :Status: Approved .. contents:: Table of Contents TL;DR ***** This blueprint describes the plan for migrating the utilities libraries present in Avocado and related projects to a standalone and central repository. Such a repository, called autils, is about having well tested utilities that will keep test writers efforts in the features to be tested, and away from debugging and troubleshooting those libraries. While this repository initially targets Avocado users writing tests, it should be generic enough for consumption by third-party developers outside of the Avocado community. It presents a glimpse of the infrastructure that will guarantee quality criteria and will ease the onboarding and maintenance of such libraries. Besides the plan, with the migration steps for the current libraries, this blueprint also presents the tasks already known to be necessary in order to achieve this goal. .. note:: When reading here, in this document, "avocado.utils", please take into consideration Avocado and Avocado-VT utility libraries. Motivation ********** 1. More than just test utility libraries Our current libraries are not only useful for writing tests, they are basically system level utilities that could help not only test writers but also day to day tasks. Splitting the repository can increase the adoption by other projects that will choose to use this standalone repo because of its inherent value. 2. Better reuse of libraries and code between Avocado, Avocado-VT and third party projects Currently we are duplicating efforts by maintaining common libraries in Avocado and Avocado-VT. By concentrating those libraries in a standalone repository, we will make sure they will be well suited for different use cases. By having more users, the quality and collaboration of those libraries will be improved. Also, third-party software will be able to import those libraries without the need of having the entire Avocado framework as a dependency. This will reduce the learning curve of future potential users that don't need the entire framework. 3. Making it clear to our users that Avocado framework is not Python-centric: Currently, those utility libraries can only be used by Python tests and the Avocado framework was designed to support tests written in multiple languages. By having those libraries outside the core harness codebase we reinforce the language agnostic characteristic of Avocado. 4. Reduce traffic in the main framework repository and allow for a scalable community on its own standalone project As evidenced by “avocado-misc-tests”, a more focused standalone project will attract a larger number of similar minded developers and users. By being a simpler standalone repository, with its own specific contributions guidelines, the entry barriers are also lowered. Specification ************* To avoid the need to change our current imports, our first attempt was to keep the "avocado.utils" namespace regardless of the location of its codebase by using the Implicit Package Namespace (PEP-420) approach. Namespace packages are a mechanism for splitting a single Python package across multiple directories on disk. Unfortunately, to be able to convert the avocado.utils.* modules and packages into a namespace package, we would need to remove the `__init__.py` files from multiple places. That would break, among other things, the current Avocado egg based deployment model used on spawners such as podman. Because of that, we decided to have a new namespace for the utility libraries. We ran a poll to choose the name and `autils` (Avocado Utils) was the winner. So our goal is to host common libraries under `avocado-project/autils` repository on GitHub and make them also available on PyPI under `python-autils` package name, since the `autils` name is unavailable. Utility onboarding ================== The primary goal of autils is to be a central place for modules that ease the interaction with system level features and interfaces. As such, if a user needs to interact with such a system level feature or interface, he/she may propose a new module. For instance, if a user finds himself/herself constantly interacting with disk partitions, he/she may want to propose a “partitions” module that provides basic APIs for that. In this case it makes sense to have it as part of this central repository. However, if the utility intends to ease the interaction with an esoteric feature, system or device, this may not be a good place for it. As a general rule, if a system level feature or interface is generically available to a multitude of users, such as the partition example above, it is a good candidate for this repository. Common sense is much appreciated here and we can always review the proposals during the PR submission or within a Github discussion thread (even before the proposal is ready for submission). Besides that, the proposed utility must have at least one MAINTAINER that will be responsible for keeping track of Pull Requests, tests and quality requirements as described below: Quality requirements -------------------- It’s expected that both new modules and changes to existing modules will need to observe a full set of quality requirements and general standards that will either be described (documentation) and enforced as automated checks as much as possible. Among the topics related to the quality requirements and standards, the contribution documentation and automated checks will cover, at least: * Logical, stylistic, analytical and formatter checks * Docstring checks * Pylint * PEP-8, PEP-257, etcâ€Ļ * Unit and Functional Tests Coverage Due to the current code quality and given this is a new repository we would like to reach at least 90% of coverage. This way, users writing tests relying on these libraries will have a guarantee of a good level of correctness and quality, redirecting the debugging and troubleshooting efforts away from the well tested libraries. The new standalone repository will have predefined pipelines to run those checks. Newly added libraries will automatically be subject to those checks. Metadata description -------------------- Every module will need to provide a description including: * Utility description * The utility category (ie: network, cpu, security, etc..) * Maintainers information * Main contact (We need a contact point, a single maintainer or a group) * Maintainer Group members (this will be used to block merges) * Supported Platforms * Supported execution method (Local vs Remote) We intend to flag modules that have the capability of acting on remote systems. This will probably be a broad classification, just for informational purposes because unfortunately we have way too many ways of executing code remotely. We don't hope to address a complete standardization of the remote execution APIs just yet, but we want to at least bring some visibility to it. For instance, the textual category and description will allow users to search for modules based on their interest. Also, our pipelines will parse that data field containing the list of supported platforms, and will trigger tests on those platforms. Users will then be able to see the results of those tests on each of the platforms defined, giving them quality assurance and coverage there. The exact format of the metadata file is still to be defined, but a possibility is to define a JSON schema that will be provided on this new repository. Maintainability =============== As described above, each library must follow the new standard and keep that same level for future versions. Because of that, new PRs must be approved by at least one MAINTAINER and pass the basic checks pipelines that will be provided as part of the autils repository. Besides the automatic checks, each MAINTAINER is responsible for keeping the consistency of that library with the overall project. Release ======= As a new and standalone repository, there will not be any type of release version sync between Avocado and autils. Avocado may release, say, version 100.0, while autils won’t have a release. Because of that, autils will be versioned at a different pace than Avocado. Every week an automatic job will be executed to determine if there is a potential need for a new release: * If the number of commits since last release has reached a threshold; AND * There is no open discussion with the tag “release-decision”: * Creates a new discussion (with the "release-decision" tag) to run a poll between all the MAINTAINERS, so they will have the opportunity to thumbs up if there is a need for a new release; * The link to the new discussion should be sent to all MAINTAINERS; * If the decision is positive, a round-robin selected maintainer should be responsible for coordinating the release it self; * And the discussion should be closed * If there is any existing open discussion, the bot job should comment on the same discussion with the updated list of commits and ping again the MAINTAINERS, give them the opportunity to thumbs up based on the new status; Manual release process could also be started if a MAINTAINER think it is a good timing (i.e: Security bug or critical bug it was fixed). Backwards Compatibility *********************** The current libraries are not going to be removed right away. A transition period is expected for migrating each library into this new standard, and because of that libraries in transition should be available, receive bug fixes for at least one LTS cycle. Each library in our current code base, should receive a status flag, informing users what their current status is, according on the following table: 1. maintainer-wanted The library is using the legacy mode, and it is not yet converted to the autils repository. This should be the status for all avocado.utils.* (including Avocado-VT utils) libraries upon the creation of the autils repository. 2. adopted A maintainer has picked up this library to be ported to the autils repository. It’s not expected yet that a working version is available on autils, but someone is working on the migration. During this period, the avocado.utils (or Avocado-VT) library should continue to receive both bug fixes and improvements. It’s the maintainer’s responsibility to make sure that the work in progress version of the new autils library includes the same bug fixes and improvements merged during this phase. 3. migrated This means the current avocado.utils library is deprecated in favor of the migrated version at the autils repository. During at least one Avocado LTS cycle period, the avocado.utils version of the libraries should: * Include warnings in each deprecated method or function, pointing users to the equivalent method or function in the new autils library * Keep receiving critical bug fixes backported from the new autils version At this point, this library has an official maintainer who will be the main point of contact for. 4. abandoned * There are no plans to migrate the library to autils. Once at least one LTS cycle has passed after the “migrated” status, the legacy library can be permanently removed from its repository of origin. Security Implications ********************* None so far How to Teach This ***************** We could provide a few support material as reference: 1. An "onboarding guide" on how to send a new library to this repository 2. A template library directory structure that can be used as bootstrap for a new library 3. Existing libraries in this new repository can also be used as examples Related Issues ************** Todo: before "approved": Create the following epic issues on GH: * Migrate one library as a "reference implementation". I.e: "network" or "archive" * Create an improved version of our contributor guide focused on the autils repository * Create pipeline jobs inside the new repository * Create release polling bot References ********** avocado-113.0/docs/source/blueprints/images/000077500000000000000000000000001513561306000210165ustar00rootroot00000000000000avocado-113.0/docs/source/blueprints/images/BP003/000077500000000000000000000000001513561306000215425ustar00rootroot00000000000000avocado-113.0/docs/source/blueprints/images/BP003/Resolution-to-task.drawio000066400000000000000000000035411513561306000264770ustar00rootroot000000000000007VtbT+M4FP41leAhKJeWKY9Q6OxKs9IKWM3M08hN3MSLE2dsp5f59WvHzrUuTWlhKBtArX18/75zfHpOw8CbxKvPFKTRXySAeODawWrg3Q5cdzR2xasUrJXAuRw6ShJSFGhZJXhAv6AW2lqaoQCyRkdOCOYobQp9kiTQ5w0ZoJQsm93mBDdXTUEINwQPPsCb0q8o4JGSDod2Jf8DojDirYYYFH21gEUgIMuayLsbeBNKCFeleDWBWGJXwKLGTbe0lvuiMOFdBvz7K7pb/sO+OLMpeXgcDe+/Xn2xxmqWBcCZPu89ZARnHJFkICf0xDr22X2WJGCG4fnAux64l1gseDOjohTKkiW6PKEkyBvtdM0jklhZgjiHjJu7ZxSp3rLLRSqU4/pRlmT1h2MeQ+HPDFEYi/My4z7EO4hTUUhmLM3r4gAGqRrHUiCPyPhaU335M5Nc3PgEE6p2R8PZmTfM55mI10bxXJbzReYk4dYcxAiv1bBySYGfGOPdAIoA3pCWk220RBAvIEc+2D6GgYRZDFI0b+6D5RYkd8FikCuAapwB/ymkJEsCq3VAdzQqZ22VzytUCpDnCOv54zVGM8mdiQis3qcK5Z6fN+MnFfPkV5ocHvr+Yez0VBxARa7cdoMQYTG+FcCFdJKdeXEb2LscrqQ84jEWAkfywyl5ghO1V9HFs/MfZa24kCckgRJgjMJEVDGcy2kWkEr08LUWxygI5Eo3ywhx+CB2L5ddCucuZDkoUDoUu9xZ3QNppyTnhKuaSHukz5DEkFNxY9i6tfCW62Z1WXla51LLopqXdQqnCrR3D8uZKw8oCtoJ7uEQr3Y6RPftHaJ7NIfYX7VHt++DHOIHNu2X2vb4tUy7iChqtl3wEqCFUf8lH5ZGNb++JbAGPdhhRi9VOK2vH8wK2/A9AvY00LFGdYttfjopxDlZHZH/YNAd/QI7hvpb2vBkD8cyBnRnf54/T+AWtntefzOvOz/F9HSeEp3d0i49p6fE6ZYowEziK3/QaUFilyDUSvsQ/jyBJrYcN111IWs4rggaXtXIss1k7ZG4ahnKPvFBD+wewHbPdPWwdgL05fmqt79pfruzMF3FjAOeMeVnW3exyXX2UJqhPOhWKMRibTuicN44asR5mp9xKv4c99OFLX5l1Dm2ZdphWt+LzQENIW+M/zHDIHlq9HqHeJrTBs6nsvO49unEHhoukE1oSqzBS7V4V+6rmYAypag6pcWOn5TunpUuhMdPXTl96uo9BAHbU1dun7o60fipSF25xtSV26euTpTXPnX1oejs9gVpz+kpcfq2qStjTLHv17e9dp2Mdm2Lxvuwuw+7X9FK+rD7SA+MvIe42zXE3S0AWQRSWZxjuLqWD5wLNGAS6OKtjwFjyG/iCFeIf5OQX4x07Xut5Xal2cgr66KSiBPVBsnq93pbNSyvFeO2MsNIRn24+1m4mkE+n52AQeNR+k2ia0SODDwWMgox4GjRfADfxK1e4W+CxNFq6ZumHnl2Sz3UwfWo+hPz7YlGLYUctSZSyGxMlKtaeewDtM/7P2vfuKv22b327ad9olr924nqXv3vjnf3Hw==avocado-113.0/docs/source/blueprints/images/BP003/Resolution-to-task.png000066400000000000000000001027721513561306000260040ustar00rootroot00000000000000‰PNG  IHDRi›!a\sBIT|dˆ IDATx^ė¸ÅŌ†Û„EDA*‚Y¯×,ææŒYÁœ¯1bĀ0c@Ė¨˜PTĖ•¨ˆ &0ũûį,;ŗ=;ŗģŊģÕĪÃœĶĶ]]ŨũMMuÕ×ķcūÉüŅĸ(›ūųĮß%8ß|ķ•M/Úąj °3{ÄßMĸĶTŨ}^žËWŨĢCGĮúSÖuPV ø‚žËWÖÉĶ΋ŽéĸĢX;ȧßAĐwųōéW_ŲPŽėųĢ é}AßåĢŠE ƒՀ‚´.Ž˛kĀwô]ž˛O  PT (HUŊÚ¸‹|AßåsŅąÖŠ\ (HWîÜUäžƒ •īĪ?˙4 -´Yxá…ëtß´iSŗãŽ;šk¯ŊÖ,ēčĸŠÎÉģīžköÚk/ķ駟FļûĖ3Ī˜ÕV[Í,ģė˛æ˛Ë.3üąšå–[R“Ĩ{÷îæŠ+Ž0Mš41]ģv5wŪy§´}ūųį›n¸ÁĖž=ÛlˇŨvfāĀf‘EÉŲoįΝÍá‡.:ĸnƒ ęę-ĩÔRæčŖ6˙ûß˙R“ŲĨĄ(ũ0Ā|đÁ‘z<á„LÛļmÍ駟îŌ]Áu¤ V>˜–* ¤§M›&€H™1c†×kŦaŽŧōĘ´T"í¸‚ôŽģî*÷ß˙ū×üú믆— /´Ę`îžûni÷¨ŖŽ2C† 1<đ€9ûėŗÍŗĪ>+}ížûîfŗÍ63įž{î<ŨŪ~ûíæŅG5÷ßŋá… _iÚĩkW4ą¤‹ĻZmØU• ŌŒĢõž{îĸ\pÁæŽ;î0Œkë­ˇ+Ë+÷Ļ›n2˙ũˇYzéĨĨΊ+ŽhŪ˙}ŽožųÆ4lØPžĮō ‚ČÅ_l&OžlnŧņFéÃū…V0įwžiŨēĩéßŋŋ™8qb%Öî{īŊg9äŗË.ģ˜ŅŖG›/ŋüŌ\sÍ5fûíˇĪ9e¤?úč#3|øpĶĢW/ķæ›oš9s昍7ŪXžáõöÛo˘˛ V> ŋîēëæięķĸŖĀŋÃ;˜C=Ôėˇß~Ō”ũûöíCåžęĒĢ `˙ûīŋ›Ī>ûL^(ŧH°tŋûî;sđÁ‹^Đũ)§œbN<ņDŅīŪ{īmēté"/žnžųfĶŠS'´¤g͚eŽ;î8ķęĢ¯Ę—4ķEšôŌKĨ?æ•Âŧ6ĖlēéĻŽË?o=éŧ*Ō ÅÖ@%ƒ4–ôžûîk6ß|sųüøá‡ÍYge^yåųôĮ]ÁĻčVYe3eĘųäŋõÖ[ÅęeŗcsÎ9f˙ũ÷7~øĄ˙ķĪ?¯sw„4 Íķŧ(°¤­ģ° k™×^{mķØc ō‚¤_~ųåzS Pa%"K‡Ė_|aX`sĐA hËÎ;īlvÛm7ą´ƒ…ņ`%O:U~œË’æĨ  įĨŌŒ'LnÜMŊ{÷@‰^—Xb yávÚiæ‡~0ƒ ’`Ī˜žūúkŗÁ0vØaæļÛn“ú&L¨Ō€úˇß~kø"˜9sĻYũõÍC=dÖZk-ŠËÄW_}%ãC_˙ųĪLķæÍSÛ6 ŌŠŠR*T•Ō|Ū#ķ_ũ%–Ų1Į#›K ŋëĒĢŽjzöė)ęxüņĮ8GŒa–YfsŅE‰õ€P €į§Ÿ~’6)m´‘9ķĖ3Å ´>é¸ û#ŦŨ•VZÉl˛É&æĮ”ū°@ąÜydÜ?˙üŗŧ@āŧÜrËÕĢÖ§OąČGŽiæŸūzŋÃ5BXā¤ąØ­˙v‹-ĂîÖ­›Ô‰é0š駟~Z^’”Ģ¯žÚŧõÖ[ŧXÕüņ‡iÔ¨‘üŽųAŽÆ‹Å‹xų uoüîÖ'Í×ÎĐĄC͆n(Ī÷čŅC^Ā|ÁP_|qķÎ;ī˜å—_~ũĨņƒz ­#õFøüÄjĀĸaÂĸŠŒž+ ¤­Oë‰Oô1cÆČß\|Û4€œƒ1Üld@0ÁÂÃĒåŗ>XĨøxŠS(HqÄĄíbq„Ŧk k;ø˙āLbxāâsENŦn[Hå?餓Ė'Ÿ|"Ÿø,f—lw@ļ%ÍK ׎{˜ŌarŌčŨj˙Xã+g?ķáĨÄ …9ÂÔ=cĀ ‚\¤Š×ŦY3ŗā‚ ĘđsžžøŌ đ…DŋXĐÅ(9AZF†üP,f6°° Ѓ‘dKŗRAšQŗųņÅâ: yä‘âf8ųä“C•‚U‡õ XāËÅâŚŗV(–e›6mę@3‡mÖ÷‰__k˜ģÃZŌšÚŏíŌXĪ¸@ÖYgąōģõÖ[O)‘7.üĐøjs•K.šDüäVî\î\Eø…m”/)|ÔôOÁzÆŒ^ i,į3Î8ÃđâĸŦŧōĘân|ˇØb q…0>|ėœ X׆ięķbžr•˛ƒ4BÕâÁŸxX9üÁŋeIëÁHí‚4n Ü|oŗÍ6æ‘G1}ûö5Ī?˙ŧ|Ōc-c!˛F°*ņģŧØīģī>q>€.~kŦ8ü˛2žĩ¤yŽč‡—^zÉU€K#DąĐ0ë“&.Ŧ]€Õ¤™Ur7~üxąĸí'ū /ŧ ĀÉ ĘZ˜šVÁāÁƒåtwdGwāŋ˜ąČ—\rIq{´lŲŌôë×Oô€īrĄ M›O=õ”ŧ`°xqŲÜ{īŊâģægîąĮæŽģî=˛×ƒ‡§žzĒœnˆˇˇ_QŦæmˇŨļ^t Á§9uđ ķ˙Ļ ^ —_~š€Ÿâ6NÚFwdˇ›íŪswp(ÆË—cĀįēÕV[Ʉ¨šÃEŦí`áPW`ÔÁ!õ9t$Ŧ7{ +K—q~˙ũ÷ĸ€u!–4/7ĀwąÅkš/,iŦ{9/Ö!/,÷ėčŽã?^ž*ĀHžŽųâë õq}ņ|ŅŖ;,HëÁČŋ#ų@ZFj¤“´öZĀoģk•ŧj+ÅHĘÖQ¤OēÖFPV>Öƒ‘äÛŽR,éä#­Í°Âq7āb¨ĻBL6_O<ņ„|1Ģä=8Ŧ僐փ‘äKSA:š}nk—ŽĸVĒĨ•Âá.!yÅ,yAēVFŦŌķYŌz0’|y*H'ץļPŊČ Ō ŊF|đA9ŧĀ āˆzBy8 =Iž9¤“ëP[¨^ hÆašՃ‘„ Ė<^- M‘–|Éj†(".˛Ĩ9ļB4OpžaBČĸ’X i;­gˆž q†đ?ĸ0āņ ŽGHlĄŧIŸOk|qÛQŽĢąõõ`$™Ģ¤I3&V7;eē M˜ É(Äķú Ō$—KNš=ĄŠ3…°7B I™/¤$}ž>ĶxFA:-ęÁH2%V#H“œBF!\„´bIŸÆ>G¤ŧÄârE¸ąŅ$’aU’ČAb°“ ģ1Õ¤nÄO +Ū¤I“$%Bv }yšPH(á˙XŌalƒŒ—™˜¸E83ĸ/Æ|ž’6Ŗ‚t%ÍV•ĘZM Mē7VܨQŖĚĨA:ŒÅ këZL . ĸnX¤ˇÜrKą )XÂXŽ¸WāČĀZ…ü‰zd+BĶ‰ui^DpQ‚lx€6šåĶāE8â.¤!‰"-'V=iŅq@:ŒĩΤÃØyÉņÂâ…F:7)ûļÔHëÁH:(Š#õõX) ũ%)Ø\ÖJĩŖa`MÂŲ0ZjĖ H‡Ĩ5ç w‡=$㞌| M:5āk‹eqãAŠ2r`Ã÷e QP¤ÃäÁåÁ‹×å7Ūg,q@+™LBŠũ?tŦAũĀ æîˆb­sé(ļA^Rp{ĄE*8n€ģĻAZF⁡ŒT&Hㆀ‘gSsŌÁ‚đEcÉR\@~\–= –u’¤aƒéŽ‡jø^ņ}cŨâĸāĶ7ôš‚4–4W‚ÁG!ĢŸ8sHsČÔĪ¸qãD^6,gH™đQ'iļA+7‘,đ—đRĩî’x;ļŧĩcš;‚–´ŒÔ?aõ`¤°Å\)–tžŅŲũ5 ĀA@„;¤ąö`ŠãÎ@@Ā†F3ŸOŸ2ūcØō`xã@Ī26BąkÛė‚ørŠķDFa–4 }ÎáŸøy‰pŖ –rH[ËÚęiúôé’[aēÁ {ž+Hs¨ÉË}đBÄDÄō‡ą îŗĪ>BÅË@æđˇ 7Ēā†˛Īį›KŸ~_HëÁČÜ)ԃ‘t–rĩ4āˆ@ˆŸ `Â2EŦt"5OXîî$N8,ēW ˜eŗ¸13ø¨éęOų(„ŗáûæ2Ãä FwŊ„ÅΗ`é ŌôĮ%0čAGĘ5]0ĮšXŌXḁ ŕXīšįž2^^€šØšũņ;ÆĪ ׎,ųāķéŦŪ⡤õ`äßIÉŌz0âļ€Ģ¤ŨF›^-{‘* V°~ą*!ŌR™¨Ōz0ĸ#åXÆ Ō…i=x›xŽ°(š5Š–ĘÕ@=Öƒ=)ĮRV.LëQ MŒ0~XÜ.ŲžâÂzĶ§ĘĨØî%ô`dîtéÁH:ËVA:=j+ÕŠ‚@ZFū] z0’|c(H'ץļPŊˆŌĨVƒŒ”Zãåé¯Ö@š4o"6lX\˜ÖÉäÛ8ėyÁļ’>&‰.ÖĪM”w6ÚBŒ6 $Ž%ČʗũLRų]Ÿ'ã’{LÉĒ$c“°E¸B|)Ūƒ´Ŋ-9—Âô`ėe”LŽZi@6Âņe‹*¤r“έā…”¤ĪģôÉ „č‘Be ˛ōe÷—T~×įI°!Ÿ„ķ tŽ„úP*¤õ`ć哎 ÕŌ$}ĸM:6ÅūŸŗ ¸&Čūƒˆ”okI?ÍĪą(ƒ"ŖķÎ;Oø?¨ĪmŲaŦo\†ĘÍ×Ä6Ķ7°×e?oۏ˞Gģ$ĸä’3¤I áÅ2gÎ!^‚]XmôB8ąßŧ¨Ž:ę(IĶ&4вōaųÚRĖņg¯ZW×\sM‰åĻîTqŨ—ÅköAA*Cņ5Pí •F!€Eæ^ĐŨņķĪ?›įž{N@!ģŠdbŲ…ąžÁ÷™Đ”)S$U¯K(Nųd>lģö<ÚĘ%g¤I”!.æ<ę_~ųå’ąH&%€Ė „Ôr’R@ôA.å)ÕøŖV4ŽÆĀ ƒŋ}( Ō>ĖBËPí odC2Ä˙Ž>é Ȇąž‘‰G–)Üđ%“FmKHĮeĪcŽr• H“č6tčPáų 0^¸§Ą%•@†ä‰Tv{BK$ ¤ĶØvƒ„ŠŒD22ų ņĨ¤]ĄĢc?LYIŸkˇÚF'ž?61iĀ¤äúT*¤ķ%{…š;iÖŽŠëū‚lës H‚ ŧĐІFt\öŧ°õiÜ.Ŋ{÷ŽįkĮjūčŖägü˛'üž={ö.WNsüšÆÂéæ:uĒcōķe¤Ōz0Rj}8â¸ãŽ“MËÉģ‚tŧíg_"ų’Ŋāĩ b,>Y@4 äČ(¤]X߸0šOŸ>‚Ÿ5 ¤ã˛įš€4O€.L€Q…4vĀšŲÅÅŨ‘æøŗeÃŌį‚xP˛ég㭔âԎŌz0bL%ŒøaZÃo¨ o3šZúø†šbŠ `qUÖsHGų¤aoÃBāÃXßŅŀ…ĘūÄ僄KÁįąh!üĮm€O:.{^˜œAKúĢ¯žö?üĐíÚĩj1ųŲ˙ũEŒPäFdE?–•=3Ž­ˇŪZŽũJsüQ3΋}Áí8>–T@ZF*ã`¤_ŋ~ ŌėBWšëģvÚÉ\qÅTD/„YŌaŅˆ -(‡oĖabšXßxų’P¯ŒĪUQÜ |Ō{Q{^˜œaŅ^r ë‘Xčá˜cŽŽküŅđUã ˛ōqĄãÁį™4ĮūÂJÛļm%ę$xQ0˛rq­%Öƒ‘Ę8Q.lËš‚ta­—ū)M+ŊΓô‹OF*÷`„Eĸ ]ØVŠFÖ$ąÂÖB9žŠÅ‚§#•y0b–‚ta[Ŧ–@ē“Äė5šfˇ ‘&>—Xî=ŠĖƒéd[°Ú@:™6ôéRk HëÁČŋŅ•t0B<*'×6ŧ ˜á“†+Ģ7ģÔXĪ֒~ôŅG…ļæūÆˇKéÕĢ—đ]āŖļF mš@ĸØįpw¸€t˜+HÛqfŗņųąĘ*WŠX ­#z0RŒĨ^) oėŲ`ļŪzë‰ÕŠ• ŲI'$\Īø¤ˇØb ķᇊO×ôĸ€3,p0éŊôŌKžg,ō(ŽbŸsé0–=ÂëÂ@ēE‹rÉî’÷ß?”O}ŌųVNôīc´ŒäžĻHaōEH8—¯Åõ%’ f¸48p›6mšøĨO=õTš•šrîšį [üŪ‡v˜üæ0Ž;ö8ˆƒMŸ1˙æ÷a–4íY Ũė$+WĻ\,{ø™Ã@WÉvÛmgŪ~ûmņœė–‹OŖ;’­ėX ­#š¯)J6ú´+–KSŖš¨¸ūr[ûõCõ@ZFV–YŅƒ‘Ō.Îbƒ`ŌŅ[>é¤3TŨĪ×i=‰i=)Îf(6&•ēØō)H'Ąę~>ļģ#ŠŖ#s¯)RŸt˛MRlL&‘ÄŸ}æIĮ§Īû­D ­#s¯)Ōƒ‘d‹Üwô]ždÚק}×@,.t0ú9W¨æjã9ßAĐwųjc•Ôî(¤kwîŊšī X ųŽ8â ]ģä’K$å69Bô6ß|s!_‚ÛŖœ…dbšųZæ’Vûī‡zHˆ”ˇĶR (H—FĪÚK„Š‚i*<Ž|đrČsš äDë¯ŋžyųå—åvėYŗfÉMߍ7ö¤ÃĖ™3 É+ÄA+H§šēōˇUÎ/†Ö¨e ÄÁRë*Ž|¨Ãjđf"¨:wîlĻL™bVXasÆg„ZŌdB%úÉ'ŸH[Ûnģ­¤ķoŦī3Īh°xaĢ#lGU8äĻ-^ $š`!ŸvÚiÂ7Âm=2b(NŖ,é:˜§žzJ’t`Ę#k2ģ`mĶ'I9ôËË FĀŊ÷ŪÛėšįžÂB"Ē h„aäkBËŋPÖÕPv ÄÁR Wž(Fv@“ToëšG9Ũ]ģvzĪsÎ9GRÄ>ø`áp†Ú‡zë­ˇžG#mÔĢŽēĒ¸O_Rŋy1üđkœūųŠGö#„M”8Āė´ĶN"Ļ0†KúĘ+¯”z¯ŊöšČí))îÁØķ"ŦaĐ?~ŧdSŽ5JūĀŪ7bÄy„vN>ųdI‡Īõr+õœûԟ‚´OŗQŖ˛ÄÁRĢÉE>,UK˜! ŦÃË/ŋŧžČ. Ĩ ØŋōĘ+uāŨ(V2€Ui,eHœ(€ä]wŨU'ßwÜ! 9hĐ ‘ ęTøĨūN:IŸ0ㅁô€ĖčŅŖëDā8JvŪyįy@šËž{îšēŸķ5c-ķĸēįž{ĖkŦaÎ>ûlĶ¤Iš4AK} ¤™XŧxãÃ;`E‚Cš'ÂĐ´mÛ6įbÔˇyéfČK'Íŧ=ŕ/ Kēe˖â×nŨēu@ūų§p9Âų@zȐ!âļ đ7,¸ķ7îęp üX°đ@› ÁSÔÁ!€?|øđ:öØcIkĮ Û§•ƒßërP '5–ü†nh`ķ[mĩÕĘ9Õ^ö¤ĄIäTzĢ­ļ ‹ "nĸŠøŌŧÍíĄHpa*H—nŊÆÁŌI6ˇ§¸ōĨŌМrŗ >gx›ã@ܤiwß}÷×în’Ųf›m"A+Ël ØÛ;î¸ã< ͍ôXíļ`u㇧.;Ė€D¸ā>˛UËŧˆ ŌøÁø4ÁߕĢÎL Ÿ{€4÷ąå˛¤įĖ™#ˇTđ)„ŧAûöí+wžĨq(‚ †ôœ„ ‘Vā˙âo|uš,i>]ųā“Å‹O0´õP$ŊíĶëŲ­Ĩ¸ōĨŌ„āq÷ Ž@ƒ5hNY×øĨŲW¸)pId—¸ …¯_Ö;Å"FYŌ¸$x #D˙Pąâū úcäȑr` C4uqĢāûÎŽK'ōsøˆ_›Ûa´”¤éŋ“cß°š@š7,×úpxĀáIŸ>}„Ü’§4E¸ ß ŪyįÉČ9ą†’žŖÜ,äGNâé\ū6=IgKÅÁtzuo%mų\|Ō îŒiC(ʐ‰ ŌЛânĀea÷JHŧ¸E0nˆ ÁØbŸņâ `üéčÚp= °\uŠ÷ V5ÄfZ€4Á÷(ÅÂI3N~ Ęoßž}Ŋ–]@šĪŦnN{)pîrR͂Œr5Ä9ÁĪÆį26hĐ@|o+/(æK€[•)€õB -$›#XЅŠ¤ŗĨŌÁt¤úˇßåK:ŪŋūúK,Z,uûՙ´MžgđÅI\uXáā׉Ŋ‰&~Ģ­ÔŨ(Ȥ æŒ^xá:ri''Ōø‡•8V/Ā˜!nIæ– ›~Ŗ@š36l("ā/ãÂN,ülļ1Ĩöįz(RØöđ}—¯0­˙û×yaÉõ‘fɗø‚ˆ˜iܞ\ŌĢ%ˇĘŌ¸¸ĩ˜CÆ8%HĶîW\!ôćōi†?Ō6ŒzÍŌz(OĶžƒ ōiQ ”SąË)¨ö]đ¤ĢSëÅs™ •Ģ'ę¸+GŌĘjUAē˛æĢę¤UŽž)uK×zÕŖ™d#QNĻ?}:Ąyįˆ" IDATtÃ&T GģÎĨk=†VVQŧie§+ëú(zįēa‹Žâ’uā:—ŽõJ&¸įÅiĒŌ´ĮĢ ļFũjO7Ŧ_ķ‘Dךt­—D–jz6HįË8$τˆ‰¸ Æ/ŊHĮ†3ƒßķ;•Hb!“$&möėŲrĮárœÃ¨EŨéĶ§×ķĶäā¤”’rÚ¨Q#iúQڃqëŌK/Ļ0ūr@–Nú)DãēK!ŋįŇ„O°ŪQ‚‰+Ų“=vėXIpáž:Ž'‚ŒûÛč7l $Ã@¨NģdTB9 iĶZk­UMkŠ ąč†-Hm^>ä:—Žõŧd„J¤­Õ .  "iŲüžHŽRØč6Ŋ_ŋ~ÖP‰rĄŠâ€4”‡d ī[ƒ`…= "&8ødG!l}đŲüP*ú‰# å(ĀNú9ŠčÃ ŧ4ēté"Ŧ]íÚĩ6žß˙]îf N¤áÖ‘K5?ūøc‘‘ŸÁO6Úf¤Ã“VÎ5H€ēŪéæœtödÅvé žŽõ*V) îŌŽTĨ¤ƒwŗaé XŽ˙†´ˆB äü­tĮŧXĐnu&”+¸ R:t¨ÜnLĘ6@LĖI1t)<œ ƒúąš|@ Hŋûîģ’Áû׸§ ŠRx; ŋ€lđÆ Āš´v@:l Xī=ÁLĄ žëZÃÕ ›ōŽ.csŽséZ¯ŒCņĒk'ļįswX7n\ iîĸˆŽ¸°ž4i’4Žn´ĀåōÎ`ąmâ"ĀŨ„In[!y„:´‡{eĨ•V’›#Ž=öXqGXĐé,ąĸšMÂ\XũQqžÜx ŸI2\š°€— 6.7Āĸį -õ5 ļzV„ë\ēÖĢÍ$IQ@šët–Yf‘ ŋ2>_84¸‚"îQŖāūĀ’¤q;pĮ×nQ° ņ+sũ<–4\ĶƒãBá6 n„6ÚHÜ'L<îRÁi~Æå¸9(đDSÆ¯ĖŊjAŌ%5~ųå—ōÁŸŨ´iSy„— >p@:l XŪũû÷¯ģˇ Ē|p]Q-Ũ°Õ3ûŽséZ¯z4“l$EiܧœrJ=ēOnpÁ/ ņ>ļ¸p.Âä@w€,Ĩk׎âžāÆ{Ŋ<Ö.‡{‡v˜üĀėŲŗ§üŒ›áéā0ß2Vë7Ū(lō3Ü-8Ō€7˛ĀÄĮß?žqڄ΋™ž f ÜÔįĨÂíČ&Lŋaū¤ÃÆĀ8ũņ‚1b„¸apÍÔúĨˇēa“m`ŸžvK×z>­œ˛Äé|‚Z×Q DaXNfK÷ PXĄBƏ{Ÿ^¸ oĀĢ+ëՂ4€†ËåĀ”C@Ā”›đI¯˛Ę*âãÆG hķ _@™{ų9/ë/ˇŅÜeˆe‹Ģŋ4}DEwđ<“<ŗöÚkËXtd‡o:×h@æĐŋ6Žƍ{¤Ö‹nØęYŽséZ¯z4“l$Ei,L,DÜ%VBßpōXÖI /|ā(j,z-ņ5 6žÎ|}Âu.]ëų:ÎRËUĩ ˜ÃÇ~XÂõ8”$Ę⡇J¤c\-¸Dzôč!1Ī¸2°ž éĶ_ēaãëĖ×'\įŌĩž¯ã,ĩ\U Ō(’CJbĒ)„×áÆ ú#I!ĸ7 ŽBšLķ¸ãŽKŌdM?ĢļzĻßu.]ëUf’$UN&Š>]‹Đ [=ŗî:—ŽõĒG3ÉFĸ LútB č†M¨@wK×z ­Ŧĸ(H—UũÚšnØęYŽséZ¯z4“l$ŌųîLĻ‚øOįēm<;T0~ĢáOėšįžæ ƒĒKôIŗmÚ˛ã ĢÔE7lŠ5^ŧū\įŌĩ^ņ$­Ŧ–cƒ4YzpT čR_Ašlə3gJzy1)U¤KĩŌ´Ÿ$p_×zIdŠĻgcƒ4 ¤^‡ÅA“íŋÆrË-'ŠĪ°Ũ‘HBåÅ_”Ëa!Pĸ "#`•ŖĖ˜1C¨GIá&•üÄO4{ėą‡ ‚4I$đq0Ņ¤vŽaô§$Ē*‡„á.7pā@‡…ŦÄAƒIú9dG$˜^Gč &>.´Í.Q–4áx¤¯ÃOĪ 9Œ3llšĶ°aÄ1č.Ņ… 9H™'9í’íHv'I;Pŗá a.˜“(jVĩ¤ĢiK—o,ŽāëZ¯|#ņĢįÔA@…7`€ę“Ŧ@ĀqÔ¨QÂëŧÁ)qŀ=ŠØ ŋ#ŚßÃķ1qâDynøđáæũ÷ߗ CÚ$k‚#ĀŪ‹(úS’Xf8=x)ā2āoâĨáw†N•¸i^(đN“:Žœ€# GØŋ‹Ōd=ōŌāAæ#2@Ķ662%ƒ&àîØąŖđkī k žx†vÉpŧwß}wŗß~û w7Šé0õQxÉđ;úĸfUökSVĒ4ŽāëZ¯RõļÜN uHRđPˆw|ÁHŸ¤ÅÚx @r‘EŠs—tčĐAø Ãõ×__Û˛Ķa…“͐Ō¸YH+‡ƒÃf FŅŸÂû%Ísøîēë.iØël:ˇ•ßŽ"¨¨eIō€;°đątÃƖũE×úãĨ”‹”uxCø’ RÁō5?5:$œ1‘fœT{¸R¨EÍĒ öļĒÍö\Á×ĩ^mjqŪQ;tđą|î@Āāo[pYāŌĀš„ë ĩWē[+Ō#ø3°ŧŗ™áh Â-•ˆ;Á–|ô§0æYĐø¸†ĮŌ}ËfG{Üļx[Në°EŌ¸j6l(ZšVú éäXË,{žĩ. Û?9œŪŧĐxápŗŒ-pjķÂ3čKžDU¸Ip›đR‹ĸfUV8HCŽāëZ/ ™ĒĄĸ€ô5×\#n[øÔævHøųËkŒ5ÖXCȘ¸† WnüĮnL;ūgĀkšĪ}–9ä!pŠĸ?Åō„[Kž‚˙™ži\ŧ(ėĨq&3 ¤ƒō“™Č˜ņ­‡~ą~)ŧ¨FŽ)% į`Ž×]wŨz—dËÍX–3ū~ŦiڌĸfUŽ3ûZ7LŽāëZO5=WEi>ķDŊ8°âŠ)8Ļn\–Į0â–1ūT|¨0ËáĶæ@ūgndáë“æj+ü¯üœkąĸčOqĀGœƒ<ü×DcŌožųĻ€Vú˛Ë.+/,_Ā V=ŦQ,Vëē.˜(9Ø´ €ōÁø66(\ƒ…qoĩÕVr¨Éˆ¸™xą!>iüęŧØø/9nŖUß3…—W¸9øÃW .Ļ(jVi…ƒ44ā žŽõŌŠڈ Ōųm#1¸Ëāz+üŅ€îˆũšŋ´u9āŋ&O}Äbĸ;¨ g‡āQ—?øo9( Ŗ?Åę$âāƒ+ķÖ[o­ķ—„ŧ4‹žh vqŖ;žøâ ĄWÅ2įŠ¯üQĸ;,M+2ä[.}bíÅ Ķ~u@—0<ÜF6ēŲ/,g{NĀË‹0~%DQŗ*Hį[Õú{ ¸‚¯k=—>kĄNQ@š( ,_JŽ”+ŦđÕ1ĸÅ č†õcŌÂu.]ëĨ!S5´QļŽ‰r+k”¸lŦgņÚöļōrËĻũĪՀnØęY ŽséZ¯z4“l$U Ō\‹…ģÁ&ßpȈW‹?4 ֏yHC ךt­—†LÕĐFę ] JŅ1”NēaK§ëb÷ä:—ŽõŠ-oĨ´¯ ])3UĨrꆭž‰uK×zÕŖ™d#QNĻ?}:ĄtÃ&T GģÎĨk=†VVQ¤ËĒ~í\7lõŦךt­W=šI6édúĶ§j@7lBzô¸ë\ēÖķhheEAēŦę×ÎuÃVĪpK×zÕŖ™d#QNĻ?}:ĄtÃ&T GģÎĨk=†VVQ¤ËĒ~í\7lõŦךt­W=šI6édúĶ§j@7lBzô¸ë\ēÖķhheE@ēŦhį5¯Ë-^ķŠ¨p¸‚¯kŊ WGjâĪ—Ų ŌŽędq‘Vž¯PG՚OKúûjĶ€+øēÖĢ6ũ:éšs]\Žõbt­UUŪkĀuŨģÖķ~Ā%PA:†ĸ]—kŊ]kUՀ÷p]÷Žõŧp‰TŽĄh×ÅåZ/F×ZU5āŊ\×Ŋk=ī\"¤c(ÚuqšÖ‹ŅĩVU x¯×uīZĪû—H@éŠv]\Žõbt­UUŪkĀuŨģÖķ~Ā%PA:†ĸ]—kŊ]kUՀ÷p]÷Žõŧp‰TŽĄh×ÅåZ/F×ZU5āŊ\×Ŋk=ī\"¤c(ÚuqšÖ‹ŅĩVU x¯×uīZĪû—H@éŠv]\Žõbt­UUŪkĀuŨģÖķ~Ā%PA:†ĸ]—kŊ]kUՀ÷p]÷Žõŧp‰TŽĄh×ÅåZ/F×ZU5āŊ\×Ŋk=ī\"¤c(ÚuqšÖ‹ŅĩVU x¯×uīZĪû—H@éŠv]\Žõbt­UUŪkĀuŨģÖķ~Ā%PA:†ĸ]—kŊ]kUՀ÷p]÷Žõŧp‰TŽĄh×ÅåZ/F×ZU5āŊ\×Ŋk=ī\"¤c(ÚuqšÖ‹ŅĩVU x¯×uīZĪû—H@éŠv]\Žõbt­UUŪkĀuŨģÖķ~Ā%PA:†ĸ]—kŊ]kUՀ÷p]÷Žõŧp‰TŽĄh×ÅåZ/F×ZU5āŊ\×Ŋk=ī\"¤c(ÚuqšÖ‹ŅĩVU x¯×uīZĪû—H@éŠv]\Žõbt­UUŪkĀuŨģÖķ~Ā%PA:†ĸ]—kŊ]kUՀ÷p]÷Žõŧp‰TŽĄh×ÅåZ/F×ZU5āŊ\×Ŋk=ī\"¤c(ÚuqšÖ‹ŅĩVU x¯×uīZĪû—H@éŠv]\Žõbt­UUŪkĀuŨģÖķ~Ā%PA:†ĸ]—kŊ]kUՀ÷p]÷Žõŧp‰TŽĄh×ÅåZ/F×ZU5āŊ\×Ŋk=ī\"¤c(ÚuqšÖ‹ŅĩVU x¯×uīZĪû—HĀų2ũüSĸž´Õ@N üķŋK@Ņĸ(§¤}Ū$åTŽö]| ønUų._ņgH{(§X ŌåœíÛø‚žË§K¨ē5  ]Ũķ[Ŗķ}—¯"&Y…,X ŌĢNLKžƒ īōĨ5ڎŸPös^jJ*ßAĐwųjjąÔā`ôæ›on.ēč"ŗÉ&›ĖŖē=÷ÜĶtĐAfˇŨvËĢÖ/ŋüŌœ~úéæã?6oŊõVdũĮ\úkÖŦYŪvÃ*|ûíˇŌĪöÛo_pQvīŪŨ<˙üķRåīŋ˙–ŋįŸ~ųģK—.æ˛Ë.‹ŨīģīžkZh!ĶĄC‡yžMk\Úg Xâļ-ô=žwŪyGdūá‡Ė /lŽ>úhĶšsgyäžûî“ žX`sä‘GJ›Ûnģ­ųúë¯åå7xđāPK6âĖYžČī A—ļĶ¨S¨|AKēĐũ1zôhŗä’KšöíÛËPFeÎ9įÃĪŗ‹î?öGœ9sYŸŠ€4`˛Å[€uėØŅ<ķĖ3—^zŠ€ôG}d~˙ũwŗÎ:ë„Ęôæ›oæi€e•UV€LŪxã yæĄ‡2Ë-ˇœyꊧđXČ¯žúĒ9īŧķÔ4h ?{éĨ—ÄáM÷ųįŸį´¤IŦÉ#FˆKåöÛo ŸļO>ųdSÚĨœŸ~úŠšæškBĮ˜ ŌÔŊ§Ũƍ›>}ú8_}õÕŪŧĖøžņÆÍŌK/mvß}wŗĪ>ûžNrYŌÁņüôĶOĻS§Nō‰Œ;gÚ´i€ķ˛Ë.+ÖøŗĪ>+ ؞{îšōrE˙ŒĢÍĩ¸Ė™K[…‚ KÛiÔ)T> Ōiė;æ Ã7cvŅũá×ūp™3—õ™ H2Xə-›nēŠø™]|Ō<ã˛áŗ!7{öl¤`ŋ—_~š€éŪ{ī-VįÖ[o-ֈ-ų@ËŪļ‰5ũúë¯ ŸtŌIfĖ˜1üÛmˇ9ķĖ3 Ÿ˜a%¤wÜqGąlwŨuWyE>Úžįž{ĕĶĢW/ŗᆠˆÛâ Ōŧ”x9ō"ŗ…y”O8áņį3'čfÅWŦĢËIA:÷,&é4ö’áâ¸ķÎ;åë1יŒîü ]Ęũá2g%i>û{ė1ąÖláķúˆ#ŽHŌ={öĢœ‚ÅL{AKúøãë÷„-ŋüō‹X§¸!Ŧ`@u…Vđ[oŊõōZŌ-Z´0gœq†4ɇu#GŽ”6°Ž~܇z¨´ÛĀ¤×_}ŠÂŦ_VķĐĄCÍŖ>jŪ˙}ËĢŸģ‚ôwÜ!/Æ` /2^¸{p+áîĀšGŽcŽ9Æėˇß~ĸ+éâ€tŌũø˛ĻĮon¸áĶŧysT÷‡ŋû#lÎ\@9ģN*–4Ö&‹'hIo´ŅF˛ˆ’XŌ€CiŌ¤‰[¤qmā°î‡0 0ĀšĪz,â|–4>ėK.šDšÃũ€õ‰ĨuÎŗŸ|ō‰6ūeÜ'Q%ےƏŽEģÃ;D>÷ķĪ? ¨2~\$Ž ÍËäĘ+¯”h€|…1vØaæÁ4*H¤“ėÎVpŗą&û÷ī_ī^÷‡Ÿû#jÎōíÉ\ŋO¤ņķÉĪg>Oĸ °D:@ °!|(Ŧ¸¸;<møži‹g8,yāÄߊë€>ųܡÖ(#ī‰'ž(~l@ƒM)Ÿūø­OšÃ:,X^>|%ÚD™:uĒXĨ‹-ļ˜šå–[ęÃƘ Ōŧ(đ—ß|ķÍâÎĀWŽ‹•˙ŋ˙ũĪŦŊöÚu~įˆĪ›gÛrË-ÍQG%>Iäč‘#8žYŗfI=\>üũë¯ŋš~ũú™Ž]ģʇŠ×_Ŋ|.˙ũ÷f›mļ7 ˙æk…—XŽh\ķæ2g.‹˛Pw‚KÛiÔ)T>ë“N˛?8ģā…‹‘aÃ8ÃƤûÏũgÎ\Ög* Ĩ‰¯Œƒ.>Ö &ČAa]QŅ˜„XĀž^Š=!Í ĮÁÖV /ĀĢ–gņ÷rІ%Ę)8 ŧā‚ ŠKËYĮŽ+Ī6jÔH>ûņcŠHĶŅDY,˛Č"üÁøäũ÷ß_"'‚ÖjØsEw ¸q°đņ•sÄK s¨É‹kvyąI9,âečrK˙|Yd‡˙Ũ/ŊōRÁ/ÍÜčî-čūp×UZ5¤c€4î|ĸ?HØŠÖ˛×^{ITKŽ‚+ë”SNIu螃 īōĨ:y‹iŨF\ŊÅا…—rqh_ÕĢßAĐwųĒwečČÔŨĄkĀ ø‚žËįÅ$ĒEĶ€ē;ŠĻZmØUžƒ īōšęYëUĻŠŌŽ´—iŅlĒ~ÂŨā!d0*s°Đöõšüđ‹!ŸîüëBkĖÕ@Q@:íe4›…N&qĘ(%ƒiԅļ§ĪĻb€`a’ä~*mųt¤9;ÕßV"vĨŊ$q‚l:2Fˆ‰HÚ œ-Hŗ įŋƒëĀ88Č*$!#WšXáČ,\iĨ•„ ‰ōôĶOKüd+ÂpGŠ9V4‰4֒ļyŲôĻ$ŽĐdJųøŽĢɤ?´A0m •O÷GÚ3Q›í Ōŋũö›3í%m¤sr¤CĄIÖ€äŅœÃ@šį0šÍ YĶŌ¤‹CDÖ”žAw4aôĻX?|đ„ØhIW…‚`ēR„ˇVˆ|ē?J5;ÕßOÁ O€+í%1ˇ¤$۔o2õ ˙äFWØÃú˞Ļ(†‡ƒôj{I¤yI„Ņ›V˙R(ß ÁRJ[ˆ|ē?J9CÕŨWÁ Z\i/áĮ€_> ĀN ž WÆŨÖ_\†Üč‘G‘Į˛ÃčMĢ{”wt…€`)%.T>ŨĨœĨęí+HÕF{ ö\ ã÷–t6Hŗ¨q'Õ‡`Ü#ŌaũÁ^,düĀ<Į*NHƒđIcIçĖe.ļídĶ›Vī(˙Č ÁRIž†|ē?J5[Õ×OÁ ÍM".´—đ1´°vĩlŲRn á‘?HŒ‚4›ĐjBÕ ÂîÆũ}Ü8Î5Uė‡õ׎]ģz3Ué”)SäÅØBī Ģ]>æP˛ũ\ôĻÜ˙Į˜9dԒŽŌÁt%ĒßZ!ōéū(æŒÔVÛƒ4Ž´—DSđēPĄā„ĐžÃC,Ū m(— r9,LtЃBģ —2ˇ‰‡õ—=e2$H Ž>šáÎdxŖŖ,i{ÂčM5ēŖxŖ,ž4ķļ\ˆ|ē?J9CÕŨWÁ ]ŨjŅŅ•R…€ ĘWJ h_åԀ‚t9ĩ¯}‹¤u!¨Â5  ­ĢŖėP.û¨k@AÚãÉŠŅ¤keĻuœ…h@Aē­é3Šj@A:UujcUĻé*›ĐJŽ‚t%ΚĘ\* ”¤_ũøgsô5“ĖZ+61ˇļbŨįüų9÷ŽĪÍCcž7dūŊÎJÍåG-gÚļ\Øôēmš<ō›zú ū­§Žh:o¸øLöėʓŋšmz šjĻ|=Gnäî´fSĶī6ĻÁ‚ķ™|}ŪôÄ×2ÖĖcfŅF ˜žĩ6›Ŧ>÷Åw'ūjN8Å|2}ļY˛é‚æėũ–1{lŌŧT뤨ũ(HĪU¯îč=YËû#ņõY¤yÃĪŅž}ûĐÍ<ėĨ™æ†Įŋ6Ģ.ÛĐüōûßõ@úÂ{§›Qc2CĪ\Ų4mŧ€9õæŠæ‹īæ˜ág×ORĄņĪžœmöė7ÁŧØuŠ,ãĻūfv>oŧš˙ŦvfŨ•›˜[žüÚ\˙ØWæĢ;š…˜›ÁhË-O}cn{æķ@¯vŌÎņ7L1 ĖoĖM'Ŧ0ĪĸdĪŽŧKĻ˙Í:65=÷nef˙ņˇŲĩī'f‡õ3'īļ´‰ęsô‡ŗĖ‘&™—.koZ,ļyō­å…6~āš™.æ3ëžđžéąg+sđ6-Ė›~6{]øŠyöÂÕLģÖ ‹  ĨhŧÚAZ÷ĮŋĢH÷Gü•Š%}ûíˇČûÉ +LųÍŦ’”ĢžaŪŸü[=~ūŊŸL‹f ÕYŧOŋķcÆjœjÆ^ŋÆ<Íí~ūsĐVKæ´"û ũÂ|ũãŸæęŒõlK‡ŖĮš›O\ĄÎ"ĩ?ßáœņæđíZ˜}6›kū›žūĄ™|ë:fá…ęz”ėŋņƒ9éÆ)æĶAkI;+ūžš'ķ˛Ų`•&ō˙Ūw}aĻg^8ČÕ'/Œ§ŪūQ^”ßįüm–éúŽyãĒŽfÚ7ŗEo]Ķąn\ŨŽždVjÕP^•^Ē¤učūH˛GKŌVČËørÎĀéˇL5¸4‚`KG_ûŪ\ũČWæé VË9æƒ/ûĖl¸ę"æ¸ÎKÕũ~ˇžL—˙..€,+öŽ~Î*fíˏq1,uĀÛKvuy™ä*šdŸņũâŠĀZĻuÕ$qGô;dYųb ˙#whiØr Õį__uGô]-ž ›ûGĪĖ¸mĻ›×t0ƒ2.ųŌčų/GI˙Œ?Ę|9 >å_ˇQ’…PÎg¤˙ÕžîÜ{˛Ö÷GAî¸,:wî,Ģ‹tjü¯Īŧ[oŊ5Ôõ‘oâw”qE<ÕoUŗø" ÖÎ-{~dÎŲŋĩŲzíĻ91 ~Š IDATeĪ >1;ŽßĖšũŋ€ŧ˙%ŸŠ}|¸yxé ē¤ŊY­Íŋūę6ŋc9oUņ‰ģ‚tvŊ/gūaēôožũéOą†ˇ]g13$ã?_`ūųōöyUæ+ŖßĐé™q/ ~ōÛN[ÉlŪqQsåƒ3Ė{“~?ŧ-×=ú•ÁE‚Õ^éĨAZ÷GîUŠû#ūnõƒū3Jg™fđ+ß~úJbËËãf™c¯›lŪŊvL†Zîrųgr(yęîK×UØéÜņâŌ8tÛú–4n‰Ą=W2˙ÉXŪúĮ’~íĘfåĖc! M›Ÿņ‘9xë%M÷[ HŸ˜q…4i8ŋĐŊ­¸BÂú„y ?{ĶĒųBæõņ?›ũ/ųĖđĀÍ*ĢŦ’WåŊ{÷6˙üķOŪz,Bęæ+&L0wŨuWžjú{Õ@Y5 ûŖŦę75Ō=ö˜ŲgŸ}ĖGa–\rÉŧŗqŪyįå­Ķ§OŸŧužũö[3hĐ sß}÷™]vŲ%o}­ (‡t”CëõûŦyF 0÷Üsyųå—Í ,PôYųë¯ŋĖ&›lböÛo?sōÉ'Ŋ?í@5Dē?’h/ųŗ Ō˙¯Ãã?ŪüđÃæÎ;īLŽÕ<-tíÚÕ4kÖĖ\{íĩEīK;P ¤ĄŨihą°6¤zÛ~ûíÍÆol\\…ŠÛˆ¯zĖ˜1抧ž*´ }N5P čū(‹ÚÕ'TûŒ3Ä Ņˇo_9LLģpHxîšįŠ[eéĨ—NģymO5PT čū(ĒzCWK:K5/Ŋô’ŲrË-H7ÜpÃÔfåĩ×^“Ā¨QŖĖĻ›nšZģڐj ”ĐũQJmĪíKA:‡ÎoŊõVsņÅ›W^yÅ4oŪ<ņŦĖœ9SÜ(gžyĻ9ôĐCˇ§ ¨ĘŠŨĨÕž‚tˆž{õęeƎk}ôŅÄ3BˆŨZk­e.¸ā‚ÄmiĒ4 ûŖtŗ  Ąë}÷Ũ×,ŗĖ2æĘ+¯,xFN9å3}útsīŊ÷܆>¨đQē?J3+ Ōzūũ÷ßÅMAĸËqĮ{FŽģî:3xđ`ņo7lØ0öķú€jĀg čū(Íė(HįŅķ| @=lØ0ŗŨvÛ9ĪĘȑ#Í^{í%~íŽ;:?§U•¤Ũş-i?øāƒæ˜cŽĀ]qÅķ>1qâDön¸ÁėžûîyëkÕ@%k@÷GqgOAÚQŋũû÷7<ōˆ=ztŪ'6Ûl3ĶĨKĶŖGŧuĩ‚j 4 ûŖxŗ¨ CˇŨēu3üņ‡2dHčS‡v˜iĐ šéĻ›b´ŦUU•¯ŨřCé˜zŨjĢ­Ė6ÛlcÎ:ëŦyžŧđ Í3ĪøĀ<øāƒā}úô‘=Ŧin0yã7ęŪ}÷Ũ†ÃGū˛Ũ„Õ͞=Ûėžûîb°īąĮķč/HŋúęĢĻuëÖzį Ģ-DßAĐwų<žZ- ¤ŌXĖ¸+ld}XÉÂõÎ9įå\Püzöė)÷f‡āq€Čáâ•W^Yoh´‡Õ ØōīîŨģ‹›ÛÃCŦė…ZH\.„㚧Ņ)Ŧ’"7á;ú._‘§G›/ŗ é2ËŦŨW™|AßåĢ˛å ÃÉŌ€‚´.‰˛kĀwô]ž˛O  PT (HUŊÚ¸‹|AßåsŅąÖŠ\ (HWîÜUäžƒ īōUÍBЁäԀ‚´.Œ˛kĀwô]ž˛O  PT (HUŊÚ¸‹|AßåsŅąÖŠ\ ¤ ŌpoĀTGx]vŲsĪ=…ŸcˇŨvËĢ­/ŋüŌ{ũņĮ›ˇŪz+˛>ü ô×ŦYŗŧí†U S’ĐŊ:ÜFԃ„ B0EpŠŲĨK—.’­ˇDÉüíˇßŠŪļß~û¸ÍÖĢOâQ.˛Ģ\Æ™ŗėį}Aų|Ļō:t¨šųæ› $fË.ģŦųß˙ūg6ŪxãDkC.RiȐāÖđ é‰'šnŨēIŒ3 ‘¤ˇÚj+ĄMmÛļmÁZ#ųf­ĩÖ2ûîģoÁm¸>w6ąâÄ{')Q2?üđÃæ…^0W\qEÁ]|õÕW’0DJžwÎĒ¤ƒcō‰Ę÷•W^‘Œ^’ÅV]uUs˙ũ÷ ŋ$ iņ_Š‚tВ6l˜¤؛mļ™đz@Š% uéŒ3Ėu×]7†°Čxfōäɒ´Ō$ŋĀŽ×ĻMIŒŲyįebÕkV_}uķŨwßI8E „Zyå•ÍÅ_l†nŽēę*yš`Õ~ÁrÚi§ ¨NŸ>Ũ|ķÍ7æ?ūĖĘå—_Ūü÷ŋ˙5Œ“ö)¤Ģ“<ķÚk¯I›šÆ˜ Ō$ā`I?÷Üs’„ŗÚjĢ Í+t¯…ČLßĮ{ŦųķĪ?Ĩ-ƒ4}dƒbc}ôŅĻsįÎ"ķ}÷Ũ'™žč‹ŒPxQĐÃļÛnkžūúkyų <82c3ΜåÚ.–j9ˇR\ųrtš¨|Įoxá˛7)ŋũö›éØąŖ°Q˛o4ŲĢœ+Ë­īĸ€4¸Å[€ą žyæûi2Ií†S:Ŧŧųæ›yA`!€LČjØzč!5éSO=%€Į‚´  Ō7ŪxŖÜ(CĒ9ŧ!¸crYŌ;Ö$éč¸Tnŋũvąđi6>Ā*V }|úé§BÕ6ÆlĻ.V/íÂ{Bú<ā|õÕW xôqeƂūüķĪŒū駟L§N ü*¸?ĻM›& 8ķéËü(lXĀ&B^ŽčŸąc‰š—9ĢEö…ĘŨãd]°'ŲüJåëēēËW¯( Č`%dļlēéĻâgvņIķŒË†ĪiŽ)Ø/)â€'nø6ÜpCD[ō4–Ŋmk šĪE€ UøHHgßnģíĖ™gžipÁ„•lŪqĮÅ˛åf Ĩ+mßsĪ=Éiž4x9ō"ŗ…y”ų ƟĪœ@'ģâŠ+ÖÕaķ*HĪUGRKÚ*_Ö,†{3Ę@*iĪaFĖ|™_dÖŅ?‘âS0ŗ%—Âē;¨ Q֚-|^C7š¤áüĀ `1Ķ^ВæÂ.Ā=a „O€7Ÿu <účŖBøÄK k:HˇhŅœqÆŌ$_0Ž9RhW3.Ü\|`Ûģs)4¤aņŖ~Đ?ˆõ ¸"[!2AfA^0ŒÁ^dŧđ‹ãVâSk9Ž9æŗß~û‰…Ĩ Hû@åkŨ|%ņuĢĨr4ālIcĩr0hKpĶ۟YÆÚŧá†ęYŌm´‘+%iĀ €ĄĀ”°A×/ë~›.( h wC>&Ã^‹ûëKW€øÉ'Ÿ`ã_æƙ¨’ ŌøŅąhwØa‡ČįâČi^&VÁJ˜¯0&ČĒ`.ďŠ H—›Ę—5_eœ-pɆ–ĘŌ€3Hģ ˂4ūa>ų9ÄÃįI´–(@Hs˜@r/aXqqwÎÆáVmņ ‡‰<đ€ø[qĐ'ŸûVrįĸõ;0@\ X„™mšå–´GØøôĮˇn}Ō0bŲōōá+ßeęÔŠb•.ļØbr“š=D c6HĶ?L~„Há‚ÁŽ‹[†P)™9ĖD>€9xņpPËęŦYŗdl¸|øû×_•ĶũŽ]ģĘgŗAk”—Cļl6ņ;+7ŋG6d†_ĻMK`Cl1āĘßqAB$RŨ)l@øI°˜3æKbŊõÖĢ'éōXéŧ€Ō¤'Uv׀ĩ¤Ķ ōĨW\Ė's›Ë•¨TžųAē”Tž.s–¤]ŠJą"Hė¯\ <œ Rbr1VOų ĢĪ¤q›Ø(°pąlųŧŽžhLąÖšRĄėEœí“ĘƧ#Ö5/[ė…ŌP ZžjËŦ‡5œMņåîāåÂ×;%(#/0ÆÅŲRëÖ­ëdå×í§IOęQá5ĢÁ’vŨXŧIŠ|YĐīBâĪ ķJQ*_7úárPų†Í™Ëūqvw¸R•â‡nØ°Ą‹ãļĀZÍļ¤Ã@š¨ H~Ŧ%ÍŨyøR]AŸ+ĀIÁ7 ÁoJ ėw¸8yäáLX〴Ĩ6 ~V˜¸x‘„4–t6Å+rˆû„‚…‹ÛYĶéüQ‘pÕä;ŦMJOę˛ČōÕŠvŨŦŸ$Tž0?˛npéõīßŋCĄRųēŅ—šĘ7jÎōí ~ī Ō.F8íëä+h}ŌX ˆYŽĸU¤o˜í°Bą8yY@ďĪŽÂ'!/ŦsÚá1›ę4ʒæ8:y|ŌøŅa’ãoĸPÂ@:ו\øē§L™"ō°š‘–Ŋ8 ÍÄiZƒô¤Œ—›Ų×Xc ųš`œāâŠá™4éI]ÖEž:ÕŌųÆČīĶ ōå •đ;Î,ėķa}+•ī\úárSųƙŗ\sY4 ‰Æ€*ëØFwD!Ÿäø~!+‚÷øcųtŠî`ˆDJItžhkIÂŖL”–mđR€ Õ)ŸĄaV>ĘÃ˙ĮK`å`_:ž_{ĨV.wG.|”Ī-2ÜÃÁ)<ÖŽ–4/ĸ ėȤ'EX˙øâŲŦ6_(|§AOęJŽuj ¤“PųâÆâ 3ûö"“r}5)•īV=VN*߸s–ŊoRi×MYŒz€4~oŧqÎæVÜX”–˜ŋrh›ņ5P+ _3ū?‘%’(•o˛yŦ*Æ߇•’ĢpĀBô–°ŋ4  í×|đ&H+•omæŽ[õ Í!ņŅ-[ļ”“đ`ÄCrõi ih@A: -–§(V*_#IniPų:Ũ^ž% Ŋւ¤ka–uŒ…j j,éB Ī•_ Ō埕Ā_ (Hû;75#™‚tÍLĩ´ ” ¤ãЗ ŲR’Ą€Á[ž“´—ũ,Ų•„ųQˆ$!†Õ’+‘Jž÷Ū{Įî.Šf2ņÄĨl$ņ…ŒËŽ;JŒwZĨV@ZŠ|ĶZ1šÛŠf*ߒû¤1Ė]čKáڀˆ„–$…˜f2sÅ-'i7×ŗ„’‰hšK m?Šf2ņÄĄl$‰gȐ!ÂŪa“‚tüY iĨō¯Ë\OT3•oj M˛†–Ú“Ė6g™č8íeŖ0déŲPHZ!9XŌÂĄū$zđāÁ’5)lu6ëŽ$Å_Ü@\Ãiķ¤I“$Ûb§‡nŽēę*IQīŌĨ‹ô,…Đœō%–\“ ŌQ´ĄīŧķŽđ0Ėš5KtÅi0ōŠHĶÎ;ī\'2ąŪŲã!ŖrĐ AŌē@įă“ČÂ<‰uøŨœ9s$Y(HoĩUHŠAŸô=nÜ8ép%ŌJåĢTžŽK(UwGHÕ…)| /ŧpŊTq@ėƒ>€Ę.A’#~GļäI¤ZBfĀÎŧoø—ųĻ/8Ŗ-Ōž{æfÅ*„æ*Đ0ęÔlŖ ŅC$Ö=î^\dd’9HÆdĄ/[/–Ā‰g!Œ‚"ž’å–[N˜˙xq‘ŠI:;–¤œ7hĐ@~ŅũdĶ›ē,,hiMÍ[Į‚´Rų*•oœT2Æ"&Ū2šú¤ŗAŠæ–’:SĀKÔj´Œ| IDAT,učCyaXF:ĩlBsE\é(ÚP.ĀbƗ ËW!Ųtßlō§(æEĀs­’-¤š“UÍ ë=-šä’uu¤ŨļJZ>s֙Rų.%JW*_ˇĩG-gvĄbŒ˛¤i\ÖZ.¤ĄūČ°Æmũ KĢyčĐĄBЏ@…ÉĪķt\šĶ(A:m(ÔŽ0āÁÍ /,^62Ž WKšË°Žáą…vo\2ŧč8|…ß*^bp}(HģmvŲ€´Rų6Ĩ+•¯ÛڋŌ.TŒ€F8 üļ5( )‡a”BAß,äD–Dž,Ŋ(.’| —æÔ¤ãІrĨXˇnŨäƘė[cĸ,i\¸w‚ÄQšäcëk¯Ŋ֐ŽĢ íļQ\@ÚeŌJå[¤•Ę7˙tļ¤ķ7e >_Üâ'îÜšŗíįiqW@€”]ü­üMƒŸ•,ÜøXąąy  g#8  ÷+ō ū_nRÉE]XÍ)r†Q§fû¤ÃhCąrY¤\…(ģ IcGŪ )Žī™û )Áņ‡ģ„߲Ë.+7ÚĐ&”¨öĢĢš/Ŧi\,´•Moę2Įę“vŅRî:Jå;īĨJå›=Ĩ Ō€%`€ß“‹`—Xb ĩ#ē ʒ‹î@|>@ˇ1ēøZĄ3Ôč Îhüš¸ø7ņ’Ÿöô paABĀ„%ŸM]X(ÍiÜčŽlÚPôÃ!×eÉÂdi‚EĖ!_f’1#`‚ãáДčŽ#jƒgņÛs`Ęက2Ú¸DøÂĄ" ‚ôĻŲô—ÁåÃá'Ņ7čĢļ¸Š†Xî¤ÅÅRMÚG’įĶ’/ŌJåģėkĨōÍŋ2SéüŨųYCiNË;/i`ąFáģ|.ãÖ5îĸ%?ë(Hgæ%,ÉĀN™Ōœwņú‚žËį2;ēÆ]´äg鐎šSüööžÅ\˔ßį^Ŧâ;ú.ŸËŧ„t­ŦqųZGAÚיŠ!š|AßåĢĄĨR“CUŽÉi÷kĐžƒ īōų5›*MÚPN[ŖÚ^l ø‚žË[áú@Ei "A:ß!Hšg Šb´Ü˛ųØŋī Xų\“Ŋ˜OĨōm6Ī˛.5•/Ų„ÎĻJx*g86”6éž+HOw›Dđ™3g ›É-ļĀö&ƒÉü)w‰ĸ-ˇl>ö_ŒŖ‡R˧Tžnŗã•/ņŪ$ŸA\F < }đ Á4™´¤ Ō›mļ™géHÉ6 Ŗe!B†O2Įb‹-&IAŠ YĪ‡` ‰#dÕ1pkIÃøÆ˙‰>¸ūúëÍOĨōĢųj¤ōåˇ—„f(’šT IZRél:Ō(ZQ\ äąĮ†Á >d˛É2tévíڙ6ÚȜtŌIÂįÁ`ĸhAyËŊöÚkBéIJ9yđ)“­×°aCál>æ˜c$=Jvdæ†>3!|âĶjŋũö3īŊ÷ž|ꉑ¨HázŖ:M:Š•ö|Z XŦq§%ŸRųæéjĸōe„` ô GydC’ĩ™*HgĶ‘FŅŠbÁÂôFÚ6Kn8 Bæ„õUDą´)-Ö3œØä¸Ŋ~(ŲąØI冚ŽBę;Ÿ:đˆ`9A:ŒŠ4ɤUÛŗi`ąô’–|ųX"•Ęw.)F_Ģ•HåË8_Đx ā JŖ8ƒ´ c6i­(ĖmÜô??YHãrhŌ¤‰¸8`ĮkŨēunxĢ5oŪ\,v@îŠ3Î8C~2áÄĀ§¨X¸đ^DÉĄŅĀÅåA r?Ķ^¤Ã¨Hƒūô4&˛’ÛH ‹Ĩų\öG>V*ßš3Üŗ•Då ŋ_âíÛˇ7jZgrÎ íBŘMĸE+Š˙Ģ°Ļ°ČqqŌÜųī4™pö +b  OÚZŌpGãžČG ¤Ŗdđ]A:A*R>ņ´ĖՀ –SW.ōšėĨō;‹AŸtž=ļ|Ŗō%˜á€ŠaÎĻŌ,Î íŌi6HGŅŠō9Ÿ1TšXˇ°įŨyįŌüĘQĀ Ə Ũ'÷úE42†Ņ‚âŽŌQ˛G4†–b”7j)r†Qēčēšę¸€`9Į›–|Jå;/HGíŲJĸō%`€Kš­g ÍõZTÆ?F+Ę' q„PŽâĒ€ėNh@š›T°Tš¨ļUĢVĻC‡E‡2ŽîČļ¤QŒŊô5-hŽ’= ¤š07H1Š:ŒŠTŖ;ĒĮ’vŲ”J囤Ŗöl%Pų~˙ũ÷âŅ &°á‘GqY‘uRé$Ōvg#i-ĩŖ´,ÕbiĖwųŠ5nÛŽŌœ[ÃŅí+H—W˙Ú{Fžƒ īō{åËđU*ßâ΀‚tqõĢ­;hĀwô]>'ĒRë4§>Pų’GšéŸDŠĢ Հī čģ|…ę]ŸĢ 8[Ōŗįü‘xD 7ø—k#qcÚ@ÕhĀwô]žĒY:œPÖ…Qv ø‚žËWö TŠĒ’ƒô‹/ž(éØË/ŋŧđ]fwß}÷™8ԌEÕHĻņ .ėßŖĄ–Ú÷}—/{­ä;č+÷ÚR*ßx3*HÝņîģīd88:å”wŧ„Û]tŅE’Yȟf͚y ŌAZTßŧŠ÷§ļī X ų”Ęwˆ!§@K~ ¤ Ō~øĄé}Ūšæūaäėyđ [ĖÍ7ß,ÄJd㑘˒Žĸ ũ駟Ė:ëŦ#´¤$ŗ ÄÛ¤“ŸsÎ9’­8iŌ$ááāEųiŲđv@Š 1Ķ°aÃ$6›BZ:É%d7ō7/‘M6ŲdĢš„˛$ ĀīØąŖP.ąÄõä˜:uĒŲ~û텸‰`üožųFÚ"™†äú‚Õ8Cāü°œ"ų§­ējĶÔPZō)•¯Rų˛.K ŌXŌ†XĐÅüwM(™ŠO?ũ´-ÜĐÁžđTãJyîšį„đ„˙“ž °Û H8¨á~%ë‘BĘ9Ą6ôåî ƒJU¸¨8âą,˙ˆ•åķĪ?7[lą…€n÷îŨͯŋū*éėüŸ—˙&û žß~ûÍüį?˙Ö/§ją¤‚ÅŌ]Zō)•¯Rų˛FSéîŨģ™‡2 Ãí(€ LyáÅŅÂca‹+HGŅ„F1KY†ĨÆĸíÔŠ“ąL°äaŊđÁæĨ—^öģ1cƘ š,dwG4V/€O—š@~Ŧå`ą ũĘ+¯˜Ĩ–šKŋČˉ/}īŊ÷–ū ĻĖN1/d+ũ™´@°XzHK>ĨōU*ßBÖh* m;Îįîpé(šĐ¨”q ŌãƍKׂĨŊáÅē4,¯` ‡˜ÜŦ`ķˆiø¯ą’å%ô bã"ÉŌV~wÉ%—ˆģåęĢ¯ĪāÁƒÜą´q…pŖM­–´@°Xús‘Ī…ĒTŠ|ŸˇžRųÆ[Š^‚tMhÔđâ€4íĀ‡uŊ +ČuYXģ”(†<‰[(đIcIs]V.=z´\@ÁŊ˃ƒS8rá¨q™ņb¯Ũ‰7}ÕQÛË9Rų\¨J•Ęw^ÎĢRųæ^å^‚tM(>inb“ÖēUėĐâ‚4î\+\ßuË-ˇÔ"F4t„úÁ…Íõ8Xá|ŋķÎ;âęA.kÁsĶ ÷+õ˛ĶN; @ã栜~úéˇ ÷$Ë FåęÛË%ũĻ%ŸRų>'w*•oŧ՜*HįëÚÕŨEŨ¤‘’n@Ô^beI㖘ą šd€å ’BÜ \H{Ũu×Ձ4?ƒ#ۂ4ŽüŌ¸¤‰Ą>ˇÉÔrI ‹ĨôäËiĨō=Yž4•Ę7zå–¤‹ĩ‘ m—(5×\Ķ~øá…61Īs֒ž0aBč:„îŗĪ>†ÃÅZŋJ+-LmŗōA>Ĩō-ÖėúßnMƒôë¯ŋ.ˇŒsņĀĸ‹.šÚlåi.Žå&îtäīZ/>€`Ôø Ÿ‚tíiĸ)ˆ‹&ꂘë4KHÂá#Ž\%ĩnE§éķMsƒm)HKŗÚŽ‹œAÚĨ1­Ŗ(D>€ ī–t!zÕgĒCÎ ­TĨÕ1á>ŽBAÚĮYQ™|Ņ€‚´/3QÃr(H×đäëĐķj *@ZŲéōÎŗפëOä]]ģv5?ūøŖyA#á<1dģFE •jĸÃxwzôčaZĩj%į.ZŌŅ@Ē íBUšŽØõ[Q.†VK×f-ôŦYŗ2tžīFVŗßŋķėŗĪJ"ĨËŧ}é 専túû&Uváî ĒáC‘Th@ŊE‹utŸ„‘š iļd&]zéĨuT wŨu—0БVMl3‘͛7¯—Æ=vėX!.‚•ˇžŖ=R¯!8Z{íĩE“÷ßŋ$“Ž ąDL{ėą‡ü!ƒö˛ lØķ.ŧđBá…fávéŌE6˛ŗø°āĩÎ.pYũõ×2ŦĻƒ:Hū&•=j p€Ų`ƒ dŒ'Np>|ø<œŲÕ¸8¤ëĪj63ĸũR ē;0T ]ŖÖ‚ÅąqãÆ Ũ{ĄaÆî(~;tčJĢûŪ{ī…‚4 '„bėˇlĻHF  CŲ;bÄÉÚŊũöÛňĄn•p51ĨŌq¨JYpŧA! đdbX(¤Ę˛p(XäOCDīŋīׯŸüä˙X´É„ŗ đ…ÁĮAát€Ü&Ē°°CŦÕGyD€ÔĻƒS—ˇ=/ *@Ī ÃŦ^ЛF.@›ę ' ĖŒ•/ Ę÷ß/VF5kĩ,ÖZičmyŲŗfY“€…/&č‚Ť1< ]Ŗ€4Ü3ŦG ûë0 †{/Ō0ZŨ(Ÿ4{‰Ë0(CL$Že_ŽÁ>!Ocˆbi¨‹!F%\-ëŪuŠ€´íĖÅŨ ´üM Ō}ōi5`ĀážĀ’fŌøŦ2dˆdæaUŗ ˛ maYvŧ­;ƒŸąXđ2P€tlõ°ĒąÚmŠ[ˇnbų˛p,`ō{\1€ŋ]Øa ^oŊõĖõ×_/,w^>_|ą€tØøĻíņãĮËæŠĩR mį4Ÿģƒz. Íē-tŌö& Ā Qk€đâāÆ"\|a´ēQ ā÷ėŲS†ĖžÃ"9r¤ėå`Ą_ö˜•#X+<ŒJ¸÷Į|™Ag^~˙DŽŨ%NÚ¤a†ã‹Â'ž0—ÂFm$äûl\,]ŪĒ€47¨`e㎠pÛ )v2yëcãcæDë™EŒŸ?6Ö9î,i@ˇ ?gC؂õ€ËƒēžI—\Ɩ[n)ŽČt(ô_,|¸:øôŗ|ŲX]üŧüo ŌõW— Hs¨XčŌa´ēQ Ąa÷){fH{G6HcŒqļDąVˇ­F%ėēĢĨ^Y,i€Ÿ4Ÿō€ĢĨûÜtĶMå Ėđjp/ã ¸ōÖˇ‡wy0ēk€Ž\ŠpÁbƒ…āåfîCHņ—ágÆ@cåã˛lIÛæo^üœ°(, ,^,k|ŲŲ…>ąčų"āŗŸ8§ķ€tÔ [⠀_ŧ°†*×mU{QŽŌŦÉB×hF˛\´ēQ Í:gŸr+û“—ûŒ}Á9 Ž@ ë“æg'ÁēôF%\íû!{|Š‚t>åAU  îģīžr˜—M÷ (î ,Yno€™´ąBoģí6ąĒámæ 7‚AšH–O5,cŦvŲĢ‚…AaŅā/Ļ=üƒ¸:đAÃéŋܞœ­čãŽ`ąōb‰Šî€¸— @ËˈÁ­ˇŪZįßõ9tÄĘįÅÂÁ ÷4ÖBŠ%v™OKšuXč ŌšhuÃ@šĩO ŧ8¸93ÂRfŋfĶ ŗšēŽčŽėēVOš¨„]tXMuĘŌ€+Ÿ@å.€wđ`Ž‹BJ{=VĄōÛÅõ‹kAKn (HûŊ2ŠEĢëS] *aŋĩ=¯teiüUœø–ŗ`ã^Á¯‡%Žĩ€Õ>jÔ¨yNĄãȉpŦg”K›Ä{ãÆĐĸ ]ik ˜´ēų@ēXT•65 ŌL<Ō„1ŲÎY4øŖ“|…øžY`XéDyāfáŗO‹‚t%­bĶęFt1Š„+iÕ¤+m`*oåh@Ũ•3W*ié5ā ŌQ!xjQ ĒéB5§ĪՂ¤ka–=Ŗ‚´į¤â•UUŌųî,ĩ†ƒžļ`x`ąØúđĢ“FnŌožž¤ũ)HĪ%ŪĘNĪĨWÂVßzë-á”)Gņm¯•CĨî3u&Y„`u.UņmáAÍŌLÂQ  {EÔ:H“•ã!I ų š葍WęBh)…5¤L(ĩĩÖ_ę w…lČĨL˛ˆ° s×~Âņ°'‚ßķ;ŌĸYÛmˇ°ā!(Y~ÄX•Q I+Ô%>ČŗKöwÜ!ŧd,†ŅŸŌ>ŲˆČA~÷îŨ… ‰ė@h" Ĩ‚Œ˛¤ą–ˆfcvėØąMkŽą‘6›]Hu'¤,ʕW^YzH¸Á’&už„ sĐáF‘ĒK¤ ņÚüŒęnģí&ė|´E ?cHÂ!ˆĢ%=Ÿč7i!s–L[Gä‘GJf)ē†9–;ČÄč+×`_{ėą˛ö™ChØpŪØž€#†äĢ\ũ‘H–]Čä…WÃrwhFļ-T Æ –:Ŧ0†Č ´{ :UÖqÔČĖZd ÃŨ•đ•Tĩö|ÉAÚZŊĒŠ„Ŧą ø?‰¤QÛ¸b>átR˛YŒW]u•yûíˇ……J*5I¤Įڅ8ŧPÚTĶ0úS˛YT–ëƒD€‹†ŦĮBŠ Ŗ@z™e–>=TŦlL@4ll¤ĸgtÂf##}´lt@FA,-¤ŠîdM’&ģ$UöÖ Ŧą)SĻHßôC–': $‘(øUxŠ(H'itJ,œ°)BK€ņqíĩ× ¯:†Kšĩ˜3æœŋį0fO„õĮÚ –(foRĘúa?ŋZÉ1€oŖ&G€=Įē‰ĸķ­5M:ŪT@úōËú>Ã(‚Ĩ$$Y„7r°ØIfA’ÛOÁŌÅbŦ`Ââߤ‹R"Āa§ÃĐ`-P`Ģ̀Ÿ†ë‚´TR¯#Jũ)DHôCĘ9NRÂi˜ķ Ĩ‚Œi6´•ČĨ°L°äÃÆiT°Xš õ#éęčæ…GJŧĨ‚å‹„Øo6 /+ƄĨD7›ąŗ™ų Î \&đ/ĀļĻ ¤˙ūûoųbä… ũ§]ŸĖë-ŌQ{Ā¤y‘‡õ— Q MŠ:ëĘēb‚ !D6-ûˆ¯4K1œ”ôųúH¤ƒ!xųÜv’Į'Ö$…Ü~8/x[cŠ`ͧ đĐ6Vā,ļM@ŸĪs.”(úS>W]uÕēĪ<>ßčƒ JŌ|Ú´ķ MkØØødĩ,apNcÃ3ŧįŽ/ôÅfĸ:ZKp&đéĘ˓Íđōō’˜šd€—.[ø=. tŠ ¤Yg|ŲŲÂá^v4ĖŪŌÃY~–lŽÚŽ ÍË:Ŧŋ¸ ͗+üë”ėķ #Ŧm\pėK^ú–ÉQÁ6 ” ¤Gmøô§āTČĶD …GƒÂ§–4 uĀÂÆĄ`Aãß#uāˆ¸ö œÎ”(úSüēMš4‘O| TĢÔgĶp WĄTQ ČZųųúĀ’Fî°ąAU S°–ą˛˛IÔņ%bÅŌ¸Œ`´—d/Ŧ|†ø÷Ņ˙įÂĶu×]Wĸ,}đ9éü .9Pŗ%ßÁ.ŽÃ;LžVø:˛–4†BÔ‚4 ĖWdFö 7L’öâ +O°?|ĮÁÂŲn7ö™ØGŦËl˛§°CzÖ(gŦ+\nZŌĶ@Ų@ˇ‹ƒE‰eČCËÉ"3fŒ†ņy€ãÜX rũ_{WUu‡?ö˜ŲÂ(°ĩ=”R×FD´¤ļ…BŠTĒ@‘ƒPQD”Ã"ĢE@Yj@ŽŌãR(ZzŒ BB0 aß! Ŋß&™Iōfždfō^Ūīū•“šķîī~÷Ūīūîwķ{Iāü?'5%’7‰„Ū'Ŋ`^¸FwP[æ…/9YôŌŸr`ģ$1ž†ž““›‡S›ŊĶTz$M˜ÄLēlšV^,UÔˇŠ^ŸE—v“h)×Pßäß\lŒîā=#`¸āč5ķ‚Ę™ûš$Oā&Dœ…Ūq&™[67-J$BŌ•'iâN‚et ˙抈NOŠŒŦ‰‰‰Ņîs¸s\ôÖ=^: t*(‹0„ķœķ—é@yäܧãŖ×^Ų5JBæé‹dĀuGĮ†›ßĻâɓĻLÆšÅ5ÍyĘĩĘuíšīÄ/ĨrTIģjĒ•ëBåžíšV”aP|3 Iž‹@J`°{œt`P–VŦŠ€­Iš)/éŲō˛ƒĮDÆH3ęBJā’ÖŌ’õ°5Is¸xãŅŽ…ņĢ<Ž•}aĻõ†ÕZ I[kŧÄÚĀ"`˜¤k–´f'„¤í4ÚŌ׊€ôELęû!iŸC*ŦFIWŖÁ´jW„¤­:rbw IK*Æ@ §5Û’.?nYnâOËOāČ÷y8ūnˇ ëūuË>:ƒ‹š… Æėá­Đ?Ē!v|}Ŗ–¤šÕ-T‰ëF>Šy#[—kä`ÚuLZ})§ōŌ¨füŽ÷mZá$ÚüŸ xsK6.ä"ĸE,Ķ‘m*Ž€ōdģëÃķ Š1umöqüXĢc‹`Ė˙c8Z…8RBxjķËCWņj\roŖ†J¸9z@žy$TûmüķʓØs$ĩkŨ…a†`搖*ÄĐēëƒĻĢĐÍĘgųŌƒ@R1ZsrĘj;‘4ãŲųc”~ũúéÂ{üÔ ™—Š§~Úk!ģ’ô—‡r0ō­4|üJ't ¯‹¸Ũį1m]&’×tGp{Æě…*&~ĘQ,Ũ÷vĒīÖ^~Á-ôw“cš#öáP|“œ‹_Ī=Ž]s„ˆ–îųbÕFņøŦcøpZzwŦ5ŸžÅÛ;Î ~i$‚jē3Ÿ'ÛËvøÍ-§Ÿr ›ĻtÔČtƆL$eäaëôxj3˙f1ēŒNĀS;ĸo׆āĻĐwRĸfû9bQę×ĀÂQá¸r­ƒf'ㅁÍ0üĄ@MiŸļãSOZR1útllķ0;‘4Ā_ęq­č’I+5;Ãæ§ē‘4É+ûbęŲHûz^~1Z?€Ë#Ņ&Ė=ģŨüŋgku)˛*[žH¸Ēŧčīņí˛Č’ž[šŽÍë`ĘĶÍŨĒĪŲ”…ŗW ąTyĪÎŌmLVo§‘¤kņd; ŗŨ3ņÕÂŽčŌē.ū°0=Û×ĮÄ_ŨŖ=bį+¯<āÄwēÃS›MÔƒSâôÆŪ%M?<- ąŠ„ŸūYS´qPÃŖÅŨ|ÃŽķØē÷">šÕɒkĘg$-Š-9ūĻ0ZHēâaøoRn9’.[sûūKxũƒ,|Ŋ8Ōí8Orî˙R"ö-ęĻIeË;Ÿœ‰š^ŦŗÔĒMāŊ‰Ž ’Îģ ÷wn€įŸpd­dyJy§ƒ~ŌD“R**Ų^Pt ŸÄ_Æ=ĄaŨšXŋķ<6~ĄTyÎõ”į;ųŨ ĐK^ņ|[xj“ōÅŊcö°VxR6R˛n zæ1|6§3xBˆž™„Œ ŊJĖÚ4Wy׊8ļÚ]62Åä7`„ĪHZR1@[ĒTˆ€Hšéi™6–Ų™ŒČ™ČŠiÃe`IDATBõ^›å¤÷)ōĄŊ~Rô‰p—3Ļ­ËĐäYŋ/Ílč ū[ÛNãģôëX÷b)!¯øøŒĻãRFp-1¯§āą>ņė/K yČŧãšũ‚ qģ~Į›íŦ[T| ÃÕIaˇÚ,hkŗ&AØ>ŗš7 ‚ˇ6)û W›GŨÚwáRn‘Ļ9Ô ņÉ×0TŲ–âĸãS{¨äšĖŋ•ˇ•–ĸa’–TŒ’ŠŅ_Û$íÄΈÜáŦë‰čâv_ĀåųŽU$Ûŗ}=ˇĄĄņėwØģ +Ú6s—@œW)OúĶo¯ā3J“-ÍŨ| i§ķąfB;ˇįQ–čĄd‰oËüpĀËĮđ›~M1âöe]Ųša„¤'Ž:‰œŧb,S2JÚ5@ī~ũÎsšŨÔŨõÚėß]åõQōFÜKpŸōđyj<'“7×tú”fSŠBBŊ,IĢēûk ûõš†IZR1J*FÍD!銑Õ#ē5ŸĶČlË_"4¯ŗl!ųž—‰ũJęĐ+$ŽąËĶqde÷™„úw¯õ4˛s-$ītEŪĢo“7=āÎŖ4’¤ RQ1BŌÔĩĢKÍGz9rŋ_Ŋ^¤éÉWDáũŸ×m39ķÖ)Šd׎×ĀąL_ŸËĘŖæe!ŸAŨ›ú:Ë_˙yŸ˙ī ļÍtĪūį¯ųėëį&io K*FoÉįzI'éÃ'ķ3'{æwAXãōÍ'14-ûâMŦįîŖ")ŽŠ5†ëQî3ū0Æ h†QŅĄø*1GéĀiØŖŧØđĐÚp­›Ŧ4_^ĖmV2Ŗ'ūĮđ8ęŨ cčÃō:ˇ* ŅĶ¤I–ũ#ĸŌ¤ûÆqÜŖ6^lÖTâéā•÷3qTyŧôčõÚ<˜z O*M|įÜ.Z›$wFŸ ũEÆ>Ļ$ t3ëāØļ8ĩPĶĪ§ĢđđBŗ¯LŸ‘´¤b4ûP›×>;‘´‘Q˜÷a6oĪVqąÃ肃an oGaūÖlŧ÷ų9ŠčײzB{ ŧ¯ąö¯1ęhOgü´kĄNuĄ@iØš„?AES$eæĄ…"Ë×b[#úĮ¯ļlŨmû.aöFFy GģzZ¤GGEĖ,?Ÿœ¨…ˇŠƒ'Ûk×ĒáŨA™‚qŌ‡N\{C›_‹mUĸ¯{j“Ą‡+vœÕ.YŪß/mŠ‘ũeE2aå Ĩuįhšõs…•;ŗÔņI›ĨCb‡õ’ļŪ˜‰ÅC@H:pXKK:IËÔô’–ŲQåIWųˆ&F@HÚăcĶ„¤í2ŌŌĪ;A@HúNP“īø!iŸÂ)ĢfIWŗĩbw„¤­8jbs ¨2’–TŒ’ŠŅ9Ʌ¤ĩÜĨ+"ās’–TŒĨĶ@R1[BŌÆp’ZöDĀį$m$7¤b”TŒŽËMHڞä#Ŋ6†@•´Ķ4#ŋī—TŒöHÅčĪ—N[ úĩĖž‰Tļō}s#ā3’–TŒåZR1›üf'AŗÛg eŠeU|FŌNŒČF IÛyôMŌwŗ“ Ųí3É0Š~B@HÚOĀĘc#`v4ģ}Ƒ–šVD „¤­hŧØ\}0ûYĒŌŌ+"đ_3}-h┙IENDŽB`‚avocado-113.0/docs/source/blueprints/images/BP003/Resolution.drawio000066400000000000000000000023771513561306000251250ustar00rootroot000000000000007VhLc9owEP41nkkO7vgBDDkGStpDesHJ5NiRbWGryJIjyQHn13flFwibBiaP6bThgvRpdyV9367XYPnzbPtNoDz9wWNMLc+Jt5b/1fI81/Gn8KWRskbG03ENJILEjdEOCMgzbj0btCAxloah4pwqkptgxBnDkTIwJATfmGYrTs1dc5TgHhBEiPbRBxKrtEa9qbPDv2OSpM3OI6dZyFBr2wAyRTHf7EH+wvLngnNVj7LtHFPNXUtL7XdzZLU7l8BMneLw6zldbO7lrRve8OBuPFo+XN3aTZQnRIvmvs1hVdkSIHjBYqyDOJY/26RE4SBHkV7dgOSApSqjMHNh2ITDQuHt0XO63e0hazDPsBIlmGxbBmuP0pxu9shvOU73iPdbnlEjeNJF3pECg4aXMzjyehwt8QrDdYADz1liyWmhCGeWPoHv9ggEKpTJklSCr/GcUy4AYZyB5WxFKD2AECUJg2kE1GHAZ5pYArl53SxkJI71NoOymMK9vTLjE5V5N2H8geSdoEzfnYUyry7t7LTyr/WzA0v1JYe9J1SLEgoYJaoyHXCVhbbSfsH9fL4IgrPFhYxwqs+fFaZ4pf42facn6jt6L32nA4XXKzbnYlkwhkKKLyulDnW1wWRNWFzLmJcq5cwuGFE6E4bNC0HMZPGv7/RIT3+6wz4CPxZE4AyoloPnGMgvyxtAaz+ZI2Yk2uSx0I1iFtX5o08nkvDCH1VxgF/HGF7qcbXJijNlr1BGaFm7dVsCf+ADOSgIoj20C9ZbSTF9wjpNj/tIxKQtsSAr8xyyau/6FDJDVQLUiyGK1kmVz/bBBb3xuIt6ML7csdKSDBXWxM9KSsJjhU7r75ua5U99PkwfeICtq/ct7Z5E0evU+ZTiFVJUye0YgkDFRHaMn/Qb/Mm6/MP9cGL2Q3fgVdSdDDRE132vjnj1Ykf0Pr4jem/WET+ftW9e4K/qiP9PbXfzl2p7en5pw3T3K79a2/urxF/8Bg==avocado-113.0/docs/source/blueprints/images/BP003/Resolution.png000066400000000000000000000517111513561306000244200ustar00rootroot00000000000000‰PNG  IHDRŨUkPę˛tEXtmxfile%3Cmxfile%20host%3D%22app.diagrams.net%22%20modified%3D%222020-05-25T20%3A43%3A56.624Z%22%20agent%3D%225.0%20(X11%3B%20Linux%20x86_64)%20AppleWebKit%2F537.36%20(KHTML%2C%20like%20Gecko)%20Chrome%2F81.0.4044.129%20Safari%2F537.36%22%20etag%3D%22Sv-ovi9mBoTkP63cILmY%22%20version%3D%2213.1.3%22%3E%3Cdiagram%20id%3D%22r4VAieNWgqVfXoFI8GZJ%22%20name%3D%22Page-1%22%3E7VhLc9owEP41nkkO7vgBDDkGStpDesHJ5NiRbWGryJIjyQHn13flFwibBiaP6bThgvRpdyV9367XYPnzbPtNoDz9wWNMLc%2BJt5b%2F1fI81%2FGn8KWRskbGLZAIEjdGOyAgz7j1bNCCxFgahopzqkhughFnDEfKwJAQfGOarTg1d81RgntAECHaRx9IrNIa9abODv%2BOSZI2O4%2BcZiFDrW0DyBTFfLMH%2BQvLnwvOVT3KtnNMNXctLbXfzZHV7lwCM3WKw6%2FndLG5l7dueMODu%2FFo%2BXB1azdRnhAtmvs2h1VlS4DgBYuxDuJY%2FmyTEoWDHEV6dQOSA5aqjMLMhWETDguFt0fP6Xa3h6zBPMNKlGCybRmsPUpzutkjv%2BU43SPeb3lGjeBJF3lHCgwaXs7gyOtxtMQrDNcBDjxniSWnhSKcWfoEvtsjEKhQJktSCb7Gc065AIRxBpazFaH0AEKUJAymEVCHAZ9pYgnk5nWzkJE41tsMymIK9%2FbKjE9U5t2E8QeSd4IyfXcWyry6tLPTyr%2FWzw4s1Zcc9p5QLUooYJSoynTAVRbaSvsF9%2FP5IgjOFhcywqk%2Bf1aY4pX62%2FSdnqjv6L30nQ4UXq%2FYnItlwRgKKb6slDrU1QaTNWFxLWNeqpQzu2BE6UwYNi8EMZPFv77TIz396Q77CPxYEIEzoFoOnmMgvyxvAK39ZI6YkWiTx0I3illU548%2BnUjCC39UxQF%2BHWN4qcfVJivOlL1CGaFl7dZtCfyBD%2BSgIIj20C5YbyXF9AnrND3uIxGTtsSCrMxzyKq961PIDFUJUC%2BGKFonVT7bBxf0xuMu6sH4csdKSzJUWBM%2FKykJjxU6rb9vapY%2F9fkwfeABtq7et7R7EkWvU%2BdTildIUSW3YwgCFRPZMX7Sb%2FAn6%2FIP98OJ2Q%2FdgVdRdzLQEF33vTri1Ysd0fv4jui9WUf8fNa%2BeYG%2FqiP%2BP7XdzV%2Bq7en5pQ3T3a%2F8am3vrxJ%2F8Rs%3D%3C%2Fdiagram%3E%3C%2Fmxfile%3Eīå> IDATx^íŨ ŧŽÕØ?đÍ%IERœTšPH¤Q#ŪæD"QiPoƒĄH)¤‘ōV4 ĨÔ1• iŌ,¤pš*R‰ūŸīzßĩ˙÷yΟg­}žŗŸi]ŸĪųœŗĪ^÷ē×ēîõ[×ĩŽu ĪúĪūķŸĒPá@áĀ¤qāYt“ÆëōĸĀē˛ &™t“Ėđōēē˛ &™t“Ėđōēē˛ &™t“Ėđōēē˛ &™t“ĖđōēdĐ=ëYĪ*Ü*(hÁTįŽ,ĐĨvZžLáĀ¨q€PJÅGŨ¨­Ž2ßŲēŲÂÖŌiá@3 čĘę(˜dĐM2ÃËë  čĘ(˜dĐM2ÃËë  čĘ(˜dĐM2ÃËë  čĘ(˜dĐM2ÃËë  čĘ(˜dĐM2ÃËë  čĘ(˜dĐM2ÃËë †tĪ<ķL5įœsVsĪ=÷ؗ^pÁĢM7Ũ´úÜį>W=÷šĪmģžôĨ/U{īŊwĩĮ{TŸüä'ûzĩ|˙û߯6ŪxãjŽšæį _øÂę}ī{_õÁ~°ëc?á„Ē;īŧŗúžĐļīŗÎ:Ģzį;ßÚ,°ĀÕŨwß]-šä’]ĪÛŪöļęœsΊ}ôŅę¨ŖŽĒŽ;î¸Đ÷WŋúÕęŊī}ouæ™gVÛlŗM×Ū׍ŽFtŋūõ¯Į>ôī˙ûjįwŽ^ūō—WĮ|[nŊõÖÕzë­€—B˙ū÷ŋĢg?ûŲ)MģŪčėWŋúÕXߡß~{Ø`N9å”ęÍo~sWߙ:1c‹/žx…įčᇎYd‘Žņč駟ŽöÚk¯ę‹_übuÕUWUŋøÅ/Ē÷ŧį=xW_}uõģßũŽ:đĀ čēúå;t%]tą;ŸwŪy•…úä“O†wà 7Š`¯]“t{ŪķžWŊë]īĒ?üđęŸøDõå/š˛Sm°Ááã’,¤'iōéOēúÍo~Sũöˇŋ­vß}÷°Øėî'tRõÚמļēíļÛĒ]vŲĨÚb‹-ĒkŽš&,ŠĪ~öŗÕ&›l‚;ė°ę+_ųJõ¯ũĢÚwß}Ģ8 Ė°éŊõé:ŋˇÁŦ¸âŠÕG>ō‘°(Į×?ūņęŨī~wuŨuׅw¯ŗÎ:GķÎ;ouÁ„š˙ķŸ˙Ŧ^ôĸU§žzjč¯ēy晧ē˙ūûÃīQü™†đÍo~ŗZiĨ•Ēī~÷ģÕË^ö˛1I×Ôī‰'žXŲ,júôé•oxá…VK/ŊôL_û§?ũiuŲe—U|põųĪžZuÕUÃØũ˙jĢ­Vm´ŅFá[I7‰¨t€°ũöÛW¯ũëĢ#Ž8ĸúĀ>TĒäŸūô§ę•¯|eõo|#|´ˇŧå-áƒY¸Ī‡>ôĄ°0É˙ŋņoŦöŲgŸę/xA&ĐäškŽTRģîÍ7ßúšīžûÂĸ_}õÕĢī|į;՛ŪôĻ| ûŅ~Ö1ĮSũđ‡?Ŧūöˇŋ…ôõ¯ŊúÃūĐøŪN r*'UË&Ņ4Žo}ë[L—_~y˙ATŊõ­o­–ZjŠĀsX~ųåĢĶN;-H•oŧ1 tsĖ1GĐ0¨Ž^ŌšúĨöėc Ē+ ūÚxę´ÛnģpŲ(ŠŅ€ ÔÆū†7ŧ!4Ũpà č&oáUt$0ØÅ}p€đõ–]vŲęÜsĪ­Ö^{íđ u¨>úŅÎ: š:ujØUŅÅ_\}æ3Ÿ Ą2ŌĢ_ũęęÁ ;ûO<1ĻFŊęU¯ m-;ņã?ú°PļÜrËꁨvŨu×4›ōüüķĪ€ÛôŪVБ˜ņœú÷ŋ˙ŊZtŅEƒ„#Ũڍë9ĪyNĩÃ;ā'I…lÚ´ianxHŋ?˙ųĪáŦĪtM’Ž t¤SSŋ4‰ī}ī{a“C´„˙øĮÕŲgŸ=ĶōĄ ü÷˙wPaß&V§ēÉF\ tQŊ$5¨F×_}ø;îž -´Pe ‹•$¤>Õ%• €í°ˇÜrK>WXa…đ3đՍO=õTPŧ“„Ŗ‚"‹6ūŧųæ›é |uj÷ŪVĐÕĪt6Røį?˙yPÛË{IՓO>šúÉO~RmˇŨvaūT=†ú‚ˇ €8QĐVSŋ—^ziā35‘|õŸësŽF›+žéˇ€Ž@Ģŋr<õŌšéÖ[oÛŊŠMĩ¯•ę ŖÎŦ˛Ę*Õ~ûí7S; ŗ0ôPڑ­TY+č,š¨ų~æ›ož .5ŊˇčüŽ ŊÕV[iĐn\õ~ŒÛĻCj:ĪR{/šä’Đä¯ũkž¤0)Aį˙pHįC’īĄ‡ Ųxę%I×Ô/€wŗŪūûī6*ŗņØPmz,™ÆƒŠ¤ëĮ´ÜrË•ŌGņņ,&’H{ęŖô¯xÅ ’Κįã˙xP'ŠpÎ7$sxtĻéY }ĮwŦyä‘`ŅžąĄIŌ1,yä‘ÁĀBŠ:[Úí=ßôŪN sū$AųË_†16ëŒ3ÎįY*5˛ÁبlOJÚPœ?“e°nHyéK_,¤øiĖŒEŒITQ*5u╁w4õÛ*Ųš$ŨĪ~öŗąsv4îÄŗ\äK]Ÿ€Î0,g;%Õīũī0fuÎU‚z]ŌyŽEĶNĖ’gĄYŦK,ąÄL c0qGD­u…ØTŋv’ÎYĶų‹ŠÄ˜¨Î—íŪÛ t~o¤ÅÛ4.†$’ÖĶxAÍ Hh $؋_üâ°y8×AwūųįcßoļŲfÁĒËø2eʔ`ČĄY–ë¯ŋū˜õ˛ŠßTЇoāJĮ&ášxĩšĨīäŧjNøēíļÛö`ÎüĘĄž§ë —A´p €Ž,‰ÂIæ@Ũ$3ŧŧŽp €ŽŦÂIæ@Ũ$3ŧŧŽp €ŽŦÂIæ@Ũ$3ŧŧŽp €ŽŦÂIæ@Ũ$3ŧŧŽp €ŽŦÂIæ@Ũ$3ŧŧŽp €ŽŦÂIæ@Ũ$3ŧŧŽp €ŽŦÂIæ@Ũ$3ŧŧŽp`ļŽ°ļp p ™b&SčY˙Il™ƒä”—6…Ãā|Đ Ķ—/sé čzÆúōâQå@Ũ¨~ų2īžq €ŽgŦ//UЍę—/ķî čzÆúōâQå@Ũ¨~ų2īžq g ›Õ"†{îšgHĄ­ÚNˇk¯Íί!ĄíמöĩEz"4ĢĪOäå™îr § ›•"†Ōu_yå•Ą¸c IU.Ûo¯IViĩėʘÍęķygyĻģč+ЙZJCĩĶÔP0B *éĩĮ+v(ˇWōæË­¯j6Ŋ4áŌƒ+„ĄđEģb„RtKŨ-Mēg¤VÉK^ŌXdŅ<ÔV“:üu¯{ŨØS€rå•WŽ{ėąPš‹´oēīxĮ;BáH†’YŌŋˇ>__Ęn)ĶĨ´”‚Ō‹Ÿ~úéÕk^ķšęC+Žøío; āB“Īž]jC…;”ˇR@žūĻb‡÷ÜsOXp€Ĩ‚ c(ŌˆĮ{l¨apŅE…\øã#TÔCCÕH]áĮ+Ž¸"HĒĻ÷Ÿß¯ĩÖZÕ /<ÃWDUNũžŨx pÕ7P€CĨTķPģ->ßē\Ô-h›‹ÚæŽĘ^{íĩažČFdŖÄB“Īž‚nĸE Ug‰ ŗ¸Úa´đ˙ō—ŋ„"ÎĒõ(ŋü˙ķŸ˙üPėP՞ņŠĒ˜tąô•ĸjØŠrĶôŪÖę1õĪZ]ģņ(Ŋl<@Ž‚OŦe^~<Đyw,,y衇*îA+PžJQ`VM•Ô.4ųč)čfĨˆaiŅŽ#`S-U”4S6*)ĸpĸÅ<^]4ÕsMT-ĩNŠ,6}Ę:hڍĮũ^éâ‡~8”đĻN #í”īBĘ)+Ėā¤bÎÛßūöj™e– }ŨtĶM“ŋÚĘút“SÄ0‚ÎsŠEÕNŗhÕoĨĻMĘ$ë8IgÁ{īŊ7HŒĻ÷ļ[[uĐ´OŊõåÔ7wŽõîvęåēëŽ[) ŒH8Ō\ aĪx7ĩréĨ—kSp0ųč+ĐĨ1´āëgēÔ"ŒÎ6ŦÎqÔ-5ÔÔ-ŗ(›@į÷žQä‘apĄž:“5Ŋ×øšÎtjiS͕k‚’Š+*"Iũu~¤Ž’Úņy$T'\)fRΝ9ũN9…Ijj&°‰Ž–\ĄŪp ¯@‡)E ĩĢƒ.ĩŖįĸĩPõUUUealWŒđŽ;îÖK*&IQˇ^ŽWüŅ{Æŗ^úę0Ë%‹¤’ÄMãą!ėąĮÕ˙øĮ aŠ†TãÖįëīÁģŠS§†ĢVOĒĩŗjŦ§î>SH›NĄŪq g ëŨ”‡÷Í@§ÚhĶÅģ ‚QˆcAĄŪq €Žwŧīú›[Ë5×_āŽ‘Ēęī\°ëī.Ļs €.W}߲ tîãÎ:ëŦęÔSO(—šžgøX@7AƕĮ &ʁē‰rŽ#Ũ~ûí•w‘ĸ@×iœ~?kK¨<Ëž]j”w(–¸Ë/ŋ<¨o¤‰ņr@œĀâo’Ž›Ž­ļÚj&Ūę§>õŠ 7Ūxã ’ÖŠ茓*zß}÷5~RˇČ™gžŲØĻč:ŗĶīsMi?kč)č&e\;ė°C¨yæ™g&ÉÖ$éZA׉}bĪÎ8ãŒ.CtÁėüe @Įmë”SN ˇ‰¨‘‹-ļX86ĐqPŽ‘ąŨ]wŨ5)ĐnœÚwú}'>”ßw=ŨŦDpŪ=ųä“C ævÛmWŲÍŠdš’.••Bˆ¨‰ÜÁ…§ūĒĢŽZqÄ3Ü}qNæOÉđrÉ%—„āZį¸&"Ĩ…Üp~nēvgēúsã3į÷Šü(í&΁ž)äDÄ)ŗĖ P%5]œ ¯"†ŠŊ÷Ū;œņꆔēzŲÄ:ž™Ŧ‰;í´Ķ VC‰,.oĶM7 †•čÕ¯/ €ÄÔØH1jđzë­7ö*€\ĘpÂOŌ˜éXG# 7âĖ ÔíÔËNãøÚi_>åɉp ¯@—e@ŨcZgA¤ˆpáđ+­‡hĒāUW]5čÖXcęčŖČĻ3 Ŗ‹” @ÄČálÆÜĪ˛×^{U\pAĪųįŸ<úY7Aj…D&}*-ëĨSŽČž™wŪyƒU“dëÆ8ÄųÚ<¤[`št´i´]§q’æ1‘…Sž™8ú tĻ‘eĀSŪĸ¤N:įXđ€čĖ%ĮųE°ææ›o2•¯.éDƒ“2R1l´ŅFÖËéĶ§Wx`HuĀÄīŽÍ{šUÅ;3`l*a‰%––H ­ßŠQƒíH3}ˆ°aH뀞zꊐ–Ęln@´B{PĶ=ķėe—]Vug§ßO|ų”''žn"ƒ-Ī  č†á+–9  čęs•Á č†á+–9  čęs•Á č†á+–9  čęs•Á č†á+–9 fčŠ e°…“ĖŅ1)ôŦ˙$ļĖArʋK›Âaâ@> č†é˗šôŒt=c}yņ¨r €nTŋ|™wĪ8P@×3֗* čFõ˗y÷Œt=c}yņ¨r €nTŋ|™wĪ8P@×3֗* čFõ˗y÷Œt=c}yņ¨r €nTŋ|™wĪ8P@×3֗* čFõ˗y÷Œt=c}yņ¨r €nTŋ|™wĪ8P@×3֗* čFõ˗y÷Œt=c}yņ¨r` @W/Æ1ĒĒĖģˇHĖT’4ȁ]7'Ä™Ō¨pā˙8’Ļåô×ŗ)9ƒL™tiS8Ãn¯ŋœū čržTi;4ČIʤsú+ Káhi3tČIĘäsú+ Káhi3tČIĘäsú+ Káhi3tČIĘäsúë9čžyæ™jÎ9įŦæž{îąš-¸ā‚ÕĻ›nZ}îsŸĢžûÜįĻĖ9šÍOúĶj›mļŠ~õĢ_ĩ}æûß˙~ĩâŠ+VK.šd¨?~÷ŨwW_øÂ’ßĶŠá{ßûŪę¸ãŽĢæŸūjįwŽžō•¯„Gyä‘Pŋü7ŋųMuįwļífË-ˇŦŪõŽwmŧņÆcõÎ=ôž0ÔJ˙ā?Øi(]ũ};ūÖkÄ7ŊtīŊ÷Ž–^zéPg~vRHRƑĶ_߀î×ŋūuXāč÷ŋ˙}Xˆ/ųËĢã?>eÎÉmRA÷æ7ŋ9,ØWŋúÕÕ_˙ú×Ęæ`3čí´ĶNÕWŋúÕĐī{ŪķžęĖ3ĪŦž|ōÉđž-ļØĸēøâ‹Û‚îK_úRõíoģēā‚ *€Õ7’Ûoŋ=l\§œrJe.“Eŗ ēŋũíoÕ*ĢŦR]zéĨÕ +Ŧ0ۆ’”Aäôח 3IRåŧķÎ }⟨žüå/W&ˇÁ)1×\s)tꊧV˙ū÷ŋĢŊčEĄÍ˛Ë.[ŨqĮa!’ķĖ3Oxžd¨/ŠO~ō“Õũ÷ß_ũĪ˙üOxGüų%/yIõŅ~´š2eJuĖ1ĮT÷Ū{ī˜¤kę÷ļÛnĢvŲe—˜kŽšĻúŨī~W}öŗŸ­6Ųd“qŋYŨĪūķęĸ‹.Ē>üáWųË_Â†ãąˇ“t¤0ĐŽšæšã‚ÎKm\Ú}ä#ŠŪôĻ7Uī|į;ĢvØ!Œ'ūü˛—ŊŦqÜ'žxbŧO?ũt5}úô°A\xá…AũņŦŪņŽwžāũ>đjŸ}ö üŨvÛmĢ­ļÚ*l4˜ĶN;­zãßXÕ% f¯ŊöĒn¸á† épæŒ>ũéO‡÷ųŽČwũú×ŋ^Ŋîu¯KY˙Imr@’ŌaN} :‹nûíˇ¯^˙ú×WGqDõÍo~ŗúЇ>T]wŨuÕ ,ÔCŅÂ}éK_Z=đĀAÅ:ëŦŗ‚Tōņė–‡zhĩãŽ;V?ûŲĪĒמöĩáCRÛĸzŲ: ô<ā“ø`h?ž¤ŗ P9Ķ&Õ:ķi7õūcûXŋ…¯/xÁ ÂvĀTūķŸĢ/~ņ‹aã^szøá‡ĢWŊęUhģîēkuöŲg‡öŋøÅ/f>účŖ‰ũ§?ũŠzå+_Y}ãߨV[mĩĐvŨu×­ūđ‡?„ųá×Zk­U-ŧđÂ)ë?ŠMHR:Ėé¯o@Gu3đũë_aįÜc=ÂĮ˛S:ˇL:ĩ:øāƒÃüŠ^€pÉ%—TK,ąDuôŅG‡ŨՂ@>ž…ôÄO„>Ņk^ķšęC ģôDAGŨlęwšå–ĢÖYgęņĮī#!HVB+Q I5‚9ۋ_üâąf@GÕ AGĸÆķīß˙ū÷jŅE n÷ŨwŸA˛ĩJ: k7Đ}ī{ß ›:餓Ē˙øĮH¤Ū?˙ųĪjŪyį ŋķ}Œkžųæ  l&Æĸ 0:ˇ°ÍŒ6rîšįVk¯ŊvxūĀ * =˙ųĪ¯~ō“ŸTË,ŗLʚĪn“’”ÎsúëĐÅ3ŨJtũõׇŋ•âŖ Ād(¸å–[‡:‹Ãni‡%u,Žú‚ß|ķÍĢˇžõ­ĄÍDA÷îwŋģą_‘4!I‘ÅU˙šūáHˇŊímáĖbœ¤b:ŽUũj•t6%ĒՕ ŽÚIēĻqžG#Oũgā;ė°Ã‚úūėg?;l2—_~yøFΐuŪ3Q;ã9č´[hĄ…Ē9æ˜#Œ8i74Dƒņ^nvPHRŪŸĶ_߁Î}Ė[oŊ5¨jhˇŨv jŨ~ûí×8ģîá‡>>㠉dˇĩ ÕÎŋÔRKÎyņ!žœĢœUšÔË(éÆë×90t¤•s5ÖRØâ}Å+^Œ6q×īēO}ęSáœĮ=žzI5wŽŠV@›Ž3ž÷#ŌÍ9 _':’í ƒĒlDhųå—ę=0Ŋá oĒ§ųũã˙gę¨JFI§Ŋsšī4ĐUU``7”cņĘ nŊ¤Rרn¸aõ­o}ĢúøĮ?^ũđ‡? *if' íúÎ->ŦķÁųįŸT/‹ ˆœûė˛Î5fŽ’ÎsŦ{9Ģ”Ä:  A-ČxĻ;ũôĶûu6K¯îšįž åĸJ_'НqÆaCĒĢ—­ÖKį_@ûå/Y-˛Č"AÍ\ląÅĒ#<2đÁŲÉylĸ Ķįe—]6 ‰Šüĩ¯}-œũü#Ęũ×UįœsNā#°Õ )ûīŋ8ūķŸĒ*5›ô÷,ĸ^ÚxmfåL—"kÚ´Į} gę#iuÔQG…ŗ‰f´đ|`;9‹šķÃâ‹/^ÆJ+­4fŊ´Ã:Sjm´Ņ ÖK*¤3•įü[@~ėąĮ†EJõ‰÷tŅzŲÚoĢ:Ų¤^2;ĶœYÖ_ũĀąiĶĻIds3O ‰Â(R'?;/ļ3¤h˙–ˇŧ%\ÃP _ôM™įc=@|‘t6+`zŪķž¤ ¤#}ČĶÆā;7ÉÚjŊ|˙ûߤ>Đ9BĐPX2mÚ;jxžX/•Ō¤Û’3åÃŌ(Šg¤‰+ƒaŖŲáŒĐĘŖn¯ŋœūz~Ļļ3Yķ!%ŠwTēa"w‚ŽRžûŨīfvQHRƐĶ_] Gû° iGÅ¤ÚąĘ ąē2všB˜”’”qäôW@—ÂŅŌfč8’”ÉįôW@—ÂŅŌfč8’”Éįô×÷ sĘĄ#XáX[˙ŋƒRŧÜSÜچõÍŲŠÉģŨĨøDúîÖ3Ŧƒ.â]W°2ōÃDŽ8LÔ¯qVŸīÖürûÉIJß9ũ ,踚+Š—ß)Œ™] s­árÛ}Rŋ‚ÎeĩģLnuŽV 1Ķģá"7šÕį'ōÎn<“’”÷åô7P sŲÍã„/ _M’ÎũP“w?KŋMwAįĖëîæ¸!Ųõ] ķRŠŪķîô¸™YŦîšÜg‰:¸īžû‚ âôë.ãŨ{ņ(ėhāp?E zŸ{+>Ĩŧm˜úĮ`÷]æâ2˜ļ;GŪ&ÆÍÉģ•¸e‘RæŽę?ˇ‹ ˆ’N˙øįŽŽCøž{îîÆ8'ģwkŠæāâįëūĐŊ'ķy­?Ÿ˛@ûĄMHRƛĶßĀ€p¸?]}õÕá‚7Ē—@Ņä%ĪÛÁE4O–§žz*xœˆ6€˛Ø[ÍŌŧW€“9đ,P–4—ÚÜĒxČ°°!NģúįoŠĪ(éøwē`9Ā{ÆĨžņķ>áA#ô‡c0Uzl^¤ļĀV‹žw‹žYģ:ĩ]ģ¨€:áŪiŗ@.¨ũLŌ5Es˜¯ Š§5Ôåŧw™cũų”ÅŲ/mr@’2æœūtÜģė˛W^ye6¨ē&/yŌŒôF‚¨€tÛE€Ũzë­$"ŠėėÔY>Ž¤ glíxŗk!=ę lŽl¨m„$fô‡l‹z tœļšŸ!a,¤.7¨Đ5E¤€Ž)šÃĻe˛AqßâĸŠ€î91P .Âå QŊĸ‰•!ÅnĪįŌBĄ$uĐ5š1‰'ã{T"ūŽ@Į} ˜"E/y€į’d¤M’‹ãntMãĄbÚ¨rčæ›owl$%БbQÄ°,đ:āF‹ZŧIĶ*éššP*eŒN aH°N ãĄ/’ĀîÅČāėâėHúP ŠZÔJžĄtSAGŌIAÁ;ņēpĻ$Ņš@ĮčQįĪ]wŨÆ`ķ@$›hgŧY]J4G7K-˙S›dTOgøp}ūCŽdJ™JNĄ^˛’v,¯uę_Š¤ŗķÄ˙Á~@€ÂN:éœÉœŋD#đ gāˆŠĄEãCĖIâ,¤ Ī~ŽÅM›€Æ į;@ļ)ˆ¸&ɚ@%_üđ=ôPp‚•aCrŧęĒĢ’AĮČcķÂ՛EĶø›ĸ9ļÛnģā”ms0Æ$j°ˆoj|>eqöK›¤Œ9§ŋmQėģīžáœDØŨIŽĻENŠ˛Dƒ( Gî÷TMÖKĒŖEfqļzÉcž3žw •aô@ĖīΎ‚kSšÆSˇ^rã"QIn‹?tŪGbŗ˜ŠPž#-ĪüIGJRģ…ÍP]oë­ˇķĩĄÍD{x IDAT!:ĪüÎüm€TMĒ1I[>eöC›¤Œ7§ŋž]ʄģŲ&&Öąˆ›ˆt˛ësĖ-4˜ČIĘ sú+ káh§}v|Į¨Ŧ…—9 I™eNt sGåCÍm=kĨ|˜ŌĻ8’”QįôW@—ÂŅŌfč8’”ÉįôW@—ÂŅŌfč8’”Éįô74 ãÖÅ"™SŖ …™­mę5&ō|Ķ3.Îã9‘TΗHî]H§R=ęĄÛãĪ™?Ož™Ŧš,ĢũD9 IwNC:‹” ŪõĶ{;Ē×(Hafk›Y}>坎GXGe&›ÕŖē=ūÔųģĻ‘!Œ# Ŗ€î˙‰žKd.YܯPüŲ%1_A•ƒ0¯(éÜßų;~8×kȆ•S#At@ëķą˙Üčwv.ļĮg+č\Čķ•C’#´čw…øâŌŨŖG!nYŽ2bÔCŦaœŗsū­āĻqøfÜcĐ č8.ķ0ąyvÔÕK)˯¸âŠĐJõš5xĸԟ¯÷=‘č}7Î:č,X÷‚"´—G Oŗ°š’šä–/?\XG_Đɚ“D–°€nFî Ŧ¤ã÷ĮųĀ˛Ļžéę É­‘€uí@—ā0ÕA'ÂBŌ]~šČ|ÅŪ Íáē`œŽšŽÅ€Ūē/h;Đusūtéɘ{ēNQMę%Đ1>P-ŅD@—[#A˜M;ĐåF'4-Ô:č䀔˜ļ~V%ÕÔ)đ~ĪųZ€$¯|)SA×ÍųĐ č:EđKt>p†‰yņį€Žîã8ĐĨxÕ×k$¸oēÜč„Đq¸"‘íˆÛšōaÆ(ōfæ@HRø—Ķ_߃Nęsöü Ũ9šø•‰Ë])å’XFe$Ŋģ!ž‰.ũŽ'ž‹bK Žę?GI|õ¨Y›ŧûeđb8‘Sj>uܗ|ōÉ!ũēģ2ÆÔÅĐTģ@†­Xk€ŋĨj@ęxÛtܡšiДÍ,e”6y™Sø5T “Ŗ€ÎÄx]Xā.Z]ķw”‡qúôéĄ8ŧdŦ§ŋĄ] ŖJ›Ņå@HR¸”Ķ_] GK›Ąã@HR&ŸĶ_] GK›Ąã@HR&ŸĶßЀŽŗąËīX€žyŨuLíūŨk’ĸwÉŌK/î]H‰˙]/Ōëąûûs@’‹œūt•ę`ŌŠ+ŽRō ŗáÛxČ!‡T ´Ō´iĶÂģ…Ôđá ãŌÛåĩ$¯@îœ' @ņÍtxĘđÁŧņÆÃßM’N$5ŸMŠ=ôĐĘeé+zŖ¸ā÷^‘ĸ-8CoŊõÖĄNH(Ü,",HØB3r $)ŧËéo(@‡)rPŠģ# ëŌ­ūo9Ep§"?üđāƒÉéšÛpo°Á3ņ¸ôxS§N ĀášÅ͋ķąø9ž‘bįöÛoŋbÕ}Ø;^1ŪŨNŊ‘`üÆŗå–[Đmžųæ3Œ…gŠø$Ŋįž{‚§ Įķēä’KÂ3@n,\ߊÚ:3lr@2R S…ÆBŗkķ™”b‰:PÉĻN) ÛtĶMƒ jĄ"Žd@&,¨ŨÂ:’ŒCu”8įœsΘsĩĀR‹^Å‹œ?¨1Î5×\—.@ą´įÃ"ā›sÎ9ƒŽ^8ģ’Ô‘Ė¸ôMʞwŪy!#4ÍŋôU¨€.p g§é¤^ĻJ:Ą1Ā5÷Üs}ąyĀëߍˇ@îĖ3Ī ōˇ?¤ō7õTČJ-¸•s3zo;ĐqÔæԌԑ—Gˇ‚Îģâ8üNŊmļŲĻÚi§‚ąH<Ôĩ×^;¤Ą§B* ëč¨k‚M]r(tú•úŨ™‹ęéIę v SĢ `IIjk+訊¤b$EC:蠊Wũ””Úã?žŪTh|älú)<Ėéo¤ÎtTÄn¸!„9ĶYŧģø:Æj&Kb+å‚N„;ˆvÂ!UځnŸ}ö €aŠeH8ĒŠ VĒĩ<0$u‘+ZŨī„8)É*Š‘ņ‹žPĄÕ9ąPŨrv†”…“rĻ#uT2UXÆ/á5TąUW]ĩ­õ2tÆ ‹96I:j¤ˆu†—ŗĪ>;¤y:ŌxŽ8∍î,tQŠŦæã<'™Q$å†Å’zę¯§đuXÛt{ũåô7’n?<0s7Ĩ Đą\^1¤ČdŖä‘w“1搤Œ'§ŋēŽfļ‘…ĖŲ‘DeØčEIxãŲģH › 5s $)|Ė鯀.…ŖmœĨÜË1f8wu“ځŽZéęBú?‘õ…Ús $)ŧĖ鯀.…ŖĨÍĐq $)“Ī鯀.…ŖĨÍĐq $)“Ī鯀.…ŖĨÍĐq $)“Ī鯀ŽŖÅû?eŠ n›¤Ė2§ŋ]JhO crÚĐåpkđÚæ€$ev9ũ č:ų^ōqÁĖ[ã˛Ë. Ą7Š‰pŋâķč÷~ĮÁŲĨ8O—âĨōŽđæũgžy&„ĮhûĐCÍįƏ‘uĐŲŧķÎúæɯ?—Õ’Â -ōŗhãXxᅃ%‘O¤ģŗ)SĻ/ŋˇ‘đå€-ĒÕ/Â[?´Äš.ĖeÄé Vh÷6ÍÁåēôōúu ôˆõjĢ­–˛Ž†ēMHR‘ĶßP€.J%>ˆšä°Ü°üĖߒĶ1`đöˇPôČ# āz#@–kĐÉ Í›D}‚\ĘĢP\<ķ”ÂŨ˙q>ļ°ECˆĩd j˜ßŖ;:!:€ĘŨŒë_IąsÆ`ØjĢ­Â…ˇZ ÜО~úéļS'î]üE96‹ŸSČũŸ0ŖĻ9čÛ<¸ŋq#¤%Ü'Īá~¤@—ÚAĮõÉnŽH"€°ŗ[Čū͉ņãNËÂUŨ‡„CR…ÕČ tįž{nuĀ-ĀBĀÉ ˆĀōé<ãŒ3‚§÷Äx:RZ(ĐÉÅiŅ×Cs¤ˆŌÃī˛‰Äú[FđqķēĻ9Žœ¯E@ $Ŧx?;ķ(SŽdJáSNC%éԋ‹a;õđĢEeŸzvGĩS¤e]§d ĻĸōŋäĀŒDƒģŒÖFÔŲå–[.„ö¨Gũ‹ ˛ČŊč,~RŽsr$Ē!ŠÜQP‹`Uū˜.Ũ#XāÚL€Ži‚ZIÜXŌ+eņŒJ›¤đ$§ŋĄo{õܐsō[Æaxà 7 ŋŖn’t@GÍSdRšDÂ9—‰'éÄÚņc¤˛r$FÔ6EH¨Ģ˜MŨäútūOđ(ĩ‰“ĶčœË„Üԝ S>¨äE6įA•i‘MÃčšæ@2ĒXDâ!jĒ#{Ž9æHyõĐļÉI rú*ĐQ•ĮĒ‡Įˆ0wŽá•ŋØb‹Õ ˇß~û`ā ~1ˆ Úyįƒ:(ĸ<žéH#ÆųGü>øāđĘ*ķŗd\q6#UT’ĘÁ˙Qo`€E¤ƒŋŲŲRŸÂH4īâ(]'jĄö6 ÁĒ˛ˆų[dĐ5ÍÁ<ĖÁûl8R9P{ŠÂŖžÂ!$# ēN“ŽĒ ++c] ą°-8R‚ņ$æ5ņ˙#•-nRƒ”jMįl&ŊŖpp*vĻSžØŅ Ú{Lr$˙o3ˆįÍhŊ” …äĄZ:×A;ëĨįj<ŗúęĢ‡š¨ą‹ˇoú0F įBĒ25–::ęT@7‹+ ßîÔ¨§Q’0ÕSE–䛲)IJĪ 6€GâĘį@]>ĪfxĸŸ@įŘĸ"ŦëFgĒĻôŠS§ÚRAe%dĒ#éæ ĸP> čōyÖˇ 30FwzČuĩ‘usVˆÅ”ÚjƒqĨ!nn¯Ŋöš•.GúŲē‘ūüeōŊā@]/¸^Ū9Ō( éĪ_&ß Đõ‚ë īė”“$Ą‹Ž6™ėĒ?ę#ŧũíosčędjy@9§O6Đ%pœÃÉôėWĐMV՟ē„…ųMr@<)æåB—ĢU“'EģŠ6žŋúęĢC˛ XF‹å×>’W’ƒ4—-ŽcŋJW^Kiū”˜Ë•‹ą)\ČÅ7Ķ>—+×Ėû§Ÿ~z(^‚x­Hë>ķą 또–ãĩH ŽZŠ¤kĒúĶ4ˇņ–“jC"XG%Uâ m\ä\Ž§9ųŧax˙pā^÷ráķÉWÕ7iĘįS$]¸sœ˛?äö× tí*ڈk“’Ž“˛{-āåzeĄø—*ŋį§yīŊ÷\tŅEĄ,‹‡W‡cāáˇØ.\ˆG‹ģ4>™@NEķˇû:ņmÜÛ)y%îŽĢ˜lĶģEėšÁīr@×Tõ§in}zØ l $‘ŌejÔá Ū+ËĨžūnģíļĄ„ ŖëuI§Îšj7ĐtH8ß Šëā}ąJ’īĒnž’bŨĸ¤ŧ3§ŋžIˇũöÛWrŨSˇŗŧUŊ3ĒõĐG´p*|ā‚Š%Ŗ2ŠäãŲ-åđßqĮC…U Ų‡”Û?Ē—M BĪ>IÕK‹§Š_cV„Dé( چtĒÕÉÂŗ‹‹Tj¨§€EX'uõī ëd>¤˜Ú h<ĐŲ¨œ€i“jų4›z/I.•ĐÂĮWĩ l`j5¨^¤"‘ xÍI1L•‚đj×]w­Î>ûėĐ^]„:č€ôŅG ĩ!WQGOU%ÅK´%áÕO@øĩÖZk…Ō\Ũĸ¤ŧ3§ŋžuĘĀÕ°s*âácŲ)[ĻNĘ6Ą‹/ž8A%–Xĸ:účŖÃî‹YøøŌO<1VRŲ'UyėŌuŗŠßå–[.TzüņĮÃI’Õ†ĐJÔBÕ|læl­•WÕĢ#1•›RĐŖNTQ}ÄÚ@GĸÆķ¯ķ —$Üîģī>ƒdk•t@×4n S.,æ<餓Bš,@"õT&ŠE2}ãRyˆD›‰ąhŒÎ­ņLGQ)ÖđSDĶ†‹( ­øČ2Ë,“˛æŗÛä€$ĨķœūútņLgwŖ)ˇäoDeŖ†Dƒ`2PĮ| ŗ8ė–vXRĮâĒ/xRÃI›‰‚NÄĻ~IDŌ„$EWũįú‡#=ÔēsfŅŠ)Ö6WۛJÅĐŌJ­ęWĢ¤ŗ)Qõ¨ŽŅ¸ŌNŌ5čđ=yę?Ÿŗ&õŨĻ`“ąAøFΐuŪ›ĩĶ¸"č´[hĄ…Æ q'í†4D4ī%áfå€$åũ9ũõčLĐĮt–ĄĒĄŨvÛ-¨uj‚7‘]—tđņ…H$ģm”vT;ŋēotÎkŒņėā\åŦŌ¤^FI7^ŋ΁) #Ũ¨œkŦąFÂ¯*ŗú6Fj•Ķ9ÎYgP.‹ęe1‘sŸ]ÖšĀėĀtžcŨSŽ˜ÕŒ JbP d<Ķ1ß7õ () ŗĀ⁒ˤ\TŠ”TNk)æúÂTĘŲ†TW/[­—Îŋ€Fb.˛Č"AÍ\ląÅB]p|pvr›(čôŠjĢ ƒDĸ"+ éėį˙QÔĩSk­~ĻÛ˙ũÃųÛõˆ5@Í&ũ=‹¨—ø`3+gēY›ĐĻč<î9#PI+ĩĢ%H4ģ …įÛÉYԜ_|ņp¯ĩŌJ+Y/í°Î¤šâŒuëĨNŌƙĘsū­ ?öØcÃ"Ĩú´Z/[ûmU'›ÔKF`§bšƒ3‹ ¨ˆŅ"­Žą…4Ŧ#‰ķb;CŠöŒ0Ža¨…Œ/¤Idž=öXđMDŌŲŦ€I™hŌŽ†@Ō‘žäŒic𝁛dmĩ^*ÎIę# …tˇQhī¨áųbŊLTJ“nKΔwKõŒ4qe0l4;œZyÔíõ—Ķ_ĪĪtÃļ`&k>¤$õŽJ7LäNtWžF3ģ($)cČ鯀.…Ŗ}؆´ŖbRíbŊō>föX]ģ\!ĖNĘIĘ8rú+ Káhi3tČIĘäsú+ Káhi3tČIĘäsúë{Đš@e ŒÎБŦp,Š­˙ߎA)^î) nmÃúælÅäŨîR|"}wëÖAņŽ+Xųa"ל &ę×8ĢĪwk~šũä€$Ĩīœūt܊ÜĩēHõtŽ5\nģOęWĐšŦv—É­ÎÕ  !fzW"\ä&BŗúüDŪŲgr@’ōžœū t.ģyœđäĢIŌšjō’g ãˇé.ČáœyŨŨGfģž‹a^*ur'Æ{Ū73‹Õ=—û,Q÷Ũw_pACŧGŧÃeŧ{/%¤i4p¸Ÿ"ŊĪŊŸRŪ6Lũˆc°û.sqĖ۝#oãæäŨJܲH)sGõŸÛEDI§üsWĮ!|Ī=÷ wcœ“Ũģ5EspņŠķå˜āŪˆųŧ֟OY ũĐ&$)ãÍéo`@8ܟŽžúępÁÕK hō’įíā"š'ËSO=L‘ĸ—<ĀsI2Ō‹ŋ&ÉÅqˇēĻņP1mT9tķÍ7‡;6’čH1ž&(ū,|ŠÎ^÷Męeģ¨€Đĩ‹æ°éˆmœ6mZpũĸFbŨ‚ŽÚ'ŠVk( Đ n´¨Åˑ4­’Ži‘ ĄRÆč†ë:ú" ė^Œ Î.Ύ¤•ĒE­äĘA7t$ŧķ¯ gJ­ tŒuūÜu×]a 6D˛q’vƛUĐĨDsÄqŗÔō?ĩIFõt‘Üį?äHĻ”Šäô7ę%‹ igÁrĻūĨH:ģ1O|9G€…t:Ķ9“9‰FāAĪĀa HLX4>Ĝ$ÎBÚđėįXÜ´ ˆ€`Ŧpžd›‚ˆk’Ŧ tQōÅ˙ĐC…€^Q6$į@Ņ Š cäąyᇠÅĖq` ēIDATŽęÍĸiüMŅÛmˇ]pĘļšc5XÄ7ĩ?>Ÿ˛8ûĨMHRƜĶßĀ€ÎĮļ(öŨwßpN",4’Ģi‘“ĸ,‘Ā ŠĀ‘ģÄ=U“õ’ęh‘Yœ­^ō˜īŒįŨBe=ķģŗŖāZƔĻņÔ­—ܸHT’ÛâO÷‘Ø,Ļ"„īH Á3?EŌ‘’Ôna3TWāÛzë­Ã|mhãEsˆÎĮ3ŋ3 U“jLŌ֟OY ũĐ&$)ãÍé¯īA—2ánļ‰‰u,â&"ėús &r@’2Üū čZ8Ú)EŸ_T7•ĩĐār $)ŗĖ鯀.tc¨š­g­”SÚôr@’2ęœū čR8ZÚ r@’2ųœū čR8ZÚ r@’2ųœū†tÜēX$sj¤0ŗĩMŊÆÁDžozÆÅy<'˛‚ĘųÉĄ éTĒG=t{üŠķį‘#Y¯=ŽYøzö å€$eĖ9ũ č,R&x×Līí¨^Ŗ …™­mfõų”wēa•™l"TzčöøSįīžËŪ,æ"üÉ5I?PHRƛĶ_߃Î%2—,îW(ūė’˜¯ īÂ\ŧĸ¤sį˙íøuâX\¯Q VNŅ­ĪĮūsŖÜŲšØoœ­ s!ĪoTIŽĐĸÜâ‹{Hw6éךešĘˆQ$S¤Ų9˙օÉØ´ęĒĢŽehVã1OfĘBžmr@’2ŽœūtvQ& ĪŽēz)eųW\>r+ÕkäÖH "՟¯÷=‘č}7Î:č\ŧģ™ Ŋ´€N$:ĄiœMÖKų]ÄŠ*ā’†ęøšĮsž¯G­G=­×čæüņ¯‰ÔdU­'Ž2Ön—ŗNÄxmr@’ōŽœūút)î‡6%:ĄžBúr@’ŌkNt)MhSĸ˜ÔGMr@’2ėœū čR8šĐĻč†1:!FԏĮjn,UĀēž4ÉIĘsú+ Káhi3tČIĘäsú+ Káhi3tČIĘäsúyĐM¤&BĘG˜•6õZŗŌOyļ™9 IácN:Â9u RĨÍx5ZCkRûęF;ŅõZŨčŗô13r@’ÂŋœūútRŸķ°į_čÎÉůL\îĸH)—Ä2*#éí\ÆōMtiėw<ņ]û[ZpT˙9J:āĢG-ČjÜäŨ/ƒɜ˜RķŠ“āęä“Oé×Ũ•1& Ļ(†ĻÚ2lÅZü-…èãmĶqßæb\¤AS6ŗ”RÚäedNá×PNŽF`:ãua cq9ĖßQÆéĶ§‡âđ’ąōŧ õĻ›n .d) sŲ^ZhįŨ/՝KuīrY-?PŽWj-Ø\|ˇĢ ĐTģĀxŦ5Āë†GÎ<ŪCíô{ŅMĩRHiS@×v XœÜ¯A>øāƒĄÚŒP‘¨jĒ|Ãé—0É`Ą#ų'…ž¤‚Žsn=jĄw?i*ÁkŦžŗîēë†,Ëü Ibcä"ÖŽ6@SíāŠ ãĄ/į$ Ž:Ž ¨Pw8#™RŪ˜Ķ_ĪÕËNQ§‚üíîĀW/Éa¨uūˆ䛘ēzÔB;ī~ČÆß%¤GX^û$¤Laü*Ûyņ7%—•ĩš^UG8Đ ááÛŠ PkQ‘”EQÚĖȁ¤đ.§ŋžƒŽS”A}q:ĶņuŒuęĖ<Ō-J:…š¨r;!uîø+:ã5ĨgoįŨOŊL];/ūTĐÅ9ļF;¤,„ŌĻ™9 IácN=]§ ĩ.N;HRLô¸4ëvg:u ÔpĻŖęIŋlŧėE*\{íĩĄŌŽŗ‰ŲtíŧûSA×Å ęNč]tŅąZwÜqG8Ëŋ¸B*/pęv9ĶuZ9ퟒ”7åô7p ŖBĘ-ĸ€†szi˛N!Ĩļ*”9K9*˙:Æ9:&D+8sųˇß7I:ũEëeĢw*čôŅT ]í‚Xk¸€Løė`ŽŒŲø‹õ2 EŌÍĀœa"ėíä€<‘>Ë3ÃÁn¯ŋœūú^ŌMô3ĐM”sŖņ\HR8’Ķ_] GK›Ąã@HR&ŸĶßЂ.…QĨÍčr $)\Ę鯀.…ŖĨÍĐq $)“Ī鯀.…ŖĨÍĐq $)“Īéo¨@'Ãą˛žŌ‡ķā`bwĨ 5¯ž™Ŋ$—ûî]{HÚ˙-ŊĮf×…&‡9 IQN:~•˛™ÜxÄa™›”TuŧņŸ|ōÉpˇĨ>v?‚Θt.eiw¯MHRŪšĶßĀ€Ž'Š¨z ŠČ Žd2 ķÆį—xĐA5J:ž&Bo$g՗4zÜÅü›t<äC*Å@ZiÚ´i!áĢ°ž"ŧa\|ģĀ֐ģ4'Á8ā€ā/:Įs„nx”– v’NáĨĨ\úKĐĘĢĻ•ĸGŠK~īĩŲˆ¸ā-}9OŽH8ü,ĸ,úĨZNĘžŒ69 IONC:LŽ]AēxāãĒ—;īŧs—Q+K˜Ē7b؄¤ø¸ 6Ø`&>—>oęÔŠA]&Ž^€Čü#8âˆJÔīÔh§v 9;9F7NnO9!ĩģņÆø„ qiĢđÚ€OâØ{îš'xÛ\yå•áWąK.š$<ĸŸũöÛ/¸ŋˇYĨ,Ļam“’äô×× #Iĸ3åÚb÷VŊĻN) #IHĖëŽģnl1 Ī!ŨIÆŠYôįœsÎØø—ZôĒæ›P#ņu€,ŨšwŠæôW@—ÂŅŌfč8’”ÉįôW@—ÂŅŌfč8’”ÉįôW@—ÂŅŌfč8’”ÉįôW@—ÂŅŌfč8’”Éįô×SĐĨLĻ´)˜]āHĐ-Đuk˛ĨŸÂ~ā@]?|…2†‘â@ŨH}î2Ų~ā@]?|…2†‘â@ŨH}î2Ų~ā@]?|…2†‘â@ŨH}î2Ų~ā@]?|…2†‘â@ŨH}î2Ų~ā@]?|…2†‘â@ŨH}î2Ų~ā@]?|…2†‘â@ŨH}î2Ų~ā@]?|…2†‘â@ŨH}î2Ų~ā@]?|…2†‘â@ŨH}î2Ų~ā@]?|…2†‘â@ŨH}î2Ų~āĀl]?LŽŒĄp _9šs%91QŋN´ŒĢp`Đ8P@7h_ŦŒwā9P@7đŸ°L`Đ8P@7h_ŦŒwā9P@7đŸ°L`Đ8P@7h_ŦŒwā9P@7đŸ°L`Đ8P@7h_ŦŒwā9P@7đŸ°L`Đ8P@7h_ŦŒwā9đ˙ÜĻvį™^IENDŽB`‚avocado-113.0/docs/source/blueprints/images/BP003/Task-scheduler-flow.drawio000066400000000000000000000025451513561306000266020ustar00rootroot000000000000003VjbUtswEP2azLQPycjyReYxQEKZoYVpyrQ8MSIWtqhjGVkmDl9fKZZvuZhAQgJ9SbxnV9Jqd49Wdsc8mWRnHMfBd+aRsAOBl3XM0w6EECH5q4BZDiBHAz6nXg4ZFTCiz0SDQKMp9UjSMBSMhYLGTXDMooiMRQPDnLNp0+yehc1VY+yTJWA0xuEy+pt6IshRB4AK/0aoH+iVEdSKCS5sNZAE2GPTGmQOOuYJZ0zkT5PshIQqdEVY8nHDNdrSL04isckAJx6bz+7d6eDWuX9GqI8fhzddR/smZsV+iSe3r8WIRfLvmLM08oiaBkiJcREwn0U4vGAslqAhwQcixEwnD6eCSSgQk1BrSUbFHzW8Z2vppqY5zfTMc2FWCJHgs9ogJd7UddWwuVSMyzekdrE2ThpKWMrH2qqb+eJs6M8y/jC6PLq7Ro9R1C3KDXOfiBY7WGZTkoCwCZH+yHGchFjQp6YfWJejX9pVKZMPOmurM9jm5BMOU71SBzqhdPf4Tj746uEneUxJImQGtUYuVCpLa74eEXL0bZJSQZZ1v3DyV/oA1C4gUBJcqqmqgFTOp4GcaBTjeeyn8tBoFsvafD0RLkhWg5ZDrrVmcXhMK8YaBQ+DGlsdsH1WVvIK7ZdXNVaBHrI3I1YX9IDRoFbPttwX6DWXrginMk6E755zcEPOoS05Nx/a5xzPagYxo5FIajNfKaCqK1ic+rqXGabdPHJfsLdNa6Gycg+qOiu38vbSc/daesabSs/oAbhQeq7zSSrPOkTlWc7rKm/RHrpwp5XXFsG2VnQtq46rPsGpunet7T3tHepD9xwd8aNDt6DCnwMcBO03ux1ydidcXCaPCXqGBV0b6d9GYi1jIWP5iaHnqJK2LcWLCtqU4oVf70pxa5PbJpbvelsw++NxGZqHJvPRZ2jqzYaOyte2939Tcz7yrdGGi0Rt792L9ra9B2I7LxN7JMNbe4n8P6htoj1Se31Ntkd+SCOaBG2hb09GcV/6MuyfXwzkEuDyh7o4/Tzvnw2+rk6V1B8+W7L4m9RZcauCaEW23NdnS4rVx7icVtUHTXPwDw==avocado-113.0/docs/source/blueprints/images/BP003/Task-scheduler-flow.png000066400000000000000000000557341513561306000261110ustar00rootroot00000000000000‰PNG  IHDRP;ē3ÃŨsBIT|dˆ IDATx^í œŽUûĮ/Û4ƒĨū˜JxЈ{öTöFÖloĘŌbI–%{ÅKR”%Ū7K˛&ZˆôÚĒW"QömŦũĪīä™Ķ,ĪcîåÜ÷üÎįs7™šīsŽķģÎķ}ŽŗÜįdĻôĸ@ņŒ3ļɝ;wėɓ' ?>2ŊTÜkõŒˆˆˆĪ‘#ĮžŖGÆ]ž|yĻ˛§×ę^ė͐^*šžë™={ö)/^ėĐĨK— 5ĘRĻLɕ+Wz–Äčē;vLļlŲ" .ŧ0yōä?2gÎ<ũÔŠS6:G€úÛņŅyķæ]Qŋ~ũ‚ãĮ"4ŊįlĀ´{÷îg/^ŧ÷đáÃĩT ~ņ^-ük1ę_ߊ‚įvõá‹0`€k™>Ē6lØ0Q_‚;DK¤{Ŗ–¨7üļ•čļĮÆÆļ™1cFTØķ#hßžũ™¸¸¸™ėΛãÔ_XiIņČČČ­ˆp˛ÛŽ -§OŸN¨GļlŲ¤TŠR2jÔ(Š^Ŋē•õŗ$¯7ŪxC*WŽ,%K– +ŋüųķKLLŒŦZĩ*ŦįŌz3ēķ 8_ZåʼnĨ´ jÁķ¨"š–…šmÖ­[ˇŪ/ŋür'm@Õ°<ũôĶēXÕŨ5 "j&YöîŨkÔĕšá–|ųōÉėŲŗĨiĶĻaÉä@adĪž=/L˜0aŒŌ”ã2ay͞› P{tu5WŒ}ĒŽ^LĩjÕĩEÄšaƄr{÷î-/Ŋô’|ūųįRĄBYļl™<÷Üs˛k×.)[ļŦĖœ9SūņČĨK—4x´[nšE~øa:t¨ž?K–,RŖF AÄØącGŲąc‡”(QB $ƒ–~øAÔ ]Æ˙ũß˙ÉĉåÁÔ6Œ3F˙ûˇß~Ķ‘Ļ‚*TH—H}úôŅQōØąceŌ¤IrčĐ!iܸąL›6M˛fÍĒķ}ėąĮdßž}ŌŖGmGąbŏ@aīęÕĢE Íp,ÔŅ–|a¨!Ž°Ō ĩŽđėÁƒ#ėžÃ~´hŅĸōÁ$D ŪöíÛåįŸĩöT˙0>|¸tčĐAn¸áYž|šŧûîģŌĻMi׎¨ĨVĸ"hũĖʕ+u^É-WŽœ9rDæĪŸ¯#Ū Čūũûuš÷Üsŧøâ‹R§Néßŋŋ¨ņCYģv­%`8zôhiŨēĩlܸ`’J͚5ĨaÆ°öë×OÛ|âÄ ™>}ē|ōÉ'ČUĒTq čÆĢČ9^i™ÕĘ6ÃŧŽMôÚt3ũŠ?TrÜÆÄc 0ŨäW^yEZļl)SĻLŅ‘â’%K¤nŨēvO>ų¤(Øk˜Íš5KwûķäÉŖŖKD Š´E‹:íÛˇ¯Œ9RC‘æûīŋ/ ”ōåËëhō‰'žûîģO0.‹¤–Iƒ ô}čÂ#Ÿyķæé1\DÍ›7—īžûN ÚĐÕ,¸œ;wNE _§Į@͐AlųŲuŧ…˙Ŋ@:Á'Ø`‚kŊéĻ›tT§&:t4YĩjUĩ! 2Dwš(5NĢģí¸īĢ¯ž’įŸ^ €Bš:uĒtęÔ)U€"2Å•ŠēuW Dw\ę2žãĮKTT”.‘hb€"”,ĸåëŽģN!TĒTIÐFŠŽŽv­ ¯ÉI€Úđ‘šļ, ĐkĶÍô§\hđ(Æ4ŅŨũ÷ŋ˙-=ôP1î‰ņĐ@t1Æ8gÎŨW¯›Ę /ŧ ģŪˆ@2@lܸqŠčōß˙ũ:JmÕĒ•/^\žzę)%ĸX\€4Ō×_-XKšhŅ"ųõ×_åË/ŋŧ*E>sįÎÕQ§zķ'!L€!˙@zöėYVŦX‘¨éŸė#@Ų…"Œ(@U¤H}áÕDĩŦJnģí6Š]ģļ_Ä$ Æ*X@õņĮדDčZž˜š@Ņ%GÔ8ĸˆ.1 Č"ĸÅĢŠˆ:ĩŽ_ŋ^wģ1ĻēyķfĘA7C(“D*ė@”Œ‰.€ŗYŗf:BÅīšb’ å.\XΜ9ŖķYētŠū  ŗ īBË6ŦHÔ0‡XdŽE]ŗđ€‰Ž3ē˜9ĮäŒZjĨ….3`‡ ĀîĐŨf]tĖä1BræĖŠŸÁ¸e`ö|įΝzlõ‹/žĐŅëŗĪ>‹×užø‰É%LŪˆj›4iĸ‹|ņl¯^ŊôäĘ{ũõ×å÷ß׀Äl;fėa"dDĮ(SŊœ Ŗ[ĖÎģ‘Ø…wCõ¤Ë$@Íņ…•–¸P++€(Ā ÔĘŧŊžjŽ Ps|aĨ%¨•j–jŽCPs|aĨ%¨•j–jŽCPs|aĨ%ž¨•bø-/ԏ æøÂJKP+Õ4,/Ô‡ æøÂJKP+Õ4,/Ô‡ æøÂJKP+Õ4,/Ô‡ æøÂJKP+Õ4,/Ô‡ æøÂJKP+Õ4,/Ô‡ æøÂJKP+Õ4,/Ô‡ æøÂJKP+Õ4,/Ô‡ æøÂJKP+Õ4,/Ô‡ æøÂJKP+Õ4,/Ô‡ æøÂJKP+Õ4,/Ô‡ æøÂJKP+Õ4,/Ô‡ æøÂJKP+Õ4,/Ô‡ æøÂJKP+Õ4,/Ô‡ æøÂJKP+Õ4,/Ô‡ æøÂJKP+Õ4,/Ô‡ æøÂJKŒ(Ž4Ž‰‰‘›ožųšę—Öį“+G)ã,øīŋ˙^.^ŧ¨OälŨēõ5ŲčôC¨ĶŠ'_jŽ/Ŧ´Ä€6jÔHúõëwÕ1ÆáT4­Ī'W yėØ1šá†ô靰'}z! æx‰5ĮVZb@G­ĪnĮšī8UķᇖáÇëŖ…Zĩjé6#""ôQÅ8į°įĪŸ_ßķūûī˙íųäDJęy§){öėŅy"ūŊoß>žˇß~ģ=zTʗ//ˆxqĖ2€ŠS=‘Į›ož) °Ō?iʋM“|–>L€Z*§1™P¨ŗÜqÂf… dá…úÜuœŨž={v}4q5¤UĢVú(ãŸ~úIräČĄ Æ9ė]ģvŊęųäÔÅ1ÄÉ=Ÿ@Ņ…ĮąĮ°(β/Qĸ„Ŧ]ģ6áúĪ>ûLˇlJ"@Mņ„jŽ/Ŧ´ÄH€ūķŸ˙”âŋëŗՑé!rüĪū#ŅŅŅú\öGyDōæ͛ E0€“čėŲŗÉ>.@ßyį™3gŽ|üņĮē¸S§Néŗæããã%SĻLVúčšķ"@¯Y:Ë$@-—Ôˆ hũúõeÆ :úDēté’ä˗O0ĄƒŽ4ē|ųršûîģeęÔŠR¸pá"Pä•Üķá@účŖČ0ņķŲ˛eĶ3íÃĘ&€úË/ŋH` 4qūŨwßÕ]ú?üĐJXšjŠœiʌM“|Æ>l @˖-Ģ#Ëx@qčĐĄ˛råJ=։(HXæ4fĖ™={ļēŅķæ͓ŋKđķÉŠ¨6šį16ú¯ũKj׎-€cģvíd˙ūũWcŗŖFŌcŗ‡’;î¸C0îY´hQųōË/õ„ÖĢ¯žjŒŗ Pc\Á1Ps\aŠ%ÆĀ7nœ 6Lž~úi1b„ŧũöۂHp{ë­ˇt׸W¯^§Į-žņÆR˛dI Üāį“R 3÷É=câĒ`Á‚ōĐCéY@ņĀ ë@1q„x ) 2 ĖÂc"  íĩפJ•*–:(-™ iQĪÚgZĢ§)šPSņ“¨9Ū$@Íņ…•– VĒiX^¨9!@Íņ…•–ø ŋÄ8eRŠM›6 KŖŦŅÔŧPs]ŋö™\ÂûõQQQŌŖG„[ęÖ­++VŦø›øÅáÇåúë¯ŲĖņãĮkņēgâôâ‹/ĘäɓåÜšsr˙ũ÷ëՁB. Ė Đ0ŗņvÔFq]ĖÚV€ę…EčX×¯5%P,7Zŗf^nôÜsĪĨPŧO?aÂ)]ēt’f`+=,ŽĮûøג’(–] 0@ƒ:gΜŌ¤IŠZĩĒŧđ ×RLČĪ !Keû¨íģR€+U…JĪž=õĻ!X›YŊzuĩeΜ9ÉŨ–°ĶQ0@û÷ī¯ß@ÂķøYĻLŠS§ŽtîÜ9Y€b­&âc#’+`ų›ā‰Š…úˆZąëCąPū–[nŅģ6ĩmÛVC‘õã?Žwe (։VŽ\Y/ĘĪš5Ģ~ģŠRĨJēĖW^yE6mÚ¤ķŗ3 vĒ^ŪhxzyånWēhŅ"éÛˇ¯|ũõ×f÷Ū{¯ŪJnˇ¤@ņæļŋCÔÜÆŗ);(aa>6I.ôȑ#z{ŊīžûNoU‡!Ø:eĘyꊧôĸ~D•x˙ũąĮĶ[Ųa]x,¨G„‰WQ}ôŅŋW¯^=iܸą¯‰ĩSŨđō&@ÃĶË+wģPD §OŸN ātë­ˇęąÉäv[@>ûėŗ˛jÕ* ˇā”@ņļŸėB(î9yō¤~à ā —-[ĻŖRlf‚×BąĄIƌõ=uÃnRØl$qÂûûØųžŗĢą v)~žhøš˙„zŋüėÁƒ#';Sâ1Đß~ûMWbį"|ČŅ%FT‡ˆ.šŨ’`#&€´ožųFw‹Ã(&Ž°ņGĄB…B(†ĀĨK—ęûą‘2ēņØHc؍ “Wx'Ûîū(‰‰"DĄÁ°Æ—FˇnŨd÷îŨzœ›—Ø™°‹žÚÁ*^ \-”…2īd @}Ø8Ô~šÛÕGLĩjÕl­]b€"âdĐíÅĖą36MƎN-[ļÔģ5aį(Ŧ,(WŽœ^Xˇn]Â2&ė,ØâXąA &Ž0QæDR“tԗĖI˙õ­äDÁ,#IP6Œâj­æV5›ag7>1@ąfąąE&Î:čIŒo&ĩÛRđ,<ĸ=t‡1ˆbĸ+ĸYŒ+"Ęđ ;Ü?øāƒWũ.)wG fhĐ `2 ŋa'ōÂĪ.]ēčuĸ(¯ˆĸģ< Ųy|IāoXķ āÃį+a͎„îģÚŠęŧ‚;Ökí´Ŗ æžhxzyæn5“=EuõÚ¨îi”gŒĻĄ)*Đž}û3ęKhĻŠ˜;S*3 @Íđƒ-V`,T-š‰ ƒ´Ĩ fjģØOUEÂû´]éđ @ÃĶËkwG+ˆŽPgTž(;ģķ^Æ+öĸÛŽžĪ¨Ũų÷ĒĄ…ZĘî_ŧb{z°“M^Fw^-Ņé Æ÷24jÔ( Ūđ!LÍu< ‰I8õFÖõžũjŒu:ģífú‹5Ķ/vXU\MŒ´QGôÆĒ…ä…Ô:ÂH; ažiW@MÂÅĢuą{Ô*85ێ÷B9a”vYmɁĩEVfę°¨ōؖÅąŅą øCÔ~ô\-ø­í9—Ņā$ @Ų,\Q€uEvją¨Å‚2ģĐ @CĶ‰w™­jļ|kę[×ĻĢŠ éĘŨæT–5Į´äÚ @¯];>™Đ4ˆĮGQ€5ÆéË4}ųÛ¯ĩ%@ũęYÃëE€î š’hH2ņ&Ģ @­V”ųšĄę†ę,“¯ŋą øBÔnô^%zĪg´øī  lŽ(@€ē"; ĩXÔbA™]h  ĄéÄģĖV€5Û?žĩŽõ­kĶUÅĐtåns*K€šã Zrí  ×ŽŸLƒhÄãŖÆ(@€ãŠôešžüí×Ú ~õŦáõ"@ wÍ I4$™x“Õ  V+ĘüÜP€uCu–É7‘Ø<ŠĀŗĘęaęęĢŽ ę ´›ú˙Qę ŽqžŦö”Œ@=å.{Eęįau]T×uåU×ueUWæ+˙>Iĩ¨€Ũ  v+ĖüíR`¤Ę¸§ē"‚ 8¯ū˙euõŗĢPæK‚ @ŲŧĒĸĐßÔu]PΊ˙ŋQ]Œ>ŊęUŲM€zĖa4÷*‚ŖPFŸlŽ+@€:.9 ´Pā(”Ņ§…Â2ĢĐ @CĶ‰w™ĢĸPĖĘc֝cŸæúɗ– žtkēĒĸĐéęęĀąĪtåw#*K€áGŒ(ž1cÆ6šsįŽ=yōdĄķįĪG:R* [ˆˆˆø9rė9zôhÜå˗gĒ v† pDÔ™Ũ-${öėS.^ŧØĄK—.5j”ĨL™2’+W.wbéÉ*pėØ1Ų˛e‹,\¸đÂäɓ˙Ȝ9ķôS§NuĻdæ)@€šį+-ŠÎ›7īŠúõë?~|ĄiĨ´Îä˜vīŪũĖâŋ÷>|¸–*õgJf)Ą(@€†ĸ’GīQđÜŽ>|1āÍF&/+0lØ0Q_‚;DKxš~ŗõ›G¯ÔŨöØØØ63fĖˆōiĶ]ĩÚˇo&..n&ģķ渞5ĮVZR<22rë"ØmˇRVwķBwž@įãããK+K8ąäŽ;té¨N°Ú5Û>Ŧ[ˇnŊ_~ųå,VįÍüÜU gĪž&L˜0FÍÎs\Æ]W čo‹ ûT]Ŋ˜jÕĒŲ’?3uOÕĢW‹šáX¨{.¸ĒdF †8ÂJ3Ô:ÂŗŒd÷ŨJUÍČ Ũø|ųōÅĢuŧØēÉeP—`Sņ¨dSÖĖÖm2dĐ[~vŨv`€ė1ĩGW#r%@pĮ@Íqƒå– –KjN†¨Až0ĮZbĄļT-Ą‘ŦYŗJŊzõDŊ“`r¯^Ŋdܸq˛yķfšķÎ;ÃĒJūüų%&&FV­ZÖsÁ7wėØQŪ|ķÍ$ŸéĨ—ö§)SĻˆzĩUÖŽ]+UĒTšærŨxuCõ¤Ëä8Š9ž°ŌßôŌĨK’)SĻ͖-[&[ˇnÕ˙îßŋŋĘO?ũ´ūwõęÕåîģīN¸W-’={öČĻM›¤FĸV-XŠŊíy ļKrhČRyęFWĒÖĄ 6,5j”,_ž\>˙üsŠUĢ–ŧ÷Ū{Ĩ˙ũØcÉž}û¤GōÆoHąbÅ"ĐącĮʤI“äĐĄCŌ¸qc™6mšŽzķäÉ#uëÖĩK‘‚€fRIŊ…%ĨJ•’ 6č?Ŧ°gāĀ2gÎi׎Üpà WE S§N•#FČoŋũĻa;}út)R¤ˆ¨MX䊧žŌĪ,XPZ´h!x5e׊SG˙|îšįd׎]RļlY™9sĻüã˙°ĩą ļĘVæhXryæfWĒŪ‚’%JČM7Ũ$¯žúĒ|ú駈č6?ūøãR´hQ9q℆Ô'Ÿ|"'NÔŨhtá,X€uŽv5k֔† JŸ>}ĸJ !´lŲR!¨MRBčŽ;´=jų¨…č:߯žú* j‹?šãŽ;tž @ž>ø ĖŸ?_Ū~ûmQ¯PjčÂĩˇ€üüķĪ˛råJ]\ˆb‡.:tĐ`Ɨ†‰ĩSŨđō&@ÃĶË+wĐΝ;‹ÚŽMöīß/7ß|ŗ<ķĖ3:šC´ŠŪ”ÂærîÜ9Ŋĩ^ųōå5@á͛7ONŸ>­ŖÎæ͛Ëwß}'ÛļmĶŨrĀŨī”Râ4ĐļmÛj "Ū{īŊčjNšîēëtĒŪ7<×ēuk™={ļ¨MĀ›ĩ`-,†"~øá=Ą¤vĀŌ@ĩ3 vĒ^Ūhxzyån[ęŦ˛ķ×_•ë¯ŋ^wËņÅ`bˆĄdɒVV>hXrŲz3jĢŧŽeN€Z$}`Y–7aFcžˆĀwîÜ)Y˛¸ŗŨ*j‘s-Ȇĩ@Dŗ @-r †°t ã›X PŽ\9yíĩ×ôÂ|ˇę–ō/—5ĮVZB€ZŠĻay æ8„5ĮVZB€ZŠĻay æ8„5ĮVZB€ZŠĻay æ8„5ĮVZB€ZŠĻay æ8„5ĮVZB€ZŠĻay æ8„5ĮVZB€ZŠĻay æ8„5ĮVZB€ZŠĻay æ8„5ĮVZB€ZŠĻay æ8„5ĮVZB€ZŠĻay æ8„5ĮVZB€ZŠĻay æ8„5ĮVZB€ZŠĻay æ8„5ĮVZB€ZŠĻay æ8„5ĮVZB€ZŠĻay æ8„5ĮVZB€ZŠĻay æ8„5ĮVZB€ZŠĻay æ8„5ĮVZB€ZŠĻay æ8„5ĮVZb+@qîŅŋĩŊ83SĻLú˙īŧķN}¤o8 'râĖwœŸ8ávėX}Ō&~ķŨq–z‘"Eô™đ´˙ūúly<Ÿ86ŖN:ú¸ää"P#?ũô“\Ëß|” īŧķŽ jÅŲF ÔļāŦ{œsßļm[ mDÖ8Î'GWŽ\YŪ}÷]}ČÜųķįõÉĄH8zyĶĻM:?;j§ēáåM€†§—Wîv ‹-Ō'V~ũõ×f÷Ū{¯Žų-VŦ˜†\Ž9ô1Æ_׎]úŲgŸé/uwņlJ]˛d‰>^Į'—‚ŠŖ”oēé&}Ē&Ž!؊㖟zę) wD•Į×Įŋųæ›zĮYHˆ0ģwī.>účߊĢW¯ž4nÜXƒ×ÎD€ÚŠnxy áéå•ģ]("ĐĶ§O'pēõÖ[õØdtt´>OįŽį͛7AGD ˆŸ}öY}Ö:āœRh¯^Ŋ¤D‰vĄ÷āp8€ ā —-[ĻŖŌå˗˘1cäîģī–Œ3ę{(ę†ŗáüˇĸ† "k׎ÕųžŗĢą v)~žŠtĨʲzøŲō ˇĖėN‰Į@ûí7=^š}ûvÕĄKŒ¨Ũæ͛5@(ĀięÔŠR¸pab@ûæ›otˇ8€bâęÃ?”B… …P !€K—.Õ÷=zTwã?ųä=1†ĄL^:tHúôéŖတ˜(B kčÜ­[7ŲŊ{ˇg͖-›Ũ˛';Ta{Á"ĢT5(Į3E¤P| SģĮ3•MG†ē"âdĐíÅĖ<ÎRØĐ@¸#`„1Ώ>úHÄX)"SL… P@㑀ĨāĀ% IDATWJ)¸ ˆsĈ:ZDŲËĐœ'&§`#†0<1NDÄ_~ųĨŽŦ‘9īßŋ_{:uNŧ‹(y¨ĄĨG æMęēPLôTŦXQCe˖-zVēYŗfz\ŨbDvX….ûŧyķdņâÅ c ˆBī¸ã™5k–ž| ¤”ēđČoõęÕzr*T€Nœ8QGŸ"ĸO )œ:uJ6lØ -[ļ”víÚÉ< Wā xØēnŨē„eLˆĸ[\ëĶO?­'Ž0QæT"@R:õrĐÔ5ōâŽtũúõz;""BO œ:tГ<ßIJDĻ Đë%K–,yÕ,<ĸ=t‡ĸ˜čAĊg0Žˆ(Ā ¤ūķŸōāƒ^õģ¤œb˜ĄAƒ‚É$ü@„Č ?ģtéĸ׉ĸŧ6mÚčî~đ,GŽ{Ž=wųōå™ęž†(E€ †4EÍȞ=û”‹/včŌĨK†Fe)SĻŒäʕËA XT8 ;vLļlŲ" .ŧ0yōä?2gÎ<ũÔŠSÃÉÃĻ{ PÔĻĻefļŅyķæ]Qŋ~ũ‚ãĮ"4ÍtRJVĻŨģw?ŗxņâŊ‡ŽĨîũÅÅZ ¨‹ÍĪáĸ<ˇĢ_Ė€.™ÅY­Ā°aÃD} îP-auŪaäG€ a4ߊn{lll›3fDy¸4=Höí۟‰‹‹›ébwž%@ĶÅg˛xddäÖD°ÛîŖ;_ @ķņņņĨU­Ü˜X"@ P˙| ’Ģ‰šmÖ­[ˇŪ/ŋür˙×6}Õ°gĪž&L˜0FÍÎģ1.C€ ū˙ĀaėSuõbĒUĢæ˙ĘĻŗŽ^ŊZÔЌ[cĄ(ę˙OœZGxöāÁƒ‘ėžûĪ×čÆį˗/^­ãÍęBíPԅfį|‘¨ä|Š,Ņ2dȀrôN(ęp“s§8ÔŨ)•uDæ Ií[Œß8!ÉhÜM¨q.ąÎ Ô:-ĶššVÍ|ŪũęĢ¯äž{îšJÁ\˙žnŨē2sæLyôŅG­ômy ļIvÆhؒyâO4ąĸĨJ•’YŗfIéŌĨĨE‹2oŪ<9}ú´Ž:›7o.ß}÷|ûíˇĸ6ÖõĘĒ¨÷ūĨW¯^2nÜ8Q¯¯ĘĸE‹ä‰'ž>ø@GŦX|^Ŋzõ„1PDžgΜŅ[Æṉ'jđæĖ™ĶÎ%@ÍqjŽ/Ŧ´ÄS}øá‡ĨM›6ēێ(ņÅ_ÔŨl$ŒŽZĩJ˛eËĻ˙­Ëu×]''NœĐđhÕÆÃréŌ%}ũüķĪ:šD„ųųįŸK… d׎]Rŧxņ€žöÚkēëŋwī^Q[ũéÉĒ>úČJũm͋ĩUŪ°2'@ÒË37{  1PL"Ũyįōã?Ę÷ߏ7n¤UĢV2wî\uĒ…đÃ?Č< /ŧđ‚žYG]yô?˙ųtęÔ)!]ąb…ÔŽ]; ˆ6ŖŖŖ[Ä´oŋũļ¸WjŽ§Ps|aĨ%ž(ĀDēŨUģąËûīŋ/͚5Ķ ‡ "e˖•ûîģOš>ųä“:ŠTģíëąOĀGĉč-Æ@ŅāeL€īæ͛¯F¨×_Ŋ•úۚjĢŧaeN€†%—gnö,@Ą0&6mÚ¤ģô1112räHyũõ×å÷ß—J•*éeNj[7=š0–+W› k ž={Vߧvĸ’#FHTT”ŽP{ė1éÛˇ¯Î éwŪŅJõęÕĩÛģg C PsÜE€šã +-ņ@­Ŧp¸y}üņĮz튗fßu$@Ãõļ}÷ öiëfÎh2ęc6Ã˙<~ü¸oŤ”—jŽˇPs|aĨ%h2jb’ kLķįĪ¯'ŧ¸g*jåG%my iĶĪÔ§ PS=c]¨"Z”j‘†eC€æ+Í!@­T3my iĶĪÔ§ PS=c]¨"Z”j‘†eC€æ+Í!@­T3my iĶĪÔ§ PS=c]¨"Z”j‘†eC€æ+Í!@­T3my iĶĪÔ§ PS=c]¨"Z”j‘†eC€æ+Í!@­T3my iĶĪÔ§ PS=c]¨"Z”j‘†eC€æ+Í!@­T3my iĶĪÔ§ PS=c]¨"Z”j‘†eC€æ+Í!@­T3my iĶĪÔ§ PS=c]¨"Z”j‘†eC€æ+Í!@­T3my iĶĪÔ§ PS=c]¨"Z”j‘†eC€æ+Í!@­T3my iĶĪÔ§=Pœ]„Ķ5‘._žŦĪüš Y°`4lØ0dŋųæiÚ´Š>Õ3qÂNô8gĮãĖyœ­„3—ŧ–PsęxŲ˛eĸ*+Ŋ{÷–&MšHļlŲ đ^ŧxQwÕ§1cÆȖ-[dÖŦYVęîH^¨#2‡T’LžģÉ7Å9ī’)S&=vyë­ˇęčsŨēuŌĸE ŅZĩjéč)ЅīÕĢ—†ė’%K$sæĖW9péŌĨŌĩkWYģv­DGG{Κ¨9.#@Íņ…•–ø īŊ÷žLš4IĐG4 @â'‰ŋĄ;ŪŦY3?~ŧėÚĩKĒTŠĸ‹.:Î}NŗgĪ–ĄC‡ĘâŋåļÛnŗRsĮō"@“:ՂĐT%ōä ž(&~šM›6I‰%dßž}R¨PĄ€}Zk9rD ¤uëØąŖ†, XÆôŲgŸé˙ßēuĢ~K—ÖŦY#… ļRkĮķ"@—<Ų Ps|aĨ%ž(i׎Ŧ^ŊZōäÉŖá9xđ`=ëŽh˛C‡Ž3f”{îšGŪzë-Ŋū3xhĪž=eīŪŊR¯^==kņÔā„(7oŪŧVjo{^!4ˇ2äuuMU×'õ‡Ę'5fXT”7˛IM æ ?&ļŌŗõĻÜÎZ @ (kú¨ëIua°x°ēF[d!yHHÔĸ–eX6¨aąŌœdZD•ņœē:ĢëœēŽS×NuYųĒJ€ZŲ”Í‹5Ö5i7,@KĒĮĢĢ†ē.]' 9ŖŽWÕĩęJ$Šŋ!" \×ōī ęųԂŽ´WĐC9¤&ŋq<äĖ€ŠjœīėÁƒ#såĘåAëirJ `ÅAž|ųâÕZØŦWîģIũÜĻ.,„ũs1ėŸ °[§ŽķęÂBX\™‚ū?ņŋCųō¨™Ž(@€ú°)¨I‘íqqq1ÕĒUķaíŌw•0ĄģãđáÃ%)ņ„ú÷‹ęĘŽŽ(uÅĢëmuĄKĪd“¨MÂē™­š•Ļ^ėũōË/gqĶ–mŊjUÁ… &ŒQģV H&÷gÔīĢ c )"Ô_Ŧˇ„9BÔŸí ¸zĩqĢzõ1‚Ũx˙8Ũ÷ œ/­j… ĸä>טPBD:W]mũŖ‚Y5!@Íō‡eÖdĪž}Šęęĩ™1cĸ&(Đž}û3jhfæŠS§Bí–įSÕ>čƒĒ[ÔXפŨ0Œ…vīŪ=fĀ€äz{i/ƒ98ŖĀ°aÃđŽRcŸÎĀR’T€õwÈV]Qŋ~ũ‚ęÃÅîŧ÷œnģú<Ŗ6?ŲĢ&ŽjŠp<Ķ 7 9Ã.SНW;uPûcfhÔ¨Q–2eĘaj—ÚiĪĐÄ^ĨęuÕ “'OūCŊĮ?=Œn{Ú `!+@€†,•įo,ŽfįÛäΝ;öäɓ…Ô:Âā5ƒž¯œŸ* ÖņÆįȑcÚ…?NÍļc?žÄFƒÕīūÜEÅŲ´JĮu Aš Î6@–FŧŦ_ŦIä=ÔË͙ļSg @ Pg[KŖ>R€%@}ԜY*āŦ(ęl‹ciTĀG  ¨š3ĢBœU€%@mq, øH”õQsfU¨€ŗ  ¨ŗ-ŽĨQ)@€ >jÎŦ pV”uļÅą4*ā#PÔG͙UĄÎ*@€ Îļ8–F|¤J€ú¨9ŗ*TĀYPÔŲĮŌ¨€ @ P5gV… 8ĢJ€:ÛâXđ‘(ęŖæĖĒPg @ Pg[KŖ>R€%@}ԜY*āŦ(ęl‹ciTĀG  ¨š3ĢBœU€%@mq, øH”õQsfU¨€ŗ  ¨ŗ-ŽĨQ)@€ >jÎŦ pV”uļÅą4*ā#PÔG͙UĄÎ*@€ Îļ8–F|¤J€ú¨9ŗ*TĀYPÔŲĮŌ¨€ @ P5gV… 8ĢJ€:ÛâXđ‘(ęŖæĖĒPg @ Pg[KŖ>R€%@}ԜY*āŦ(ęl‹ciTĀG  ¨š3ĢBœU€%@mq, øH”õQsfU¨€ŗ  ¨ŗ-ŽĨQ)@€ >jÎŦ pV”uļÅą4*ā#PÔG͙UĄÎ*@€ Îļ8–F|¤J€ú¨9ŗ*TĀYPÔŲĮŌ¨€ @ P5gV… 8ĢJ€:ÛâXđ‘(ęŖæĖĒPg @ Pg[KŖ>R€%@}ԜY*āŦ(ęl‹ciTĀG  ¨š3ĢBœU€%@mq, øH”õQsfU¨€ŗ  ¨ŗ-ŽĨQ)@€ >jÎŦ pV”uļÅą4*ā#PÔG͙UĄÎ*@€ Îļ8–F|¤J€ú¨9ŗ*TĀYPÔŲĮŌ¨€ @ P5gV… 8ĢJ€:ÛâXđ‘(ęŖæĖĒPg @ Pg[KŖ>R€%@}ԜY*āŦ(ęl‹ciTĀG  ¨š3ĢBœU€%@mq, øH”õQsfU¨€ŗ  ¨ŗ-ŽĨQ)@€ >jÎŦ pV”uļÅą4*ā#PÔG͙UĄÎ*@€ Îļ8–F|¤J€ú¨9ŗ*TĀYPÔŲĮŌ¨€ @ P5įđĒRPÉÕDÍļëÖ­[ī—_~9‹˙k›žjØŗgĪ &LŖfįŨ—!@ P˙ā0öŠēz1ÕĒUķeĶY W¯^-jhÆ­ąP”õ˙'N­#<{đāÁHvßũįktãķå˯ÖņfuĄv(ęBŗsžČ?TržT–čˆ2d@9ú?'”u¸ÉšSęŽîŽ”J€:"sH…¤ö-Æoœd4î&Ô8—Xgj–i͉MĢ‚f>o4@úé'éÕĢ—ŦYŗFŽ?.ˇÜr‹tęÔI˙i÷îŨ˛|ųréÚĩë5ŠûÆoHåʕĨdɒa=Ÿ?~‰‰‰‘UĢV…õœĶ7 N+ž|y¨9ž°ŌŖZŠR%ųöÛoeȐ!rã7 €ˇvíZY´h‘¨×N5H—,Y"Ûļm Kĩ´GYM˛ČėŲŗĨiĶĻa=O€Ļ*{¤Mĩ‘øáŖzŨu×éņĶO?ÕZĢ˜UĒT‘‘#Gʛož™āƒŗgĪĘW_}%Ī<ķŒėØąC ,(¯žúĒÜ˙ũ˛uëVÁÆ(”9sæH“&MäĨ—^JxļOŸ>2jÔ(;vŦLš4I:$7–iĶĻIÖŦYåķĪ?—Į{LöíÛ'=zôĐ /VŦ#Đä?(ę>ĻZŖZĩjUYˇn<ōČ#Ō AŠQŖ†Ü|ķÍēRčŪ?øāƒĸ–éČ{īŊ'e˖ÕC¤:eĘŨ­ßŋŋ†áΝ;ĨD‰:âT Ėåžûî“ 6hŽ=ZZˇn-7nÄēI Ųš5kJÆ `íׯŸ-ZTNœ8!Ķ§O—O>ųD&Nœ¨!Î.|˛í‹%@S…n0 ŋūúĢØG}$GŽŅz×­[WŪ~ûm Ãģīž[ÔëŠē åXj Q¯ĻŠzŋ_wīĮ'ę5UÁšH´mÛļúY$ĩk‘†ōûīŋ¯ģđ-Z´yķæÉéĶ§uÔŲŧysųîģīäƒ>ĐŅĻzc ›tČšsįôå˗'@†ĖŽ†,•§n4 %1f HN:UwąŸxâ yũõ×˙P@sÖŦYĸ6–K—.éëįŸĩŠ†čāÁƒeĐ AIŅ-"ĘlŲ˛éŋ#˛Å²eËcą2čÛˇ¯ū[tt4ģđ)7sF Œ@=Âk5ÖX€îÚĩKa>đĀz QĻځ] ` Ž@—.]Ēī}á…4(Ņ…GW> /žøĸļėKļjÕJæΝĢŖNĩ1q‚ž€wņâÅ"PŒĩ"­Xą"#PF !,•§n4 č˛˙ã˙ĐbĒō3ßXΤŪŨ—#FčŽ=ÆHą#û;īŧŖ#FtģŸ|ōI \<ŗgĪ ÅŌĨKë4 +VŦÚĩkKģví¤wīŪœÍš5“įŸ^˙3˙WÅ0@á…åĖ™3zR  ÆO@œc ÉļuF Œ@=Âk5ÖX€ĸB›7o–ūũûË_|ĄĮ&ąôŅGÕŋ˔)“žÔÁŦģÚEO uėØQÆråĘéņʇ~X1b"*1@‘_õęÕõēū˜uŨĒŽŸ-+đ¯ŒPԆfe^–ž(“Ê›H8-€¸ Y°`4lØ0d…ŋųæ}ü÷ß˙ˇg.\¨Ī>™ō8g#ÅÄĄœˇ)7^ŅĻ˛g¨ēŽsĐŽUĒŦ–g|Q5ŪE×d §\C.WĒTŠ„ŗÚí}rŨŋŋÜ~ûíōņĮKųōå5°×¯_/Ÿ~úi¸E¸~hVeTÔÃėŒ@]¯ģi ĻyÄ{|Đ/ŋüRe|ôčQ‰ŒŒ”‰'J5ôqĮ8Qŧté’TŽ\YKŒĶ8č… ¤N:ōĐCIëÖ­eãƍúDO$œ áī’={IDATŠČįË{-%3Šc YÔeרפ˛Ũ^Ôv‰])Ā7ŊëŽģ¤[ˇnŌļm[™3gŽ>×}ĮŽ2ū|}Tņ˛eËDUVŸߤIɖ-[@Ū‹/ęŽzâ4fĖ}öüŦYŗ\qPZ Me~Ę;ˇēė˜…O‹Ųž|–õĨ[Áôäŧ—wáqū{DD„>/c—ˇŪzĢŽ>q&|‹-4DkÕĒĨŖS¤@g²K–,‘Ė™3_%ÄŌĨKĨk׎˛víZ‰ŽŽöœH.ŽõœVvL€Ú­°;ųû īŊ÷žLš4IĐG4 @â'‰ŋĄ;ŪŦY3?~ŧėÚĩKĒTŠĸ‹.úĖ™3¯ōĀėŲŗečĐĄ˛xņbšíļÛÜņNK L°Ĩ1>žēĢÔ-)Nš Š‹čÅ;|PLür›6m’%JČž}û¤PĄB 8æØącŌŧysŠ[ˇŽÔŦYSj׎­ŸÁĪŅŖGëŽ=fá¨ģũųķį÷ĸ_ĩÍ!D ž­›a†§ēl‹5Ėc™ã €nÛļMOœč†÷ë×OƎ+§OŸÖãšGŽ‘Aƒ0ä'ŌącG Y@30‰ôŲgŸé˙ßēuĢ~K—ÖŦY#… ļHfw˛!@ĶuLjŗ ō@!iģvídõęՒ'O ĪÁƒëYwD“:tĐpĖ˜1ŖÜsĪ=ōÖ[oéõŸÁë@{öė){÷î•zõęéY{Œ§'Dšyķæ5Ë{ŠXC€:æ.Ô1ŠÍ*Čŗ5KF3­!@ķ ę˜ÔfD€šåK­!@-•3ĨĖPĮ¤6Ģ Ô,Xj jŠœ¨crz¤ 5ÎwöāÁƒ‘šraW3&?)€ųōå‹Wkaņú&“Ŋ 0ĩW_3sW“"ÛãââbĒUĢfĻ´ęšĀ„ZllėŽÃ‡—¸æLø`¨  Ą*å§ûÔŦô0õúcī—_~īE3ųHĩĒā„ ƨ]Ģø¨ZĻV…5Õ36ÛU\ŊÚ¸UŊúÁnŧÍJ;˜=ēī 8_ZģĶÁĸĶkQhzõ|öėŲ§¨Ž^›3fļ9K¯RøĻŪíÛˇ?Ŗ†ffž:uĒŗo*evEPŗũc¯u íŪŊ{Ė€ėíŲĢ´ũš6 īúsėĶ~ŠƒK @ÕÛ¸ŌĸDWÔ¯_ŋ úđEą;oœR5Ũvõ%xFm~˛WMÕRü’ęCŧÁ*PĢ”ôr>čÎĢŒ:¨ũ134jÔ(K™2e„05×Ŗ€&ö*U¯Ģ^˜p"Ģ@¨€;  îčÎRŠđ¨œČ*P*āŽ¨;ēŗT*@| ę'˛ T€ ¸ŖęŽî,• P(@€úĀ‰Ŧ î(@€ēŖ;KĨTĀ  >p"Ģ@¨€;  îčÎRŠđ¨œČ*P*āŽ¨;ēŗT*@| ę'˛ T€ ¸ŖęŽî,• P(@€úĀ‰Ŧ î(@€ēŖ;KĨTĀ  >p"Ģ@¨€;  îčÎRŠđ¨œČ*P*āŽ¨;ēŗT*@| ę'˛ T€ ¸ŖęŽî,• P(@€úĀ‰Ŧ î(@€ēŖ;KĨTĀ  >p"Ģ@¨€;  îčÎRŠđ¨œČ*P*āŽ¨;ēŗT*@| ę'˛ T€ ¸ŖęŽî,• P(@€úĀ‰Ŧ î(@€ēŖ;KĨTĀ  >p"Ģ@¨€;  îčÎRŠđ¨œČ*P*āŽ¨;ēŗT*@| ę'˛ T€ ¸ŖęŽî,• P(@€úĀ‰Ŧ î(@€ēŖ;KĨTĀ  >p"Ģ@¨€;  îčÎRŠđ¨œČ*P*āŽ¨;ēŗT*@| ę'˛ T€ ¸ŖęŽî,• P(@€úĀ‰Ŧ î(@€ēŖ;KĨTĀ  >p"Ģ@¨€;  îčÎRŠđ¨œČ*P*āŽ¨;ēŗT*@| ę'˛ T€ ¸ŖęŽî,• P(@€úĀ‰Ŧ î(@€ēŖ;KĨTĀ  >p"Ģ@¨€;  îčÎRŠđžhËüđCiÖŦ ZÁÁHÔ;UŨö]#GŽ,ĘČĶôĻLû¨€; íׯßn՝/搞hßúõë÷[´hQN‡„a1T€ xP œXŧxņHeēKŪ¨šm?ŠfŪ˛sÂȃ-š&SĀĒZ‘sJÍÎįp XO´eõęÕ'¯\šōzaT€ x\5j_ĩjUU ģ׉šP5ö§Æ>æاĮ[5ͧ)pe,t ĩšHķš+WŽŸW¯^}3ģī67fO|ĸēņÕĒUÛwėØą[lŽ’ųUK—Ίo’žadsS`öTĀ' ā%ÕsŒĻëlŽ’ųUüĄ’Í:0{*@ü¤@† PũjŖ¸Ėš P— @˙ž¨KÅR¯*@€ ^mģ vcÜZm1øˇzėŪŊ[>ųäéŌĨ‹Ŧ]ģVĒTŠ’b]ķįĪ/ĨJ•ŌĪ¤”BŊ/5a‘OLLŒ¨Ĩ0ŠŨĘŋĒJ€Ú4C7 ͛7¯<ũôĶW=ÔącGQƒ˛iĶ&Qköô=)%õf‰\ũõRĩjU4tųĶõ(ęųŠČû7&NSĻLIˆ@sæĖ)eʔ‘QŖFÉōåËåķĪ?—ZĩjÉ{īŊ'QQQYĒ7Lôs€ęšsį4Tß|ķM)P €žīÎ;īÔyMš4IīĢ6ŸŅŅ$ŌØącõī:$ؐfÚ´iĸŪrĶå=öØc˛oß>éŅŖ‡ŧņÆRŦX1F n(ęáæû§éhŅĸEåƒ>¸Ē.… ’`€Ūpà RĸD šéĻ›äÕW_•O?ũTƒ÷tęÔé*€‚}ûöėžƒüÛĩk'M›6•qãÆéû.]ē¤˙­–ąČˆ#¤yķæÄ ,ØØX8p ÔŦYS6l(}úôÁæÚÆ'NČôéĶõ0Ú.Q+° īŨ&H€ ŪmŊW,On ËŌ’hįΝeōäɲ˙~šųæ›å™gž‘ &\Đ1cÆhđáByĪ=÷HĻL™t‰("VėZŽPŖëŋeËiŅĸ…ŽF1&‹¨`ũîģī4ÜmvëÖMƏ¯ŖZõō†¨“Pˇ@”õpķũ+ET9zôčĢęđ%Đ_|Q  X#G}âžā.<"Eŧ֋ˆR-”–ččhÁĢ{õęÕĶ÷Ũ~ûí˛bÅ ]ŪŨwß­ķÚąc‡kEDŠÎÔŅ;ūŧ\wŨu˛lŲ2Q§ˆz]XGļHȓ]xo7?”õv žŌ…e 4Ѕ Ũôøøx=ƒČč˙ûßĢ@› ­Zĩ’šsįę¨3sæĖ ÚĒÃĨxņâ (ÆXVŦX‘¨‡[ J€z¸ųūZ Ptķ—,Yĸ'ŽĐīŪŊģ.ė‹/žH īŋ˙>v,—įŸ^j׎-C† ‘˛eËęąĶ… ˙3gô¤ŌŌĨKõODĨõn$@ PīļŪ+–‡: NŠ™ōĮ\>ûė3Aôˆ1Đ×^{MĪö'^܅Į¸+ēé¯ŋūēüūûī˜mĮ8ŠÚ*QĪÂciÆVgĖ˜Ą'Ą0;ĪäMPԛ-—VS @ Pš!M ŪT€%@ŊŲri50@”5 Ō*āMPԛ-—VS @ Pš!M ŪT€%@ŊŲri50@”5 Ō*āMPԛ-—VS @ Pš!M ŪT€%@ŊŲri50@”5 Ō*āMPԛ-—VS @ Pš!M ŪT€%@ŊŲri50@”5 Ō*āMüPđ}Ŗē„āĩî!ÜÆ[¨ *āw€ŽUuŧ[]ÛÔõ‚ē>NÁņ(?T€ „Ĩ€ßZDŠąK]8ö”ēvĢkē%ĄVĶáÍT€ x  €`ue ē˙Küģ1ęwÍÔqÅŨ'ÕĪ˙Škšē^ jé #FŒ^@0.úŦ4€+@áÛ°aÐ?ß|ķ4mÚTž˙ūûŋ=sņâE}°ÚØącåĐĄC‚ķˆ’K/Ŋô’ā "ŧ–%K–„ķ×÷ĮÄÄČļmčDˆ>O˙ƑÅ8f8`~rq!ŸŸūYŸûœ>ųäŠS§ÎßĘĀ=xŋ˙ūû%"âĪæ’7o^yđÁ'y(P É*=zTú÷ī/~øĄ>|XnŊõVißžŊ>÷'yė œ?Č÷ã\$ü NuëÖM82'ƒÎŖĮ=ČG+ãpģŒŅÜE›8ŗŠD‰úw¨÷{īŊ'UĒTIČēW¯^ōÖ[oéķëošå–ĢĘÄz¨ãæ͛õiĸwÜq‡ 8Pxā}_Jēá„ŅĒUĢęķî+T¨ŦŸũø¯tĨ˙>u]V*ņ3p˙;Šŋᓀ+{/Ē˙?ŽŽāOuēhp#~ôŅGõ!isĘÃmā)€ģķÎ;eØ°aōë¯ŋ& P€RÜäĀ°mâĉ`<8:822R˙)€âÄͤ Eāīø‚ųņĮeøđáō駟ĘĻM›4PƒÎ}ĮŅĀë¨QŖôáqT—.]ôyņš3gN}Œ1@T˛dIEr Î œ?ūU`a#ü‡€ 4H–/_Žˆ” 1ĘíĐĄƒ`Ŧŋ_ŋ~ Å,Z´HÚ´i#¯ŧōŠ4hĐ@ûļ=ũôĶ:ĪōåËkģSŌíÛoŋ•ØØXŲž}{’_Náļ-¯Üī€ĻEĪÔà Ždp^ũÄ×ö+ę~ĸŧĶ=@ŋüōKũÁ &ĀĒFHā4Éõë×ëŗŌ+WŽŦäŨšsgBzáÂŲ=ôĐCŌģwo\P@1%€vęÔIŸ—ŪŗgĪ#Į€“îŊ÷^Y=ZjÖŦ)͛7ˇ  Á:\ˆŌ‚ĶģīžĢŖĪŨģw'D­øûž={tD nģí6ɑ#‡ŽĐåãHc+Šŧđ…€čžÂ‡:1@˙ũīËŧyķ=D¸ˆĪ–)SF:vė¨œöîŨ+7Ũt“bjÅsđ=@ܲeË´|^=õŦßZNyã3uĄ[ŸE]Ž€ķp^J÷ŊëŽģ¤[ˇnŌļm[™3gŽ>Ķ|ĮŽ:Â1Ŋ˖-ĶŅ Ų¤Iɖ-[@^DZ8Â78ĨPä…#‚qŦo‘"ERč˙û_yâ‰'dŨēu˛víZņ €„Ú…5 ŽĮäɓĩ&kÖŦšĒ~8ú_68ō8qĒV­šxŒ3`kĢV­tw;9P,\¸PŖ{œ&Dp ø[Ķ‚fˇß~ģî.C3ÔoÕĒUúŧy ?üį?˙Ir 48ŖäēđčĒŖĢtâÄ yøá‡u„ķ (zĐQ*ÆM§OŸŽ‡#ũŸ~úI; đ ˆP1ū‹û 'Ö ˙†65ŌŅwzI~h”rd~uũ‚CĶ=@Ņ5ž4i’ū0áÃ@â'‰ŋáƒÛŦY3ũaŨĩk—îÖ¸ˆ¨šÄ)Ĩ!čōa(  øpOØ>č,XPN:•0é¯ģ¤čūÛ1\—ũë_:bDœž|ōI=‡5qBŠî,ēĮ€âŒb(CIM"…Đā1Pä…(åR0@a;4 |ņáË_”ŋüō‹ū‚„ļø2Ã&&ĒĐđĨņņĮëz‡"? W "N/Éī Įé û÷ī׍đÂXētˆL yėØ1Ũ-Å8Æ ­āüĸ$ēöÁ)%€nܸQ$hRŗįø O˜0AGš˜cR ˇ øĸ@Ũž}öŲĢę§ĮøáŨM$Ô2&W c0@+LĖ`r C‰gáÃ(&¸Zˇn­Įaŗg˙sŽ4 ( ÚĪ#R„v˜ėÃ$XũúõõXnpÅÜšs Đ(B€ū%Nē(ēl˜(8=t;ąéôéĶz\ŨlĖô"!ŌdÍĀ2ĻĪ>ûL˙˙Ö­[]đ@J  ˜áÆLuâ.|R´a"¯ā8}ôŅGō2ĻpÆ@U ,?Âėō˜1cŽ˛='OžÔŅ`ôûīŋëŽ0žd­ĸŧ¤ŠĄtáĄĨE/ Į¤Öx"Đŋ ô›2eĘULƒÁ— ‹ē”˜ˆ‚Ī1ŽûÔSO%Ė¨§´ĪbežhąÂÃ=é% é &4r€‹ļ_&o+€aLœÖ§zģ6ĄYO€ Ąĩ›îÂø'ŪŦÁB`’ÃĻĸ˜­ ŌcŠYâ•6ëzÖ(ęz#D÷ct˜č`ōž˜ąĮđƀ1!•žJ€Ļ§öÎēRK @ PK3ŖéITšCiņ„ēÆĨQėņˇ)“M)ū1…‡úxē[ÆĒ0^ž/xųPđÎFxO˙Ģ¯ž Ģjxåcĩ‰wxJœ Ū—ĮÂzŦ@ĀŒ>f§ë3Ã*7¯@28ŸW _R×_ „¯­FčĩéƧŦRKŽ°æo/]k  XX×:WŦXĄ—@aņ=Æ[^kŲ|ÎL4ÎW"FĀÛ^ĨupŸ5ĶũéĮĒÄÅO,ÂWŧŠYŊzuŊÃfąkYņ{lv‚WTąN5 X÷‰uąx>°/'ÔDT‹×#+UĒ¤ÅÅÄ Öh&õškúQßŋ5ŊМW"Î8{cĶĸä7 ]4t­x§ $(ŪÂÁ>¨X8ŽŪ'ĮÛWxģ ¯`bŅ?Ū.Âģęxˇģk׎ Å[Wxm;3ĨÖ5Į-ŧ.‰ D˜ü§Ā€ÆŠša'ō?wwą>­RYÖH)[ŽZ/:s R Š¯Šˆ­ŪđQ=ô[O#GŽė´ŧ2"ĐwŪyGŋa„]•RÚū EãÍl/‡ H‚ŖT:Æ? ¤b›ąĢwāļŠę¨MÂ2Û?H Đß~ûMž{î9Ŋƒ:>؂ ¯.bü›” ØīęcŸË… ëīÜ#2Å{ņÁ›‡ëŒáėĢŠÍ=đZcâ÷įé˙(āĐhĒ‚ ŠJÄŌĸ@b€"âÄ,ų›ožŠˇãÃģëؘ$x_O,G‰ąNlX€âŊuŒ•"2Mná?"Tŧ€qĪĀžiąĪšĢ€CŗđŠ @€Ļ*oH‹‰ŠMPđÖ `‡]„0[Ž}N1^‰7ŖfĪž­_-E—ģ&aSŽĀ$ĸPŧŋ?kÖ,=ųœ°ąļˇÃÄQzzĨ1-žņōŗ­MU"4U‰xCZH Pœī„ŊHąą0&NllōöÛoëņM,GBdŠŨŸ°Ž˛ĪÂ#"E7Ņ)ēô„­Ö`xbWzLV1ųO‡ŪDJU84U‰x Ļ)@€ū垉dZë¤=TĀpPÔđ&Jķ¨€š  W|ŖfKĪŠŖ%"R[mŽ+i N*€SÔé įÕj?\p1š>Ē&~V3¨7—.]ÚEX4 ^QˆęäÕ}ęȔ[ÜļŲu€Ēo’8ĩxúaËD¨HMŧ`Ą^˙] zŽąŠŨk÷ß]¨Ē`KĩĻo˛:Ąđzģ+ËüŠđžę”ØãjÉ[U“9n×Æ€âÕŧ“6lČÎnŧÛ́åSŗ@÷][}Jâú×"`M6 Ēū}ÕɂũÔN=؞Š‰ P*¤ 48ĄŪNŠū8ʉL(fÕvŠąĐĸ 5ĄYĐ*`žWÆ>wĢąĪbĻXg @• eՒĻ ęũįŧŪĮD¨(€ÍdԞ XēTAũnŗ)ʘPhRGAtņĉ#‰šŌDhpWDžjËCĀŗž˛dšģÖ\]ēi…ueUw~Ž:š!ßđáÃsrbɤæB[¨€s `ÂHárBm@sPuۛ›yT0 ÛúĒŲųūå˗ŋÔ˛eËëÕĖ›>‡o,9׀YpRŧaôã?ŠZ‘#sæĖ9žqãÆLjļ}¸˛Áˆ Ŗ¤´0 {[Ēˆ´Š:hė^uÄ˙Š0>pp”“žeYT€ ØŦ€ž;¯N8¤ŽaųBEœķUqŽ¯ķL­Ę˙įîŌvÃÉIENDŽB`‚avocado-113.0/docs/source/blueprints/images/BP004/000077500000000000000000000000001513561306000215435ustar00rootroot00000000000000avocado-113.0/docs/source/blueprints/images/BP004/current_requirements_resolver.png000066400000000000000000000431241513561306000304630ustar00rootroot00000000000000‰PNG  IHDR1y÷nisRGBŽÎéĻtEXtmxfile%3Cmxfile%20host%3D%22app.diagrams.net%22%20modified%3D%222021-09-02T21%3A01%3A06.926Z%22%20agent%3D%225.0%20(X11%3B%20Linux%20x86_64)%20AppleWebKit%2F537.36%20(KHTML%2C%20like%20Gecko)%20Chrome%2F92.0.4515.159%20Safari%2F537.36%22%20etag%3D%22QXCnKVXvrrFj1-GO3PlT%22%20version%3D%2213.9.2%22%20type%3D%22device%22%3E%3Cdiagram%20id%3D%22Q5DIlvNph6NFuipN24rI%22%20name%3D%22Page-1%22%3E5VhNc5swEP01PqZjBMb20bHTZKbtNBNnmqY3GTagiUCOEDbur68E4tuJkzQMbn1Cu1qtVm9XTxIDcx4klxyv%2FW%2FMBTpAQzcZmIsBQsbQnMiP0uwyzUhJSuFx4mqjUrEkvyEfqbUxcSGqGQrGqCDrutJhYQiOqOkw52xbN3tgtD7rGnvQUiwdTNvaO%2BIKP9NO0LjUXwHx%2FHxmw55mPQHOjfVKIh%2B7bFtRmRcDc84ZE1krSOZAFXg5Ltm4z8%2F0FoFxCMVrBmwmdvgjseOY315%2Bv%2FlydXfx%2BOtMe9lgGusF30Ik3c2kkoWgIxe7HA7O4tAF5XE4MM%2B3PhGwXGNH9W5lAUidLwIqJUM2KV4BvWYREYSFUufISIHLjg1wQSTGXxsGAXFdNdM5psTbO2KmOwrLBxYKXTWGLWW9HmkOybNAGQX8sm6BBSD4Tpok9dLb1cVtmX4jz6lfSb2tdVhXnFc4LpMiGzovb8iR2crRDTzFhEOQLmiWLs55VIWMhivM%2Fy5pVUCtTgAd9w2o9ULRiy07zaKfHlnVj%2F75qp8eWdmPW4i2MJNn1Fo1HygkM3V6SiggdHVz4VAcRcSpQwcJET8Vyp9GWrqv9CwSnYBU2Gkhmxjc1tnbAFcGx2LuwCF2bCehgvJoD8i5jgPFgmzqYexDXs9wzYgMsMixWU8xmjZyl0WvB1XP5oafccPPsOFHYO6BaPlJy6BY9PsrY9ImRBw9ppvMphKx8xWv1Yn9FKtbS7przqJ026j9aNjrJE1u3i9bnvq%2BdqMab9uo3Zz3aIh63qh5QJ3lQ6jj7tAF7yiSYaLek9G%2BfrVA64IiJV58VxmkxPtqXzkslTqgVk0LGfkcrNe%2BOBhZjf1rofeRMGpegaYNRx2zsLHnXnpKNDw9Oh5u30E74eGX3xzHkY0jIGL7ZIk4Z4bDTDzqk4kt9EFMbI17ZuL%2F86nUa3E03ziG9c63UhHQhz%2BWpFj%2Bjs3My5%2Fa5sUf%3C%2Fdiagram%3E%3C%2Fmxfile%3EŖ0î IDATx^í ÜUS÷Į%Dƒ ‰D(CÉĘĢâ5•H”!Š„"Ņ A¤I‘"SũK” I2•ĖCĻx)Ķe*BƒūŸīöžë>ˇûtĪŊ÷ÜsĪ>{íĪ§Oõ<ûėŗ×Zŋũ;k¯Ŋ÷ڛmØ°aƒhQ ¨T–j`3%1K-§ŨV ¨Œ$ļŲf›ŠJbŽ—œnÅsĖÁ,"žK˜K ŋ‰KJČ vÉžŽÉë2ž•ÄąžkƒÚ5yqBĖdû*‰9b}×ĩkō:c%1× ,¯kƒÚ5y]ÃļzbŽYœ‰9hõøŠŦ$_ۖ*™’˜ƒFąČJb16niĸ)‰9hô‹Ŧ$cã*‰ũ­×HÛ5H+‰šfqĩ’XŧAŽ$oûĻ•ÎĩAíšŧŽAZIĖ5‹Ģ'æ Åã-˛’XŧíĢž˜ƒ¤í¤•Ä\ŗ¸ƒƒZ§“ņš’XŧíĢž˜ƒ¤í¤•Ä\ŗ¸ƒƒZ=ąxƒ\I,ŪöUOĖAŌv ŌJbŽYÜÁA­žXŧAŽ$oûĒ'æ iģi%1×,îā VO,Ū W‹ˇ}Õs´]ƒ´’˜kwpPĢ'o+‰ÅÛžę‰9HÚŽAZIĖ5‹;8¨Õ‹7ȋNbŸ}ö™ėĩ×^›ÔōŽ;î(˖-‹ˇ%B”ÎĩAļŧŠéÁœōQ.ĘmGŋ˙ūģŧōĘ+ ŠŸ{î9ŨË[oŊe~_ĄBš˙ūûåŅG•Į<­Ū×ŦY# 4˛eËĘčŅŖĨzõęōđÃK˙ūũeƌrōÉ'‹˙8đĀåÖ[o•Ŗ>Z˜ŸtŌIŠŸū‡iøbę0e*‰ųÁÄąĮ+üņ‡@PÛmˇ!ĻË/ŋ\f͚%76mp Ļ!.ÆÁÕW_éôüÍ7ßČ.ģėbÆĘŲgŸ-]ģv•å˗ËĶO?mĒƒYˆŠ1öíˇßšŸ}úé§æãũöÛo›vĻ1QLûF–ÄŽŊöZ3fŒ0%Ŧ\š˛Ņ_)b[|åÎ:ë,9í´Ķä—_~1∠O`¤O<1Ģé$ävꊧš˜Ũ‘G™h÷C‘*UĒČėŲŗ$vĶM7%žfT„(éÄæ§˙a^I,Lm‹ÁhjˆÄ&ĀŪį%Ņë… Ü/fF0iŌ¤ŦĻ“âÖ[o-&L0ĶI>äW\q…üôĶOfŦ\zéĨ‚‡ÉŊûîģ˛Į{Čøņã a~÷ŨwæãiL„ĢŨ’žvdIŦYŗfō×_ÉŧyķJč‡UMˆ‰iÆčĐĄƒņ|øsĖ1Į˜¯—Wr!1⃠<2oZH{zęÔŠfÚęybžgæŊĸÃ;ģûîģÅO˙Ã4ŧ’X˜ÚNOb~0qÁČ#ņÄžHĖO˙3õ%Čß+‰ŠÍĖmĨ#ąL˜Xĩj•ÁŪQ2æ!ŋ]wŨÕÄ´ 1Ļ›,ø-^LŒqgEaq‹…*ÆáŪO8†Į]wŨeVá)~Æ„ßžUĪŠ}bŪJJ& J`’)$ËĖlšĀ5g ÅęÕĢeȐ!†`ørđ…9蠃ĖŒ€(DÄô˛I“&R¯^=CŒlĩ˜6mZZ}â×D,,T­ZÕ¸×ëY)"ŽáĮķĶ˙  ę§%1?Z ŽN:˄ Ŗ ĢV­Z™DfŦB'9r¤ųû–[n11*B`ž-ŒV)“ˇ_¤JÂļ ČoŽUļUĢVMØF1lØ09ũôĶÍ#ŒŸūYZ´haˆâgL§9-YAbˆÂ>+ „WD§÷ßcĀN8ÁHJmË?ū(ėÎĮ‹bŸ ž ˛8Ā6ˆ§žzĘ×>1–¤ģwīnö1M…üx/{Í(~HĖO˙ũ™+˜ZJbÁčŅo+›Ú'ļ)LŋūúëBüõ7Ū0Øc–ŅącGƒG ęã?ŪÄŽØ.3íã9VŌųŗŌŋhŅ"ķ÷a‡&ŧ}bŪIHŽ>ŗƒW2 ŋz Ē^äH,(Á´Ō5 $Ļ舓”ÄâdMŸ˛(‰ųT”VŗBJbV˜)ØN*‰ĢOm­¸P+Žū‹ōv%ąĸ¨]_Z (‰HąQnVI,ĘÖŅžeĢ%ąl5ƒúJb10ĸŠĐ€’˜ƒ`PsĐč1YI,ÆÆ-M4%1c‘•Äbl\%ąŋ5āiģi%1×,îā V‹7ȕÄâmß´Ōš6¨]“×5H+‰šfqõÄ´xŧEV‹ˇ}Õs´]ƒ´’˜kwpPët2Ū W‹ˇ}Õs´]ƒ´’˜kwpPĢ'o+‰ÅÛžę‰9HÚŽAZIĖ5‹;8¨Õ‹7ȕÄâm_õÄ$m× ­$æšÅÔę‰ÅäJbņļ¯zb’ļkVsÍâjõÄâ r%ąxÛW=1IÛ5H+‰šfqĩzbņyŠ$oąUē 68Ŗ@Ž%ŪđđŧŲ†˙ũKŋ\ņ6¸köuMŪxŖwcét:éšÅu:é Åã-˛’XŧíĢ}IÛ5H+‰šfqĩN'ã r%ąxÛW=1IÛ5H+‰šfqĩzbņš’XŧíĢž˜ƒ¤í¤•Ä\ŗ¸ƒƒZ=ąxƒ\I,ŪöUOĖAŌv ŌJbŽYÜÁA­žXŧAŽ$oûĒ'æ iģi%1×,îā VO,Ū W‹ˇ}Õs´]ƒ´’˜kwpPĢ'o+‰ÅÛžę‰9HÚŽAZIĖ5‹;8¨Õ‹7ȝ ą+¯ŧRžūúk™:ujŧ­éS:×uÜäU<—zÁIlīŊ÷–O>ų$ņV^¸Ë.ģČÉ'Ÿ,ƒ ’J•*ųzšWûėŗĪäĪ?˙”ũöÛ/÷FBzōÃ?4„{ÜqĮėqԙ¤ŧŠįLÚ.ųû°ņ\ĖŽũŒ3ΐ‹/žØHˇ~ũzųčŖäŠ+Ž8@ĻL™’VŦŊnŨ:)[ļl€-æßÔ 7Ü üņ‡ÜvÛmų7VJ Aę‚u2Ā†ƒ”WņœaÂÆsÁHėĸ‹.’Ģ¯žē„ô˙÷˙'įž{Žüú믆H–.]*—^zŠŧđ Æ;;餓dčĐĄRąbEķÜüųķĨsį΂WÕ°aC9õÔSeäȑōŸ˙üG&MšdÚĮƒņĘ駟n<ž#FH˛ûŨŊ{wųå—_äĮ”—^zIVŦXaHƒŸC¨ũõ—zčĄ2jÔ(ŠS§ŽüöÛo˛íļÛšß 2Äŧīˆ#Ž0íŌ_ž45jÔ0SÕ]wŨÕŧūųįŸ—ĢŽēJ>ūøcŲyįĨS§Næ˙›ožš\sÍ5ōÃ?HÕĒUå‰'žŸūŲčáÖ[o•>}úČĀM=Ú\˛d‰ }ēņĐ~˙ũwŲb‹- 1A b~ÆĀŪqĮåžûî“ĩkׂ¸đ f÷ŨwĨM›6˛õÖ[ĸȆĮŊöZy衇„ŋiŖzõęŌŖGY¸pĄ<ōČ#RĨJ0`€i“ļņÖx}}øá‡eåʕRģvmŠYŗĻĖž=Û<ėąĮJƒ døđáōũ÷ß˞{î)cĮŽ5Ī,^ŧXN<ņDéׯŸ´oß^Žģî:šķÎ;eüøņæ÷īŊ÷žxāōæ›oĪræyĪ{ųå—å­ˇŪ2žkP%ČATŸ ŲNō–FbŠįhā94ûâ‹/Ė†đF^ũuiܸąŦZĩJʕ+gđ Šđ3í•W^‘cŽ9Æx-xE5$’-‰A"“'Oú@ánŧŊ™3gJ“&MS^ŧAúvøá‡ãß͛77ŋįg‡v˜Ü~ûíæ˙ąŌŪÚŦYŗdŪŧy‰qyË-ˇČSO=e~ÆûųũûīŋŸø=ÍsmÛļŨˆÄ 1¸ƒԅč_Đm)o:SD8cÆ ã2ÍezĮŠ)ĶālH "Ĩ žgU”)đi§fūđŗL$Fŋ×ŦYcú°ŨvÛĪUûų]ОXęGYņ\:‰…įĐH HŪ˙ũrÉ%—˜ēĪ>û˜8ĶLļX°å‚@>1/V()xc—_~š|ūųį&āĪŠ 1+Œrã7š?ŧ*~GÜ =zt‰-éH„ÅȔmüÛ[idÅ4[OŒž<÷Üsf+Û/ "<<6ö"W&{ōÉ'M|â„D‡ Ļ[,ōã°@c€Ĩ­N*ž˙Ī¤zbaãš $–'ūJ}m‰ę=ql7HĪÄũØ ¯â9w$üØQî]Ûô“jôÜ5kàÎ]ēŸ´A^ÅsîWË]wÖ>iàRš6ČĢ$–ģÅ­%ąÜEÖ'mÔAZÉ5yƒÔ m)‰Ų`Ĩ€ûčÚ vMŪ€áųæbAbl’eđõęÕ+ĒŌŲ0Ë<+­Q-Ž jÛäU;ĶŲĀʎy šŊØËÆVRāyä‘r×]w™ã?ŌÚđ{r‹ą – Ĩ´™jôoŋũÖäã€6ŋ'KFˇnŨäwŪ‘ōå˛CÖä ÛrË-Mģ"Ķíp|‰#Oœ`ķ,… ŗėIŖäãĀ7ųÎŌž\œÛ\°`9ÎAYN plŠ˛Šžpb€3ĸôÜb´‘îísD ĸöRå;“ŸW F›Šįčá9{æ™g y…#B;Â3ÂKŖxŲ\!–.]ē˜ķ‡¯Ŋöš!2˛zŋúé§MōqAâM7ŨdÚ +ƒ•ŠđŒįyáũņ3Ȓķ›x~x=Ho¯Ŋö2G— 1RáÕA€^ė8’”ü3īwHæHá}›8qĸ9?šŠž msø˛+TQ Fŗ˜âšôąU <‡Bb¤wÆķō 9´ ¯G}ÔdwĀÁËa:ÉšE<1Ļyxo€&9×†7$E ‡ŋņæH˙ėĻi$Äĸ ŖiĶĻ&]ÎųįŸoŪEÖV¯ā ‘áĢ[ˇŽ™ÆĻžķˆ*ųw^Ÿ—Ļ‡ßA˛x}$rÜT_0z˖-Í4ģEI,í‚GÅsécĢx…Ä Ļh^áp4—‡@"Äžˆ5q虸ūŧķÎK$€'“Z<ķ’â%įĸpĶRũúõÍ!jĻĢĘ9ėÍArHŒ÷ã}‘ŗĖ+Påʕ ‰á%2õ°~ $vÎ9į˜xWđĒđø˜ânĒ/i,÷˛(‰Ŗ]0Ąx.}lĪĄ–ėŲŋ"Ũ3S<<–O?ũ4kęŨģˇI I‘~‡Øž9Æ(\ÎŲWcĘFāžĖ|°ņė˜š=øāƒ&ĀŸ|‰dB}H ˛Ąí_|1l‚÷͚53$ÆÕr;ė°C"ČO%ōž‘6(ŨĘ!$FZ!/ģĢ7¤?\˛ŠžÃčÁ įhˇR(Ō†ÄĪĨ­bā9ģ÷Ū{ÍÍ>:t0iwđŧ¸ ƒ 9S8bcŌ!ęŊņÆfđW¨PÁԁ$ø9ˇ˙´hŅÂL3“ûlv%˜ÎĻĨü!Áô'f†rI‹yĶ îEPŒ­ášE‰…/°OŌ;âs˙ū÷ŋÍĘ(ÄF Ÿ”@Š…ūąŠIŧX}ÅŗdŠËû6՗b=ÚôLī IbŠįŌĮV1đ ‰á‘āđ0Û˜Úą­€B~°ë¯ŋŪÄÂN9åC(ÄŽ rk1d‹+ŠxGxQxpŠ[,ČãĪsŦîāįj5ë\2B}VB Üãâ!AlL?‰ņ{â$\$Ÿo"bŠ!}$&–Ž0]äwäđ'ķ,SdļhāŲQ6ÕŽvĶéd0ĕÜJ!ILņ\úØ*žC#1ĻQ*¤Á†PŊ‚Į…WF+ŽĨPƒ:Ēē*”ŧÉ÷™FIv—ņė$‰‘}ī‰{™ęą —­x{tP”°X_ 5¨ë`Ā JŪ(’˜ëxv’Ä/Ä5HũÍ7ߘé+™q-…ÔQÕWĄä"‰šŽį‚“XTAîZŋ 5¨ŖĒG×äĒ Õ¯PĪNJm7; ¸6¨]“7;4Ø_[IĖ~f-kƒÚ5yŗ„å(‰YnĀ\ēīÚ vMŪ\0aķ3Jb6[/Įžģ6¨]“7GXXû˜’˜ĩĻËŊ㎠j×äÍv>Š$f§ŨōęĩkƒÚ5yķ‡…+‰Yh´|ģėÚ vMŪ|ņaÛķJbļY,€ūē6¨]“7ˆXՄ’˜Uæ ĻŗŽ j×ä %ö´ĸ$f­ëŠkƒÚ5yŠ% )‰Yb¨ ģéÚ vMŪ ąbC[Jb6X)ā>ē6¨]“7`¸DžšRI,ō=×æĨîpĨr-ņր‡įD‹8‹  Ķ]ĩ–*3u\čqļyœeS<—´Ž3$懜t įĄŲüâÔo=Û5Ŗ$–dAWŒn;h]īŋ_œú­gģ>•Ä”ÄlĮ°sũ÷KN~ëŲŽ@%1%1Û1ė\˙ũ’“ßzļ+PILIĖv ;×ŋä䷞í TSŗÃÎõß/9ų­gģ•Ä”ÄlĮ°sũ÷KN~ëŲŽ@%1%1Û1ė\˙ũ’“ßzļ+PILIĖv ;×ŋä䷞í TSŗÃÎõß/9ų­gģ•Ä”ÄlĮ°sũ÷KN~ëŲŽ@%1%1Û1ė\˙ũ’“ßzļ+PILIĖv ;×ŋä䷞í TSŗÃÎõß/9ų­gģ•Ä”ÄlĮ°sũ÷KN~ëŲŽ@%1%1Û1ė\˙ũ’“ßzļ+PILIĖv ;×ŋä䷞í TSŗÃÎõß/9ų­gģ•Ä”ÄlĮ°sũ÷KN~ëŲŽ@%1%1Û1ė\˙ũ’“ßzļ+PILIĖv ;×ŋä䷞í TSŗÃÎõß/9ų­gģ•Ä”ÄlĮ°sũ÷KN~ëŲŽ@%1%1Û1ė\˙ũ’“ßzļ+PILIĖv ;×ŋä䷞í TSŗÃÎõß/9ų­gģ•Ä”ÄlĮ°sũ÷KN~ëŲŽ@%1%1Û1ė\˙ũ’“ßzļ+PILIĖv ;×ŋä䷞í TSŗÃÎõß/9ų­gģ•Ä”ÄlĮ°sũ÷KN~ëŲŽ@%1%1Û1ė\˙ũ’“ßzļ+PILIĖv ;×ŋä䷞í TSŗÃÎõß/9ų­gģ•Ä”ÄlĮ°sũ÷KN~ëŲŽ@%1%1Û1ė\˙ũ’“ßzļ+PILIĖv ;×ŋä䷞í TSŗÃÎõß/9ų­gģ•Ä”ÄlĮ°sũ÷KN~ëŲŽ@%1%1Û1ė\˙ũ’“ßzļ+PILIĖv ;×ŋä䷞í TSŗÃÎõß/9ų­gģ•Ä”ÄlĮ°sũ÷KN~ëŲŽ@%1%1Û1ė\˙ũ’“ßzļ+PILIĖv ;×ŋä䷞í TSŗÃÎõß/9ų­gģ•Ä”ÄlĮ°sũ÷KN~ëŲŽ@%1%1Û1ė\˙ũ’“ßzļ+Đj;įœs¤N:mСo_Ų°aCÆzē™Ęâŋåá‡ÎTM¯ČJŠįŦԕ¨l-‰=ņÄŌĻMšđ ĨZĩjĨīĶ§OÆ:ũúõËXgŊ2~üx™2eŠ´hŅ"c}­ đŖÅŗ-Ĩ¯c-‰!Έ#dŌ¤IōĘ+¯H™2erׂĪ'ׯ_/5’3Ī|xf ”RŖ[ˇnōíˇßĘäɓsnCT ĄÅķĻĩKûã?Ė4°]ētÉGwÜq‡Ü{īŊ&žļÕV[eũŧ> RŠgI ‘-ZdˆlęÔŠrÜqĮųÆÔėŲŗåôĶO7qĩzõęų~N+Ē ŠÅséڍĨ'æ‰;}útéÜšŗ!¤ÚĩkgÄØįŸnˆoĖ˜1ŌĒUĢŒõĩ‚j L (žĶk;Ö$†ČC† ‘3fČK/Ŋ”oGu”´lŲRŽšæšŒuĩ‚j P¨P DJ›$ąO>ųĤ}fûÁ‡~(+WŽŒTįĩ3˙h bŊ˛īžûšm$¤ŅŽ[ˇŽĒ'EŠg{ ‘ žK%1.Â`s(ëŧ”4ä—×M pĪĀģīž+ܚC°—Íģ\<Ąåo (žíBB6xŪˆÄH?ĶēukŠ_ŋž z+qŲe|z öž˙ūû2mÚ/(ĖJ IDAT4“–ČÕĸxļßō™đŧ‰qÄŌŧysšá†ė—Ūq   ŗfÍ2œ¸ZĪņą|ix.Ab¸ÜĢW¯6Ķ-ņĐá€ōåË;9ĩT<ĮÃÉR¤Ãs‚Äz˛å`éŌĨ:…Œ‘íqÅkÕĒ%,ÁģėW<ĮÄIĸ¤Ãs‚ÄnŧņF!ƒ$Š>´ÄK¤"Cm˙ūũã%Ø&¤Q<Į×ÔŠxNąn :účŖã+ŊŖ’Ŋđ æÆ&—bcŠįø‚=Ī ĢTŠ’N%cjwĪ˙å—_b*áÆb)žãkęT<'HlŗÍ6“ 6ÄWrĮ%sÍžŽÉëŧ“íĢ$æˆõ]ÔŽÉëŒb*‰šfqr.9æiģ&¯kVsÍâJbZ<Ū"+‰ÅÛžiĨsÍ3qM^× ­$æšÅÕsĐâņŲZģįž{2Ļ’nÛļ­Lš4)/ r8˸¤ ŠKqÍ3ąA^ÅsîŖËZ##yÍŧ2lØ0sŲ9ĪŧRŊzuŠW¯^îÚ%ąŧÔ‰‡m 1ÅsîPą–ÄREžčĸ‹Œˇ´dɒÜĩ‘æI%ą@ÕY”Æl 1ÅsîЈ5‰­_ŋ^úöí+'N”ožųFĒUĢ&§œrŠÉļÍ6Û­qgßUW]%_ [íîģīn’æqBž’Jb˖-“† Jãƍ×ĮMI'žxĸŧôŌKæg6u>zĩQŪteÅszÄšÄ Ģ>}úȃ>(ûīŋŋ|ųå—rá…JĢV­éhHáĖa÷áÇËvÛmgŧšË/ŋÜäŪ:î¸ãJØīŋ˙.Mš41øôĶOKšråĖļwīŪ†I mCąqPįŖWåMGbŠgIŒķüŲu×]ŌsÃ7äzņ´*UĒHˇnŨ„L^Y¸pĄėļÛn˛ãŽ;&HėŲgŸ•3Î8CHëōōË/ įņl-6ę|tmŖŧéHLņė ‰ąĒiáU}˙ũ÷ō×_-@NL )\p<ōČ#réĨ—šÁ:ę(ãayśN|đÁōĀČë¯ŋ.ģė˛K>cĒčĪÚ8¨ķQšōĻ#1Åŗƒ$ƅ¸ķæÍ3—ã’ŽeË-ˇ4ŠļYÎöHŒ8ÝwŪię,°B… æ&pĻĄe˖5žØėŲŗM–[žûíˇĨN:ųŒŠĸ?kã ÎGi6ʛŽÄĪŽ‘Y8 $H‹üY^Üä‘X˛ZVŦXaŧ-ęßvÛm†Ė ą_|Qžzę)šä’KLõųķįË[l‘Ī¸*ęŗ6ę|fŖŧŠ$Ļx.ą ė¯]ģÖd0ŊũöÛĨK—.FÄöÜsOŲ|ķÍåģīž3˙‡ĐÚ´iS‚”Ž9æGãšēäÕIâax ‰Ą 80ŸqUÔgmÔų(ĖFySILņė ‰!2÷đc×>ņRŲBbW˜í°ÃRģvmŗZŲĩkWãšąÚHœläȑæīÔ-ŖG6[0˜Ļ˛Ĩ‚éeŋ~ũäæ›o–}öŲ'ŸąÚŗ6ę|”cŖŧéĻ“ŠgĮĻ“ˆ QąĨ‚ŋ!+ˆ†=iˇņÂŌ/_ž\zõę%oŧņ†°…‚UɎ;J÷îŨÆRI Rd_^Ô2ĩÔ}bųPL៍ ‰)žcNb… ņyƒƒ:íģ&o>ē˛ņŲØÄÄlT~ąúėÚ vMŪbáĒXīU+–æ‹ø^×ĩkōZEyĩ’XQÔ^ܗē6¨]“ˇ¸č ˙íJbáëŧčotmPģ&oŅr”ÄBVx^įÚ vMŪ(`,Ė>(‰…ŠíˆŧËĩAíšŧYhŨ°šÄ-Z$õë׏l./ō—ą1–ŖOQ*Ž j[äU<į6J”ÄrĶ›¯§”Ä|ŠŠā•”Ä‚Qą xļîpŋ\œĄË÷đš-ƒ:˜ĄgĪeÁŠįÜ,^0OŦbŊfõ駟ĘÔŠSe͚5&S*ipĒV­jzËąŸë¯ŋ^Ū|ķMķ{2Ŗ4HČļę.P`JFč2eʇŗGŒ!;īŧŗ¤3:Īsŧˆŗ1ōķ’’ÍõŖ>’=öØC†* ũöÛOƎkēB{ôč!¯žúĒéGĶĻMM?Hg]ZáËE&ŒŸ~úÉd—åXSļ:xīŊ÷¤Aƒ&‘#šŅHÄųÎ|Š’XöÚS<‹Iīu<ę‰ATd‹ ÛÃųįŸo2ŠļhŅÂäõ‚ÔH qŽ‘ŦĒ ÎņãĮŌā\"ŋ[ˇn!"<Rķō7g!ŧwŪyG>øāƒ11Ú=ûėŗåņĮ7įũŧƒƒáŲ)¸1‰ŧa''˙i§&ø^ētŠyOŖF AŌ/ōōũõ×ĻdËHW0:ŠŦO>ųdißžŊüį?˙‘‹/žØŲ”)S|õīã?6ËÉëOâÆ8Ā¤ÚΧ(‰e¯=Åķß$u<JbĖāã ¯ÜrË-Æ;ƒ8øxYäĩßvÛmM•_ũÕd’˜Î=÷\,ęÖ­kž#=õá‡nRøĐŪĩ×^+cƌ19ú+WŽlępÁ‡ÅyūŦŗÎ*•ÄvÚi'sŨ+dŧ ˙čâÎÔŋĪ>ûLöÚk/“"ĸ(‰e¯EÅķß$u<Nbx`LÍŧBî.ŧ˛?āM<öØcf@CRLĩŧa€°˙ū2xđ`ųíˇßŌĸΛNB$xFdŸā™ä’éd° íš\0ؙgžiHŦYŗf&ĩ5SÔäšāņ‘ŗŦ4OĖkÃûũŒ3ĖKL!āLũķH Ī˙A%ąėĩĪ˙Œ‰¨â9pģė˛ËĖ•i^™;wŽ‰i-X°Āx\Lá Ģž={ Ũâ9áQy$F čžûî2ŽĻ+‰¯€„:tč`bn^3ŊƒŠã´iĶĖÔ0šėŊ÷Ū&6‰rČ!&oXj@i-ĂįX‰áÍá}yåšįž3íúՁGb\T’+Ė~ūķ„’XöڃÄĪÕĖė$ĘxœÄČ Ž—ãbU§žzĒ™^¨ÆkbJH.{ DRŗf͉Ą,ę@Ä×R‹Gb;w6ŧuëÖ2}útķ Ķ¯Lī 5¤I~ąä˛ũöÛKÛļm ‰AŧÄÁŧ r=Șø]i$ÖŽ]ģÄõpԁ0ÉQæWJbŲNęA6$ĻxŽ&QĮsā$F"B.ßđŠ—âbņŽģî2I ŊÂԒ˜w@LÜRÄb9ˆÂÁyįgrá3ÅKŪėJ,læĖ™fĒÆܝƒLī`1,š_|ņ…‰qQHx衇&bbxŠŦ.R'Ų[ŧxq"Ũui$F&YV4ŊBŸ† bČۏ”ÄĸCbŠįjīQÆsā$Fā‚ pÎV Xœ1,<ąæ͛›é?Ãb€›¸‰ąPž|yŗŊ€lĒxtüĪ ¯‰?["’IlÕĒUĻ>zÚga Ķ;XÁduo‹w2-%žÆJbęę$ņ<‚ü[oŊĩ™B2UÆ L7ĶV´ĐSfžâžN8áß:€ä‰Ŋét2w2 ĘS0UdĘČJ)ģâŲŸÆYq,A j›ô„ŧļ˜ëx”Älšk} bPÛ¤3×äĩÉ6Aôĩ`ĀƒčœļQ ¸6¨]“ˇ0¨‰nĢJbŅĩMÁzæÚ vMŪ‚'ĸ §%ąJ•*™íŪaįˆö]ģ•ƒ8xÎBFę …š˛æÅŗ5ĻĘēŖŠxNÄÄH—C`›ũVZâĨ6ŗa˜ũnŽÅs|-Šį‰q4†ÍŠŅ/ Ō‡ũpŠŲ>â%eIiĪņĩn*ž$FRBļč”2^Æ÷\oļ”xšĶâ%aziĪņ´r:<'H ‘ŲmO–ĶqãÆÅSJÕŠS's2#YŽÅsü,žĪ%H ‘‰%p~0jWŽÅĪ…—ˆ;8#ęR,,UĢŠįÂã,Ŧ7”†įHŒÔÉäč"S‡˛uĩ2,÷\n.8!šĖjÔ¨\ã–ĩ¤xļĖ`iē› Ī‘˜×Žø„ ÷ü^¤ģQB‹. 04)ĀIN8€Œˇ.N!Kŗâ9ēØM×ŗlđ\*‰Ņ0ÁQRęĖ™3Į$&ä Š–hj€@\ŦBļ[r¸šÄ÷kÅŗ_Mŋ^6xŪ$‰_”Âõ@wtNˇÚrøpĪJbáãMߨ\JbĢ4ú ēlôč[G{˜­\ÆŗzbŲĸEëĢ"¨%ąĨĐ]rŲč…Ö­ļž\ÆŗzbáãMߨ\JbĢ4ú ēlôč[G{˜­\ÆŗzbŲĸEëĢ"¨%ąĨĐ]rŲč…Ö­ļž\ÆŗzbáãMߨ\JbĢ4ú ēlôč[G{˜­\ÆŗzbŲĸEëĢ"¨%ąĨĐ]rŲč…Ö­ļž\ÆŗzbáãMߨ\JbĢ4ú ēlôč[G{˜­\ÆŗzbŲĸEëĢ"¨%ąĨĐ]rŲč…Ö­ļž\ÆŗzbáãMߨ\JbĢ4ú ēlôč[G{˜­\ÆŗzbŲĸEëĢ"¨%ąĨĐ]rŲč…Ö­ļž\ÆŗzbáãMߨ\JbĢ4ú ēlôč[G{˜­\ÆŗzbŲĸEëĢ"¨%ąĨĐ]rŲč…Ö­ļž\ÆŗzbáãMߨ\JbĢ4ú ēlôč[G{˜­\ÆŗzbŲĸEëĢ"¨%ąĨĐ]rŲč…Ö­ļž\ÆŗzbáãMߨ\JbĢ4ú ēlôč[G{˜­\ÆŗzbŲĸEëĢ"¨%ąĨĐ]rŲč…Ö­ļž\ÆŗzbáãMߨ\JbĢ4ú ēlôč[G{˜­\ÆŗzbŲĸEëĢ"¨%ąĨĐ]rŲč…Ö­ļž\ÆŗzbáãMߨ\JbĢ4ú ēlôč[G{˜­\ÆŗzbŲĸEëĢ"¨%ąĨĐ]rŲč…Ö­ļž\ÆŗzbáãMߨ\JbĢ4ú ēlôč[G{˜­\ÆŗzbŲĸEëĢ"¨%ąĨĐ]rŲč…Ö­ļž\ÆŗzbáãMߨ\JbĢ4ú ēlôč[G{˜­\ÆŗzbŲĸEëĢ"¨%ąĨĐ]rŲč…Ö­ļž\ÆŗzbáãMߨ\JbĢ4ú ēlôč[G{˜­\Æŗ3žØĐĄCå†nÁƒK׎]Å3úȑ#åē뮓ČUW]•-v´žj (P<˙ŖvgHlÕĒURĩjU)[ļŦ”/_^~øáŠRĨŠüūûī˛nŨ:ķ˙ *úRÕ@ļP<;HbˆÜŗgO6l˜ŦYŗ&ĄråĘI÷îŨåæ›oÎGZ_5PT (ž˙Vŋ3žÂōõÚ~ûíåĪ?˙L€oË-ˇ”å˗ĢVÔá¨/ĪEŠgI,ÕS/,—ĄŖĪDIÉŪ˜ĢxvĘKõÆÔ ‹ŌpÔžäĸdoĖU<;Gbž7ÆęĢ‘ Ëečč3QŌŪ˜Ëxv’ÄøzučĐA&L˜ ą°(FíKNpĪ›$ąO>ųDzč!™3gŽ|øᇲråʜ”Ŧ^+V”}÷ŨWŽ=öXi׎Ô­[ˇđ/ĩė Šg{ – žK%ą+¯ŧŌx*:u’-ZHƒ ¤råĘöhÁąžūüķĪōîģīĘOúčøJī¨d/ŧđ‚I7äRlLņ_°§â9Ab•*UŌŠdLíîšāŋüōKL%ÜX,Ås|MŠį‰šœ2žæūG2×ėëšŧ.`8YÆdû*‰9b}×ĩkō:ㄘJbŽYœÄq›m&6lpFr×äuÆ°˙TIĖ5‹+‰9hņx‹Ŧ$oûĻ•Î5ĪÄ5y]ƒ´’˜kWOĖA‹Į[dkIėž{;nŌ:mÛļ•I“&åeÁĶO?Ũĸ&Q\Škž‰ ō*žs]֒ Čkæn.zë­ˇLÎ3¯T¯^]ęÕĢ—ģvDDI,/õEâaHLņœ;TŦ%ąT‘/ēč"ã--Y˛$wm¤yRI,PuĨ1HLņœ;4bMbëׯ—ž}ûĘĉå›ož‘jÕĒÉ)§œbrŖmŗÍ6Fk_}õ•É¯Īņ˛ÕîžûîBÆNČSRIlŲ˛eŌ°aCiܸąņúžyæ9ņÄåĨ—^2?ŗĄØ8¨ķŅĢōĻû(+žĶŖ Ö$YõéĶG|đAŲ˙ũåË/ŋ” /ŧPZĩj•HGC gģ>\ļÛn;ãÍ]~ųå&÷ÖqĮW‚ĸ!ŧI“&†Ÿ~úiáZ,Ļ°Ŋ{÷6ÄHJhŠƒ:ŊÚ(o:S<;HbœäĪŽģūęĢ¯–'Ÿ|2OĢRĨŠtëÖMČtā•… Ęnģí&;î¸c‚Äž}öY9ãŒ3„´./ŋü˛pĪÖbã ÎG×6ʛŽÄĪ’ĢŠ^Õ÷ß/ũõ—ŅäÄ´rÁČ#<"—^zŠœtŌIrÔQGË+Ūtōāƒ–x@^ũuŲe—]ōSEÖÆAŌl”7‰)ž$ąsÎ9Gæ͛'?ü°Ž… EIĩÍrļGbÄîŧķNS‡d*T+Ž¸ÂLC˖-k<ąŲŗg›,ˇ<˙öÛoK:uōSEÖÆAŌl”7‰)ž#1ÎBHųŗŧ8¸Č#ądĩŦXąÂx[ÔŋíļÛ ™Ab/žøĸ<õÔSrÉ%—˜ęķįĪ—-ļØ"ŸqUÔgmÔų(ĖFySILņ\:bØ_ģv­É`zûíˇK—.]Œˆ)ėšįž˛ųæ›Ëwß}gūĄĩiĶĻ)sĖ1&ŽÆ5uÉĢ“ÄÃ<đ@C8p`>ãĒ¨ĪÚ8¨ķQ˜ōĻ’˜âŲACdî ā ÆŽ}â ¤˛…Ä 'Ž0Ûa‡¤víÚfĩ˛k׎Æscĩ‘8Ųȑ#Íߊ[,Fmļ`0MeKĶË~ũúÉÍ7ß,ûėŗO>c-´gmÔų(ĮFyĶM'ĪŽM'ĸbKCV zŌnã…¤_ž|šôęÕKŪxã a Ģ’gęŪŊģŅX*‰AŠė Ã+ã‚ZĻ–ēO,Š)üŗq!1ÅsĖIŦđC!>o°qPįŖ}×äÍGW6>›˜˜Ę/VŸ]ÔŽÉ[,\ëŊJbÅŌ|ßëÚ vMŪ"BĢ(¯V+ŠÚ‹ûR×ĩkō]áŋ]I,|ũŽ j×ä-:ĀBXČ Âë\ÔŽÉŒ…Ų%ą0ĩ‘wš6¨]“7"0 ­JbĄŠ::/rmPģ&otNO"KbėĻį@6›Oƒ*wÜq‡Ųāzß}÷Õ¤•í¸6¨Ŗ"o!0Mz¨æ͛KûöíKÅĸŸ:VųŽ$‰‘ŧ°V­ZōÅ_Jb#"’˜^žö -Ļ9‰BRÎM‘˜Ÿ:aë#Č÷EŽÄ8“Ø´iĶ„Œ¤“~ėąĮdŨēurĶM7Éũ÷ßo2Pp@›ķŽ—]vYĸ.)ĸ9BôŪ{ī )v4h ƒ ’ũë_&#+)¨Ŋ™Į8`#]~ũõ×&]5Iûí7“v‡Ŧ¤AĄ|ôŅG&ƒ+mq¸œôÔå˗nV1b„9\NyķÍ7ĨGōęĢ¯J™2eŒLüžô×Å.QņLÂŌCąå- Ķä¸kdHųõ×_MöáÁƒKŗfÍĒ!m¸ųüķĪ Î82Įš^rÚ!—WHŌÉųāԒZ‡6:wîœHĘXÚi§L„É“''¯QŖ†ItĀyãLc",;–öžČ‘§õ§OŸnHÃØØör|?^˜6jÔHž{î9CbČæŦ#„ƒâĪ:ë,ķsÎ8R¯ #@.d§Øk¯ŊdÔ¨Q&5ä’\ÖŦYcˆųdųėŗĪLdŗ ]0 ĸ/$T¸îK—.•úõë›~r^Ŧôãƒ>0Ų5ŠYŠ=¨Ã–ŊØōĻÃ4dr衇š{Ǝk°K^ģqãƙ ˇtņQ_üb#UÔĩ×^+ā”3ģÜ™įŗĪ>[ČRœZRë068/Lzu ˜…¨Ā(7.Q>ũôSķņæCĪ;Ķ˜۞鈚ņNŲlÃ˙ūUlŖŖ`U{ĶIkOĀ€xc^ŧČõÅalŧ/”Í˙ÉÎJÁ0¯ŧōŠ  9üđÃeīŊ÷.u:ųøãËŠ§žjž9ōČ#ī9äC @HŽč‘ũHNi ŲžvÚi†Øčį˜1cLN˙ʕ+›v¸„ø—‹@´Å,ÅļoØ˛GAŪTLã}ņáãčy^„: ¸đôÁø÷˛ “BŠ$-¸bë­ˇ6YYJ›NĻÖa&CžŧŸ~úÉ|ØÉhŒ’Đ`=öœ…ž={štU|ŧ3‰°íi%‰ALöųįŸ7ĶB¯ā%ĩk×Î|ĄļŨv[ã­ZĩĘäã’ūŸėNg"1b L?ųŌyĶBŪ…Ą§NjR\{$æyf^_ :Ūw÷ŨwPHĻÉrœÅ,QÔaĘySIŒ >™Sđü“?ĖÜņĀG¯ŒÄ¤x" ˜N^čʅĘ ÔŦYSŪyįķŅ‡4I:tčP9÷Üs ō‡1Ā-a~ÆD˜ļL÷ŽČM'édĒÁI\ČTŽ¯Q2)A¸ę\ĸ‹Ąøác˛!nFŒÁķ|2‘y¸‰v’ q6Œ h<#fÆMIÉ$FŒ„†+žšũ`0ŨLŽ=QÔaĘyS1ÍŨ`Š”įɅŲCÕĒUDČĀôĖ™3 .š./Ÿf.$F›ĖF¸Ņ‹8ņ0œnû‚āđÂX`–ŅĄC“;/Ķ˜Ķ–Ö’ą‚ķSĻL1ąĻÔB°< ‹/–!C†Ŗ@(|u2‘yõÉØ 1&“%ĐG}Ôˉ{ŒÄ2R ņŊwŪš¨vÂ SQ7•ÄđÂđúÁfj!V‹×ž\ø`/X°ĀL/Á á ¯˛NŌ&3Ļ“„?Ā;} TąÍ;×ÄŲ…āąųaÚŌ:cE‚"øILlØ°afjį”Lûør?Ãg5Ķ+ž :SŧķĪ?ߐXŨēuÍ gēÂ׎e˖,Ôõ ˆAņũņž^ô)ŲƒXŊôØÅ4|u˜ōGA^Äō10`€ÉBĖšX°bZČø`–’î›Ô:Ŧäƒ}VDq Č^L9蠃Ė6 Əˇõ3&´Ĩ5$†› ' ĪJNō>ą˙ū÷ŋækÅĒ 1/‚úũ|IđxpÉ÷Ûo?ȋB;/)- ”ĻHŽÔÔ|) ēB 7=Ī‰Ņ&ËäĀlėĸžžÅ.QÔaę ōĻÃ4 EŪeÎėcÖqņÅ›3…gøđUpYąbEŗjæų(SX ā#Ë*øĸE‹ĢáÉúMWG€Ķ+ėÃŖŖ@„88ĖHŧ’iL„iKkHŦØJ‰ûûŖ0¨ÃÔąkō†ŠÛ(ŧ+’Ģ“QPLœûāÚ vMŪ8cW=1×Ŧ[ŠŧŽ j×äu æꉹfqŽflĻĀ4{lEV‹­iKLIĖAŖĮXd%ąˇ4Ņ”Ä4zŒENKbe¯“wx9Æō;'û€Ø[ĮĄbWŠâ9ž–NÅs"‹;„Ų‹Â--ņŌ›‚ɏÆū5WŠâ9ž–NÅs‚ÄØ ĘqÎj‰—ČÅQ,N2¸RĪņĩt*ž$F~.vôę”2^Æ÷\oî.ā Š+EņOK§Ãs‚Ä™ŖĢW¯6™&ĩÄC:u2gí8ĢįZQ<ĮĪâéđ\‚Ä™X7ŠÜpà ņĶ€cqpxÖŦYNÅÂRMŦxŽčKÃķF$FÎíÖ­[›ĶņŪ!Ķø¨Á Ipšš°äũ÷ߗiĶĻ™ĖŽÅŗũ–Ī„įHĖWœ<Ū¸o¤é ŋ–nŋˆ. 04ųŌÉF8€\S.N!Kŗâ9ēØM×ŗlđ\*‰Ņ0ÁQ.š;úDIDAT ¯é Ém¤%š e ‰öHŸÍ.ņũZDņėWSů— ž7IbÅE{ P ¨6­%1Eˆj@5`ĩū8 ”IENDŽB`‚avocado-113.0/docs/source/blueprints/images/BP004/dependency_graph.png000066400000000000000000000343341513561306000255570ustar00rootroot00000000000000‰PNG  IHDR-c“P3sRGBŽÎéVtEXtmxfile%3Cmxfile%20host%3D%22app.diagrams.net%22%20modified%3D%222021-09-03T18%3A09%3A08.490Z%22%20agent%3D%225.0%20(X11%3B%20Linux%20x86_64)%20AppleWebKit%2F537.36%20(KHTML%2C%20like%20Gecko)%20Chrome%2F92.0.4515.159%20Safari%2F537.36%22%20etag%3D%22iu_rMbKjJ-v4w3xtOUp2%22%20version%3D%2213.9.2%22%20type%3D%22device%22%3E%3Cdiagram%20id%3D%22ikDY0Pxp_RKBuqn0TieK%22%20name%3D%22Page-1%22%3E7Vldk5owFP01PG5HSEH3cf3Y7kxndzrjQ%2FUxQoTMROKGqNBf34tJ5EO2WmtHxuWJ3JObm9x7DoGAhUar9JvA6%2BiVB4RZTi9ILTS2HMfuoQFcciRTiDuwFRAKGminApjSX8SM1OiGBiSpOErOmaTrKujzOCa%2BrGBYCL6rui05q866xiE5AqY%2BZsfoTxrISKEDp1%2FgL4SGkZnZ9h5VzwobZ51JEuGA70oQmlhoJDiXqrVKR4TlxTN1UeOeP%2Bg9LEyQWJ4z4GW%2BfetP5q%2FfF2JO3rLFLHvPHnSULWYbnbDleAziDRf5kmWm6%2BC9b%2FJ1Dpc8lg%2FJnqUncID5gelh0Q%2BtUF%2BZ8f%2BnQE8mEiSmglUnAHhRYE5lLgcKADoBY7iLqCTTNfbznh1IFbBIrhhYNjRxslbiWdKUBHp5WoxQVhN6S4Qk6Yf1tw%2Bswu1A%2BIpIkYFLWlV0ZpSi7V0hq76GopKiDIa1kMND4IJraGi6%2F4J61Hbqh%2FdDvVPj%2FsbUf2079aO7pf7m3Ltt5%2F75brh3kNcu7r22cz%2B5W%2B5v%2Frjvt5378f1yf2PqBw3U1%2BsXB0%2F5aQksn%2BEkoX61bCSlclZqz0vtcZ52zxiZMWJY%2BqxszMtGMWhvmVHNbJDg6JBW4wKS4Rvhk9NHHYlFSOSpl6NjbkvkuQ3kGUwQhiXdVpfbxKie4Qen%2B%2FvGSGdQe11wa6JQaepR5dNeLRBCtUC9WiBVh6NAe4Ed0r5cc4%2F%2FR3O9CzRnt19zqNPcFTRn9tmri%2B6Le8lWp0bdRnjoTOH1O%2BFdQ3hNn9E%2B3xP2XNF5neiuITqn2%2B2q37NOCs%2FthHcN4TV9Of5873bniq7b7f4sOjCLn2DKvfiViCa%2FAQ%3D%3D%3C%2Fdiagram%3E%3C%2Fmxfile%3EÍoV| IDATx^흐UEÖĮ[‘  Š€uU@ĸ€ĸĢƒãJp•$¸ģ‚bĀ…DÉA1 Ŧ€ˆ kJŅ-)GÄTpdÉ(ĸŦ(Ä@P _ũû˜aæŊ™wįŊúŪsĒŦg ÷ŨîūŸî˙ë>} ŋ˙ūûīFEPK8AIËMi7E@PŌŌ‰4_}õ•Yąb…Yģv­Ų´i“Ų˛e‹áoûöí30ŋüō‹)Qĸ„)[ļŦŠTŠ’ŠV­šŠ]ģļšđ MŊzõLãƍåo*Á@ĀV}*icūļoŋũļyã7 Ÿ;wî4M›65õë×7_|ąRõęÕÍ駟.DaA\ØŪŊ{åyˆmãƍf͚5&''GžĪČČ0­[ˇ–Ooƒ>•´ŧ3V´Æ.jæĖ™föėŲB2mÛļ5™™™ĻQŖF)÷˙?˙ųÉÎÎ6YYYBj]ģv5Ũģw—Ũ˜Š;„MŸJZîĖ+ßē|ųr3qâDŗhŅ"ĶŖGķ—ŋüÅ\vŲeŽeũúõæĨ—^2Ī=÷œiÕĒ•éĶ§iÖŦ™kíEíÅaÕ§’VÔfrœņōKüČ#˜ĨK—š~ũú™^ŊzÉQĪ+áH9yōd3~üxĶ˛eK3tčPŨyĨ~ØõŠ¤•ÂäÃW!ĢQŖF™#F˜Áƒû>¤G}Tú2lØ0!/gDAŸJZÎæDhž^ĩj•éŨģˇØŦƍgÎ=÷ÜĀŒí‹/ž0 ›×¤I“Lƒ Ķˇ v$JúTŌ ę,tą_ؐîēë.3eĘsīŊ÷ēØRj¯ž:uĒéŲŗ§™>}ēØØTâ#5}*iEl% 8Đ,X°@nĶqč6|Ü6rģØĻM3vėXˇ›ŗîũQÔ§’–uĶ´øžķÎ;ÅtΜ9ĻB… Å‘ĮßüîģīL—.]Ä1uƌˇÜæĸĒO%­āÎÉ´ö ÷…_ũUËV¸N:é$q“ˆēDYŸJZ˜ũü"•´BNZØ<ˆ\¸pahFzã7J,cm\ĒO ˜ÍBŽ7n•đp˙ũ÷­˛aĨl\W^yĨxĐGéVQõytfčNĢ¨béŋãˇĶ°aCÉĘ`Ã-ĄS˜šU$kÄʕ+#áĮĨúĖ!JZNW‹%ĪˇhŅÂüõ¯ ´VĒPâĮõĪūĶ|đÁŠž*đßW}*i~’ĻŌAB9Ö­[g^~ųåT^cÅw;vėhęÖ­ęÕįņSQwZV,Íä;I°ėĨ—^*Iú‚š“üœ=IČim6lØĘ kÕgÁų ¤ålūiüwHĐ„āg¯Ā"ȚDƒaôßR}*iyĩŽ|i‡üI;w6ÛļmķĨ}?=įœsĖŧyķB•Kõ_ŸēĶōsĨĨší[oŊÕ\qÅĻoßži~sđ_7aÂķᇚšsįŋŗIöPõ_ŸJZIN  ?†íŖyķæf׎]ž&đ .$<ãŒ3Ė˛eËBaÛR}&Ö§’VPV]Šũxøᇠ%,ɍU!× 'œ`{ė1ë!P}ÉŠOŸJZÖOīŖĀĻCÕ7sē*rÎSå' 6=Õ§1‰ôŠ¤ô•˜D˙( 5dČąéD]°é=ÚęōdĒĪÜYOŸJZ!Xå÷ßŋŠ\šr¤ÜŠ ÷‡={ö˜§Ÿ~ÚZÍĒ>sUOŸJZÖNí܎s$´%ŠÛpĮ2r´°UTŸšš‹§O%-[gö˙úM&R<āŠčŦr*^ã!OĻSÛDõYPcųõŠ¤eÛŦÎ×ß×_Ũ8Ļ|YŠĒ„|[ėh׎]Ē¯ōüûĒĪ‚įק’–įĶ2Ŋ L{čĐ!3fĖ˜ôž¸ˇyæ™â–_N<ņDÉÛEĖTŌ9ųä“=ëWŦĄAƒ™2eĘXD}–.]ZvŦ—_~š?Cø›’_ŸJZ~h!m›–™™)“Ę+IDZųÛŋúęĢÍâŋ%¯ģ—Bēšėėl+cƒĒO‚ī_yå!1¯%ŋ>•´ŧÖ@šÛ#ĪĨdōôJb¤UĒT)“““sŦŲŸ~úÉŦ^ŊÚāIvQäÕW_57ß|ŗW]“vČԊcbķl=õÔSæîģī6åʕ‹‹‰Ÿú,Y˛¤áxŠ°{' žÍ›7Ë߸Ą&Šd­Zĩ|Õ§’–§đ§ŋ1~ņëņ2 MŒ´8ú>|¸Ā †nF%';$æĨŽ&##Ãđ4áØJU¤xĀpėÉO^AĶ'úíÔŠ“ÉĘĘ(‰‡ôēĸS~}*imV;ėOŊÍÖ­[=Íī„´HsÛmˇ9UjŗËĢYŗĻŲŋj/ráÛäė‡Äûí7 QÁ'+/yQŸûö퓈‹vcß~û­)[ļŦ čÄe~}*iyŊ; 1‰Øʗ(Qâŧ5īņí1áx¸fÍķāƒ apŒøøãĨ䗗Bđ4;š#GŽxŲlŌmaā†ŽØyÉĢRĨJžé3ŅΙ~ļmÛV*“#K—.5W]uUŌãMõÁüúTŌJQŸŋĪ„'PÚK)ĘOŸØ]akĢ^Ŋē—];Ö}°Iøņá–nūüųžéŗ0ŌęŨģˇ™u§•" ~ŨOęüÎĨ`­crĖūÁÕ8AĀ‰œ)ŨĀO=❡#­ŧÎĨ_q*6mš8™"ܯ]ģ֔/_ŪY)>­ņ)´¯%V-.Ä"Θ1ÃtëÖÍSØ4öĐÜEŨķļK.šD~Œ¨1éĩhėĄ×ˆģܞY ›äėžps Ŧ¨OŸ>ĻQŖF.#PđõšåÁäņôÉ%…BęÔŠ#î+ėčŊöˇ‹Bŗ<8ĶgāŸÖüKU¤ų´?muPķi9‚ËŽ‡5ĶeŽž4sŠs6Ų^jæŌd‘˛ė9Í)žĢ0ÍoŲä-ĸģš#>\ú<6­Ū’ĢX­ÆŽIŽÕxÂĨĪãFŖuō×ÉŗQíĒO­{hãŧuÔg­Hœ¸"ą# ō°ęS+Ld*ē׍M›6™æ͛Kîv/ĶÔ¸7"go&¨–+úe˖ųâKäŦˇE?­úLŦOõˆ/zūXķY%ąŦ5™0a‚T؞;wnh†ŽúŒ¯O%­ĐLqc–/_n:wîlļmÛĸQ%7l@ķæÍ3͚5Kî <ĨúŒ¯O%- &¯“.âšL ¯Áƒ;ųšÕĪr-N|ŠÃ&ĒĪ‚UŌ Ų,ĮB>Ŋ,vጤ-!ŽŠŌ~ÄÅš=nÕ§’–Ûs,ī'ˆzŨēu’Œ/ėBQØēuëZY˜5YŨ¨>GJwZÉÎ˞#Ī\)V™:uĒ!mIëĻsÕg.ĸJZéž]yßĒUĢLÆ %Õą™܆˜´ÆĒ5hĐĀíæ|ŋęSIË÷IčEž{î9C=2?V¨PÁ‹&=iƒœáąÔ7=zôđ¤Í 4ĸú<ĒŨia6ēØŠ(mrá….ļâíĢɯT¯^=Š^5Q}*iEbÎßyįæāÁƒž—3wÜ6mڈį;QŖ*aŌg—.]$š }ęN+ä3ÅŲŗgK)sō~Ī™3ĮÚwčĐA ,Œ1B ,DYđ߲]ŸĨåœú×)i…hæ˙đÃr“FüŨoŧ!n§žzĒ”YķÍ7åĻL§—M6.lXLđjÕĒ™ūũû›ģîēKŠ,LŸ>ŨP(ĒŽËv}:ŲaÅôŦ¤’O=ēũû÷KÕ>ķ …5)°‰`ĄBđĖ™3­¸UŒeŽäX˜×†EūųwŪyGˆ‹x˨JXôéDJZNĐ čŗHeBĩ”={öv\19í´ĶĖ3Ī'÷ b wīŪ-„…AÁ÷åĻ›nräâ€{ö!Ü °!q­Ny2ˇdũúõŌgl2­Zĩ’⎊æÃĘÉɑã"äĮŅ2Ę}ÆĶŸ’–ÅŗúË/ŋ” hRŅ`TĪ+‹/6UĒT‘ N…_jnņīÂæÕļm[“™™™–ÛFnŗŗŗMVV–ØŦēvíjēwī^ŦUĸqũüķĪB\›7o#>ųÅĸ,ļë3ŋ,ÍīžûŽĮ!ŒÖn åÉđųâ’iÚ´ŠÁ…"¨]ģļ€Ë–-+GJŽz0{÷î•įˇlŲ" úÖŦYcØņ|FF†iŨēĩ|ē)ė $Ä9ĒŅŖ­úŒéOIË™Ė.čž{î‘ŖU§N<ŽŒdāƎ_o‰ŋíÛˇOˆ ‚¸ °J•*‰3(Ć}ŠÛ=˛2đ7/;îíÚĩ3O<ņ„—Mž-õ ¨JZŸZĮwpøđáb§‚°š4ibYīũéî÷ß/ĮE.%ØuA¤*ö" ¤e‘înŋũvķõ×_K8ūW*ÎĀíņĮâēå–[œ}YŸ JZQEâŽP]'Vā`Ú´iô8¸]|ë­ˇdׅ=7ûPŌ ¸Î–,Y" ėžûî“P•ÔĀ=â:räˆøtq9 bJZÖÕ /ŧ ŪíدōƸËVumäȑBZüĮmĻŠ(iTO,(Č ûU”ŗ¸­ž×_]v]¸Ž 2Äíæôũi@@I+ ĻûŨēu3;vėŌ"K§Šģlßž]ˆ‹(Œô¸k¨%­é†ÅƒũĒN:‘ÎXā—JpD;wŽ×u×]įW7´Ũ"PŌ ČyīŊ÷„°øÅgņ¨øƒAį8ŖrT|đÁũé„ļZ(JZ˜ ¤‘!ūû•†›ø¯Ī?˙\~<¸UÄH‡ŋJpPŌōYä˛â–ûUĒ|Jčš'Mšõ9.RáY%(ių¨r>á8ĘËë˜<‡mUĶč†]q‹Ü0Ēø€’–:āfûÕšįž+š¤T‚Ā† „¸ČlÁq‘˛W*ū! ¤å1ö”ø"$§Qõ ōü›#*áÃ?âjÔ¨QŠoĶ¯%­â"WŒīaˇb‡ÅįmˇŨVŒ7čWüFûģ.ˆ‹[FīPŌōķŅŖGKéoKēîR3d_…¸¸8ųĮ?ūáR+úÚD(iy078 ’0ŖîYgåA‹Ú„ÛPĸ â"++ģ.R^Ģxƒ€’–‹8“ûU͚5 åŊT‡U„zč!!.Žū*î# ¤åÆ˖-“ILĄ“ú}.uG_ë"īŋ˙žėēn¸á)ƒĻâ.JZ.āK‹˛íúëëĀ|%9ō!.Šy°ëēā‚ ØËptII+Íz¤B27LدŽēęĒ4ŋ]_tž|ōIÃĨ s cĮŽAīŽ•ũSŌJŖÚ¸'nÂŗĪ>;oÖWŲ„5'ŲuuîÜŲŒ3ÆĻŽ[ŅW%­4¨‰b!*Ę{Š(Ô}„¸<(ĮÅ5j((iB@I+E ņæ†Ōĸŧ—Š"ŽŠÜ0B\Ô^TI%­0$a„…Ķ(åŊTxPŅ™]˙é[ęsDIĢ˜Ž;ÖL™2EėW-[ļ,æ[ôkQA€[EHĢdɒ˛ëĒRĨJT†žöq*iRJŌúé§rCxÎ9įã ú•¨"€ĪķâēūúëŖ CJãVŌrß7ß|#ĮAr_‘¸OE(ķįĪ—]W˙ūũ̀ŠķŠHGI+IõôŅGBX]ēt1”÷RQRAā‹/žâĒ\š˛ėēʗ/ŸĘë"õ]_H‹˜ŧ+V˜ĩkךM›6I01ÛˇoŸÁŗø—_~1%J”ÜܔsbgSģvmsá…šzõę™Æ{šésŪŧyr;Čä",GEHÄ-R{‘šuõÕW§ëĩIŊĮļu”g¤õöÛonQøÄ(Ų´iSSŋ~}É !QDāôĶOĸ‚° . ž‡Øˆ¨_ŗfÉÉɑį322¤20ŸnɸqãäĘ;Ä5×\ãV3úŪ#@Øģ.ĸ)úôéã*ļŽÃŧ ¸JZėĸpļD)LÛļmMfffZ˛>’Ķ(;;Ûdee ŠQņŠ6ėÆŌ%={ö4¤Ú…°jÕĒ•Ž×ę{|ōÉ'B\įŸžėēJ•*uė~Ā:dʕ+W,äl_‡ųí i-_žÜLœ8Ņ,Z´ČôčŅClA—]vYąOæKëׯ×ō­ˇjÕJ~­RŠlŗgĪé3×Ō–Š"āŊ{÷6K—.ââ4‚Ŧ\šRūŸĖ!Mš4Iē+ļ¯ÃDM+iÁč<ōˆ€Nų%Ē—pÔķJøEâ(Gz|§¸^vēķbauęÔÉPŪKEđr¯ąëb.ß{īŊrZá“S ö¯ĸ$ ë°°1Ļ´ +ųˆ#ĖāÁƒ‹ÂÕõĮ>@_† –t>+Ē CX8RŪKEđ ÕĢW q5hĐĀœxâ‰fęÔŠĻL™2†Ü]ü-‘„a…yʤĩjÕ*Ö›FkĘbE¸VƛפI“ U6uí&L˜ ĮÁk¯Ŋ6(CĐ~Dlĩ¯ŧōŠ9räˆ qķÍ7›W_}ĩ*aY‡É¨;%Ō†D:v&l_ƒ*üJaT'Į6ļüBi(Ü/ Ŧ ‘nPņÔ~š?¸lXcÔYŒ‘VéŌĨĨŒŽ?1 Ë:LÕb“ÖĀ͂ äŧmC 8lUÜ.ļiĶÆ7ˆāÆq_0ë„NH7}Np“O>YŽ…‡.ĐN‡ Ļ $ ëĐ)Å"-ĒËā˜6gÎSĄB§múöüwß}'í8Ģ˛ģ‚°Úˇo/™&U  đûīŋ ~ų嗆H vVŸ}ö™¸AđÊ˙"'.œl_‡dHq*ŽI‹…Nų$ËV¨pĮ@éņŽ‹ļŽKûn .œĢ_{í592ÚžŲ@0ܕœˆ#Ōb‡E&F› +ŽŽUĢV•\X*Š€M„iB\§œrŠŖu˜4iqvfKēpáB›ô[h_oŧņF1hÆl\Ą˜$´č:4&)Ōâvw|Dl˛a5sąq]yå•âA¯ĮÄĸĐŌ÷]‡G5P$iá˙Ņ°aCÉĘ`Ã-ĄĶ‰Å­"Y#•(ĖiĪé{õyE č:ĖEŗHŌjŅĸ…¤e ˛VĒ“?.\>øāƒT_ĨßW\A@×a’¤EHĀēuëŽų„¸ĸ€ŧ”šuëÖM:ä' ŨÖnD]‡Į+9áN‹ ËK/ŊT’ôEÁKd‚ĢIEã4Č:ëF‡čē Ÿ´đĮ"A_‚ŸŊš/Y“hЊ߈WũĶvĸ‡€ŽÃ$I‹<<”ôŪļm[äf ÕuH¯œJ>ŽČĻv]‡ņ×aܝ֭ˇŪjŽ¸â Ķˇo_W”䗒遰 ąĒ(~" ë0ū:,@ZœĄ›7onvíÚåi??'GŪļI$xÆgH–HĩmE+Ņ뇮ÃÄë°i=üđƀMŌbDUČÁE`ęc=UtÜ># ëĐH.ŧxë°iaĶĄjŽ›9Ũ}žE6OÎyĒüDŅĻW$8ú€'č:4&Ņ:<Ž´(/4dČąéD]°é‘˛ÆÍōdQĮXĮ]‡š¸Ä[‡Į‘Öũ÷ß/oŖäæháāū@Už§Ÿ~Zז"ā)ēsᎷ#-Ž„ļd"u{Å2˛EUQŧD@×a.ÚņÖá1Ō""đTtV9Š¯ņ'ĶŠŠ"āē ĸœ#-ęŠ8Ļ|YŠN2ōm(ŪŽ]ģT_ĨßW’B@×aA˜ō¯Ãc¤EP&ĨˇĮŒ“¸é~ˆÔągŸ}ļųúë¯åÕøH}úé§énÆŅû $ĩæ(úĒĸx€ë0V­*™ņųQy+˙:Ņ ÁĶâc%É}é„6)üX‡yI‹ōdd—ˆ'åʕ3M›6õ\ų×á1Ō"0˛đZvīŪmĒW¯.ÁŅpøđáŌ…~ũú‰7úŠ§ž*'¨æ‡ø…‹cÕ6ũGĀų–Ŧ!žr{‹õCōââûNkōäÉĻwīŪĮŽ†øÃä˙srr$§2{öl)gīĩčNËkÄĩ=ŋwZ…iĀ/ŌJ¸Ķōã, @U˜p|\°`į3ZmZžCųũX‡ÖÚ´ü¸ĩØŧyŗšā‚ Šœ¨ØŲvîÜiĒTŠRäŗé|@oĶ‰Ļž+üX‡A'­„ˇ‡~ø‡Œ9RėXÕs1ÄįRäņŽpŒüûß˙žŒŪĶöŒúiĨ J}Q’øąƒNZ ũ´üđÄ­S§ŽųüķĪ%á;ųũą¸Íäß<(ĮHũ{)ęī%ÚÚøą8—–/_^Ōą{) =âŊŽyĘkhīŅŖ‡™>}z\:tč`æĪŸ/˙ÆqōŧķÎķ /=ô jmčxŊi6ŲÛCžõß0öĐëčō˙ûŋ˙3“&MUáŖõ§?ũ)îÄÍëxĘqrØ°ažMpÍōāÔÚĐ˙đzÚ@Z ŗ¸lŲ2ÉS¯ĸø€ŽÃÄëđ8›VL9äĶá,Ųˇo_?ôåk›¤w&|hîÜšžöCWtÆ_‡qI‹p™Î;›mÛļEnæ`K˜7ožiÖŦYäÆŽēã¯Ã¸¤…ęˆēøâ‹ÍāÁƒƒĨI{ƒ›ÃƍĩúŽ‹ëĢ! ë° ^ I‹35§ųôŖ؅3ÕĻū4é/°a‘îYmYŠãŠoHēŧšnŨ:ķōË/§G~KĮŽMŨēuĩ0k€uÕŽé:<^ķ wZąĮČīCq ĘÇUˆr'ũ…Ö7 Ģ†í—ŽÃ\IZĢV­2 64~W{vkÚÛÔ¸qcŗråJ)Ģĸ]‡H‹GÉ*:qâDCÁ *Q§Åę9āŠ¨M%krzŠ(AF@×áQíšĶŠ)qāĀfíÚĩ’û*,Bž’šQUWE°]‡H …RȑÔĮ$æŗ]ÚļmkĒV­jfĖ˜aûP´˙C LëŌ€Ô4u˛“ŪiÅæ~#ŋūúĢÕÄÕž}{ŗhŅ"3~üx=FlÁÛ<ÜĪ>ûLŠĨžöÚkROÁöua1Ž—^zɑZ“VlĮE†Ev\6Ų¸°aTĩjÕĖ}÷Ũ'´ØčŅŖĻ+n"@Ĩ÷wŪyĮlŲ˛Å|ôŅGR­RĨJ™O<ŅüøãbĒáG×öučd‡ÃŧX¤Å—9[S@uæĖ™ĻQŖFnę0-īŽe@lĶĻÍ1ÖŪŊ{ŝŖRĨJâō@émE œ|ōÉBP‡.Đ~h_yåų{ÖĄSŧ‹MZ4ÄmÆ]wŨeĻL™h?ŽXĩ*ūÄģ%d×Å/Äī§“DŸ÷"4ƍ'kŒ#ԑ#G¤ĨK—–€~.b–u˜,Ę)‘ā?ŌģwoSŊzu9H‹Å0@ĒSSų§0?ŦĮ\Ü: Žk¯Ŋ6Yüô9EĀUēví*ģĒiũųĪ›V~ Ë:LĖ”I+ÖĄŖF’ŠŅA˛&ø™žPrlčĐĄÉ`!áJØšØ9ŪqĮI}GRÜ@`õęÕæîģī–ZŽ‰œʔ)cŪ{ī=qöN$aX‡Eá™6Ōĸ!Œ…€ļtéRĶ¯_?ĶĢW/O ’ĀoōäÉb lŲ˛Ĩ•Ķāg<˙ąs‹ČXT¯xūųį…°˜Ë„Īa7æ333ĶPą( ķ#\(IDATÃ:,lŒi%­XCä⨅[6$v/”ErKÖ¯_/×Ļœí[ĩj%îŠäÃÚŊ{ˇW•*U丨ĸx…Ļ~ôŸ}öYĶ´iSi–3ūŸlēMš4Iē+ļ¯ÃDu…´bÁøüJĖž=[l^8tōk‘ŽÛFnŗŗŗMVV–ØŦ8ûwīŪŨņÎĒ°Ā¯ÛĮ,ÄUĢV­¤'‹>¨8Eā“O>‘ŨÕųįŸ/„…{CL8A:tȔ+WÎékåyÛ×aūAģJZyŖ,Ōoŧaø„døå¨_ŋž$Ŧ]ģļË–-+GJuāĀƒ[ĪãŗB‚>œëđ[áųŒŒ ĶēukųtK¸\`›q]sÍ5n5Ŗī0ü¨CXØa9%¸)ļŽÃŧ˜xFZyÅ!Ûnü @Hümßž}BTāáC…3(Ć}ŠĢ^˛2đ7¯„ô˧M›&ģ9E ]<ôĐCb§bwuõÕW§ëĩIŊĮļu”/¤•ĸ{ˆŨąGũȑ#Ö;íŽmāŽÃîĒråĘBXå˗ˇmžõWIËôß|ķ\*°Ë{á…|SUr˜?žV˙ūũŏPÅJZÎđ’§īšįžc0¨ŪŖĸ$‹n8ØGŲ]]ũõÉ~MŸËƒ€’V1§9¸pBezm‹(f—õk>"Āeģ+lĩ„“áNŖR<”´Š‡›|‹‚Ž‰TŋũöÛSx“~5ĖpkaņßđáÃÃø@‹ŌæC† ņ¸um.(5G„•ŽėģŠô%ĘßUŌōAû;vė‰ ‹Saׇ.GēÉ 6ČîŠ,ģuUüC@IË?ėĨLŲÖ­[ÅÎåe&V‡l]Ķå!,ębRtBÅ”´|Öĩ"I(ČâhŪŧšĪŊŅæķ"@ŧ7ß|SvWW^yĨ‚”´ Ģ[ˇnB\TRņâGŲ]Q<ÂĸVJpPŌ ˆ.¨Œ‹Å2hĐ €ô*zŨ Ę8ˇ‚\’<øāƒŅĀ‚+iHIÛˇoâ"nĢuoāĮ‚ÄŽ`Ũu×yÛ¸ļ–4JZICåŨƒšaÄ@Ægx×pD[âĮ‚î)§œ"„EŲ:•ā" ¤PŨPĻ ąįĩŠ;PsÂęÕĢ—úÍšqÚßĒ¤•vHĶ÷BnqB…¸nŊõÖôŊXß$đÀĄ˙¨TŽbJZ×Ķ’%KÄÎÕŗgO3pāĀ€÷֎îíŪŊ[vWGŽÂâ–PÅ”´,ĐÕļmۄ¸.ēč"3mÚ4 zÜ.žõÖ[BXāųČ#ˇŖÚŗ„(iY49H(øÕW_ÉqąjÕĒõ<]7nœxļŗģjßž}0:ĨŊpŒ€’–cČüũeĘæĖ™#ÄÕ¤I;cIëß˙ŊėŽöėŲ#„uîšįZŌsíf<”´,œ3gÎ4÷Üs——ÅØå­XąÂŦ]ģÖlÚ´ÉlŲ˛Ev~ûöí3äH˙å—_¤‚2ä¸ OIPø…^hęÕĢg7nėyŒåŌĨKÅY”šƒO<ņ„…Úv¯Ë6ę4”´Ü›ŽžųŨwßģ Wõ p­­ˇß~ÛP!™OŌG7mÚÔÔ¯__2@HąO?ũt!* â‚Ā¨ûĮķÛƍ͚5kLNNŽ<Ÿ‘‘!ˇu|ē)'N”č|¯4<ę(Ō6ë36W”´Ü\5.ŋûË/ŋâ"Õī”)SŽkÚ{UĒT1uëÖuÜ vQėæ¨*É´mÛÖdffĻ%‡Y?ŗŗŗMVV–dŌŊ{wŲĨK~ūųg9nŪŧY ‚˛ØŽĪüēSŌ Ál†¸¸Æį¸XšreŪŨ‰KIV–/_n؝,Z´Č×žd…—]vY˛_wüÜúõëĨĪĪ=÷œiÕĒ•éĶ§iÖŦ™ã÷äũģ9Ģe˖fŌ¤I)ŊËö/‡AŸņt ¤eûĖü_˙‡jö…ʗ/o4h`>,ÆįŠ+:J~‰šūĮūC:ŽœõŧŽ”TĩĄüdÃXŠŗķš:uĒôc;šĘĸ*aŅg"ũ)i…hfSĻ 'TvIxĶŸp Bũû÷O8JȊœ^#FŒ0ƒöG}TúB­HúžŦÜ{īŊfåʕBX—_~y˛_ ŨsaŅgaŠQŌ Ų´3fŒ,xJ\!øsíÚĩĢĀ(W­Z%™8ąYáŋ$7Ę­qš€Í‹#ģÆDÂM&ĮAn'!Ŧ¨JXô™Œū”´’A)āĪ@P?ü°™?žÜÚá—“ĶN;MĘ\å]ƄÆ{v(AŽ{ė1Ļŗ{Ė/ŗfÍÂÂĮsQ•°č3Yũ)i%‹TĀŸÃ˙íˇßŠ=k˙ūũĮõ…ÕĢWË߈_\°`ÜÚPQ†ÛFnÛ´icƎ{l\íßyį!´(gÁ‹>,/%-'hüŲ~øÁPĸlŲ˛eâ[ĩnŨ:sꊧšC‡IŽsRŨāPˆG}… >šÜî}÷ŨwĻK—.☊}Ž]b­Zĩä8XētikƑÄv}§•’VēgRĀŪĮĩ7$…×:ĮHūßVéĐĄƒ¸c`¨ŋ˙ūûmFZú;Šíú䇈rlÜx;%-'hYú,ŋČ´š°bĐsL$›kq~Ą-U_n‡IŸ>…NôŠ¤–™œ`Ø<¸a[¸pahFzã7Ęma^WhWÄ@TŸ{ęšÎ­7kīŋ˙žU6Ŧĸ”‚‹:„ããŨ*õ}[˙]õyTsēĶ˛uŅoüv6l(Yl¸%tĒnɁCia~\NßÔįUŸššQŌ ę,Mą_-Z´`ę ûaĨ8Dƒ7ĸܘ†]TŸJZĄžã„rāî@,bØĨcĮŽ’ÉÂIȏm˜¨>×˜î´l›ÁEô—`YRÕđ¤Đˇ`&ä‡āę 6+ČÚ­~ĨëŊĒĪ‚H*iĨkvä=øī?*ÁĪ^AB5‰úûxÕŋTÚQ}*iĨ2˙]I;wîl¨Ū59įœsĖŧyķRÎĮ$ÜTŸņõŠ;­ ÍŌûBP4qx}ûöMņMö}}„ æÃ?t”ô0čŖT}Æק’VĐgn’ũÃöŅŧysICãeŋ$ģįúc$ÄSž¸Ëâ$tŊƒP}&Ö§’–ÃÉÔĮIMķûīŋKnŦ¨ 9¸H|øØcYęĶHNĩxúTŌ˛~z62;¸™Ķ=čP‘sž*?a°éŠ>I¤O%­ ¯Ä$úGY¨!C†ˆM'ę‚MoôčŅŽ—'sgÕg.ēņôŠ¤åæėķčŨ¤i! `”ÜA‹ûÅ<ž~úiĐO3ĒĪ\LãéSI+ũsÎķ7r$´%ŠÛāÄ2r´°UTŸšš‹§O%-[gö˙úMæJ<āÉ ¯r*^ã!OĻSÛDõYPcųõŠ¤eÛŦÎ×ß×_]‡Ã”/+U•o‹@qŠÕÚ&ĒĪ‚˯O%-ÛfužūLKxJ‡y-Ûˇo—"ĢŽ‰¤gžyĻ9īŧķ$ÃΑ~äp4h)SĻŒ•AÔ~é]QP”Đ?.}ŧ”üúTŌō}Ú"6-33SHÂK™;wŽTq†¨ U€(¨Ķ§—Bēšėėl+cũŌgI+ŋ>•´ŧ\M.´Ež%JÉä镐 õüŖÁ šöÚkÍ-ˇÜ"7˜[ļl‘KĪ>ûLū­I“&&''ĮĢŽI;ôĮÄ æŲzꊧ¤Vcšråâbâ‡>éH^Ōâæõĸ‹.ŠÛŋ:uęxž=$ŋ>•´<]NéoŒô3ĪŧLCC¸Áŧ9ËķMúé'CžĢŦŦ,yfɒ%Ļe˖é|‚7rTÍČȐ#kЄc+UtxāÃą'?yųĄĪü¤…nƒTK2ŋ>•´‚6Ģö§bŊfëÖ­žå€ĮŽ…ˇ6RŖF ŲYQ*ŋPŌ~ņâÅĻUĢVbįōRČ!_ŗfÍEkŊėCĸļČŲOˆÎoŋũ&!*ødå%/¯õëgŪVĐH+ŋ>•´‚0“SčCɒ%ÅŽäUôŋūõ/sķÍ7K)*A…į  ĮVv4GŽ Zפ?\áS‡)UĒÔqäUŠR%Oõiiåק’V §uōâך@i¯„Š0TxFFŽi† æUĶŽÚ›„\4æĪŸīŠ>ã‘Va6-Š‰@ē^KŪyŽ¤å5úinĪëÖķĪ?o(ŠPéyøđáiQę¯ úN‹ ‹˜3pwZ…i€ ßØ ŊŨiy‰ļmym!“Už‘ÛoŋŨĖš5˃Q:k"č6-./0ÆÕĻ4ŌR›–ŗųø§Ŋžm"šjÕĒr„áÃ<ģ…üB8 ĮHʞcŖôšWĸˇ‡Î‘˛!^oë3Đßđ;.¤W¯^fŌ¤IĮaôķĪ?‹ßVėūįŧõĶrŽtIKũ´œë3ĐßđÃú“O>§Ņ6øm1éņ|'ÎĨ<ƒą€Hũxģ1ˇ€UxįČ™´Ô#Ūš>ũ ŋbÕūũī‹iaŲ% ,šĪ>ûlO1ÔØCįp™´4öĐš>ũ ?ŗāk4eĘ9~úé§æĮGR KtëÖMHÍËVLQšåÁų” 2ii–įú ô74˙RAõh>­@OYĮĶ|ZŽ! ū4ĶeŽŽ4siđįĢ“jæR'hYôŦæĪU–æˆˇhâ&ŅUÍŸH6>ĸÕ[rĩĻÕxlœÁ‰ûŦÕxÂĨĪãFŖuō×ÉŗQíĒO­{hãŧuÔg­Hœ¸"ą# ō°ęS+Ld*ē׍M›6‰“įŽ]ģ5˃íŗ;O˙ņĩÁСoߍ*šĄL˜0A*l“ģ>,ĸúŒ¯O%­°Ėpc$rįΝ%”&j‚ ˆj2͚5 ÍĐUŸņõŠ¤š)~t Ä"^|ņÅfđāÁ!Yâáp-žqãF+ĢīĨ$ÕgA„”´Šš5–ũ;ļ*Nķéeą ŋ`"m 6,*J‡Á–•GÕ§’–_kËĶv ĸ^ˇnyųå—=m׏ƈoŦ[ˇŽ•…Y“ÅKõysUŌJ÷ė ČûV­Ze6lhVŦXa5j^Ĩ¯ĤQdaåʕĻAƒé{q@ߤúTŌ čÔLoˇ¨œC=2?V¨P!Ŋ/÷ņmä §ĸvŸ>}¤ŒYTDõyTĶēĶ ųŒ§ˆÂÚĩk%_X„üJõęÕ3cĮŽ ː’‡ęSI+éÉbķƒ”ü:xđ ™3gŽÍОS(ƒ"3fĖ°~,Å@ÔõŠ;­âÎËž‡ŋeĢl&.뤓N Ĩ?–Ķée}*i9-?Ī/4™N!.›l\Ø° ŦjÕĒEz‡•ęEUŸJZ“PqēŽMdÁ‚R1Į†[ÅXæJʖEŅ†U”ŽŖ¨O%­ĸfE˙[( ŠR”"Č~\øaõėŲĶLŸ>=Rˇ„N§\ÔôŠ¤åt†„äyü~z÷îmĒW¯nƍ¨Bs `vîÜ)…`Ŗā‡•ę´Š’>•´R-–Ÿ‘QŖF™#F"Țāgú2lØ°P‡æ¸5mĸ O%-ˇfEī%(—ÉžtéRĶ¯_?)a_ĸD ĪF@ŋɓ'›ņãĮ›–-[ Y…1øŲ+@ÃŽO%-¯f’íŋ úE‹‰ ‰kuʓš%ëׯ÷l2­Zĩ÷0åÃr ˇdßV}*i%;"ôŋÔÜ.Ξ=[l^mÛļ5™™™išmä60;;Ûdee‰ÍĒk׎Ļ{÷îēŗrq~…MŸJZ.N–0ŧšōd”Ŋį’iÚ´ŠŠ_ŋž$Ŧ]ģļ€Ë–-+GJŽz0{÷î•įˇlŲ" úÖŦYcrrräųŒŒ ĶēukųTņ0čSIËÛ9cuk8Ļ’5‚XF~Ŋ!$ūļoß>!* â‚Ā*UĒ$Πö)bÉĘĀßT‚€­úTŌ ÆüŅ^(Š@’(i% ”>Ļ(Á@ā˙40ˇ ēÂĘIENDŽB`‚avocado-113.0/docs/source/blueprints/images/BP004/dot_representation.png000066400000000000000000000630541513561306000261710ustar00rootroot00000000000000‰PNG  IHDR ØÃÃsRGBŽÎéUtEXtmxfile%3Cmxfile%20host%3D%22app.diagrams.net%22%20modified%3D%222021-10-19T20%3A17%3A16.794Z%22%20agent%3D%225.0%20(X11%3B%20Linux%20x86_64)%20AppleWebKit%2F537.36%20(KHTML%2C%20like%20Gecko)%20Chrome%2F94.0.4606.81%20Safari%2F537.36%22%20etag%3D%22dI4AMvyDcCAjtft1XqpG%22%20version%3D%2215.0.6%22%20type%3D%22device%22%3E%3Cdiagram%20id%3D%223wabN0Xgkd7uKRWGGPpy%22%20name%3D%22Page-1%22%3E7VjbjpswEP2aPLbCECB5zG23UrtSpVRq81R5wQFvAVPjLNCvr1nb3Jxs2TQSibRP8Rw8Y8%2BZOQ5mYq3i4p7CNHwgPoompuEXE2s9MU1gWDP%2BUyGlQBzDEEBAsS8nNcAW%2F0HKU6IH7KOsM5EREjGcdkGPJAnyWAeDlJK8O21Pou6qKQyQBmw9GOnod%2ByzUKAz023wTwgHoVoZOHPxJIZqsswkC6FP8hZkbSbWihLCxCguViiqyFO8CL%2B7E0%2FrjVGUsCEOD7v88%2FJbuYDxzP6dLhfu4s74IKM8w%2BggE4ZJhh959mLPrFRE8GCcc24s8xAztE2hVz3Jedk5FrI44hbgQ5ilohB7XCC%2B9nJPEraVgYCyRaHBlNtyC4gyVJzMDdSM8VZDJEaMlnyKdFAcl10zbyoGVN%2BF7WopEMouCerIDZF8ILl8A69TjVfO16%2Bq126KV%2BBcG7G2RmxWZjjZk58pvTFyLePayAVAZ9Dnx6E0CWUhCUgCo02DLik5JH7F2drgVjPnCyGpJO8JMVZK9uCBkS73qMDsR%2BX%2B0ZbWTgarxuuibZTKSHi%2BLafK3Kl4ldG4vVjKb1hZq6RfLyrniByoh14h05H%2FVpAGiP3rqNCbhKIIMvzc3cfFK%2B5ocmIoY0fa4JqF1D%2BlantEJRk6he9KOlNJ7kAlOWMqyT2qJPO2lVT%2FRY2npPm7kC4mpNlAIbljCml2%2Bg2PZOy29TSdjq4noF9NdE4Tf1HdnbnlRTDLsHdCHm8XB7hSaQx9WztR71Y97SPlVNhgBckVvhLME6nbye1fGHpNIrKUTu1PAb04oHdfBnYvkKBBC%2FTScHXW%2F9GD%2Bi3u7B4EZxzQN96D9pg9aM17rwjmmU1Yn4Wqm91LNSE3m89qYnrzcdLa%2FAU%3D%3C%2Fdiagram%3E%3C%2Fmxfile%3EøŪ IDATx^í] œÕÖ^(|âÔ¸D5§pŠNēLŽ‘J“ĄŒJž\B8BGšĻFd"šPČAH)Q茩ĄÂIår Š“ú¨pJ"ƒJMÆ!ßīYõŸ3Æ\ū—÷žŸũûÍo\Ūw_žĩŪũėŊÖŪk•;~üøqa!D€#(G0Rî4 D@đ üōË/’““#[ļl‘mÛļÉΝ;e׎]˛wī^9pā€:tHōķķåØącRĄBŠTŠ’TĢVMĒW¯.ĩkזøøxŠ[ˇŽ4hĐ@5j$ Rž|yĒA@ žT°6 Ëd}ņ, äååÉĒUĢäíˇß–õë×ËƍuōÆOÆ Ĩ~ũú:ąc‚ĮD ŋråĘJ ‚Ç+1€ @ ŒíÛˇËÖ­[•HđĶŦY3iÕĒ•\sÍ5ŌŽ];ŠRĨŠM*ÆjíF€úb7ÂÁĒŸúō_yzŠrsseéŌĨ’••%˗/—öíÛK›6m¤uëÖŌŧysä­* ‰÷Ū{OÖ­['kÖŦ‘•+WJĮŽ%))Iēví*5kÖ´Ē)ÖcÔ›€ hĩԗâë xõÕWeūüų˛lŲ2INN–.]ēHbbĸœvÚiŽŠãO?ũ$ŲŲŲ’™™)K–,‘Ν;KĪž=ĨS§NŽõ …‡õ%<œøÔ¯P_J××H6¸§žzJfΜ)qqqŌģwošũö۝øK‚„°`ÁyîšįäāÁƒ2pā@šûîģéCpqVĄž¸ž›Ļž„/4WH ##C&OžŦĻžAƒŠMŪĢ>‰3f¨Éhøđᒒ’âÕŽļ_ԗĀŠÖ–Q_"ƒÕQ˜3gŽŒ;Vą#FŒÆGÖ[Ÿūāƒdâĉę¤9r¤ôë×ĪÅŪ˜Ņ4õÅ 9[5JęKtH:B˜8ĶŌŌԜ2jÔ(O¯ü˂;ƒôôtÁvsܸqJh,Ö"@}ąĪ ×F}‰MÂļ“&Øũ'L˜ ũû÷­ˇz{öėŲ’ššĒū‹5P_ŦÁŅ”Z¨/ąKÚ6€ųdđāÁR¯^=ĩ˙׊S'öŪzŦ†={ö¨Ÿ`ĮŽ2}út_™ˇ<ĨP_ŧ&o÷‡úb|l!Øæ°ę‡C+å ėtāāÆ‚ČĨM}‰3“ß žX+}ËIāŪ{ī՛žsįΕĻM›ZÛ[×ļiĶ&éÛˇ¯Ū<ž4i’‡{ę­ŽQ_¨/‘h$õÅz}ąŒĒáļÛnS‡).~9yŅ+%˛ķYÜ/Ā38Ā-Z¤!,XŠG€ú"B} ˙렞ا/–âķ Ôâúā˜éË›!0׈åD¨/'âA})ũ ĄžØĢ/1“‚ŗ!Ūâüā˯ā.â!‚ÜąüŠõĨxM ž õÅ~}‰‰ĀĐ:tĐĀkŖGæYÂCŽsd63ņø(õ%<)üõ…ķK$Z‹žDL¸¨1mÚ4 ¤fâ1ĐHSøYDœĄ!C†uĄŒúÆP_8ŋDĸ9ąčKD$€ĢÚMš4ŅT&]‹DĨ=‹ eHišyķf#BLP_bĶęKlø™öv´ú ´lŲRzõęeD(ģ!&ŦfÆ v5á™zŠ/ą‹‚ú;†&Սž„MˆÖ‡@i/žøĸI˜Ú2VdPC`Ŋ GĨžX§:Ôë°4ĄĻHõ%,€ũšvq 6ˆŅ@V DÅíjä> b>ę‹ĩE}ąĪ ×Šž„E×^{­æ˙ R>ˇGąĢzë­ˇÜîŠåíS_,‡T#ÔR_ŦĮ5¨5Fĸ/e’Nw,\¸PV¯^Tŧ\WÛļmĨ{÷î:-D}ąO¨/öaĚÃ՗2IgÛáČôr2xŋ Š*áhĮ‹ ę‹}’¤žØ‡mkW_J%ŒŒ ÉÉÉĄ3ØF ™-!!ARRRllřĒŠ/öãL}ąã ĩŽž”HČ pöŲgk´Æ Ogé€o÷îŨš‡Ā¯…úâŒä¨/Îā”VÂŅ—I`ęÔŠz+ÉQXėEÉxpJhčĐĄö6dcíÔÁ-R5õÅ9ŦƒĐRYúR" \tŅEš#˜žûÕļ;ä(ūä“OėoĖĻ¨/6[LĩÔį°BKeéKą$€ķë&LĐČ,Î €ÜŠŠŠzÃo…úâŧĨ/ÎcîįKĶ—bIa\‘)ėŽģîōķ¸}Õwœë}ķÍ7}nšúâŧĒQ_œĮÜĪ-–Ļ/'‘@nnŽ:„<Č(ĄJQãââÔA\ŗfM[Ž­)ęKløEû6õ%ZäĖ|¯4}9‰f͚%°!1Fķʂã\đÁ 0ĀųÆŖl‘ú%pŧF}ąDƒĒ(I_N"ÄÄDŊĀtË-ˇ7†úōË/ëÅŧėėlot(Œ^P_ÂÉĻG¨/6ĐjKŌ—H //OM@?ūø#MA.(ļlUĢVüŽRĨŠ =ˆŦIęKdxYũ4õÅjDƒ]_Iúr deeiÎ`8(YÜAāēëŽä˜MJJr§´J}‰,›ĨžØl@Ģ-N_N 䂭QŖ†<øāƒ…ĀûÃzôŅGe˙ūũžČEL}q_Ÿ¨/îËĀO=(N_N d‚Âũ^sOŦīŧķŽÜ˙ũžČũ°}Ä*ĮtņÛĢ…úâÉP_ŧ# ?ô¤¨ž’ƒ5JđÛĪ+čSO=Uj׎-{öėŅuáŋ7ļ‘#Gʸqã4töĀ]>’Ī§§§{: Ŋ_õ%}p]"ėõ%|Āĸ•?VĪO>ų¤Î'˙øĮ?¤QŖFá7ęą'‹ęK ü’%Kŋũ\pjdV­Z5=jŽĐŊFČ+œœėéü~՗pôÁoúO} _bEį‡pŪüöÛouQ‰0ŅŪzë­ž ōXtŒEõĨ€ęÔŠŖų„ņ;H%œŪk$€ ōãˇW‹_õ%}đ*æ%õ‹úb¯ÄpbßãŸūô'YŊzĩŊ•PûņãĮ?V¤ -Ē/$€˜A`<üļģÔáÇËƍõNŠL™2EĪĮ/[ļLēté"#FŒĐžL›6Mī/<đĀé˙ũīëû¸ø€ø58ō4~üxŊäV’9áąąCB÷Ŋ{÷jĪ<ķŒļ_”ōķķuÔšß}÷4iŌDÃi\~ųåvCŖõ#ÆĮ™gžŠŋŊZėŌ—C‡Éī~÷;Ũ‘2DƎ+›ožYæΝ+•*UŌæ2Čoßž}rÁč%Gœ¨BĀq=ū| ‰~éĨ—ę˙ˇiĶæ$ũ€ū\qÅ˛mÛ6Yŗfūš4ũDũˆÍūđÃĢn`UxÖYg nb"ōn÷îŨÅiũ ĸžĖœ9Sį„;wę -{ė1ųčŖ¤yķæŌ´iS?PŽ;&ĩjÕŌīÜ?üöüđũ÷ßËgœĄõĨĨĨé{¸ą8ú€ˆĘČ­R¸üķŸ˙Ô9'##Cž~úiųęĢ¯t§Ųc§-ĢāļũäɓuNBwĖeĐ#´ ßl¨_͚5ĶąCß^ũuiŨēĩ rīã?._~ųĨÔ­[Wŋ¤ ˇ՗ĀÉ (o… ­+Ēįp ĮĮ‰ŧ´$>*€?iŌ$uˆ^ũõrîšįĒ°1cčļ1 6ÔÉīÁN‡ 8&vĻ8ŸĖC˜X0ųã >>z˛y%€† &-Z´Đ0?˙üŗ|öŲgjf˛ģ@ŠĄH˜ĖŧZėŌ—˙üį?JúÛļmĨ˙ūĒđČwîžûîĶ…ō1cņ€„6 v|ø_|ņ…îdaÄĮˆ•t ū;Ũúõ럠øpAöp–ũī˙ū¯^Ō+M?Ą7^xĄö}ųüķĪUgATˆŌˆū8­?AĶØŦ!;L† Y|똁ëĉåâ‹/–-[ļčä/k׎•ĢŽēJŋíŋũío͐7t 9<0ˇ`ō‡~mßž]'øoŧQį'œšÄü™#Žū sGãƍuFä_Æ=÷Ü#O<ņD™Ÿ- ‡ęÕĢëoLĖĐcƅ?¨Đg“XC˙Đ>°Áˇ>ŽZĩJŋ,`Â)EõĨ€ʕ+§+,ģ >s# 1’¨œsÎ9:ŲoÚ´Iæ=īŧķTãÎ;īԉûŲgŸÕ0×!ŽzĮŽú‘ãã‰G>Ø?üáēâƒclŽ‰Ą0 ôíÛW'zõŦ@ÔmÁ‚‚3ļN§äíXėę_h'‡úwíÚĨ ŽāˆCˆ lÉą ûûß˙ŽĢ č tđđQ"Í%v§Ÿ~ē(vļX$ŧōĘ+úƒõ†n(Đ|Đø°@ō8€R–~báUŲ_ūō%čV¨‹/–Ν;ģĸ?vÉ#Zũ(ú^$ũÃâ 0ČXcÁąaÝ o, 0abw>xđ`ĩ`!"áXįė 1ĪôîŨ[įĖ°`Q‚ų ~ĖXBGą Ā$Ž•8Lč_ũĩŠU>ô˙÷$pĶM7釋úđ°u‚r`Bž>}ēū_ƒ ”%Ą X!”dĶÃtúĐ(ģ|ô…I ]ģvj^(Ž€Ŋaž°ģme ^!ųa‘€I>ėŧ>ūøcŨ9âÖ2œā`ļpáBŨšaaŊØēuëIM‡ęĮ"īā‹ L¨”ĻŸĄ ō>ôėŲS_Á8gÎ%•Ķú4}ÁŽ‹B˜D`vlÕĒ•`>Î ˜|°ĀBnåʕ:  kVĪŅĖ!Ũ‚<ąĸ‡é0´ŗ(JXBĮ° ÄÂ2T`R„ s"úTZ Í9˜Įūüį?ëŖ˜Û@€0ƒáß@ĄĐūĄē0nĖ›E rcGN)q'`—ˇh§°õÆ‚Åļ zõÕWŸDØâYLŧ°ņb›…ív,`Kė(`KÎūL õęÕ+v'PxRAJDü ė…IĢËķĪ?_ÍQ/ŊôŌ ŨĶc5bw ĸ7\ĖB“4HrByÃG’v|˜āąCÃäÕ"V |ˆ˜„ņƒ­sŅĒ˙Ž]ėø Ŧ4ą:*K?Z ˜Â$Đ¯_?í H+F§õ'ˆú“0†i˛ÁĒ;>ØÃQ:vė¨€Ė4ĄU;ū/šų!t¤ī‡ĖKØÕaž)J˜o@3VâĄâÂô¯ũK¤áæ@ ˛ũë_• TėŅöß˙ū÷ēûxíĩ×4âs¨`ūžä’KÂúĖJô 8uÚ#d/ÅV[j ļŨĐĘ-d*‰@† “ lÅpäâœ?>ČâĖAøĀąZˆØNa[2Ag‚ “Ā… ŋųæ,&"ģ‹É§= OŌXbŅ6PČ “L<øÁJÎ3˜1ĄÃĻ‹˙Ã+ClÛņ!Ö ›Qũ€ GŖC>˛ôļaø¨Ę29Š?AĶ|k°ųãÅw <Îåc:1ŌĮ°Ā/ƒ|JŅĖ‘@asVåX Ā Ķ%úˆkYA8Csv“ ;Ŧæą+…Yŗ°Ų˛đ)Á9§”`A[Đ},zāË §”x:ČŠsß!§Vp˜Đᜅa–Íöxŗ$ĀvĢtØ}á˜DØųÁāp&&ŖĪaĢĨÁj{hgQÔ1ŒĻ'ؗ! üĀA1ąĶ°ģ˜|î;DV6ĄS ČaėⰃÃßCŽYŦúqŠöbč&}|$°­âĀvØĒcáPX?đ>t2v8q„6JŌO,V°€9 „~¤đ1b'€ū:­?AĶ,Č„C ˜hņwü6s˜ą3Į܁ Ģcø1G2?wd¸ŦÚĮŧŸæĖU ĸuëÖéĄ˜¯Ë*Ą9‹J˜š0ĖK ĢE}›ĄúB§'a>ĮbéŌĨ:÷a‹~„SJŧ'āÔ PÉÊ Ė æÄÖtØJCJ#˜ƒĀ–pa›!âēÛûĸB…ÍÄĐ@°ëc “ļ^˜Š’ļž¨Ģ ŧ:†N“o€†ä:v yálCō‡û)&öŽ]ģĒS:=‚ '‚ Ø„ŽˆâŖ…ŋ§¸“>ˆ&GčHiú ŨƒÎBOąĀĀŗ¨§4p2vd§õ'hú<1ÁB–X|Á^ “d¨ĀŠ ܋žČ‰d~ˆ–°€ŪĄ}ÖX B_Ã9ašs`ęé@†XɃŠ;ę^xŪ ´ ķ4æN˜ĘÂ-%Ūf,˜p!´˙9“cÁxũ2ŽbĨģ3(8Fˆ#‚8š„ŨĻĶÅT}Ád Ÿ ƒX¨ųŠ¸yAĩÄØAŒ é29*¤×I'Ø`ŸÆîß v•8į lŗ8Ŋât1M_āûƒ%!t”~E/ij)Ž_¸[ķĨ[ņĘJŒ"ĘøđŪQ#“ãÃ{ %°˙bB€+R˜*`‹vŖ˜Ļ/0ũŊûîģz{ö…^đe¨7w%æ`Ļ(7>ßâÛdĻ(īČÂ=ĄžøAJŪéc‰™Å˜3ÖBbÎXoČÁ/Ŋ žøERŪčgŠ9†ŅE‹C&\ŪbqÜVÄ \ņzĄž¸/!ę‹û2đSŠĶ—‚°Î=ãŽgą¸ƒœŒ8#`^/Ô÷%D}q_~ęAqúr ‚2ÁķÎâ8K ˆčõB}q_BÔ÷eā§§/'.r…9‘WĀOā9ŅWÄô@ $üƍY¯ę‹ģĸž¸‹ŋßZ/I_N  ×ėFWđYœE7”üR¨/îIŠúâö~lš$}9‰pmÎ\šgqœßřs\‡÷KĄž¸')ę‹{Øûąå’ôå$@<Äô '1‚đjŸąUCŒ$ŲAîŋę‹;’ĸž¸ƒģ_[-M_N" áuCéËü:hŋõ¨‘(ũV¨/ÎKŒúâ<æ~ną4})–Īō•ÅK÷[Áo…úâŧĨ/ÎcîįKĶ—bIƒEŒuj‚šÅ^āƒA&!ÄųökĄž8'9ę‹sXĄĨ˛ôĨD@ĖtÄå.-^ōé6×Ū¯…úâœä¨/Îa„–ĘŌ—Ią„ā ^ž|yAĻû âĩ1 pōĨÂ!Œ”„~-Ôg$G}qį ´Žž”Hš{‘°ĮEíS ÛB"’””ûq¨fę‹ũ@S_ėĮ8H-„Ŗ/Ĩ’Ā@Ę3\`ĸoĀzՀ­ķvîÜi}å.ÕH}ąxę‹}ØąæpõĨL˜3gŽ,\¸PŨžÅZÚļmĢšŽ‘P;(…úbŸ$Š/öaĚÃ՗2Iā =ļũû÷"VŽŒiöėŲjfCŌį ę‹õĨžXikŒD_Â"œÂųõ-[ļø2•›×„ŊgĪÍS‹ķõ8´B}ąVĸÔkņ zm‘ęKX$ĐŌŌŌdĮŽt[ AØUÕĢWOMĩP_Ŧ“,õÅ:,M¨)R} ›2IÁ‘9pā@°´eŒ3gÎTGģ2‡Å õ%VE¨/ąchR ŅčKD$€3§Mš4‘7JĶĻMMÂ֒ąnÚ´Iš5k&›7o6âîõ%6ĩĄžÄ†ŸioGĢ/‘@ÅéiĶĻémb&ž _ÍÅö˙!C†ę4PYP_ĘB¨ø˙§žp~‰DsbŅ—ˆICnYœm_ēti$ũ4úŲŽ]ģę ?äļZPԗČĨžp~‰DkbŅ—¨HCøāÚĩkkNb–Ō@ÎāŊ{÷ú2L´U˛Ĩž„$õ…ķKøÚ"š“<–ų%j8vė˜´k×NĸtėØą‘ôŲ¨gGŽŠ!šW­Z%*T0jė…K} Oôԗ_qĸž8§/Q“ēxāĀéĐĄƒ@=ztxŊ6čŠ1cÆhž+VHõęÕ yņCĨž”ŽԗņĄž8Ŗ/1‘ēˆmHRR’´oߞ;‚B2Ê™Â7n,HÁˆÛÖČÖfzĄž¯!}ÉĘĘR3ĢéåÍ7ßÜzEĒUœ2ãürĸFXŠ/1“@hGĮnÁŌGđĢˇĢá8GŌø%K–čN[\ātë­ˇMXáQ_ūûQÖ“wŒ˜ø_zé%ũn`:…ž$''ËŦYŗ¨/…8Āj}ą„B6<$/@\ųųķįy|Į´zöėŠyŒŠ ÅF8(t¨„ū$’šūúë[øŠ/'ë‹iŠđÆoč‘ķĩk×Lü…ŋ¯Äšú"RÜüb…žXFĄÎā8 œ sįÎ5ęB.jôíÛWåEžqÆōũ÷ßČĢRĨJrĘ)§HŊåõ×_—æ͛[!K_ÖA}9Y_|)Č(;ũūûīË 7Ü GŽ‘ŖGJ~~~AM§Ÿ~ē|÷Ũw'ÔL}ą^_,'H „`GŽbBLāĒ6rÆY\XhÃ3Ī<ŖĘ ¸ųæ›5‰ ’É°P_¨ĸÉĢ”č•W^) ‚;īŧS“E įk5Æ@áĖž]ĪÎΖoŧQM§ D#™‚ŠĢžúbčŋUa „ēˆh’X)O˜0!Pų°ęOMM՝N8Ņ@aÂvļĪ¯ŋūZžDp˛"S_Ŧû¸ũRSˆđ=œsÎ9Ōēuk5—57ęKėRļĐEÄ‚°ā05j”¯SU"e[zzē:Ā1ų‡›“>p9ˆ +>w'*2õ%öÛ/5÷āā"Đâ˙Â)ԗpP*ųGH Ô~@‚YLU<đ€ûCP˛ņãĮÛۘĩSũ!t’€ĮåÄÉaPĢV-5!Ât¸oß>î,„šzk!˜6WE°`+ĒįeŠ'ց].,"„1Bzã" wÖāL}ĩG§j! 8…tŒíđÊĀB¯Ū„ū™ģkđĨžZƒŖ“ĩœD;ÆļøÅāo¯Ū„jän vlŠŸącčF $7PĄM~h1€'ĸG‰kÔ¨ĄĮˆq/Į‹ņwœG¨īũû÷Ķ7ÄÔË(@ķČ+$"’nđƒ‹­ŸÅi DE~‹aÆIšråäøņã2eĘ́(ˇ8-Ä>ÔĮđąōâ“$/J%Œ>ņà ¤0 ‘@ō‘b ú_-H>–!?Ā؅GˆCę_ôØyéM’€—¤E_ø!FZĄWHŅáGŊ‹7/žEđĸT"ė?Č# D˜ˆ†\gŽė˜ ôÔË$O‰#úÎđÌ;î"Íz^~xš$ā)…ŲG~ aŝ@ä@qf~x‰$ā)EĐGA`‰ė-ķžĻ^Wæ$Ę–løBĨ9¨lŦ¨Oecäį'H~–^åēÚ IDAT)}į‡ž`IĨãD= OüüIĀĪŌ+Ŗīü€Ë.I dŒ¨?eëOž AŠÜD-E’@ņБĸV)ßŊHđČ"ī0?č’1# œŒ õ%ōoĖĪoü,ŊúÎģx°H'âB=‰āŖ ČŖ$€2œađ?%’Ā1Ą~„ķī’@đdZęˆøĄŸIāW<¨†M…†K0Pöüā˙+t’ ĀĀ)ā„!“ ÕÁ¯‚7¨†NÜ Pđ4Hœt!tšõXŒEĀô‰Āԝ€ér7öƒ/fā$jƒŅNAI€Āž0$ęƒŅ§CL#?øĸ¨˜8A˜D&ʗŸwŲĘÆȨ'L›(L!ĶäjÔGã`I1Ä×Mš0L “äÄīŅî1‘ėFاõ›2qL‘ŖO?3Ot›$ā 1xŗ&L A&äįÍ/Į_Ŋ" øK^Ž÷6čIPI črsüCpƒ$ ×ĒĄyB " Y^Vé4ëų/$jCXub  UNa))Š ’@T°™ųR'˜ ‘@åcæ—æė¨IÎâíûւ6Ņ…‚&ß(>IĀGÂōJWũ0áüōË/’““#[ļl‘mÛļÉΝ;e׎]˛wī^9pā€:tHōķķåØącRĄBŠTŠ’TĢVMĒW¯.ĩkזøøxŠ[ˇŽ4hĐ@5j$ Rž|y¯ˆā„~øAžŽRHT„¨đÚē——'ĢV­’ˇß~[Ö¯_/7nÔÉ? 6”úõëëĎ =&üʕ++€>ŦĂQ€0ļoß.[ˇnU"ÁOŗfͤUĢVrÍ5×Hģví¤J•*QagåK^“ƒ•cc]Î @pį@ļâö”››+K—.•ŦŦ,Yž|š´oß^Ú´i#­[ˇ–æ͛ë$oUIŧ÷Ū{˛nŨ:YŗfŦ\šR:vė(IIIŌĩkWŠYŗĻUM…]Ûø‡ŨQ>čiHž÷;įÆDôęĢ¯ĘüųķeŲ˛e’œœ,]ēt‘ÄÄD9í´Ķė§Ÿ~Ōܙ™™˛dÉéÜšŗôėŲS:uęäHÜĀŨ‘ąĮ 8yđtbB‚˙Š§ž’™3gJ\\œôîŨ[nŋũvG'ū’$BX°`<÷ÜsrđāA8p Ü}÷ŨļųœĀ;xZĘ•„I€ēa vNL2yōd5õ 4Hmō^-đIĖ˜1CMFÇ—””Kģj'Ζv”•ų’€oDåũŽZ=A͙3GƎĢÎØ#FHãƍŊÂo=üāƒdâĉę¤9r¤ôë×/æž[oĖb@€$1zgVLT˜8ĶŌŌԜ2jÔ(O¯üËB;ƒôôt9kܸqJhŅ+pĻ]ž|HÁ—ąã#ŒeÂÂäģ˙„ ¤˙ūŽ÷ŨŽgĪž-ŠŠŠę/DRbÁ3’vøŦ™Ė”ģíŖŽtâ‚ųdđāÁR¯^=ĩ˙׊SĮö>:ŨĀž={ÔO°cĮ™>}zXæ­HqtzLlĪ˙ü/CĪŽ Ü ļŦúáPÅJ9č;8¸ą;(ÍW.~AĮ‹ãŗ’€Ŋø_{YŲŊ÷ŪĢ7}įΝ+M›65¯M›6Ißž}õæņ¤I“NwY¸j;$Û!fÅMhÕpÛmˇŠÃŋœŧčå‰ā~.˜ÁžhŅ" aBđŠ„ĖčIĀ 9ģ>ĘÂ[˖-5Ôâúā˜éË›!06lØ 7ŪxŖŧöÚkz š…؍IĀn„Y!"@dÎnŨēé–_Ā]‚Å‹kÄSĩÂIHNĸmx[ˆĐ‰]Â=Œ=Úp4Nū˜1cäÅ_ÔŨ"˛' 82ÛĐpÍp‚^yå•Ü”ĸØ R)œå!Շ؉IĀNtYw0˙ –?}e+|Čiķ °’€Ũŗ~Á1Pdö‚ã“%<ā8Gfŗ⎏†WŸ"á!@'>%¸6mÚ4 ¤fâ1Đ(aEœĄ!C†X|.Ú~đŊā#@žŒ]!BA4iŌDS=štĖ*ĀqĄ )-7oŪVˆ ĢÚe=f!@0KŪŽŽ'zõęeD(ģ€Eˆ $ĢÁ‰!"`$;Pe Ōpä‘%6p¤õ">[Ģ|ÛHĻHÚÁqÂū\ģ¸Äh BŠM!ú(nW#ˇr´ųœî3Ûķ$˙ČĘ7=ŊöÚkõBXō¸ >"ŽbWõÖ[ošŨļ0H¨ÛÃÁi … ĘęÕĢŨîJāÚoÛļ­tīŪ§…'YwDp˙ĀĩŽŗípdz9ŧ_AGĒJ8Úqį‚…X…IĀ*$YdddHNNÁ6ęĖl ’’’bc+ŦÚ$H&IÛÆą"/ĀŲgŸ-˗/į™vqÆŨ‹Ž;ĘîŨģ5 ˆ’@Ŧō}E`ęÔŠz+ÉQXėEÉxpJhčĐĄö6Äڍ@€$`„˜íäE]¤9‚é °køŖø“O>ąŋ1ļxHąũÄųõ &hdg@HîÔÔTŊÁBbA€$ z|W@˜čöíÛË]wŨEDB÷Ū|ķM†›vī 7C˛t[nnŽ:„<Č(ĄājQFãââÔA\ŗfM[fSAC€$4‰:<žYŗf lԌä0đ"z+>˜8ß8[ $ĀˆŌ$&&ęĻ[nšÅÜęË/ŋŦķ˛ŗŗ FC’@Ŧü~^^žš€~üņGš‚\Đ˜„ĒV­Ē hĒTŠâBØd AŠ.!++KsÃAÉâ×]w 'qRR’;`ĢžG€$ā{ē7äŽQŖ†<øāƒîuÂđ–}ôQŲŋ?sŽą Ÿ$ z†ŋ‹Ėa¸Ā bî)Â;īŧ#÷ß?3š'ߡLđŊŨbUŦXQũ•+Wv§lU>Ŧ~#GŽ0–õ!*HQÁƗČŦOŸ>ōᇠ—¸ė˛ËdŪŧy Üį˛üÚ¯ ™ü¨QŖô7‹ģ ų|zz:“Đģ+ßļNđ­čÜí8ō,Y˛DđÛí2pā@ųøãeíÚĩ–t‰qĐũ?˙ųšŧP×÷Î;īԉļsįΒ™™iI[VT‚ŧÉÉÉĖ/`˜ÖA0PčV šN:šOŋŨ.ņņņ‚÷V‘@=Ôá"äK€ķĩRĨJz1Ëk$°gĪÍ;Œß,D RH‘"ÆįÄ úöÛo]„0ÖĩjÕ:A"Ÿ~úŠÔĢWO/PaE˙ŨwßI“&M4ŧÅå—_ŽĪ"ę)n9ôŅGrꊧęeˇ)SĻČšįž+_|ąlŲ˛Ĩ NdKûōË/ĨB… jújÛļ­įHąƒÎ<ķL!ÄB"E€$)b|^€™$??_'Gˇ ÚĮä ˙‚ .ЉũĻ›nRûøäɓeØ°aŌĸE ũ÷ŸūY>ûė3í*ō`3bÄųá‡$--M.ŊôRŲ´i“,[ļLī? ?ÂÜšsŧ@*¸‡ÆUW]å98vė˜îRpBˆ…DŠI RÄøŧ"PŽ\99~ü¸ëh|ūųįJ˜Ŧ1IÄS­Z59ãŒ3dĮŽÚOLæ˛ļ`Á]éãrUģvíÔ§qúé§ë…7Lĸ^xĄŽæĨ¯ŋūúŸūŨĢ$ā%y¸Žė@Đ"†Œ/xe'€~%Đߋ“Vü?ü°4mÚT'~ėf7n,×_Ŋ :´ C—ßH€;~“ą @ˆ=ƒßõ‚O 8øâ‹/äüķĪWûūK/Ŋt‚„Î:ë,9īŧķÔv:[˙îģī | üãõ„Qųōå}ˇ OĀāŅ‚Ą“,ŅÄ*ŧr:(´ōŋâŠ+4’fČ„É|ßž}긆/ā›ožQSV˙p7lØPĘ°Ŗß|ķ͂@p*Ã$Ú 8ĖDĄâUsO™øZ7f’€uXU“Wî „N!’)L=8Ę §đ“O>Šņtn¸áũ3&JœúÁIŸ6mÚ艠ŪŊ{Ģsá°ŋ˙ū{Ųĩk—fčēä’KtWŅ­ˇŪĒī"`ŪW_}%ķįĪ— HˇnŨԟpß}÷š.wŪp]žîIĀ×âs¯ķ^ē1IĀßōs­÷Œäô'5ĖØAŪ‘…{BđŖÔ<ĐgFõ€~ëŖˆzG~ė IĀRķ@Ÿ™OĀBø­ Ė'āYøą'$?JÍ}ff1áˇ.0ŗ˜wdáĮžü(5ô™9†= æö†|Ũ ’€¯Åįnį[ļl)=ö˜\}õÕîvÄāÖqr)55UīH°h DƒßQîŊ÷^ŠQŖ†hcqmExëI“&šĶļę{HžĄ{ČĘĘŌÍHmČâHņ‰āxIIIît€­ú’€īEčŪōōō4,.dá7‹ŗ fPÕĒU5n:ŗh Dƒß)@ 11Qzõę%ˇÜr Qq܈ÆMæėėl‡[fsAB€$$iē0–YŗfiX¤cdqÜ@ ¤8Û0[ $@‰ĶųÁ €2u!MBÎáP\\œėŪŊģ ‚s­ŗĨ !@’4] "jÂAŲŋ—z`^ŗO?ũ´Ŧ\šR/^lŪā9bK X §™•!?Â1¯[ˇÎL\5Ōiâ~@§N\hM ’@¤éâX_`ƌjŖfąø` $Č#ĀBbE€$+‚|_˜:uĒŦ_ŋ^-ZDDlFāļÛn“V­ZiŌ"+$XäûŠb ÁAŧ|ųrMŪÎbÜ׹cGu#…& ˆ’@ŦōũŊ+''‡ĮEmÔ  MHH””Õ&!@0IÚŒĩnŨēz‰žëÁ†/ķvîÜi}åŦŅXHƊŪžĪ™3G.\(ĢW¯ļ§ƒkmÛļ­tīŪ]úõëg0 ēՐŦF”õ ’ĐÃlÁ{Ö)ÃėŲŗÕĖ†¤ō,DĀJHVĸÉēœÂųõ-[ļH:uˆJŒėŲŗG5j$¸SA,DĀJHVĸÉē HKK“;vĐIlN`WU¯^=7nœĩą "p"$j„m ķ™´­ WÜŽ†ãœ;~(N#@pqļ§ĀG€ä(ČC0ū|#âhĪž=5/’ņĐĀÃ Hn Î6 ĀņQ8AįΝkԅ2\ëÛˇ¯:Ëy ”„›ÜDŸm+¸P†ˆŖČQlBˆ „‚@Ž`DeXh~n#@p[l_@ˆ‰Áƒk ´É“'2ú(ĸ>\ëMŸ>Ą ¨ûž@€$ā 1°!}+å &*VũŠŠŠēĶa4Pęģ— xIė‹"€8C 8LGåëT•H ™žžŽpLūĖ@%÷$¯I„ũ)@žÜ.ÆÄ9bÄ_™O`Ūš8qĸîĐöOÅö*$¯J†ũ*@ ##CũmÚ´Q‡Ē—“Øcå÷š5kÔūŸ’’BIO#@đ´xØš0§ābüqqqšŦĻGž¸_€ķūČ˙ûėŗĪĘÁƒÕî `0gą¯#@đē„Øŋ“@Ž]\0[ļl™$''K—.]$11ŅQBĀğ-™™™˛dÉéÜšŗ^üBne"ā'H~’ûzšššjņvx ņˆ`2B¤Ō-ZHåʕ-CėđáÃōūûīËÚĩkÕԃ8?„‡8HQŗfMËÚbEDĀIHNĸÍļlC //OoÃ&g,RZ"6~6l(õëחøøx Άø<ÕĒUS’@¨„°Ā$čĐ!A\#šÛĩk—lßž]ļnŨĒq}đƒTpRÃ'›žUĒTąm<Ŧ˜8…IĀ)¤ŲŽŖĀ‡““Ŗ“÷ļmÛ4ŗ&vLđ˜č1áįįį+€*UĒ¤Ä‚Q€0ŲĢAƒJ$ ´ņ;*A6æ$§f;žCDЧO™7ož 0’€‰Rį˜x@ƒˇ!ˆŨøņ㉠0’€‘bį ą ¨UĢ–š„` ÚˇowT # )vģ\@;räˆTŦXQ/vq7@Ŋ0’€‰R7|Ė…w!(¸0\) >IĀ`á›:ôÂģ€Ü ˜Ē 7I€:`ØÔ¨QC…âœ?Ž‹âī¸gpôčQŲŋ?}FiK …N!Ē'ō 6Lʕ+'Į—)SĻhŧD-Åi!"` $S$Íq‹@ˆ0’€Š’į¸’ÁtHĻk€áã' Žž¨F#@0ZüŦĂQ€0ļoß.[ˇnU"ÁOŗfͤUĢVrÍ5×Hģví¤J•*Á1’€š˛÷ũČssseéŌĨ’••%˗/—öíÛK›6m¤uëÖŌŧysä­* ‰÷Ū{OÖ­['kÖŦ‘•+WJĮŽ%))Iēví*5kÖ´Ē)ÖCE€$ā(ÜlĖ ^}õU™?ž,[ļL’““ĨK—.’˜˜(§všՇUĮO?ũ$ŲŲŲ’™™)K–,‘Ν;KĪž=ĨS§NaŊĪ‡ˆ€W xEėGŠĀÆ˙ÔSOÉĖ™3%..Nz÷î-ˇß~ģŖI!,X°@ž{î99xđ  8Pîžûnú¨Ķž@€$ā 1™ŨɌŒ ™øā™8qĸ:ŠGŽ)ũúõķMßŲQŗ ˜%o_ŒgZZššSFåé•Y€bgžž.0g7N …x ’€—¤Ážčäģ˙„ ¤˙ūAdöėŲ’ššĒū đ $¯HÂđ~Ā|2xđ`ŠW¯žÚ˙ëÔŠ8DöėŲŖ~‚;vČôéĶ}eŪ œ08 HT×€íĢ~8TąRzÁNnėč+ē´Ŋ?>’€÷ečŪ{īŊzĶwîÜšŌ´iĶ@ĩđā6mÚ$}ûö՛Į“&M2fܨ÷ xO&FôĄnģí6u˜â◓Ŋŧ0îā‚ā‹-Ō,DĀiHN#Îö4>B- Ž.€™^pą ą‰qXˆ€“œD›mip6ÄÛAœÜ`ųÜ%@<"ÄAB;"ā$§f;ēčĐĄƒ^=z4)‚Ā˜1c4ŪŠ+¸# v8†IĀ1¨Ín>8A¯ŧōJîJQėŠÎrúĖūfœ=IĀ)¤ o§[ˇnjæ  lE€fŗŋ—ũ0Ÿ 1"@ˆ@ž^68ŠĖ^p|˛„‡įČlÆãŖááŧĸG€$=v|3 plÚ´iHÍÄc a@Tė#8>Š8CC† ᅲhAä{a!@ &> Ņ¤IMõhŌE°h°*î\(CJË͛73ĄU ˛ž“ P)lC e˖ŌĢW/#BAØ"BL Y͆ ėj‚õŽIĀp°køˆŠ@i/žøĸ]MS/2¨!°Ŗ#rGJpn3ƒũšvq 6ˆŅ@–"ĸâv5r+3Ķčŋ=’@đeėø¯ŊöZÍ˙¤|ŽƒX¤ADÅŽę­ˇŪrģ+l?`&Pˇ‡ƒĶ@ .”ÕĢWģŨ•Āĩßļm[éŪŊ;O N˛îˆ$ā.ūkgÛáČôr2xŋ‚ŽT•p´ãÎ ° ’€UH˛ÉČȐœœ:ƒmÔ˜Ų$%%ÅÆVXĩIL’ļcE^€ŗĪ>[ 5nÜØƖĖŽw/€o÷îŨš‡€…ĊI Vųž"0uęTŊŒä(,ö"€d<8%4tčP{bíF @0BĖöōĸ‹.ŌÁô؏5|ČQüÉ'ŸØß[<$Ā‹Øūâüú„ 42‹3 $wjjĒŪĮ`!ą @ˆ=žĢ L42…Ũu×]DÄ!poāÍ7ßd¸i‡đr3$ KׁąåææĒCøāÁƒŒęŪĄ&e4..NÄ5kÖt°e64HA“¨Ãã™5k–ĀFÍA/ĸˇ˛áƒ0`€ķŗÅĀ @Œ(ŨHbbĸ^`ēå–[Üé€Á­žüōËz1/;;Û`8ôX Ċ ÁīįååŠ čĮ¤)Č=€I¨jÕĒ‚ßUĒTqĄl2‚ E—Ɛ••Ĩ9ƒá dqëŽģN“8))ɝ°Uß#@đŊŨrרQC|đA÷:axˏ>ú¨ėßŋŸšˆ ׃X†Oˆ=ÃßEæ0Üā1÷áwŪ‘ûīŋŸ™Į܁ī[& ø^„î ą‚*VŦ¨ū€Ę•+ģĶ ļ*‡VŋĀ‘#GKˆú$¨`ãKdÖ§Oųđà †Ë\vŲe2oŪ<îsY~mž$āWÉšÜīįŸ^VŦXÁû.ËÍãž@‡äŽ;îđ@oØŋ!@đ›Ä<Ō߇zHN9å=z´Gzdn7ƌ#G•GyÄ\8ō¨ D Ų/bՉã‰\}ē¯ؕá˜.~ŗH DŠŸWL~Ô¨Qú›Å]|>==IčŨƒo[' øVtîvų–,Y"øív8p |üņĮ˛víZKē‚Ä8HčūŸ˙üGM^8 S œ¯ß~û­Ô¯__eze„ŧÉÉÉĖ/`‰ôÍĢ„$`žĖ-q:u4Ÿ0~ģ]âãã î­"=z¨Ã;D?ū¸æôũũī/M›6•7ŪxC2+đ6mÚ¸=|ŲŗgæÆo")$HãķŠbaUŒßn„ąŽUĢÖ Íúé§R¯^=]ŠcE˙ŨwßI“&M4ŧÅå—_ŽĪ"ę)n9ôŅGrꊧęeˇ)SĻČšįž+_|ąlŲ˛Ĩ NdK[ŧxąūHæüķĪ—Į{Lē ŽqãÆš5ü‚v;čĖ3ĪÔB,D RH‘"Æį˜IōķķĨB… Ž!‚ö1ycÂŋā‚ tRžéĻ›Ô>>yōd6l˜´hŅB˙ũįŸ–Ī>ûLûŠüØÁŒ1B~øáIKK“K/ŊT6mÚ$˖-Ķûȏ0wî\A¯† ž0ÆÜO; IDATáÇËOĶO?]/ŧaŊđ u<0/}ũõ×æ Âƒ\ētŠ Ņ;b&Á_Ŋzu×1đ’<<;$ˆāâÃ!ŧ°@_Š’@čīÅI +ū‡~Xíú˜ø1†ÆËõ×_/C‡-ČĐU L›6Mîšįø_ũu53yĄp'ā)øˇ$˙ĘÎ՞{Á'P |ņÅjˇ‡}˙Ĩ—^:ŖŗÎ:KÎ;ī<ĩ‡ÎÖŋûîģßÂ˙øG=aTž|ųbw01ábĖF0Áí•BŸ€W$áĪ~ü)7×{í•ĶAĄ•˙W\Ą‘4Cæ LæûöíSĮ5|ß|ķš‚°ú‡Cv~8•aGŋųæ›åÕW_8•a íœ fĸ… jh8–×ŦY#ŋûŨī\Įŋpx:ČSâđ]gHž™7:ė•{ĄBˆd S&k8…Ÿ|ōI§sà 7čŸ1Qâ„Ī—_~ŠĮ:q"¨wīŪęÜF8ėīŋ˙^víÚĨē.šäŨĀD„´™ œ„jÖŦ™î0B¤“Û…÷ܖ€ŋÛ' ø[~ŽõŪK7†‡ "Ī<ķŒÔŽ][233ÕQŒS;ČÁ‹P×pã$ĪUW]ĨxáYœîé+}øœ“<Ę /ŧ ¨˙‡# ũúõ+gøāpģđÆ°Ûđwû$ËĪĩŪ3vkПÔ0cyG~ė IĀRķ@ŸEÔBø­ Œ"ęYøą'$?JÍ}f>áˇ.0Ÿ€wdáĮžü(5ô™™Å< „ßēĀĖbŪ‘…{BđŖÔ<Đgæö€D˜cØbđu/HžŸģoŲ˛ĨSģúęĢŨíˆÁ­#‚ŲᎠˆ’@4¨ņEāŪ{īÕ:ĐÆâ8Úē˙~™4i’;`ĢžG€$ā{ē7€ŦŦ, ŅŒÔ†,î €ŸŽ—””äNØĒī ø^„î ‰U–˛ÜĖ+āîļŒ˜AUĢVÕXH¸éĖBĸA€$ j|§ÄÄDéÕĢ—†W`q܈~îšį$;;ÛŲ†ŲZ  JœÎfÖŦYšŠ éYœE÷iĀ€Î6ĖÖ…I Pât~0ā†L]ČÄE“søÃ'ģwī.ȃā\ël)H‚$M—ÆŌ­[7ƒ˛˙ū.õĀŧfŸ~úiYšrĨæ?f!ą @ˆ=žĢ ?Â1¯[ˇŽˆ8„Ōiâ~@§Nj‘Í’@P%ëđ¸_`ƌjŖfąø` ¤9ŽYˆ@ŦbEī+S§N•õë×ËĸE‹ˆˆÍ Ņ}ĢV­4é ˆ’@Ŧō}Eą„ā ^ž|š&ogąîëØąŖ:„‘B“…ĊI Vų~ČŪ•““Ããĸ6ꎅ&$$HJJŠ­°j“ ˜$mÆZˇn]ŊĀD߀õ`À‹y;wî´žrÖh,$cEoĪĀįĖ™# .”ÕĢWÛĶ€ÁĩļmÛVēwī^bÎcƒĄáĐc@€$x|ĩx„f Ū°NCfĪž­f6$•g!V"@°MÖĨā”ίoŲ˛EęÔŠCTbD`Īž=Ō¨Q#ŊSA,DĀJHVĸÉē HKK“;vĐIlN`WU¯^=7nœĩą "p"$j„m ķ™´­ WÜŽ†ãœ;~(N#@pqļ§ĀG€ä(ČC0ū|#âhĪž=5/’ņĐĀÃ Hn Î6 ĀņQ8AįΝkԅ2\ëÛˇ¯:Ëy ”„›ÜDŸm+¸P†ˆŖČQlBˆ „‚@Ž`Díׯĩ€¸ŠIĀUøŲx„˜ûŦ,īŋ˙žŦ]ģVM=ˆķƒ@xˆƒ„5kÖ´Ŧ-VDœD€$ā$ÚlË6ōōōôæ1lōpÆ"Ĩ%bá§aÆRŋ~}‰×ālˆĪS­Z5% „j@ Lō‡Ä5Bģ]ģvÉöíÛeëÖ­×?Hõ'5|¸é[ĨJÛÆʉ€SœBší8Š|999:yoÛļM3›abĮ‰~~~žˆ RĨJJ ™Ŋ4h D’@ŋŖdcN!@p iļã9@}úô‘yķæ)° ˜(uŽYxā4x‚؍?ž¨# )vģ€ZĩjŠIĻ }ûöq7@ĩ0’€‘bį ą Ā´#GŽHŊõbwÔ  ˜(uÃĮ\x‚‚ģÕÂāá“ žŠC/ŧ aĀŨ€ŠÚĀq“¨F!€]@5ôX(Îųã¸(ūŽ{G•ũû÷Ķ7`”Fp°$ę€Qā4ĸz"_Á°aä\šrrüøq™2eŠÆûGÔRœb!Ļ @0EŌgą„H€đS ˜*yŽ[ PLG€$`ē>~’€á Āá I€J`4$ŖÅĪÁc7|^1"`($CĪa @ 2IĀhņsđÜ PLG€$`ēpüÜ PŒF€$`´ø9xî¨Ļ#@0]8~î¨F#@0Zü|¸¤¤¤xĩĢėPHTĪ"0gÎ;vŦ:cGŒ!7öl_‹vėƒ>‰'Ē“zäȑŌ¯_?ßô5 ’€YōöÅh1qĻĨĨŠ9eÔ¨Qž^ų—(vééésÖ¸qã”ĐXˆ€— xIė‹Nū°ûO˜0Aú÷īDfĪž-ŠŠŠę/°¯ @đŠ$ īĖ'ƒ–zõęŠũŋN:CdĪž=ę'Øąc‡LŸ>ŨWæ­Ā ƒ*@€$@epØūąę‡C+å ėtāāÆ‚ KÛûã# x_FîáŊ÷ŪĢ7}įΝ+M›6 ôX nĶĻMŌˇo_Ŋy^|*zHŅcĮ7Ã@ÁĻM›ĻÔL<DÅ>‚ãŖˆ34dČ^(‹Dž$°`âCŅ €PMš4ŅT&]‹ĢâŪÁ…2¤´Üŧy3CLX*ë9 ’•Â6Zļl)Ŋzõ2"„] "Ä’ÕlØ°ÁŽ&X¯á Wģ†h ”öâ‹/ÚՄ1õ"ƒë1ú¨1"wt $Gá6Ŗ1Ø˙‘kˇ`ƒ Ôi)"ú(nW#ˇ2ķ8~đÛ# _ƎđÚk¯ÕüŋAĘā8ˆEDÄQėĒŪzë-ˇģÂö†I `u{88 ´páBYŊzĩÛ] \ûmÛļ•îŨģķ´Pā$ëî€Hîâ¸ÖqļŽL/'ƒ÷+čHU G;î\°Ģ X…$둌Œ ÉÉÉĄ3ØF]€™-!!ARRRll…U›„IĀ$iÛ8Vä8ûėŗ5ZãƍmlÉėĒq÷øvīŪ­yXˆ@ŦbEī+S§NÕ[ÁHŽÂb/HƃSBC‡ĩˇ!Ön$#Äl˙ /ēč"ÍL_€ũXÃ7€ş|ō‰ũą…Ā#@ŧˆí ίO˜0AC ŗ8ƒBr§ĻĻę} " $XĐãģŠÂD#SØ]wŨEDB÷Ū|ķM†›vī 7C˛t[nnŽ:„<Č(ĄājQFãââÔA\ŗfM[fSAC€$4‰:<žYŗf lԌä0đ"z+>˜8ß8[ $ĀˆŌ$&&ęĻ[nšÅÜęË/ŋŦķ˛ŗŗ FC’@Ŧü~^^žš€~üņGš‚\Đ˜„ĒV­*ø]ĨJzĀ&ƒ€I Rti YYYš3Jw¸îēë9‰“’’Üé[õ=$ߋĐŊ wp5äÁt¯†ˇüčŖĘūũû™‹Øp=ˆeø$XĐ3ü]dÃũ^sOŪyįš˙ūû™yĖ=øže’€īEčÎ+¨bŊę¨\š˛;`Ģrøđaõ 9r„ą„¨Q!@ˆ 6ž„@f}úô‘?ü`¸ŒĀe—]&ķæÍcā>—åā×æI~•œËũ~ūųįeŊŧā˛Đ<î tčĐAî¸ãô†]đ$ŋIĖ#ũ}衇ä”SN‘ŅŖG{¤GævcĖ˜1rôčQyä‘GĖ#’@ÔЙũ"V8žČÕ§ûz€]Žéâ7 ˆ’@¤ˆņyEÉäGĨŋ.­[ˇÖˆĨŸ~úŠ\xá…a5Õō“O>Š+æüãŌ¨QŖ°ŪķÃCH>ŸžžÎ$ô~–ûHđ PüĐ%äX˛d‰āˇĶ4˙ũīË=÷ÜVđ´oŋũVj׎­×`2šõÖ[ÃzĪéq×H.99YƎ[bwWĪ0ŋ€$æŋ>ü'3Oô¸N:šOŋŊ^p‚ yy˙ô§?ÉęÕĢŊŪŨ‚ū}ūųįrÁHZZZŠ$°gĪ~ŗH DŠŸW3+lü.ŽĖœ9SĻL™";wîÔ8Ė=öØcōŅGIķæÍĨiĶĻ˛qãF}õØącRĢV-ƒ‰ qppjéŌĨ˛oß>9īŧķä/ų‹ 8PŸ/l:ëŦŗäŒ3ÎĐú0Y>\oĐbeŒ> Â)r.˙üį?Į*322ä駟–¯žúJw ŨģwםBĨJ•Ę”2nKOž¯ādP~~žT¨Pá$D`ŖÆĒ“!L/˜ė1Ųaâœ8qĸ\|ņÅ˛e˝|ãããeíÚĩrÕUWI—.]äoû›ž8‚1q.ŋürY°`üũīLŒ7Ūxã $Pŋ~}´SŋaÆ:ųO›6Mļoߎ<ž‡ųuÂtuß}÷iœü2s5nÜXŗ‘8A01=ņÄeJ„ƒ ÕĢW×ߘ€SRR”0Ž˙ųŸ˙QŌ+­Ōp2dˆĀķ˙ČÖŖGÅ˛¸đw°H DŠŸWʕ+§+Ũ⠒Î6Lc =üđÃJ6lĐũ9įœ#?ū¸N˜>7xđ`yā4üB#cõ{ķÍ7+`bîØąŖ&ŦųėŗĪÔ4‚]EáĀųįŸ/§žzĒvæ“?üá2ū| oŨģwoDaļ™´mÛVV­Z%¸íŒÉúįŸ–]ģvé.“8Vō¸N‚œ‘#Gʸqã4–?å†n7ŪxC233uâ.Ģšsį–ŠĶœ9s¤˙ūešƒĘ’U–”†I€úĨížøâ iŌ¤‰|˙ũ÷j.jÕĒ•ÜtĶM:Ąá=˜5°ĀÉĸ•+WĘĨ—^ĒŽŪŊ{÷j Vį LЈM„I„€R €8 ˙}›6m vEI`ĮŽ‚Ä*0)mŪŧYoAŖOĨ• LŸ>]ūüį?ëŖ > Ė` ˛Ú6Ĩá. p'• ķĨß PĸB ,ŸL"Hx“vXu#12‘Ą`BĀG3MhÕę LH .Ô “8VīČ^ģ}q$PØf2/uîÜYWåEIĻ"vØa„ &d„Ãø×ŋūĨ&ĢpHĻ'˜­P°Ģųë_˙ĒD€d;á´QNá’}QŠ0_" PbA ´ĶAß|ķÚüa~ŲčģīžĶsųXą†œ›XíÃÆ ĶĪ+¯ŧĸ—BÜ­[ˇĒIæŠ+ŽĐ.ž˙ūûŌĸE }~ŅĸE1“@asĐ×_­NÕ~øAs%ŖhģŦ xĄ@Īž=•ėPŽŧōJYŋ~Ŋú.@r!sPIm Ōp ‘ü ãĮ/Q\<‹&ķ]î¨Q!PÚ=8uáˆíׯŸšr0 âīø3læ(p*ÂĀ*ļų“Ne\Ã;įž{Ž:Žąƒ€ūƒXwh“*|8ŠrĄM\€ƒCļŦ"8†aæÂ8ĐGŒĻ-8jËjŖ,œp: ŽnøB ¤'¤Š+ŧ'P–´ø˙Ĩ!@ ~D…@i7†qJėK/Ŋ¤+ØëqRaL’Ą§*Nđ=‘ƒėëkÖŦQŋœÉ8'3ˆÂ ĀnÎh´’BwŪy§:a‹;ņT¤ āH f¯ēuë*„ō+”ÕFY8ÁĖg3LT ˜ĩŠ+ŧ1• ķ%šƒ¨ą `Eė L–8_É įöũTB$0cƌ‚û nõŸąƒÜB>ír' 9:>ŠXĸˆbâ„N[8P_{í5Įû_Zƒˆ-ßCIw ļmÛĻGDŊ@Œ"ę)õņ]gHž™7:K>8€ß}÷]5ëŧđ ž=Q’9Č $Ā|Ūø&üÚ ’€_%įrŋ™YĖejž™Åŧ# ?ö„$āGŠy ĪĖ1ė!ˆ0Į°7Äāë^|->w;߲eKsõÕWģÛƒ[Įe:Ü#Ā…<" $hPã;ŠÂÕ¨QCƒ˛¸ƒîN ję¤I“Üé[õ=$ߋĐŊdeei8œôaqÜŋ@Ø DFe!Ņ @ˆ5žŖäååi€¸üąÄŧ„Ę>p™ŦjÕĒ9Xˆ@4ĸAī €sūˆđ‰¤1,Î"€øKˆ[„Û,D ZHŅ"Į÷„I€s>YœE÷pëá7Xˆ@´ĸEŽī)ššš}3œD,„Ė:`BLĻŨģwkV5"-$h‘ã{ ”ˆĻÉâ |‡| ‹/vĻAļXH­sCü|DäD(fg@îÜ@3" $XĐãģ ŋâč„Â(û€ųG€…ĊI Vųž"€”ŠČĒUZôMBe ČŋŒŧÍC‡ĩĻBÖb4$ŖÅoŨāKâå˗ B-ŗ؃÷!u%Âå˗ˇ§Öj$ŖÄmī`3224A ‹Ú‡3Ž…&$$HJJŠ}°fŖ %nû‹‹¸ĀD߀õXÀ‹y;wî´žrÖh,$cEoĪĀįĖ™# .”ÕĢWÛĶ€ÁĩļmÛVēwī.ũúõ3ŨjHV#ĘúIčaļāŊë”aöėŲjfCRy"`%$+Ņd]ŠN áüú–-[|™:ŌkbÜŗg4jÔHp§‚Xˆ€•ŦD“u ––&;vė “؝ĀŽĒ^ŊzšØž…XIĀjDY_Č<GæĀ‰J”Ėœ9SíĖ%€|­LHeBÄĸEgڛ4i"7n”ĻM›F[ąīmÚ´Iš5k&›7oæŨ cĩĀū“ėĮØčpZhÚ´ię'@–đ@”PØ˙‡ ÂĶ@áAƧĸD€$%p|-|‹gۗ.]ūK†?ŲĩkWÁ æ6\>IĀŲ„ÂM׎][sŗ”ŽrīŨģ—aĸŠ(Ž @pf6rėØ1i׎ ōØąc H Œ9RCr¯ZĩJ*T¨@œˆ€íl‡˜ „8pā€tčĐA =šĀA`Ė˜1€oŊRŊzuâCA€$āĖl$„ĖIIIŌž}{î ŠvČ–••Ĩf3"ā$§f;`GĮ'nÁŌG nWÃqÎ?§ 88ÛSā#@rä!˜?ž‘ĮGq ´gĪžšÉxčāĮá$7Pg›āø(œ sįÎ5ęB.‚õíÛWå<ĘÂMHnĸĪļ\(CÄQä(6!ÄBA G0"ƒ2,4?ˇ ¸-ļ¯ ÄÄāÁƒ5PÚäɓ}Ņ@‡ŽõĻOŸÎPÔ}O @đ„؉ˆ>Š•ō„ •ĢūÔÔTŨé0(õŨKŧ$ öE@œ!ĻŖFōuĒJ¤„LOOW8&æ ’{ ’€×$Âū _ncâ1b„¯Ė'0oMœ8Q whû§b{’€W%Ã~ ‘‘Ą~‚6mÚ¨CÕËIėąō‡ƒ{͚5j˙OIIĄ$‰€§ xZ<ė\˜Spą ū‚¸¸8MVĶŖGOÜ/Āyä˙}öŲgåāÁƒj÷Į0˜ŗXˆ€× x]BėßI ×..˜-[ļL’““ĨK—.’˜˜č(!`âĪÎΖĖĖLY˛d‰tîÜY/~!ˇ2 đ$?I‹}=ÜÜ\ ĩ€x;ŧ†xD0!Ri‹-¤råʖ!vøđay˙ũ÷eíÚĩjęAœÂC$„Ā¨YŗĻemą""ā$$'Ņf[ļ!——§7a“‡3)-›? 6”úõëK||ŧgC|žjÕĒ)I TBX`’?tč Ž‚ÜíÚĩKļoß.[ˇnÕ¸>øAĒG8Šá“ĀMß*UĒØ6VLœB€$āŌKķéaIDATlĮQāCČÉÉŅÉ{ÛļmšŲ ;&xLô˜đķķķ•@•*URbA€(@ČėÕ A%’„„Úø• s ’€SHŗ"@ˆ€ xP(ė DĀ)ūšŪÕ1õ}5IENDŽB`‚avocado-113.0/docs/source/blueprints/images/requirements-resolver.png000066400000000000000000001121261513561306000261110ustar00rootroot00000000000000‰PNG  IHDRę!ŗš€IDATxÚėŨˆ]Ũŋôõũvė!ļS;mÉį€Ą 4` A†y(QĸyÄ ‘4ŌPĸ>”TcI1…H#ŒhÄHŖÄ6Ö¨Š¤šBJŖšļą7õĻmîmlŸÖ؛^ĶÛ´ĻmnoœĪsÖúΚ5{Ÿ9393sÎĖë‹äœŲg˙XûĮĖ~ŸĩÖN ˜AGWĘšĒœ])Ë+åđJ™S=°3ŽŽ”/=åãJšŖŠļĪrļ˜b9>;°īīNAŨîÔö°MįĖ¸ą‹n˜}WũüHõū“4ėjØ5Z|ūVž.Æ‹1ÃN47¯}­žĘ:œ¨Ūk[ÔŨČ¯ŋÍ˸—V[ĖįãĮÕĪNô,#Ærģ˜ˇåižą=ˇéF~/æsļcōôåŗwrũ¤ŽmœËõXĻm[õÄ4ŅÂęcūcí@ŗ-{Ö%õ,÷pŗīnįų<Í˙?ÚņŲCÕž+ëچIįsũ,äc L{%¯ķn™ļ î鈟_ĖĶ\qŲŲņãc°ûūî>Ú^ļÉržą‹@)ZûÄũ÷ōĪÎŦ”Ī+åû4 oâF4‚Ĩ7ixGķûņŲsnåy=Ëķ.úÆQ+ëpnčo´<{™×į]ū7Ö!‚ŖWųõÃŧü—ųķ{ļķ]ŪÆįųŊû+åužO Đâũ: äzø”?{3×C,÷tĮ6>Ž–ķ$ŋÛ0Ÿ§ë ęžæeÄ:]¯ęđÖûą­Û#Õžē•KŠˇ:Ä<]í÷ō°†ųŗmpú.×ŅÛüúEÚ8œÚŠ`b‚ēåÔč,ĻaøYBŨØW]­î"Ŋ’§‹ō]>ę{ą9žRŗëđģmõUÂØ{åŌ+Ķjļgņ™hEø ­Ä%¤ž˜VCč+ÕųPĖå}ų(Oķ8ÕQ_%”>UMįÚņfûĘĩāA~]|[mËãüŗqįĶÕ2ËļmÔĸŽÔíŅŧŧļnëõÎMĢáû“\‡ Õ2ęí­—?ßÔũũŽz-_fÉסōEÆõןkš0!%^WķyŊĘīßiÎÛ'Õ% ÔLĐrĪũ…ü~×ā÷wōÜBuŗvģcēi˛AŨûæsķų†÷IĮ˛ĻĩŨ Ë2.7ĶŨLŨAĶÛ*tYęŠSÍē—müvŒmė ęĻÕ 2åzŪ(@č ęÎ5Ÿ­įûmZ~Ö.埝Ž‚†Ž›õķiwŸ6;mAŨëŧ.ušP %T)JČ]wYŽŸ—hžŲÎú¸ZJĢÁĖ¤‚ē2Í\>îæĢã¤síģžķössœ” įUĩísy?VĶ]NëƒĖúX^nļą­ˇr.Ūę8>Õ{í õļœąäi5įÔŊ-Öí\uŪļûërŗŋōöžlŽŸsÍu­k{˲ë/ æĒõ>Õ\ŖŪWŸ_Ču܆z)_[ãũis`OXî :n§Õ3w›RēW¨>ĄcŪåæˇŊŠ\cē‚ēļĩRš|ĶąŽw›yômgߘNõōNĻĩ­hęR˜k[ØÆŽ‡I”–JĨeKĖ÷øûą]îá|c˙%ō>ZęØöŽÖpĮĶÚ ķnO=õmīN™•§žžËAĮbĮņ{ŗc^g›åM.­‹iōAŨ‘Žc2ŽĄŽnĒī›ķ˛´†ĢNŨ-Eī6ׇ×=Ûx Y÷˛wÆØÆž îuZōͧģž^KŖ[–SˇíøœOķy:×QˇrŨ×ĮٞcāXĪöÆ|úžĖXLk[E–kTW7×įųŽ×3öÕ3ŋB&kš'č(7–÷RwVƈZ”œKÛԝHũZ)į7ØÎq‚ēŌĸčŲËųÚ n./īAžTáĮfēž†厤ՎlĨĩĶ… ļ=ĨÕÖCW›ãa0ÆrwŌ´uĪs rxr;×ųí´>Œu\•ĀöJ5īûË<–&ÔĩJWÜŽãū]3Ž1ø6ēÆԟ}×ŗœi5ÚĖ6vãWĒķęMŪ7ߤ[­>ėŠŸRG[ŠÛštmķûje;y> šēēŪ3}Ė˙uŗŸēžt)-ŦË%Č?īW(ĀdõŨD—îj=Vē˜vSt=ԝL[ ęFuģw;Į ęĘMéå1–ķĩA]-‚ƒ#qSŧQ`]ØÎĻÕņĐĸõĐĨ´~|ŋĸt#ŧØ ƒM.wģÍÂuĨ{õũžē{šúāĶ=įC1HÛԕqŅîŽ(iÄzö{õōJ÷ō7#–qc ÛØw~Įy-ōžOkCģQAØĶÔÔŊŨbŨ~Ų no4×ãÁ˜įÃ`Œē*ëũvƒũTŽõø–ˇĢëÔwsöMęo‰ņ(ß\F׊šü˙v ö2U}sZēoĩãˇ]K[ ęRžš~×qÃë­p.m°ãuķy^]ƒÍGK“Õ6}MPw>­ˆ@ũŲQAaģÜĮŠģa ")!kWÂ2Āũáæsƒ –ģĶf!¨ĢĪ…ēĩR†ŽĮØŗ1ÎßAęfæŌփē×iĩÕÕ8u°ÕuŸŌjWĪQúļąkųŖZqŒ—0õŪˆéĘp =ûg+u[žfŊ‘2fŨąŽŸŽÎŋv{GœãeŊŸo°ŸŠ{ųúÛ˙~ƒē`‹únÎæŌj—ÉËiõI°×:nüJXs?ß .åŸinNĨՖ+'ōëīōMßVƒēū=Ë7ąĒE Ŋwų&tiƒí'¨Ģo”åûĪä×īĶjP¸™ îQ'ĘSƒü:B‘ãyD‹Æ'y?”'Î.åuģŲą–›õpá`^ŋSšN^4ëPBĨ\Ž§õ-ĀuõÔ×Ŗy~HĢOW.­”žwL_΋2F]yĒéÁfē›Íö÷ĩ2=™ļԕsš ‰ķö<qŪŽ:÷Úå=éŲÆCųũ2vÚ m=¨;œëõJĪgG…ĨĢō…žíÛJŨŪëŠÛōÔír–nūmËå…<Ũƒf{떁¯S˙—õ—1ËœKå*÷8é×'Āäē9[JĢK¨ËŨŽ›žīōÍt™&nŸuܜ^lĻ{WŨ„n%¨ įĶęƒęnlĮĮØÎqƒē”oî?5Ëy‘Öŋ™ îLZ;ŽÜhŸNĢÁe)īĶjČ:¨x:bšsiõéŧõŧž7Aȁ*D(åsžŸīÚzÔ­=fŸŽøųõf_‡ŌĨņ^ŽÛØ>—”}p${ŅúęXžîbĩëí˙>O{&ĪķL>Ī>Ļ­…I‡ķgcßäu:RßgFœˇŖÎŊvy'ōöŧÎ˙ŸËįđëüūņæø'¨+´‰z.Ąįëŧ=gr=.VûfTëÆXŸWÕgįķļŊM[ęŽäųÅ´';ęļî–^û yŊŖå÷N4įã­jŋœŠŽÍ#y§ķū|—6ū2#5ĮÖįü/Û`ž úĖ7qËiíSC[ yšŌŦīæt1Owĸ "Úu¤ĩOČ<¸Á˛įķr—Sw÷°ží\ČīĪuls×ō¤Õ§Ũéųų ­đèåo–UoËņÔũ´ÍAķ™žĀlŠÚw‡GÔ_=ŨBĪ>ëĒ§žíŨ)ŗÔEŊÉĶŽöe„H]p;^Zi)ZĻ)­ŲÚí?ŅLWZm–ąđŠqä2Ī7Í:Æō/tÔÁVƒēpĒY÷æwÕãuQ‡%ø~Ÿß‹ķāuZNßcĮyōŧųė´õ1ęĘ~íĒÛķįāŖŽéÎ4×­2îŪĮęũŽ/3Ú/–Į8—ĘųvͯN€Ų3ęæ”É*-‡ŽíŗížĻ nF‡ÉĄËī/§ÕûJP| ­´įFLW–ĩÔ,7^×­+K;Ęą´˛čŠƒÅŽutLߡŧšj9Ŗ‚ę…1— Ī§­×Ŗit8=ʑüšĻœpŨŽ žm°JĐdÄ1q¤įsƒ4úK›ōô×C €™#¨Û~qcaU´ČŲOaœĻ n7œÛįÛĪÎ)EyĒ*fĶe7uÛŽ pŨúÎīÃíÔ ęØ^Ņú.žt)]~ŋQ%@—ũÔEhō4 OØ>Ņ…÷c.—Uė›ĖRöŠ#v-ėˆũÔĀĔn[ĨĖēŦüqŌŨvŠ &d/uņ4ŧĪy;u°3.įsîCŽĄciŨLïŨB` {)¨ģ[m‡ îë-§a EeŠš†t•ĐŠ*mŗAŨÜJ9ž†-dŽmrYƒ•r*  Û°-ãu +åÄJ9ŊRNŽ”ŖyģXëKŽĶŊ,‚Č{võWš”Vø×ųÜģ۔ŨčÁ`<…7Zô}—¯;ûŅršŪĀũp^ŋų¯ããų÷{÷X¤I˙Ŋq(_ã.įŋÍr].åŸĪįׇĀlÔ=­ŪéÎŦ”wÍôo:n€Í4‹+åAķŪ§•rĨc}ŽVĶŧŨ`}ßö,īKĪ6-åõøÜ1͇üGŽĀnU,į÷ø6>í9Îß´QOČ|Øs-xĩA@„67Ļh˙LKāáÁÍŧękĖ—= ŧMZq­#ųXønogųō`RâīšÕ5îIū[-ū*OķM~}Öa0Æ ęžĻū ėcZũæ6ĨõÁŲËŸŊŅsĶ?é nnƒõč[ö6AŨכĻ .B ø2!ÂøkiĩģíR>įŸÜÆu˜ļĐfZ‚ērœ šĐ"ÖmqÆĪøŊqŲĨāĢËĮÂ^n™x*M>0ģ˜įųmu­;ĶœWWō4GfŗaÜ Ž´ž‹?0—ĶÚnĻíˇāƒ´>+ßōÆō^7?[ÛLPˇ˜×įMõŗwiĩÛ]8ÖļĒ‰›‚…ŧŽ7ĶÚą´îđ>8œ×3Ö%ēažlnúŽVīĮ`ũ×{nlc>wōtķî'š›ôōúpĪ:œ¨Ūk[ÔŨČ¯cŧHÃ։%9ožäú}Öˇ´,ˈ‰č.ų,Īįzūüb^ÆË|ė]čØÆÅĀ<ËĶŨ븹+Û8Ÿ×ŠL{ģ îæãäcū˙éŽm‰cõQŪîZ[ž¨öãŧmĪķŧęŽŪąŊˇzæq-MļĩÅN0ĶÔ=jnZ[įĶöW‚ēŅĮÉĀ¯]öŠíĖâwā§ ~?E ãĒ`vŒÔÅ‚m€UcwĒ÷i}HW˙YŨ…ö^ĮÍÜf‚ēbÔuõ|#¸iƒŽŽN§éÂrĩÎsŨ”:‰pķsŽënį?¸ŋonxåĪžOĢAPŧ~Ņė×RË=ëpnÄ ~Ô÷Ģŧ>ßįcb1—×ųũXī›Jëŋ=ÛY‚ÔgųŊyåŠĄßwŦĪĄüšOųxģ^-÷LĮ6>Éëķ{œß‹ųÎWĮKWPWöÝ|Ü<ɟŊ=æšô:—kyŸķv–åŪČĶĩãĄüū•ģ†LKPw¨ÚīŨØÖ_,ôĩ†ēœÖ\§ōąú<'7ĒkFųÂācu|×Á÷‘|<<˟Ŋ™Ö‡Vįķ<ķņö<­†îå<įņ 7NhWPˇ”ĪŸ§Õ9×5¯šŧ_åéõėįXŋûyšgų˙'›ízYë7Ē÷ÛuąĖ yYĪĒeÖŋCĢúũĻZŋiŧ' ŸOŨáe׃|=}–Kģ_ģŽĄËš Ēũø$u¯0Ÿß’įŗÚžĶÛpžĖįy/į}ô$×ß|užŸ×åqž†w…@õņü ×Ų‘´úeLŊŸÎtWíA×rŠ÷Ņĩü÷BYΊęøŊ–ßgiũ—0á@^ÆŲŧ^÷ō~¸“6—ˇ­Ô{×5`÷GŲ÷7:ūv8X?e9uÔuūĻ:ĪTuq)×õ¨sŦīX}_ũ;[ÕmũE×ģŽëbŲåZÖ~ĄĀ u]7ŋĒŸßԝéøėõ´ļĨŪvu§Ķú§P>ÍË[Nģ;˜ųrĩ=‹ÕöKUØV˙}8‡uĢ÷^ä÷ę ņxZoRAŨ—ęæjĄĒ÷ĪÍ<įō Kũ´Īå*ČĒˇįyĮ˛–r ÷¨9?67ŅsųũÕú”m|Ö,įFĮņØv}=žÖŒåX˙¸ÁqR–ûĒYny˙Ruƒô%ßĻæFõKڝ°x/u—šzW_ ¸§=įÎŗ|ũē]…rKu%p—CŸ;ų˜ũ˜ÖˇbīīĶúĐúZžūQR>ĨõÃŒÔĢ–}7‡ßįõģМ[ĒëiiÉú%oq¯úq=ĪķCZ;FV_P׎Q7_Õ{™_æĪ5ŋcžæzx˜ë¤,wŖ°înę~xR{}äzz›ëéFļœq =ÍךwÕ~,-Éī4u\ļīq5˙2ív´Ž=QOŸōōŸ4×ČgyŲåÚŪ>pįLūlųéIŽûimëŽō{ˇmáz,­m‰?Ÿ?wŋšĻ´ •—åhžFŋĪåFŪŽˇšŽŋĶĘ9u7­Ž•{/˙˙UZmÅŨŽįéjß_Īå}Ū7 ÕtgĢsâ}Ūŋå‹˛7M˜W~Īß­Žąōeέ\įĢũ3Ęéj{^æy~“ëčK<Lë‡õ¸—ų*/įf^ßgÉ8Ŋ3ÔŨqŗŗQPwhËŨ(¨;ŋÅ nŖ1ę>æīŨlQ×~ŖqÄMįíüŗ…ĒžģēS>˜pP÷žųÜ|G –š@ęęÛYē  :nr‹SÍēŸëš9ėÚÆ6¨+7šcáĀ&ÎĨŗ=aЋęõķ|c9×Lķd¯!ĶÔŨĘëqr›‚ē—št{įGĖo1__^57øƒ|>Ŋ­Žƒr ģŨs==Õq-Üč/íyü:‡*‡›ãûE>—Íņ|Ĩãšō9Ow´įÚs(õˇ–t\ŗË{WzŽ—ĶÚļÔɧ´ļeęą*ԛDPWÖ§ž,äũöxƒ Ž [Ęī ĪÕõäRZ?tDVnGPwŠúũy¤cÛÛe~—Öļ¤î/J¨÷ŧzīZĪßåwۉfŋ]Ēęās^ÎåĒūĸîŋĪXP/åcú^Įzŋnö_ žoVį]ûģĒlãķf¤Õņ/Ûß_ĪĶÚÖv%Ā>Đŗ7šcŠklÜĮųØŪč ēŋsÎ7×Á––`š|÷°ųũķ͘×vĀNuƒM,wŖ nÔĪGuåf´|sÜØŊK;?†ŌrOĐqģēxڔˇÕNų|טnßĨÉumŊŽęíiGŠįŅˇ}cWÕË;YíķvĪ››âÍlc×Ã$ę–ĸĨÅʼnMœK]Átšņ*.¤ĩ!ėršŨ§ōMKPwˇgŋodÜ Ž´Z=Ų„/6˜_Ų×]ãæ]iŽƒ˛ ‡;æųĻyīȘaN}KũÍųĻ ɧîV¤‡ķqē˜ƒ†oRwĢžM=ŒÔŊÎAJk._c^5ŋcēž ø>m§Ũ}ōk_€Uļį^Z}0F[ø|ŊĶBŦq‚ērCųrÄ:žß`;Į ęžMĢ]˛6ZÎ×usųfę^žqŠŠqēž:~ö°Ų ų†ņ^uãēQ×Úi5-AŨĩכQÆ ęNU׊8.îįĀjŖĐæ戛ū“MŌ u­ã m>¨+Įh×p Í´oĶÚÖQŖÍaˍŽŧ븏Ô}Ikģô§&ôųŌlûŨMėĪŽkkß9\ŽĢmų”VĮ™l°Ė§i}ëãŽ:øÔŗŽķcîۛ×ÁÅŧ/Ú÷Ë„oŌú–ĄåÚøĒãsšuy×s\”yÔ_6|Hk[¯šu(­ņæ›ŋÚkøËjŋˇåMs-/cĒļĮiW+Ņrí8\V{–ķŽÚgsyēŽ–Đošzē‘ē‡6(áãį\×G;~ŪŽÃ‘ę÷ĘĮŽ:ĒˇûIu,.¤Õ.ø]Ûö1MĮƒgöŊę ūŌü!_xF› ęŽæ°įzĮüBZm•ĩOHė °Ęö=!ŽÜ|tis=ԝL[ ę–ŌxZĩãuĮĶøZøÚ ŽÜ€ĮŌj+ģķc,÷hĮĪ^¤õ-ĸî§Õ– åáŗhZ‚ēoĮ‰ų|ÚzP×wũ­ģņŌÎuå÷ÃĨ´:Ū—Žzī ęF}ŅToīĶûmÔž]HŨC8”¨ũŲ‘ę3כy]Ŧ~÷}Įøbę_ŗŽģvęųŊTÖĄžŪĩÁŨŊ|^ÍuÔɛëX–ÕˇžmËŊâQõ÷FĶmԗOß5ŋ{ŋkæ×ĩü'Šģõd™Oü>øÜņšû˛„}íC"J—á{ÍįKËÄōÅÚ¨/ŧĻáéŨûŪNuepūš\.7?Ģŋ?Ķüėüˆumo„nĨĩ¨XŦū Ü|nŠšA­ƒēiiQ÷͈›ĸō-ųbõûëæÆĻ—¯)ēēX^K[ ęRZ4ŊíJs:ß4\Ú`;Į ęæķ Į뎛ˇ ųúĖ‚ēķyÛš“Š{ėŦŽsŠŊ1-7Ŧˇ{æyĢįæQPˇ9‹iĩ[Û¨`Š<éw+A]}ė-åc;Ž—#ΓrŊ;ŪąŒ.~ģCAŨŠÔũ°””Ö?ä¤ âßUĪįķq}šZ‡ųŽån6¨ûú[ņŊHĢ-ƒ&Ô z‚œåĮĪé*[üĘ ŽXė –ÚqN¤ĩOc--ļÚ­ãO:ļõPķûũmSwåXŠŸ&ŪžMŨÁfšŽl–}¤ã\}Ųs]ĢļņSĮ6Æ1PÂâųæ9ĐŗŸõf)­Ž]yēįÜŲčzÚ>$"\jŽëŗim0;HŨ_-¤îŽÂėá îlZ;ÖWŌu B~!­7ŽAߎęŠûA‡Ē?€ßĨŅ’؍VMË#ū0_¨nëāķvęoYV?ĄĢUĐÅfēīĶj—˜­uåæŊŨĮošĐží7¨ —;öņ‹&LØLPwĒĒ‹Õ ĐģŽcãtĮ1ū´cšˇŌÚîJoRwkĒú†˙Ę _CĻ)¨;oäK7žoķ~?S…b¯šërŗ}3_+NäkÍĮæÜ)O;Ŋ˜¯[Ģã:ø{™÷˙ų*(xœ?{9îPĩÜ;×Ôí ęęcŽŒqÛō]^ŋgÕuĨ\OKhŊ˜Vø2˙•ęÆ˙@ZmqöŽš_×ųqĻ'¨;š?÷:īˇĨ|Ũ.a÷‘ uĨĩōŗjÉû˙Ssí(ë|/×Õbõ{¤nŨģÕ n1­~Ąq9×ī­ę÷ߤƒē‡=ë5Ÿß—ˇo)Ÿ;īrYjæQBŊĨŧ/Ëõōzs­.­ģsŋĘåSUwßõw§;ÖķJu -åĪ<čÛÖ}åÚĐõäđŽ–ĨåŨåŧœãyŸ~Nk[#žė ˇúZŪHkŸž[ŽõŦĖŸũœ6ū°kĖģ{iíĶb˗zu0û$Ī˙\u~ŊĖį×1L‡CųļsŠģëeyŋ+Ā:Ņķķ6¨ä?¯æĢ;9ŧ›ą^GķĸwķMāˇyúĨj™gzļ§|înžIXhnHÎæ?ôË4ˇōüĻy ˙Ĩ|ķ¸œēŸÖVߌ,Wp÷Ũø—éŽo°6c./w9m<ŽŪטĪëŊœēčßŦ…|Œ.ölËą1ë¨ÛũĐįRę`|–LSPWöįŨ´žeîį|íYč8ˇž§õãj^kΝĨ|“ÛÎķnsí¨ŋh¸Yo÷›Ī~ĘÁF}líDP7—ˇ­­ŸûuįY;ŪŲķ*¸‰ízœÖ‡ÚįĶęS/įĢúûžÚöz{Íīv™¯ĶÚ°{ž.¨+!ÆįfŌúŋĢŽÚ/ļԕßYOĒuyQ;“ęžOĢ_H´§Õģëõ`Įųō2­ũÂėfGØĩØĖīûŧoc[ëˇ'mpˇØķ;æVŗßŪ§õ]\ëŽŦÅręn5X¯Ĩæ÷c{žÆúŸj~•kJęXūŖf¨Öˇ(Á÷¨/…úäÛ {`ģu°5e<ļhYÃ4D—á{iõĄ9ŗĸ}ĸ)0eIPû… ļ.ÂēøËÃK"´‹‡&ĖÍČúĖëŪŽ€éUÆn)eN•Āž%¨€]ãjÅĶ˓ĪĒØ^ËiØr.šŋŸV„QJ„u Ē ØĢ¤awÂŖ3´ÎGķ:°ûö”ŌŊĩ.ŸĶđi16•îíĀž6HÃ@äę.¯ĮĄ•ōxĖiK 3°ûö”åŧo/­”oĶ0ÎûÆ MGPw7¯Į8Nįéí>öŠAšŊ fÚüJš˛R^¤áĀüˇWĘąÔÔY)÷VĘĢ\îä÷Zō<Ÿ­”ˇ+åŅJ9ÕLs<ĪëužWü˙Dõķ+åM^ėÎį÷/įr:îiv‡l[ÔČ¯ËēŧĖˊutŦs<9ôI^߇ųķ—ķzŅåōR5ŨŗåĪ}Îī;Ô=Lãu!‚Éčūú:­†vo{ęÆO-}ßņūÁ´6¨ģ”_Ķ1íé´öAē=ę˜îpžæ`~ŨvyR”‡S”ŸíDPw3ŋ>Ô1Ī×ËĒ[ĪÅ˛Ę˜wˇNlՕÔĀ•`ŽuĨÕ\WKšŌ"­]ĶđIŦ Ít7Ēé" Œnˇíøn÷ō4KųõNue<žëÍtߤĩ“ˆéĸÅßÅfēų4~+BčaZŒĩ-ŲĸeÜ Ŧx:kû4ØšEāv8—Ō­~Rkt @ëyūĖķb~ī~žĻ<€"æy$O}ˆ€īi5¯ōЉ—ķ{“ęÂŊj;Îåå}Ėë\–Ąâ›\7ßæåŠ>{ĘáĀ׈Ā-ÆŠ+ã­E×Ķķi}PAUgŸ›i¯§õ-ãJ7Ö/U‰î@3¯OÍ4ŅoŠšO<‰ĩ„†e,Ŋíęb}"€|ŸV[ŅMëĮđ‹ēzŲŦsŦßE‡“!ÔrZßeĩ?n ŅZîĀˆé"ü:šįšÔ3́<ŸQĶD×ŌA>¨b;ą6>îx˙P^įcÉ_`"Τa˸ŗÍûeėēkĒļ_´Ļ‹qúbLēsī\ū7^G‹ēEU;ã`>H"ãæŊ^)ˇ“€Æ?vāĀĢųŸšˇennîŋWĶ0Â?ņ›–>˙Ÿûš/ÛYæ~Õ¯ú^MĀ‚:˜‚:˜‚:˜‚:˜‚:˜‚:˜‚:˜‚:˜‚:˜‚:˜‚:˜‚:˜‚:˜‚:˜‚:˜‚:˜‚:˜‚:˜‚:˜‚:˜‚:˜‚:˜‚:˜‚:˜‚:˜‚:˜‚:˜‚:˜‚:˜‚:˜‚:˜‚:˜‚:˜‚:˜‚:˜‚:˜‚:˜‚:Æ4X)ĮTĀ6Ôõú2FŠ§Ŋ;ÁeŸËķ\ίųõÕ]Ŧûšė¤ų•ržyīm.;!–ķÔUØ‚ē^ŒEPsuD)bē\vÔĖ˸´‹õņ!¯×Nz˜Ö‡r;Ô=K;Nû• ŽWeģ՚Ē ęvÛņŧ>K;ŧܨ˙Ũ ęvŽ Ž×f‚ēÖNT¯Īä×VĘŕrgĨ\[)G{>jĨÜĖåDZÔČīm–¯#<û. ģŪ^Y)‡:æ?ˇRž])ˇŌ°å߉üšsiŧđ-ZžcēŌUõöJšž×īP^΁fÚãy}īæí>›?_×áë•ōžŠßԕemTˇGĒåtÕOŠÛCy?ÜÎë2—×á›jÚorŠˇķFŗīk'ōĪoåúOû`HP×k3A];F]IĶ0hú_Îål5]„AōįßŦ”ų˙ĪŌÆcÔÅëč–ų. ÃŦˇųŊOimpt š_„m/ķ˙§ņ[í=OÃām”Ĩ<˙/y¯ķö>Íī ĒmžŸVģ?ÍÛđ%Ļ„uĪōļ|NkģŋÍÛĶ~Ė˙–éÚŽšWōûīķrŪį×ĢiJŨ–ũđ%¯Ī\Z?FŨĶŧ^åe—ũŸšĐ,ûN~˙û\ŸĢe\uåÖÔõ*ÁŅšžŌNÛuņ^´¸*ÁS´ėú”įY\ĖĶ]ĢŪ‹íc/¨‹aÔ\~īd~īQ5ŨÍü^ŊÎ1Ũį4^Pˇ§Ũhēyē“Õ{gĢõ4ī]k>#ŋ˙mõ^_×טîNUˇ‡Ķ0}SMw*­peēųj=5u[ęhÖļŪkƒēÎWŸŸKQZDŪĒöMŦã÷IPôÔõ÷‰¯eÚ6¨‹°mž™îaķŲhu3×Lw=ÔŊéXīWi5ܚĪëņ¸cē[iŧ î۞mŠ-ĨÕđ,õlķ ŋ^ÎÛ×ÎīXZ ‹ž Žk}î7uû¤gē<ŪmęöIĮē÷u‹ÍtwĶÚ1üĸ5āûŽũz6 ę€>‚ē^Ĩ įrOi§mƒē—ķ,NũšĶ•–qe9ƒÔÔu}ļˇŽĨū`čt/¨‹u~´Á4e^]O…-­Íû˜EËĩ¯"¸{ŪąŽ›y˜D[ˇŅÂîûÔũ´ŪøŲĢĻnģžÚÛÔuËW›mŒ–“;Ļ;˜u@A]¯¯ŖŽëŗu˜´Øņšb9Ôu}ļˇĘ|.ŽąŒ>ąī.l0MéęynÄĪųõŧŪĨëmüĄæÃ4Ų .ū˙ąšž-ĪFÔmß~|Úŗė6¨ëÛ7sIPôÔõÚî .DHÕÕōjÜuuGRkąŗiã Ž|ūĐÛ_Öˇ+ü.­ ąnį××ķüK÷Đå4Ų .BēgcėģŽēíۏãuz–}8 ę€>‚ē^;ԕąĖÚqÔŽĨÉu!ę>ēyļãĨ=Lu—S÷8x­h%÷Šg›K—ÖAU7]ĮÅ4Ų îI^§Ĩfē…<ßĢ#ęļo?ŽÔ=čYöwIPôÔõډ ŽëɤËiü§žŽÔ]¨–-ēĸu\yXÅFA]Ėëö˜uPÂĒÛy9‡Ģí­CŦx€Æ§üķâx–í6Æ"ĘZĢē}ģ‰ē0´f劯u7ŨŽēíۏãuGōz?ĪÛīŸ¯öĢ XOP×k'‚ēĄM„:â|ŸVCĩIuáZZîKūųFO}-­äNYsyžõrĸ%_yë O÷Mžo”˛^į˙_Ę˙Ū¯æyąš×ÃĒnߎYˇķú|NĢO‹-ŨnĶˆēíۏãu!°ņĄZ˙ø˙å$¨úęz-¯”Ŗ›˜ļm!ÖõŲÊ;‹XŅ-2‚œh=ļ§[Č?ŸĪ¯#–YÍËo•yÆĪ"T+!ØąžmŠ'”F t`“õļ”—s,/§´ŪĢģÆv\Ęķ?WũėXĮē/įiNVu{|uÛq>/ëRŽëZWŨöíĮžēäyĖwĖûDūŲ´:ßW`AŨžcĩu=Lĸt+=0ĄåD˸Ëī—qøö“;+åQZ?.`…㎖QËļߔ.ŋĢēˆúÖ? `HPˇoÄuŅš.ZwÍoķrĸY´ĒÛī-Įæs}—qú ę` ę` ę` ę` ę` ę` ę` ę` ę`Â~üã˙™Å6S~ëo>ō÷ˇ;¨ûŠŸúŠ_ØĖ:­L˙?ۛĖŦÅÅÅŗ‹ŋî×˙Ŋ˙ũø_žlwøļåŅõß~YüĮũ?ü—~û?ÖŪ`ĻÍjX'¤`Ī™ĩ°NHĀž5+a€=oÚÃ:!ûÆ´†uB:öi ë„tė[ĶÖ éØ÷v;ŦŌ@ļ[a;Ö é ĮN…uB:ØĀv‡uB:Ķv…uB:ؤI‡uB:ØĸI…uB:øJ_Ö é`BļÖ é`Â6Ö é`›ŒÖ é`›mÖ é`‡ô…uB:ØamX'¤€]RÂē[×~ŋvS„uŋé7üƏB:`ō~4ŋøs?úG~ÍEQe÷ĘĘåøßHû܏į?ũēųOQEQv¯Ŧ\Žßú ¨Ô)ŠĸęÔ)Šĸ(‚:uŠĸ(‚:uŠĸ(Š Aĸ(Š Aĸ(Š"¨@P§(Š"¨@P§(ŠĸęÔ)ŠĸęÔ)Šĸ(‚:uŠĸ(‚:uŠĸ(Š Aĸ(Š Aĸ(Š"¨@P§(Š"¨@P§(ŠĸęÔ)ŠĸęÔ)Šĸ(‚:uŠĸ(‚:uŠĸ(Š Aĸ(Š Aĸ(Š"¨@P§(Š"¨@PˇŊå_øŊņËü‡˙zoųŊ˙õ_ûō;ūŗˇ_žŊūæËĄŸū?fūūėī˙šŸl[ü_¨1ŊeÔqåßũåËŋõŸūå/˙ėīúŲ™ŪÎúü÷īūUû^PĀ~ ę"×į_ū•/˙å˙ú _~Ãŋōgföūŋyú7˛=ņĄÆæĘāßxųÃ1Ųv/k3~ūŨßûōÛĪ_˜Ų@˛ø+ŋđ÷g‚:uã›å€KPˇõ-ØūÖßųåę.ZYNSPūΧøåŸūwūŧ NÔ°7‚ēŽ.†đŋ˙ō÷ūÁ?\3Ũ?ķīũŸ3y˙;o˙_?tQâ˙BņK„HÅNuôO}Xw\Ūøīūī/?ķöīŽ™îüoŋ(¨Suė Žoēũ?ųų5ĶKKPˇ“A]ßōĸö_úkŸ~2]´øÔ)‚:ötP÷[ĮĪŦky×N­ėĸESé˙ūą?ũˇžœų}Šwž˙ĩ?ûÃŧbŒąŌ}ņũÉ_üōOū›î‡pæŲŸ˙Û?”:¨‰ų•÷Ŗ´ķõķx8Fy?ū_Ūå•÷cbŊūĮį˙īãōÅvÜúŸūŸ5áP´Æûŗ?÷wøy#í?˙Ŗã‡ų´ëS/3ļãˇüÛ?ķC+ÅROÎüŽ˙â¯üdŪņ˙>Å4ąQ˙]õøˆåūõŋ9 Ņĸåã‹ŋôw~čĻÚ5–`L[Ö%ęŠŦËßü۟øüßøđ~˜&ÆĸkëŗnUë×ÖaÔŲīģ˙×håVĻúųŲŋú˙ũP›}É8A]۝šëŽzˆ`9ÖŖˆ˙G‹ŧz;Ûc9ŽÃ:œŒ}Ņ‹},ųã/é'û7ŽéQŸë ęâõ1üOũÎWë>ˇô¯ūŲ–SīĪúįņ˜?ņ3ŋô“}û6ļŠĢ5lŊŧØ˙qŧÅgËįâxÔ ¨›’ î_üūâšéÚnŖŅ⮄V]ę Ģ~0AžÔ"xŠ‡tƒxŒZįQ?īŖŽ"#,‰€ąaV }"éĄÆoûŨ?ÛģĖcš.7žû!”ëĢ6TŠņØúæ"h‰0§ūL1E2}ŸũR>ÛÖg×x…ŌÕ-Ûúļ!‚ÁIuĪ_\ķP‰6ŽŪŠ}Ũ†`x:–ûÆ7Œe:6Ę>‰é6 ęâ8+ákˆPqT+×ãęã#ömŸ˜6>Û>yļˆ ˇ=/g­­ €=ÔÅ {["”Ģ¸Ņ¯[ŽE :؈›üŽĐŠ "´ĒEKŦ@J°RĪs'ƒē.TÆtāņßXķÜ؆ØÖŌ"°Ru@ÖļøŠĪÅ{^––líLcžmčķūÁīׄBuk¯ŌŽO0Öˇ/¨+ë­ŦbyõüBy˛kėˇøYŊ?bŊãŊĀÖĮQė˙vb?D ģz73†\bļĮe´këļ „ëc0ęĨÔ{}<ĮqWˇ>Ŧë!Ö7Ž˙ؖ¨§Zۂ­=žc>ąü6ĀŒũ:N×׍ēÄÖar},Įúļá`}ĀÕuQ”uP×e3Ģ AŨ>õĩíW‡!ÕĄGtŦœō~„]-ÖJ‰€§¯ĢíNu.•.‚ŋũ÷ü…Ÿt­ÃĒē•Wgu+¤Ō•ĩ+¨‹€ŠnŊÕļdĢ[\Õ!OŊž1˙z]ëā4ÖˇŦgü[w9mƒēhęîŋŅMŗ/ô5F]=ߟžųķëZ¨EXAbûŗI>õ5ęĒŽģčÎŲļ–m­ˇ'ēĢļĮAÔEÛ"1‚Ī8ŪãøŒų×u^‹m­Īƒöx.ËČE8VoížĒöĪũ¯;[⍠.ŖÅj_Páe,#æWŽA‚ē) ęĸĨ[v”` nÁÕļš‹ Ŗūyk­šB;~YĖˇŒņĩA]+]OŒ­ÃąQ?¯[MÕˌeÔØuĀ×~ļo¯íÂÚŽKũķŌ2Ž}?ÆØõšļ{į¨ ŽnŨû;ZŸÅ4}cëM2¨‹}Ô¨.Ĩ<:öKPĮaÛ*.ZķEØØÎŋ.ņųú3]ãÖ-ÚęãcTËšžVsŅē˛+n×ĩvĖÄôęV˜}AŨfUA‚ēm ęâu´´iĮ¨Š–D]áCÛ2.ˆęRˇŌ*-‰"XéWŦ+4Úé ŽnõÖ5îWlSģuWÕ:ë ÛĘ8}}­žF}ļíēŲŽKŨˇnU5*ˆk[\m&¨kŸ ܆]Qw]17¨ûŖęÃOēS×ÁoÔsŨ˛­¯+jÔW[GõØuuhËę-Øâ3ĻļãÆ>čën\Jt?í ÉFuqvĩđĢĪĪzü¸öØoˇšíŽ^ō6¨ëzx… AŨ.>Lĸ>Z÷´aŨFc[ĩJĀ3*ŧš† ŽîfØׅu#ãum Õ.ˇīŗ›QoįFA]_ũlÔE‰.Ņu(ÛtmæĄ}“hĸËėęžŲvķĨČĸå\_XW„ą]uĶõDäQĮ娠ŽíĘ­#ë.ŊZÖ­QG=øŖKiņ؞Įm—_A‚ē]ę"$hÎļ›e„ĩ“.‚žRZęÔˍąÅēÖ­î*¸ĶA]W‹ĸúAąÎŖļŗ–ļ#¨ĢĐŨ.G­GbmgPWBŽøY´fĢģ.oĨĩÖ¨§žļcûÅ˛Únļuëąh 7ĒŽÚ.Ûå؎.ĐņŲŽ§ĀÆûeÚh‘ˇ™uu‹ËQ˙<Äz>íÃ9ęc?ÖyŖc´tįmƒēYéuėÉ ŽXm+Š:LŠ–7uˆŅ5F]tyl[âEË :PhģF+Ąē‹ãf‚ēQÛ4nP×5ļZ´ëęēXūߎĩˇ]A]bvQuŪÕ*j;ƒē-ēîÖcšEØU?PdTȡ™ Ž Nģę6ēʎŖ.Ãö¸+uŦwy@C9Žcß´OâŨęu"ŽÔÕ•ˆsąîÖÜ/íC-Úã1^wۂ:u3Ôu=ø!‚‚:ĒË#ꟕ,DčaI âß:āĢŸ@!GÛõ°ę" ÷iž“ ęâŊzcÛęu.cž•‡lgPW‡_Ņš,œ:¤+ë-˙ęú™TPW?l#öeŨzŽ} Fû`ŽÖk[ ę"l÷u=]ũā„ö8‰Ī–Ā+B×x8C cëPŦ}C=_üŧ¯…ß$žúZ—ŽŽ¸]-Qc™uˇāúŧŠ¯ËēĮ<uęf,¨køŨīę@¤°b ûøy´xĒ[ÅĩŨôęVHåsĩáKÔE+¨zyÅ˛"ŠCŠIu]ëÛëV?˜  Üļ#¨‹:¨ˇĩ„ƒŅŠŦîuY§_ÔÕLˆ 'ļš´ ĢÜû&ĄÍbLēēnbŨF==u3A]WĀķ¯[‘ũ‰ŸųĨ5ëû/öWũ0†xŋ<č"‚š:TŒú+Ũšã˜Žëļ Áę'´–î­Qßu ¸ļ5Ũ¸A]†.ļ%ęŧmiËh$QĒ‚:u3ÔE‹­:ĀˆpŖL?Žqŧę§jļMŧŽĮöj[ ÕaJ;@Ũ­ą}`AÛÔÅ:ˇĄ\Ģ}`ÂvuĨ›cß8pĨĨXŊž6¨ĢŸÔÛv)§^úúđ5A]WØzŊ#xkŸ`Ü/m ŋXĮQÅ(ãĶuĪu0Ø%ZąĩŸ'¨k[4ļ‘hËú“ŋ8r=âįēž ¨›˛H„7ĨŒšö§oūüšiëVuQ~ÛīūŲ€(D˜A\W}O‘ŒĐ"Bˇō™h…-ãĸÅXßS_ë.š%„‰Ā*Z Å:D8ĶˇMŅ:Ēŧ˙¯ú3m—Įļ{a,;ščDKĢaē‚¨z™]ãÉÕˍ'znæŗ ÆûQuw×Õēļ!ĻíÚūę§ėĢÅJĢ˛ë'DŊDwāøl™&Ö)Ö-–Û~ö•ē^ĸEYßtå)­õôõ˛â؋m‰nŲeÅēĮąÚÖw]¯Q‡ąîĨUhc1īØ÷]cЕŨZãX,uŸ‹Šuwמs°muZ—RˇQâœŲ¨ūbyąÜz=âŗqˇĶF=Œ{Ô ¨Û‡eŖ NQEP€ NP§(Š A NQEP€ NP§(Š AŨū)1Č~tQÚ§Ą*ŠĸęÔ)ŠĸęÔ)Šĸ(‚:uŠĸ(‚:uŠĸ(Š Aĸ(Š Aĸ(Š"¨@P§(Š"¨@P§(ŠĸęÔ)ŠĸęÔ)Šĸ(‚:uŠĸ(‚:uŠĸ(Š Aĸ(Š Aĸ(Š"¨@P§(Š"¨@P§(ŠĸęÔ)ŠĸęÔ)Šĸ(‚:uŠĸ(‚:fŨ~õ¯ũģ?ūGãgE™­ō>˙čW˙š_ūŅÕ¯üčG?ú•ôãš_ųŅOũÚ_Ž÷Ս2‹eårüÜo$`VĖ­”“+åîJų”Ë/Ž”ŋŧR>ä×ŋ´RîäéæTLF į"|{ŋR^Ž”Ÿ_)~ĨüΕōũJųMųßKųũ7yē÷Ih_åÄJš•†aۋ•ō{VĘ^)īVĘų4 ŪĸĢāŲ<ũ™•ōjĨĖ¯”syē?˛RŽ­”gIhc‹pîfļŽ‹qWVĘoÉ˙FĐv}Ĩ,äi# {Ņ|ūŅJšœ˙ŨÕüš+å7§a‹;Ąt8š†\„s¯Ķ0”;”öm> ķAõ^˜ĪīŸhæ5HÃąęÕ{K+åv†sÔE0w0 íā‡p.ē¤žÉåZ~¯8–†!Z´Ē[îø|„y÷{æ-ęwŧ8 [ÜÅōÎTī íØWĸE\l”Ek¸6œ Ņú-žčZC—:Ļy—Öļ°ĢÅgbŦē3=?_NÃ0Æˇk[ä íؓJ8ÁXtmņįNtL7ŸēĮĄër'O3ĘņŧŧQķ9—§iģÕ%´{ž„vĖ pE8-ßúšĸoē.Gō<ÆXÖnm0M„ŅUö}^ĪžųցŖĐ€Šĩ•.ŖC×åÉJš0æ´ēŊËËŲČb†ząîßĨa€×GhĀT‰p+BŗÛâIĢãVãŒC×åT>v3ØŲ•ōbŸ‰îaļđ;;æôB;v\„sŽEËļįĶit ´b3ãĐĩ"øŠî›-ŦsŦëĨM~f9 žiüÖ~B;ļՁ4l]×Į´špŽØĖ8t].æåoÅá4 ÎnáŗąŪņ”Ú‡›\īŽĐî„C €Í*á\Tōŋgķû›ą•qčZeŦš#_ą=WĶ0$܊$cÜēÜbģĨM~~Ü'ßĀ"Š–rĘ}L[įÂVĮĄër# [¤}íļE×ŲS_1 #d‹ĀîJÚ\‹ÂBh@§ÎEwÖh9ŨK#X[üŠųmuē.lEØˇ4mņâĸûí ŦĶũ4 ÚÎ}å|„vûX´n‹Đ*šh9÷ĩá\ņĩãĐu‰y]™āļĮ6_ŸĐŧŽ§¯īÖ[íö‰ÎEŌhíS< õāæ=‰qčēDPÁßüįš˜ˇ˙Čįy& 8ņhBķÚėAđÜNÃpęyš\8&9]—i8FŪ¤Īu1IsšnË^—&4_ĄĀ ‹ 'v"ā‰ įĐį?éqčēD@÷|ë(æ}~æģë$ęæjšlk@ĄĀ 8š†OGįUš|8WlĮ8t­ųŧÛBEÕĶˇiūƒ4/ZÆ'&ŨÚPh0E"œģ–†ãŖŊÉ˙?ēMËÚŽqčēDuę/ZžŨÛæeDŊ=ÍõöÍ6-Ch° "”‰.•;΅í‡ŽkyŅŌm°u-÷ĸuāÉXÖéŧŋžlķūÚlŖŨ_vbē.wōōvĘŠ•ō:Mv,š>r^Čû0ÂĪĨ=xÜė9ņdÖīŌî„,;1]—Ŗy[v¸Žc;¯îāōbûĸ%ä‡ŧÜ;°LĄĀ&D8w) Į‚‹–lŅē,ēeÎíĐōwrē.Ņ-ôÂ.Õ{tí=ŧ Ë-ŨŠ/ėā~Útˆ§Ž^Lģ΅‡ŽKtA}ĩKː>ŲĨeGKÂxāÄë\;IhėkÎE ÁPtŒ'ÆÃæwx=vkēV„sņ …ovqŸÄ:ŧX)gwqbû#ŦŒĐîč.,_hė 1Ųš´ûá\ą[ãĐu‰…§`E×ßhU¸°‹ëáųŧŅĘq°Kë!´ö”įĸ…ÖÃ4 įæ×vqv{ēV„bŅĸīȔėŗ¤îLÉąs57ģŲÚ1ÄØ}ņđ‹hõ(´fF´;“Ļ+œ Ķ0]—+åöíŋÄ"Œ:>%ëSīˇ‹S°ßĸKŽĐ˜ZÎE7ÖčÎá\to0lqJÖmÆĄër(¯×Ō”íĪZwķÁ]ĸÅat~“ĩi ´ĻB„81ø˙4†sÅ4C×%Öëō”îßGSēnņDāčļœ8>Eë%´vT„s”ÄfŅ,Æzģ´RNŲzNÛ8t]bŊ"DœŸŌõ¤a;˜Ōã0LŨaī§é a…vĀļ‰aÚÚ0­ãĐuy‘†-ūĻY´¨{4Åë!įÕŧŋcŦŋ…)\Į:´{“˙Ô%،įĸ%P´Š–i1ÎÛĄ)]×i‡ŽK´{>Į@1VŨ™)_Īhoįũ)Mo@+´ÆĄÁŦ„sÅ´CךOĶõTՍœČëģ0ëz8 [žIĶßZQhŦ3ĢÁ,ŒC×%ējŪ›ąc$ē=ߜĄõ]ÎĮE´Z<áĻY´<›Õ``–ÆĄëZ÷čž9˜ąãe!×÷ą[īŗiļZ[Ļ$´€}!‚ŠčĘ-Ę“(ĪĐúĪÚ8t]"`ŧ6ŖĮO„^ņŒš[ī8n.įãææŒ7B;ØCâÉŦRÔá܉܎Y‡ŽËŅ4;cŊõyš†k˜E‹+åVvqNĖĪāņ#´€á\„)1~[„1žØÉŨ–Y‡ŽK„\įg|âa īō16Ģ"č}˜†ÁīŲŨĄLąh-t1­įæft{fyē.§WĘĢ=°!†ņ`lG´,‡MDwŪåŪĄLį"Äz˛R>¤á“D#šå0h/ŒC׊ũĘÉ=rÜÍįí9ĩGļįÛŧ=ŅĘîЌo‹ĐvЁ•r.­įæ÷Āļí…qčēD7äĮ{ė8<™÷ÕüŲžØŽīŌ0 ž†-:gĐļA„s1–V´øų˙×{%$ŲKãĐĩĸE`„?‡÷āq!ņõ=¸ŋâ+ŅåúĘ:Į„vđ" ˆfm8w`mã^‡ŽK„>ˇ÷č1]¯#„<˛ˇ-ZtŪOçôžÛcÛ&´€1D8ŨXŖĨR„sŅŊ5Ŧ…=¸{mē.‡ō6.îácöB>a¯:žV[{žÜƒÛ'´€J´"‹AųÛpn¯†;{uē.Ņ ōō>8†Ÿįcv/;“†AVŒ5xdnŖĐ€})šhs' [\E‹xāĀŌŪæŊ<]—ØÆ;æ÷ÁļI{ŋå`9oã<}—ĪŨŊ|ž íØķ–Ķúpîāßæũ0]—$ŋŨGĮvtažˇOļu!ooœĮWĶŪc# ģ‘†ãõ í˜i'Ōđq“áMŒÍ6ØÛŊ_ÆĄëxļĪŽķxČIti>šļ9Îã'÷S]_Ī„vĖ„ci}8whm˙~‡Že7Į÷áqc-žNûŖģo{ž?])¯VĘ7ûhģ…vL­Ž1Žėŗ:ØoãĐu‰Ž÷öņyđ ×Á~OlŽ ōIځ•Đ€]w8p=ė×qčēę!žÜ{pŸąíīķšąÅą! Ģģio?pĸĐ€]9Ŗ+ëË|#7¤Į÷i]ėįqčēD0sÍ)ōÃCRžėķ:XČĮ‡üī~=7ęĐîuÚð Ļõá܉}^'ûyē.Ņb(ZP?´*{ąRÎĒŠŽĨĩ酴[›ĻÔũ`×ÆžÁŽ–A1æZ´ģ“ö×-û‡Ž[B;€=~SƒÛ?LÃpîa~-œ[Ī8tã‰`%B–ÃĒb¤Į+å˛jčŨĘŖ‹p´°ģ˜„š]„v{@„pŅ*Ŧ į<™ŗ?00ŨønåÂhƒ|ū TÅHGŌ0ԌŽÔ§UG/ĄĀ ™Ī7š÷r8Ũ[#tŌ­lt‡nsåúZTc‰uē'Æ;Œ*Zĩßo4ĄĀŠ–_§ŌúpNˆ˛1ãĐmM„Nߊ†MŖ1VŨU1v}KÃęžss,B;€]ž‘–'1ļS´lŠÖ'—Ō°û&3ŨÖE}E÷DãnN)ĸhą9žŌÚ5ēŖßPw›:քv;`9­įĒ–ą‡îëĖå-Ãļ&ÎŨ›ĒaKįíí|Ũ‹kžx|B;€¸Ņ¨–M1ŨdDwÄgĒaË⸋ø˜ĒؒxÂptģŽßĒŽ‰ü.ÚŒá˜Ē‰1ŨdÄĶ‚…L_/žŧü"iÍų5–s>OÃđ‰ÍÚlāčJš–†­EŪä˙Q-[fēÉēš†,áë=])UÃW‹ĐS˙õ„vYtåjÚŖĒåĢ‡nōbÄÉxˆ“<īߊĪ‰ˆní—Ķ°[û­äi×_Khė;‡ōÍĪË|37EēNæ†Ũ8tÛ#‚ĪĢĒaĸĸ>¨†‰‰€îV>˙/'œ˜„ÚŊKB;`9˜Ö‡sÆVšãĐmŸcųFũ€Ē˜¨’ĸí7Ēbĸåë@\ÎĒŽ‰ˆVÉ'ĶęĮ…vĀLŠpîRŽ“ö>ßä,Ģ–‰2Ũö‹ņÔΊ†máGJZM^|›xáÚ0QB;`ĻD÷Ģ6œ;™Œ“6iÆĄÛgō¸úŨ>ņ€ŽëĒaÛDkÛhšø( Įdr„vĀTŠp.Âĸ'i8ā~ÜxŸJí`ē­ë8–UÅļ_?âxö„įí=–ŋËõ|; ƒ~&KhėĒ¯Ģ įN']Øļ“qčvVT萆­pŲ^ėßHÖ¸W\¯ˇMWhøđ”c`ĸ"œ‹ÁÉĻa8˙~ëfoÛ‡nį•V^Ņ㊝W ;"Žëûiø`ŸQëįíĶ†vq-á!„vĀ–D׆sņÚ0ˇŸqčvĪ­\Ø9Ņõ5‚ŒEUącŽ§Õ/NĒŽm'´ļ$ščÆŨY#œ‹î­ įvŽūCˇ{įēWī;īzžî°ŗâzã1>NÆ Ü)B;`ۆŽpN떝eēŨãŌ}§vŁ|ü/ĢŠ]ųp1 [đFxä;[÷B; ÷æ@8ˇķŒC7â|x“t3ŪMņÄč×Éؗģ%Z’^ĪŋŽ&-Š§å÷˛Đö07ĶÃ8tĶuƒAéUąëĸEéUÕ°ĢiØÂÚĩiw¯I~_ĀubĨÜLçüEqÅûģĘ8tĶ'ˆ§Ēa*Lžē;-ŽæķâÕJųFuėĄėņDŋ6œsãģûŒC7}ĸ{_´:Ļ*ĻF„OTÃÔ(]’cŸUģJh3$n ŽĨá8[oō˙Ģ–Š`ēéįÉŨm˜oåO7(GˇpŽĮįęÖM1Žd<ąķs.gķ47Ģiî7¯oæ÷Šsų3Ķō#^ämazöɅ4üō'ÎÁĐtėĄL™#i}8§ÅÃô0Ũt‹Úi{Ēžž¤ÉuËyžįĒ÷næ÷ĘØn‡Ēåo›×Oķ{ÅÕü™Áí›ãųŧŅ5|ē,äß3ōŋöĪtÚĀ.ŠņčĘ­ŗžĪ7ęēíMãĐ͆0ģ\P‚ēIZNëƒēX·fēAZ>nÔ-äĪL[|kĨÜv˜NĨō%Düē| 1M„v°â&ē įN¨–ŠdēŲáv´ØšßĻųo&¨[LŨ­ŲÚm9­usųgĪķ5!ūŋT]/ž&¨;PŊžĪË=ŊÁūą<͑j›ž6XČÛvÜá:ĩĸUhŒ]c؝RSGht0˙Aũ,˙}'ßlš‡nļÄž:ˇķßLPwˇgÚļKęrZ ęų˙u)aÜ×v}=—_GwíÕüc ŧëÍ:F øĸY3īAŗŒ­ŠāûUŌbkÚ}“÷ĶĶ¤…÷´ÚĀ,ĨõáÜI7ŠSŋĪŒC7["üyšÍË(AŨ §¨ĻŨJP7ĒEŨ¤‚ēO+åģ4l­Ú^å÷W7ūQsÎįŸŊÎĶŊP]FđwŲa;õæō50Ž…÷ŌtyČú}%´€‹ųæ&ē}Č78§’ĀgÚ‡nv÷[äKw1fƒqčfÛbžų܉puģĮ¨Ģ—ŗ]A]{ŒŌÚ îîˆmŧ—&ÔE€]jĪ8ŒgN<`$ē/ŋą˙fJœs§ōšü1 íØ#"„kÚx}@ÕĖãĐí ˇĶ°ØNØAŨüzącŨŸ¤Éu!žtũŊëįĖŠ–Zņ%G>žä;{ËDĢĄ3˙í‡|ÃzŪÍåĖîKãĐí ‡ķ~ÜŠ}¸• Ž €o§éęÎį×g:ΛiōA]ˆpđĻÃyĻÅņëũdč€Y˙GhĀԚKŨáÜĸĒ™YÆĄÛ[§õ=ØN› ꎧõ×á|-™æ .BĪ?ßTŸ›øŌŌn;‚ēXf´l=搞iåKØ—7“/Afy? í˜}ƒ. įf›qčöž8Oߤí˛ŧ™ îPžÉ'ĨF0]ä?Ĩa‹Ŗię‰jŨ_¤a¸uũ*mOPWÖīEŌ}/ˆanįߥß%cļÎ2ĄģzĶī‰h{ķ†Ņ8t{OėĮWųr'E˜tuĶÉוŅ"Ŧû& ƒą˜GimT^m–ĶļŧšÖ†y—š×íg–›åͯÍ|ōûËįÎĨüŗ‹yēé^ncũ>ÍËboˆ¤PGČû­ę˜y}C€ø"€‰‰V#Ņ=įû|ķ]v„s{į†Â8t{WÜ>U Ûâx>gįTyxÎv‰`įëđžŗœ†­%ŸįßģėßąB;&vچsÆ*Û[ŒCˇˇ•ņĖŽĒŠmaYt…@î@uS^‚ąŨ-ŖŽås—Ŋįlž6?tmŪS„vlÚŅ|ķ÷&—kųf”ŊÅ8tûCœŋwUÃļ×qŒOcęŊÍ˙Æëë;tĶ×éoė†=)öīå4lí|+ söâūÚĐéHZÎiŗ7‡n˙äü%Uą#u}! Į¨ģÖw…ŨN1fčÛä!{Y´ŒŊ™ĪįËöõž$´ā˙oīl ė*Ë{ŋ3™Læ33I&ɐĪ! `€€b‰%1ÕXŌBŠØčE^¨ąĻČõŖP›Ö,•Xšče7Žú-XliE KŧrYH)ģ°Ÿ|h”) Éšį?g?øäeŸ™sæcŸŊ÷ųũÖú¯™9sÎ>ûŧį}÷ûž˙ũŧĪ 0ŧ”FKYQõH< Āœ+ö$€aÎAō™Č5ęė›dđžOŊQĀ´€Ė (9-eŊ?Ėj‰+æÔyč`<\×€z¸5ĒäF¨idēčäŠj 4ínŠ˙îĻh`˛H2įÖP,P䥃ņ°:žö´SP'ƒQåæĀ Euĸ]a‘Š 'VQP#Ū´Û˙Ä´€ AI•ˇD•bšä(/æÔ yč`"¸;žčŒEÔŨD1ĀĐō×ķĸJĒ†k# _¨L;7IæÜéyZ ~ČC…&5÷R 0tũy(b#ŒEķnHŨcĶjFIoe¤ÜtĮx]„ącŸĖ‡&˛>iÉ4Ņŧ0^T‡aR ãD7ĄvÄ}ÜÆJ0F0íāU$™s"ō?Áø L4—Äõ `"ÁrÅĀPTYN­ 'ÎĻ8``Ú08h  Á%æŒōĐÁd Č-Æô…‰ĸ/ŽSl ÅÚ¸īS¸šâ€IĢcÚĐáÔ yč`2QôĶvŠ&m p7×+˜„zĨĨÕ×EÜ\Æđ 9-cŊ6îØo‹tė0‘uŒČ˜vae<‘ąą3œ`n@!…íyž ß F•ˆæAŠrŠĨÎĐÍkE8! yĶ g˜9ˇ3Öļˆœ˙Á4yč ,ëōE9GQų7Q P€1ķmņxyÅ9Ķ Ó? š•7FwˇG˜soČCEDKļ?L1@° Q07 hc5EėkÉUäL;€cæœ2į”n Å€˜vPˆë†øĻČ9­÷ßwęÍy蠙QÎŽķ)h2N¯ûLÚ ™ĐG+ddTo¤8 IĀ´€ÜĐRwÄ,]¸0į Ų 4;ëËz0b‰74'Z=Ā’ohF4æš7ãh&0í sxsîņøĩ%"Ą24䥨Ô{–˙Aŗ÷­ ( IŲ÷Š˛cE4Ū´{<ž#ŸÎŧŌB[ŗ__€î0į y!Ā¯ŲUō43įĮ“4€f]”ĮŽÕ5Ќ`Ú@*˜9§á2įdNp§ šōĐüšžx :DQDwG•čj€fīėÆžæ ,„fĶ&íܡ-Ē˜s;#Ė9A:€WŗŊŦĢ)€WÆOšŒIP™;ÜĪ'ÎŖ8 ÉÁ´€1.eÎíŒĩ-~  Ų!@õIĻĀÁlû ¨°:âF'€ĶFd)ZrˇKšļ0į*‡`d)ņaŠā ēŖJz„ĩĀAœU‚nŠØxĀĀ´€aĖœĶ]-™sĘ!ą†b8ōĐŒĖÚxÂEî!€Wŗ!ĒÜĨ}ŒĖ™fH P$¯€iФūŪč×;1aÎŧōĐԆú“ˇP Uš1ĒÜ€WŖU ÛãyÉÖS ÚüĶ€Æ Đ´‡ v΋ûyĻņQŲÕ,ëÚxüuã/æõED Ŋe2Ü^֞¨b<ЈĒC:€úی&TĢ) €QŅÄęļø÷nŠ *ĢâųËũņÜ’Á´Č fÎi ¸;ĒܕÚBāLxŒsv˙žr˙ôđâC—˙s[GĮäĄ=•Û˙5 -ŲY>÷Ũęcų.Q–ÚĶ‚я-?â¨[ûûû7æ =ĩ64tÖ '¯ų—Áe‡=\ū Īō]Žsœ2gîŪ™ũsëM€Ôž.>ôįG{Â7]~ÄĻ´§hņâÅg ŋę+K]öSæP(sžDy5w`ūOfõĪũģɞCĩ/YvØļžŊ/Ŧ\ĩúĀæūyé3×ŨTúę”îzhWéū‡÷ ÔpŨņũGK7įßKŸúüWJ7m.ÍčíÛ ™‹¤:dáÂ?,O€~yؑCû/¸đbÚĘl{RŨ|˙Ÿ~ŦtÜĒ“Ÿ6­MËļ.‰2éÛÖŅqVšS|¤ujë/ŊũWnOí÷žūg>C{Byօ—\ēŋĢģ{wŖͅîîįvu÷ŧČ åYøŗKK}3g=—€‡îEK–=˛ūŦsKĖĄPžõߡ|čᖖ–ĮÆÛGŨ3Ŗ÷ŲoŋBE…0ëæĖ&jā—Ūqæ9ų>P!tÁ.ųE|W¨!Ë"zgÎēš<`{‘īÆŦĢDÖ5¤=ĩļĩŨpÆúß}ŠīÅ\č훹ŗËöΖšÁ e5kÎÜ'ĸnÜą|ÅŅ÷2‡BEŅĻ÷ũN™2åģcíŖ•‹čoŋö- Fī|īJƒKģŋĖÂŃī?rhåūīũx7ß*ŒŽ:ö„ŸE•mŪSeZ{ûģ–qÔs´'T$tĘŠĪļL›vaē¨ķ.>ôqÚ*’N9í /ĪXđŅ´§AåŸd…Š6‡X°čîFĖĄŽ?ų”yĖņčŖP‘tؑG˙į˜æPSĻLųÚĻ÷}đ1 IēĀ+Ųŧ–#¤Ŧ Ũˆv|ųžT°| ßŪ×Ú:퉔wkíîîŲE{BEĖ_2ŊŖķé´ÛS[ÛôŸíøōÍûøPŅÚSwOīķiī^9eęԛ˙čĸKžå;@E›CēėđįãËSC͘˙c>T¸>ę–ģ^liiųeŊ}ÔĘöŽÎ]¸Ö¨ˆúâ?}ŗÔÕŨũxš=ĖâåËߎ”(TDqÔ1?×.‘iîHyü‰¯}Ž˛GE”ęvÚíéđ#áÆ,*h”ęÚÚ!2Åö´RcLæP¨¨s¨ŽÎÎ_Ļ9‡zŨëßø''¯9ōG…ÔāōÃ\ī˜oû›Ī:÷_)˙•Ō÷~ŧ;ˇßA¸îßüÃ×Kûä5ĨO\ķĨ†Ÿ›•ˇĘZúoįoiHyįåûŨCŖ!Œ:„0ę0ęFFB…7ęîzhWiũYį–ʇIÔĸÁĨĨ/ūĶ7sWū2äŽ9áă“!ĻĪÔčD™ ÕĘûˆŖ-}ãžNú9|õŽJ§­{siĮ—oÁ¨CŖŖaÔaÔ!„Q‡FeŽ0ęmÔ)šęä5§ŊbĩMoū{Ũ™ŋ?lĐŲã]]š2ëĖx Ë% FŨŸŊū•r=gŪpYëŧtNöøĒמnRĪáú[īūŽõ^uaÔaÔ!Œ:Œ:„0ę¨c…0ę2`Ô}dÛ_ŋbuė Ĩšįũ_K3}d]^–IĘčĘBä\’ÅĻsë™Ņ;ÍčMSEĶYyßüŸ´s9gīƒQ‡FF¨èCŖ!Œ:æPŖ.FŨā˛Ã‡ÍšŠ­­U!‹B“>sŨM¯DdÉ “´„rãĻÍÃĻXßŦŲĨ×ŋéw‡ sÕĩ7?_•"ÉdXũíמuĐsŪ˙§~ŽŽ§Č3ŗ|m*mŊüĒW–k*ZNëœõ?=_Įļc(BMīa‚úŋŽë¯įø%˛ö˜–œ*GŸÎ¯Úg‘ĄĻã,?â¨á÷9ūÄ×›]ūÜG*ķß<ũˇ_‰RTžēĐ@ģā‹‡ RËˇß÷J9[˜dĒÚ˙.ŊōŗÏŨūo?~}¯:ž>‡–˙^rŲ•}^oęwÁ§Īˇųƒ>üųT~:–ōįŨņũG_e†ZŲËđõåĄīU&¤ÎEߑĘŌĘŖ!Œ:„0ę(s„Q‡Q‡FBudę˜YķkĪ¨ú<™sö<6aD–™}^2ˆŧņ'Ŗ()›Ė6o˛ųĨŠļ4SŌæ2ytÜjšŨ,2Ė/!5é¸Õ–žĘĶc2“’>KI˜”ĪOŸÃ^;Ú˛UŨ×iŠņ˙âãÃægŌķm ˛ 5˙¸^cĮ‘ĄFäÉ “YgË|ķŸ7”™tJŸĪ›u#}˙*KEh†ë1Œ:„0ę¨ŖĖFFBuaÔRÎ93PÎ}ĮųUŸ'Ã͞'“*4ęd)úMĪST™=Žh4=W›™n2eę13„ŧæŖ÷d)ŠLæ voU”™Œ-EmY$žvq MD}.3Ŧdėé\edfÔIŠĶį!ĨsĩĮíũ|™éüõ9ô\oŪfԍd†É\ĶųøŨje°Ų˙}tŸÎUŲ†ūÜT.zLĮŅ÷ ãZĄ>Ÿ7ųôģ™œŪPUyë\u,‹P´2öFž}˙zĖo’!ŗNį¤sđyĮU‡Q‡FBuaÔaÔ!Œ:Œ:„ kÔyŗí=ī˙PMFEŪų×Ú˛R“\2įBķĮGÉü˛ĮmŖŠ¤eļ^2}|¤ž 8B>jn¤uŖu~)ŽßôÁŒ,oDyĶLįeŅkĩl!ĖvĨM2ėd™yŠ%ŽaTœLzĄiŠãęņjQzÕrÔYšiĢž–°†å˛áqŊŠįëĀXsīaÔ!„Q‡FBuuŖŖĄÂuʏfæ‰ĪŲ6ŌķĖ i3o|ɔ‘aã $ËmæM*™VĄQæD3ƒJf•ōŧ™AžÛxŒ:í•ôũų…įfŸ§Ū[õžuÕį‡sãųåž–w˟ļüÕ"âÍĻE3˙d°%-Cũũˇm:hŖjߟ-V}O’_Nlĸũ-ķ2ÉÔõ9õôũbÔ!„Q‡FF¨èCŖ!Œē*R˜CZYmG×p™dŊFL"û[į™$UŖaĘegį+#I›Q(šĪōÖM„Q7ZÔŲx:™2ĩÔøSŸ˙ʈˑUnö¸ĘĮGōŲ˛W•Ax|EGZ^ģ0o ™ŸŖu*įjߕ~öz•_’Qgæ+FBuaÔaÔ!Œ:Œ:„0ę¨ĢAZļXm ĢE{ÉÄ w†õFr™ų×(:Īž/ķO˙O2idú\lŖafN)ĒL;ŠÚãiu~éĢ_&+ĖLÄŅŒ:•‹Ī-įĨĨŧIFĘÉōÄyãĶrÃų%ĩ–Oe-ƒĖį‡ĶĻ#u–Û/<ˇ¤čFŒē‚J_Öhōîšūö;Âh vø<„QW$éBĒúmT„Q‡Qw°”wŖZ˙aģƒMö9čŽĻŊ§4"Œēɐę´ōŧhŽÉ†–É(ē!iĐĢúhu3)GMĩȉZۘ¤Á}8ÉʛlŲ’XmaQZ#uv.2ė’Î#ɨĶšé:nFĶx:ņĻeĻú[†™ˆĩ,}UœßœC¯W?ŽëŒßĩÕĸ MfĐŲ÷Ĩ%­ž zCԛoŠÜŗĮ-ŠĪ6ĩ3[>ĢkŸŊ‡CíQzĘÁg›‰`ÔXÕļUˇö ÍWú‘1¨+‚Ŧ3kĮ‹0ęŠnÔUÛ^Ū¤ĸ%؝Ė›Nãx ŒēZ$CÎįŽņŌ€Úįˆ Ë~­ ‹Ž.—KŗIí.—G%MÚ,ĒAQá’.Œēâuj~įÆPĒÍxCF;ÉJ“÷jÉęĮŖj“}Œē‰7ę,8Ā›uÕęŋ7…Â]_mųŦī—|ęn,ųzã )Ÿo¤9ž?†"ž”§Nīcįîû5˙\oâ×¨K:ļ-ÅĩåĻŖuĒßXM:FxŖČož‘´S¯næŲq5^y(ŲØAÆĸ}‡>rĪ$ŗT† T9ûsõ|u7ęÔ°Ģ­ƨèèèèèĢÅDĐ ÂúŽpB1™mŒ:ŒēF˜ŌG{Âđ8(Ü-Îvƒɨŗģūĩ$ģöīĢģŖųvĻ˙å5˛.iŌĻ6ņèkŖÎįáR}PŨqį'Š>b§¤čoāčēĄ2ņQV*Ÿ‰Ž`Į¨Kרŗ¨7-ƒ o)ÚM7=CĶțXŠü˛ųēqaĒ^¯h-ŋ„|EÕųcëīĐ đ‘–2įüëÕOŠīĶķeDŲądjyãĐL<;žĘSÛOzOEËŲãļ+­}Ŋ§ŪO˙Sä Ę¯žūUu<Šŧ՞TNÕn øÍ"ü9ų•:ßnõ2CÃvę G=Įĸ"un*/ Ճ0ōÜĘÆōJ2ûėq˙|ũæ¸Ã¨Ë¸Q7ÖŖŖŖaÔaÔ%™q@ų˙´“FF]֍:EĢØ`Yø0zÅ/3Ō¤e´ˆë[ę5ęÂú-cÎįŊņéIŠž% ŖŖ.[Ÿß"|4!õƁڔMøÕÖōžÜģVésšIŠk‹œUųøIžĖŒē|uūģUųKļ2IIŅf2‚j1aėøcM- ķĒõõzŪDÉ2Âd:É ËĮúWpc)īŅÎSĪÕõÉĖĐŅÚo-åle™t]ĶĩoĸË/MOŖ.ŖNy4 ÔĪŅŒ:U$uęPՙĘu–ËÛL9F0ęŠkÔÉpP[°|:ŠĒĐNƒKŨõ´;<ēx[Čŗ;ˇ6ĸž­¨'i‰ îÔh"ĻšÚ”îđčxļUˇŋp+÷ƒÚ¨ÎWį¤ļ§I^ŗ h1ę˛cÔŲ%)Wˆ¯˙Ē§Ē˙j?Iõ_ƒMõ'ÖõŊÖ×û$ŖΖĖé}ÕfÍ8ŅOĩ ˛ôŪjWj'JŪöQ2ctGTį§ģĪz‡ÖöŧYŖĪĒ˛°Á›~ęųI‰1ęōgÔųņN5ĶHã-Q=ļúĻzjõÅ&ēúŨī˜į˙W¯Q')_^Ōdږ×Ē~ĢŽĢ …mĮ÷ij2úŦOĶëtė¤sRÎ$ŊFį¯6¤~.é3(2@Į7ƒE†ƒÚTåĄ÷ļļjmÚڒ%-MMĢŽĨļŠōÔy'5­üÕ?ë{´įęw=.%“Ô‡ë:~Úc×f7ę,REu%4´ÜKíaëåWŊĘ ¯ĩ>øú>RôéZŽ÷V=Öyéũí9ú]Qĩj֞“v‘ÔëÕNô<Ŗ"eÂąáh†ĩ˜ņ}Ū_įļ+ÛéĶ› ú;ÉÄQß&cÃú;õģ~‰ßXŽ-u“¯‘v}-˛ÔŪ}˙ žFõ]ũoĩÍ5Æ+Ŋ‡Ú„ßD"iã< ŒēԍēZ—žúuŲĄęqļF]V:{ĖO"|.ŸČÔįL° ģU>d:”7+Ôé&åĢđīm7 Z“ÎÉžb”cÔĨmÔųAž–IØŋÖú¯‰CĩūDuŨÚThÔŠ˛ļ I‘-#đ}Y’|~1™p–īĨZÛŗĪĢĪ”ÔîíųEĪ§Ô FMú9ֈ[úšTOü˛ØzŒ:Õ=QįëēßE0\Ōã'ÔŪ­]„cŋP>ÁĩÎÕڐŒŊ”ę×ŧŠ.}­–—¯–ŧ}aNĄ¤ļ+#Â_7|žŊ֖…Mf$0F]˛,ąŊÕõ2…|ŨŽ'cXŦž‡sŸ¤å×ū1ŋ4ÎęËHmÁ›yŪ<ĮŒŖmá#æęŲĐA}XĩœvĒßŪ$Ŧ–ĢË?æēZ¯-uu“™Ë˛Ú¸+#NÆ ;4ÃÍXŒēuÕä„ĩuö˜gÖųT !Œē<u–‹H…ęĩxY>ŋ}¸Õ}ũ´É‹Ũuõƒ$34$o6ؒ&ŊÆ­ÚāVīīßˌžÉNčQ‡Q§ún;Äéw?ȡú§¤ÕĢ§ēģoĪĩŨÉ$ũnõ_ĪĩīBÁ?ĻÉœå — ibb‹Ōrkgē+žˇÚĸ‡ô˜ŽåķēØ ØįTŌquMĐsís* Ŗ.ßF]ĩ]ä4)U=e“Õ$ŖN˙ˇúŽ:§ŋGZ˛íۘęļEˆ)bČ×Gßwø>Éú)MÜí}5ųOęĶô¸Î]íČڅڀŸ?ŽŪOĮÔs}ž"›Đ+ŌĀL›Ė觌zÉįō MŊŸī?ÕÆŦŊé=}y(ę\´ŪĐđÆg˜Č\ŠÚŠnšŲõĀŒģ6r)qŗuēąá܇›&„yēę­c1ęlâ¯ēŖą–ęŽ¯'zžú1ĩĩKģAĢįéqŗ)ŠNm@mĖL†Ņ–Îų›Kc‰LT?¤6Š6ĨžĖú%Ŋ¯•Ŗ/++_õ—IF]ŌĩÅ?×_[0ę&_Ēëļ3˛~oĻąŧŽęg4^SûTŲëƕÚûdhé­ļĨzŪŦŠV0ęrjÔébm‘ßöY˛ŽĢw'1ę0ę&è ;㤉œß><\äۗ~ˇA•Ŋ^wNĢåŅōĒ°Úyųɚdۏג ŖŖnxŖÎŪĶ}žj`‡QWLŖÎ×=/ģŪWÛLbŦ9ęĒEąČ öË7-âE&YRũÕd=<ĮˇS™Íá"‹ŪĶq}¤¨ĪŖĨņ^ØVÔ™Y0R¤‚īWĢå¨ŗįĒ†KøĖāņŸÛ›<áûÚ&•‡•Ÿ™ay`ÔĨ›—Mæl5ÃÎ_—íûĒĩ>ŒÕ¨ wUļ÷ ķåiܧ>JĪˇ%ĸv o¤øö5ŌŽ­c1ę|”U¸üĪŸõ‘fXĢ}ļŦŪŽ-jŖ][0ę“Ā¨įĀSNtŌaßņÔbÔųŠ.Tv×Wōw){Œē"uá+i)yĩDŧ2!4X’Šæ#"üqũ@+4ų’rrųI[RÛĢwÉFF]ŊFßõÕvāR=ķ?Yũ×äŨīæëŋ7šũĸŅrøx%å×Ō{ëxIīíŖyĒ-IôQfXļüD&Š„Q—oŖÎnB†c˜´:MēU¯mL&sB‘gĄĄäŖą}ôË?ÍÔ¨v.>Į¤Ũ|ĩ }xCČrŲ9…K_ÃvĻHoôÕcԙų’4žôQxöųFJŋâ¯3ŠōË^}NfŒēÆIŅgŠSÔŖObĢvŦ>čfO-õaŦF]˜Â ŧÁSM>Ÿ•o‹ūĻ’Em'ÉĸģĨZŖ„t¨fĘ,ôqž‡7…“°Æq"IDATÆĄv ûģ¤k FBxuÉQį¤ē€û ›ÉoÌ0ęšŅ¨ķyˇ4ųĶ Lą0ĸh$ŖNFƒ7ę4š­í1xĀ¨K;G]’ĒÕä'Õ˙z:kŖ~ɝ-sĩ˙éŊõ^ŠD '\#u>ą}^ŋÛgĩļ7ŌŽmuŲ7ę|TKĩLI×ė‰6ęėÆL˜Ÿ'4•|=¯V'G3ę|ŋfËŲĖ¨K2Ŋd¤„Æ´%ŗOĘå—š×cÔY_i‘{^jĪÕnbUëÚĩGŸÉņ#E9aÔMŽôŠžĢžÉD ˙˙WW}ū “ÉuĩևjFE‹V3ęĒ]F[ųcTk‹–.%IŪlLĘg§k>ģ"¸-ęĪuan?ŋZCe-˛ŖŽžk FBxu1ęFēØy’‚Q‡QWQgíF?á ۘ:á$)iBdƒO›@…Kõh{uY2ęl9¨L­‘ęŋūWméĢ&.zĖĸü„^E Xä…ī‹lŌĻ˙ų%H֞’–žz3!œ4Ųį5ã ŧÕ,m¯Œ:ŋĖį3ķŌd;-ŖΎí“ÅûĨŪ….ģNŠČŠļôÕįū˛ås¤hÔ0ŠĪ6Ŗđ‘Mz?;gOKũĻv ¨Į¨ŗĨw2 ÂĪcf‹_~7ÚX×L ĪĸšúŖ™:íĨ¨įđ˙ūF‰íļhßY­õÁę{x|KŅSĢQįß×GÛÉ`TUģÕ8Î/÷}žú™0Úę3ė$õƒa÷yäÂ]ūųfV[úęÛ ]Ķęšļ`Ô!„'Q—ŖÎwŠ~[uŨÍŅ¤GÁDo•LŖĀ¨Ë›QgƒĩÜipjQ9ū¸6pTûŅGwBÃ|E6 2ŗÁīrŠAĄŪG…æ9FF]#Œ:K:Ö["įÍ.Ģ˙úŸEWøÍĀNZ n† žkšė|N!›0ųĘ};ˇÜŽf°čjƒ~é•}^ŋ‘‹E4čŗŲNÕŒŒēüuánÅēŽËS}“y ¨1_7F3ęėX2„4YÉÔi×W_÷ežY›ōģL* É>ƒŪĪ"YÃ>Í"ķô<“ß Âúo:¨™ŲāķĻjYģoCáN>ĘÉ&÷IF7>õģÃGUÉŅ1tÎūÜ|„áhc]7Ü]Ŋ‘cÕf_úęķŌŠ^ÉāUŊ×÷î#3­O¨ˇ>XĨŽeõOmØę@­F7ŧÔūÕt<›ŠŠ]ûŨĪ}ūn[Ģķm§TßÕŽÔ÷¨ßT=ĩžSįoQ ~Ŗ}NÛáÕoú Ÿfú›ŋvŊĐΛęvMKēļč8I׌ēüIuK}Ž—Ė^}˙ē7ëÍ˙ZLu< ŒēĖuęŦŗĶ[“.ŋ,¨YwHÁ¨Ã¨ŗAŽ_’ š–j#IƒDߞŧü@ÖڔĪ¤ã)ōÁjúÛjŖŽ‘FOd­ēÖŸKŅ›rĄ|’dÔi0iõß&~bĨ÷QD„žcīáŖ.dĖ%å×2ķĐ^Ŋ—M~t,ĩ=ËsĨŋąs$FŨÄKuÂO\GÚÍځŨÜIęĶt\ŋä6”úE‹´ķí×ōYųöã#œŒ:op„yÂ,Ē.IęW}„S-c]_†:įŅŒŒēɓž÷¤k¯Wxķc´úāŖŊÔFÂT!Ö'ÔcÔŠžû%ņĄ|{÷ĮŅ5ġÍZvIU}ô¯I’ä&EÕUë?Ã~Éß,ŗ˛Đs’Žēø|táĩe¤|{uųĶ%Iãĩ‘–jMj{jŖa< ŒēTLŠ–HMhô\ŋŦN¯KĘ}ĨˆŸ$ÕŠ•īŖŖnŦŌ„$Ė­h…KL“Ú“îH†9H4¸ķŅęôôˇ7Ių=4h•Ņ Į5PĶŦ$c–ŲŨ\?ˆ w+èè›(ir`õ6)§PŌäFQžūk¤;ķv_§5Ų÷“û¤ūDm+Ší$›[vķHį vŦČ({ž7´5yQô‚ĩ=Eh$m&a†Œžë'FšĖTÛyŖ.FEJ*7–B3“Wßø}û>ĀO¤U}tP¸ŧģZKĘũ¤ĮT×ô™AĒ † Ģī6'Ōūæ“úƒGį.qU¨Ÿķ!éw- ŖÔFÂņ =××Ɠáōqopč|ėķëĩúŧA ī@}i¸¯–ąŽ_*Øčˆ 6“¨Ŧđyę뮙\Šč ëd=õAõ×{ĒējíĖ×Ájķ,Ŋ‡7št<[–ëĪOuŪˇŊ&i#˜‘Ž=ęŋÂ|ęUĢEG…ÆēūNęĢÕ÷ų›Āē¨LĘŖWëĩŖ.ßFžSûū}šŊYt“Î>7FF]ádšI˛œˇŖŖŽ‘>ĩ‘ŠeČÉ°‰WRTRRē›ütuyŽ˙IũI¸ķŪX—0Œv,‡Š8ĐDÅO¤ü2Ģ0Iˇ?v3ĩŊf2ę„úŽĮēR@õ*ēbīS-J,Œ¯Ĩ}x“°–Īoí},I歜Ģú\#ũŋųhÛF› u¯ŽģõĖ%j­>âDg-õģÖ6SË1j-kĩ<_åWë5i´k F]ž:_O-Č –(đĸh¤Åđ$0ę¨krųeV2Ôi(:Î×OĘ ŖMŧü’&ãj{Š8°č Ũe1€Q—ŖŽh)jŲØĸh7“q¤h$‹RRģnt."Œ:„0ę˛fÔYú‡ji~ai­ŠU¤§Oą`ĢķÔĪčq-AWDĻĸÁ-:OķE|úüŠŠ~õ›éĒŽ!É Öyčų’~—)­ąšV^čø:§¤ČUŨhUŧ^gé‚üM“×÷Ē÷ô‘„Šîĩ%é:W•AXfúœzĸlõĢ˛Ņą’vpƨCŖŖ.GōK\“rŒ4SžŒ:”Ļ|rņ$i‰ å„Q‡QWŒh‰,l"Q‡F]–:Ĩvs+ꆇ_rĘ6ņ}Oaģ&ûÕ Ą´dÜ"ŧũŧČįJ4É\KĘqé—Ŗ”S›Ų˛ņ¤˙[ĘŸ–ÁK†_e7}Ã2 WJaÔ!„Q‡Q—CŠSP‡ ÎG›~ęīZō!Œ:4véΊ–zX^JŨ!ÖĶ<ß Å¨CĄ´„I}J˜WĢdm[yéÂčŒ:„0ęšŲ¨Ķ ]ĩëĢú Ÿ§Î 8EĄÉô’áfų°ũf\>ōÎtKéØvsÄv —ąeK´ũÎÍvloÔ)§ĸĨPņšŒ•ßRĮĐq;͝ė<,ŖåtU´íFŽ´ŠāĶëŊy¨ĪĄų˜ךø œôū~ŗ)ŸįÔ¯ÎĐyéüķŪ×bÔ!„Q‡FBuu¨Š„Q‡F]ŒējŌrÔđu>?ŖmBb›…F]ĩŠšŗČ9ũßįãļDŪ¨ķŅzZŪjûMÁĖ(ŗīEįfĪĶŌU{žĖ:3"Í<Ŧ–ŖN›,Ų_ŋ¤V†Ļ™}ö9ŧQW”ÜÅuaÔ!„Q‡FF¨èCŖ.eŖÎīúĒeĻæ{SŽ8å_“)įŖî” fԅģ{›ŠĻU Šx SØÎÆŪ¨ŗe¨áyûÍMBŖÎīđ­Īdųî$ÛÅØĸĢuvL=ßŋŪīšlfaøūuaÔaÔ!„Q‡FBuĖĄFF]ŨF]-ģû|oúŠ%ŦZÚiĻU’Q—”ÕGÄi Š ;-ˇ MšZŒ:ÜĐ(ķËY•OΌH/ÛŅv4ŖNfbŌë%KOdĪUy`Ô!„Q‡Q‡FBuaÔ1‡Buu“fÔYn9}n[î)Yžļ$ŖNģŋúchI¨Ŋ§™d’ßĐaĸŒ:‹öŧ0/i¸4ĩšQ§üzļ„×?_ųëÂ%ŊöūEÚŦ Ŗ!Œ:„0ę¨èCuuaÔeШķF˜-‰U4™-]M2ęüĄ!Ļ%´ö¸ĸęÆ˛ôu$ŖNį¨H:=Ļ (lã [Ģfĩģmx^Ú4BFd¸É„åšĶqĩô׎keQG'ƒ0ę0ę¨CŖ!Œ:æPaÔĨbÔų%Ģˏ8jxĮS™tfÔÉ´ͨ“ŠĨ|xļœTšđ”?ÎįŠģä˛+'Ĩ“´S­=WQqÚÖŪKīëŖâ,oé3×Ũ4|ž~“}n;ŋF ¨èCŖ!Œ:„0ę0ę¨ŗ´ĶĒåYķ;šV“L--5ŖKyędŪÉPĶ1´ÖĸÖ´‰„ĪįĨ%Š2ÉüFW]{ã°ņ§×(ēNĪĶî¯a¸đŧũqĩĖV…Xč8ūũtūÚÍÖoD!É 4ŗNĪąČ>E×éœ,?Ÿmža†bøūár_Œ:„0ę0ę¨CŖ!Œ:æPŖŖnR$ãJxᎰõJFYh–MĻ”—Nįæ– ÍȑžŖ˙ĨyÎuaÔQîŖŖ!Œ:„0ę¨èCŖ!Œ:„0ę¨èCuuaÔ!„Q‡FF¨èCŖ!Œ:„0ę0ę“'Á  ~Ũ[ÛôĮŌędfôöíŖÜQ‘ÛSšF]gW× ”;*r{JĶ¨+÷…ĪRî¨Čí)eŖŽ9*|›j™:õá´TīĖY(wTäöT¯Qˇ§ü""€PaõÕ;P'ķã´:™ō{o2P„˛Üž:ģē~žZ{jiŲO{BEnO}ŗf?•V{š2eĘ>Ú*r{šwȂįS4ę4‡ĸėQauÃ×ŋģ¯\Īd…ĐÄôQĶ§ˇ×e|?ô÷_ŋûg*ĒŽųģ¯Ē“š=ÍĨzjˆ”=*ĸ>sŨMĨ}}÷Ĩ՞ē{zž =Ą"ˇ§‹–ėLĢ=•ˆ?§=Ą"ˇ§å+ŽŪ•ĸQ÷Ѝ˙į{{){TT]ņŲëĩ"éÖ´T˙œyĪĶGĄ"÷QmmíwÕĶ&Ž>ë­įŨKáĄĸęÜw\đŖr=ŋ,­NféáGŪķąO^CŲŖBęīŊ腖Ö֏§¸ â歗_EŲŖBę‚ /.-\rčgSŒPŊūŖ˙_,E…mO'žrę-)uWŋíŨ›HŲŖĸę”ĩ¯˙Ašž8­uä1Į?Ā Uī¸āÂ'ëõ$ÖôôĖø)…‡ŠĒ™ŗûbē&­NæE‹.Xû†õäX@…Ô‚EKžJŗ=•ŲxĘÚ3ū‹˛GEÔŌÃVėMģ={ÂIŦĸ@…ÔáGsāØUĢĪLą=­é›9ëQĘQ÷ė|ŧ4mZÛ3åzž0­ĩę”S/>mŨ›)TH͘ŋĢî1ߔ)SžûÁŋ¸œŽ/dû˙ŨS---DéŌŪÛ7ķŋŽŋõ.žT(}ęķ_)ĩwtũ íöÔŅŲõ$í ą=Å7’RmO­­Ķv]ëˇŲôŽ=Í_´äɔÛS4eęÔ{.žô“Oķ ĸéíī~ߊÂNģšŲ?g/c>T4]ų7˙â´imßKŖ’cΚpT´;A3gÍVn… iÜæ,øčI§œĘDFJ˜=0áShO-ĶĻ]xÜĒÕ{H0ŒŠÔ?-Yē|o[GĮYQúl9üČcã{@EjOK[ą˙¨ĄãŪՀö4Ô6ŊÜĨPúâ?}ĶĸéŌnPG­ŧü¤SÖ˛Š*T5gîĀ¯Æ3‡ÚۈŗÖGÍ_¸ø?™CĄĸč7O㎖––/;jA;VĘE§PQž]ëSĪX˙Xy"ō…P7pāÖßÛ7k×ģ˙øƒûš+„ōܞÖũÎŲĪ´wtŪÛčöÔŅŲõ3Ė:”gŠ/Xö[_ę›9ëÁFˇ§iĶÚ~øö÷üņ“|/(ĪíéĖs6–,úQƒÛSÔĶĶķ!íXÉ å} yĪŒŪÚ::ū j,ũŗûį=)ŗŽ9ĘķęõoúŨ'ÛÚÚī™¨>j}gw÷ž?|×{÷sgåM;ž|KiūâÁ_Æ9ēŖÆĶ?kÎÜ˙wâoüæËä[@ylO‡.?ü™ŽŽÎĖJ{RdŨĘW“å˛=~äĐŪ™ŗúoÉJ{jmvįĐqĢȉrŲžV ­ÜˇhpéiOŅôÎÎ3ģēg<ˇqĶæs(”'iéöīüūÆũ3zûv—Ģōę(ôĖ_øƒ“~ãÔũôQ(}ÔâC—=ÕÚÖvÃD÷Qsæ|ŽŖŗķĨ7ũŪ[\zåg‡0Ž6ĘĸS­ílûë’&Ķ§ˇ+1÷yQļhí9ķ#åÎoĪŠ¯ãË[/ŋŠö„2ߞd†uvuũ<‹íIŅß]ŨŨģלö†įiO(íé5Ģ×ė-÷wvöŧ;kíI9ëÚ;:ŸxíëN†ö„ōО”xæėūŨŗįÎũŖ({ Ė_ŧäKåūķe-Ée…˛ÜžtÍ_û†õēē{^™ }ŗf˙CÚSYįÍ_đžŊÚŨyŋš|—(íŠk¸=•ĮNO,X¸ä–ŧ´§}}7•ûĨ'iO(K’éĨöÔ7s֓K–.ģ­ģwÖ9oOÃs¨‹–|`éō#žW>īg˜CĄŦĩ'™ŨË?ęۇ,ZtA ēWõQŗfÍÚ´xéōo{ĖĄPįPåųČ¯z{ûnČÁ˜Fã˙‡ÕĘũéŸDIENDŽB`‚avocado-113.0/docs/source/blueprints/template.rst000066400000000000000000000031441513561306000221200ustar00rootroot00000000000000:orphan: BP??? ##### .. Please refer to BP000 for detailed instructions on how to write a BluePrint (and remove this comment and :orphan: tag from your final document). :Number: BP??? :Title: :Author: Your Name :Discussions-To: avocado-devel@redhat.com :Reviewers: :Created: :Type: :Status: .. contents:: Table of Contents TL;DR ***** Short summary of the contents of this blueprint. Motivation ********** Why is this needed? Current problems and benefits of the proposed solution. Specification ************* How is this going to be implemented? Can contain pseudo-code or architectural descriptions. Topic 1 ======= Topic 1 description. Sub-topic 1 ----------- Sub-topic 1 description. Topic 2 ======= Topic 2 description. Sub-topic 2 ------------ Sub-topic 1 description. Backwards Compatibility *********************** How will this break users of previous versions? Users with tests using older APIs? External plugins such as Avocado-VT? Security Implications ********************* Do you have any concerns about security with your proposed solution and what are they? If there's functionality that is insecure but highly convenient, consider how to make it "opt-in", disabled by default. How to Teach This ***************** What is the best way to inform our devs and users about your new feature/solution? Consider both "how-to" and reference style documentation, and if appropriate, examples (under ``examples/``) using the feature. Related Issues ************** Here a list of all issues related to this blueprint: #. References ********** [1] - Reference 1 [2] - Reference 2 avocado-113.0/docs/source/conf.py000066400000000000000000000204201513561306000166570ustar00rootroot00000000000000# -*- coding: utf-8 -*- """This is used as config file to generate Avocado's documentation.""" import errno import importlib import os import sys from avocado.core import parameters from avocado.core.varianter import Varianter from avocado.utils import genio # pylint: disable=C0413 from avocado.utils import path # pylint: disable=C0413 from avocado.utils import process # pylint: disable=C0413 # If extensions (or modules to document with autodoc) are in another directory, # add these directories to sys.path here. If the directory is relative to the # documentation root, use os.path.abspath to make it absolute, like shown here. ROOT_PATH = os.path.abspath(os.path.join("..", "..")) sys.path.insert(0, ROOT_PATH) # # Auto generate API documentation # API_SOURCE_DIR = os.path.join(ROOT_PATH, "avocado") BASE_API_OUTPUT_DIR = os.path.join(ROOT_PATH, "docs", "source", "api") try: APIDOC = path.find_command("sphinx-apidoc") APIDOC_TEMPLATE = APIDOC + " -o %(output_dir)s %(API_SOURCE_DIR)s %(exclude_dirs)s" except path.CmdNotFoundError: APIDOC = False def generate_reference(): avocado = os.path.join(API_SOURCE_DIR, "__main__.py") result = process.run(f"{sys.executable} {avocado} config reference") reference_path = os.path.join( ROOT_PATH, "docs", "source", "config", "reference.rst" ) stdout_text = result.stdout_text.replace("['/run/*']", r"['/run/\*']") with open(reference_path, "w", encoding="utf-8") as reference: reference.write(stdout_text) def generate_vmimage_distro(): yaml_path = [ os.path.join( ROOT_PATH, "selftests", "vmimage", "variants", "vmimage.py.data", "variants.yml", ) ] reference_dir_path = os.path.join( ROOT_PATH, "docs", "source", "guides", "writer", "libs", "data", "vmimage" ) reference_path = os.path.join(reference_dir_path, "supported_images.csv") config = {"yaml_to_mux.files": yaml_path, "yaml_to_mux.inject": []} varianter = Varianter() varianter.parse(config) try: os.makedirs(reference_dir_path) except FileExistsError: pass with open(reference_path, "w", encoding="utf-8") as reference: reference.write("Provider, Version, Architecture\n") for v in varianter.itertests(): vmimage_params = parameters.AvocadoParams(v["variant"], ["/run/*"]) vmimage_name = vmimage_params.get("name") vmimage_version = vmimage_params.get("version") vmimage_arch = vmimage_params.get("arch", path="*/architectures/*") distro_arch_path = f"/run/distro/{vmimage_name}/{vmimage_arch}/*" vmimage_arch = vmimage_params.get( "arch", path=distro_arch_path, default=vmimage_arch ) reference.write( f"{str(vmimage_name)},{str(vmimage_version)}," f"{str(vmimage_arch)}\n" ) generate_reference() generate_vmimage_distro() # Documentation sections. Key is the name of the section, followed by: # Second level module name (after avocado), Module description, # Output directory, List of directory to exclude from API generation, # list of (duplicated) generated reST files to remove (and avoid warnings) # References tag API_SECTIONS = { "Test APIs": ( None, genio.read_file("api/headers/test"), "test", ("core", "utils", "plugins"), ("modules.rst",), ".. _tests-api-reference:\n", ), "Utilities APIs": ( "utils", genio.read_file("api/headers/utils"), "utils", ("core", "plugins", "utils/external"), ("avocado.rst", "modules.rst"), "", ), "Internal (Core) APIs": ( "core", genio.read_file("api/headers/core"), "core", ("utils", "plugins"), ("avocado.rst", "modules.rst"), "", ), "Extension (plugin) APIs": ( "plugins", genio.read_file("api/headers/plugins"), "plugins", ("core", "utils"), ("avocado.rst", "modules.rst"), "", ), } # clean up all previous rst files. RTD is known to keep them from previous runs process.run(f"find {BASE_API_OUTPUT_DIR} -name '*.rst' -delete") for section, params in API_SECTIONS.items(): output_dir = os.path.join(BASE_API_OUTPUT_DIR, params[2]) exclude_dirs = [os.path.join(API_SOURCE_DIR, d) for d in params[3]] exclude_dirs = " ".join(exclude_dirs) files_to_remove = [ os.path.join(BASE_API_OUTPUT_DIR, output_dir, d) for d in params[4] ] # generate all rst files if APIDOC: cmd = APIDOC_TEMPLATE % locals() process.run(cmd) # remove unnecessary ones for f in files_to_remove: os.unlink(f) # rewrite first lines of main rst file for this section second_level_module_name = params[0] if second_level_module_name is None: main_rst = os.path.join(output_dir, "avocado.rst") else: main_rst = os.path.join(output_dir, f"avocado.{second_level_module_name}.rst") if not APIDOC: main_rst_content = [] try: os.makedirs(os.path.dirname(main_rst)) except OSError as details: if details.errno != errno.EEXIST: raise else: with open(main_rst, encoding="utf-8") as main_rst_file: main_rst_content = main_rst_file.readlines() new_main_rst_content = [params[5], section, "=" * len(section), "", params[1], ""] with open(main_rst, "w", encoding="utf-8") as new_main_rst: new_main_rst.write("\n".join(new_main_rst_content)) new_main_rst.write("".join(main_rst_content[2:])) # Generate optional-plugins OPTIONAL_PLUGINS_PATH = os.path.join(ROOT_PATH, "optional_plugins") API_OPTIONAL_PLUGINS_PATH = os.path.join(BASE_API_OUTPUT_DIR, "optional-plugins") if not os.path.exists(API_OPTIONAL_PLUGINS_PATH): os.makedirs(API_OPTIONAL_PLUGINS_PATH) with open( os.path.join(API_OPTIONAL_PLUGINS_PATH, "index.rst"), "w", encoding="utf-8" ) as optional_plugins_toc: optional_plugins_toc.write( """.. index file for optional plugins API ==================== Optional Plugins API ==================== The following pages document the private APIs of optional Avocado plugins. .. toctree:: :maxdepth: 1 """ ) for path in next(os.walk(OPTIONAL_PLUGINS_PATH))[1]: name = f"avocado_{os.path.basename(path)}" try: importlib.import_module(name) except ImportError: continue path = os.path.join(OPTIONAL_PLUGINS_PATH, path, name) if not os.path.exists(path): continue output_dir = os.path.join(API_OPTIONAL_PLUGINS_PATH, name) params = {"API_SOURCE_DIR": path, "output_dir": output_dir, "exclude_dirs": ""} process.run(f"{APIDOC_TEMPLATE % params} --implicit-namespaces") # Remove the unnecessary generated files os.unlink(os.path.join(output_dir, "modules.rst")) optional_plugins_toc.write(f"\n {os.path.join(name, name)}") extensions = [ "sphinx.ext.autodoc", # pylint: disable=C0103 "sphinx.ext.intersphinx", "sphinx.ext.todo", "sphinx.ext.coverage", "sphinx_rtd_theme", ] master_doc = "index" # pylint: disable=C0103 project = "Avocado" # pylint: disable=C0103 copyright = "2014-2023, Red Hat" # pylint: disable=W0622,C0103 VERSION_FILE = os.path.join(ROOT_PATH, "VERSION") VERSION = genio.read_file(VERSION_FILE).strip() version = VERSION # pylint: disable=C0103 release = VERSION # pylint: disable=C0103 html_theme = "sphinx_rtd_theme" # pylint: disable=C0103 htmlhelp_basename = "avocadodoc" # pylint: disable=C0103 latex_documents = [ # pylint: disable=C0103 ( "index", "avocado.tex", "avocado Documentation", "Avocado Development Team", "manual", ), ] man_pages = [ # pylint: disable=C0103 ("index", "avocado", "avocado Documentation", ["Avocado Development Team"], 1) ] texinfo_documents = [ # pylint: disable=C0103 ( "index", "avocado", "avocado Documentation", "Avocado Development Team", "avocado", "One line description of project.", "Miscellaneous", ), ] intersphinx_mapping = {"python": ("https://docs.python.org", None)} autoclass_content = "both" # pylint: disable=C0103 avocado-113.0/docs/source/config/000077500000000000000000000000001513561306000166275ustar00rootroot00000000000000avocado-113.0/docs/source/config/index.rst000066400000000000000000000011511513561306000204660ustar00rootroot00000000000000.. _config-reference: Avocado's Configuration Reference ================================= This is current Avocado Configuration reference. You can adjust the values by two ways: - Configuration file options; - Command-line options (when available) Some options that are used often are available for your convenience also at the command-line. This list has all options registered with Avocado so far. .. note:: Please, keep in mind that we are in constant evolution and doing a huge improvements on how to configure Avocado, some options here can be changed in the near future. .. include:: reference.rst avocado-113.0/docs/source/guides/000077500000000000000000000000001513561306000166425ustar00rootroot00000000000000avocado-113.0/docs/source/guides/contributor/000077500000000000000000000000001513561306000212145ustar00rootroot00000000000000avocado-113.0/docs/source/guides/contributor/api/000077500000000000000000000000001513561306000217655ustar00rootroot00000000000000avocado-113.0/docs/source/guides/contributor/api/.gitignore000066400000000000000000000000061513561306000237510ustar00rootroot00000000000000*.rst avocado-113.0/docs/source/guides/contributor/api/api_utils_heading000066400000000000000000000006231513561306000253610ustar00rootroot00000000000000This is a set of utility APIs that Avocado provides as added value to test writers. It's suppose to be generic, without any knowledge of Avocado and reusable in different projects. .. note:: In the current version there is a hidden knowledge of avocado logging streams. More about this issue can be found here https://trello.com/c/4QyUgWsW/720-get-rid-of-avocado-test-loggers-from-avocado-utils avocado-113.0/docs/source/guides/contributor/api/core/000077500000000000000000000000001513561306000227155ustar00rootroot00000000000000avocado-113.0/docs/source/guides/contributor/api/core/.gitignore000066400000000000000000000000061513561306000247010ustar00rootroot00000000000000*.rst avocado-113.0/docs/source/guides/contributor/api/test/000077500000000000000000000000001513561306000227445ustar00rootroot00000000000000avocado-113.0/docs/source/guides/contributor/api/test/.gitignore000066400000000000000000000000061513561306000247300ustar00rootroot00000000000000*.rst avocado-113.0/docs/source/guides/contributor/api/utils/000077500000000000000000000000001513561306000231255ustar00rootroot00000000000000avocado-113.0/docs/source/guides/contributor/api/utils/.gitignore000066400000000000000000000000061513561306000251110ustar00rootroot00000000000000*.rst avocado-113.0/docs/source/guides/contributor/chapters/000077500000000000000000000000001513561306000230255ustar00rootroot00000000000000avocado-113.0/docs/source/guides/contributor/chapters/ai_policy.rst000066400000000000000000000116111513561306000255270ustar00rootroot00000000000000Policy for AI-Generated Code ============================ Introduction and Purpose ------------------------ This policy outlines the guidelines and requirements for the use of Artificial Intelligence (AI) generated code within the Avocado framework project. As AI code generation tools become increasingly prevalent, it is crucial to establish clear standards to ensure transparency, maintain code quality, uphold licensing integrity, and address potential ethical and security concerns. This policy aims to facilitate responsible and effective integration of AI tools while safeguarding the principles of open-source collaboration. What is AI-Generated Code ------------------------- Any code, code snippets, configurations, documentation, or other programmatic assets produced, suggested, or significantly modified by an artificial intelligence model, tool, or service. This includes code that is directly used, or heavily adapted from AI suggestions. Transparency and Disclosure --------------------------- All contributors are required to be transparent about the use of AI-generated code. - **Explicit Attribution:** When submitting a Pull Request (PR) or commit that includes AI-generated code, contributors **must** explicitly state that AI tools were used. This has to be done in the commit message and PR description. - **Example Commit Message/PR Description:** .. code-block:: python :emphasize-lines: 5,6,7 header <- Limited to 72 characters. No period. <- Blank line message <- Any number of lines, limited to 72 characters per line. <- Blank line Assisted-by: <- artificial intelligence model, tool, or service which has been used and how big part of contribution has been generated (percentage) Reference: <- External references, one per line (issue, trello, ...) Signed-off-by: <- Signature and acknowledgment of licensing terms when contributing to the project (created by git commit -s) - **Originality:** If AI was used as a brainstorming or refactoring tool, but the final code is substantially original and reviewed by the contributor, a general disclosure in the PR description is sufficient. Licensing and Copyright ----------------------- Contributors are responsible for ensuring that AI-generated code complies with the Avocado's license (GPLv2). - **Be Alert to Possible Copyright Issues:** Do not assume that AI-generated code is free of licensing obligations or copyright. Treat it with the same diligence as code copied from other sources. To the extent practical, ensure that AI-generated code does not infringe third-party copyrights. - **Review for Training Data Leakage:** Contributors are expected to enable any code similarity matching or blocking functionality offered by their AI tools if it is available. This helps to identify or prevent potential leakage from proprietary or incompatibly licensed training data. If such instances are identified by the contributor, the contributor must either (a) ensure that the AI-generated code is used in a manner that complies with its license, and ensure that such license is compatible with the license of the project, or (b) the code should be re-written before submission. Code Quality, Review, and Testing ---------------------------------- AI-generated code must adhere to the same quality standards as human-written code. - **Human Review Required:** All AI-generated code, regardless of its source, **must** be thoroughly reviewed by a human contributor. This review should be as rigorous, if not more so, than the review of human-written code. AI contribution best practise ----------------------------- - **AI as an Assistant:** AI tools are encouraged as assistants for boilerplate generation, refactoring suggestions, debugging, or exploring different approaches. They should not replace critical thinking, understanding, and human review. - **Understanding is Key:** Contributors must fully understand any AI-generated code they submit. Do not submit code you do not comprehend or cannot debug yourself. - **Iterative Refinement:** Do not blindly accept AI output. Enforcement and Dispute Resolution ---------------------------------- - **Non-Compliance:** Failure to adhere to this policy may result in the rejection of Pull Requests. - **Disputes:** Any disputes or concerns regarding the use of AI-generated code will be discussed and resolved by the Avocado maintainers. Disclaimer ---------- This policy is a living document and will be updated as AI technology evolves and best practices emerge. Contributors are encouraged to provide feedback and suggestions to improve this policy over time.avocado-113.0/docs/source/guides/contributor/chapters/contact.rst000066400000000000000000000004661513561306000252200ustar00rootroot00000000000000Contact information =================== - Avocado-devel mailing list: `https://www.redhat.com/mailman/listinfo/avocado-devel `_ - Avocado GitHub repository: `https://github.com/avocado-framework/avocado/ `_ avocado-113.0/docs/source/guides/contributor/chapters/environment.rst000066400000000000000000000101221513561306000261170ustar00rootroot00000000000000Development environment ======================= Using a container ----------------- Having reproducible development environments is hard because developers will usually choose their preferred environments, and those will never be the same for two people. Using a container can be a good option if you're unsure whether your development environment is sound or compatible enough. The Avocado project maintains a container image that can serve as a pretty good starting point for development too (it's actually intended to be used on some CI checks, but don't mind that). You can pull the image from:: quay.io/avocado-framework/avocado-ci-fedora-40 It's a simple image, with just a number of known needed packages installed. This is the full definition of the image: .. literalinclude:: ../../../../../contrib/containers/ci/selftests/fedora-40.docker You can use the information there to apply to your own environment if you choose not to use the container image itself. Installing dependencies ----------------------- You need to install few dependencies before start coding:: $ sudo dnf install gcc python-devel enchant Then install all the python dependencies:: $ make requirements-dev Or if you already have pip installed, you can run directly:: $ pip install -r requirements-dev.txt If you intend to build the documentation locally, please also run:: $ pip install -r requirements-doc.txt Installing in develop mode -------------------------- Since version 0.31.0, our plugin system requires Setuptools entry points to be registered. If you're hacking on Avocado and want to use the same, possibly modified, source for running your tests and experiments, you may do so with one additional step:: $ python3 setup.py develop [--user] On POSIX systems this will create an "egg link" to your original source tree under ``$HOME/.local/lib/pythonX.Y/site-packages``. Then, on your original source tree, an "egg info" directory will be created, containing, among other things, the Setuptools entry points mentioned before. This works like a symlink, so you only need to run this once (unless you add a new entry-point, then you need to re-run it to make it available). Avocado supports various plugins, which are distributed as separate projects, for example "avocado-vt". These also need to be deployed and "linked" in order to work properly with the Avocado from sources (installed version works out of the box). You can install external plugins as you wish, and/or according to the specific plugin's maintainer recommendations. Plugins that are developed by the Avocado team, will try to follow the same Setuptools standard for distributing the packages. Because of that, as a facility, you can use ``make requirements-plugins`` from the main Avocado project to install requirements of the plugins and ``make develop-external`` to install plugins in develop mode to. You just need to set where your plugins are installed, by using the environment variable ``$AVOCADO_EXTERNAL_PLUGINS_PATH``. The workflow could be:: $ cd $AVOCADO_PROJECTS_DIR $ git clone $AVOCADO_GIT $ git clone $AVOCADO_PROJECT2 $ # Add more projects $ cd avocado # go into the main Avocado project dir $ make requirements-plugins $ export AVOCADO_EXTERNAL_PLUGINS_PATH=$AVOCADO_PROJECTS_DIR $ make develop-external You should see the process and status of each directory. GPG Signatures -------------- This is an optional step for most contributors, but if you're interested in ensuring that your contribution is linked to yourself, this is the best way to do so. To get a GPG signature, you can find many howtos on the internet, but it generally works like this:: $ gpg --gen-key # defaults are usually fine (using expiration is recommended) $ gpg --send-keys $YOUR_KEY # to propagate the key to outer world Then, you should enable it in git:: $ git config --global user.signingkey $YOUR_KEY Optionally, you can link the key with your GH account: 1. Login to github 2. Go to settings->SSH and GPG keys 3. Add New GPG key 4. run ``$(gpg -a --export $YOUR_EMAIL)`` in shell to see your key 5. paste the key there avocado-113.0/docs/source/guides/contributor/chapters/how.rst000066400000000000000000000226151513561306000243620ustar00rootroot00000000000000How can I contribute? ===================== .. note:: Except where otherwise indicated in a given source file, all original contributions to Avocado are licensed under the GNU General Public License version 2 `(GPLv2) `_ or any later version. By contributing you agree with: a) our code of conduct; b) that these contributions are your own (or approved by your employer), and c) you grant a full, complete, irrevocable copyright license to all users and developers of the Avocado project, present and future, pursuant to the license of the project. Report a bug ------------ If Avocado crashes on you, or if one of the Avocado "selftests" fail, congratulations, you may have just found a bug. And If you have precise steps to reproduce, awesome! You're on your way to reporting a useful bug report. Don't be afraid to report bugs, even if you're not sure if they're valid. The most that can happen is that we find out together that this is a feature instead! Avocado is using GitHub's issue tracking system for collecting and discussing issues. If you have a possible candidate, do not hesitate, share with us by `creating a new bug report `_. Suggest enhancements -------------------- The same is valid when suggesting a new feature or enhancements: Don't think twice, just `submit the feature request `_ using the same link. Our community will evaluate if the feature request is valid and when it might become a part of the Avocado Framework. Contribute with code -------------------- Avocado uses GitHub and its pull request development model. You can find a primer on how to use GitHub pull requests `here `_. Every Pull Request you send will be automatically tested by the `CI system `_ and review will take place in the Pull Request as well. Remember that we do have a small “Feature Freeze” period right before the release day (usually no longer than one week). It means that during this time, no new feature can be merged into the master branch. Git workflow ~~~~~~~~~~~~ - `Fork the repository `_ in GitHub. - Clone from your fork:: $ git clone --recurse-submodules git@github.com:/avocado.git .. note:: The ``--recurse-submodules`` option is used to checkout the contents from the `avocado-static-checks `_ repository, which is not needed for general Avocado installations, but which is very important for development purposes. - Enter the directory:: $ cd avocado - Create a ``remote``, pointing to the upstream:: $ git remote add upstream git@github.com:avocado-framework/avocado.git - Configure your name and e-mail in git:: $ git config --global user.name "Your Name" $ git config --global user.email email@foo.bar - Golden tip: never work on local branch master. Instead, create a new local branch and checkout to it:: $ git checkout -b my_new_local_branch - Code and then commit your changes:: $ git add new-file.py $ git commit -s # or "git commit -as" to commit all changes .. seealso:: Please, read our Commit Style Guide on Style Guides section manual. - Make sure your code is working (install your version of avocado, test your change, run ``make check`` to make sure you didn't introduce any regressions). - Rebase your local branch on top of upstream master:: $ git fetch $ git rebase upstream/master (resolve merge conflicts, if any) - Push your commit(s) to your fork:: $ git push origin my_new_local_branch - `Create the Pull Request `_ on GitHub. Add the relevant information to the Pull Request description. - Check if your Pull Request passes the CI system. Your Pull Request will probably be ignored until it's all green. Now you're waiting for feedback on GitHub Pull Request page. Once you get some, join the discussion, answer the questions, make clear if you're going to change the code based on some review and, if not, why. Feel free to disagree with the reviewer, they probably have different use cases and opinions, which is expected. Try describing yours and suggest other solutions, if necessary. Then, proceed to make the changes. This is a typical workflow: - Code, and amend the commit(s) and/or create new commits. If you have more than one commit in the PR, you will probably need to rebase interactively to amend the right commits. ``git cola`` or ``git citool`` can be handy here. - Rebase your local branch on top of upstream master:: $ git fetch $ git rebase upstream/master (resolve merge conflicts, if any) - Push your changes:: $ git push --force origin my_new_local_branch Please communicate to the reviewers what the summary of changes are. Also, make use of GitHub's features to ease the reviewers' life, such as marking comments as "resolved". Reviewers should make use of GitHub's "compare" feature to more easily verify the changes since the last iteration. After your PR gets merged, you can sync the master branch on your local repository propagate the sync to the master branch in your fork repository on GitHub:: $ git checkout master $ git pull upstream master $ git push From time to time, you can remove old branches to avoid pollution:: # To list branches along with time reference: $ git for-each-ref --sort='-authordate:iso8601' --format=' %(authordate:iso8601)%09%(refname)' refs/heads # To remove branches from your fork repository: $ git push origin :my_old_branch Code Review ~~~~~~~~~~~ Every single Pull Request in Avocado has to be reviewed by at least one other developer. All members of the core team have permission to merge a Pull Request, but some conditions have to be fulfilled before merging the code: - Pull Request has to pass the CI tests. - One 'Approved' code review should be given. - No explicit disapproval should be present. Pull Requests failing in CI will not be merged, and reviews won't be given to them until all the problems are sorted out. In case of a weird failure, or false-negative (eg. due to too many commits in a single PR), please reach the developers by @name/email or other means. While reviewing the code, one should: - Verify that the code is sound and clean. - Run the highest level of selftests per each new commit in the merge. The ``contrib/scripts/avocado-check-pr.sh`` contrib script should simplify this step. - Verify that code works to its purpose. - Make sure the commits organization is proper (i.e. code is well organized in atomic commits, there's no extra/unwanted commits, ...). - Provide an in-line feedback with explicit questions and/or requests of improvements. - Provide a general feedback in the review message, being explicit about what's expected for the next Pull Request version, if that's the case. When the Pull Request is approved, the reviewer will merge the code or wait for someone with merge permission to merge it. Using ``avocado-check-pr.sh`` ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The ``contrib/scripts/avocado-check-pr.sh`` script is here to simplify the per-commit-check. You can simply prepare the merge and initiate ``AVOCADO_CHECK_LEVEL=99 contrib/scripts/avocado-check-pr.sh`` to run all checks per each commit between your branch and the same branch on the ``origin/master`` (you can specify different remote origin). Use ``./contrib/scripts/avocado-check-pr.sh -h`` to learn more about the options. We can recommend the following command:: $ AVOCADO_CHECK_LEVEL=99 $ ./contrib/scripts/avocado-check-pr.sh -i -v .. note:: Before first use you might need to create ``~/.config/github_checker.ini`` and fill GitHub user/token entries (while on it you can also specify some defaults) Share your tests ---------------- We encourage you or your company to create public Avocado tests repositories so the community can also benefit of your tests. We will be pleased to advertise your repository here in our documentation. List of known community and third party maintained repositories: - https://github.com/avocado-framework-tests/avocado-misc-tests: Community maintained Avocado miscellaneous tests repository. There you will find, among others, performance tests like ``lmbench``, ``stress``, cpu tests like ``ebizzy`` and generic tests like ``ltp``. Some of them were ported from Autotest Client Tests repository. Documentation ------------- The process of contributing documentation is very similar to the process of contributing code. The only noteworthy difference is that instead of running ``make check`` to verify the soundness of the changes, you should build the documentation with a command such as:: $ make -C docs html If there are any warnings or errors introduced by your changes, the documentation will fail to build, showing the cause, such as in:: Warning, treated as error: $HOME/avocado/docs/source/guides/contributor/chapters/how.rst:239:Unknown directive type "foo". .. foo:: bar make: *** [Makefile:48: html] Error 2 A similar check is also performed by the CI, via readthedocs.org. avocado-113.0/docs/source/guides/contributor/chapters/introduction.rst000066400000000000000000000006421513561306000263020ustar00rootroot00000000000000Brief introduction ================== First of all, we would like to thank you for taking the time to contribute! We collected here useful pointers on how to participate in the Avocado community and how to contribute. And keep in mind that our procedures and guides are far from perfection, and need constant improvements. Feel free to propose changes to this, or any other, guide in a pull request. Happy Hacking! avocado-113.0/docs/source/guides/contributor/chapters/plugins.rst000066400000000000000000000433741513561306000252530ustar00rootroot00000000000000.. _writing_plugin: ************************* Writing an Avocado plugin ************************* What better way to understand how an Avocado plugin works than creating one? Let's use another old time favorite for that, the "Print hello world" theme. Code example ============ Let's say you want to write a plugin that adds a new subcommand to the test runner, ``hello``. This is how you'd do it: .. literalinclude:: ../../../../../examples/plugins/cli-cmd/hello/hello.py This plugins inherits from :class:`avocado.core.plugin_interfaces.CLICmd`. This specific base class allows for the creation of new commands for the Avocado CLI tool. The only mandatory method to be implemented is :func:`run ` and it's the plugin main entry point. This plugin uses :py:data:`avocado.core.output.LOG_UI` to produce the hello world output in the console. .. note:: Different loggers can be used in other contexts and for different purposes. One such example is :py:data:`avocado.core.output.LOG_JOB`, which can be used to output to job log files when running a job. Registering configuration options (settings) ============================================ It is usual for a plugin to allow users to do some degree of configuration based on command-line options and/or configuration options. A plugin might change its behavior depending on a specific configuration option. Frequently, those settings come from configuration files and, sometimes, from the command-line arguments. Like in most UNIX-like tools, command-line options will override values defined inside the configuration files. You, as a plugin writer, don’t need to handle this configuration by yourself. Avocado provides a common API that can be used by plugins in order to register options and get values. If your plugin has options available to the users, register it using the :meth:`.Settings.register_option()` method during your plugin configuration stage. The options are parsed and provided to the plugin as a config dictionary. Let’s take our Hello World example and change the message based on a “message” option: .. literalinclude:: ../../../../../examples/plugins/cli-cmd/hello_option/hello_option.py The code in the example above registers a **configuration namespace** (*hello.message*) inside the configuration file only. A namespace is a **section** (*hello*) followed by a **key** (*message*). In other words, the following entry in your configuration file is also valid and will be parsed:: [hello] message = My custom message As you can see in the example above, you need to set a **default** value and this value will be used if the option is not present in the configuration file. This means that you can have a very small configuration file or even an empty one. This is a very basic example of how to configure options inside your plugin. Adding command-line options --------------------------- Now, let’s say you would like to also allow this change via the command-line option of your plugin (if your plugin is a command-line plugin). You need to register in any case and use the same method to connect your **option namespace** with your command-line option. .. literalinclude:: ../../../../../examples/plugins/cli-cmd/hello_parser/hello_parser.py .. note:: Keep in mind that not all options should have a “command-line” option. Try to keep the command-line as clean as possible. We use command-line only for options that constantly need to change and when editing the configuration file is not handy. For more information about how this registration process works, visit the :meth:`.Settings.register_option()` method documentation. .. _registering-plugins: Registering plugins =================== Avocado makes use of the `setuptools` and its `entry points` to register and find Python objects. So, to make your new plugin visible to Avocado, you need to add to your setuptools based `setup.py` file something like: .. literalinclude:: ../../../../../examples/plugins/cli-cmd/hello/setup.py Then, by running either ``$ python setup.py install`` or ``$ python setup.py develop`` your plugin should be visible to Avocado. Namespace ========= The plugin registry mentioned earlier, (`setuptools` and its `entry points`) is global to a given Python installation. Avocado uses the namespace prefix ``avocado.plugins.`` to avoid name clashes with other software. Now, inside Avocado itself, there's no need keep using the ``avocado.plugins.`` prefix. Take for instance, the Job Pre/Post plugins are defined on ``setup.py``:: 'avocado.plugins.job.prepost': [ 'jobscripts = avocado.plugins.jobscripts:JobScripts' ] The setuptools entry point namespace is composed of the mentioned prefix ``avocado.plugins.``, which is then followed by the Avocado plugin type, in this case, ``job.prepost``. Inside Avocado itself, the fully qualified name for a plugin is the plugin type, such as ``job.prepost`` concatenated to the name used in the entry point definition itself, in this case, ``jobscripts``. To summarize, still using the same example, the fully qualified Avocado plugin name is going to be ``job.prepost.jobscripts``. Plugin config files =================== Plugins can extend the list of config files parsed by ``Settings`` objects by dropping the individual config files into ``/etc/avocado/conf.d`` (linux/posix-way) or they can take advantages of the Python entry point using ``avocado.plugins.settings``. 1. `/etc/avocado/conf.d`: In order to not disturb the main Avocado config file, those plugins, if they wish so, may install additional config files to ``/etc/avocado/conf.d/[pluginname].conf``, that will be parsed after the system wide config file. Users can override those values as well at the local config file level. Considering the config for the hypothethical plugin ``salad``: .. code-block:: ini [salad.core] base = caesar dressing = caesar If you want, you may change ``dressing`` in your config file by simply adding a ``[salad.core]`` new section in your local config file, and set a different value for ``dressing`` there. 2. `avocado.plugins.settings`: This entry-point uses ``avocado.core.plugin_interfaces.Settings``-like object to extend the list of parsed files. It only accepts individual files, but you can use something like ``glob.glob("*.conf")`` to add all config files inside a directory. You need to create the plugin (eg. ``my_plugin/settings.py``):: from avocado.core.plugin_interfaces import Settings class MyPluginSettings(Settings): def adjust_settings_paths(self, paths): paths.extend(glob.glob("/etc/my_plugin/conf.d/*.conf")) And register it in your ``setup.py`` entry-points:: from setuptools import setup ... setup(name="my-plugin", entry_points={ 'avocado.plugins.settings': [ "my-plugin-settings = my_plugin.settings.MyPluginSettings", ], ... Which extends the list of files to be parsed by settings object. Note this has to be executed early in the code so try to keep the required deps minimal (for example the `avocado.core.settings.settings` is not yet available). .. _plugins_execution_order: Plugins execution order ======================= Avocado lets plugin developers to define plugin priority, which ensures the default execution order.The plugins with higher priority will be executed earlier than the plugins with lower priority. The priorities are on the scale between 0-100 and as default all plugins have priority set to NORMAL (50). For easier usage, avocado has predefined values in: .. autoclass:: avocado.core.extension_manager.PluginPriority :noindex: :members: :undoc-members: To define a priority for plugin, you have to create class attribute ``priority`` same as you're defining ``name`` and ``description``: .. literalinclude:: ../../../../../examples/plugins/cli-cmd/hello_priority/hello_priority.py Now, the plugin ``HelloWorld`` has priority high and will be executed before every plugin without priority variable (default priority). As a plugin developer, you have to consider that users can change the execution order by ``plugins.$type.order`` option. In such case, at first will be executed the plugins from ``plugins.$type.order`` and then the rest of plugins by its priority. For example the default order is [plugin1, plugin2, plugin3, plugin4] the ``plugins.$type.order`` is [plugin2, plugin4] then the real order will be [plugin2, plugin4, plugin1, plugin3] .. _new-test-type-plugin-example: Cacheable plugins ================= The results of Pre-test and Post-test plugins defined in :class:`avocado.core.plugin_interfaces.PreTest` and :class:`avocado.core.plugin_interfaces.PostTest` can be saved in cache. This is very useful If the results can be used by other tests or even other avocado executions. As an example of such plugin is `Dependency resolver` in :ref:`managing-requirements` which installs dependencies for test into the test environment. As a default, all pre- / post-plugins are noncacheable. To make plugin cacheable you have to set plugin variable `is_cacheable` to `True`, like this: .. literalinclude:: ../../../../../examples/plugins/test-pre-post/hello/hello.py New test type plugin example ============================ For a new test type to be recognized and executed by Avocado's nrunner architecture, there needs to be two types of plugins and one optional: * resolvers: they resolve references into proper test descriptions that Avocado can run. * discoverers (optional): They are doing the same job as resolvers but without a reference. They are used when the tests can be created from different data e.g. *config files*. * runners: these make use of the resolutions made by resolvers and actually execute the tests, reporting the results back to Avocado The following example shows real code for a resolver and a runner for a *magic* test type. This *magic* test simply passes or fails depending on the test reference. Resolver and Discoverer example ------------------------------- The resolver implementation will simply set the test type (*magic*) and transform the reference given into its **url**: .. literalinclude:: ../../../../../examples/plugins/tests/magic/avocado_magic/resolver.py Tests contained in files and associated data ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ A ``magic`` test does *not* depend on a file, only on the "magic" word (either "pass" or "fail"). Because of that, there's no need to provide information at resolution time about the file(s) comprising the magic tests. For most other test types, though, the test will be contained within a file, and/or may be comprised by many supplemental files containing test specific data. While local execution of a test will easily find those files, in order to be prepared for the execution of a test in different environments (by different spawners), it's recommended that the resolver (and discoverer) provide that kind of information. A feature that is strongly recommended to be implemented by resolvers and discoverers of tests that are file-based is the ``.data`` directory support. Whenever a test contained in a file has a matching directory with the ``.data`` :data:`suffix `, the test can get quick access to these files with the :meth:`get_data ` method. To do so, use the ``assets`` keyword argument when creating :class:`Runnable ` instances. The ``assets`` keyword argument takes a list of tuples with ``(type, path)`` tuples, which will then be available at the :data:`Runnable ` attribute. Actual examples are available in the implementation of the ``exec-test`` and similar builtin resolvers. For the file that contains the test, it's recommended that resolvers and discoverers use the type :data:`avocado.core.resolver.ReferenceResolutionAssetType.TEST_FILE`. For other data files, use the :data:`avocado.core.resolver.ReferenceResolutionAssetType.DATA_FILE`. Runner example -------------- The runner will receive the :class:`avocado.core.nrunner.runnable.Runnable` information created by the resolver plugin. Runners can be written in any language, but this implementation reuses some base Python classes. First, :class:`avocado.core.nrunner.runner.BaseRunner` is used to write the runner **class**. And second, the :class:`avocado.core.nrunner.app.BaseRunnerApp` is used to create the command line application, which uses the previously implemented runner class for ``magic`` test types. .. literalinclude:: ../../../../../examples/plugins/tests/magic/avocado_magic/runner.py A runner is free to make use of all the information in the :class:`avocado.core.nrunner.runnable.Runnable` that the resolver implementation populates. In this particular example it only makes use of the :attr:`uri ` attribute. If a runner needs to behave accordingly to some Avocado configuration, you need to declare that configuration in the :attr:`CONFIGURATION_USED ` class attribute and then you can access it in :attr:`config `. Activating the new test type plugins ------------------------------------ The plugins need to be registered so that Avocado knows about it. See :ref:`registering-plugins` for more information. This is the code that can be used to register these plugins: .. literalinclude:: ../../../../../examples/plugins/tests/magic/setup.py With that, you need to either run ``python setup.py install`` or ``python setup.py develop``. .. note:: The last entry, registering a ``console_script``, is recommended because it allows one to experiment with the runner as a command line application (``avocado-runner-magic`` in this case). Also, depending on the spawner implementation used to run the tests, having a runner that can be executed as an application (and not a Python class) is a requirement. Listing the new test type plugins --------------------------------- With the plugins activated, you should be able to run ``avocado plugins`` and find (among other output):: Plugins that resolve test references (resolver): ... magic Test resolver for magic words ... Resolving magic tests --------------------- Resolving the "pass" and "fail" references that the magic plugin knows about can be seen by running ``avocado list magic:pass magic:fail``:: magic magic:pass magic magic:fail And you may get more insight into the resolution results, by adding a verbose parameter and another reference. Try running ``avocado -V list magic:pass magic:fail magic:foo something-else``:: Reference magic:foo might be resolved by magic resolver, but the file is corrupted: Word "magic:foo" is magic type but the foo is not a valid magic word Type Test Tag(s) magic magic:pass magic magic:fail Resolver Reference Info avocado-instrumented magic:pass File name "magic" does not end with suffix ".py" golang magic:pass go binary not found avocado-instrumented magic:fail File name "magic" does not end with suffix ".py" golang magic:fail go binary not found avocado-instrumented magic:foo File name "magic" does not end with suffix ".py" golang magic:foo go binary not found magic magic:foo Word "magic:foo" is magic type but the foo is not a valid magic word avocado-instrumented something-else File name "something-else" does not end with suffix ".py" golang something-else go binary not found magic something-else Word "something-else" is not a valid magic word python-unittest something-else File name "something-else" does not end with suffix ".py" robot something-else File name "something-else" does not end with suffix ".robot" rogue something-else Word "something-else" is not the magic word exec-test something-else File "something-else" does not exist or is not a executable file tap something-else File "something-else" does not exist or is not a executable file TEST TYPES SUMMARY ================== magic: 2 It's worth realizing that magic (and other plugins) were asked to resolve the ``magic:foo`` and ``something-else`` references, but couldn't:: Resolver Reference Info ... magic magic:foo Word "magic:foo" is magic type but the foo is not a valid magic word ... magic something-else Word "something-else" is not a valid magic word ... We can see that the reference "magic:foo" resembles the magic words by type but it is not magic words ``pass`` or ``fail``. Consequently, the resolver can provide the user with information about potentially corrupted references. This can assist the user in identifying typos or reference mistakes. As the creator of the resolver, you can use the :data:`avocado.core.resolver.ReferenceResolutionResult.CORRUPT` variable to notify the user of such a situation. Running magic tests ------------------- The common way of running Avocado tests is to run them through ``avocado run``. To run both the ``pass`` and ``fail`` magic tests, you'd run ``avocado run -- magic:pass magic:fail``:: $ avocado run -- magic:pass magic:fail JOB ID : 86fd45f8c1f2fe766c252eefbcac2704c2106db9 JOB LOG : $HOME/avocado/job-results/job-2021-02-05T12.43-86fd45f/job.log (1/2) magic:pass: STARTED (1/2) magic:pass: PASS (0.00 s) (2/2) magic:fail: STARTED (2/2) magic:fail: FAIL (0.00 s) RESULTS : PASS 1 | ERROR 0 | FAIL 1 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0 JOB HTML : $HOME/avocado/job-results/job-2021-02-05T12.43-86fd45f/results.html JOB TIME : 1.83 s avocado-113.0/docs/source/guides/contributor/chapters/releasing.rst000066400000000000000000000100701513561306000255260ustar00rootroot00000000000000Releasing Avocado ================= So you have all PRs approved, the Sprint meeting is done and now Avocado is ready to be released. Great, let's go over (most of) the details you need to pay attention to. Which repositories you should pay attention to ---------------------------------------------- In general, a release of Avocado includes taking a look and eventually release content in the following repositories: * ``avocado`` * ``avocado-vt`` How to release? --------------- All the necessary steps are in JSON "testplans" to be executed with the following commands:: $ scripts/avocado-run-testplan -t examples/testplans/release/pre.json $ scripts/avocado-run-testplan -t examples/testplans/release/release.json Just follow the steps and have a nice release! How to refresh Fedora rawhide ----------------------------- This is an outline of the steps to update the Fedora rawhide packages. This example is based on updating from 82.0 to 83.0. Update downstream python-avocado package ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ #. Use pagure to create a personal fork of the downstream Fedora dist-git ``python-avocado`` package source repository https://src.fedoraproject.org/rpms/python-avocado if you don’t already have one. #. Clone your personal fork repository to your local workspace. #. Checkout the ``latest`` branch, and make sure your ``latest`` branch is in sync with the most recent commits from the official dist-git repo you forked from. #. Locate the official upstream commit hash and date corresponding to the upstream GitHub release tag. (eg., https://github.com/avocado-framework/avocado/releases/tag/75.1) Use those values to update the ``%global commit`` and ``%global commit_date`` lines in the downstream ``python-avocado.spec`` file. #. Update the ``Version:`` line with the new release tag. #. Reset the ``Release:`` line to ``1%{?gitrel}%{?dist}``. #. Add a new entry at the beginning of the ``%changelog`` section with a message similar to ``Sync with upstream release 83.0.``. #. See what changed in the upstream SPEC file since the last release. You can do this by comparing branches on GitHub (eg., https://github.com/avocado-framework/avocado/compare/82.0..83.0) and searching for ``python-avocado.spec``. If there are changes beyond just the ``%global commit``, ``%global commit_date``, and ``Version:`` lines, and the ``%changelog`` section, make any necessary corresponding changes to the downstream SPEC file. Note: the commit hash in the upstream SPEC file *will* be different that what gets put in the downstream SPEC file since the upstream hash was added to the file before the released commit was made. Add an additional note to your ``%changelog`` message if there were any noteworthy changes. #. Download the new upstream source tarball based on the updated SPEC by running:: spectool -g python-avocado.spec #. Add the new source tarball to the dist-git lookaside cache and update your local repo by running:: fedpkg new-sources avocado-83.0.tar.gz #. Create a Fedora source RPM from the updated SPEC file and tarball by running:: fedpkg --release f33 srpm It should write an SRPM file (eg., ``python-avocado-83.0-1.fc33.src.rpm``) to the current directory. #. Test build the revised package locally using ``mock``. Run the build using the same Fedora release for which the SRPM was created:: mock -r fedora-33-x86_64 python-avocado-83.0-1.fc33.src.rpm #. If the package build fails, go back and fix the SPEC file, re-create the SRPM, and retry the mock build. It is occasionally necessary to create a patch to disable specific tests or pull in some patches from upstream to get the package to build correctly. See https://src.fedoraproject.org/rpms/python-avocado/tree/69lts as an example. #. Repeat the SRPM generation and mock build for all other supported Fedora releases, Fedora Rawhide, and the applicable EPEL (currently EPEL8). #. When you have successful builds for all releases, ``git add``, ``git commit``, and ``git push`` your updates. avocado-113.0/docs/source/guides/contributor/chapters/results.rst000066400000000000000000000017451513561306000252670ustar00rootroot00000000000000Implementing other result formats --------------------------------- If you are looking to implement a new machine or human readable output format, you can refer to :mod:`avocado.plugins.xunit` and use it as a starting point. If your result is something that is produced at once, based on the complete job outcome, you should create a new class that inherits from :class:`avocado.core.plugin_interfaces.Result` and implements the :meth:`avocado.core.plugin_interfaces.Result.render` method. But, if your result implementation is something that outputs information live before/during/after tests, then the :class:`avocado.core.plugin_interfaces.ResultEvents` interface is the one to look at. It will require you to implement the methods that will perform actions (write to a file/stream) for each of the defined events on a Job and test execution. You can take a look at :ref:`writing_plugin` for more information on how to write a plugin that will activate and execute the new result format. avocado-113.0/docs/source/guides/contributor/chapters/rfc.rst000066400000000000000000000010671513561306000243350ustar00rootroot00000000000000Request for Comments (RFCs) =========================== What is a RFC? -------------- RFCs were the main mechanism used in the early days of the Avocado project to discuss larger ideas. RFCs have been pretty much supplanted by BluePrints (see :doc:`/blueprints/BP000`). Previous RFCs ------------- The following list contains archivals of accepted, Request For Comments posted and discussed on the `Avocado Devel Mailing List`_. .. toctree:: :maxdepth: 1 ../rfcs/lts .. _Avocado Devel Mailing List: https://www.redhat.com/mailman/listinfo/avocado-devel avocado-113.0/docs/source/guides/contributor/chapters/styleguides.rst000066400000000000000000000051641513561306000261260ustar00rootroot00000000000000Style guides ============ .. _commit_style_guide: Commit style guide ------------------ Write a good commit message, pointing motivation, issues that you're addressing. Usually you should try to explain 3 points in the commit message: motivation, approach and effects:: header <- Limited to 72 characters. No period. <- Blank line message <- Any number of lines, limited to 72 characters per line. <- Blank line Assisted-by: <- artificial intelligence model, tool, or service which has been used and how big part of contribution has been generated (percentage) Reference: <- External references, one per line (issue, trello, ...) Signed-off-by: <- Signature and acknowledgment of licensing terms when contributing to the project (created by git commit -s) The header is the visible part when reviewing commit logs and should make it easy to identify which part in the source tree was affected. The message helps reviewers to understand why certain changes were introduced in the code, esp. when they are not obvious. Also, they can give important context if debugging or refactoring code in the future becomes necessary. A good example of a commit header and message:: commit b94a186a2d638b791f03bcc0c270e7412de0d9bc Author: Cleber Rosa Date: Mon Aug 11 19:07:00 2025 -0400 avocado.utils.gdb: remove module level variables used by plugin These module level variables were used to control the behavior of the gdb utility from the command line plugin. The plugin was never migrated to nrunner due to the parallelism aspects, so these are not needed anymore. Signed-off-by: Cleber Rosa Signing commits ~~~~~~~~~~~~~~~ If you've set a GPG signature, it's a good idea to put it in use when committing your changes. To sign your commits, add the ``-S`` command line option, such as in:: $ git commit -S And if you are merging branches:: $ git merge -S .. warning:: If you use the merge button on GitHub, the signature will be performed with GitHub's own private key. Please check whether you find that acceptable or not. Code style guide ---------------- Avocado uses the Black code style checker, and thus, you should follow its very opinionated style. In reality, it's recommended to use your editor or IDE features to make sure the style is applied automatically. Please refer to the `black documentation `__ for more information. avocado-113.0/docs/source/guides/contributor/chapters/tips.rst000066400000000000000000000044051513561306000245410ustar00rootroot00000000000000Avocado development tips ======================== In tree utils ------------- You can find handy utils in `avocado.utils.debug`: measure_duration ---------------- Decorator can be used to print current duration of the executed function and accumulated duration of this decorated function. It's very handy when optimizing. Usage:: from avocado.utils import debug ... @debug.measure_duration def your_function(...): During the execution look for:: PERF: : (0.1s, 11.3s) PERF: : (0.2s, 11.5s) .. note:: If you are running a test with Avocado, and want to measure the duration of a method/function, make sure to enable the `avocado.utils.debug` logging stream. Example:: avocado --show avocado.utils.debug run examples/tests/assets.py Line-profiler ------------- You can measure line-by-line performance by using line_profiler. You can install it using pip:: pip install line_profiler and then simply mark the desired function with `@profile` (no need to import it from anywhere). Then you execute:: kernprof -l -v avocado run ... and when the process finishes you'll see the profiling information. (sometimes the binary is called `kernprof.py`) Remote debug with Eclipse ------------------------- Eclipse is a nice debugging frontend which allows remote debugging. It's very simple. The only thing you need is Eclipse with pydev plugin. The simplest way is to use ``pip install pydevd`` and then you set the breakpoint by:: import pydevd pydevd.settrace(host="$IP_ADDR_OF_ECLIPSE_MACHINE", stdoutToServer=False, stderrToServer=False, port=5678, suspend=True, trace_only_current_thread=False, overwrite_prev_trace=False, patch_multiprocessing=False) Before you run the code, you need to start the Eclipse's debug server. Switch to `Debug` perspective (you might need to open it first `Window->Perspective->Open Perspective`). Then start the server from `Pydev->Start Debug Server`. Now whenever the pydev.settrace() code is executed, it contacts Eclipse debug server (port `8000` by default, don't forget to open it) and you can easily continue in execution. This works on every remote machine which has access to your Eclipse's port `8000` (you can override it). avocado-113.0/docs/source/guides/contributor/index.rst000066400000000000000000000006451513561306000230620ustar00rootroot00000000000000=========================== Avocado Contributor's Guide =========================== Useful pointers on how to participate of the Avocado community and contribute. .. toctree:: :maxdepth: 2 chapters/introduction chapters/how chapters/environment chapters/styleguides chapters/plugins chapters/results chapters/rfc chapters/releasing chapters/tips chapters/contact chapters/ai_policy avocado-113.0/docs/source/guides/contributor/rfcs/000077500000000000000000000000001513561306000221515ustar00rootroot00000000000000avocado-113.0/docs/source/guides/contributor/rfcs/lts.rst000066400000000000000000000345361513561306000235200ustar00rootroot00000000000000.. _rfc-long-term-stability: RFC: Long Term Stability ======================== This RFC contains proposals and clarifications regarding the maintenance and release processes of Avocado. We understand there are multiple teams currently depending on the stability of Avocado and we don't want their work to be disrupted by incompatibilities nor instabilities in new releases. This version is a minor update to previous versions of the same RFC (see `Changelog`_) which drove the release of Avocado 36.0 LTS. The Avocado team has plans for a new LTS release in the near future, so please consider reading and providing feedback on the proposals here. TL;DR ----- We plan to keep the current approach of sprint releases every 3-4 weeks, but we're introducing "Long Term Stability" releases which should be adopted in production environments where users can't keep up with frequent upgrades. Introduction ------------ We make new releases of Avocado every 3-4 weeks on average. In theory at least, we're very careful with backwards compatibility. We test Avocado for regressions and we try to document any issues, so upgrading to a new version should be (again, in theory) safe. But in practice both intended and unintended changes are introduced during development, and both can be frustrating for conservative users. We also understand it's not feasible for users to upgrade Avocado very frequently in a production environment. The objective of this RFC is to clarify our maintenance practices and introduce Long Term Stability (LTS) releases, which are intended to solve, or at least mitigate, these problems. Our definition of maintained, or stable --------------------------------------- First of all, Avocado and its sub-projects are provided 'AS IS' and WITHOUT ANY WARRANTY, as described in the LICENSE file. The process described here doesn't imply any commitments or promises. It's just a set of best practices and recommendations. When something is identified as "stable" or "maintained", it means the development community makes a conscious effort to keep it working and consider reports of bugs and issues as high priorities. Fixes submitted for these issues will also be considered high priorities, although they will be accepted only if they pass the general acceptance criteria for new contributions (design, quality, documentation, testing, etc), at the development team discretion. Maintained projects and platforms --------------------------------- The only maintained project as of today is the Avocado Test Runner, including its APIs and core plugins (the contents of the main avocado git repository). Other projects kept under the "Avocado Umbrella" in github may be maintained by different teams (e.g.: Avocado-VT) or be considered experimental (e.g.: avocado-server and avocado-virt). More about Avocado-VT in its own section further down. As a general rule, fixes and bug reports for Avocado when running in any modern Linux distribution are welcome. But given the limited capacity of the development team, packaged versions of Avocado will be tested and maintained only for the following Linux distributions: * RHEL 7.x (latest) * Fedora (stable releases from the Fedora projects) Currently all packages produced by the Avocado projects are "noarch". That means that they could be installable on any hardware platform. Still, the development team will currently attempt to provide versions that are stable for the following platforms: * x86 * ppc64le Contributions from the community to maintain other platforms and operating systems are very welcome. The lists above may change without prior notice. Avocado Releases ---------------- The proposal is to have two different types of Avocado releases: Sprint Releases ~~~~~~~~~~~~~~~ (This is the model we currently adopt in Avocado) They happen every 3-4 weeks (the schedule is not fixed) and their versions are numbered serially, with decimal digits in the format .. Examples: 47.0, 48.0, 49.0. Minor releases are rare, but necessary to correct some major issue with the original release (47.1, 47.2, etc). Only the latest Sprint Release is maintained. In Sprint Releases we make a conscious effort to keep backwards compatibility with the previous version (APIs and behavior) and as a general rule and best practice, incompatible changes in Sprint Releases should be documented in the release notes and if possible deprecated slowly, to give users time to adapt their environments. But we understand changes are inevitable as the software evolves and therefore there's no absolute promise for API and behavioral stability. Long Term Stability (LTS) Releases ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ LTS releases should happen whenever the team feels the code is stable enough to be maintained for a longer period of time, ideally once or twice per year (no fixed schedule). They should be maintained for 18 months, receiving fixes for major bugs in the form of minor (sub-)releases. With the exception of these fixes, no API or behavior should change in a minor LTS release. They will be versioned just like Sprint Releases, so looking at the version number alone will not reveal the differentiate release process and stability characteristics. In practice each major LTS release will imply in the creation of a git branch where only important issues affecting users will be fixed, usually as a backport of a fix initially applied upstream. The code in a LTS branch is stable, frozen for new features. Notice that although within a LTS release there's a expectation of stability because the code is frozen, different (major) LTS releases may include changes in behavior, API incompatibilities and new features. The development team will make a considerable effort to minimize and properly document these changes (changes when comparing it to the last major LTS release). Sprint Releases are replaced by LTS releases. I.e., in the cycle when 52.0 (LTS) is released, that's also the version used as a Sprint Release (there's no 52.0 -- non LTS -- in this case). New LTS releases should be done carefully, with ample time for announcements, testing and documentation. It's recommended that one or two sprints are dedicated as preparations for a LTS release, with a Sprint Release serving as a "LTS beta" release. Similarly, there should be announcements about the end-of-life (EOL) of a LTS release once it approaches its 18 months of life. Deployment details ------------------ Sprint and LTS releases, when packaged, whenever possible, will be preferably distributed through different package channels (repositories). This is possible for repository types such as :ref:`YUM/DNF repos `. In such cases, users can disable the regular channel, and enable the LTS version. A request for the installation of Avocado packages will fetch the latest version available in the enabled repository. If the LTS repository channel is enabled, the packages will receive minor updates (bugfixes only), until a new LTS version is released (roughly every 12 months). If the non-LTS channel is enabled, users will receive updates every 3-4 weeks. On other types of repos such as `PyPI`_ which have no concept of "sub-repos" or "channels", users can request a version smaller than the version that succeeds the current LTS to get the latest LTS (including minor releases). Suppose the current LTS major version is 52, but there have been minor releases 52.1 and 52.2. By running:: pip install 'avocado-framework<53.0' pip provide LTS version 52.2. If 52.3 gets released, they will be automatically deployed instead. When a new LTS is released, users would still get the latest minor release from the 52.0 series, unless they update the version specification. The existence of LTS releases should never be used as an excuse to break a Sprint Release or to introduce gratuitous incompatibilities there. In other words, Sprint Releases should still be taken seriously, just as they are today. Timeline example ---------------- Consider the release numbers as date markers. The bullet points beneath them are information about the release itself or events that can happen anytime between one release and the other. Assume each sprint is taking 3 weeks. 36.0 * **LTS** release (the only LTS release available at the time of writing) 37.0 .. 49.0 * sprint releases * 36.1 LTS release * 36.2 LTS release * 36.3 LTS release * 36.4 LTS release 50.0 * sprint release * start preparing a LTS release, so 51.0 will be a **beta LTS** 51.0 * sprint release * **beta LTS** release 52.0 * **LTS** release * 52lts branch is created * packages go into LTS repo * both **36.x LTS** and **52.x LTS** maintained from this point on 53.0 * sprint release * minor bug that affects 52.0 is found, fix gets added to master and 52lts branches * bug does **not** affect 36.x LTS, so a backport is **not** added to the 36lts branch 54.0 * sprint release 54.0 * LTS release 52.1 * minor bug that also affects 52.x LTS and 36.x LTS is found, fix gets added to master, 52lts and 36lts branches 55.0 * sprint release * LTS release 36.5 * LTS release 52.2 * critical bug that affects 52.2 *only* is found, fix gets added to 52lts and **52.3 LTS is immediately released** 56.0 * sprint release 57.0 * sprint release 58.0 * sprint release 59.0 * sprint release * EOL for **36.x LTS** (18 months since the release of 36.0), 36lts branch is frozen permanently. A few points are worth taking notice here: * Multiple LTS releases can co-exist before EOL * Bug discovery can happen at any time * The bugfix occurs ASAP after its discovery * The severity of the defect determines the timing of the release - moderate and minor bugfixes to lts branches are held until the next sprint release - critical bugs are released asynchronously, without waiting for the next sprint release Avocado-VT ---------- Avocado-VT is an Avocado plugin that allows "VT tests" to be run inside Avocado. It's a third-party project maintained mostly by Engineers from Red Hat QE with assistance from the Avocado team and other community members. It's a general consensus that QE teams use Avocado-VT directly from git, usually following the master branch, which they control. There's no official maintenance or stability statement for Avocado-VT. Even though the upstream community is quite friendly and open to both contributions and bug reports, Avocado-VT is made available without any promises for compatibility or supportability. When packaged and versioned, Avocado-VT rpms should be considered just snapshots, available in packaged form as a convenience to users outside of the Avocado-VT development community. Again, they are made available without any promises of compatibility or stability. * Which Avocado version should be used by Avocado-VT? This is up to the Avocado-VT community to decide, but the current consensus is that to guarantee some stability in production environments, Avocado-VT should stick to a specific LTS release of Avocado. In other words, the Avocado team recommends production users of Avocado-VT not to install Avocado from its master branch or upgrade it from Sprint Releases. Given each LTS release will be maintained for 18 months, it should be reasonable to expect Avocado-VT to upgrade to a new LTS release once a year or so. This process will be done with support from the Avocado team to avoid disruptions, with proper coordination via the avocado mailing lists. In practice the Avocado development team will keep watching Avocado-VT to detect and document incompatibilities, so when the time comes to do an upgrade in production, it's expected that it should happen smoothly. * Will it be possible to use the latest Avocado and Avocado-VT together? Users are welcome to *try* this combination. The Avocado development team itself will do it internally as a way to monitor incompatibilities and regressions. Whenever Avocado is released, a matching versioned snapshot of Avocado-VT will be made. Packages containing those Avocado-VT snapshots, for convenience only, will be made available in the regular Avocado repository. Changelog --------- Changes from `Version 4`_: * Moved changelog to the bottom of the document * Changed wording on bug handling for LTS releases ("important issues") * Removed ppc64 (big endian) from list of platforms * If bugs also affect older LTS release during the transition period, a backport will also be added to the corresponding branch * Further work on the `Timeline example`_, adding summary of important points and more release examples, such as the whole list of 36.x releases and the (fictional) 36.5 and 52.3 Changes from `Version 3`_: * Converted formatting to REStructuredText * Replaced "me" mentions on version 1 changelog with proper name (Ademar Reis) * Renamed section "Misc Details" to `Deployment Details`_ * Renamed "avocado-vt" to "Avocado-VT" * Start the timeline example with version 36.0 * Be explicit on timeline example that a minor bug did not generate an immediate release Changes from `Version 2`_: * Wording changes on second paragraph ("... nor instabilities...") * Clarified on "Introduction" that change of behavior is introduced between regular releases * Updated distro versions for which official packages are built * Add more clear explanation on official packages on the various hardware platforms * Used more recent version numbers as examples, and the planned new LTS version too * Explain how users can get the LTS version when using tools such as pip * Simplified the timeline example, with examples that will possibly match the future versions and releases * Documented current status of Avocado-VT releases and packages Changes from `Version 1`_: * Changed "Support" to "Stability" and "supported" to "maintained" [Jeff Nelson] * Misc improvements and clarifications in the supportability/stability statements [Jeff Nelson, Ademar Reis] * Fixed a few typos [Jeff Nelson, Ademar Reis] .. _Version 1: https://www.redhat.com/archives/avocado-devel/2016-April/msg00006.html .. _Version 2: https://www.redhat.com/archives/avocado-devel/2016-April/msg00038.html .. _Version 3: https://www.redhat.com/archives/avocado-devel/2017-April/msg00032.html .. _Version 4: https://www.redhat.com/archives/avocado-devel/2017-April/msg00041.html .. _PyPI: https://pypi.python.org/pypi avocado-113.0/docs/source/guides/reference/000077500000000000000000000000001513561306000206005ustar00rootroot00000000000000avocado-113.0/docs/source/guides/reference/chapters/000077500000000000000000000000001513561306000224115ustar00rootroot00000000000000avocado-113.0/docs/source/guides/reference/chapters/avocado_instrumented.rst000066400000000000000000000072701513561306000273660ustar00rootroot00000000000000 The Avocado instrumented lifecycle ================================== The Avocado instrumented test goes through multiple phases during its execution. This page describes in detail each of those phases and overall lifecycle of instrumented test execution. INIT: This is the initialization state of test. Each test is created in this state. During the initialization, the test attributes are set, environment variables are created, and test skip conditions are evaluated. SETUP: The test is in this state when Test.setUp() method is being executed. TEST: During this state, the actual test is running. TEARDOWN: During this state, the Test.tearDown() method is being executed to make clean up after testing. FINISHED: This state is the end of the test's lifecycle, and the test is in this state when the testing and clean up is finished. during this phase, the test results are reported to runner and the testing is finished. The whole Test lifecycle is represented by diagram:: +--------+ | | | INIT +------------+ | | | +----+---+ | | | | | Test|initialized | | | | Test|skipped +----v----+ | | | | | SETUP +-----------+---+ | | | | +----+----+ | | | | | | | | Test|set up | | | | | | | | +----v---+ | | | | | | +-----------+ TEST <------------+ | | | | | | +----+---+ Setup|timed out | | | | Test finished or timed out | | | | | | | | +------v-----+ | | | | | Test|skipped | TEARDOWN <-------------+ | | | | +------+-----+ | | | | | Teardown finished or timed out | | | | | +------v-----+ | | | +---------> FINISHED | | | +------------+ Timeouts ~~~~~~~~ The avocado instrumented tests can be affected by three types of timeouts. Test timeout, Task timeout and Job timeout. All of these timeouts behaves the same from the point of view of Test’s lifecycle, but the lifecycle will change base on in which state the timeout was reached. Here is a description of different Test’s lifecycles based on the interruptions: Timeout in SETUP INIT -> SETUP(interrupted) -> TEARDOWN -> FINISHED (ERROR) Timeout in TEST INIT -> SETUP -> TEST(interrupted) -> TEARDOWN -> FINISHED (INTERRUPTED) Timeout in TEARDOWN INIT -> SETUP -> TEST -> TEARDOWN(interrupted) -> FINISHED (ERROR) .. note:: If either the Task or Job timeout is reached after some other interruption (test or task timeouts) the test is killed in the phase where it is (probably TEARDOWN), and it won’t properly finish. avocado-113.0/docs/source/guides/reference/chapters/runners.rst000066400000000000000000000515471513561306000246530ustar00rootroot00000000000000.. _nrunner: The Avocado default runner (nrunner) ==================================== This section details the default Avocado suite runner called "nrunner", also previously known as N(ext) Runner, and the architecture around it. .. tip:: a suite runner is an implementation of the :class:`avocado.core.plugin_interfaces.SuiteRunner` interface. It's the component that runs one or more tests that are contained in a :class:`avocado.core.suite.TestSuite`. At its essence, the nrunner architecture, when compared to the previous runner architecture (now referred to as the "legacy runner") is about making Avocado more capable and flexible. Even though it started with a major internal paradigm change within the test runner, it also affects users and test writers used to the legacy runner. The :mod:`avocado.core.nrunner` module was initially responsible for most of the nrunner code. As development continued, it spread around to other places in the Avocado source tree. Other components with different and seemingly unrelated names, say the "resolvers" or the "spawners", are also pretty much about the nrunner architecture. Motivation ---------- There are a number of reasons for introducing a different architecture and implementation. Some of them are related to limitations found in the legacy implementation, that were found to be too hard to remove without major breakage. Also, missing features that are deemed important would be a better fit wihin a different architecture. For instance, these are the limitations of the Avocado legacy test runner: * Test execution limited to the same machine, given that the communication between runner and test is a Python queue * Test execution is limited to a single test at a time (serial execution) * Test processes are not properly isolated and can affect the test runner (including the "UI") And these are some features which it's believed to be more easily implemented under a different architecture and implementation: * Remote test execution * Different test execution isolation models provided by the test runner (process, container, virtual machine) * Distributed execution of tests across a pool of any combination of processes, containers, virtual machines, etc. * Parallel execution of tests * Optimized runners for a given environment and or test type (for instance, a runner written in RUST to run tests written in RUST in an environment that already has RUST installed but not much else) * Notification of execution results to many simultaneous "status servers" * Disconnected test execution, so that results can be saved to a device and collected by the runner * Simplified and automated deployment of the runner component into execution environments such as containers and virtual machines nrunner components of Avocado ----------------------------- Whenever we mention the **current** architecture or implementation, we are talking about the nrunner. It includes: * ``avocado list`` command * ``avocado run`` command * :mod:`avocado.core.resolver` module to resolve tests * :mod:`avocado.core.spawners` modules to spawn tasks Basic Avocado usage and workflow -------------------------------- Avocado is described as "a set of tools and libraries to help with automated testing". The most visible aspect of Avocado is its ability to run tests, and display the results. We're talking about someone doing:: $ avocado run mytests.py othertests.sh To be able to complete such a command, Avocado needs to find the tests, and then to execute them. Those two major steps are described next. .. _finding_tests: Finding tests ~~~~~~~~~~~~~ The first thing Avocado needs to do, before actually running any tests, is translating the "names" given as arguments to ``avocado run`` into actual tests. Even though those names will usually be file names, this is not a requirement. Avocado calls those "names" given as arguments to ``avocado run`` "test references", because they are references that hopefully "point to" tests. On the nrunner architecture, each one of the test references given to ``list`` or ``run`` will be "resolved" into zero or more tests. Being more precise and verbose, resolver plugins will produce :class:`avocado.core.resolver.ReferenceResolution`, which contain zero or more :class:`avocado.core.nrunner.runnable.Runnable`, which are described in the following section. Overall, the process looks like:: +--------------------+ +-----------------------+ | avocado list | run | -> | avocado.core.resolver | ---+ +--------------------+ +-----------------------+ | | +----------------------------------------------------+ | v +--------------------------------------+ | ReferenceResolution #1 | +--------------------------------------+ | Reference: /bin/true | | Result: SUCCESS | | +----------------------------------+ | | | Resolution #1 (Runnable): | | | | - kind: exec-test | | | | - uri: /bin/true | | | +----------------------------------+ | +--------------------------------------+ +--------------------------------------+ | ReferenceResolution #2 | +--------------------------------------+ | Reference: test.py | | Result: SUCCESS | | +----------------------------------+ | | | Resolution #1 (Runnable): | | | | - kind: python-unittest | | | | - uri: test.py:Test.test_1 | | | +----------------------------------+ | | +----------------------------------+ | | | Resolution #2 (Runnable): | | | | - kind: python-unittest | | | | - uri: test.py:Test.test_2 | | | +----------------------------------+ | +--------------------------------------+ ... Running Tests ~~~~~~~~~~~~~ The idea of **testing** has to do with checking the expected output of a given action. This action, within the realm of software development with automated testing, has to do with the output or outcome of a "code payload" when executed under a given controlled environment. In the nrunner architecture, a :class:`avocado.core.nrunner.runnable.Runnable` describe a "code payload" that will be executed, but they are not executable code themselves. Because they are **data** and not **code**, they are easily serialized and transported to different environments. Running the payload described by a ``Runnable`` is delegated to another component. Most often, this component is a standalone executable (see :data:`avocado.core.spawners.common.SpawnMethod.STANDALONE_EXECUTABLE`) compatible with a specific command line interface. The most important interfaces such scripts must implement are the ``runnable-run`` and ``task-run`` interfaces. Once all the ``Runnable(s)`` (within the ``ReferenceResolution(s)``) are created by :mod:`avocado.core.resolver`, the ``avocado run --suite-runner=nrunner`` implementation follows roughly the following steps: 1. Creates a status server that binds to a TCP port and waits for status messages from any number of clients 2. Creates the chosen :class:`Spawner `, with :class:`ProcessSpawner ` being the default 3. For each :class:`avocado.core.nrunner.runnable.Runnable` found by the resolver, turns it into a :class:`avocado.core.nrunner.Task`, which means giving it the following extra information: a. The status server(s) that it should report to b. An unique identification, so that its messages to the status server can be uniquely identified 4. For each resulting :class:`avocado.core.nrunner.Task` in the previous step: a. Asks the spawner to spawn it b. Asks the spawner to check if the task seems to be alive right after spawning it, to give the user early indication of possible crashes 5. Waits until all tasks have provided a ``result`` to the status server If any of the concepts mentioned here were not clear, please check their full descriptions in the next section. Concepts -------- Runnable ~~~~~~~~ A runnable is a description of an entity that can be executed and produce some kind of result. It's a passive entity that can not execute itself and can not produce results itself. This description of a runnable is abstract on purpose. While the most common use case for a Runnable is to describe how to execute a test, there seems to be no reason to bind that concept to a test. Other Avocado subsystems, leverage the same concept. One example is the :ref:`sysinfo-collection` which describes what kind of system information collection is to be performed in a ``Runnable``. A Runnable's ``kind`` +++++++++++++++++++++ The most important information about a runnable is the declaration of its kind. A kind should be a globally unique name across the entire Avocado community and users. When choosing a Runnable ``kind`` name, it's advisable that it should be: * Informative * Succinct * Unique If a kind is thought to be generally useful to more than one user (where a user may mean a project using Avocado), it's a good idea to also have a generic name. For instance, if a Runnable is going to describe how to run native tests for the Go programming language, its ``kind`` should probably be ``go``. On the other hand, if a Runnable is going to be used to describe tests that behave in a very peculiar way for a specific project, it's probably a good idea to map its ``kind`` name to the project name. For instance, if one is describing how to run an ``iotest`` that is part of the ``QEMU`` project, it may be a good idea to name this kind ``qemu-iotest``. A Runnable's ``uri`` ++++++++++++++++++++ Besides a ``kind``, each runnable kind may require a different amount of information to be provided so that it can be instantiated. Based on the accumulated experience so far, it's expected that a Runnable's ``uri`` is always going to be required. Think of the URI as the one piece of information that can uniquely distinguish the entity (of a given ``kind``) that will be executed. If, for instance, a given runnable describes the execution of a executable file already present in the system, it may use its path, say ``/bin/true``, as its ``uri`` value. If a runnable describes a web service endpoint, its ``uri`` value may just as well be its network URI, such as ``https://example.org:8080``. Runnable examples +++++++++++++++++ Possibly the simplest example for the use of a Runnable is to describe how to run a standalone executable, such as the ones available on your ``/bin`` directory. As stated earlier, a runnable must declare its kind. For standalone executables, a name such as ``exec`` fulfills the naming suggestions given earlier. A Runnable can be created in a number of ways. The first one is through :class:`avocado.core.nrunner.Runnable`, a very low level (and internal) API. Still, it serves as an example:: >>> from avocado.core.nrunner.runnable import Runnable >>> runnable = Runnable('exec', '/bin/true') >>> runnable The second way is through a JSON based file, which, for the lack of a better term, we're calling a (Runnable) "recipe". The recipe file itself will look like:: {"kind": "exec", "uri": "/bin/true"} And example the code to create it:: >>> from avocado.core.nrunner.runnable import Runnable >>> runnable = Runnable.from_recipe("/path/to/recipe.json") >>> runnable The third way to create a Runnable, is even more internal. Its usage is **discouraged**, unless you are creating a tool that needs to create Runnables based on the user's input from the command line:: >>> from avocado.core.nrunner.runnable import Runnable >>> runnable = Runnable.from_args({'kind': 'exec', 'uri': '/bin/true'}) >>> runnable Runner ~~~~~~ A Runner, within the context of the nrunner architecture, is an active entity. It acts on the information that a runnable contains, and quite simply, should be able to run what the Runnable describes. A Runner will usually be tied to a specific ``kind`` of Runnable. That type of relationship (Runner is capable of running kind "foo" and Runnable is of the same kind "foo") is the expected mechanism that will be employed when selecting a Runner. It's recommended that a runner takes the form of an executable that follows the ``avocado-runner-$KIND`` naming pattern and conforms to a given interface/behavior, including accepting standardized command line arguments and producing standardized output. This gives the runner the highest probability of working with different spawners, including ones that would run on isolated or remote environments. .. tip:: for a very basic example of the interface expected, refer to ``selftests/functional/nrunner_interface.py`` on the Avocado source code tree. A Runner can also be, at the lowest layer, a Python class that inherits from :class:`avocado.core.nrunner.BaseRunner`, and implements at least a matching constructor method, and a ``run()`` method that should yield dictionary(ies) as result(s). Avocado may support in the future the usage of such runners directly, which can speed up execution, but limits where those can be run to pretty much the same machine. Runner output +++++++++++++ A Runner should, if possible, produce status information on the progress of the execution of a Runnable. While the Runner is executing what a Runnable describes, should it produce interesting information, the Runner should attempt to forward that along its generated status. For instance, using the ``exec`` Runner example, it's helpful to start producing status that the process has been created and it's running as soon as possible, even if no other output has been produced by the executable itself. These can be as simple as a sequence of:: {"status": "started"} {"status": "running"} {"status": "running"} When the process is finished, the Runner may return:: {"status": "finished", "returncode": 0, 'stdout': b'', 'stderr': b''} .. tip:: Besides the status of ``finished``, and a return code which can be used to determine a success or failure status, a Runner may not be obliged to determine the overall PASS/FAIL outcome. Whoever called the runner may be responsible to determine its overall result, including a PASS/FAIL judgement. Even though this level of information is expected to be generated by the Runner, whoever is calling a Runner, should be prepared to receive as little information as possible, and act accordingly. That includes receiving no information at all. For instance, if a Runner fails to produce any information within a given amount of time, it may be considered faulty and be completely discarded. This would probably end up being represented as a ``INTERRUPTED`` kind of status on a higher layer (say at the "Job" layer). Task ~~~~ A task is one specific instance/occurrence of the execution of a runnable with its respective runner. They should have a unique identifier, although a task by itself won't enforce its uniqueness in a process or any other type of collection. A task is responsible for producing and reporting status updates. This status updates are in a format similar to those received from a runner, but will add more information to them, such as its unique identifier. A different agreggate structure, :class:`avocado.core.task.runtime.RuntimeTask`, is used to keep track of the extra information while the task is being run. Recipe ~~~~~~ A recipe is the serialization of the runnable information in a file. The format chosen is JSON, and that should allow both quick and easy machine handling and also manual creation of recipes when necessary. Runners ------- A runner can be capable of running one or many different kinds of runnables. A runner should implement a ``capabilities`` command that returns, among other info, a list of runnable kinds that it can (to the best of its knowledge) run. Example:: python3 -m avocado.plugins.runners.exec_test capabilities | python -m json.tool { "runnables": [ "exec-test" ], "commands": [ "capabilities", "runnable-run", "runnable-run-recipe", "task-run", "task-run-recipe" ], "configuration_used": [ "run.keep_tmp", "runner.exectest.exitcodes.skip" ] } Runner scripts -------------- Specific runners are available as ``avocado-runner-$kind``. For instance, the runner for ``exec-test`` is available as ``avocado-runner-exec-test``. When using specific runners, the ``-k|--kind`` parameter can be omitted. Runner Execution ---------------- While the ``exec`` runner given as example before will need to create an extra process to actually run the standalone executable given, that is an implementation detail of that specific runner. Other types of runners may be able to run the code the users expects it to run, while still providing feedback about it in the same process. The runner's main method (``run()``) operates like a generator, and yields results which are dictionaries with relevant information about it. Trying it out - standalone -------------------------- It's possible to interact with the runner features by using the command line. This interface is not stable at all, and may be changed or removed in the future. Runnables from parameters ~~~~~~~~~~~~~~~~~~~~~~~~~ You can run a "noop" runner with:: avocado-runner-noop runnable-run -k noop You can run an "exec" runner with:: avocado-runner-exec-test runnable-run -k exec-test -u /bin/sleep -a 3.0 You can run an "exec-test" runner with:: avocado-runner-exec-test runnable-run -k exec-test -u /bin/true You can run a "python-unittest" runner with:: avocado-runner-python-unittest runnable-run -k python-unittest -u selftests/unit/test_test.py:TestClassTestUnit.test_long_name Runnables from recipes ~~~~~~~~~~~~~~~~~~~~~~ You can run a "noop" recipe with:: avocado-runner-noop runnable-run-recipe examples/nrunner/recipes/runnable/noop.json You can run an "exec-test" runner with:: avocado-runner-exec-test runnable-run-recipe examples/nrunner/recipes/runnable/exec_test_sleep_3.json You can run a "python-unittest" runner with:: avocado-runner-python-unittest runnable-run-recipe examples/nrunner/recipes/runnable/python_unittest.json Writing new runner scripts -------------------------- Even though you can write runner scripts in any language, if you're writing a new runner script in Python, you can benefit from the :class:`avocado.core.nrunner.app.BaseRunnerApp` class and from the :class:`avocado.core.nrunner.runner.BaseRunner` class. The following is a complete example of a script that could be named ``avocado-runner-magic`` that could act as a nrunner compatible runner for runnables with kind ``magic``. .. literalinclude:: ../../../../../examples/plugins/tests/magic/avocado_magic/runner.py :language: python :linenos: For a more complete explanation on the runner scripts and how they relate to plugins, please refer to :ref:`new-test-type-plugin-example`. Runners messages ---------------- When run as part of a job, every runner has to send information about its execution status to the Avocado job. That information is sent by messages which have different types based on the information which they are transmitting. Avocado understands three main types of messages: * started (required) * running * finished (required) The started and finished messages are obligatory and every runner has to send those. The running messages can contain different information during runner run-time like logs, warnings, errors .etc and that information will be processed by the avocado core. The messages are standard Python dictionaries with a specific structure. You can create it by yourself based on the table :ref:`supported-message-types`, or you can use helper methods in :class:`avocado.core.utils.messages` which will generate them for you. .. _supported-message-types: Supported message types ~~~~~~~~~~~~~~~~~~~~~~~ .. autoclass:: avocado.core.messages.StartMessageHandler :noindex: .. autoclass:: avocado.core.messages.FinishMessageHandler :noindex: Running messages ++++++++++++++++ This message can be used during the run-time and has different properties based on the information which is being transmitted. .. autoclass:: avocado.core.messages.LogMessageHandler :noindex: .. autoclass:: avocado.core.messages.StdoutMessageHandler :noindex: .. autoclass:: avocado.core.messages.StderrMessageHandler :noindex: .. autoclass:: avocado.core.messages.WhiteboardMessageHandler :noindex: .. autoclass:: avocado.core.messages.FileMessageHandler :noindex: avocado-113.0/docs/source/guides/reference/index.rst000066400000000000000000000005671513561306000224510ustar00rootroot00000000000000======================= Avocado Refefence Guide ======================= A collection of technical reference that explains the inner workings of Avocado. Useful for those trying to understand the Avocado architecture and expand it either through plugins or modifications to the core parts. .. toctree:: :maxdepth: 2 chapters/runners chapters/avocado_instrumented avocado-113.0/docs/source/guides/user/000077500000000000000000000000001513561306000176205ustar00rootroot00000000000000avocado-113.0/docs/source/guides/user/chapters/000077500000000000000000000000001513561306000214315ustar00rootroot00000000000000avocado-113.0/docs/source/guides/user/chapters/about.rst000066400000000000000000000030331513561306000232740ustar00rootroot00000000000000About Avocado ============= Avocado is a set of tools and libraries to help with automated testing. One can call it a test framework with benefits. Native tests are written in Python and they follow the :mod:`unittest` pattern, but any executable can serve as a test. Avocado is composed of: * A test runner that lets you execute tests. Those tests can be either written in your language of choice, or be written in Python and use the available libraries. In both cases, you get facilities such as automated log and system information collection. * Libraries that help you write tests in a concise, yet expressive and powerful way. You can find more information about what libraries are intended for test writers at :ref:`utility-libraries`. * :mod:`Plugins ` that can extend and add new functionality to the Avocado Framework. * A Python API for creating custom jobs and test suites for more advanced users. Avocado is built on the experience accumulated with `Autotest `__, while improving on its weaknesses and shortcomings. Avocado tries as much as possible to comply with standard Python testing technology. Tests written using the Avocado API are derived from the unittest class, while other methods suited to functional and performance testing were added. The test runner is designed to help people to run their tests while providing an assortment of system and logging facilities, with no effort, and if you want more features, then you can start using the API features progressively. avocado-113.0/docs/source/guides/user/chapters/advanced.rst000066400000000000000000000134341513561306000237350ustar00rootroot00000000000000Advanced usage ============== Custom Runnable Identifier -------------------------- In some cases, you might have a wrapper as an entry point for the tests, so Avocado will use only the wrapper as test id. For instance, imagine a Makefile with some targets ('foo', 'bar') and each target is one test. Having a single test suite with a test calling `foo`, it will make Avocado print something like this:: JOB ID : b6e5bdf2c891382bbde7f24e906a168af351154a JOB LOG : ~/avocado/job-results/job-2021-09-24T17.39-b6e5bdf/job.log (1/1) make: STARTED (1/1) make: PASS (2.72 s) RESULTS : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0 JOB HTML : ~/avocado/job-results/job-2021-09-24T17.39-b6e5bdf/results.html JOB TIME : 5.49 s This is happening because Avocado is using the 'uri' as identifier with in the current Runnables. You can change that by setting a custom format with the option `runner.identifier_format` in you `avocado.conf` file. For instance: ``` [runner] identifier_format = "{uri}-{args[0]}" ``` With the above adjustment, running the same suite it will produce something like this:: JOB ID : 577b70b079e9a6f325ff3e73fd9b93f80ee7f221 JOB LOG : /home/local/avocado/job-results/job-2021-11-23T13.12-577b70b/job.log (1/1) "/usr/bin/make-foo": STARTED (1/1) "/usr/bin/make-foo": PASS (0.01 s) RESULTS : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0 JOB HTML : ~/avocado/job-results/job-2021-11-23T13.12-577b70b/results.html JOB TIME : 0.97 s For the `identifier_format` you can use any f-string that it will use `{uri}`, `{args}` or `{kwargs}`. By default it will use `{uri}`. When using args, since it is a list, you can use in two different ways: "{args}" for the entire list, or "{args[n]}" for a specific element inside this list. The same is valid when using "{kwargs}". With kwargs, since it is a dictionary, you have to specify a key as index and then the values are used. For instance if you have a kwargs value named 'DEBUG', a valid usage could be: "{kwargs[DEBUG]}" and this will print the current value to this variable (i.e: True or False). .. note:: Please, keep in mind this is an experimental feature, and for now you have to use it in combination with :ref:`documentation`. .. note:: Also, be aware this feature it is meant to set custom Runnable identifiers strings only. Test Runner Selection --------------------- To effectively run a job with tests, Avocado makes use of a well described and pluggable interface. This means that users can choose (and developers can write) their own runners. Runner choices can be seen by running ``avocado plugins``:: ... Plugins that run test suites on a job (suite.runner): nrunner nrunner based implementation of job compliant runner And to select a different test runner (if another one exists):: avocado run --suite-runner=other_runner_plugin ... Running tests with an external runner ------------------------------------- It's pretty standard to have organically grown test suites in most software projects, and these usually include a custom-built, specific test runner who knows how to find and run their tests. Still, running those tests inside Avocado may be a good idea for various reasons, including having results in different human and machine-readable formats and collecting system information alongside those tests (the Avocado's Sysinfo functionality), and more. Avocado makes that possible using its "external runner" feature. The most basic way of using it is:: $ avocado-external-runner external_runner foo bar baz In this example, Avocado will report individual test results for tests foo, bar, and baz. The actual results will be based on the return code of individual executions of /path/to/external_runner foo, /path/to/external_runner bar and finally /path/to/external_runner baz. As another way to explain how this feature works, think of the "external runner" as an interpreter. The individual tests as anything that this interpreter recognizes and can execute. A UNIX shell, say /bin/sh could be considered an external runner, and files with shellcode could be viewed as tests:: $ echo "exit 1" > /tmp/fail $ echo "exit 0" > /tmp/pass $ avocado-external-runner /bin/sh /tmp/pass /tmp/fail JOB ID : 874cab7e2639f1e2244246c69a5e0d3e1afefee0 JOB LOG : ~/avocado/job-results/job-2022-01-19T15.33-874cab7/job.log (external-runner-2/2) /bin/sh-/tmp/fail: STARTED (external-runner-1/2) /bin/sh-/tmp/pass: STARTED (external-runner-2/2) /bin/sh-/tmp/fail: FAIL (0.01 s) (external-runner-1/2) /bin/sh-/tmp/pass: PASS (0.01 s) RESULTS : PASS 1 | ERROR 0 | FAIL 1 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0 JOB HTML : ~/avocado/job-results/job-2022-01-19T15.33-874cab7/results.html JOB TIME : 1.10 s .. note:: This example is pretty obvious and could be achieved by giving /tmp/pass and /tmp/fail shell “shebangs” (#!/bin/sh), making them executable (chmod +x /tmp/pass /tmp/fail), and running them as “SIMPLE” tests. But now consider the following example:: $ avocado-external-runner curl redhat.com "google.com -v" JOB ID : fa68dd49a4c00e5a3c2e0fe45c6b3b0ed1b6495e JOB LOG : ~/avocado/job-results/job-2022-01-19T15.37-fa68dd4/job.log (external-runner-2/2) /bin/curl-google.com: STARTED (external-runner-1/2) /bin/curl-redhat.com: STARTED (external-runner-2/2) /bin/curl-google.com: PASS (0.28 s) (external-runner-1/2) /bin/curl-redhat.com: PASS (5.39 s) RESULTS : PASS 2 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0 JOB HTML : ~/avocado/job-results/job-2022-01-19T15.37-fa68dd4/results.html JOB TIME : 6.38 s This effectively makes /bin/curl an “external test runner”, responsible for trying to fetch those URLs, and reporting PASS or FAIL for each of them. avocado-113.0/docs/source/guides/user/chapters/assets.rst000066400000000000000000000112351513561306000234670ustar00rootroot00000000000000.. _managing-assets: Managing Assets =============== .. note:: Please note that we are constantly improving on how we handle assets inside Avocado. Probably some changes will be delivered during the next releases. Assets are test artifacts that Avocado can download automatically either during the test execution, or before the test even starts (by parsing the test code or on-demand, manually registering them at the command-line). Sometimes, depending on the use case, those assets can be a bottleneck for disk space. If the tests constantly use large assets, it is important to know how Avocado stores and handles those artifacts. Listing assets -------------- To list cached assets in the system, use the following command:: $ avocado assets list This command supports ``--by-size-filter`` and ``--by-days`` options. When using the former, use a comparison filter and a size in bytes. For instance:: $ avocado assets list --by-size-filter=">=2048" The command above will list only assets bigger than 2Kb. Avocado supports the following operators: `=`, `>=`, `<=`, `<` and `>`. Now, to look for old assets (based on the access time), for example, 10 days older, use the ``--by-days`` option:: $ avocado assets list --by-days=10 Registering assets ------------------ To manually register a local asset in the cache, use the ``register`` command:: $ avocado assets register *NAME* *URL* Where ``NAME`` is the unique name to associate with this asset and ``URL`` is the path to the local asset to be manually registered. The ``register`` command also supports the ``--hash`` option, which allows the addition of the file's hash. Fetching assets from instrumented tests --------------------------------------- The ``fetch`` command allows the download of a limited definition of assets inside an Avocado Instrumented test. It uses a parser on instrumented test source to find ``fetch_asset`` calls composed of simple strings as parameters, or at least one level of variable in the same context with a string assignment, and fetch those assets without running the test. The only exception to strings as arguments is the ``locations`` parameter, which allows the user of a list. Following are some examples of supported definitions of assets by the ``fetch`` command: .. code-block:: python tarball_locations = [ 'https://mirrors.peers.community/mirrors/gnu/hello/hello-2.9.tar.gz', 'https://mirrors.kernel.org/gnu/hello/hello-2.9.tar.gz', 'http://gnu.c3sl.ufpr.br/ftp/hello-2.9.tar.gz', 'ftp://ftp.funet.fi/pub/gnu/prep/hello/hello-2.9.tar.gz' ] self.hello = self.fetch_asset( name='hello-2.9.tar.gz', asset_hash='cb0470b0e8f4f7768338f5c5cfe1688c90fbbc74', locations=tarball_locations) .. code-block:: python kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora' '/linux/releases/29/Everything/x86_64/os/images/pxeboot' '/vmlinuz') kernel_hash = '23bebd2680757891cf7adedb033532163a792495' kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash) To fetch the assets defined inside an instrumented test, use:: $ avocado assets fetch *avocado-instrumented* Where ``avocado-instrumented`` is the path to the Avocado instrumented test file. Removing assets --------------- It is possible to remove files from the cache directories manually. The ``purge`` utility helps with that: $ avocado assets purge --help Assets can be removed applying the same filters as described when listing them. It is possible to remove assets by a size filter (``--by-size-filter``) or assets older than N days (``--by-days``). .. _assets-removing-by-overall-cache-limit: Removing by overall cache limit ------------------------------- Besides the existing features, Avocado is able to set an overall limit, so that it matches the storage limitations locally or on CI systems. For instance it may be the case that a GitLab cache limit is 4 GiB, in that case Avocado can sort assets by last access, and remove all that exceeds 4 GiB (that is, keep the last accessed 4 GiB worth of cached files). Use the ``--by-overall-limit`` option specifying the size limit:: $ avocado assets purge --by-overall-limit=4g This ensures that the files which are not used for some time in the cache are automatically removed. Please, note that at the moment, you can only use 'b', 'k', 'm', 'g', and 't'as suffixes. Changing the default cache dirs ------------------------------- Assets are stored inside the ``datadir.paths.cache_dirs`` option. It is possible to change this in the configuration file. The current value is shown with the following command:: $ avocado config | grep datadir.paths.cache_dirs avocado-113.0/docs/source/guides/user/chapters/concepts.rst000066400000000000000000000227641513561306000240140ustar00rootroot00000000000000Basic Concepts ============== It is important to understand some basic concepts before start using Avocado. Identifiers and references -------------------------- Job ID ~~~~~~ The Job ID is a random SHA1 string that uniquely identifies a given job. The full form of the SHA1 string is used is most references to a job:: $ avocado run examples/tests/sleeptest.py JOB ID : 49ec339a6cca73397be21866453985f88713ac34 ... But a shorter version is also used at some places, such as in the job results location:: JOB LOG : $HOME/avocado/job-results/job-2015-06-10T10.44-49ec339/job.log Test Resolver ~~~~~~~~~~~~~ A test resolver is Avocado's component that will take a reference you know about and will turn it into an actual test that can be run. This reference, explained next, can be pretty much any string, but it'll usually be some form of text containing the path to the file that contains the test. For more information please refer to :ref:`finding_tests`. Test References ~~~~~~~~~~~~~~~ A Test Reference is a string that can be resolved into (interpreted as) one or more tests by the Avocado Test Resolver. A given resolver plugin is free to interpret a test reference, it is completely abstract to the other components of Avocado. When the test references are about Instrumented Tests, Avocado will find any Instrumented test that **starts** with the reference, like a "wildcard". For instance:: $ avocado run ./test.py:MyTest:test_foo This command will resolve all tests (methods) that starts with `test_foo`. For more information about this type of tests, please visit the :ref:`Instrumented` section of this document. Conventions ~~~~~~~~~~~ Even though each resolver implementation is free to interpret a reference string as it sees fit, it's a good idea to set common user expectations. It's common for a single file to contain multiple tests. In that case, information about the specific test to reference can be added after the filesystem location and a colon, that is, for the reference:: passtest.py:PassTest.test Unless a file with that exact name exists, most resolvers will split it into ``passtest.py`` as the filesystem path, and ``PassTest.test`` as an additional specification for the individual test. It's also possible that some resolvers will support regular expressions and globs for the additional information component. Test Name ~~~~~~~~~ A test name is an arbitrarily long string that unambiguously points to the source of a single test. In other words the Avocado Test Resolver, as configured for a particular job, should return one and only one test as the interpretation of this name. This name can be as specific as necessary to make it unique. Therefore it can contain an arbitrary number of variables, prefixes, suffixes, tags, etc. It all depends on user preferences, what is supported by Avocado via its Test Resolvers and the context of the job. The output of the Test Resolver when resolving Test References should always be a list of unambiguous Test Names (for that particular job). Notice that although the Test Name has to be unique, one test can be run more than once inside a job. By definition, a Test Name is a Test Reference, but the reciprocal is not necessarily true, as the latter can represent more than one test. Examples of Test Names:: '/bin/true' 'passtest.py:Passtest.test' 'file:///tmp/passtest.py:Passtest.test' 'multiple_tests.py:MultipleTests.test_hello' 'type_specific.io-github-autotest-qemu.systemtap_tracing.qemu.qemu_free' Variant IDs ~~~~~~~~~~~ The varianter component creates different sets of variables (known as "variants"), to allow tests to be run individually in each of them. A Variant ID is an arbitrary and abstract string created by the varianter plugin to identify each variant. It should be unique per variant inside a set. In other words, the varianter plugin generates a set of variants, identified by unique IDs. A simpler implementation of the varianter uses serial integers as Variant IDs. A more sophisticated implementation could generate Variant IDs with more semantic, potentially representing their contents. Test ID ~~~~~~~ A test ID is a string that uniquely identifies a test in the context of a job. When considering a single job, there are no two tests with the same ID. A test ID should encapsulate the Test Name and the Variant ID, to allow direct identification of a test. In other words, by looking at the test ID it should be possible to identify: - What's the test name - What's the variant used to run this test (if any) Test IDs don't necessarily keep their uniqueness properties when considered outside of a particular job, but two identical jobs run in the exact same environment should generate a identical sets of Test IDs. Syntax:: -[;] Example of Test IDs:: '1-/bin/true' '2-passtest.py:Passtest.test;quiet-' '3-file:///tmp/passtest.py:Passtest.test' '4-multiple_tests.py:MultipleTests.test_hello;maximum_debug-df2f' '5-type_specific.io-github-autotest-qemu.systemtap_tracing.qemu.qemu_free' .. _test-types: Test types ---------- Avocado at its simplest configuration can run three different types of tests: * Executable tests (``exec-test``) * Python unittest tests (``python-unittest``) * Avocado Instrumented tests (``avocado-instrumented``) * TAP producing tests (``tap``) You can mix and match those in a single job. Avocado plugins can also introduce additional test types. Executable tests ~~~~~~~~~~~~~~~~ Any executable file can serve as a test. The criteria for PASS/FAIL is the return code of the executable. If it returns 0, the test PASSes, if it returns anything else, it FAILs. Python unittest tests ~~~~~~~~~~~~~~~~~~~~~ The discovery of classical Python unittest is also supported, although unlike Python unittest we still use static analysis to get individual tests so dynamically created cases are not recognized. Apart from that there should be no surprises when running unittests via Avocado. .. _Instrumented: Avocado Instrumented tests ~~~~~~~~~~~~~~~~~~~~~~~~~~ These are tests written in Python or BASH with the Avocado helpers that use the Avocado test API. To be more precise, the Python file must contain a class derived from :mod:`avocado.test.Test`. This means that an executable written in Python is not always an instrumented test, but may work as an executable test. The instrumented tests allows the writer finer control over the process including logging, test result status and other more sophisticated test APIs. Test statuses ``PASS``, ``WARN`` and ``SKIP`` are considered successful. The ``ERROR``, ``FAIL`` and ``INTERRUPTED`` signal failures. TAP producing tests ~~~~~~~~~~~~~~~~~~~ TAP tests are pretty much like executable tests in the sense that they are programs (either binaries or scripts) that will executed. The difference is that the test result will be decided based on the produced output, that should be in `Test Anything Protocol `_ format. Even though such executable can be seen as test suite from avocado point of view, it will be considered as one standalone test. If you want to get result of each test of such executable, you can get generated tap output in debug.log file. :ref:`avocado-log-files` .. note:: The result of Tap test is based on the importance of individual results types like this: `SKIP -> PASS -> FAIL`. This means if one of the tests in TAP output is `not ok` the TAP test result is `FAIL`. If all tests are `ok` or `skip` the results is `PASS` and if all results are `skip` the result is `SKIP`. Test statuses ------------- Avocado sticks to the following definitions of test statuses: * ``PASS``: The test passed, which means all conditions being tested have passed. * ``FAIL``: The test failed, which means at least one condition being tested has failed. Ideally, it should mean a problem in the software being tested has been found. * ``ERROR``: An error happened during the test execution. This can happen, for example, if there's a bug in the test runner, in its libraries or if a resource breaks unexpectedly. Uncaught exceptions in the test code will also result in this status. * ``SKIP``: The test runner decided a requested test should not be run. This can happen, for example, due to missing requirements in the test environment or when there's a job timeout. * ``WARN``: The test ran and something might have gone wrong but didn't explicitly failed. * ``CANCEL``: The test was canceled and didn't run. * ``INTERRUPTED``: The test was explicitly interrupted. Usually this means that a user hit CTRL+C while the job was still running or did not finish before the timeout specified. Exit codes ---------- Avocado exit code tries to represent different things that can happen during an execution. That means exit codes can be a combination of codes that were ORed together as a single exit code. The final exit code can be de-bundled so users can have a good idea on what happened to the job. The single individual exit codes are: * :data:`AVOCADO_ALL_OK ` * :data:`AVOCADO_TESTS_FAIL ` * :data:`AVOCADO_JOB_FAIL ` * :data:`AVOCADO_FAIL ` * :data:`AVOCADO_JOB_INTERRUPTED ` If a job finishes with exit code `9`, for example, it means we had at least one test that failed and also we had at some point a job interruption, probably due to the job timeout or a `CTRL+C`. avocado-113.0/docs/source/guides/user/chapters/configuring.rst000066400000000000000000000212151513561306000244760ustar00rootroot00000000000000Configuring =========== .. warning:: Please, keep in mind that we are doing a significant refactoring on settings to have consistency when using Avocado. Some options are changing soon. Avocado utilities have a certain default behavior based on educated, reasonable (we hope) guesses about how users like to use their systems. Of course, different people will have different needs and/or dislike our defaults, and that's why a configuration system is in place to help with those cases The Avocado config file format is based on the (informal) `INI`_ file specification, that is implemented by Python's :mod:`configparser`. The format is simple and straightforward, composed by `sections`, that contain a number of `keys` and `values`. Take for example a basic Avocado config file: .. code-block:: ini [datadir.paths] base_dir = /var/lib/avocado test_dir = /usr/share/doc/avocado/tests data_dir = /var/lib/avocado/data logs_dir = ~/avocado/job-results The ``datadir.paths`` section contains a number of keys, all of them related to directories used by the test runner. The ``base_dir`` is the base directory to other important Avocado directories, such as log, data and test directories. You can also choose to set those other important directories by means of the variables ``test_dir``, ``data_dir`` and ``logs_dir``. You can do this by simply editing the config files available. Config file parsing order ------------------------- Avocado starts by parsing what it calls system wide config file, that is shipped to all Avocado users on a system wide directory, ``/etc/avocado/avocado.conf`` (when installed by your Linux distribution's package manager). There is another directory that will be scanned by extra config files, ``/etc/avocado/conf.d``. This directory may contain plugin config files, and extra additional config files that the system administrator/avocado developers might judge necessary to put there. Then it'll verify if there's a local user config file, that is located usually in ``~/.config/avocado/avocado.conf``. The order of the parsing matters, so the system wide file is parsed, then the user config file is parsed last, so that the user can override values at will. The order of files described in this section is only valid if Avocado was installed in the system. For people using Avocado from git repos (usually Avocado developers), that did not install it in the system, keep in mind that Avocado will read the config files present in the git repos, and will ignore the system wide config files. Running ``avocado config`` will let you know which files are actually being used. Configuring via command-line ---------------------------- Besides the configuration files, the most used features can also be configured by command-line arguments. For instance, regardless what you have on your configuration files, you can disable sysinfo logging by running: .. code-block:: shell $ avocado run --disable-sysinfo /bin/true So, command-line options always will have the highest precedence during the configuration parsing. Use this if you would like to change some behavior on just one or a few specific executions. Parsing order recap ------------------- So the file parsing order is: * ``/etc/avocado/avocado.conf`` * ``/etc/avocado/conf.d/*.conf`` * ``avocado.plugins.settings`` plugins (but they can insert to any location) - For more information about this, visit the "Contributor's Guide" section named :ref:`writing_plugin` * ``~/.config/avocado/avocado.conf`` You can see the actual set of files/location by using ``avocado config`` which uses ``*`` to mark existing and used files:: $ avocado config Config files read (in order, '*' means the file exists and had been read): * /etc/avocado/avocado.conf * /etc/avocado/conf.d/resultsdb.conf * /etc/avocado/conf.d/result_upload.conf * /etc/avocado/conf.d/jobscripts.conf * /etc/avocado/conf.d/gdb.conf * /etc/avocado_vt/conf.d/vt.conf * /etc/avocado_vt/conf.d/vt_joblock.conf $HOME/.config/avocado/avocado.conf Section.Key Value datadir.paths.base_dir /var/lib/avocado datadir.paths.test_dir /usr/share/doc/avocado/tests ... Where the lower config files override values of the upper files and the ``$HOME/.config/avocado/avocado.conf`` file missing. .. note:: Please note that if Avocado is running from git repos, those files will be ignored in favor of in tree configuration files. This is something that would normally only affect people developing avocado, and if you are in doubt, ``avocado config`` will tell you exactly which files are being used in any given situation. .. note:: When Avocado runs inside virtualenv than path for global config files is also changed. For example, ``avocado.conf`` comes from the virual-env path ``venv/etc/avocado/avocado.conf``. Order of precedence for values used in tests -------------------------------------------- Since you can use the config system to alter behavior and values used in tests (think paths to test programs, for example), we established the following order of precedence for variables (from least precedence to most): * default value (from library or test code) * global config file * local (user) config file * command line switch * test parameters So the least important value comes from the library or test code default, going all the way up to the test parameters system. Supported data types when configuring Avocado --------------------------------------------- As already said before, Avocado allows users to use both: configuration files and command-line options to configure its behavior. It is important to have a very well defined system type for the configuration file and argument options. Although config files options and command-line arguments are always considered ``strings``, you should give a proper format representation so those values can be parsed into a proper type internally on Avocado. Currently Avocado supports the following data types for the configuration options: ``string``, ``integer``, ``float``, ``bool`` and ``list``. Besides those primitive data types Avocado also supports custom data types that can be used by a particular plugin. Below, you will find information on how to set options based on those basic data types using both: configuration files and command-line arguments. Strings ~~~~~~~ Strings are the basic ones and the syntax is the same in both configuration files and command-line arguments: Just the string that can be inside ``""`` or ``''``. Example using the configuration file: .. code-block:: ini [foo] bar = 'hello world' String and all following types could be used with or without quotes but using quotes for strings is important on the command line to safely handle empty spaces and distinguish it from a list type. Therefore, the following example will also be well handled: .. code-block:: ini [foo] bar = hello world Example using the command-line: .. code-block:: bash $ avocado run --foo bar /bin/true Integers ~~~~~~~~ Integer numbers are as simple as strings. Example using the configuration file: .. code-block:: ini [run] job_timeout = 60 Example using the command-line: .. code-block:: bash $ avocado run --job-timeout 50 /bin/true Floats ~~~~~~ Float numbers has the same representation as integers, but you should use `.` (dot) to separate the decimals. i.e: `80.3`. Booleans ~~~~~~~~ When talking about configuration files, accepted values for a boolean option are '1', 'yes', 'true', and 'on', which cause this method to return True, and '0', 'no', 'false', and 'off', which cause it to return False. But, when talking about command-line, booleans options don't need any argument, the option itself will enable or disable the settings, depending on the context. Example using the configuration file: .. code-block:: ini [core] verbose = true Example using the command-line: .. code-block:: bash $ avocado run --verbose /bin/true Lists ~~~~~ Lists are peculiar when configuring. On configuration files you can use the default "python" syntax for lists: ``["foo", "bar"]``, but when using the command-line arguments lists are strings separated by spaces: Example using the configuration file: .. code-block:: ini [assets.fetch] references = ["foo.py", "bar.py"] Example using the command-line: .. code-block:: bash $ avocado assets fetch foo.py bar.py Complete Configuration Reference -------------------------------- For a complete configuration reference, please visit :ref:`config-reference`. .. _INI: http://en.wikipedia.org/wiki/INI_file Or you can see in your terminal, typing:: $ avocado config reference avocado-113.0/docs/source/guides/user/chapters/datadirs.rst000066400000000000000000000041121513561306000237540ustar00rootroot00000000000000Avocado Data Directories ======================== When running tests, we are frequently looking to: * Locate tests * Write logs to a given location * Grab files that will be useful for tests, such as ISO files or VM disk images Avocado has a module dedicated to finding those paths, to avoid cumbersome path manipulation magic. If you want to list all relevant directories for your test, you can use ``avocado config --datadir`` command to list those directories. Executing it will give you an output similar to the one seen below:: $ avocado config --datadir Config files read (in order): * /etc/avocado/avocado.conf * /etc/avocado/conf.d/resultsdb.conf * /etc/avocado/conf.d/result_upload.conf * /etc/avocado/conf.d/jobscripts.conf * /etc/avocado/conf.d/gdb.conf $HOME/.config/avocado/avocado.conf Avocado replaces config dirs that can't be accessed with sensible defaults. Please edit your local config file to customize values. Avocado Data Directories: base $HOME/avocado tests $HOME/Code/avocado/examples/tests data $HOME/avocado/data logs $HOME/avocado/job-results cache $HOME/avocado/data/cache Note that, while Avocado will do its best to use the config values you provide in the config file, if it can't write values to the locations provided, it will fall back to (we hope) reasonable defaults, and we notify the user about that in the output of the command. The relevant API documentation and meaning of each of those data directories is in :mod:`avocado.core.data_dir`, so it's highly recommended you take a look. You may set your preferred data dirs by setting them in the Avocado config files. The only exception for important data dirs here is the Avocado tmp dir, used to place temporary files used by tests. That directory will be in normal circumstances ``/var/tmp/avocado_XXXXX``, (where ``XXXXX`` is in actuality a random string) securely created on ``/var/tmp/``, unless the user has the ``$TMPDIR`` environment variable set, since that is customary among unix programs. avocado-113.0/docs/source/guides/user/chapters/dependencies.rst000066400000000000000000000203551513561306000246160ustar00rootroot00000000000000.. _managing-requirements: Managing Dependencies ===================== A test's dependency can be fulfilled by the Dependencies Resolver feature. Test's dependencies are specified in the test definition and are fulfilled based on the supported dependency `type`. Test workflow with dependencies ------------------------------- When a dependency is defined for a test, it is marked as a dependency for that test. The test will wait for all the dependencies to complete successfully before it is started. When any of the dependencies defined on a test fails, the test is skipped. When the dependency is fulfilled, its metadata will be saved into the avocado cache, so avocado will be able to reuse it in another tests. Also, the dependency metadata will stay in cache after the Avocado run, so the second run of the tests will use dependencies from cache, which will make tests more efficient. If you want to know the state of cache, you can use cache interface with `avocado cache list`. .. warning:: `avocado cache` interface works only with metadata about dependencies. Any manipulation with `avocado cache` interface doesn't affects the real data stored in the environment. .. warning:: If any environment is modified without Avocado knowing about it (packages being uninstalled, podman images removed, etc), the dependency metadata in cache won't be updated, because of this, the resolution behavior is undefined and will probably crash. If such a change is made to the environment, it's recommended to clear the dependencies cache with `$avocado cache clear`. Dependency logs --------------- Each dependency will create its own log directory where you can find logs related to this dependence. Dependencies logs related to one job are stored in `{avocado_dir}/job-results/{job}/dependencies` and for each dependence logs the symlink to correct test logs directory is created. Therefore, if your test has dependencies, you can find dependency logs in `{avocado_dir}/job-results/{job}/test-results/{test_id}/dependencies` Defining a test dependency --------------------------- A test dependency is described in the JSON format. Following is an example of a dependency of `type` `package`:: {"type": "package", "name": "hello"} To define a dependency for the test, use the test's docstring with the format of keywords ``:avocado: dependency=``. The following example shows the same package dependency showed above inside a test docstring:: from avocado import Test class PassTest(Test): """ :avocado: dependency={"type": "package", "name": "hello"} """ def test(self): """ A success test """ It is possible to define multiple dependencies for a test. Following is an example using more than one dependency definition:: from avocado import Test class PassTest(Test): """ :avocado: dependency={"type": "package", "name": "hello"} :avocado: dependency={"type": "package", "name": "bash"} """ def test(self): """ A success test """ Defining a dependency in the class docstring will fulfill the dependency for every test within a test class. Defining a dependency in the test docstring will fulfill the dependency for that single test only. Defining a job dependency ------------------------- Managing dependencies across multiple tests within a job can be streamlined using the Job Dependency Feature. If your tests share common dependencies, there's no need to specify them individually for each test. Instead, you can utilize the Job Dependency Feature to apply a set of dependencies to every test within the job. Once it is enabled, Avocado will read the list of common dependencies from the specified JSON file and automatically apply them to each test within the job. Enabling the Job Dependency +++++++++++++++++++++++++++ To enable this feature, you have two options: 1. Command-line Option: Use of the `--job-dependency` option of the avocado run command to specify the path to the dependency JSON file:: avocado run --job-dependency=dependencies.json tests/ 2. Configuration File: Alternatively, you can set the path to the dependency JSON file using the `job.run.dependency` configuration option in the Avocado configuration file. .. tip:: If you're using the Job API, you have the flexibility to define different dependencies for each suite. Simply modify the `job.run.dependency` value in the suite configuration during suite creation. Dependency File Format ++++++++++++++++++++++ The dependency file follows the same format as the test dependency defined in the docstring. If you have multiple dependencies, ensure to encapsulate them within a JSON list:: [ {"type": "package", "name": "hello"}, {"type": "package", "name": "bash"} ] Supported types of dependencies ------------------------------- The following `types` of dependencies are supported: Package +++++++ Support managing of packages using the Avocado Software Manager utility. The parameters available to use the package `type` of dependencies are: * `type`: `package` * `name`: the package name (required) * `action`: one of `install`, `check`, or `remove` (optional, defaults to `install`) Following is an example of a test using the Package dependency: .. literalinclude:: ../../../../../examples/tests/passtest_with_dependency.py Pip +++ Support managing python packages via pip. The parameters available to use the asset `type` of dependencies are: * `type`: `pip` * `name`: the package name (required) * `action`: `install` or `uninstall` (optional, defaults to `install`) Asset +++++ Support fetching assets using the Avocado Assets utility. The parameters available to use the asset `type` of dependencies are: * `type`: `asset` * `name`: the file name or uri (required) * `asset_has`: hash of the file (optional) * `algorithm`: hash algorithm (optional) * `locations`: location(s) where the file can be fetched from (optional) * `expire`: time in seconds for the asset to expire (optional) Podman Image ++++++++++++ Support pulling podman images ahead of test execution time. This should only be used explicitly if a test interacts with ``podman`` directly, say by executing containers on its own. If you are using the ``podman`` spawner (``--spawner=podman``) this will have no effect on the spawner. * `type`: `podman-image` * `uri`: the image reference, in any format supported by ``podman pull`` itself. .. _dependency-vmimage: VM Image ++++++++ Support downloading virtual machine images ahead of test execution time. This allows tests to have their required VM images downloaded and cached before the test execution begins, preventing timeout issues during the actual test run. * `type`: `vmimage` * `provider`: the VM image provider (e.g., 'Fedora') * `version`: version of the image (Optional) * `arch`: architecture of the image (Optional) Following is an example of tests using the VM Image dependency that demonstrates different use cases including multiple dependencies and different providers: .. literalinclude:: ../../../../../examples/tests/dependency_vmimage.py To test the VM Image dependency: 1. Run the tests:: $ avocado run examples/tests/dependency_vmimage.py 2. Check the cache to see downloaded images:: $ avocado cache list 3. Run again to verify cache is used:: $ avocado run examples/tests/dependency_vmimage.py The vmimage runner will automatically download required images if they're not in the cache and use cached images when available, without needing to clear the cache first. Ansible Module ++++++++++++++ If you install the Ansible plugin (``avocado-framework-plugin-ansible`` from PIP or ``python3-avocado-plugins-ansible`` from RPM packages), you will will be able to use the ``ansible-module`` dependency. * `type`: `ansible-module` * `uri`: the name of the ansible module. All other arguments will be treated as arguments to the ansible modules. Following is an example of tests using ansible's ``file`` and ``user`` modules: .. literalinclude:: ../../../../../examples/tests/dependency_ansible.py avocado-113.0/docs/source/guides/user/chapters/installing.rst000066400000000000000000000105661513561306000243370ustar00rootroot00000000000000.. _installing: Installing ========== Avocado is primarily written in Python, so a standard Python installation is possible and often preferable. You can also install from your Linux distribution repository, if available. .. note:: Please note that this installs the Avocado core functionality. Many Avocado features are distributed as non-core plugins. Visit the Avocado Plugin section on the left menu. .. tip:: If you are looking for Virtualization specific testing, also consider looking at Avocado-VT_ installation instructions after finishing the Avocado installation. Installing from PyPI -------------------- The simplest installation method is through ``pip``. On most POSIX systems with Python 3.9 (or later) and ``pip`` available, installation can be performed with a single command:: $ pip3 install --user avocado-framework This will fetch the Avocado package (and possibly some of its dependencies) from the PyPI repository, and will attempt to install it in the user's home directory (usually under ``~/.local``), which you might want to add to your ``PATH`` variable if not done already. .. tip:: If you want to perform a system-wide installation, drop the ``--user`` switch. If you want even more isolation, Avocado can also be installed in a Python virtual environment. with no additional steps besides creating and activating the "venv" itself:: $ python3 -m venv /path/to/new/virtual_environment $ source /path/to/new/virtual_environment/bin/activate $ pip3 install avocado-framework .. _installing-from-packages: Installing from packages ------------------------ .. _fedora-from-avocados-own-repo: Fedora ~~~~~~ Avocado is available as a standard Fedora package. Simply run:: $ dnf install python3-avocado The exact version of Avocado is dependent on the Fedora version and its release constraints. If you're looking to have the latest Avocado release, please use Avocado's COPR repo, by running:: $ dnf copr enable @avocado/avocado-latest-release $ dnf install python3-avocado Enterprise Linux ~~~~~~~~~~~~~~~~ The latest release of Avocado is available on the same COPR repo described previously. To install the latest Avocado release on Enterprise Linux 9, run:: $ dnf copr enable @avocado/avocado-latest-release $ dnf install python3-avocado Latest Development RPM Packages from COPR ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Avocado provides a repository of continuously built packages from the GitHub repository's master branch. These packages are currently available for some of the latest Enterprise Linux and Fedora versions, for a few different architectures. If you're interested in using the very latest development version of Avocado from RPM packages, you can do so by running:: $ dnf copr enable @avocado/avocado-latest $ dnf install python3-avocado* The following image shows the status of the Avocado packages building on COPR: .. image:: https://copr.fedorainfracloud.org/coprs/g/avocado/avocado-latest/package/python-avocado/status_image/last_build.png :target: https://copr.fedorainfracloud.org/coprs/g/avocado/avocado-latest/package/python-avocado/ OpenSUSE ~~~~~~~~ The OpenSUSE project provides packages for Avocado. Check the `Virtualization:Tests project in OpenSUSE build service`_ to get the packages from there. Debian ~~~~~~ DEB package support is available in the source tree (look at the ``contrib/packages/debian`` directory. No actual packages are provided by the Avocado project or the Debian repos. Installing from source code --------------------------- First make sure you have a basic set of packages installed. The following applies to Fedora based distributions, please adapt to your platform:: $ sudo dnf install -y python3 git gcc python3-pip Then to install Avocado from the git repository run:: $ git clone git://github.com/avocado-framework/avocado.git $ cd avocado $ pip install . --user To install an optional plugin run:: $ pip install optional_plugins/ --user I.e. for the HTML plugin:: $ pip install optional_plugins/html --user Check the directory ``optional_plugins`` for additional features you might be interested in. .. _Virtualization:Tests project in OpenSUSE build service: https://build.opensuse.org/project/show/Virtualization:Tests .. _Avocado-VT: https://avocado-vt.readthedocs.io/en/latest/GetStartedGuide.html#installing-avocado-vt avocado-113.0/docs/source/guides/user/chapters/introduction.rst000066400000000000000000000613721513561306000247150ustar00rootroot00000000000000Introduction ============ Avocado Hello World ------------------- You should first experience Avocado by using the test runner, that is, the command line tool that will conveniently run your tests and collect their results. To do so, please run ``avocado`` with the ``run`` sub-command followed by a test reference, which could be either a path to the file, or a recognizable name:: $ avocado run /bin/true JOB ID : 89b5d609d4832c784f04cf14f4ec2d17917d419a JOB LOG : $HOME/avocado/job-results/job-2023-09-06T15.52-89b5d60/job.log (1/1) /bin/true: STARTED (1/1) /bin/true: PASS (0.01 s) RESULTS : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0 JOB HTML : $HOME/avocado/job-results/job-2023-09-06T15.52-89b5d60/results.html JOB TIME : 1.49 s You probably noticed that we used ``/bin/true`` as a test, and in accordance with our expectations, it passed! These are known as `executable tests` (``exec-test``), but there is also another type of test, which we call `instrumented tests` . See more at `test-types` or just keep reading. Running a job with multiple tests --------------------------------- You can run any number of test in an arbitrary order, as well as mix and match instrumented and executable tests:: $ avocado run examples/tests/sleeptest.py examples/tests/failtest.py /bin/true JOB ID : 2391dddf53b950631589bd1d44a5a6fdd023b400 JOB LOG : $HOME/avocado/job-results/job-2021-09-27T16.35-2391ddd/job.log (3/3) /bin/true: STARTED (1/3) examples/tests/sleeptest.py:SleepTest.test: STARTED (3/3) /bin/true: PASS (0.01 s) (2/3) examples/tests/failtest.py:FailTest.test: STARTED (2/3) examples/tests/failtest.py:FailTest.test: FAIL: This test is supposed to fail (0.02 s) (1/3) examples/tests/sleeptest.py:SleepTest.test: PASS (1.01 s) RESULTS : PASS 2 | ERROR 0 | FAIL 1 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0 JOB TIME : 4.19 s .. note:: Although in most cases running ``avocado run $test1 $test3 ...`` is fine, it can lead to argument vs. test name clashes. The safest way to execute tests is ``avocado run --$argument1 --$argument2 -- $test1 $test2``. Everything after ``--`` will be considered positional arguments, therefore test names (in case of ``avocado run``) Using a different runner ------------------------ Currently Avocado has only one runner: ``nrunner`` (the new runner) But some plugins may use their own runners. You can find a list of current runners installed with the ``avocado plugins`` command:: $ avocado plugins Plugins that run test suites on a job (runners): nrunner nrunner based implementation of job compliant runner During the test execution, you can select the runner using the option ``--suite-runner``, where the default is the nrunner one:: $ avocado run --suite-runner='runner' /bin/true Interrupting tests ------------------ .. _signal_hanlders: Sending Signals ~~~~~~~~~~~~~~~ To interrupt a job execution a user can press ``ctrl+c`` which after a single press sends ``SIGTERM`` to the main test's process and waits for it to finish. If this does not help user can press ``ctrl+c`` again (after 2s grace period) which destroys the test's process ungracefully and safely finishes the job execution always providing the test results. Interrupting the job on first fail (failfast) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The Avocado ``run`` command has the option ``--failfast`` to exit the job as soon as possible. Due to our current runner architecture, tests are executed in parallel by default. The ``--failfast`` option work on the best effort to cancel tests that have not started yet. To replicate the same behavior as the legacy runner, use ``--max-parallel-tasks=1`` to limit the number of tasks executed in parallel:: $ avocado run --failfast --max-parallel-tasks=1 /bin/true /bin/false /bin/true /bin/true JOB ID : 76bfe0e5cfa5efac3ab6881ee501cc5d4b69f913 JOB LOG : $HOME/avocado/job-results/job-2021-09-27T16.41-76bfe0e/job.log (1/4) /bin/true: STARTED (1/4) /bin/true: PASS (0.01 s) (2/4) /bin/false: STARTED (2/4) /bin/false: FAIL (0.01 s) Interrupting job (failfast). RESULTS : PASS 1 | ERROR 0 | FAIL 1 | SKIP 2 | WARN 0 | INTERRUPT 0 | CANCEL 0 JOB TIME : 1.57 s The default behavior, that is, when ``--failfast`` is **not** set, is to try to execute all tests in a job, regardless individual of test failures. .. note:: Avocado versions 80.0 and earlier allowed replayed jobs to override the failfast configuration by setting ``--failfast=off`` in a ``avocado replay ..`` command line. This is no longer possible. .. _the_hint_files: The hint files -------------- Avocado team has added support to the "hint files". This feature is present since Avocado #78 and is a configuration file that you can add to your project root folder to help Avocado on the "test resolution" phase. The idea is that, you know more about your tests than anybody else. And you can specify where your tests are, and what type (kind) they are. You just have to add a ``.avocado.hint`` in your root folder with the section ``[kinds]`` and one section for each kind that you are using. On the specific test type section, you can specify 3 options: ``uri``, ``args`` and ``kwargs``. .. note:: Some test types will convert ``kwargs`` into variable environments. Please check the documentation of the test type that you are using. You can also use the keyword ``$testpath`` in any of the options inside the test type section. Avocado will replace ``$testpath`` with your test path, after the expansion. For instance, below you will find a hint file example where we have only one test type ``TAP``: .. literalinclude:: ../../../../../examples/hint-files/.avocado.hint.example Let's suppose that you have 2 tests that matches ``./tests/unit/*.sh``: * ``./tests/unit/foo.sh`` * ``./tests/unit/bar.sh`` Avocado will run each one as a ``TAP`` test, as you desired. .. note:: Please, keep in mind that hint files needs absolute paths when defining tests inside the ``[kinds]`` section. Since Avocado's next runner is capable of running tests not only in a subprocess but also in more isolated environments such as Podman containers, sending custom environment variables to the task executor can be achieved by using the ``kwargs`` parameter. Use a comma-separated list of variables here and Avocado will make sure your tests will receive those variables (regardless of the spawner type). Ignoring missing test references -------------------------------- When you provide a list of test references, Avocado will try to resolve all of them to tests. If one or more test references can not be resolved to tests, the Job will not be created. Example:: $ avocado run examples/tests/passtest.py badtest.py No tests found for given test references: badtest.py Try 'avocado -V list badtest.py' for details But if you want to execute the Job anyway, with the tests that could be resolved, you can use ``--ignore-missing-references``, a boolean command-line option. The same message will appear in the UI, but the Job will be executed:: $ avocado run examples/tests/passtest.py badtest.py --ignore-missing-references JOB ID : e6d1f4d21d6a5e2e039f1acd1670a6882144c189 JOB LOG : $HOME/avocado/job-results/job-2021-09-27T16.50-e6d1f4d/job.log (1/1) examples/tests/passtest.py:PassTest.test: STARTED (1/1) examples/tests/passtest.py:PassTest.test: PASS (0.01 s) RESULTS : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0 JOB TIME : 1.49 s Runner outputs -------------- A test runner must provide an assortment of ways to clearly communicate results to interested parties, be them humans or machines. .. note:: There are several optional result plugins, you can find them in :ref:`result-plugins`. Results for human beings ~~~~~~~~~~~~~~~~~~~~~~~~ Avocado has two different result formats that are intended for human beings: * Its default UI, which shows the live test execution results on a command line, text based, UI. * The HTML report, which is generated after the test job finishes running. .. note:: The HTML report needs the ``html`` plugin enabled that is an optional plugin. A regular run of Avocado will present the test results in a live fashion, that is, the job and its test(s) results are constantly updated:: $ avocado run examples/tests/sleeptest.py examples/tests/failtest.py JOB ID : 2e83086e5d3f82dd68bdc8885e7cce1cebec5f27 JOB LOG : $HOME/avocado/job-results/job-2021-09-27T17.00-2e83086/job.log (1/2) examples/tests/sleeptest.py:SleepTest.test: STARTED (2/2) examples/tests/failtest.py:FailTest.test: STARTED (2/2) examples/tests/failtest.py:FailTest.test: FAIL: This test is supposed to fail (0.02 s) (1/2) examples/tests/sleeptest.py:SleepTest.test: PASS (1.01 s) RESULTS : PASS 1 | ERROR 0 | FAIL 1 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0 JOB HTML : $HOME/avocado/job-results/job-2021-09-27T17.00-2e83086/results.html JOB TIME : 2.80 s The most important thing is to remember that programs should never need to parse human output to figure out what happened to a test job run. As you can see, Avocado will print a nice UI with the job summary on the console. If you would like to inspect a detailed output of your tests, you can visit the folder: ``$HOME/avocado/job-results/latest/`` or a specific job folder. Results for machine ~~~~~~~~~~~~~~~~~~~ Another type of results are those intended to be parsed by other applications. Several standards exist in the test community, and Avocado can in theory support pretty much every result standard out there. Out of the box, Avocado supports a couple of machine readable results. They are always generated and stored in the results directory in ``results.$type`` files, but you can ask for a different location too. Currently, you can find three different formats available on this folder: **xUnit (XML)**, **JSON** and **TAP**. **1. xUnit:** The default machine readable output in Avocado is `xunit `__. xUnit is an XML format that contains test results in a structured form, and are used by other test automation projects, such as `jenkins `__. If you want to make Avocado to generate xunit output in the standard output of the runner, simply use:: $ avocado run examples/tests/sleeptest.py examples/tests/failtest.py --xunit - None [stdlog] 2023-12-21 08:42:54,721 avocado.test parameters L0141 DEBUG| PARAMS (key=timeout_factor, path=*, default=1.0) => 1.0 [stdlog] 2023-12-21 08:42:54,721 avocado.test test L0345 DEBUG| Test metadata: [stdlog] 2023-12-21 08:42:54,721 avocado.test test L0347 DEBUG| filename: $HOME/src/avocado/avocado/examples/tests/failtest.py [stdlog] 2023-12-21 08:42:54,721 avocado.test test L0353 DEBUG| teststmpdir: /var/tmp/avocado_tx32gp2p [stdlog] 2023-12-21 08:42:54,721 avocado.test test L0354 DEBUG| original timeout: None [stdlog] 2023-12-21 08:42:54,721 avocado.test test L0355 DEBUG| timeout factor: 1.0 [stdlog] 2023-12-21 08:42:54,721 avocado.test test L0356 DEBUG| actual timeout: None [stdlog] 2023-12-21 08:42:54,722 avocado.test test L0548 INFO | START 1-examples/tests/failtest.py:FailTest.test [stdlog] 2023-12-21 08:42:54,722 avocado.test stacktrace L0040 ERROR| [stdlog] 2023-12-21 08:42:54,722 avocado.test stacktrace L0042 ERROR| Reproduced traceback from: $HOME/src/avocado/avocado/avocado/core/test.py:638 [stdlog] 2023-12-21 08:42:54,723 avocado.test stacktrace L0049 ERROR| Traceback (most recent call last): [stdlog] 2023-12-21 08:42:54,723 avocado.test stacktrace L0049 ERROR| File "$HOME/src/avocado/avocado/examples/tests/failtest.py", line 16, in test [stdlog] 2023-12-21 08:42:54,723 avocado.test stacktrace L0049 ERROR| self.fail("This test is supposed to fail") [stdlog] 2023-12-21 08:42:54,723 avocado.test stacktrace L0049 ERROR| File "$HOME/src/avocado/avocado/avocado/core/test.py", line 785, in wrapper [stdlog] 2023-12-21 08:42:54,723 avocado.test stacktrace L0049 ERROR| return func(actual_message) [stdlog] 2023-12-21 08:42:54,723 avocado.test stacktrace L0049 ERROR| ^^^^^^^^^^^^^^^^^^^^ [stdlog] 2023-12-21 08:42:54,723 avocado.test stacktrace L0049 ERROR| File "$HOME/src/avocado/avocado/avocado/core/test.py", line 801, in fail [stdlog] 2023-12-21 08:42:54,723 avocado.test stacktrace L0049 ERROR| raise exceptions.TestFail(msg) [stdlog] 2023-12-21 08:42:54,723 avocado.test stacktrace L0049 ERROR| avocado.core.exceptions.TestFail: This test is supposed to fail [stdlog] 2023-12-21 08:42:54,723 avocado.test stacktrace L0050 ERROR| [stdlog] 2023-12-21 08:42:54,723 avocado.test test L0642 DEBUG| Local variables: [stdlog] 2023-12-21 08:42:54,735 avocado.test test L0645 DEBUG| -> self : 1-examples/tests/failtest.py:FailTest.test [stdlog] 2023-12-21 08:42:54,736 avocado.test test L0740 ERROR| FAIL 1-examples/tests/failtest.py:FailTest.test -> TestFail: This test is supposed to fail [stdlog] 2023-12-21 08:42:54,736 avocado.test test L0733 INFO | ]]> .. note:: The dash ``-`` in the option ``--xunit``, it means that the xunit result should go to the standard output. .. note:: In case your tests produce very long outputs, you can limit the number of embedded characters by ``--xunit-max-test-log-chars``. If the output in the log file is longer it only attaches up-to max-test-log-chars characters one half starting from the beginning of the content, the other half from the end of the content. **2. JSON:** `JSON `__ is a widely used data exchange format. The JSON Avocado plugin outputs job information, similarly to the xunit output plugin:: $ avocado run examples/tests/sleeptest.py examples/tests/failtest.py --json - { "cancel": 0, "debuglog": "$HOME/avocado/job-results/job-2024-02-08T16.24-bee291e/job.log", "errors": 0, "failures": 1, "interrupt": 0, "job_id": "bee291e496e7a54b3e81e3836abd860aa779d37b", "pass": 1, "skip": 0, "start": "2024-02-08 16:24:48.169658", "tests": [ { "actual_time_end": 1707405889.5167582, "actual_time_start": 1707405889.207349, "fail_reason": "This test is supposed to fail", "id": "2-examples/tests/failtest.py:FailTest.test", "logdir": "$HOME/avocado/job-results/job-2024-02-08T16.24-bee291e/test-results/2-examples_tests_failtest.py_FailTest.test", "logfile": "$HOME/avocado/job-results/job-2024-02-08T16.24-bee291e/test-results/2-examples_tests_failtest.py_FailTest.test/debug.log", "name": "examples/tests/failtest.py:FailTest.test", "status": "FAIL", "tags": { "failure_expected": null }, "time_elapsed": 0.038685322972014546, "time_end": 889039.845678904, "time_start": 889039.806993581, "whiteboard": "" }, { "actual_time_end": 1707405890.2360377, "actual_time_start": 1707405889.2069798, "fail_reason": "", "id": "1-examples/tests/sleeptest.py:SleepTest.test", "logdir": "$HOME/avocado/job-results/job-2024-02-08T16.24-bee291e/test-results/1-examples_tests_sleeptest.py_SleepTest.test", "logfile": "$HOME/avocado/job-results/job-2024-02-08T16.24-bee291e/test-results/1-examples_tests_sleeptest.py_SleepTest.test/debug.log", "name": "examples/tests/sleeptest.py:SleepTest.test", "status": "PASS", "tags": {}, "time_elapsed": 1.011205994989723, "time_end": 889040.816592906, "time_start": 889039.805386911, "whiteboard": "" } ], "time": 1.0498913179617375, "total": 2, "warn": 0 } .. note:: The dash ``-`` in the option ``--json``, it means that the xunit result should go to the standard output. Bear in mind that there's no documented standard for the Avocado JSON result format. This means that it will probably grow organically to accommodate newer Avocado features. A reasonable effort will be made to not break backwards compatibility with applications that parse the current form of its JSON result. **3. TAP:** Provides the basic `TAP `__ (Test Anything Protocol) results, currently in v12. Unlike most existing Avocado machine readable outputs this one is streamlined (per test results):: $ avocado run examples/tests/sleeptest.py --tap - 1..1 ok 1 examples/tests/sleeptest.py:SleepTest.test **4. Beaker:** When avocaodo finds itself running inside a beaker task the beaker_report plugin will send the test results and log files to the beaker lab controller. Happens automatically, no configuration required. Check the `project website `__ for more information on beaker. Using the option --show ~~~~~~~~~~~~~~~~~~~~~~~ Probably, you frequently want to look straight at the job log, without switching screens or having to "tail" the job log. In order to do that, you can use ``avocado --show=job run ...``:: $ avocado --show=job run examples/tests/sleeptest.py ... avocado.job: Job ID: c9ca6a96d34fd0355f5f121af7fa87eef734a196 avocado.job: avocado.job: examples/tests/sleeptest.py:SleepTest.test: STARTED avocado.job: examples/tests/sleeptest.py:SleepTest.test: PASS avocado.job: More information in $HOME/avocado/job-results/job-2023-09-08T15.27-c9ca6a9/test-results/1-examples_tests_sleeptest.py_SleepTest.test avocado.job: Test results available in $HOME/avocado/job-results/job-2023-09-08T15.27-c9ca6a9 As you can see, the UI output is suppressed and only the job log is shown, making this a useful feature for test development and debugging. It's possible to silence all output to stdout (while keeping the error messages being printed to stderr). One can then use the return code to learn about the result:: $ avocado --show=none run examples/tests/failtest.py $ echo $? 1 In practice, this would usually be used by scripts that will in turn run Avocado and check its results:: #!/bin/bash ... $ avocado --show=none run /path/to/my/test.py if [ $? == 0 ]; then echo "great success!" elif ... more details regarding exit codes in `Exit Codes` section. Multiple results at once ~~~~~~~~~~~~~~~~~~~~~~~~ You can have multiple results formats at once, as long as only one of them uses the standard output. For example, it is fine to use the xunit result on stdout and the JSON result to output to a file:: $ avocado run examples/tests/sleeptest.py /bin/true --xunit - --json /tmp/result.json $ cat /tmp/result.json { "cancel": 0, "debuglog": "$HOME/avocado/job-results/job-2023-12-21T08.46-a4f410d/job.log", "errors": 0, "failures": 0, "interrupt": 0, "job_id": "a4f410de7969bc8d6148712796e027b110027fa3", "pass": 2, "skip": 0, "start": "2023-12-21 08:46:21.078811", ... } But you won't be able to do the same without the ``--json`` flag passed to the program:: avocado run examples/tests/sleeptest.py /bin/true --xunit - --json - avocado run: error: argument --json: Options --xunit --json are trying to use stdout simultaneously. Please set at least one of them to a file to avoid conflicts That's basically the only rule, and a sane one, that you need to follow. .. note:: Avocado support "paginator" option, which, on compatible terminals, basically pipes the colored output to ``less`` to simplify browsing of the produced output. You an enable it with ``--enable-paginator``. .. _sysinfo-collection: Sysinfo collection ------------------ Avocado comes with a ``sysinfo`` plugin, which automatically gathers some system information per each job or even between tests. This is very useful when later we want to know what caused the test's failure. This system is configurable but we provide a sane set of defaults for you. In the default Avocado configuration (``/etc/avocado/avocado.conf``) there is a section ``sysinfo.collect`` where you can enable/disable the sysinfo collection as well as configure the basic environment. In ``sysinfo.collectibles`` section you can define basic paths of where to look for what commands/tasks should be performed before/during the sysinfo collection. Avocado supports three types of tasks: 1. commands - file with new-line separated list of commands to be executed before and after the job/test (single execution commands). It is possible to set a timeout which is enforced per each executed command in ``[sysinfo.collect]`` by setting "commands_timeout" to a positive number. You can also use the environment variable ``AVOCADO_SYSINFODIR`` which points to the sysinfo directory in results. 2. fail_commands - similar to commands, but gets executed only when the test fails. 3. files - file with new-line separated list of files to be copied. 4. fail_files - similar to files, but copied only when the test fails. 5. profilers - file with new-line separated list of commands to be executed before the job/test and killed at the end of the job/test (follow-like commands). Additionally this plugin tries to follow the system log via ``journalctl`` if available. By default these are collected per-job but you can also run them per-test by setting ``per_test = True`` in the ``sysinfo.collect`` section. The sysinfo is enabled by default and can also be disabled on the cmdline if needed by ``--disable-sysinfo``. After the job execution you can find the collected information in ``$RESULTS/sysinfo`` of ``$RESULTS/test-results/$TEST/sysinfo``. They are categorized into ``pre``, ``post`` and ``profile`` folders and the filenames are safely-escaped executed commands or file-names. You can also see the sysinfo in html results when you have html results plugin enabled. It is also possible to save only the files / commands which were changed during the course of the test, in the ``post`` directory, using the setting ``optimize = True`` in the ``sysinfo.collect`` section. This collects all sysinfo on ``pre``, but saves only changed ones on ``post``. It is set to False by default. .. warning:: If you are using Avocado from sources, you need to manually place ``commands``/``fail_commands``/``fail_files``/``files``/``profilers`` into the ``/etc/avocado/sysinfo`` directory or adjust the paths in your ``avocado.conf``. avocado-113.0/docs/source/guides/user/chapters/logging.rst000066400000000000000000000063621513561306000236200ustar00rootroot00000000000000Avocado logging system ====================== This section describes basic of the logging system used in Avocado. For more detailed information about avocado logging system, please go to :ref:`advanced-logging`. Tweaking the UI --------------- Avocado uses Python's logging system to produce UI and to store test's output. The system is quite flexible and allows you to tweak the output to your needs either by built-in stream sets, or directly by using the stream name. To tweak them you can use:: $ avocado --show STREAM[:LEVEL][,STREAM[:LEVEL][,...] Built-in streams with description (followed by list of associated Python streams) are listed below: :app: The text based UI (avocado.app) :test: Output of the executed tests (avocado.test, "") :job: Avocado job related logs (avocado.job) :early: Early logging before the logging system is set. It includes the test output and lots of output produced by used libraries. ("", avocado.test) Additionally you can specify "all" or "none" to enable/disable all of pre-defined streams and you can also supply custom Python logging streams and they will be passed to the standard output. .. warning:: Messages with importance greater or equal WARN in logging stream "avocado.app" are always enabled and they go to the standard error output. Storing custom logs ------------------- When you run a test, you can also store custom logging streams into the results directory by running:: $ avocado run --store-logging-stream STREAM[:LEVEL][,STREAM[:LEVEL][,...] This will produce ``$STREAM.$LEVEL`` files per each (unique) entry in two places. One in each test result directory where the stream is separated by test and one in job log directory where the stream will be gathered from all tests into one place. This might be useful when you don't use parallel run and you want to have logs from all tests in one place. .. note:: You have to specify separated logging streams. You can't use the built-in streams in this function. .. _avocado-log-files: Avocado log files ----------------- Avocado use three main files for logging information on different levels. All of them are stored in ``{avocado_dir}/job-results/{job}/``: :job.log: Logs related to job run under avocado.job namespace. :full.log: All generated logs in one job run. It contains all logs from avocado internal to logs from all tests. If you run your tests in parallel the logs might be overlap since the logs are not sorted and they are stored at the time when they have been created. :debug.log: Logs generated by one specific test and they are separated from other tests logs. Logs are stored in ``{avocado_dir}/job-results/{job}/test-results/{test_id}/debug.log`` Custom loggers and levels ------------------------- A Python based test, either directly or indirectly though a library, can create and use custom loggers and levels. These log messages will go to the ``full.log`` file of a job, but you can also use features such as the ``--store-logging-stream`` option documented earlier to have these log messages in a file of its own. One example test is included in Avocado's source tree and can be run with:: $ avocado run --store-logging-stream custom_logger:35 examples/tests/logging_custom_level.py avocado-113.0/docs/source/guides/user/chapters/operations.rst000066400000000000000000000173541513561306000243600ustar00rootroot00000000000000Basic Operations ================ Job Replay ---------- The process of replaying an Avocado Job is simply about loading the source Job's configuration and running a new Job based on that configuration. For users, this is available as the ``avocado replay`` command. Its usage is straightforward. Suppose you've just run a simple job, also from the command line, such as:: $ avocado run /bin/true /bin/false JOB ID : 42c60bea72e6d55756bfc784eb2b354f788541cf JOB LOG : $HOME/avocado/job-results/job-2020-08-13T11.23-42c60be/job.log (1/2) /bin/true: STARTED (2/2) /bin/false: STARTED (1/2) /bin/true: PASS (0.01 s) (2/2) /bin/false: FAIL (0.01 s) RESULTS : PASS 1 | ERROR 0 | FAIL 1 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0 JOB HTML : $HOME/avocado/job-results/job-2020-08-13T11.23-42c60be/results.html JOB TIME : 0.41 s To run a new job with the configuration used by the previously executed job, it's possible to simply execute:: $ avocado replay latest Resulting in:: JOB ID : f3139826f1b169a0b456e0e880ffb83ed26d9858 SRC JOB ID : latest JOB LOG : $HOME/avocado/job-results/job-2020-08-13T11.24-f313982/job.log (1-2/2) /bin/false: STARTED (1-1/2) /bin/true: STARTED (1-2/2) /bin/false: FAIL (0.01 s) (1-1/2) /bin/true: PASS (0.01 s) RESULTS : PASS 1 | ERROR 0 | FAIL 1 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0 JOB HTML : $HOME/avocado/job-results/job-2020-08-13T11.24-f313982/results.html JOB TIME : 0.39 s It's also possible to use the other types of references to jobs, like the full directory path of the job results, or the Job IDs. That is, you can use the same references used in other commands such as ``avocado jobs show``. .. _job-diff: Job Diff -------- Avocado Diff plugin allows users to easily compare several aspects of two given jobs. The basic usage is: .. code-block:: diff $ avocado diff 7025aaba 384b949c --- 7025aaba9c2ab8b4bba2e33b64db3824810bb5df +++ 384b949c991b8ab324ce67c9d9ba761fd07672ff @@ -1,15 +1,15 @@ COMMAND LINE -/usr/bin/avocado run sleeptest.py +/usr/bin/avocado run passtest.py TOTAL TIME -1.00 s +0.00 s TEST RESULTS -1-sleeptest.py:SleepTest.test: PASS +1-passtest.py:PassTest.test: PASS ... Avocado Diff can compare and create an unified diff of: - Command line. - Job time. - Variants and parameters. - Tests results. - Configuration. - Sysinfo pre and post. .. note:: Avocado Diff will ignore files containing non UTF-8 characters, like binaries, as an example. Only sections with different content will be included in the results. You can also enable/disable those sections with ``--diff-filter``. Please see ``avocado diff --help`` for more information. Jobs can be identified by the Job ID, by the results directory or by the key ``latest``. Example: .. code-block:: diff $ avocado diff ~/avocado/job-results/job-2016-08-03T15.56-4b3cb5b/ latest --- 4b3cb5bbbb2435c91c7b557eebc09997d4a0f544 +++ 57e5bbb3991718b216d787848171b446f60b3262 @@ -1,9 +1,9 @@ COMMAND LINE -/usr/bin/avocado run perfmon.py +/usr/bin/avocado run passtest.py TOTAL TIME -11.91 s +0.00 s TEST RESULTS -1-test.py:Perfmon.test: FAIL +1-examples/tests/passtest.py:PassTest.test: PASS Along with the unified diff, you can also generate the html (option ``--html``) diff file and, optionally, open it on your preferred browser (option ``--open-browser``):: $ avocado diff 7025aaba 384b949c --html /tmp/myjobdiff.html /tmp/myjobdiff.html If the option ``--open-browser`` is used without the ``--html``, a temporary html file will be created. For those wiling to use a custom diff tool instead of the Avocado Diff tool, there is an option ``--create-reports`` that will, create two temporary files with the relevant content. The file names are printed and user can copy/paste to the custom diff tool command line:: $ avocado diff 7025aaba 384b949c --create-reports /var/tmp/avocado_diff_7025aab_zQJjJh.txt /var/tmp/avocado_diff_384b949_AcWq02.txt $ diff -u /var/tmp/avocado_diff_7025aab_zQJjJh.txt /var/tmp/avocado_diff_384b949_AcWq02.txt --- /var/tmp/avocado_diff_7025aab_zQJjJh.txt 2016-08-10 21:48:43.547776715 +0200 +++ /var/tmp/avocado_diff_384b949_AcWq02.txt 2016-08-10 21:48:43.547776715 +0200 @@ -1,250 +1,19 @@ COMMAND LINE ============ -/usr/bin/avocado run sleeptest.py +/usr/bin/avocado run passtest.py TOTAL TIME ========== -1.00 s +0.00 s ... Listing tests ------------- Avocado can list your tests without run it. This can be handy sometimes. There are two ways of discovering the tests. One way is to simulate the execution by using the ``--dry-run`` argument:: $ avocado run /bin/true --dry-run JOB ID : 0000000000000000000000000000000000000000 JOB LOG : /var/tmp/avocado-dry-run-k2i_uiqx/job-2020-09-02T09.09-0000000/job.log (1/1) /bin/true: STARTED (1/1) /bin/true: CANCEL: Test cancelled due to --dry-run (0.00 s) RESULTS : PASS 0 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 1 JOB HTML : /var/tmp/avocado-dry-run-k2i_uiqx/job-2020-09-02T09.09-0000000/results.html JOB TIME : 0.29 s which supports all ``run`` arguments, simulates the run and even lists the test params. The other way is to use ``list`` subcommand that lists the discovered tests If no arguments provided, Avocado can lists tests discovered by each discovered plugin. Let's now list only the executable tests:: $ avocado list /bin/true /bin/false examples/tests/passtest.py | grep ^exec-test exec-test /bin/true exec-test /bin/false Here, as mentioned before, ``exec-test`` means that those files are treated as executable tests. You can also give the ``--verbose`` or ``-V`` flag to display files that were found by Avocado, but are not considered Avocado tests:: $ avocado -V list examples/gdb-prerun-scripts/ Type Test Tag(s) Resolver Reference Info avocado-instrumented examples/gdb-prerun-scripts/README File name "examples/gdb-prerun-scripts/README" does not end with suffix ".py" exec-test examples/gdb-prerun-scripts/README File "examples/gdb-prerun-scripts/README" does not exist or is not executable golang examples/gdb-prerun-scripts/README python-unittest examples/gdb-prerun-scripts/README File name "examples/gdb-prerun-scripts/README" does not end with suffix ".py" robot examples/gdb-prerun-scripts/README File "examples/gdb-prerun-scripts/README" does not end with ".robot" tap examples/gdb-prerun-scripts/README File "examples/gdb-prerun-scripts/README" does not exist or is not executable avocado-instrumented examples/gdb-prerun-scripts/pass-sigusr1 File name "examples/gdb-prerun-scripts/pass-sigusr1" does not end with suffix ".py" exec-test examples/gdb-prerun-scripts/pass-sigusr1 File "examples/gdb-prerun-scripts/pass-sigusr1" does not exist or is not executable golang examples/gdb-prerun-scripts/pass-sigusr1 python-unittest examples/gdb-prerun-scripts/pass-sigusr1 File name "examples/gdb-prerun-scripts/pass-sigusr1" does not end with suffix ".py" robot examples/gdb-prerun-scripts/pass-sigusr1 File "examples/gdb-prerun-scripts/pass-sigusr1" does not end with ".robot" tap examples/gdb-prerun-scripts/pass-sigusr1 File "examples/gdb-prerun-scripts/pass-sigusr1" does not exist or is not executable TEST TYPES SUMMARY ================== Notice that the verbose flag also adds summary information. .. seealso:: To read more about test discovery, visit the section :ref:`finding_tests`. avocado-113.0/docs/source/guides/user/chapters/plugins.rst000066400000000000000000000260121513561306000236450ustar00rootroot00000000000000Understanding the plugin system =============================== Avocado has a plugin system that can be used to extended it in a clean way. .. note:: A large number of out-of-the-box Avocado features are implemented as using the same plugin architecture available to third-party extensions. This guide considers "core features", even though they're still 'plugable', those available with an installation of Avocado by itself (``pip install avocado-framework``). If a feature is part of an optional or third-party plugin package, this guide will reference it." Listing plugins --------------- The ``avocado`` command line tool has a builtin ``plugins`` command that lets you list available plugins. The usage is pretty simple:: $ avocado plugins Plugins that add new commands (avocado.plugins.cli.cmd): exec-path Returns path to Avocado bash libraries and exits. run Run one or more tests (native test, test alias, binary or script) sysinfo Collect system information ... Plugins that add new options to commands (avocado.plugins.cli): journal Journal options for the 'run' subcommand ... Since plugins are (usually small) bundles of Python code, they may fail to load if the Python code is broken for any reason. Example:: $ avocado plugins Failed to load plugin from module "avocado.plugins.exec_path": ImportError('No module named foo',) Plugins that add new commands (avocado.plugins.cli.cmd): run Run one or more tests (native test, test alias, binary or script) sysinfo Collect system information ... Fully qualified named for a plugin ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The Avocado plugin system uses namespaces to recognize and categorize plugins. The namespace separator here is the dot and every plugin that starts with ``avocado.plugins.`` will be recognized by the framework. An example of a plugin's full qualified name: ``avocado.plugins.result.json`` This plugin will generate the job result in JSON format. .. note:: Inside Avocado we will omit the prefix ``avocado.plugins`` to make the things clean. .. note:: When listing plugins with ``avocado plugins`` pay attention to the namespace inside the parenthesis on each category description. You will realize that there are, for instance, two plugins with the name 'JSON'. But when you concatenate the fully qualified name it will become clear that they are actually two different plugins: ``result.json`` and ``cli.json``. .. _disabling-a-plugin: Disabling a plugin ------------------- If you, as Avocado user, would like to disable a plugin, you can disable on config files. The mechanism available to do so is to add entries to the ``disable`` key under the ``plugins`` section of the Avocado configuration file. Example:: [plugins] disable = ['cli.hello', 'job.prepost.jobscripts'] The exact effect on Avocado when a plugin is disabled depends on the plugin type. For instance, by disabling plugins of type ``cli.cmd``, the command implemented by the plugin should no longer be available on the Avocado command line application. Now, by disabling a ``job.prepost`` plugin, those won't be executed before/after the execution of the jobs. Plugin execution order ---------------------- In many situations, such as result generation, not one, but all of the enabled plugin types will be executed. The execution order is set up by plugins developers to make execution more effective. To list the plugins in execution order, you can use ``avocado plugins --ordered``. .. note:: For more information about how the execution order is set, please visit the :ref:`writing_plugin` section on Contributor's Guide. Changing the plugin execution order ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ On some circumstances it may be necessary to change the order in which plugins are executed. To do so, add a ``order`` entry a configuration file section named after the plugin type. For ``job.prepost`` plugin types, the section name has to be named ``plugins.job.prepost``, and it would look like this:: [plugins.job.prepost] order = ['myplugin', 'jobscripts'] That configuration sets the ``job.prepost.myplugin`` plugin to execute before the standard Avocado ``job.prepost.jobscripts`` does. .. note:: If you are interested on how plugins works and how to create your own plugin, visit the :ref:`writing_plugin` section on Contributor's Guide. Pre and post plugins -------------------- Avocado provides interfaces (hooks) with which custom plugins can register to be called at various times. For instance, it's possible to trigger custom actions before and after the execution of a job, or before and after the execution of the tests from a job. Let's discuss each interface briefly. Before and after jobs ~~~~~~~~~~~~~~~~~~~~~ Avocado supports plug-ins which are (guaranteed to be) executed before the first test and after all tests finished. .. This is a developer information not useful for user; .. The interfaces are :class:`avocado.core.plugin_interfaces.JobPre` and .. :class:`avocado.core.plugin_interfaces.JobPost`, respectively. The :meth:`pre ` method of each installed plugin of type ``job.prepost`` will be called by the ``run`` command, that is, anytime an ``avocado run `` command is executed. .. note:: Conditions such as the :exc:`SystemExit` or :exc:`KeyboardInterrupt` exceptions being raised can interrupt the execution of those plugins. Then, immediately after that, the job's :meth:`run ` method is called, which attempts to run all job phases, from test suite creation to test execution. Unless a :exc:`SystemExit` or :exc:`KeyboardInterrupt` is raised, or yet another major external event (like a system condition that Avocado can not control) it will attempt to run the :meth:`post ` methods of all the installed plugins of type ``job.prepost``. This even includes job executions where the :meth:`pre ` plugin executions were interrupted. Before and after tests ~~~~~~~~~~~~~~~~~~~~~~ If you followed the previous section, you noticed that the job's :meth:`run ` method was said to run all the test phases. Here's a sequence of the job phases: 1) :meth:`Creation of the test suite ` 2) :meth:`Pre tests hook ` 3) :meth:`Tests execution ` 4) :meth:`Post tests hook ` Plugin writers can have their own code called at Avocado during a job that will be called at phase number 2 (``pre_tests``) by writing a method according to the :meth:`avocado.core.plugin_interfaces.JobPreTests` interface. Accordingly, plugin writers can have their own called at phase number 4 (``post_tests``) by writing a method according to the :meth:`avocado.core.plugin_interfaces.JobPostTests` interface. Note that there's no guarantee that all of the first 3 job phases will be executed, so a failure in phase 1 (``create_test_suite``), may prevent the phase 2 (``pre_tests``) and/or 3 (``run_tests``) from from being executed. Now, no matter what happens in the *attempted execution* of job phases 1 through 3, job phase 4 (``post_tests``) will be *attempted to be executed*. To make it extra clear, as long as the Avocado test runner is still in execution (that is, has not been terminated by a system condition that it can not control), it will execute plugin's ``post_tests`` methods. As a concrete example, a plugin' ``post_tests`` method would not be executed after a ``SIGKILL`` is sent to the Avocado test runner on phases 1 through 3, because the Avocado test runner would be promptly interrupted. But, a ``SIGTERM`` and ``KeyboardInterrupt`` sent to the Avocado test runner under phases 1 though 3 would still cause the test runner to run ``post_tests`` (phase 4). Now, if during phase 4 a ``KeyboardInterrupt`` or ``SystemExit`` is received, the remaining plugins' ``post_tests`` methods will **NOT** be executed. Jobscripts plugin ~~~~~~~~~~~~~~~~~ Avocado ships with a plugin (installed by default) that allows running scripts before and after the actual execution of Jobs. A user can be sure that, when a given "pre" script is run, no test in that job has been run, and when the "post" scripts are run, all the tests in a given job have already finished running. Configuration ^^^^^^^^^^^^^ By default, the script directory location is:: /etc/avocado/scripts/job Inside that directory, that is a directory for pre-job scripts:: /etc/avocado/scripts/job/pre.d And for post-job scripts:: /etc/avocado/scripts/job/post.d All the configuration about the Pre/Post Job Scripts are placed under the ``avocado.plugins.jobscripts`` config section. To change the location for the pre-job scripts, your configuration should look something like this:: [plugins.jobscripts] pre = /my/custom/directory/for/pre/job/scripts/ Accordingly, to change the location for the post-job scripts, your configuration should look something like this:: [plugins.jobscripts] post = /my/custom/directory/for/post/scripts/ A couple of other configuration options are available under the same section: * ``warn_non_existing_dir``: gives warnings if the configured (or default) directory set for either pre or post scripts do not exist * ``warn_non_zero_status``: gives warnings if a given script (either pre or post) exits with non-zero status Script Execution Environment ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ All scripts are run in separate process with some environment variables set. These can be used in your scripts in any way you wish: * ``AVOCADO_JOB_UNIQUE_ID``: the unique `job-id`. * ``AVOCADO_JOB_STATUS``: the current status of the job. * ``AVOCADO_JOB_LOGDIR``: the filesystem location that holds the logs and various other files for a given job run. Note: Even though these variables should all be set, it's a good practice for scripts to check if they're set before using their values. This may prevent unintended actions such as writing to the current working directory instead of to the ``AVOCADO_JOB_LOGDIR`` if this is not set. Finally, any failures in the Pre/Post scripts will not alter the status of the corresponding jobs. Tests' logs plugin ~~~~~~~~~~~~~~~~~~ It's natural that Avocado will be used in environments where access to the integral job results won't be easily accessible. For instance, on Continuous Integration (CI) services, one usually gets access to the output produced on the console, while access to other files produced (generally called artifacts) may or may not be accessible. For this reason, it may be helpful to simply output the logs for tests that have "interesting" outcomes, which usually means that fail and need to be investigated. To show the content for test that are canceled, skipped and fail, you can set on your configuration file:: [job.output.testlogs] statuses = ["CANCEL", "SKIP", "FAIL"] At the end of the job, a header will be printed for each test that ended with any of the statuses given, followed by the raw content of its respective log file. avocado-113.0/docs/source/guides/user/chapters/results.rst000066400000000000000000000120171513561306000236650ustar00rootroot00000000000000Results Specification ===================== On a machine that executed Avocado, job results are available under ``[job-results]/job-[timestamp]-[short job ID]``, where ``logdir`` is the configured Avocado logs directory (see the data dir plugin), and the directory name includes a timestamp, such as ``job-2021-09-28T14.21-e0775d9``. A typical results directory structure can be seen below :: $HOME/avocado/job-results/job-2021-09-28T14.21-e0775d9/ ├── full.log ├── id ├── jobdata │   ├── args.json │   ├── cmdline │   ├── config │   ├── pwd │   ├── test_references │   └── variants-1-1.json ├── job.log ├── results.html ├── results.json ├── results.tmt ├── results.tap ├── results.xml ├── sysinfo │   ├── post │   │   ├── brctl show │   │   ├── cmdline │   │   ├── cpuinfo │   │   ├── current_clocksource │   │   ├── df -mP │   │   ├── dmesg │   │   ├── dmidecode │   │   ├── fdisk -l │   │   ├── gcc --version │   │   ├── hostname │   │   ├── ifconfig -a │   │   ├── interrupts │   │   ├── ip link │   │   ├── journalctl.gz │   │   ├── ld --version │   │   ├── lscpu │   │   ├── lspci -vvnn │   │   ├── meminfo │   │   ├── modules │   │   ├── mounts │   │   ├── numactl --hardware show │   │   ├── partitions │   │   ├── pci │   │   ├── scaling_governor │   │   ├── sched_features │   │   ├── slabinfo │   │   ├── uname -a │   │   ├── uptime │   │   └── version │   ├── pre │   │   ├── brctl show │   │   ├── cmdline │   │   ├── cpuinfo │   │   ├── current_clocksource │   │   ├── df -mP │   │   ├── dmesg │   │   ├── dmidecode │   │   ├── fdisk -l │   │   ├── gcc --version │   │   ├── hostname │   │   ├── ifconfig -a │   │   ├── interrupts │   │   ├── ip link │   │   ├── ld --version │   │   ├── lscpu │   │   ├── lspci -vvnn │   │   ├── meminfo │   │   ├── modules │   │   ├── mounts │   │   ├── numactl --hardware show │   │   ├── partitions │   │   ├── pci │   │   ├── scaling_governor │   │   ├── sched_features │   │   ├── slabinfo │   │   ├── uname -a │   │   ├── uptime │   │   └── version │   └── profile └── test-results ├── 1-1-_bin_true │   ├── debug.log │   ├── stderr │   └── stdout ├── 1-2-_bin_false │   ├── debug.log │   ├── stderr │   └── stdout └── by-status ├── FAIL │   └── 1-2-_bin_false -> ../../1-2-_bin_false └── PASS └── 1-1-_bin_true -> ../../1-1-_bin_true From what you can see, the results directory has: 1) A human readable ``id`` in the top level, with the job SHA1. 2) A human readable ``job.log`` in the top level, with human readable logs of the task 3) A human readable ``full.log`` in the top level, with human readable logs of anything generated inside the job. 4) Subdirectory ``jobdata``, that contains machine readable data about the job. 5) A machine readable ``results.xml`` and ``results.json`` in the top level, with a summary of the job information in xUnit/json format. 6) A top level ``sysinfo`` dir, with sub directories ``pre``, ``post`` and ``profile``, that store sysinfo files pre/post/during job, respectively. 7) Subdirectory ``test-results``, that contains a number of subdirectories (filesystem-friendly test ids). Those test ids represent instances of test execution results. Test execution instances specification -------------------------------------- The instances should have: 1) A top level human readable ``job.log`` and ``full.log``, with job debug information 2) A ``sysinfo`` subdirectory, with sub directories ``pre``, ``post`` and ``profile`` that store sysinfo files pre test, post test and profiling info while the test was running, respectively. 3) A ``data`` subdirectory, where the test can output a number of files if necessary. avocado-113.0/docs/source/guides/user/chapters/tags.rst000066400000000000000000000160421513561306000231240ustar00rootroot00000000000000Filtering tests by tags ======================= Avocado allows tests to be given tags, which can be used to create test categories. With tags set, users can select a subset of the tests found by the test resolver. Usually, listing and executing tests with the Avocado test runner would reveal all three tests:: $ avocado list examples/tests/tags.py avocado-instrumented examples/tests/tags.py:FastTest.test_fast avocado-instrumented examples/tests/tags.py:FastTest.test_fast_other avocado-instrumented examples/tests/tags.py:SlowTest.test_slow avocado-instrumented examples/tests/tags.py:SlowUnsafeTest.test_slow_unsafe avocado-instrumented examples/tests/tags.py:SafeTest.test_safe avocado-instrumented examples/tests/tags.py:SafeX86Test.test_safe_x86 avocado-instrumented examples/tests/tags.py:NoTagsTest.test_no_tags avocado-instrumented examples/tests/tags.py:SafeAarch64Test.test_safe_aarch64 If you want to list or run only the network based tests, you can do so by requesting only tests that are tagged with ``net``:: $ avocado list examples/tests/tags.py --filter-by-tags=net avocado-instrumented examples/tests/tags.py:FastTest.test_fast avocado-instrumented examples/tests/tags.py:FastTest.test_fast_other Now, suppose you're not in an environment where you're comfortable running a test that will write to your raw disk devices (such as your development workstation). You know that some tests are tagged with ``safe`` while others are tagged with ``unsafe``. To only select the "safe" tests you can run:: $ avocado list examples/tests/tags.py --filter-by-tags=safe avocado-instrumented examples/tests/tags.py:SafeTest.test_safe avocado-instrumented examples/tests/tags.py:SafeX86Test.test_safe_x86 avocado-instrumented examples/tests/tags.py:SafeAarch64Test.test_safe_aarch64 But you could also say that you do **not** want the "unsafe" tests (note the *minus* sign before the tag):: $ avocado list examples/tests/tags.py --filter-by-tags=-unsafe avocado-instrumented examples/tests/tags.py:FastTest.test_fast avocado-instrumented examples/tests/tags.py:FastTest.test_fast_other avocado-instrumented examples/tests/tags.py:SlowTest.test_slow avocado-instrumented examples/tests/tags.py:SafeTest.test_safe avocado-instrumented examples/tests/tags.py:SafeX86Test.test_safe_x86 avocado-instrumented examples/tests/tags.py:SafeAarch64Test.test_safe_aarch64 .. tip:: The ``-`` sign may cause issues with some shells. One know error condition is to use spaces between ``--filter-by-tags`` and the negated tag, that is, ``--filter-by-tags -unsafe`` will most likely not work. To be on the safe side, use ``--filter-by-tags=-tag``. If you require tests to be tagged with **multiple** tags, just add them separate by commas. Example:: $ avocado list examples/tests/tags.py --filter-by-tags=disk,slow,unsafe avocado-instrumented examples/tests/tags.py:SlowUnsafeTest.test_slow_unsafe If no test contains **all tags** given on a single ``--filter-by-tags`` parameter, no test will be included:: $ avocado list examples/tests/tags.py --filter-by-tags=disk,slow,safe | wc -l 0 Multiple tags (AND vs OR) ------------------------- While multiple tags in a single option will require tests with all the given tags (effectively a logical AND operation), it's also possible to use multiple ``--filter-by-tags`` (effectively a logical OR operation). For instance To include all tests that have the ``disk`` tag and all tests that have the ``net`` tag, you can run:: $ avocado list examples/tests/tags.py --filter-by-tags=disk --filter-by-tags=net avocado-instrumented examples/tests/tags.py:FastTest.test_fast avocado-instrumented examples/tests/tags.py:FastTest.test_fast_other avocado-instrumented examples/tests/tags.py:SlowTest.test_slow avocado-instrumented examples/tests/tags.py:SlowUnsafeTest.test_slow_unsafe Including tests without tags ---------------------------- The normal behavior when using ``--filter-by-tags`` is to require the given tags on all tests. In some situations, though, it may be desirable to include tests that have no tags set. For instance, you may want to include tests of certain types that do not have support for tags (such as executable tests) or tests that have not (yet) received tags. Consider this command:: $ avocado list examples/tests/tags.py /bin/true --filter-by-tags=disk avocado-instrumented examples/tests/tags.py:SlowTest.test_slow avocado-instrumented examples/tests/tags.py:SlowUnsafeTest.test_slow_unsafe Since it requires the ``disk`` tag, only one test was returned. By using the ``--filter-by-tags-include-empty`` option, you can force the inclusion of tests without tags:: $ avocado list examples/tests/tags.py /bin/true --filter-by-tags=disk --filter-by-tags-include-empty avocado-instrumented examples/tests/tags.py:SlowTest.test_slow avocado-instrumented examples/tests/tags.py:SlowUnsafeTest.test_slow_unsafe avocado-instrumented examples/tests/tags.py:NoTagsTest.test_no_tags exec-test /bin/true Using further categorization with keys and values ------------------------------------------------- All the examples given so far are limited to "flat" tags. Sometimes, it's helpful to categorize tests with extra context. For instance, if you have tests that are sensitive to the platform endianness, you may way to categorize them by endianness, while at the same time, specifying the exact type of endianness that is required. For instance, your tags can now have a key and value pair, like: ``endianess:little`` or ``endianess:big``. To list tests without any type of filtering would give you:: $ avocado list byteorder.py avocado-instrumented byteorder.py:ByteOrder.test_le avocado-instrumented byteorder.py:ByteOrder.test_be avocado-instrumented byteorder.py:Generic.test To list tests that are somehow related to endianness, you can use:: $ avocado list byteorder.py --filter-by-tags endianness avocado-instrumented byteorder.py:ByteOrder.test_le avocado-instrumented byteorder.py:ByteOrder.test_be And to be even more specific, you can use:: $ avocado list byteorder.py --filter-by-tags endianness:big avocado-instrumented byteorder.py:ByteOrder.test_be A "negated" form is also available to filter out tests that do *not* have a given value. To filter out tests that have an endianness set, but are *not* big endian you can use:: $ avocado list byteorder.py --filter-by-tags endianness:-big avocado-instrumented byteorder.py:ByteOrder.test_le Now, suppose you intend to run tests on a little endian platform, but you'd still want to include tests that are generic enough to run on either little or big endian (but not tests that are specific to other types of endianness), you could use:: $ avocado list byteorder.py --filter-by-tags endianness:big --filter-by-tags-include-empty-key avocado-instrumented byteorder.py:ByteOrder.test_be avocado-instrumented byteorder.py:Generic.test .. seealso:: If you would like to understand how write plugins and how describe tags inside a plugin, please visit the :ref:`writing_tests` on Avocado Test Writer's Guide. avocado-113.0/docs/source/guides/user/chapters/whatsnext.rst000066400000000000000000000006121513561306000242070ustar00rootroot00000000000000What's next? ============ Now that you are familiar with the basic concepts and Avocado usage, you can write your tests. As said before, you can write test on your favorite language. But if you would like to use the Avocado libraries and facilities, you can use Python or Bash. If you would like to move forward on Avocado, we prepared the *"Avocado Test Writer's Guide"* for you. Have fun! avocado-113.0/docs/source/guides/user/index.rst000066400000000000000000000006351513561306000214650ustar00rootroot00000000000000==================== Avocado User's Guide ==================== .. toctree:: :maxdepth: 2 chapters/about chapters/installing chapters/introduction chapters/concepts chapters/operations chapters/results chapters/tags chapters/configuring chapters/dependencies chapters/assets chapters/datadirs chapters/logging chapters/plugins chapters/advanced chapters/whatsnext avocado-113.0/docs/source/guides/writer/000077500000000000000000000000001513561306000201565ustar00rootroot00000000000000avocado-113.0/docs/source/guides/writer/chapters/000077500000000000000000000000001513561306000217675ustar00rootroot00000000000000avocado-113.0/docs/source/guides/writer/chapters/basics.rst000066400000000000000000000166741513561306000240030ustar00rootroot00000000000000========================== Writing an Executable Test ========================== This very simple example of an executable test in shell script:: $ echo '#!/bin/bash' > /tmp/executable_test.sh $ echo 'exit 0' >> /tmp/executable_test.sh $ chmod +x /tmp/executable_test.sh Notice that the file is given executable permissions, which is a requirement for Avocado to treat it as a executable test. Also notice that the script exits with status code 0, which signals a successful result to Avocado. BASH extensions for Executable tests ------------------------------------ Exec-tests written in shell can use a few Avocado utilities. In your shell code, check if the libraries are available with something like:: AVOCADO_SHELL_EXTENSIONS_DIR=$(avocado exec-path 2>/dev/null) And if available, injects that directory containing those utilities into the PATH used by the shell, making those utilities readily accessible:: if [ $? == 0 ]; then PATH=$AVOCADO_SHELL_EXTENSIONS_DIR:$PATH fi For a full list of utilities, take a look into at the directory return by ``avocado exec-path`` (if any). Also, the example test ``examples/tests/simplewarning.sh`` can serve as further inspiration: .. literalinclude:: ../../../../../examples/tests/simplewarning.sh .. tip:: These extensions may be available as a separate package. For RPM packages, look for the ``bash`` sub-package. Environment Variables for Tests ------------------------------- Avocado's environment variables ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Avocado exports some information, including test parameters, as environment variables to the running test. Here is a list of the variables that Avocado currently exports to exec-test tests in default: +-----------------------------+---------------------------------------+-----------------------------------------------------------------------------------------------------+ | Environment Variable | Meaning | Example | +=============================+=======================================+=====================================================================================================+ | AVOCADO_VERSION | Version of Avocado test runner | 92.0 | +-----------------------------+---------------------------------------+-----------------------------------------------------------------------------------------------------+ | AVOCADO_TEST_BASEDIR | Base directory of Avocado tests. More | $HOME/src/avocado/avocado.dev/examples/tests | | | info in :data:`avocado.Test.basedir` | | +-----------------------------+---------------------------------------+-----------------------------------------------------------------------------------------------------+ | AVOCADO_TEST_WORKDIR | Work directory for the test. More | /var/tmp/.avocado-taskcx8of8di/test-results/tmp_dirfgqrnbu/1-Env.test | | | info in :data:`avocado.Test.workdir` | | +-----------------------------+---------------------------------------+-----------------------------------------------------------------------------------------------------+ | AVOCADO_TESTS_COMMON_TMPDIR | Temporary directory created by the | /var/tmp/avocado_XhEdo/ | | | :ref:`plugin_teststmpdir` plugin. The| | | | directory is persistent throughout the| | | | tests in the same Job | | +-----------------------------+---------------------------------------+-----------------------------------------------------------------------------------------------------+ | AVOCADO_TEST_LOGDIR | Log directory for the test | /var/tmp/.avocado-task_5t_srpn/test-results/1-Env.test | +-----------------------------+---------------------------------------+-----------------------------------------------------------------------------------------------------+ | AVOCADO_TEST_LOGFILE | Log file for the test | /var/tmp/.avocado-taskcx8of8di/test-results/1-Env.test/debug.log | +-----------------------------+---------------------------------------+-----------------------------------------------------------------------------------------------------+ | AVOCADO_TEST_OUTPUTDIR | Output directory for the test | /var/tmp/.avocado-taskcx8of8di/test-results/1-Env.test/data | +-----------------------------+---------------------------------------+-----------------------------------------------------------------------------------------------------+ | `***` | All variables from --mux-yaml | TIMEOUT=60; IO_WORKERS=10; VM_BYTES=512M; ... | +-----------------------------+---------------------------------------+-----------------------------------------------------------------------------------------------------+ User's environment variables ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ You can also let avocado set your own environment variables. For that, you need to pass the environment variables as keyword arguments (``kwargs``) to the exec-tests. Here is an example of Job API which passes ``SLEEP_LENGTH`` to sleeptest.sh to set the time for which the test should sleep: .. literalinclude:: ../../../../../examples/jobs/custom_exec_test.py And now we can see that sleeptest.sh can use ``SLEEP_LENGTH`` environment variable: .. literalinclude:: ../../../../../examples/tests/sleeptest.sh .. note:: All environment variables set by avocado will be accessible only during the test runtime and it won't change your environment. Disabling environment variables ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Let's imagine that your testing environment has some important variables, but they could have a negative impact on one of your tests. In that case, avocado lets you disable those variables during the test runtime. To disable a test variable, you need to set it in test ``kwargs`` to ``None`` like this:: Runnable("exec-test", "examples/tests/sleeptest.sh", SLEEP_LENGTH=None) If you need to clear the whole environment before your test, then you can set ``runner.exectest.clear_env`` config variable. This variable has two options. ``system`` and ``all``. If you use ``system`` option the testing environment will have only Avocado default variables and variables from test ``kwargs``. If you use ``all`` option, the testing environment will have only variables from test ``kwargs``:: Runnable("exec-test", "examples/tests/sleeptest.sh", config={'runner.exectest.clear_env': system}, SLEEP_LENGTH=1) avocado-113.0/docs/source/guides/writer/chapters/integrating.rst000066400000000000000000000026631513561306000250430ustar00rootroot00000000000000Integrating Avocado =================== Coverage.py ~~~~~~~~~~~ Testing software is important, but knowing the effectiveness of the tests, like which parts are being exercised by the tests, may help develop new tests. `Coverage.py`_ is a tool designed for measuring code coverage of Python programs. It runs monitoring the program's source, taking notes of which parts of the code have been executed. It is possible to use Coverage.py while running Avocado Instrumented tests or Python unittests. To make the Coverage.py parameters visible to other processes spawned by Avocado, create the ``.coveragerc`` file in the project's root folder and set ``source`` parameter to your system under test. Following is an example:: [run] source = foo/bar With the ``.coveragerc`` file set, one possible workflow to use Coverage.py to measure Avocado tests is:: coverage run -m avocado run tests/foo coverage combine coverage report The first command uses Coverage.py to measure the code coverage of the Avocado tests. Then, `coverage combine` combines all measurement files to a single ``.coverage`` data file. The `coverage report` shows the report of the coverage measurement. For other options related to `Coverage.py`_, visit the software documentation. .. note:: Currently coverage support is limited working only with `ProcessSpawner` (the default spawner) and Coverage.py>=7.5. .. _Coverage.py: https://coverage.readthedocs.io/ avocado-113.0/docs/source/guides/writer/chapters/libs.rst000066400000000000000000000025141513561306000234540ustar00rootroot00000000000000.. _utility-libraries: Utility Libraries ================= Avocado gives to you more than 40 Python utility libraries (so far), that can be found under the :mod:`avocado.utils`. You can use these libraries to avoid having to write necessary routines for your tests. These are very general in nature and can help you speed up your test development. The utility libraries may receive incompatible changes across minor versions, but these will be done in a staged fashion. If a given change to an utility library can cause test breakage, it will first be documented and/or deprecated, and only on the next subsequent minor version it will actually be changed. What this means is that upon updating to later minor versions of Avocado, you should look at the Avocado Release Notes for changes that may impact your tests. .. seealso:: If you would like a detailed API reference of these libraries, please visit the "Reference API" section on the left menu. The following pages are the documentation for some of the Avocado utilities: .. warning:: TODO: Looks like the utils libraries documentation will be mainly on docstrings, right? If so, maybe makes sense to have only documented on API reference? And any general instruction would be on module docstring. What you guys think? .. toctree:: :maxdepth: 1 ../libs/gdb ../libs/vmimage avocado-113.0/docs/source/guides/writer/chapters/logging.rst000066400000000000000000000445251513561306000241610ustar00rootroot00000000000000.. _advanced-logging: Advanced logging capabilities ============================= Avocado provides advanced logging capabilities at test run time. These can be combined with the standard Python library APIs on tests. One common example is the need to follow specific progress on longer or more complex tests. Let's look at a very simple test example, but one multiple clear stages on a single test: .. literalinclude:: ../../../../../examples/tests/logging_streams.py Currently Avocado will store any log information that has been generated during job run into specific log files based on used logging namespace and process where the logs have been generated. Avocado generate three types of log each of them with different purpose: **1. job.log** The `job.log` file is generated for each avocado job and it is stored directly in `log_dir` directory. This file contains only logs from `avocado.job` namespace and holds information about job configuration and test statuses which were run inside the job. The format of logs is::