Currently, selecting a different device in a platform to test is rather
cumbersome, for two reasons:
1. The default device type tested is the "default" device and there is
at most one default device in a platform. This means that, by itself,
choosing any non-zero device index is by definition out-of-range:
```sh
$ CL_PLATFORM_INDEX=1 CL_DEVICE_INDEX=1 ./test_conformance/basic/test_basic
Initializing random seed to 0.
Requesting Default device based on command line for platform index 1 and device index 1
device index out of range -- choosen_device_index (1) >= num_devices (1)
```
2. To choose a non-default device type you therefore need to explicitly
specify another device type also, but "all" is not a valid device type
in the harness. This means that you need to know both the device type
and the index of the device within that device type to choose the device
to test.
```sh
$ CL_DEVICE_TYPE=all CL_PLATFORM_INDEX=1 CL_DEVICE_INDEX=1 ./test_conformance/basic/test_basic
Unknown CL_DEVICE_TYPE env variable setting: all.
Aborting...
Aborted (core dumped)
```
This PR aims to fix (2), by allowing "all" as a device type. In the
future, we could consider making the default device type "all" vs.
"default", which would fix (1) also, but that will likely need more
discussion and should be done in a separate PR.
Fixes:
1. Multi import diff ctx subtest which acquires/releases external memory
via queue not associated with the context in which memory was imported.
2. Platform/Device info subtests to handle different platforms and
availability of the query.
* basic SPIR-V 1.3 testing support
* updated script to compile for more SPIR-V versions
* switch to general SPIR-V versions test
* update copyright text and fix license
* improve output while test is running
* check for higher SPIR-V versions first
* fix formatting
In preparation of re-enabling `-Wsign-compare` globally, avoid mixing
signed and unsigned integers in comparisons in test_common.
Signed-off-by: Sven van Haastregt <sven.vanhaastregt@arm.com>
This change introduces a new command-line parameter to enable
parallel execution by a specified number of worker threads. When
parallel execution is requested, tests are distributed across
the worker threads. This behaviour is disabled by default.
This does not currently work for all suites as some of them are
using global variables to configure tests. For the suites that
do not use global state, this change reduced the execution time
by up to 5x on an 8-core machine.
Signed-off-by: Kévin Petit <kpet@free.fr>
It only contains a pointer type definition for test functions that
really ought to be provided by testHarness.h.
Signed-off-by: Kévin Petit <kpet@free.fr>
Signed-off-by: Kévin Petit <kpet@free.fr>
The main sources of warnings were:
* Printing of a `size_t` which requires the `%zu` specifier.
* Printing of `cl_long`/`cl_ulong` which is now done using the
`PRI*64` macros to ensure portability across 32 and 64-bit builds.
Signed-off-by: Sven van Haastregt <sven.vanhaastregt@arm.com>
Signed-off-by: Sven van Haastregt <sven.vanhaastregt@arm.com>
The context and queue were not released when the test is not supported
in offline mode or the queue couldn't be created.
Inline test_missing_support_offline_cmpiler_ret macro, remove dead
parameter of check_functions_for_offline_compiler and slightly refactor
callSingleTestFunction to address leaks.
Signed-off-by: Marco Antognini <marco.antognini@arm.com>
When invoking for example
test_c11_atomics test-that-does-not-exist
parseAndCallCommandLineTests() would attempt to dereference
`resultTestList` which is still a null pointer.
Signed-off-by: Sven van Haastregt <sven.vanhaastregt@arm.com>
* Implement Negative Tests for clPlatform Functions
This change introduces negative tests for clPlatform
functions as well as changes to the Harness to help with
other negative tests.
Signed-off-by: Chetankumar Mistry <chetan.mistry@arm.com>
* [SQUASH] Remove magic macro from Negative Platform Tests
This change removes the negative-testing macro and all
other changes related to its usage.
Signed-off-by: Chetankumar Mistry <chetan.mistry@arm.com>
TEST_SKIPPED_ITSELF was originally located in
threadTesting.h but this no longer makes sense.
This change moves the definition to the test_status
struct in testHarness so that it can be used in the same
way that test_status' can be used.
Signed-off-by: Chetankumar Mistry <chetan.mistry@arm.com>
* Tests requiring image support use runTestHarnessWithCheck
Removing special case for images in runTestHarness.
Fixes#710
* Remove imageSupportRequired argument
Tests which require image support now specify this while
calling runTestHarnessWithCheck.
Fixes#710
Signed-off-by: Ellen Norris-Thompson <ellen.norris-thompson@arm.com>
* Reimplement buffer tests
Reintegrated and fixed test code for buffer tests buffer_read_half and
buffer_write_half tests.
Added mem_alloc_ref_flags test code, as was previously non-existent,
to test CL_MEM_ALLOC_HOST_PTR. This flag was otherwise untested and
as similar tests within the suite are used to test other cl_mem_flags
it has been assumed that this was the purpose of the test.
Fixes#439
Change-Id: I5accf986be7436d09377d0bfd7afd5de2235c329
Signed-off-by: Ellen Norris-Thompson <ellen.norris-thompson@arm.com>
* move mem_read_write_flags to a common function
Code under mem_*_flags tests have a lot of duplication, this is
the first step of moving test code to a common function.
Contributes #439
Signed-off-by: Ellen Norris-Thompson <ellen.norris-thompson@arm.com>
* move mem_write_only_flags test code to a common function
Code under mem_*_flags tests have a lot of duplication
Contributes #439
Signed-off-by: Ellen Norris-Thompson <ellen.norris-thompson@arm.com>
* move mem_read_only_flags test code to a common function
Code under mem_*_flags tests have a lot of duplication
Contributes #439
Signed-off-by: Ellen Norris-Thompson <ellen.norris-thompson@arm.com>
* move mem_copy_host_flags test code to a common function
Code under mem_*_flags tests have a lot of duplication, moved
mem_copy_host_flags code and rearranged function where appropriate
mem_ref_alloc_flags test also uses common function.
Contributes #439
Signed-off-by: Ellen Norris-Thompson <ellen.norris-thompson@arm.com>
* Remove unused NOT_IMPLEMENTED_TEST macro
This define is not in use anymore, since tests have been
reimplemented in #439. Tests should be working and implemented
or not registered.
Signed-off-by: Ellen Norris-Thompson <ellen.norris-thompson@arm.com>
* Reformat common help text
Signed-off-by: Stuart Brady <stuart.brady@arm.com>
* Reformat test harness code
This goes part of the way to fixing issue #625.
Signed-off-by: Stuart Brady <stuart.brady@arm.com>
When running sub-tests explicitly on the command line and that sub-test
fails, the test harness returns `0` from `main()`. This is problematic
in automated testing environments as failed tests can be hidden.
This patch iterates over all the test results in the `resultTestList`
after all sub-tests have completed, if any of the tests failed a
non-zero value is returned from `parseAndCallCommandLineTests()` which
is then propagated up the call stack to `main()`.
- Remove the requirement to detail values for device queries and list
extensions in the submission template. They are printed by computeinfo.
- Require that tests skipped in accordance with the conformance
process be listed in the submission template.
- Remove support for OPENCL_1_0_DEVICE. This variable doesn't control
anything anymore and the updated conformance process no longer refers
to it.
- Require that implementations running via a loader document the loader
that was used.
Signed-off-by: Kevin Petit <kevin.petit@arm.com>
* Accept OpenCL 3.0 in version parsing code and use where appropriate
There were a number of tests against 2.2 that are clearer against 3.0.
Fixes#751
Signed-off-by: Kévin Petit <kpet@free.fr>
* Remove CL_EXPERIMENTAL guards
Signed-off-by: Kévin Petit <kpet@free.fr>
* formatting
* Configure the headers for OpenCL 3.0
* more format fixes
(Patch1)
A number of tests have got their own code for checking the presence of
extensions. This change replaces that code with is_extension_available
function.
Contributes to #627
Change-Id: I8dd2233719aa8c84841ac61776437d7f6e3fafe6
Signed-off-by: Ellen Norris-Thompson <ellen.norris-thompson@arm.com>
* Harness: Added an option for a test to skip itself.
New condtion in callSingleTestFunction. This allows a test to return TEST_SKIP if the device does not support the requested test.
* Split generic_ptr_to_host_mem into two tests.
The old generic_ptr_to_host_mem test tests two different device capabilities (SVM and not SVM). This is a prerequisite for the following commit.
* Generic Address Space: Skip tests utlilising SVM on devices that do not support SVM.
Where a device does not support SVM, do not run the generic address space test(s) that rely on SVM.
Add gDeviceType set on kernel_read_write tests(image_streams)
as the device type affects the way validation is performed in the test.
Signed-off-by: John Kesapides <john.kesapides@arm.com>
Moved all duplicate 'PrintArch' functions to testHarness.c
Replaced sysctl system call used in 'PrintArch' function with uname
system call
Signed-off-by: bhargavdas <bhargav_das@mentor.com>
Update testHarness.c
