Repository navigation
Implement IDL tests #332
Description
IDL tests can be highly automated. WPT appears to have a helper for this: idlharness.js.
We should see about using that instead of writing any of our own IDL tests.
See also previous notes: https://hackmd.io/@webgpu/SJtUks9CU
Activity
kainino0x commentedon Nov 18, 2020
Note, these tests are not a high priority, because they only verify rules that should happen automatically in any compliant WebIDL generator.
kainino0x commentedon May 5, 2021
It may be the case that there is infrastructure already out there somewhere to generate these tests.
gpuweb/gpuweb#1710 (comment)
tidoust commentedon May 20, 2021
Right, the workflow is the following:
@webref/idlnpm package is released from contents in webref. Good thing about this package is that it guarantees that the IDL it contains is valid (can be parsed) and consistent (only uses types defined by some specification, no duplicate definitions, etc.). If there is a problem with the IDL, we will create a temporary patch and raise an issue (or a pull request) in the relevant repo to have the spec fixed.@webref/idlnpm package is then used to update interfaces extracts in Web Platform Tests. See Sync interfaces/ with @webref/idl 2.1.3 for instance.In other words, Web Platform Tests already contains tests for the IDL defined in WebGPU, and there is a mechanism in place to maintain these tests (this may take a week or so, but all IDL updates should eventually make their way to Web Platform Tests) as well as to assess the validity of the IDL defined in WebGPU.
kainino0x commentedon May 20, 2021
Is there documentation on exactly what the WPT IDL tests test?
It's impossible for these tests to be fully autogenerated, though. For example the autogenerator won't know how to create a GPUBuffer object, so it can't test copyBufferToBuffer.
tidoust commentedon May 20, 2021
I realize that the end of my previous comment is at best confusing. Sorry about that. You'll still need to create IDL tests in practice. The IDL extracts in the repo just provides the raw material to ease actual testing of the interfaces but it indeed does not provide the actual tests!
The IDL harness documentation describes how to setup the IDL harness to test interfaces of interest. It does not necessarily explain what tests the IDL harness runs under the hoods. The source code of the IDL harness is obviously available, but not necessarily an easy read...
kainino0x commentedon May 20, 2021
Aha, excellent, thank you for all this background info!