Skip to content

Fix out-of-bounds read of the last transforms in CCDSweep() - #693

Open
karjonas wants to merge 1 commit into
NVIDIAGameWorks:4.1from
karjonas:fix-ccdsweep-oob-read
Open

karjonas wants to merge 1 commit into
NVIDIAGameWorks:4.1from
karjonas:fix-ccdsweep-oob-read

Conversation

@karjonas

Copy link
Copy Markdown

Loads lastTm0.p and lastTm1.p in CCDSweep() with V3LoadU() instead of V3LoadA(), so the sweep reads only the 12 bytes of the position.

The bug

CCDSweep() in GuCCDSweepPrimitives.cpp loads the positions of the last transforms with V3LoadA(), an aligned 16-byte load that masks the fourth lane to zero. PxVec3 p is the last member of the 28-byte PxTransform, at offset 16, so the load always reads 4 bytes past the end of the transform.

The caller, PxsCCDPair::sweepFindToi() in PxsCCD.cpp, keeps lastTm0 and lastTm1 in 16-byte aligned stack locals (PX_ALIGN(16, PxTransform lastTm0)), so AddressSanitizer reports the read as a stack-buffer-overflow as soon as a CCD pair is swept. The masked lane is never used, so the sweep result is correct. This is an out-of-bounds read (undefined behavior), not a wrong answer.

The same function already loads transform0.p and transform1.p with V3LoadU() a few lines below, so the two aligned loads look like an oversight rather than a deliberate choice. V3LoadU() builds the vector with _mm_set_ps() from the three components.

PhysX 5 does not have this problem: its CCDSweep() takes PxTransform32, a 32-byte padded transform, so the aligned load stays inside the object. This change gives the 4.1 branch the same guarantee without changing the type.

How to reproduce

Version: PhysX 4.1.2, branch 4.1 at a2c0428

Steps:

  1. Build PhysX and the snippet below with -fsanitize=address.
  2. Create a scene with PxSceneFlag::eENABLE_CCD and a static box.
  3. Add a small dynamic sphere with PxRigidBodyFlag::eENABLE_CCD, moving fast enough towards the box to be a CCD pair, and simulate one step.
// Build PhysX and this file with -fsanitize=address, then run it.
#include "PxPhysicsAPI.h"
#include <cstdio>

using namespace physx;

static PxDefaultAllocator gAllocator;
static PxDefaultErrorCallback gErrorCallback;

int main()
{
	PxFoundation* foundation = PxCreateFoundation(PX_PHYSICS_VERSION, gAllocator, gErrorCallback);
	PxPhysics* physics = PxCreatePhysics(PX_PHYSICS_VERSION, *foundation, PxTolerancesScale());

	PxSceneDesc sceneDesc(physics->getTolerancesScale());
	sceneDesc.gravity = PxVec3(0.0f, -9.81f, 0.0f);
	sceneDesc.cpuDispatcher = PxDefaultCpuDispatcherCreate(0);
	sceneDesc.filterShader = PxDefaultSimulationFilterShader;
	sceneDesc.flags |= PxSceneFlag::eENABLE_CCD;
	PxScene* scene = physics->createScene(sceneDesc);

	PxMaterial* material = physics->createMaterial(0.5f, 0.5f, 0.5f);

	PxRigidStatic* box = PxCreateStatic(*physics, PxTransform(PxIdentity), PxBoxGeometry(5.0f, 0.5f, 5.0f), *material);
	scene->addActor(*box);

	// A thin, fast sphere that would tunnel through the box without CCD.
	PxRigidDynamic* sphere = PxCreateDynamic(*physics, PxTransform(PxVec3(0.0f, 5.0f, 0.0f)), PxSphereGeometry(0.1f), *material, 1.0f);
	sphere->setRigidBodyFlag(PxRigidBodyFlag::eENABLE_CCD, true);
	sphere->setLinearVelocity(PxVec3(0.0f, -500.0f, 0.0f));
	scene->addActor(*sphere);

	scene->simulate(1.0f / 60.0f);
	scene->fetchResults(true);

	const PxVec3 p = sphere->getGlobalPose().p;
	printf("sphere after one step: %f %f %f\n", p.x, p.y, p.z);

	scene->release();
	physics->release();
	foundation->release();
	return 0;
}

Expected: The sphere is stopped on top of the box and the sweep reads only the 12 bytes of each position.

Testing

The same change has shipped in the PhysX 4.1 copy bundled with Qt Quick 3D Physics since Qt 6.11 and 6.12 (https://codereview.qt-project.org/c/qt/qtquick3dphysics/+/758361). There AddressSanitizer reported the read from CCDSweep() in a sphere-versus-box CCD test before the change and nothing after it, and the simulation results were unchanged.

CCDSweep() loads lastTm0.p and lastTm1.p with V3LoadA(), an aligned
16-byte load. PxVec3 p is the last member of the 28-byte PxTransform,
at offset 16, so the load always reads 4 bytes past the end of the
transform. The caller, PxsCCDPair::sweepFindToi(), keeps lastTm0 and
lastTm1 in 16-byte aligned stack locals, so AddressSanitizer reports
the read as a stack-buffer-overflow as soon as a CCD pair is swept.

The extra lane is masked to zero and never used, so the result is not
affected, but the read is out of bounds. The same function already
loads transform0.p and transform1.p with V3LoadU() a few lines below.

Load the last transforms with V3LoadU() too, which builds the vector
from the three components.

PhysX 5 does not have this problem: its CCDSweep() takes PxTransform32,
a 32-byte padded transform, so the aligned load stays inside the object.

Signed-off-by: Jonas Karlsson <jonas.karlsson@qt.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant