Repository navigation
Using "unsigned int" for node::Buffer::kMaxLength prevents V8 from supporting larger TypedArrays #31399
Description
V8 developer here. We would like to allow larger TypedArrays than the current limit of 0x7FFFFFFF == 2³¹ - 1. Bumping the limit to more than 0xFFFFFFFF == 2³² - 1 currently makes the V8+Node integration build fail because Node uses an unsigned int for node::Buffer::kMaxLength: https://github.com/nodejs/node/blob/master/src/node_buffer.h#L32
Please update Node's code to use size_t for (typed) array lengths and indices so that V8 can allow TypedArrays with lengths >= 2**32 on 64-bit platforms.
(We are currently not planning any changes for 32-bit platforms, and also not for regular, non-typed JavaScript arrays.)
From a quick look, it seems most places are already using size_t; aside from the definition itself I can only see one call to Integer::NewFromUnsigned that would have to be replaced with Number::New in https://github.com/nodejs/node/blob/master/src/node_buffer.cc#L1181; so hopefully this will not be much work.
Buildbot output:
https://ci.chromium.org/p/node-ci/builders/try/node_ci_linux64_rel/b8891010098315519824
Compiler error:
FAILED: obj/node/node_base/udp_wrap.o
/b/s/w/ir/cache/goma/client/gomacc ../../third_party/llvm-build/Release+Asserts/bin/clang++ -MMD -MF...(too long)
In file included from ../../node/src/udp_wrap.cc:24:
../../node/src/node_buffer.h:32:40: error: implicit conversion from 'const size_t' (aka 'const unsigned long') to 'const unsigned int' changes value from 4294967296 to 0 [-Werror,-Wconstant-conversion]
static const unsigned int kMaxLength = v8::TypedArray::kMaxLength;
~~~~~~~~~~ ^~~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.
Activity
bnoordhuis commentedon Jan 18, 2020
This is going to need a little more consideration than just switching types.
kMaxLength > 2**31-1allows users to request file reads and writes > 2 GB but libuv currently doesn't handle those well, see libuv/libuv#1501.(It's more of a platform issue than a libuv issue but libuv is where the platform differences will be taken care of.)
lib,src: switch Buffer::kMaxLength to size_t
jakobkummerow commentedon Jan 18, 2020
Thanks for the quick fix!
Maintaining existing limits for I/O operations sounds reasonable.
If needed, you could also consider decoupling Node's Buffer limit from V8's TypedArray limit: