Repository navigation
Explore supporting StreamExecute/OLAP workloads #55
Description
Within some contexts, especially serverless, the memory limits are very low in process. So it can be desirable when performing aggregations in process, or if also streaming out to the end user, to be able to leverage StreamExecute from our API.
I will say, this is going to be rather non-trivial, but I'd like us to see what it'd take to support without fully adopting connect-web or protobufs.
Streaming requires adopting the Connect protocol, rather than simply application/json, which can be done, and is done within connect-web. https://connect.build/docs/protocol#streaming-response
To add to this, this feature is required to support OLAP queries.
cc @reconbot
Activity
mattrobenolt commentedon Aug 25, 2022
This is a functioning POC https://github.com/proxy/gist.github.com/mattrobenolt/6ca8c37185578cac8970de5a82dcfd52
It definitely needs more work. I also talked with the
buffolks, and they agree that we should vendor in their codec stuff like I'm doing here.[-]Explore supporting StreamExecute[/-][+]Explore supporting StreamExecute/OLAP workloads[/+]sabin26 commentedon Jul 28, 2024
I need to use streaming api with Cloudflare Workers. Mysql drivers that support streaming are not compatible with workers. Is there any plan to support this ?