a zig subcommand that fetches remote dependencies so that subsequent zig build executions are guaranteed to not attempt network access

Extracted from #14265.

I can think of two possibilities:

zig build --fetch

zig fetch

This is straightforward to implement; it would run this code, without actually running the build_runner:

zig/src/main.zig

Lines 4080 to 4126 in 7cb2f92

    
           if (!build_options.omit_pkg_fetching_code) { 
        
               var http_client: std.http.Client = .{ .allocator = gpa }; 
        
               defer http_client.deinit(); 
        
               try http_client.rescanRootCertificates(); 
        
               // Here we provide an import to the build runner that allows using reflection to find 
        
               // all of the dependencies. Without this, there would be no way to use `@import` to 
        
               // access dependencies by name, since `@import` requires string literals. 
        
               var dependencies_source = std.ArrayList(u8).init(gpa); 
        
               defer dependencies_source.deinit(); 
        
               try dependencies_source.appendSlice("pub const imports = struct {\n"); 
        
               // This will go into the same package. It contains the file system paths 
        
               // to all the build.zig files. 
        
               var build_roots_source = std.ArrayList(u8).init(gpa); 
        
               defer build_roots_source.deinit(); 
        
               // Here we borrow main package's table and will replace it with a fresh 
        
               // one after this process completes. 
        
               main_pkg.fetchAndAddDependencies( 
        
                   &thread_pool, 
        
                   &http_client, 
        
                   build_directory, 
        
                   global_cache_directory, 
        
                   local_cache_directory, 
        
                   &dependencies_source, 
        
                   &build_roots_source, 
        
                   "", 
        
               ) catch |err| switch (err) { 
        
                   error.PackageFetchFailed => process.exit(1), 
        
                   else => |e| return e, 
        
               }; 
        
               try dependencies_source.appendSlice("};\npub const build_root = struct {\n"); 
        
               try dependencies_source.appendSlice(build_roots_source.items); 
        
               try dependencies_source.appendSlice("};\n"); 
        
               const deps_pkg = try Package.createFilePkg( 
        
                   gpa, 
        
                   local_cache_directory, 
        
                   "dependencies.zig", 
        
                   dependencies_source.items, 
        
               ); 
        
               mem.swap(Package.Table, &main_pkg.table, &deps_pkg.table); 
        
               try main_pkg.addAndAdopt(gpa, "@dependencies", deps_pkg); 
        
           }

One thing to consider would be a slightly higher level abstraction of subcommand which would additionally build and install dev dependencies. For example, if there were a binary tool that should be available, it should get installed. But that can be a follow-up issue.

I think I like the zig build --fetch option better because zig fetch sounds like it might download a URL directly, which is not an outrageous idea considering the "Zig as Dependency Zero" motto.

Until the last sentence I thought there will be two separate commands: zig fetch for just downloading, and zig build --fetch for downloading+building.

zig build does everything, while avoiding redundant work if possible, such as accessing the network. See also #14283.

I like the elixir approach where you have to run mix deps.get at least once before you can compile. It makes it more explicit. Having zig build do everything is convenient, but it makes it further from build and would make it more complex.

Would this also mean that Zig would get hooks for sandboxing?
Does Zig intend to solve the use case of sneaky programs doing network access?

If not: Should, and if yes, what tools should the user be given to prevent this?
This includes techniques like removing dynamic linker, adjusting runtime path, on supporting Kernels bpf sandboxing [we had bindings in std.x.net], which are also relevant for reproducible builds.

Perhaps the command could be zig pkg fetch such that pkg would be the sub-command for all package related commands which are not directly building.

introduce the 'zig fetch' command + symlink support

zig build: add `--fetch` argument

Navigation Menu

Repository navigation

a zig subcommand that fetches remote dependencies so that subsequent `zig build` executions are guaranteed to not attempt network access #14280

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	if (!build_options.omit_pkg_fetching_code) {
	var http_client: std.http.Client = .{ .allocator = gpa };
	defer http_client.deinit();
	try http_client.rescanRootCertificates();

	// Here we provide an import to the build runner that allows using reflection to find
	// all of the dependencies. Without this, there would be no way to use `@import` to
	// access dependencies by name, since `@import` requires string literals.
	var dependencies_source = std.ArrayList(u8).init(gpa);
	defer dependencies_source.deinit();
	try dependencies_source.appendSlice("pub const imports = struct {\n");

	// This will go into the same package. It contains the file system paths
	// to all the build.zig files.
	var build_roots_source = std.ArrayList(u8).init(gpa);
	defer build_roots_source.deinit();

	// Here we borrow main package's table and will replace it with a fresh
	// one after this process completes.
	main_pkg.fetchAndAddDependencies(
	&thread_pool,
	&http_client,
	build_directory,
	global_cache_directory,
	local_cache_directory,
	&dependencies_source,
	&build_roots_source,
	"",
	) catch \|err\| switch (err) {
	error.PackageFetchFailed => process.exit(1),
	else => \|e\| return e,
	};

	try dependencies_source.appendSlice("};\npub const build_root = struct {\n");
	try dependencies_source.appendSlice(build_roots_source.items);
	try dependencies_source.appendSlice("};\n");

	const deps_pkg = try Package.createFilePkg(
	gpa,
	local_cache_directory,
	"dependencies.zig",
	dependencies_source.items,
	);

	mem.swap(Package.Table, &main_pkg.table, &deps_pkg.table);
	try main_pkg.addAndAdopt(gpa, "@dependencies", deps_pkg);
	}

a zig subcommand that fetches remote dependencies so that subsequent zig build executions are guaranteed to not attempt network access #14280

Description

Activity

deflock commented on Jan 13, 2023

andrewrk commented on Jan 13, 2023

kuon commented on Jan 13, 2023

matu3ba commented on Jan 13, 2023

DraagrenKirneh commented on Apr 13, 2023

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

a zig subcommand that fetches remote dependencies so that subsequent `zig build` executions are guaranteed to not attempt network access #14280