masl.src.html

<!doctype html>
<html lang="en">
  <head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>MASL — Metadata for Arbitrary Structures & Links</title>
  </head>
  <body>
    <div id="abstract">
      <p>
        MASL is a CBOR-based metadata system that is designed to work well with content-addressed
        and decentralised systems, to enable fully self-contained, self-certified content
        distribution.
      </p>
    </div>
    <section>
      <h2>Introduction</h2>
      <p>
        Anywhere you have resources that will be deployed in real-world systems, the potential
        metadata needs of those systems are effectively unbounded. This is particularly true of
        decentralised systems that need to exhibit "web-like" behaviour: in order to have
        reproducible and safe execution when content is sourced from arbitrary origins, it is
        necessary to have an equivalent to HTTP headers that is as verifiable as the content
        itself and not suffer the lossy behaviour that can occur when one treats the web as a file system (which it is not).
      </p>
      <p>
        Designing or constraining a syntax for all metadata needs would be hubris and madness.
        Instead, this document tries to minimally constrain applications while illustrating "where to stick"
        that metadata, as there are so few layers and hiding places in the DASL system.
      </p>
    </section>
    <section>
      <h2>How to use a MASL document</h2>
      <p>
        The recommended structure for DASL metadata is to insert a [[drisl]] document "between" each
        CID and its resource(s), essentially using DRISL to encode headers for one or more CID-addressed resources.
        To do this, simply replace the CID of a resource with the CID of a DRISL object that contains a
        top-level (tag 42) property named <code>src</code>.
        To bundle a set of CIDs in a logical unit, drop the top-level <code>src</code> and replace it with a mapping of paths-from-domain-root to DRISL metadata objects including a <code>src</code> property containing the resource's CID.
      </p>
      <p>
        It is preferable to nest any metadata in a top-level object to namespace your own metadata
        vocabulary (inside an object named, for example, `my-cool-project-v1`) rather than using opaque UUIDs, version-bits inserted into values, or magic numbers.
        There are a few reserved words at the top-level, but even these can be avoided by nesting any conflicts as needed.
      </p>
      <p>
        There are many metadata standards that can be embedded in this way to facilitate the preservation of
        metadata at ingress as well as translatability. For example, the IPFS-based storage system Storacha
        has a robust CID-based metadata system called
        <a href="https://github.com/storacha/content-claims">content credentials</a> which includes UCAN-based
        permissioning, inclusion attestations, and CID "equivalences," e.g. mappings of multiple IPFS CIDs equivalent to each other and/or to a given DASL CID.
      </p>
      <section>
        <h3>Using MASL with CAR</h3>
        <p>
          CAR files ([[car]]) have a space reserved for metadata in their header.
          A MASL metadata document can occupy that metadata header-space,
          and the variant using a <code>resources</code> map is particularly well-suited to be used there.
          The <var>resources</var> field can be used to map paths to the CIDs of resources contained in subsequent CAR
          blocks after the metadata header.
        </p>
        <p>
          In order to be inserted directly as metadata object within CAR files, MASL documents need to contain a top-level <code>version</code> property whose value must
          be set to integer <code>1</code> and a top-level <code>roots</code> array that must contain 0 or more tag 42 CIDs and nothing else.
          Neither of these fields has any meaning in MASL, but they must be provided in the context of a CAR header for historical compatibility
          reasons.
          Note that there is no requirements that all the CIDs in a <code>roots</code> array also appear in the <code>resources</code> mapping or vice versa.
        </p>
      </section>
    </section>
    <section>
      <h2>Fields</h2>
      <p>
        MASL is designed to host arbitrary metadata but for interoperability purposes a number of root
        fields have predetermined values. Authors are invited to add their own metadata by creating
        namespaced objects at the top level.
      </p>
      <p>
        <strong>NOTE</strong>: In examples below, whenever we represent a CID as JSON for, say, field
        <code>src</code>, we use <code>"src": { "$link": "CID value…" }</code> as a readability convention.
      </p>
      <section>
        <h3>Single or Multiple Resources</h3>
        <p>
          MASL documents are primarily used to wrap around other resources for which they provide
          metadata. This can happen in one of two modes:
        </p>
        <ul>
          <li>
            <strong>Single Mode</strong> (using <code>src</code>): the metadata is only for <em>one</em>
            resource, which is the one that can be retrieved from the CID pointed to by <code>src</code>.
            HTTP metadata, if specified, goes at the root. App manifest metadata on a single resource
            can be used if that resource a fully standalone document (e.g. a PDF).
          </li>
          <li>
            <strong>Bundle Mode</strong> (using <code>resources</code>): the metadata is used to
            describe a whole set of resources. These resources will typically be related to one another in some
            way (e.g. components that go into building an app or document). The keys of the <code>resources</code>
            map are complete paths that must start with <code>/</code> and the values are metadata
            objects that must have an <code>src</code> field pointing to the resource's CID and should have a
            <code>content-type</code> field giving its MIME type, along with other HTTP headers.
            Any other properties present which do not map to HTTP headers must be ignored.
          </li>
        </ul>
        <p>
          Note that if both <code>src</code> and <code>resources</code> are specified, then
          <code>src</code> must be ignored.
        </p>
        <p>
          The Bundle Mode has some specific processing rules:
        </p>
        <ul>
          <li>
            Default: The entry with path <code>/</code> is the default path that is loaded if the bundle
            itself gets rendered, e.g. in a browser or other user-agent. Implementations must only recognise this as the default
            and must not automatically decide to pick a given entry (e.g. <code>/index.html</code>).
          </li>
          <li>
            Relative: When loading a bundle into a web context, the root of the bundle is given an opaque origin,
            and all internal links are resolved relative to that.
          </li>
          <li>
            No directory: There is no notion of directory. If a resource is indicated as sitting at <code>/cats/reds/kitsune.jpg</code>
            this does not entail that <code>/cats/</code> or <code>/cats/reds/</code> somehow exist. As
            in web contexts, it is the full path that is matched, not <code>/</code>-separated subsets. URLs
            do <em>not</em> map to file systems.
          </li>
          <li>
            Query strings: When resolving a URL inside of a bundle, implementation must only make use of the URL's pathname and
            must ignore the query string. (Note that this departs from typical URL processing but makes it easier
            to pass parameters between resources internally.)
          </li>
        </ul>
        <p>
          There is no requirement in MASL that bundles have to be stored or dereferenced in any specific manner. The relevant CIDs
          may be loaded through whatever way the implementation knows about such as RASL ([[rasl]]) or may be
          provided in a CAR file ([[car]]).
          Note that one value of this approach when compared to bundling resources for instance into Zip archives (or CAR files!)
          is that the resource map can contain an arbitrarily high number or volume of resources; an implementation may load an
          arbitrary subset of the resources, may parallelize loading, in an arbitrary order informed by types and HTTP headers,
          or load some resources on-demand.
        </p>
        <div class="flag">
          Specify a scheme and fetch rules properly.
        </div>
        <p>
          Example with <code>src</code>:
        </p>
        <pre>
{
  "src": { "$link": "bafkreifn5yxi7nkftsn46b6x26grda57ict7md2xuvfbsgkiahe2e7vnq4" },
  "content-type": "application/pdf",
}
        </pre>
        <p>
          Example with <code>resources</code>:
        </p>
        <pre>
{
  "name": "A Simple Page With Pic",
  "resources": {
    "/": {
      "src": { "$link": "bafk…" },
      "content-type": "text/html"
    },
    "/picture.jpg": {
      "src": { "$link": "bafk…" },
      "content-type": "image/jpeg"
    }
  }
}
        </pre>
      </section>
      <section>
        <h3>HTTP Headers</h3>
        <p>
          MASL supports a subset of HTTP response headers that are meaningful in decentralised
          contexts. This doesn't preclude headers not listed here from being used, but implementations
          that support using HTTP headers should not reflect the value of arbitrary HTTP headers
          without considering the potential attack surface they create.
        </p>
        <p>
          When using HTTP headers as MASL metadata, there are two modes. If the MASL document
          contains a root <code>resources</code> field then it is a MASL document for multiple resources
          and the HTTP headers are only meaningful if they are set on values of the <code>resources</code>
          map (and must be ignored if set on the root object). Conversely, if this MASL document
          contains a <code>src</code> field (and no <code>resources</code>) then the HTTP headers must
          be set on the root and ignored otherwise. If neither <code>src</code> nor <code>resources</code>
          are specified, the meaning of HTTP fields is undefined.
        </p>
        <p>
          All HTTP headers, where specified, are lowercased. Implementations must ignore headers
          with a different casing.
        </p>
        <p>
          Supported headers:
        </p>
        <ul>
          <li><code>content-disposition</code></li>
          <li><code>content-encoding</code></li>
          <li><code>content-language</code></li>
          <li>
            <code>content-security-policy</code>: keep in mind however that runtime contexts are likely
            to already have a strict CSP that will override or constrain this one.
          </li>
          <li><code>content-type</code></li>
          <li><code>link</code></li>
          <li><code>permissions-policy</code></li>
          <li><code>referrer-policy</code></li>
          <li><code>service-worker-allowed</code></li>
          <li>
            <code>sourcemap</code>: this must point to another resource in the <code>resources</code> map.
            Implementations must verify that this is the case and ignore the header otherwise as source maps
            could be used to exfiltrate information.
          </li>
          <li>
            <code>speculation-rules</code>: this must point to another resource in the <code>resources</code> map.
          </li>
          <li><code>supports-loading-mode</code></li>
          <li><code>x-content-type-options</code></li>
        </ul>
        <p>
          Example with <code>src</code>:
        </p>
        <pre>
{
  "src": { "$link": "bafkreifn5yxi7nkftsn46b6x26grda57ict7md2xuvfbsgkiahe2e7vnq4" },
  "content-type": "text/html",
  "content-language": "en",
  "service-worker-allowed": "/"
}
        </pre>
        <p>
          Example with <code>resources</code>:
        </p>
        <pre>
{
  "name": "My Doc",
  "resources": {
    "/": {
      "src": { "$link": "bafk…" },
      "content-type": "text/html",
      "content-encoding": "gzip",
      "content-language": "fr"
    },
    "/interactive.js": {
      "src": { "$link": "bafk…" },
      "content-type": "application/javascript",
      "sourcemap": "/interactive.js.map"
    },
    "/interactive.js.map": {
      "src": { "$link": "bafk…" },
      "content-type": "application/json"
    },
    "/picture.jpg": {
      "src": { "$link": "bafk…" },
      "content-type": "image/jpeg"
    }
  }
}
        </pre>
      </section>
      <section>
        <h3>App Manifest</h3>
        <p>
          One useful pattern with MASL is to describe an entire app or document, with all of its
          resources available for content addressing, possibly within a common CAR ([[car]]). Such
          docs or apps should use Web App Manifest metadata ([[manifest]]) as it is widely understood.
        </p>
        <p>
          The following manifest fields are guaranteed to be usable:
          <code>background_color</code>,
          <code>categories</code>,
          <code>description</code>,
          <code>icons</code>,
          <code>id</code>,
          <code>name</code>,
          <code>screenshots</code>,
          <code>short_name</code>, and
          <code>theme_color</code>.
          Note: other manifest fields may be used, but their behaviour is not guaranteed in the kind
          of web and web-like contexts for which MASL is optimized.
        </p>
        <p>
          For both <code>icons</code> and <code>screenshots</code>, the <code>src</code> field must
          be a path that matches an entry in the <code>resources</code> map, and the <code>type</code>
          field that is normally accepted in manifests there must not be used and must be ignored
          if specified. Media type information for that resource is specific on the resource entry
          that <code>src</code> maps to.
        </p>
        <p>
          Example:
        </p>
        <pre>
{
  "name": "Unicorn Editor",
  "short_name": "Unicorn",
  "description": "This is simply the best app to edit unicorns with.",
  "background_color": "#00ff75",
  "icons": [{ "src": "/unicorn.svg" }],
  "resources": {
    "/": {
      "src": { "$link": "bafk…" },
      "content-type": "text/html"
    },
    "/unicorn.svg": {
      "src": { "$link": "bafk…" },
      "content-type": "image/svg"
    }
  }
}
        </pre>
      </section>
      <section>
        <h3>CAR Compatibility</h3>
        <p>
          As indicated in the CAR specification ([[car]]), the metadata object in the CAR header
          must contain a <code>version</code> field set to integer <code>1</code> and a
          <code>roots</code> field set to an array (that may be empty) of tag 42 CIDs. These
          fields have no meaning for MASL, but are expected to be set when MASL is used for CAR
          metadata for historical compatibility. Note that using versions in this way is an
          antipattern, and we expect the value never to change.
        </p>
        <p>
          Example:
        </p>
        <pre>
{
  "name": "Get in the CAR if you want to live",
  "version": 1,
  "roots": []
}
        </pre>
      </section>
      <section>
        <h3>AT Compatibility</h3>
        <p>
          When used with the AT Protocol ([[at]]), it is common that objects will need to feature
          a <code>$type</code> field. If present, it must be a string and must be set to the
          value <code>ing.dasl.masl</code>.
        </p>
      </section>
      <section>
        <h3>Versioning</h3>
        <p>
          When manipulating DAGs, it can be useful to keep track of history by referencing
          earlier versions of the same data or metadata. This can be done using the <code>prev</code>
          field, which if present must be a tag 42 CID pointing to a previous MASL document.
        </p>
        <p>
          Example:
        </p>
        <pre>
{
  "name": "Unicorn Editor",
  "prev": { "$link": "bafkreifn5yxi7nkftsn46b6x26grda57ict7md2xuvfbsgkiahe2e7vnq4" }
}
        </pre>
      </section>
    </section>
    <section>
      <h2>Lexicon</h2>
      <p>
        Making a precise lexicon ([[lexicon]]) for MASL is impossible because lexicons lack a way of
        constraining objects with arbitrary keys. However, the following may still prove useful when
        MASL is integrated with the AT Protocol ([[at]]).
      </p>
      <pre>
{
  "lexicon": 1,
  "id": "ing.dasl.masl",
  "defs": {
    "main": {
      "type": "object",
      "properties": {
        "src": { "type": "cid-link" },
        "resources": {
          "type": "unknown"
        },
        // HTTP
        "content-type": { "type": "string" },
        "content-disposition": { "type": "string" },
        "content-encoding": { "type": "string" },
        "content-language": { "type": "string" },
        "content-security-policy": { "type": "string" },
        "link": { "type": "string" },
        "permissions-policy": { "type": "string" },
        "referrer-policy": { "type": "string" },
        "service-worker-allowed": { "type": "string" },
        "sourcemap": { "type": "string" },
        "speculation-rules": { "type": "string" },
        "supports-loading-mode": { "type": "string" },
        "x-content-type-options": { "type": "string" },
        // Manifest
        "background_color": {
          "type": "string"
        },
        "categories": {
          "type": "array",
          "items": {
            "type": "string"
          }
        },
        "description": {
          "type": "string"
        },
        "icons": {
          "type": "array",
          "items": {
            "type": "object",
            "required": ["src"],
            "properties":{
              "src": { "type": "string" },
              "sizes": { "type": "string" },
              "purpose": { "type": "string" }
            }
          }
        },
        "name": {
          "type": "string"
        },
        "screenshots": {
          "type": "array",
          "items": {
            "type": "object",
            "required": ["src"],
            "properties":{
              "src": { "type": "string" },
              "sizes": { "type": "string" },
              "label": { "type": "string" },
              "form_factor": {
                "type": "string",
                "knownValues": ["narrow", "wide"]
              },
              "platform": {
                "type": "string",
                "knownValues": ["android", "chromeos", "ios", "ipados", "kaios", "macos", "windows", "xbox", "chrome_web_store", "itunes", "microsoft", "microsoft", "play"]
              }
            }
          }
        },
        "short_name": {
          "type": "string"
        },
        "theme_color": {
          "type": "string"
        },
        // CAR compatibility
        "version": {
          "type": "integer",
          "const": 1
        },
        "roots": {
          "type": "array",
          "items": { "type": "cid-link" }
        },
        // versioning
        "prev": { "type": "cid-link" }
      }
    }
  }
}
      </pre>
    </section>
  </body>
</html>