Defining Mannequin Provenance: A Structure for AI Provide Chain Security and Safety

Relating to AI fashions, one of many hardest inquiries to reply is deceptively easy: the place did this mannequin truly come from?

We addressed a part of this drawback with Mannequin Provenance Equipment, an open-source device that fingerprints fashions on the weight degree (the parameters that defines what a mannequin is aware of and the way it behaves) to confirm their origins. However a fingerprinting device wants a transparent customary to measure towards, that defines precisely what qualifies as a derivation relationship between two fashions. Right here, the business doesn’t but have a constant reply.

Definitions fluctuate throughout licensors, requirements of our bodies, analysis teams, and AI labs. The identical pair of fashions may be labeled as “associated” by one reviewed and “impartial” by one other, with each citing defensible reasoning. That inconsistency creates actual issues for licensing enforcement, vulnerability triage, and regulatory compliance.

We created the Mannequin Provenance Structure as an try to repair that. Comprised of a taxonomy, definition, and boundary specs, it is a normative reference, a structure, that specifies what a mannequin provenance relationship is and isn’t on the degree of weight derivation. This publish covers its construction, its reasoning, and the way it connects to the frameworks that governance packages already use. The Mannequin Provenance Structure builds on forthcoming work from Cisco AI Protection that describes the methodology in full, together with empirical proof for why such an strategy is crucial for each provenance and detection pipelines. You may assessment the Structure throughout the docs folder of the Mannequin Provenance Equipment.  

Why Defining Mannequin Provenance is Vital

Basis fashions don’t arrive within the enterprise as remoted artifacts. They get fine-tuned, distilled, quantized, merged, and repackaged, and every step produces a brand new checkpoint whose relationship to its mum or dad is poorly documented. When a safety workforce must know whether or not a deployed mannequin inherits a identified vulnerability, or when compliance wants to find out whether or not a third-party checkpoint triggers a licensing obligation, the query is all the time the identical: is that this mannequin a by-product of that one?

With no shared, rigorous reply, group can face compounding dangers:

• Provide chain assaults are already exploiting this hole
• Regulatory necessities assume provenance readability that doesn’t but exist
• Incident response relies on traceable lineage

Provenance is About Mannequin Weights

The Mannequin Provenance Structure grounds provenance in a single idea: the verifiable derivation historical past of a mannequin’s skilled weights. Two fashions share provenance if, and provided that, a causal chain of weight derivation connects them, whether or not instantly, not directly by way of distillation, or mechanically by way of a non-training transformation like quantization.

Shared structure, shared coaching information, shared tokenizer, and shared benchmark efficiency don’t rely. The exclusion is deliberate. A broader definition that handled any architectural or behavioral similarity as derivation could make licensing enforcement apply to each mannequin in an structure household, would flag convergent designs as real vulnerability hyperlinks, and would flood governance audits with false positives. Weight-level causation produces labels which can be secure throughout reviewers, sturdy to metadata manipulation, and aligned with how derivation truly occurs in follow.

How Mannequin Provenance Structure is Structured

The structure solutions three questions: when are two fashions associated? How does that relationship happen? And what seems like a relationship, however isn’t? It organizes these solutions as specific enumerations fairly than definitions-by-example, so each pair of fashions encountered in follow maps to a transparent class.

5 circumstances specify when a provenance hyperlink exists

• Direct descent: coaching initialized from a skilled checkpoint
• Oblique descent: distillation from a instructor mannequin
• Mechanical transformation: quantization, pruning, merging, or format conversion
• Id: byte-equivalent copy
• Transitivity: any composition of the above

A pair is provenance-linked if not less than one situation holds.

9 mechanisms enumerate the concrete derivation pathways noticed in follow:

• Id and reformatting
• Positive-tuning
• Continued pretraining
• Vocabulary-modified derivation
• Data distillation
• Structural modification with weight inheritance
• Quantization and compression
• Adapter-based derivation (LoRA, QLoRA, prefix tuning)
• Mannequin merging

Eight exclusions listed beneath are circumstances which will look like provenance-linked, however are provenance-independent. Every exclusion is a sample of obvious similarity, however in the end one which carries no weight-derivation chain:

• Unbiased replica (e.g., Llama-2 vs. Open LLaMA which share the identical structure and tokenizer, however are skilled from scratch)
• Identical-family different-size (e.g., Llama-2-7B vs. Llama-2-13B).
• Identical-family different-corpus coaching (e.g., T5 vs. MT5, which share a reputation root, however have separate from-scratch coaching)
• Unbiased runs below a shared seed (i.e., shared seed doesn’t represent shared weights)
• Architectural convergence (completely different groups independently arriving at related mannequin designs)
• Dimensional coincidence below completely different mechanisms (fashions that occur to share the identical dimension or form with out one being constructed from the opposite)
• Shared vocabulary with out weight switch (a tokenizer is a device, not a weight)
• Shared coaching goal (sharing an goal doesn’t hyperlink weights)

A rigorous provenance customary should identify them explicitly, as a result of complicated any of them with real derivation corrupts downstream licensing selections, vulnerability assessments, and compliance determinations.

Establishing an Proof Commonplace

A taxonomy is just as helpful because the proof customary hooked up to it. The Mannequin Provenance Structure accounts for 3 sources for establishing provenance (and however architectural similarity and naming conventions are explicitly inadequate):

• Official documentation: from the releasing group that explicitly names the mum or dad mannequin and derivation methodology
• Checkpoint verification: by way of hash matching, layer-by-layer comparability, or reproducible derivation scripts
• Authoritative third-party evaluation: that has been peer-reviewed or extensively cited

Below ambiguity, Mannequin Provenance Structure defaults to labeling a pair as provenance-independent. This conservatism is intentional. A false constructive in provenance carries fast penalties: a licensing accusation, an IP declare, a supply-chain incident notification. A false destructive will get caught by defense-in-depth by way of guide assessment, licensing audit, and forensic evaluation. Specificity wins when rigor is required.

Alignment with AI Menace Frameworks and Requirements

Mannequin provenance attestation may be thought-about a provide chain management, and the Mannequin Provenance Structure serves as a definitional layer that makes mannequin dependency auditable. It specifies what it means for a deployed mannequin to inherit from an upstream supply, which is the precondition for any significant query about inherited vulnerabilities, license obligations, or unattributed redistribution.

“weak mannequin provenance” and noting that “no ensures on the origin of the mannequin.”  The MITRE ATLAS framework paperwork provide chain compromise (AML.T0010) as a main preliminary-access approach. The Cisco AI Safety and Security Framework classifies third-party mannequin elements below OB-009 Provide Chain Compromise, with direct applicability by way of AITech-9.3 (Dependency/Plugin Compromise). The Cisco AI Safety and Security Framework classifies third-party mannequin elements below OB-009 Provide Chain Compromise, with direct applicability by way of AITech-9.3 Dependency / Plugin Compromise: actors insert malicious code, backdoors, or vulnerabilities into third-party dependencies utilized by fashions, brokers, or AI functions, creating supply-chain assaults that have an effect on all techniques utilizing the compromised part. Basis-model checkpoints reused as initialization for downstream fashions are exactly such dependencies.

The structure additionally acknowledges the adversarial dimension by way of AITech-9.2 Detection Evasion: deliberate concealment of a derivation relationship — metadata rewriting, tokenizer substitution, chained modifications meant to obscure the mum or dad. The structure’s dedication to weight-level proof, fairly than metadata-level proof, is a direct response to this adversary mannequin.

Mannequin Provenance Structure attracts from present frameworks that AI provide chain packages already depend on. These frameworks establish necessities or concerns that the structure helps fulfill. A proper provenance definition is a precondition for producing that documentation persistently throughout a company and throughout suppliers.

Desk 1. Frameworks, rules, and requirements that Mannequin Provenance Structure drew upon

A Residing Doc

New strategies of constructing fashions are rising sooner than any fastened taxonomy can accommodate. Mannequin merging, combining specialised skilled fashions, has change into a dominant approach over the previous few years. Past merging, the ecosystem is seeing Combination-of-Consultants architectures with independently skilled elements, federated coaching throughout organizations, and artificial information pipelines that blur the road between data switch and unique coaching. The Mannequin Provenance Structure considers these open frontiers and commits to revision because the panorama evolves.

Get Began

The total Mannequin Provenance Structure abstract is out there alongside this publish: https://github.com/cisco-ai-defense/model-provenance-kit/tree/most important/docs/structure

For groups able to put these definitions into follow, Mannequin Provenance Equipment supplies the tooling. Your complete pipeline runs on CPU, architectural matches resolve in milliseconds, and extracted options are cached for reuse. Take a look at Mannequin Provenance Equipment Github: https://github.com/cisco-ai-defense/model-provenance-kit

Entry a starter set of base mannequin fingerprints on Hugging Face: https://huggingface.co/datasets/cisco-ai/model-provenance-kit