
TL;DR
For the security-agent lab in this series, begin with an existing instruction-tuned model, current evidence supplied through retrieval, established assessment tools, and deterministic controls. Consider supervised fine-tuning only after evaluation identifies a repeatable model-behavior problem that better evidence, clearer instructions, or ordinary code does not resolve.
The RAG versus fine-tuning decision should follow the defect you are trying to fix. A revised advisory calls for an evidence update. An expired assessment credential calls for an operational repair. Repeatedly mishandling complete, correct evidence may justify model adaptation. None of these changes should grant additional execution authority.
Put changing security facts in a governed evidence path. Adapt model behavior only for a demonstrated need. Keep authorization outside both.
In This Article
- Introduction
- Choose the Mechanism by What Must Change
- Pretraining Is a Different Project
- RAG Supplies Evidence Without Making It Authoritative
- Fine-Tuning Is a Candidate for Persistent Behavior Problems
- Tools Establish Observations; Controls Establish Permission
- Follow the Evidence Through Two Separate Change Paths
- Choose Data Placement Before the Training Method
- Require a Comparison That Can Reject Fine-Tuning
- Record the Lab Decision Before Building It
- Conclusion
Introduction
Example Enterprise’s document portal uses the fictional Example Renderer component. The security team has an advisory, a deployment record, and a proposed remediation workflow. Someone suggests training a model on the advisory collection so it can identify the right fix.
Before approving that work, ask what is missing. Does the model lack the current advisory? Is the deployed version unknown? Does the assessment tool lack a relevant check? Or does the model receive adequate evidence but repeatedly produce an incomplete recommendation?
Those are different engineering problems with different remedies.
The opening article established the advisory-to-remediation architecture. This installment chooses how to supply knowledge and shape behavior within it. The outcome is a model-and-knowledge decision record, not a training run. Example Enterprise, its component versions, and the advisory examples remain fictional. No model comparison, scan, or deployment has been executed for this article.
Choose the Mechanism by What Must Change
Model weights are learned numerical parameters. Training changes parameters; an ordinary inference request uses them to generate an output. Supplying a document in that request is not the same operation as training on it. Hugging Face’s training documentation distinguishes adapting pretrained weights from starting a new model, while Microsoft’s RAG guidance describes supplying retrieved material as model input. Hugging Face: Fine-tuning
Use the following table as a responsibility map for the proposed lab, not as a ranking of competing products.
| Mechanism | What changes | Appropriate security task | What it does not establish |
|---|---|---|---|
| Pretraining from scratch | Newly initialized model parameters | Research requiring a new foundation or materially different data domain | Enterprise assessment capability merely because training completes |
| Supervised fine-tuning | Selected model parameters or adapter parameters | More consistent interpretation and presentation of reviewed findings | That yesterday’s fix is still supported today |
| Retrieval-augmented generation (RAG) | Evidence supplied at inference time | Current advisories, supported fixes, and authorized local context | That retrieved evidence is complete, correct, or applicable |
| Tools and scanners | Observations from specific supported operations | Package assessment, configuration checks, or authenticated testing | Comprehensive coverage outside the checks performed |
| Deterministic rules | Explicit matching and validation logic | Known product mappings, required fields, and permitted state transitions | Resolution of ambiguous packaging or missing evidence |
| Agent configuration and orchestration | Instructions, tool definitions, and workflow behavior | Routing, bounded retries, escalation, and draft generation | A model-weight update or permission to act |
Anthropic’s Building effective agents distinguishes predefined workflows from systems in which a model directs its own tool use. For this lab, keep the known sequence in code and reserve model calls for interpretation and drafting. Naming a component an agent does not require making its workflow autonomous. Anthropic
Pretraining Is a Different Project
Training a language model from scratch begins without reusing a pretrained model’s learned weights. It requires a suitable corpus, a training objective, model and tokenization choices, compute, and evaluation. Hugging Face’s from-scratch tutorial notes the substantially greater compute requirement compared with adapting an existing model. Hugging Face: Training a causal language model from scratch
A small educational model can demonstrate tokenization, optimization, checkpointing, and overfitting. That is useful training for the engineer. It is not evidence that the resulting model can interpret vendor backports, preserve uncertainty, or distinguish a failed assessment from a negative finding.
For Example Enterprise, the proposed decision is not to pretrain. The immediate task is interpreting bounded evidence, not developing a new language foundation.
Reconsider only when existing models fail a documented requirement and the organization can support the data, evaluation, and operating responsibilities of the alternative. Continuing to train an existing model on domain text is also a weight-changing activity; it is not a substitute for maintaining a current advisory service. Hugging Face: Fine-tuning
RAG Supplies Evidence Without Making It Authoritative
Retrieval-augmented generation supplies selected external material to a model when it answers. Retrieval can use structured queries, keyword search, vector similarity, or combinations. Creating an embedding, a numerical representation used for similarity search, with an existing embedding model does not itself retrain the answering model. Microsoft’s RAG documentation describes these retrieval and context-assembly responsibilities separately from fine-tuning. Microsoft Learn
For this lab, retrieve by asset context before asking for a recommendation. Match the component, packaging, release branch, and relevant configuration. Supply the applicable advisory passages, their revisions, and fresh local observations. Use semantic search to find explanatory material, not to override an exact product mismatch.
Preserve the conditions attached to a fix. A passage that names a target version but omits a required intermediate upgrade is insufficient for the change proposal. Where guidance conflicts, retain the conflict and route it to an owner rather than letting the model select whichever passage sounds more definitive.
RAG also needs explicit access and freshness controls. Microsoft warns that incomplete retrieval can still produce inaccurate answers and recommends enforcing access at retrieval time. In Example Enterprise, a missing source, expired evidence, or unresolved permission check must remain a visible restriction on the answer. Microsoft Learn
This is why the retrieval service and its records need their own lifecycle. A model refresh cannot compensate for an advisory revision that never entered the evidence path.
Fine-Tuning Is a Candidate for Persistent Behavior Problems
Supervised fine-tuning (SFT) adapts a pretrained model using examples of the desired task behavior. Hugging Face’s guidance recommends first evaluating whether an existing instruction-tuned model and suitable prompts are sufficient. Its TRL library provides an SFT trainer for learning from task examples, including input-and-output sequences. Hugging Face: Supervised Fine-Tuning
A suitable experiment for this series is improving the consistency of evidence-backed finding summaries. The input contains the advisory conditions, local observations, assessment status, and approved procedure excerpts. The target response preserves applicability, uncertainty, prerequisites, and rollback limitations in the required structure.
Fine-tuning can change learned knowledge as well as behavior. The recommendation to keep volatile facts in retrieval is an operating choice, not a claim that models cannot learn facts. Today’s supported target version should remain tied to a revisable source rather than depend on what a training example taught the model. Microsoft
Prepare examples that show when not to recommend a fix. Include unsupported target versions, vendor-backported packages, conflicting sources, missing configuration evidence, and failed authentication. Remove secrets and review permitted use before admitting material into a dataset. Do not turn raw advisories or user feedback into automatic continual training.
LoRA Changes the Adaptation Cost, Not the Trust Boundary
Low-Rank Adaptation (LoRA) trains relatively small weight-update matrices while leaving the base weights frozen. Hugging Face’s PEFT library supports this parameter-efficient adaptation. QLoRA combines LoRA training with a quantized base model, commonly represented at four-bit weight precision, to reduce memory requirements. Hugging Face: LoRA
These methods do not establish suitability for security analysis. The adapted model still needs task evaluation, protected artifacts, a compatible base model, and a rollback path. Size a training experiment from the selected model, precision, sequence length, batch configuration, and optimizer, rather than promising that every security model fits on one particular GPU.
The later fine-tuning tutorial will pin the model and compatible training stack. This article deliberately leaves those implementation choices open rather than selecting hardware before defining the task.
Tools Establish Observations; Controls Establish Permission
In the proposed design, a model may explain an assessment record, but it cannot rewrite a failed authentication status into a successful scan. Likewise, a persuasive change proposal cannot satisfy an approval requirement. Keep these fields under the services that actually observe or authorize them.
OWASP’s excessive-agency guidance recommends narrow tools, minimum permissions, and authorization in downstream systems rather than relying on the language model to decide whether an action is allowed. OWASP Gen AI Security Project
Apply that boundary even to a fine-tuned model with strong evaluation results. It may propose an assessment profile or draft a ticket. The controller still resolves the registered asset, checks scope, and validates the operation. Adding a scanner adapter changes the system’s assessment capability; retraining the model is neither necessary nor sufficient to grant access to it.
Follow the Evidence Through Two Separate Change Paths
The diagram shows the proposed split between changing evidence and changing model behavior. The training path accepts reviewed examples only. Both model variants remain behind the same validation and approval boundary.

The unchanged baseline also enters the registry through evaluation. The model registry represents a controlled release, not permission to use production tools. The runtime evidence bundle remains necessary whether the selected model is adapted or unchanged.
A Supplier Withdraws the Proposed Fix
Return to asset EE-APP-001, finding EE-FIND-001, and synthetic advisory LAB-ADV-001. The original lab scenario uses Example Renderer 1.4.2 and proposes 1.4.3 as the fix.
Now create a counterfactual test branch: advisory revision 2 withdraws 1.4.3 as a supported remediation, leaves the affected conditions unchanged, and supplies no replacement. Assume fresh deployment evidence still confirms those conditions.
The evidence store and pending proposal must change. The model weights need not. The expected reviewed result is that the asset remains affected but the previously supported target is no longer available. Do not invent 1.4.4.
This abbreviated JSON illustrates the expected record, not output from an executed model:
{
"asset_id": "EE-APP-001",
"finding_id": "EE-FIND-001",
"advisory_revision": "LAB-ADV-001:2",
"proposed_applicability": "affected",
"supported_target_version": null,
"blocking_gaps": ["No replacement fix verified"],
"next_action": "Owner review of supported alternatives"
}
The controller should invalidate any pending approval bound to the withdrawn remediation in EE-CHG-001. EE-VER-001 cannot establish closure until an appropriate change and its completion conditions exist. This branch does not rewrite the series’ original synthetic advisory.
The Assessment Cannot Authenticate
In another test case, the inventory observation predates the latest deployment and the authenticated assessment fails. Neither record establishes the current version.
Repair the evidence path: obtain a fresh authorized deployment observation and resolve the assessment failure. Preserve assessment_failed separately from applicability_unknown. Training on more advisories would not supply the missing observation.
The model can explain what remains unknown and draft the request for evidence. The workflow should reject a clean disposition based solely on the failed assessment.
The Evidence Is Complete, but the Draft Omits a Limitation
Suppose the approved procedure explains that rollback can restore an affected build. The evidence bundle includes that passage, yet the model repeatedly omits the limitation from remediation drafts.
First revise the instruction and output contract. Make rollback limitations a required field, and verify that each material statement is supported by the supplied passages. A populated field containing an irrelevant sentence should not pass review.
If the failure persists across representative held-out cases, reviewed examples may justify an SFT experiment. The experiment changes behavior; it does not replace the underlying procedure or authorize automatic rollback.
Choose Data Placement Before the Training Method
For the reference lab, propose self-hosted inference using synthetic evidence and an approved pretrained model. This makes the base model and any later adapter available for controlled comparison. It is a design proposal, not evidence that a model, runtime, license, or hardware configuration has been approved.
Hosted inference remains a valid candidate where the selected service permits the task and meets data-handling requirements. Compare inference and training separately: approved inference access does not automatically establish permission to upload the same material into a training service. Review retention, telemetry, support access, subprocessors, and data use for the actual offering.
Do not treat self-hosting as an access-control mechanism. In particular, a document-index permission change does not undo a model-weight update. Microsoft Research has demonstrated confidential-data leakage risks in enterprise fine-tuning and RAG systems without adequate authorization controls. Microsoft
For this lab, exclude secrets and access-restricted operational facts from shared training examples. Keep sensitive facts in an authorized evidence path and use sanitized or synthetic examples to explore behavior adaptation. Protect datasets, checkpoints, adapters, caches, and logs according to what they can expose.
Budget separately for retrieval and inference, dataset review, adaptation experiments, evaluation, and artifact operations. Measure cost per accepted proposal, including correction effort. Fine-tuning is not a saving if it reduces prompt tokens but increases unsupported recommendations.
Require a Comparison That Can Reject Fine-Tuning
Use three candidate designs against the same task: deterministic matching with a template, the pretrained instruction-tuned model with RAG, and that model with a reviewed adapter and the same RAG evidence. The first candidate matters because some findings may not need generation at all.
For the model comparison, hold the evidence snapshots, task contract, inference limits, and downstream validators constant. Separate training, development, and final evaluation cases. Keep related advisory revisions and near-duplicate product cases together where separation would otherwise leak the answer into training. Add a time-based holdout to test evidence that was unavailable during adaptation.
Score the actual decisions: applicability, support for the target version, passage-level evidence, justified uncertainty, and preservation of assessment failures. Track review corrections, latency, and operating cost separately. A lower training loss is not the acceptance condition for a remediation system.
Require the adapter to demonstrate a worthwhile improvement without unacceptable regressions. Define those thresholds before evaluating it. If the baseline is sufficient, retain it. If both models fail because the relevant passage is missing, repair retrieval rather than declaring the adapter ineffective or buying a larger model.
OWASP cautions that neither RAG nor fine-tuning fully mitigates prompt injection. Include a harmless malicious instruction in retrieved material and verify that external controls deny prohibited access even when a model requests it. Model agreement or refusal alone is not proof of enforcement. OWASP Gen AI Security Project
Record the Lab Decision Before Building It
The following architecture decision record captures the proposed boundary for Example Enterprise. It is illustrative YAML, not a vendor configuration or an executable policy. Store it with the lab’s design records and replace the unresolved model details only after selection and validation.
decision_id: EE-ADR-002 version: "0.1" status: proposed_reference_design organization: Example Enterprise task: draft_evidence_backed_findings_and_remediation_proposals baseline: model: pretrained_instruction_tuned model_revision: null hosting: self_hosted_lab adapter: none evidence: permission_filtered_versioned_retrieval responsibilities: current_facts: advisory_store_and_fresh_asset_observations known_applicability: product_specific_matching_rules assessment_status: assessment_worker_records interpretation: model_proposal_with_supporting_passages authorization: external_policy_and_existing_change_control adaptation: pretraining: not_selected supervised_fine_tuning: conditional_on_baseline_evaluation permitted_data: reviewed_sanitized_or_synthetic_examples automatic_learning_from_raw_inputs: false promotion: compare_with: unchanged_model_and_same_evidence required: task_improvement_without_unacceptable_regressions production_execution: prohibited_for_this_lab_stage rollback: prior_evaluated_model_and_configuration
The null model revision is an unresolved selection, not a usable runtime value. Successful use of this record means the implementation preserves these responsibilities and records evidence for each promotion decision. It does not mean that parsing the YAML deploys or secures anything.
Assign the model experiment to the AI platform owner and the evidence admission rules to security engineering. Application owners retain responsibility for deployment observations and change acceptance. Revisit the decision when the task, model, evidence pipeline, data boundary, or tool interface changes.
Conclusion
Training, fine-tuning, retrieval, tools, and rules address different parts of a security system. The useful design choice is the smallest change that corrects the observed defect while preserving evidence and authority boundaries.
For Example Enterprise, that means an existing instruction-tuned model with current retrieval, deterministic applicability logic where justified, and established assessment tools. Fine-tuning remains an experiment that must earn its place through comparison, not a prerequisite for calling the system an AI security agent.
The next installment, Build the Asset Inventory Your Security Agents Need, establishes the assets, owners, versions, and observations that make those decisions possible.
Before changing the model, identify whether the failure belongs to knowledge, observation, behavior, or authority.
Related Reading
- Fine-Tune Your Own Security Model on Reviewed Findings and Remediations
- Test Your Security Model Before Giving It Operational Access
- Keep Security Agents Current with Trusted Advisories and Best Practices
External References
- Hugging Face: Fine-tuning
- Hugging Face: Training a causal language model from scratch
- Microsoft Learn: Retrieval augmented generation (RAG) and indexes
- Hugging Face: Supervised Fine-Tuning
- Hugging Face: SFT Trainer
- Hugging Face: LoRA
- Hugging Face: Quantization
- Anthropic: Building effective agents
- Microsoft Research: Enterprise AI Must Enforce Participant-Aware Access Control
- OWASP Gen AI Security Project: LLM06:2025 Excessive Agency
- OWASP Gen AI Security Project: LLM01:2025 Prompt Injection