AI Practitioner — the advanced course.

• LO1.1Explain how transformer-based models represent and generate language (tokens, embeddings, attention) at working depth.
• LO1.2Compare fine-tuning, retrieval (RAG) and agentic architectures and select the fit for a given problem.
• LO1.3Evaluate hosted-API versus open-weights deployment against cost, latency, capability and data-sovereignty constraints.

Core content

Embeddings turn text into numeric vectors that capture meaning — the foundation of semantic search and retrieval. Attention lets a model weigh every token against every other, which is why context placement matters. Model classes now differentiate: fast general models, multimodal models, and reasoning-class models that trade latency and cost for stronger multi-step problem solving — match the class to the task, not the hype.

The architecture decision

• RAG when knowledge changes or must be cited — retrieval at question time keeps answers current without retraining.
• Fine-tuning when you need consistent style, format or domain behaviour — not for injecting fast-moving facts.
• Agents when the job needs multi-step actions against tools and systems, not just answers.
• Open-weights on your infrastructure when data genuinely cannot leave your control; hosted APIs when capability-per-dollar and zero ops win.

• LO2.1Analyse business processes to identify and prioritise viable AI use cases.
• LO2.2Justify build-versus-buy decisions.
• LO2.3Establish baselines and success metrics before piloting; communicate recommendations to decision-makers.

Core content

Prioritise use cases on three axes: frequency × risk × measurability. The best first project is high-frequency, low-risk and measurable — not the moonshot. Buy when the need is commodity (meeting notes, generic chat); build when the workflow differentiates you and runs on data only you have. Before any pilot, baseline the current state — minutes per task, cost per ticket, error rates — or you'll never credibly claim improvement. Define the success metric up front and tie it to a business outcome; "the team liked it" is not a metric. Then write the recommendation so a non-technical executive can decide in five minutes — transmitting analysis to others is the practitioner's real deliverable.

• LO3.1Construct layered prompts separating system rules, task instruction and context.
• LO3.2Produce validated, machine-readable structured outputs.
• LO3.3Decompose complex tasks; manage prompts as versioned, tested assets.

Core content

In production, prompting stops being chat and becomes engineering. The system prompt carries persistent role, rules and constraints; user messages carry the task; context is injected deliberately. For machine-readable output, request a strict JSON schema, validate every response, and retry on parse failure — never "ask nicely and hope". Complex tasks gain reliability through decomposition: smaller steps or chained calls, each verifiable. And production prompts are assets: version-controlled, documented, and changed only after passing the evaluation set — a one-word edit can shift behaviour across thousands of runs.

• LO4.1Design golden-set evaluations and scoring rubrics.
• LO4.2Apply LLM-as-judge methods critically, with calibration against human judgement.
• LO4.3Operate regression testing and production monitoring for AI quality.

Core content

"It seems good" doesn't survive contact with scale. A golden set — curated inputs with expected outputs — makes quality repeatable: run it before every prompt change, model upgrade or vendor switch, and compare. LLM-as-judge scales scoring but carries biases (verbosity preference, self-preference, position effects) — calibrate the judge against a sample of human ratings before trusting it. When a new model version ships, the eval suite decides the swap — never the changelog. After launch, monitor sampled production outputs with thresholds and alerts: inputs drift, behaviour drifts, and silent degradation is the default if nobody is looking.

• LO5.1Design retrieval pipelines — chunking, embedding and context assembly.
• LO5.2Implement guardrails that validate inputs and outputs around model calls.
• LO5.3Apply data minimisation to third-party AI services; engineer fallbacks for failure.

Core content

Real systems wrap the model. In retrieval pipelines, chunking strategy — size, boundaries, overlap — drives what gets found and how much fits the context budget; bad chunking quietly breaks good RAG. A guardrail layer validates and constrains both directions: sanitise and scope inputs, then schema-check, filter and bound outputs before anything downstream consumes them — the model alone is never the control. Before personal information reaches a third-party API, minimise and redact to what the task needs, per your privacy policy. And design for failure as a certainty, not an exception: retries, graceful degradation, and a human fallback queue — an automation that fails silently is worse than no automation.

• LO6.1Explain agent loops — planning, tool calls, observation — and where they fit.
• LO6.2Analyse agent-specific failure modes, including indirect prompt injection.
• LO6.3Design least-privilege tool access, risk-tiered human-in-the-loop gates and audit logging.

Core content

An agent is a model in a loop: plan → call a tool → observe → continue, toward a goal. That capability changes the risk class entirely, because agents act on real systems. Three design disciplines keep them safe: least-privilege tools (an email-drafting agent doesn't need delete rights), risk-tiered human gates (irreversible, financial or outward-facing actions require approval; low-risk reads don't), and audit logs of every action with its triggering context. The signature attack is indirect prompt injection: malicious instructions hidden in content the agent retrieves — a webpage, an email, a document — which the model then obeys. Treat all fetched content as data, never as instructions, and assume it's hostile until proven otherwise.

• LO7.1Apply Privacy Act 1988 (Cth) / APP obligations to AI data flows.
• LO7.2Map AI systems against Australia's AI Ethics Principles; maintain documentation that supports accountability.
• LO7.3Risk-tier AI uses and outline incident response.

Core content

AI doesn't get a privacy exemption: when personal information is collected, used or disclosed through an AI system, the same APP obligations apply as anywhere else — collection notice, purpose limitation, security, access and correction. Australia's AI Ethics Principles — eight voluntary principles including human wellbeing, fairness, privacy protection, transparency & explainability, contestability and accountability — are the standard frame for assessing a system; map each deployment against them and record the result. Maintain documentation — model/system cards, decision records, data-flow maps — because transparency and auditability are what regulators, clients and your own future team will ask for. Tier uses by risk (an internal drafting aid ≠ an automated decision affecting a person) and pre-agree an incident plan: what triggers escalation, who pauses the system, who is told. Awareness extends abroad — the EU AI Act's risk-based approach signals where global practice is heading.

• LO8.1Distinguish prompt injection, jailbreaks and data-exfiltration attacks.
• LO8.2Handle model output as untrusted input throughout the stack.
• LO8.3Plan adversarial (red-team) testing before deployment.

Core content

Know your attack taxonomy: a jailbreak is a user trying to bypass the model's own rules; prompt injection is an attacker hijacking your system through inputs or retrieved content; exfiltration is coaxing the system into leaking data it can see. Defences are architectural, not just prompt-level: separate instructions from data, scope what the model can see per request, filter outputs. Treat model output as untrusted input everywhere it lands — render it unescaped into a web page and you've built a cross-site-scripting (XSS) vector; pass it to a shell or SQL and you've built worse. Before deployment, red-team it: deliberate injection attempts, hostile documents in retrieval, boundary probing — the happy path proves nothing. Supply chain counts too: models, extensions and plugins you adopt inherit their publishers' trustworthiness.