⚡ AI Automation Masterclass

The AI Automation Masterclass - Evaluation & Follow-Up Plan

Kirkpatrick Evaluation Strategy

1. Introduction & Objectives

To ensure the training provided by Keith NG delivers high-impact results for business owners, we utilize the Kirkpatrick Evaluation Model to measure success across three distinct levels:

  1. Level 1: Reaction – Participant satisfaction with the content, delivery, and tools.
  2. Level 2: Learning – Verification of knowledge gain (AI concepts, vibe coding, APIs).
  3. Level 3: Behavior – Measurement of post-course implementation (active automation pipelines 30 days later).

2. Level 1: Reaction

Execution: Handed out digitally at 17:45 on the day of the course.

Key Metrics:

  • Course Relevance (Goal: $\ge$ 4.5/5)
  • Facilitator Effectiveness (Goal: $\ge$ 4.7/5)
  • Platform Usability & Tool Readiness (Goal: $\ge$ 4.3/5)
  • Net Promoter Score (NPS) (Goal: $\ge$ +70)

The Level 1 survey is detailed in the assessments folder.

3. Level 2: Learning

Execution: Evaluated via a Pre-Training Knowledge Check (prior to Section 1) and a Post-Training Quiz (at the end of Section 6).

Key Metrics:

  • Average baseline score vs. average post-quiz score (Goal: minimum 30% absolute improvement in technical knowledge).
  • Individual pass rate (Goal: 90% of participants scoring $\ge$ 70% on the Post-Quiz).

4. Level 3: Behavior & Application (30-Day Follow-Up)

Execution: Sent via email 30 days after the training.

Metrics to Track:

  • Implementation Rate: % of students with a running, automated pipeline in their business.
  • Time Saved: Average weekly hours saved by the automation (Goal: $\ge$ 4 hours/week).
  • Tool Adoption: Number of students using Antigravity, Gemini Notebook, or DeepSeek API independently.

Level 3 Survey Structure:

  1. Since the workshop, have you run or modified the pipeline you built? (Yes / No)
  2. If yes, what is it automating (e.g., video processing, scriptwriting, database entry)?
  3. Approximately how many hours per week is this automation saving you?
  4. Have you built any new automations using the vibe coding principles? If so, what?
  5. What has been the biggest technical bottleneck or challenge you've faced?
  6. Would you benefit from a 1-on-1 coaching call with Keith NG?

5. Feedback Analysis & Continuous Improvement

  • Weekly Review: Review Level 1 feedback and Post-Quiz errors after each cohort.
  • Troubleshooting Updates: If students frequently fail questions related to API endpoints or Antigravity workspaces, adjust Section 3/4 lecture times to allow more hands-on guidance.
  • Case Studies Library: Compile successful Level 3 implementation stories to feature in the marketing enrollment notice.