The first week back from Canada, I attended two conferences — though jet lag pretty much had its way with me for nearly the whole week. Things only started improving this week, and right on its heels came prep for the iPAS AI Application Planner (Associate) exam.
I’d signed up around the tail end of AIA’s AIPM course. I happened to see information about the certification on Facebook, and while the iron was hot I registered for both the Associate and Intermediate levels at once — and roped in some AIPM classmates too. So before leaving the country, on top of wrapping up the client project I already had, I also stood up a quiz and drill site using RAG and shared it with the group. Then came a full 7-week cooldown, and it wasn’t until this Monday that I actually started drilling for myself.
The first run scared me stiff: under 50% correct, with some questions I couldn’t parse at all. It had me seriously questioning my life choices — I’d just finished AIPM, how could I still be this shaky?
The next few days were spent cycling between drilling questions, turning to ChatGPT for interactive learning whenever I got stuck, and using Claude Code to improve the system. I finally finished the full set by Thursday morning.
Even then, it didn’t feel solid. The technical side is genuinely deep, and there’s a mountain of jargon and acronyms. Even when something clicks in the moment and feels learned, the next time the question shows up I still stall out — or can’t recall the application logic behind the technique.
With only 2 days left, and this being an Associate-level certification after all, I asked the AI to generate another 100 questions for each of the two Associate subjects, following the outline of the official study guide plus the content already on the site, framed around real applications across different industries. I gave them a trial run afterward and — surprisingly — they held up. Still challenging, but the questions and answer explanations were properly done. Which in turn made the gaps in the original question bank visible to me.
I figured fixing those gaps wouldn’t take much time. It took my entire Friday: Claude’s server went down once and I had to re-run, hit a rate limit once, and finally wrapped up Friday night, by which point I was completely spent. And in the end that improvement didn’t make it in time for yesterday’s exam. So on Saturday morning I simply had the AI summarize the core AI planning workflow, mapped onto practical applications across industries, to reinforce the end-to-end concept of putting things into production — and then walked in and took the exam anyway.
My take: if you’re working full-time and already have the relevant concepts, budgeting a month for prep is the safer bet. In my situation, two weeks is probably what it takes to have any breathing room. Anyway — it’s done. Fingers crossed. Next up is finishing the Intermediate subject content, plus the AIoT certification I registered for on a whim. After that, back to normal, picking up the threads I’d set aside.
What Actually Happened on GitHub
Looking at the record, the rhythm was even more lopsided than it felt:
- 96 commits between 8/1 and 8/15, spread across only 6 working days (8/6, 8/7, and 8/10 through 8/14).
- 8/13 and 8/14 alone account for 78 of them — 54 on 8/14 alone. That was the day I’d assumed “wouldn’t take much time.”
- Essentially one repo: 95 commits in
ipas-aiap-cert, 1 inai-hil-mcp. Six other private projects saw zero activity.
In other words, this wasn’t “prepping on the side.” Prep crowded out everything else.
Six Branches on the Drill Site
1. Quantifying the “I can’t find this” hunch (8/10)
My gut feeling was that working backward from the question bank to the study guide, a lot of it simply wasn’t there. Rather than patch by feel, I inventoried all 435 Associate questions one by one — extracting the concept tested and checking it against the study notes. The result: 55% of tested concepts weren’t covered. The hunch was right, and worse than I’d assumed.
The fix was 102 concept cards and 305 key points, each card in three parts: definition / how the bank tests it (with question numbers) / common confusions. I also verified a few time-sensitive updates that directly change the context of existing questions: Taiwan’s AI Basic Act took effect on 2026-01-14 (it was still a draft when the questions were written), the EU AI Act’s high-risk obligations slipped to 2027-12, and MCP has been donated to the Linux Foundation. The easiest trap with past exam questions is treating an outdated timeline as the answer.
2. Resume and jump-to-question (8/11)
Very practical features: progress saved to localStorage, jump to a specific question number, progress hints in the mode picker. Two days of drilling tells you exactly what you’re missing.
3. That D option on question 160 (8/12)
While drilling, question 160’s option D rendered 1,155 characters, with a total page height of 4,150px. Tracing it down, the cause was an asymmetry in the parser: the answer parser knew where questions began, but the question parser didn’t know where answers began. So everything after 1. Ans（D） got swallowed into the last option.
My initial call was “two isolated cases.” One line of scanning script revealed it was 51 questions across all five subjects. That’s the most practical lesson of these two weeks: measure the scope before drawing a conclusion — it’s worth running a full scan while you’re already doing the inventory.
4. A new bank organized by assessment topic (8/13)
These are the 200 questions from earlier in this post. The point isn’t “200 more questions” — it’s that the generation logic changed. The original bank follows the question-type distribution of past exam papers; the new one is built around official assessment topic nodes, with a fixed quota per node. That makes it usable for drilling one specific node, rather than being dragged along by whatever ratio the exam papers happened to use.
5. Rewriting every explanation and per-option analysis (late 8/13 into 8/14)
This is the part I thought “wouldn’t take much time.” Measuring it revealed the problem wasn’t a few badly written questions — it was structural. The per-option analyses weren’t hand-written; they were assembled by matching option text against a 106-entry glossary. That only works when options are simple noun phrases, and real exam questions are mostly scenario-based with full-sentence options. When nothing matched, you got generic filler. The actual number: 1,135 of 1,305 fields were the same filler sentence.
I deleted the whole auto-assembly mechanism and hand-wrote all 435 explanations and 1,305 per-option analyses. Tests went from about 140 to 219. Thirty-eight commits, one full day.
6. Better blank than pretending (8/14, wrapping up)
When wiring the per-option analyses into the exam review page, I realized that doing it straight would leave 97% of the wrong-answer fields in Intermediate senior-ml showing the same generic filler — which is precisely the reason I’d deleted the old mechanism: it made “no explanation” look like “has an explanation.” So I gated it on whether a question has a hand-written analysis: all 435 Associate questions show it, and for the 332 Intermediate questions the section doesn’t render at all.
ai-hil-mcp: The One Time Deletions Won
8/7 was a full day spent unifying six MCP crates that had silently drifted apart onto a shared hil-core workspace. Forty-six files, a net −1,900 lines.
The five defects I fixed along the way were all textbook: vision-mcp checked try_recv().is_ok() to detect a stop signal, but stopping was implemented by dropping the sender, so the camera was never released. capture_frame returned base64 inside a text block — invisible to the model, at a cost of tens of thousands of tokens per frame. And jtag-mcp’s diagnostics ended with an unconditional core.run(), cancelling the halt that’s supposed to precede flashing.
I also moved safety rules out of prose and into code — a voltage ceiling, and confirm=true required for erase_flash. Telling a model in a prompt “don’t do X” and making X impossible are two different things.
What the Drilling Actually Taught Me
The real output of these two weeks isn’t a question count. It’s a handful of rules that got validated over and over:
Keywords don’t find sentence patterns. To find the “the other three options are all…” construction that crushes three distractors into one clause: a keyword grep found 5 instances; semantic enumeration found 16. Another batch was 12 versus 28. Searching for a pattern by keyword finds sentences that used that keyword — not that pattern.
Judging by feel which ones to keep is completely unreliable. Of 40 explanations initially judged “worth keeping,” 33 ended up rewritten. Across all 369 real exam explanations, only 57 survived word-for-word. They escaped the first pass not because they were better, but because they read smoothly.
Tests may not bite at all. In a mutation test, replacing an entire per-option analysis with the single character "x" still passed — because the length and template checks only applied to explanations, while analyses were only checked for non-emptiness. Each batch produced 200 strings and the tests certified 50 of them; the 150 they missed were the entire reason the work existed. The guard added in batch three caught things. A guard added at the very end catches nothing.
A consistency check can be circular reasoning. Validating explanations against notes that were themselves reverse-engineered from the same question bank — agreement is the expected outcome, not evidence of correctness.
Disclosure isn’t a fix. Twice, a report honestly flagged “this is the weakest cell in this batch” — and then didn’t fix it. Honest disclosure is worth crediting, but writing it in the report doesn’t mean the problem is handled.
What’s Next
- The new bank and hand-written explanations for the three Intermediate subjects are still undone; the analysis section for those 332 questions is currently hidden.
- Around 40 known answer-key disputes are logged in the bank defect notes, none of them fact-checked by a human yet.
- Health checks for leftover option text and page-footer noise are still one-off scripts — worth hardening into pipeline tests.
- The AIoT certification I signed up for on a whim.
- And then back to normal, picking up the threads I’d set aside.
Looking back, these two weeks were nominally about exam prep, but I was really doing the same thing throughout: turning “something feels off” into a number I can measure, then turning that number into a guard I can verify. 50% correct, a 55% coverage gap, 1,135 generic fillers, 51 polluted questions — every one of them started as one vague complaint.
Comments & Feedback