Loading page
Getting things ready…
Please hold on for a moment while the content loads.
Please hold on for a moment while the content loads.
AI-Powered Technical Assessment Platform
Full-lifecycle assessment platform with AI-generated questions, event-driven state machine, and gamified ranking system serving technical skill evaluation at scale.
A technical assessment platform that uses AI to generate questions, evaluate answers, and map student skills to appropriate courses. The system uses an event-driven state machine to govern assessment lifecycles, supports concurrent timed exams with real-time synchronization, and features a gamified XP ranking system across five tiers.
System architecture showing frontend, API, real-time sync via change streams, and AI provider integration
State machine governing assessment lifecycle with guard conditions on each transition
Assessment lifecycle governed by a state machine that prevents invalid state transitions. More reliable than boolean status flags.
Question generation and scoring powered by both OpenAI and Gemini providers, offering redundancy and cost optimization.
Five-tier ranking system with per-assessment score caps and tie-breaking logic to prevent score inflation from repeated attempts.
Concurrent timed exams with real-time synchronization using database change streams instead of polling.
Four-tier role hierarchy (super admin, admin, instructor, student) with attribute-based permission checks.
Pre-computed leaderboard views avoid expensive aggregation queries on every request. They update incrementally on score changes.
Assessment lifecycles can have invalid transitions (e.g., publishing an incomplete assessment). Boolean flags don't prevent invalid state changes.
An event-driven state machine governs all transitions, preventing invalid state changes and providing a clear audit trail of assessment lifecycle events.
// Event-driven state machine: prevents invalid transitions.
// Guard functions enforce domain rules before allowing changes.
type State = "draft" | "published" | "active" | "graded" | "archived"
type Event = "publish" | "start" | "complete" | "archive"
const machine: Record<State, { event: Event; target: State }[]> = {
draft: [{ event: "publish", target: "published" }],
published: [{ event: "start", target: "active" }],
active: [{ event: "complete", target: "graded" }],
graded: [{ event: "archive", target: "archived" }],
archived: [],
}
function transition(current: State, event: Event): State {
const allowed = machine[current]
const match = allowed.find(t => t.event === event)
if (!match) throw new Error(`Invalid transition: ${current} via ${event}`)
return match.target
}Relying on a single AI provider creates a single point of failure and limits cost optimization opportunities.
Both OpenAI and Gemini are integrated. The system can failover between providers and route different workloads to optimize cost and latency.
Concurrent timed exams need real-time state synchronization. Polling creates unnecessary load and introduces latency. However, change streams introduce operational complexity: resume token management, oplog size limits, and WebSocket connection scalability.
Change streams provide real-time event notifications without polling overhead. Operational mitigations: resume tokens persisted in a dedicated collection for crash recovery, oplog size monitoring with alerting, WebSocket connections managed via connection pooling with grace period for reconnection. Fallback polling mechanism as safety net if change stream lags beyond threshold.
The leaderboard is queried on every page load. Computing rankings from raw scores on each request would be expensive.
A materialized view stores pre-computed rankings, updated incrementally on score changes. This keeps leaderboard queries fast regardless of participant count.
The hard part of Skill Mapper isn't CRUD it's that hundreds of students take timed exams concurrently, and the system must stay correct under racing submissions, disconnections, and rule violations. Three decisions define the model.
The exam deadline is computed server-side from the moment the assessment becomes active, never from the client's device clock. Client timers are presentation only; every submission carries the server-issued attempt record, so a student who pauses, tabs away, or rewinds their clock gets no advantage. This also means the server can enforce hard cut-off regardless of client state.
Each attempt has an owner and a lifecycle (started → submitted → graded). Submission transitions the attempt state atomically, so two racing "submit" requests can't both win: the state machine accepts the first transition and rejects the second. This gives idempotency a client retry after a network blip is a no-op, not a double grade.
// Atomic submit: only one transition wins.
// A retried submit sees state already "submitted" -> rejected.
const result = await attempts.collection.findOneAndUpdate(
{ _id: attemptId, status: "active" },
{ $set: { status: "submitted", submittedAt: new Date() } },
{ returnDocument: "after" }
)
if (!result.value) throw new Error("attempt already submitted")Rule violations (e.g., leaving the exam window) can fire close to the submit moment. Instead of a free-floating "disqualify" flag that could race with grading, disqualification is modeled as an event that transitions the attempt state. Whichever event wins the transition, grading logic checks the final state no interleaving where a disqualified attempt gets a grade and a revoked one.
At 500 DAU a single MongoDB instance and one app replica are sufficient, so distributed locks aren't needed yet atomic find-and-modify gives single-writer semantics. If this scaled to thousands of concurrent exams, the next step would be moving grading to a queue (reliability + backpressure) and sharding attempts by exam. That's a future trade-off, not one we've needed to make.
Prevents invalid transitions, clear audit trail, deterministic lifecycle
More code, steeper learning curve, harder to modify transitions
Redundancy, cost optimization, workload routing flexibility
Two integrations to maintain, different response formats, higher initial implementation cost
Fast queries regardless of data size, predictable response times
Eventual consistency, incremental update logic, additional storage
Real-time updates, no polling load, efficient resource usage
Database-specific feature, connection management overhead, session tracking
Change streams power real-time exam synchronization, but they require careful operational management. Here is how each operational concern is addressed:
Resume Token Management
Resume tokens are persisted to a dedicated MongoDB collection after each batch of events. On restart or crash, the last known token is loaded, ensuring no events are missed. Token staleness is monitored. If a token is too old, the oplog may have cycled, triggering fallback polling.
Oplog Size Monitoring
Oplog size is tracked via MongoDB's rs.status(). Alert triggers when oplog window falls below configured threshold (e.g., 6 hours). Oplog size is sized at deployment to handle peak write volumes during active exam periods.
WebSocket Connection Management
Each client establishes a WebSocket via Socket.IO with heartbeat pings every 30s. Connection pool limits prevent resource exhaustion. On disconnect, clients have a 10s grace window to reconnect and recover their change stream cursor.
Fallback Polling Mechanism
If change stream lag exceeds 5s (detected via timestamp comparison), the system degrades to polling at 2s intervals. This safety net prevents complete synchronization loss if change streams fail or lag significantly.
Connection Scaling
Change streams are multiplexed through a single change stream per collection with fan-out via an internal event bus. This avoids creating individual change streams per connected client, which would not scale.
Change Stream Resumption Failure
Real-time sync stops, exam state becomes stale
Persisted resume tokens enable recovery. Fallback polling activates if lag exceeds threshold. Alert on change stream cursor invalidation.
AI Provider Unavailable
Question generation and scoring fail
Automatic failover to secondary AI provider. Queued generation requests with retry. Cached question templates as fallback for common assessment types.
Oplog Window Exceeded
Change stream cannot resume from stored token
Oplog size monitoring alerts before window becomes critical. On token expiry, full sync is triggered for affected assessments. Alert triggers manual intervention.
Concurrent Exam Submission Overload
Scoring engine backlog, delayed results
Queue-based submission processing with configurable concurrency. Results delivered asynchronously via WebSocket when scoring completes. Exponential backoff for retries.
Leaderboard Materialized View Staleness
Users see outdated rankings
Incremental view updates with staleness threshold (max 30s). Forced refresh on high-priority events (assessment completion). Staleness metrics exposed for monitoring.
State machine transitions, scoring algorithms, leaderboard calculations
Vitest with exhaustive transition matrix testing. Tests cover: all valid state transitions, all invalid transitions (guard rejection), AI provider response parsing, XP calculation edge cases.
API endpoints with real database, change stream pipeline, AI provider mocks
Testcontainers for MongoDB in CI. Tests verify: full assessment lifecycle, change stream event delivery, concurrent exam submission handling, leaderboard update correctness.
Full assessment flow: login → start exam → submit → view results → leaderboard
Playwright tests in UAT environment. Tests cover: timed exam auto-submission, disqualification on rule violation, AI-generated question rendering, real-time score updates via WebSocket.
Boolean status flags allow invalid transitions. A state machine guarantees the assessment lifecycle follows a valid path.
Using two AI providers provides redundancy when one is unavailable and allows cost optimization by routing different workloads.
A materialized leaderboard avoids expensive aggregation queries on every request, keeping the system responsive under load.
Database change streams provide real-time updates without the overhead of polling, critical for timed exams with many concurrent participants.
This is a production system. The details above reflect my actual work. Additional evidence available upon request:
Contact me to arrange verification under NDA.