Institutional Sync

The system synchronizes institutional data (categories, courses, enrollments) from Moodle LMS via a unified BullMQ-based pipeline.

The scheduler uses dynamic cron via SchedulerRegistry (no static @Cron() decorator). The interval is resolved at startup in priority order:

1. Database — SystemConfig key MOODLE_SYNC_INTERVAL_MINUTES (admin override via PUT /moodle/sync/schedule)
2. Environment variable — MOODLE_SYNC_INTERVAL_MINUTES
3. Per-environment default — dev/test: 60 min, staging: 360 min, production: 180 min

Minimum interval is 30 minutes (enforced at all three layers). The interval is converted to a cron expression internally via minutesToCron().

Fetches all Moodle categories and rebuilds the normalized hierarchy (Campus → Semester → Department → Program). Parent entities are cached in in-memory Maps to eliminate N+1 findOneOrFail queries.

Can be skipped at startup via DISABLE_SYNC_CATEGORY_ON_STARTUP=true for faster dev restarts.

Syncs courses for all programs concurrently using pLimit(MOODLE_SYNC_CONCURRENCY). Each program's courses are synced in an independent transaction. Failed programs don't abort others.

Uses a 3-phase architecture to avoid deadlocks from overlapping user rows:

1. Concurrent HTTP fetch — pLimit-gated parallel calls to Moodle per course (the core_enrol_get_enrolled_users response includes a groups array per user)
2. Batch user upsert — Deduplicated upsertMany in a single operation (with individual fallback)
3. Sequential per-course enrollment upsert — For each course:
   • Extracts unique groups from the enrolled users' groups data and upserts Section entities
   • Upserts enrollments with the resolved section FK (first group the student belongs to)
   • Soft-deactivates missing enrollments

No additional Moodle API calls are needed for sections — group data is already returned by the enrolled users endpoint.

backfillUserScopes() derives each synced user's user.program and user.department from enrollment majority (most enrollments wins; ties broken by alphabetical moodleCategoryId so the result is identical across dev/staging/prod for the same Moodle state). It calls the shared pure helper deriveUserScopes() from scope-derivation.helper.ts so the cron path and the login path (MoodleUserHydrationService) cannot diverge.

Atomic source rule: If EITHER user.departmentSource = 'manual' OR user.programSource = 'manual', the user is skipped entirely. The two fields update together or not at all — preventing a user.department ≠ user.program.department inconsistency. An equality guard skips no-op writes so updatedAt is not bumped on idempotent runs.

Campus backfill (fill-if-null only): for users with user.campus IS NULL, the phase parses the username prefix (<campus_code>-<id>, e.g. ucmn-262141935), looks up Campus.code = 'UCMN', and assigns it. This mirrors UserRepository.UpsertFromMoodle's login-time behavior so cron-discovered users get a campus before they ever log in. Existing campus values are never overwritten — there is no campus_source column, so the only safe rule is "only fill if empty." Manual reassignments are preserved; an admin who clears a campus would see it re-derived on the next sync.

The phase logs an aggregate line: Scope backfill: X derived, Y manual skipped, Z no enrollments, W campus assigned.

After enrollments land, deriveUserRoles() batch-loads each synced user's active Enrollment rows and UserInstitutionalRole rows in parallel, groups them per user, and calls User.updateRolesFromEnrollments() to recompute the roles array.

updateRolesFromEnrollments() snapshots existing SUPER_ADMIN and ADMIN roles and merges them back after recomputing. Those two roles are manually granted outside Moodle — the snapshot prevents a sync from ever revoking a role it never granted. The phase is non-fatal (try/catch); role drift is preferred over a broken sync run.

user.departmentSource and user.programSource ('auto' | 'manual') follow the same pattern as UserInstitutionalRole.source. New users default to 'auto' and are eligible for sync-driven derivation. Manual assignments (admin UI, FAC-127) flip the source to 'manual', after which the bulk sync and the login-path hydration both skip the user atomically. Reverting to 'auto' re-enables derivation on the next sync.

Every sync execution (scheduled, manual, startup) creates a SyncLog record in the database. The processor creates it at job start (status: running) and updates it after each phase and on completion.

Each phase stores a SyncPhaseResult (JSONB) with:

Overall sync status: completed (all phases succeeded), partial (some failed/skipped), or failed (all failed).

Manual syncs record triggeredBy (FK to User) via CLS (CurrentUserService).

The SyncLog entity does not extend CustomBaseEntity — audit records are never soft-deleted. Queries must use filters: { softDelete: false } to bypass the global MikroORM filter.

• Cron: Fixed jobId (moodle-sync-scheduled) — BullMQ silently ignores duplicate waiting jobs
• Manual: 409 guard checks activeCount + waitingCount > 0 before enqueuing
• Processor: concurrency: 1 ensures at most one sync runs at a time