Scaling a High-Traffic Job Portal with MERN & Kafka

Jobscater.com is a hiring platform where employers post roles, candidates apply at volume, and recruiters need to find the right people quickly. Traffic is not a smooth line. It spikes around new job drops, campus seasons, and bulk outreach. As a Senior MERN Stack Developer in Nepal, I owned the architecture that had to stay reliable when those spikes hit — not only when the demo looked fine on a quiet afternoon.

The product already had the shape of a job portal: posting, applications, profiles, and a recruiter workspace. What it did not yet have was a clear split between work that must complete in the request path and work that can finish a few seconds later. Search quality, notification delivery, and screening were competing for the same runtime as create-job and apply. That is how hiring products get slow: every extra feature is bolted onto the same synchronous request.

The challenge

Recruiters judge a portal on three feelings: posting a job should feel instant, finding candidates should feel relevant, and the first screening pass should not eat a whole afternoon. The platform was being asked to do all three at once, on the same backend, while application volume grew.

Notifications were the first obvious bottleneck. A new job or application status change needed email and in-app alerts. Doing that work inside the HTTP handler meant recruiters waited on third-party mail latency and retry logic. Search was the second. MongoDB text search can get a prototype moving, but recruiter queries are messy: job titles, skills, locations, recency, and “close enough” matches. Ranking that well needs an inverted index and a dedicated query model. Screening was the third. Human review does not scale linearly with applications. The team wanted Generative AI to draft a first-pass evaluation without ever letting a model silently reject a person.

• Peak posting and apply traffic colliding with email and fan-out work
• Candidate discovery that needed relevance ranking, not just keyword filters
• Screening volume that required assistance without removing recruiter control
• A frontend that had to serve two very different users: job seekers and hiring teams

Approach

I treated the portal as a set of domains rather than a single CRUD app. Job operations, candidate profiles, search indexing, notifications, and AI evaluation each got a clear owner. Node.js and Express stayed the runtime for APIs. MongoDB remained the system of record. React stayed the client. The change was not “rewrite everything.” It was drawing boundaries so the Node.js backend could scale the expensive parts independently.

Kafka became the contract between “this event happened” and “now tell people / index this / score this.” Redis sat in front of hot recruiter reads: recent jobs, session-ish data, and repeated dashboard queries. Elasticsearch became the search system of record for discovery, fed by an indexing pipeline rather than ad-hoc queries against operational collections. That split matters. Operational data wants consistency. Search data wants denormalized documents tuned for ranking.

On the AI side, the rule was explicit: the model proposes, the recruiter disposes. Screening output is a structured assist — strengths, gaps, and questions to ask — attached to an application. It never auto-closes a candidate. That keeps the product useful and defensible when hiring teams ask how decisions were made.

Architecture

Job posting and applications stay on a request-response API. After a successful write, the service emits events: job published, application submitted, status changed. Kafka consumers handle notification fan-out, search document upserts, and enqueueing of AI evaluation jobs. If a consumer lags, posting still succeeds. That is the whole point of the boundary.

Elasticsearch documents are built for recruiter queries: title, skills, location, experience, recency, and a small set of boosts. We did not dump entire MongoDB documents into the index. Search fields were chosen for how recruiters actually filter and rank. MongoDB kept canonical profiles and application state. Redis cached the queries recruiters repeat during a hiring sprint so the dashboard did not stampede Elasticsearch or the primary database on every tab switch.

The React frontend is two products sharing design language: a candidate flow that needs to be simple and fast, and a recruiter flow that needs density — pipelines, filters, and screening notes. Shared components helped, but the information architecture stayed separate. A hiring desk is not a job board with extra buttons.

What we shipped

The delivery sequence was deliberately boring. First, make posting and applying trustworthy under load. Second, move notifications off the request path. Third, stand up search as its own product surface. Fourth, add AI screening as an overlay on applications that already had a stable lifecycle. Each slice was usable on its own. That is how you avoid a six-month rewrite that ships nothing.

• Microservices around job operations, profiles, search, and notifications
• Kafka topics for domain events instead of nested service calls in the request
• Elasticsearch relevance ranking tuned for recruiter discovery
• Redis for hot dashboard and session-adjacent reads
• Generative AI screening notes with mandatory human review

This is the kind of work I take on as a full stack developer: not only wiring Mongo, Express, React, and Node together, but deciding which parts of a hiring workflow belong in the hot path and which belong on a queue. The matching project page covers the portfolio snapshot; this write-up is the engineering story.

Results and what held up

Notification latency stopped tracking mail-provider luck. Recruiters could publish and move on. Search started returning candidates that matched how hiring managers talk about roles, not only exact strings in a bio. AI screening cut the time spent on the first pass without pretending the model was the hiring manager.

The quieter win was operational. When search or AI needed a change, we did not freeze job posting to ship it. That is the real scale test for a portal: can you evolve discovery and screening while the core marketplace stays up. If you are building a similar hiring or marketplace product, contact me and we can map the same kind of boundaries onto your stack.