Building a Certificate Generation Pipeline with Puppeteer, a Custom File Service, and Kafka

Generating a certificate sounds like a simple operation:

That approach works for a small application. It becomes problematic when hundreds or thousands of certificates need to be generated at the same time.

PDF generation is not free. Rendering HTML requires CPU and memory, browser processes can fail, and storing generated files introduces another infrastructure concern.

For an EdTech platform, I therefore treated certificate generation as a background processing pipeline rather than a synchronous API operation.

Technology Stack

1. The Problem

Imagine a student completes a course. The platform needs to:

• Verify course completion
• Create certificate data
• Generate a PDF
• Store the PDF
• Associate the file with the certificate
• Make the certificate available to the student

A naive implementation might put everything inside one HTTP request:

The HTTP request now depends on several expensive operations. If PDF generation takes several seconds, the user waits. If the browser crashes, the API request fails. The API becomes responsible for work that doesn’t need to happen synchronously.

2. The Better Architecture

I separated certificate creation from certificate processing. The architecture became:

The API no longer needs to wait for PDF generation. Instead, the system works asynchronously.

3. The Event-Driven Flow

When a student completes the required learning criteria, the application publishes an event:

{
  "event": "COURSE_COMPLETED",
  "studentId": "123",
  "courseId": "456"
}

A certificate consumer listens for the event. This creates a clean separation between course completion and certificate generation. The completion service doesn’t need to know how PDFs are generated.

4. Why Kafka?

Kafka provides an event-driven communication layer between services, reducing direct coupling:

Other consumers can also react to the same event (Analytics, Notifications, etc.) without the completion service needing to know about every downstream operation.

5. Why Not Generate Immediately?

Certificate generation is a different type of workload from transactional completion checks.

Combining them increases the blast radius of failures.

6. Puppeteer as the Rendering Engine

I used Puppeteer to render an HTML certificate and convert it to PDF:

The certificate design can be expressed using familiar web technologies (HTML, CSS, fonts, images) instead of manually drawing every element with a PDF-specific graphics API.

7. The Certificate Template

Conceptually, the template looks like:

<div class="certificate">
  <div class="header">
    <img src="logo.png" />
  </div>
  <h1>Certificate of Completion</h1>
  <p>This certificate is awarded to</p>
  <h2>{{ studentName }}</h2>
  <p>for successfully completing</p>
  <h3>{{ courseName }}</h3>
  <div class="metadata">
    <span>Certificate ID: {{ certificateId }}</span>
    <span>Date: {{ completionDate }}</span>
  </div>
</div>

The certificate template is treated as a presentation layer. The backend provides the verified data.

8. Never Trust Certificate Data From the Client

A certificate is an official representation of achievement. Data such as student name, course name, completion date, and certificate ID must come from trusted backend state.

The client should never be able to simply declare completion and request a certificate. The backend must establish eligibility through authenticated verification.

9. Certificate IDs

Each certificate should have a unique identifier (e.g. CERT-2026-000184). This enables future verification endpoints and treats the database record as the authoritative representation — the PDF is merely the generated artifact.

10. Why a Custom File Service?

I did not want certificate generation to know the details of the underlying object-storage implementation.

This abstraction makes the application easier to evolve without changing the certificate generation logic.

11. Metadata vs Binary Storage

12. The Complete Pipeline

13. Certificate States

The database should explicitly represent processing state for visibility:

14. Idempotency

Kafka consumers can receive duplicate messages. Without an idempotency strategy (e.g. a unique constraint on student_id + course_id), duplicate events could produce multiple certificates. Duplicate events must not produce duplicate business outcomes.

15. Handling PDF Generation Failures

Puppeteer can fail due to browser startup issues, missing fonts, memory pressure, timeouts, or resource loading failures. The consumer must not simply lose the job.

16. Keep Database Transactions Short

Never hold a database transaction open while generating PDFs. Create the certificate record, commit, then perform the expensive work, then update the status. Long-lived transactions increase lock contention and risk.

17. Independent Worker Scaling

The certificate worker can scale independently of the API:

18. Observability

Asynchronous systems require rich metadata: certificate_id, job_id, student_id, processing timestamps, retry_count, failure_reason, and file_key. Correlation IDs should connect the entire path from API request through Kafka to the final file write.

19. Security Considerations

Certificates contain personal information. Access should be controlled through an authenticated API that verifies ownership before generating authorized downloads — never expose publicly guessable URLs.

20. Why Not Store PDFs in PostgreSQL?

PostgreSQL handles metadata (who, what, when, status, identifier, file reference). Object storage handles the binary PDF bytes. Each system does the job it is designed for.

21. Architecture at a Glance

┌──────────────┐
│   Student    │
└──────┬───────┘
       │
       ▼
┌─────────────────┐
│ Completion Svc  │
└────────┬────────┘
         │  COURSE_COMPLETED
         ▼
┌─────────────────┐
│     Kafka       │
└────────┬────────┘
         │
         ▼
┌─────────────────┐
│ Certificate     │
│ Worker          │
└───────┬─────────┘
        │
   ┌────┼────┐
   ▼    ▼    ▼
 Postgres  Puppeteer  File Service
   │         │            │
   │         ▼            ▼
   │        PDF     Object Storage
   └────────┬─────────────┘
            ▼
     Certificate COMPLETED

22. The Deeper Engineering Lesson

The interesting part of this project wasn’t generating a PDF. Puppeteer can generate a PDF. The engineering challenge was designing a reliable system around the PDF generator.

That is why I think of certificate generation as a distributed processing pipeline, not a PDF feature.

23. What I Would Improve at Larger Scale

• 1
  Browser pooling — maintain controlled Chromium instances instead of starting a new process for every certificate.
• 2
  Queue backpressure — control concurrency so that more jobs don’t automatically mean better throughput.
• 3
  Intentional retry policies — different failures (network timeout vs invalid data) deserve different retry behavior.

24. Final Takeaway