Caching System - BigPipe

BigPipe is a Drupal core module that improves perceived performance by sending the cacheable parts of a page immediately and streaming slower or personalized content afterward. It works with placeholders and lazy builders, allowing Drupal to keep pages highly cacheable while still delivering dynamic content efficiently.

BigPipe is one of Drupal core's most powerful performance features.

It allows Drupal to send the page to the browser immediately, while slower parts of the page load a few moments later.

Think of it like this:

Instead of waiting for the entire meal to be ready, the restaurant serves the appetizers first and brings the main course as soon as it is done.

Users see content faster, even if some personalized components take longer to render.

BigPipe is especially useful for:

  • authenticated users
  • dashboards
  • pages with personalized blocks
  • pages with slow components

Core Concept

Without BigPipe:

Build entire page
   ↓
Wait until everything is ready
   ↓
Send response

With BigPipe:

Build cacheable parts
   ↓
Send page immediately
   ↓
Replace placeholders as content becomes ready

This improves perceived performance significantly.


BigPipe Architecture

Request
   ↓
Render Page Skeleton
   ↓
Insert Placeholders
   ↓
Send Initial HTML
   ↓
Stream Replacement Content
   ↓
Browser Updates Page

A government dashboard included:

  • main content
  • notifications block
  • personalized greeting
  • recent submissions

Without BigPipe:

  • user waited 3–4 seconds for the full page

With BigPipe:

  • page shell appeared in under 1 second
  • personalized widgets loaded progressively

Result:

  • better user experience
  • faster perceived performance

How BigPipe Works with Placeholders

Drupal replaces dynamic sections with placeholders.

Example:

<div data-big-pipe-placeholder-id="123"></div>

Later, Drupal streams the real HTML and JavaScript replaces the placeholder.


BigPipe and Lazy Builders

BigPipe works especially well with #lazy_builder.

$build['welcome'] = [
  '#lazy_builder' => ['my_module.service:getGreeting', [$uid]],
  '#create_placeholder' => TRUE,
];

This lets Drupal:

  • cache the page shell
  • defer personalized content
  • stream content asynchronously

Relationship to Other Cache Layers

FeaturePurpose
Render CacheCache component HTML
Dynamic Page CacheCache full pages for authenticated users
BigPipeStream placeholders progressively
Page CacheCache full pages for anonymous users

BigPipe complements caching; it does not replace it.


Decision Framework

Use BigPipe when:

  • pages contain personalized blocks
  • some components are slow to render
  • authenticated user experience matters

Avoid when:

  • the page is fully static
  • placeholders provide no real benefit

Drupal Core Configuration

BigPipe is included in Drupal core.

Enable module:

drush en big_pipe -y

No contrib module is required.


Response Flow Example

Initial HTML Sent
   ↓
Header and main content visible
   ↓
Placeholder for user greeting
   ↓
BigPipe stream sends greeting HTML
   ↓
Browser replaces placeholder

Frontend / React Perspective

BigPipe is conceptually similar to:

  • React Suspense
  • streaming server rendering
  • incremental hydration

Architecture:

Drupal BigPipe
   ↓
HTML shell first
   ↓
Dynamic fragments later

This is a powerful bridge between Drupal and modern frontend patterns.


Platform / DevOps Layer

BigPipe works best with:

  • Dynamic Page Cache
  • Render Cache
  • Redis
  • CDN

Typical stack:

Redis → Render Cache
Dynamic Page Cache → Full page
BigPipe → Progressive streaming
CDN → Global delivery

Debugging BigPipe

View page source and look for:

  • data-big-pipe-placeholder-id
  • streamed <script> tags

Useful tools:

  • browser Network tab
  • Webprofiler
  • curl -N

Performance Considerations

Benefits:

  • faster first contentful paint
  • improved perceived performance
  • better editor experience

Best practices:

  • use placeholders only for truly dynamic content
  • combine with lazy builders
  • keep slow logic isolated

Common Production Issues

  • forgetting #create_placeholder = TRUE
  • personalized blocks disabling cache
  • JavaScript errors preventing replacement
  • too many placeholders reducing benefit

SEO & Accessibility Considerations

Search engines still receive complete HTML.

Benefits:

  • fast visible content
  • improved Core Web Vitals
  • better screen reader experience because content starts rendering sooner

AI & Future Integration

BigPipe aligns with modern AI-assisted experiences:

  • stream initial content first
  • load AI recommendations later
  • progressively enhance dashboards

  1. What problem does BigPipe solve?
  2. How does BigPipe differ from Dynamic Page Cache?
  3. What is the role of placeholders?
  4. How does #lazy_builder work with BigPipe?
  5. When should you use BigPipe?

Memory Trick

BigPipe = Stream Page in Pieces
Placeholder = Temporary Marker
Lazy Builder = Deferred Content
BigPipe = Better Perceived Speed