Data Fetching Strategies

Comprehensive guide to data fetching patterns, caching strategies, and optimizations for modern web applications

Modern applications need efficient, reliable, and fast data fetching. This guide covers patterns used by companies like Facebook, Airbnb, and Netflix.

🎯 The Challenge

Problems to Solve

User opens product page:
├─ Need product data
├─ Need reviews
├─ Need recommendations
├─ Need user session
└─ Need cart info

Naive approach:
→ 5 sequential requests
→ 500ms × 5 = 2500ms ❌
→ Terrible UX

Good approach:
→ Parallel + cached + optimized
→ 100ms total ✅

📊 Fetching Patterns

1. Fetch-on-Render (Simple)

Component fetches data when it renders.

function ProductPage() {
  const [product, setProduct] = useState(null);
  const [loading, setLoading] = useState(true);
  
  useEffect(() => {
    fetch(`/api/products/${id}`)
      .then(res => res.json())
      .then(data => {
        setProduct(data);
        setLoading(false);
      });
  }, [id]);
  
  if (loading) return <Spinner />;
  return <ProductDetail product={product} />;
}

Timeline:

0ms     → Component mounts
100ms   → useEffect runs
150ms   → Fetch starts
650ms   → Data arrives
700ms   → Render complete

Total: 700ms

Pros:

✅ Simple
✅ Easy to understand

Cons:

❌ Waterfall requests
❌ Slow initial load
❌ Flickering UI

2. Fetch-Then-Render (Better)

Fetch data before rendering component.

// Start fetching early
const productPromise = fetch(`/api/products/${id}`);
const reviewsPromise = fetch(`/api/reviews/${id}`);

function ProductPage() {
  const product = use(productPromise); // React 19
  const reviews = use(reviewsPromise);
  
  return (
    <div>
      <ProductDetail product={product} />
      <Reviews reviews={reviews} />
    </div>
  );
}

Timeline:

0ms     → Start fetching (parallel!)
500ms   → Both requests complete
550ms   → Component renders with data

Total: 550ms (150ms faster!)

Pros:

✅ Parallel requests
✅ Faster

Cons:

❌ Must wait for ALL data
❌ Slow if one request is slow

3. Render-as-You-Fetch (Best)

Start fetching immediately and render progressively.

import { Suspense } from 'react';

// Start fetching ASAP
const productResource = fetchProduct(id);
const reviewsResource = fetchReviews(id);

function ProductPage() {
  return (
    <div>
      <Suspense fallback={<ProductSkeleton />}>
        <ProductDetail resource={productResource} />
      </Suspense>
      
      <Suspense fallback={<ReviewsSkeleton />}>
        <Reviews resource={reviewsResource} />
      </Suspense>
    </div>
  );
}

Timeline:

0ms     → Start fetching (parallel)
50ms    → Render skeletons immediately
200ms   → Product data arrives → render product
500ms   → Reviews arrive → render reviews

User sees content progressively!

Pros:

✅ Best UX (progressive)
✅ Parallel fetching
✅ No blocking

🗄️ Caching Strategies

1. Memory Cache (Client-Side)

Store data in memory to avoid refetching.

// Simple cache
const cache = new Map<string, any>();

async function fetchWithCache(url: string) {
  // Check cache first
  if (cache.has(url)) {
    console.log('Cache HIT:', url);
    return cache.get(url);
  }
  
  // Fetch if not cached
  console.log('Cache MISS:', url);
  const response = await fetch(url);
  const data = await response.json();
  
  // Store in cache
  cache.set(url, data);
  return data;
}

With Expiration:

interface CacheEntry {
  data: any;
  timestamp: number;
  ttl: number; // Time to live in ms
}

class Cache {
  private store = new Map<string, CacheEntry>();
  
  set(key: string, data: any, ttl: number = 60000) {
    this.store.set(key, {
      data,
      timestamp: Date.now(),
      ttl
    });
  }
  
  get(key: string): any | null {
    const entry = this.store.get(key);
    
    if (!entry) return null;
    
    // Check if expired
    const age = Date.now() - entry.timestamp;
    if (age > entry.ttl) {
      this.store.delete(key);
      return null;
    }
    
    return entry.data;
  }
}

2. Stale-While-Revalidate (SWR)

Return cached data immediately, then revalidate in background.

function useSWR(url: string) {
  const [data, setData] = useState(cache.get(url));
  const [isValidating, setIsValidating] = useState(false);
  
  useEffect(() => {
    // Return cached data immediately
    const cached = cache.get(url);
    if (cached) {
      setData(cached);
    }
    
    // Fetch fresh data in background
    setIsValidating(true);
    fetch(url)
      .then(res => res.json())
      .then(fresh => {
        cache.set(url, fresh);
        setData(fresh);
        setIsValidating(false);
      });
  }, [url]);
  
  return { data, isValidating };
}

User Experience:

First visit:
0ms     → Show loading
500ms   → Data arrives, show content

Second visit (within cache time):
0ms     → Show cached content immediately ✨
10ms    → Start revalidating in background
510ms   → Fresh data arrives, update silently

3. Normalized Cache (Relay/Apollo Pattern)

Store entities by ID to avoid duplication.

// Without normalization (duplicated data):
const cache = {
  '/products/1': {
    id: 1,
    name: 'iPhone',
    category: { id: 10, name: 'Electronics' }
  },
  '/categories/10': {
    id: 10,
    name: 'Electronics',
    products: [
      { id: 1, name: 'iPhone' }, // Duplicate!
      { id: 2, name: 'iPad' }
    ]
  }
};

// With normalization (single source of truth):
const normalizedCache = {
  products: {
    1: { id: 1, name: 'iPhone', categoryId: 10 },
    2: { id: 2, name: 'iPad', categoryId: 10 }
  },
  categories: {
    10: { id: 10, name: 'Electronics', productIds: [1, 2] }
  }
};

// Access data:
function getProduct(id: number) {
  const product = normalizedCache.products[id];
  const category = normalizedCache.categories[product.categoryId];
  return { ...product, category };
}

Benefits:

✅ No duplication
✅ Single update updates everywhere
✅ Smaller memory footprint
✅ Consistent data

🔄 Request Deduplication

Prevent multiple identical requests.

// Problem: Multiple components fetch same data
function ProductPage() {
  return (
    <div>
      <ProductTitle productId={1} />  {/* Fetches /api/products/1 */}
      <ProductPrice productId={1} />  {/* Fetches /api/products/1 again! */}
      <ProductImage productId={1} />  {/* And again! */}
    </div>
  );
}
// Result: 3 identical requests! ❌

Solution: Request Deduplication

const pendingRequests = new Map<string, Promise<any>>();

async function fetchWithDedup(url: string) {
  // Check if already fetching
  if (pendingRequests.has(url)) {
    console.log('Deduped:', url);
    return pendingRequests.get(url)!;
  }
  
  // Create new request
  const promise = fetch(url)
    .then(res => res.json())
    .finally(() => {
      // Clean up after request completes
      pendingRequests.delete(url);
    });
  
  // Store promise
  pendingRequests.set(url, promise);
  return promise;
}

Result:

Component 1: fetchWithDedup('/api/products/1') → Makes request
Component 2: fetchWithDedup('/api/products/1') → Uses same promise
Component 3: fetchWithDedup('/api/products/1') → Uses same promise

Total requests: 1 ✅

📄 Pagination Patterns

1. Offset-Based Pagination

Simple but has issues.

// Page 1: /api/products?limit=20&offset=0
// Page 2: /api/products?limit=20&offset=20
// Page 3: /api/products?limit=20&offset=40

function useOffsetPagination(limit: number) {
  const [page, setPage] = useState(1);
  const [data, setData] = useState([]);
  
  useEffect(() => {
    const offset = (page - 1) * limit;
    fetch(`/api/products?limit=${limit}&offset=${offset}`)
      .then(res => res.json())
      .then(setData);
  }, [page, limit]);
  
  return { data, page, setPage };
}

Problems:

❌ Inconsistent when data changes
❌ Can skip or duplicate items
❌ Hard to cache efficiently

2. Cursor-Based Pagination (Better)

Use a cursor (usually ID or timestamp) for stable pagination.

// Page 1: /api/products?limit=20
// Response: { items: [...], nextCursor: "abc123" }
// Page 2: /api/products?limit=20&cursor=abc123

interface PaginatedResponse<T> {
  items: T[];
  nextCursor: string | null;
  hasMore: boolean;
}

function useCursorPagination<T>(url: string, limit: number) {
  const [items, setItems] = useState<T[]>([]);
  const [cursor, setCursor] = useState<string | null>(null);
  const [hasMore, setHasMore] = useState(true);
  
  const fetchMore = async () => {
    const queryUrl = cursor 
      ? `${url}?limit=${limit}&cursor=${cursor}`
      : `${url}?limit=${limit}`;
    
    const response = await fetch(queryUrl);
    const data: PaginatedResponse<T> = await response.json();
    
    setItems(prev => [...prev, ...data.items]);
    setCursor(data.nextCursor);
    setHasMore(data.hasMore);
  };
  
  return { items, fetchMore, hasMore };
}

Benefits:

✅ Stable (no skips/duplicates)
✅ Works with infinite scroll
✅ Easy to cache

3. Relay-Style Pagination

Standardized pagination with edges and nodes.

interface Edge<T> {
  node: T;
  cursor: string;
}

interface PageInfo {
  hasNextPage: boolean;
  hasPreviousPage: boolean;
  startCursor: string;
  endCursor: string;
}

interface Connection<T> {
  edges: Edge<T>[];
  pageInfo: PageInfo;
  totalCount: number;
}

// Response structure:
const response: Connection<Product> = {
  edges: [
    { node: { id: 1, name: 'iPhone' }, cursor: 'abc' },
    { node: { id: 2, name: 'iPad' }, cursor: 'def' },
  ],
  pageInfo: {
    hasNextPage: true,
    hasPreviousPage: false,
    startCursor: 'abc',
    endCursor: 'def'
  },
  totalCount: 100
};

⚡ Optimistic Updates

Update UI immediately before server confirms.

function useOptimisticUpdate() {
  const [items, setItems] = useState<Todo[]>([]);
  
  const addTodo = async (text: string) => {
    // 1. Generate optimistic ID
    const optimisticId = `temp-${Date.now()}`;
    const optimisticTodo = {
      id: optimisticId,
      text,
      completed: false,
      _optimistic: true // Mark as temporary
    };
    
    // 2. Update UI immediately
    setItems(prev => [...prev, optimisticTodo]);
    
    try {
      // 3. Send to server
      const response = await fetch('/api/todos', {
        method: 'POST',
        body: JSON.stringify({ text })
      });
      const serverTodo = await response.json();
      
      // 4. Replace optimistic with real data
      setItems(prev => 
        prev.map(item => 
          item.id === optimisticId ? serverTodo : item
        )
      );
    } catch (error) {
      // 5. Rollback on error
      setItems(prev => 
        prev.filter(item => item.id !== optimisticId)
      );
      alert('Failed to add todo');
    }
  };
  
  return { items, addTodo };
}

User Experience:

User clicks "Add Todo"
0ms     → UI updates instantly ✨
500ms   → Server responds with ID
        → UI updates with real ID

If error:
500ms   → Remove optimistic update
        → Show error message

🔄 Prefetching Strategies

1. On Hover Prefetch

Load data when user hovers over link.

function ProductCard({ product }) {
  const prefetchProduct = () => {
    // Start loading product details
    queryClient.prefetchQuery({
      queryKey: ['product', product.id],
      queryFn: () => fetchProduct(product.id)
    });
  };
  
  return (
    <Link 
      to={`/products/${product.id}`}
      onMouseEnter={prefetchProduct}
    >
      {product.name}
    </Link>
  );
}

Result:

User hovers → Start fetching (300ms before click)
User clicks → Data already loaded!
Result: Instant page load ✨

2. Viewport Prefetch

Load data when element enters viewport.

function ProductCard({ product }) {
  const ref = useRef<HTMLDivElement>(null);
  
  useEffect(() => {
    const observer = new IntersectionObserver(
      ([entry]) => {
        if (entry.isIntersecting) {
          // Prefetch when visible
          queryClient.prefetchQuery({
            queryKey: ['product', product.id],
            queryFn: () => fetchProduct(product.id)
          });
        }
      },
      { rootMargin: '100px' } // Start 100px before visible
    );
    
    if (ref.current) {
      observer.observe(ref.current);
    }
    
    return () => observer.disconnect();
  }, [product.id]);
  
  return <div ref={ref}>{product.name}</div>;
}

3. Predictive Prefetch

Load data based on user behavior patterns.

// Track user navigation
const navigationHistory = [];

function trackNavigation(path: string) {
  navigationHistory.push(path);
  
  // Analyze patterns
  if (navigationHistory.length > 3) {
    const pattern = analyzePattern(navigationHistory);
    if (pattern.confidence > 0.7) {
      // Prefetch predicted next page
      prefetch(pattern.nextPage);
    }
  }
}

// Example: User visits /products → /products/1 → /checkout
// Pattern detected: Likely to visit /checkout/payment next
// Prefetch payment page data

🎯 Real-World Libraries

React Query

import { useQuery, useMutation, useQueryClient } from '@tanstack/react-query';

function ProductPage({ id }) {
  const queryClient = useQueryClient();
  
  // Fetch data
  const { data, isLoading, error } = useQuery({
    queryKey: ['product', id],
    queryFn: () => fetchProduct(id),
    staleTime: 5 * 60 * 1000, // 5 minutes
    cacheTime: 10 * 60 * 1000, // 10 minutes
  });
  
  // Mutation with optimistic update
  const mutation = useMutation({
    mutationFn: updateProduct,
    onMutate: async (newData) => {
      // Cancel outgoing queries
      await queryClient.cancelQueries(['product', id]);
      
      // Snapshot current value
      const previous = queryClient.getQueryData(['product', id]);
      
      // Optimistically update
      queryClient.setQueryData(['product', id], newData);
      
      return { previous };
    },
    onError: (err, newData, context) => {
      // Rollback on error
      queryClient.setQueryData(['product', id], context.previous);
    },
    onSettled: () => {
      // Refetch after mutation
      queryClient.invalidateQueries(['product', id]);
    },
  });
  
  if (isLoading) return <Spinner />;
  if (error) return <Error />;
  return <ProductDetail data={data} />;
}

SWR (Vercel)

import useSWR from 'swr';

function ProductPage({ id }) {
  const { data, error, isLoading, mutate } = useSWR(
    `/api/products/${id}`,
    fetcher,
    {
      revalidateOnFocus: true,
      revalidateOnReconnect: true,
      dedupingInterval: 2000,
    }
  );
  
  // Optimistic update
  const updateProduct = async (newData) => {
    // Update UI immediately
    mutate(newData, false);
    
    // Send to server
    await fetch(`/api/products/${id}`, {
      method: 'PUT',
      body: JSON.stringify(newData)
    });
    
    // Revalidate
    mutate();
  };
  
  if (isLoading) return <Spinner />;
  return <ProductDetail data={data} onUpdate={updateProduct} />;
}

📊 Performance Comparison

Request Strategies

Pattern	First Load	Cached Load	Complexity
Fetch-on-Render	700ms	700ms	Low
Fetch-Then-Render	550ms	550ms	Medium
Render-as-You-Fetch	200-500ms	50ms	High
With Prefetch	50ms	50ms	High

Caching Impact

No cache:
Request → 500ms → Render
Every time: 500ms

With cache (SWR):
First: 500ms
Subsequent: 0ms (instant!)
Background revalidate: Silent

Savings: 100% faster for cached data

🎯 Decision Framework

Choose Your Strategy:

Simple App (Blog, Marketing)

✅ Use: Fetch-on-render + SWR
Why: Simple, good enough
Example: fetch() + cache

Medium App (Dashboard, SaaS)

✅ Use: React Query or SWR
Why: Handles complexity for you
Features: Auto-refetch, cache, dedup

Complex App (Facebook, Twitter)

✅ Use: Relay or Apollo GraphQL
Why: Normalized cache, optimistic updates
Features: Fragment colocation, pagination

Real-Time App (Chat, Collab)

✅ Use: WebSockets + Optimistic updates
Why: Need instant feedback
Features: Live data, sync

🏢 Real-World Examples

Facebook (Relay)

- Normalized cache
- Fragment-based queries
- Optimistic updates everywhere
- Preloading on hover
- Result: Instant navigation

Netflix

- Aggressive prefetching
- Multiple CDN layers
- Smart cache invalidation
- Predictive loading
- Result: Instant playback

Airbnb

- Route-based data loading
- Optimistic booking
- Stale-while-revalidate
- Progressive images
- Result: Fast browsing

📚 Key Takeaways

Fetch early and in parallel - Don't wait for renders
Cache aggressively - Avoid redundant requests
Deduplicate requests - One request for multiple consumers
Update optimistically - Don't wait for server
Prefetch intelligently - Load before user needs it
Use proven libraries - React Query, SWR, Relay
Normalize when needed - For complex data relationships
Monitor and optimize - Measure what matters

The best data fetching strategy depends on your app's needs. Start simple, measure, and optimize where it matters most.

Data Fetching Strategies

On this page