Best practices

Best Practices

This guide provides recommended patterns and practices for integrating with the G17Eco Public API to ensure optimal performance, reliability, and maintainability.

Data Extraction Strategy

Initial Full Sync

For first-time data extraction or complete refresh:

// 1. Get all initiatives (flat list)
const initiatives = await api.get('/initiatives');
 
// 2. For each initiative, get their surveys
const allSurveys = [];
for (const initiative of initiatives) {
  const surveys = await api.get(`/initiatives/${initiative._id}/surveys`);
  allSurveys.push(...surveys);
}
 
// 3. Get detailed survey data with UTRVs
for (const survey of allSurveys) {
  const detailed = await api.get(`/surveys/${survey._id}`);
  await processAndStore(detailed);
}
 
// 4. Cache reference data
const valueListIds = extractUniqueValueLists(allSurveys);
const valueLists = await batchFetchValueLists(valueListIds);

Incremental Data Sync

Use the changedSince parameter for efficient data synchronization:

// Track changes since last sync
const lastSync = await getLastSyncTimestamp();
 
// Get all initiatives
const initiatives = await api.get('/initiatives');
 
// For each initiative, get only surveys that have changed
const changedSurveys = [];
for (const initiative of initiatives) {
  const surveys = await api.get(`/initiatives/${initiative._id}/surveys`, {
    params: { changedSince: lastSync }
  });
  changedSurveys.push(...surveys);
}
 
// Fetch detailed data for changed surveys
for (const survey of changedSurveys) {
  const detailed = await api.get(`/surveys/${survey._id}`);
  await updateStoredData(detailed);
}
 
// Update sync timestamp
await updateLastSyncTimestamp(new Date());

Change Tracking: The changedSince parameter efficiently identifies modified data by checking the aggregated lastUpdated timestamps of all UTRVs within surveys.

Performance Optimization

Request Optimization

  1. Batch Operations: Use bulk endpoints like /value-list/lookup for up to 100 IDs per request
  2. Parallel Requests: Process independent requests concurrently
  3. Connection Pooling: Reuse HTTP connections for better performance
  4. Request Deduplication: Avoid fetching the same data multiple times

Rate Limit Management

class RateLimitedClient {
  constructor(baseURL, maxRetries = 3) {
    this.baseURL = baseURL;
    this.maxRetries = maxRetries;
  }
 
  async request(url, options = {}, retryCount = 0) {
    try {
      const response = await fetch(`${this.baseURL}${url}`, options);
      
      if (response.status === 429) {
        if (retryCount >= this.maxRetries) {
          throw new Error('Max retries exceeded');
        }
        
        // Use RateLimit-Reset header or exponential backoff
        const rateLimitReset = response.headers.get('RateLimit-Reset');
        const waitTime = rateLimitReset 
          ? parseInt(rateLimitReset) * 1000  // Convert seconds to milliseconds
          : Math.pow(2, retryCount) * 1000;   // Exponential backoff fallback
        
        await this.sleep(waitTime);
        return this.request(url, options, retryCount + 1);
      }
      
      return response;
    } catch (error) {
      if (retryCount < this.maxRetries) {
        await this.sleep(Math.pow(2, retryCount) * 1000);
        return this.request(url, options, retryCount + 1);
      }
      throw error;
    }
  }
  
  sleep(ms) {
    return new Promise(resolve => setTimeout(resolve, ms));
  }
}

Data Validation

When working with API responses, ensure you validate the data:

  • Validate all required fields are present
  • Check data types match expected formats
  • Verify ObjectId formats (24 character hex string)
  • Validate date formats (ISO 8601)
  • Check numeric values are within expected ranges
  • Handle null/undefined values gracefully
  • Validate array lengths don't exceed limits

Follow Redirects

G17Eco is explicit in telling you when a resource has moved by providing a redirect status code. You should follow these redirections. Every redirect response sets the Location header with the new URI to go to.

If you receive a redirect, it's best to update your code to follow the new URI, in case you're requesting a deprecated path that we might remove.

Value ListAbout Us