Skip to content

Instantly share code, notes, and snippets.

@andylibrian
Last active June 16, 2024 00:06
Show Gist options
  • Select an option

  • Save andylibrian/9f25aceb10437e7c448ad3f3950e04a3 to your computer and use it in GitHub Desktop.

Select an option

Save andylibrian/9f25aceb10437e7c448ad3f3950e04a3 to your computer and use it in GitHub Desktop.
Upgrading Loki 2.16 to 6

Steps to Upgrade Loki Deployment from Helm Chart Version 2.16.0 to Version 6

Step 1: Compare Default Values

Key Differences Between loki-2.16.0-default-values.yaml and loki-6-default-values.yaml:

  1. Image Configuration:

    • Version 2.16.0:
      image:
        repository: grafana/loki
        tag: 2.6.1
        pullPolicy: IfNotPresent
    • Version 6:
      global:
        image:
          registry: null
      loki:
        image:
          registry: docker.io
          repository: grafana/loki
          tag: null
          pullPolicy: IfNotPresent
  2. Deployment Mode:

    • Version 2.16.0: No specific deployment mode configuration.
    • Version 6:
      deploymentMode: SimpleScalable
  3. Config File Structure:

    • Version 2.16.0: Uses config for all Loki configurations.
    • Version 6: Separates configurations under various sections like loki, schemaConfig, storage, compactor, etc.
  4. New Sections in Version 6:

    • global: Contains global configurations for the Helm chart.
    • enterprise: Configuration specific to Grafana Enterprise Logs.
    • migrate: Options for migrating from other Helm charts.
    • singleBinary: Configuration for single binary deployments.

Step 2: Review override-values.yaml and Determine Necessary Changes

override-values.yaml:

config:
  query_range:
    parallelise_shardable_queries: false
  query_scheduler:
    max_outstanding_requests_per_tenant: 2048
  auth_enabled: false
  ingester:
    chunk_idle_period: 3m
    chunk_block_size: 262144
    chunk_retain_period: 1m
    max_transfer_retries: 0
    lifecycler:
      ring:
        kvstore:
          store: inmemory
        replication_factor: 1
  limits_config:
    enforce_metric_name: false
    reject_old_samples: true
    reject_old_samples_max_age: 168h
  schema_config:
    configs:
    - from: "2021-10-01"
      store: boltdb-shipper
      object_store: s3
      schema: v11
      index:
        prefix: index_
        period: 24h
  server:
    http_listen_port: 3100
  storage_config:
    boltdb_shipper:
      active_index_directory: /data/loki/boltdb-shipper-active
      cache_location: /data/loki/boltdb-shipper-cache
      cache_ttl: 168h
      shared_store: s3
    aws:
      s3: https://@ap-south-1.linodeobjects.com/staging-1-loki
      s3forcepathstyle: true
  compactor:
    working_directory: /data/loki/boltdb-shipper-compactor
    shared_store: s3
    compaction_interval: 5m

Changes Required to Make override-values.yaml Compatible with Version 6:

  1. Update structure to align with the new sections in Version 6.
  2. Ensure configurations under loki, storage_config, and other specific sections.
  3. Remove configurations that are now part of global.

Step 3: Update override-values.yaml for Compatibility with Version 6

Updated override-values.yaml:

loki:
  query_range:
    parallelise_shardable_queries: false
  query_scheduler:
    max_outstanding_requests_per_tenant: 2048
  auth_enabled: false
  ingester:
    chunk_idle_period: 3m
    chunk_block_size: 262144
    chunk_retain_period: 1m
    max_transfer_retries: 0
    lifecycler:
      ring:
        kvstore:
          store: inmemory
        replication_factor: 1
  limits_config:
    enforce_metric_name: false
    reject_old_samples: true
    reject_old_samples_max_age: 168h
  schema_config:
    configs:
      - from: "2021-10-01"
        store: boltdb-shipper
        object_store: s3
        schema: v11
        index:
          prefix: index_
          period: 24h
  server:
    http_listen_port: 3100
  storage_config:
    boltdb_shipper:
      active_index_directory: /data/loki/boltdb-shipper-active
      cache_location: /data/loki/boltdb-shipper-cache
      cache_ttl: 168h
      shared_store: s3
    aws:
      s3: https://@ap-south-1.linodeobjects.com/redacted
      s3forcepathstyle: true
  compactor:
    working_directory: /data/loki/boltdb-shipper-compactor
    shared_store: s3
    compaction_interval: 5m

Step 4: Additional Insights and Best Practices

  1. Test in Staging: Always test the upgraded Helm chart in a staging environment before deploying to production.
  2. Backup Configuration: Backup your current configuration and data before performing the upgrade.
  3. Review Release Notes: Check the release notes of the Helm chart versions for any breaking changes or new features.
  4. Monitor Logs: After upgrading, monitor the logs and metrics closely to ensure the system is functioning correctly.

Summary

The main changes involve updating the structure of the override values to match the new sections and configurations in Loki Helm chart version 6. The provided updated override-values.yaml should now be compatible with the new chart version.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment