Fix Inventory Worker Configuration

Configuration Files

You may need make your AWS credentials file, Google Cloud service account JSON files, Kubernetes kubeconfig files, etc. available to Fix Inventory Worker so that it can collect your resources.

Mounting Configuration Files

You can mount configuration files within the fixworker container instead of defining them in the configuration.

1. Add the desired volume definition to the fixworker service in docker-compose.yaml:

   docker-compose.yaml

   services:
     ...
     fixworker:
       image: somecr.io/someengineering/fixinventoryworker:4.2.0
       container_name: fixworker
       ...
       volumes:
         - $HOME/.aws:/home/fixinventory/.aws
     ...
   ...

2. Recreate the fixworker container with the updated service definition:

   $ docker-compose up -d
   note

   Docker Compose V2 integrated compose functions in to the Docker platform.

   In Docker Compose V2, the command is docker compose (no hyphen) instead of docker-compose.

Multi-Core Machines

Fix Inventory resource collection speed depends heavily on the number of CPU cores available to the worker. When collecting hundreds of accounts, Fix Inventory Worker can easily saturate 64 cores or more.

The amount of RAM required depends on the number of resources in each account. As a rule of thumb, estimate 512 MB of RAM and 0.5 CPU cores per account concurrently collected, with a minimum of 4 cores and 16 GB for a production setup.

The following settings specify how many Worker threads Fix Inventory starts:

fixworker:
  ...
  # How many cleanup threads to run in parallel
  cleanup_pool_size: 16
  # Collector thread/process pool size
  pool_size: 5
aws:
  ...
  # Account thread/process pool size
  account_pool_size: 32
  # Region thread pool size
  region_pool_size: 20
gcp:
  ...
  # GCP project thread/process pool size
  project_pool_size: 32
...

• The fixworker.pool_size setting determines how many collectors (AWS, Google Cloud, DigitalOcean, Kubernetes, etc.) are run concurrently.

• aws.account_pool_size and gcp.project_pool_size are used to determine how many accounts or projects respectively are collected concurrently.

• Within AWS, aws.region_pool_size is used to determine how many regions per account are collected concurrently.

warning

At peak, Fix Inventory creates concurrent network connections for each region in every account. With a single cloud with 32 accounts and 20 regions per account, for example, there will be a maximum of 32 × 20 = 640 connections.

This is not a problem in a data center or with a SOHO router, where hundreds of thousands (or even millions) of new connections per second are supported. However, if you are testing Fix Inventory at home using a consumer-grade router, you should be conservative when configuring thread pool sizes.