Environment Variables Reference

Environment Variables Reference¶

Back to Home

This document provides a comprehensive reference for all environment variables used in the ByteBiota digital evolution system.

Overview {#overview}¶

ByteBiota uses environment variables for configuration management, allowing flexible deployment across different environments without code changes. Variables can be set via:

• System environment variables
• .env files (automatically loaded from project root or parent directories)
• Systemd/supervisor environment settings
• Command-line overrides

Configuration Levels {#configuration-levels}¶

ByteBiota environment variables are organized into three levels based on how frequently they need to be changed:

Core Variables {#core-variables}¶

Essential variables that users commonly adjust for different deployment scenarios. These control basic functionality and resource allocation.

Standard Variables {#standard-variables}¶

Important variables for fine-tuning system behavior. These are typically set once per deployment environment.

Advanced Variables {#advanced-variables}¶

Low-level variables that rarely need to be changed. These control internal system behavior and are usually left at default values.

Auto-Detection {#auto-detection}¶

ByteBiota now includes automatic resource detection for optimal performance:

• Resource Detection: Automatically detects CPU cores, available memory, and disk space
• Optimal Settings: Calculates appropriate resource limits, batch sizes, and network parameters
• CLI Override: All auto-detected values can be overridden via command-line arguments
• Preset Support: Use --preset auto to enable automatic resource detection

Platform Compatibility {#platform-compatibility}¶

ByteBiota workers are designed to run cross-platform on Windows, macOS, and Linux:

Supported Platforms¶

• Windows 10/11: Full support with Windows-specific process priority classes
• macOS 12+: Full support with Unix nice levels (may require elevated permissions for negative values)
• Linux (Ubuntu 20.04+): Full support with standard Unix nice levels

Platform-Specific Behavior¶

• Process Priority: Windows uses priority classes (IDLE, BELOW_NORMAL, NORMAL, ABOVE_NORMAL, HIGH), while Unix systems use nice levels (-20 to 19)
• CPU Detection: Cross-platform detection using os.cpu_count(), multiprocessing.cpu_count(), and platform-specific fallbacks
• Event Loops: Windows uses ProactorEventLoop by default (Python 3.8+), Unix systems use SelectorEventLoop
• File Paths: All paths use pathlib.Path for cross-platform compatibility

Environment Variables¶

All environment variables work consistently across platforms. No platform-specific variable names are used.

Adaptive Network Configuration {#adaptive-network-configuration}¶

The system now includes adaptive network configuration that improves over time:

• Connection Quality Tracking: Monitors response times, error rates, and success rates
• Automatic Adjustment: Adjusts polling intervals and heartbeat frequencies based on network conditions
• Conservative Start: Begins with conservative defaults and optimizes based on actual performance
• Transparent Logging: All network parameter adjustments are logged for visibility

Evolution Configuration Guide {#evolution-configuration}¶

Energy Economy Principles {#energy-economy}¶

The energy economy is critical for sustainable digital evolution. Organisms must be able to harvest more energy than they consume to survive and reproduce.

Key Parameters:
- MEMORY_RENT_PER_BYTE: Cost per byte of memory per cycle (default: 0.0001)
- ENV_HARVEST_YIELD: Energy gained from harvest operations (default: 100)
- CPU_COST_PER_INSTRUCTION: Energy cost per CPU instruction (default: 0.005)
- ENERGY_INITIAL: Starting energy for new organisms (default: 500)

Balancing Guidelines:
- Memory rent should be low enough that organisms can harvest more than they pay
- Harvest yield should provide sufficient energy for reproduction
- Initial energy should give organisms time to execute their first harvest

Reaper Configuration {#reaper-configuration}¶

Natural selection requires the reaper to remove organisms, creating evolutionary pressure.

Essential Settings:
- MIN_POPULATION_FOR_REAP: Minimum population before reaping starts (default: 50)
- REAP_CHANCE: Probability of random death per cycle (default: 0.01)
- AGE_THRESHOLD_MULTIPLIER: Age-based death threshold (default: 3.0)
- ERROR_THRESHOLD_MULTIPLIER: Error-based death threshold (default: 2.0)

Evolution Goals:
- Growth Phase: Set REAP_CHANCE=0.0 to allow population growth
- Selection Phase: Enable reaping with REAP_CHANCE=0.01 for natural selection
- Stable Evolution: Balance birth/death rates for continuous evolution

Mutation Rate Guidelines {#mutation-rates}¶

Mutation rates must balance genetic diversity with organism viability.

Recommended Settings:
- COPY_BIT_FLIP_RATE: 0.01 (1% - higher rate for population growth)
- INSERTION_DELETION_RATE: 0.005 (0.5% - balanced for evolution)
- BACKGROUND_FLIP_INTERVAL: 500000 (reduces background noise)

Research-Based Values:
- Tierra used ~0.1% mutation rates for viable evolution
- Higher rates create mostly non-viable offspring
- Lower rates prevent genetic diversity

Diagnostic Procedures {#diagnostic-procedures}¶

Healthy Evolution Indicators:
- Population stable at 20-500 organisms
- Continuous births and deaths (balanced turnover)
- Genetic diversity ratio > 0.5
- Mutation rate between 0.1-1.0%
- Memory occupancy < 10%

Stagnation Warning Signs:
- Population crashes from peak to < 50 organisms
- Zero or very low birth rates
- All organisms have zero energy
- Genetic diversity ratio = 1.0 (no evolution)
- High mutation rates (> 5%) with low viability

Troubleshooting Steps:
1. Energy Starvation: Reduce MEMORY_RENT_PER_BYTE, increase ENV_HARVEST_YIELD
2. No Selection: Enable reaper with REAP_CHANCE=0.01, lower threshold multipliers
3. Mutation Catastrophe: Reduce INSERTION_DELETION_RATE to 0.001
4. Population Crashes: Check energy balance and reduce mutation rates

Core Variables {#core-variables-detailed}¶

Resource Management {#resource-management}¶

Server Connection {#server-connection}¶

Batching Configuration {#batching-configuration}¶

Adaptive batching is always enabled and no longer controlled by ADAPTIVE_BATCHING; workers ignore the variable if present.

Standard Variables {#standard-variables-detailed}¶

Soup (Memory) Configuration {#soup-configuration}¶

Scheduler Configuration {#scheduler-configuration}¶

Mutation Configuration {#mutation-configuration}¶

Reaper (Mortality) Configuration {#reaper-configuration}¶

Environment Configuration {#environment-configuration}¶

Energy Configuration {#energy-configuration}¶

Evolution Configuration {#evolution-configuration}¶

Ancestor Configuration {#ancestor-configuration}¶

Logging Configuration {#logging-configuration}¶

Checkpointing Configuration {#checkpointing-configuration}¶

Global Simulation Parameters {#global-simulation-parameters}¶

Data Management Configuration {#data-management-configuration}¶

ByteBiota uses a centralized data management system that allows you to customize where all data files are stored. This is particularly useful for production deployments, testing environments, or when you need to store data on specific storage devices.

Data Directory Configuration {#data-directory-configuration}¶

Data Directory Structure {#data-directory-structure}¶

When DATA_BASE_DIR is set, the following directory structure is automatically created:

{DATA_BASE_DIR}/
├── logs/                    # Server and worker log files
├── checkpoints/             # Simulation checkpoint files
│   ├── server/             # Server checkpoint files
│   └── worker/             # Worker checkpoint files
├── cache/                   # Offline cache and temporary files
├── reports/                 # Tuning reports and analysis
└── test-results/           # Test execution results

Data Management Features {#data-management-features}¶

• Centralized Configuration: Set DATA_BASE_DIR once to configure all data paths
• Automatic Directory Creation: All required directories are created automatically
• Environment Override: Individual paths can still be overridden with specific environment variables
• Production Ready: Perfect for containerized deployments and custom storage mounts

Data Management Examples {#data-management-examples}¶

Example 1: Custom Data Directory¶

# Store all data in /var/lib/bytebiota
export DATA_BASE_DIR="/var/lib/bytebiota"

# This automatically sets:
# - Logs: /var/lib/bytebiota/logs/
# - Checkpoints: /var/lib/bytebiota/checkpoints/
# - Cache: /var/lib/bytebiota/cache/
# - Reports: /var/lib/bytebiota/reports/

Example 2: Development Environment¶

# Use a development-specific data directory
export DATA_BASE_DIR="./dev-data"

# All components will use ./dev-data/ subdirectories

Example 3: Production with Custom Storage¶

# Mount custom storage and use it for all data
export DATA_BASE_DIR="/mnt/bytebiota-storage"

# Perfect for high-performance storage or network mounts

Distributed System Configuration {#distributed-system-configuration}¶

Server Configuration {#server-configuration}¶

Web UI Configuration {#web-ui-configuration}¶

WebSocket Configuration {#websocket-configuration}¶

Worker Configuration {#worker-configuration}¶

Worker Identity {#worker-identity}¶

Server Connection {#server-connection-worker}¶

Resource Management {#resource-management-worker}¶

Local Simulation {#local-simulation}¶

Execution Settings {#execution-settings}¶

Worker WebSocket Configuration {#worker-websocket-configuration}¶

Worker Batching Configuration {#worker-batching-configuration}¶

Adaptive batching is always enabled for workers; the legacy ADAPTIVE_BATCHING variable is ignored.

Tuning System Optimization {#tuning-system-optimization}¶

The hybrid tuning system has been optimized to prevent excessive tuning activity:

• Event Fast-Lane Bypass: Event results (births, deaths, seed submissions) with step_count=0 do not trigger tuning assessments, preventing tuning system overload during high-activity periods.
• Step-Based Triggering: Tuning cycles only run when results contain actual simulation steps (step_count > 0), ensuring tuning assessments align with simulation progress.
• Batch Size Optimization: Default BATCH_SIZE increased from 1 to 8 to reduce submission frequency while maintaining responsiveness.

This optimization reduces tuning activity from every 20 seconds to the expected 5-10 minute intervals while maintaining real-time responsiveness for critical events.

Development and Testing {#development-and-testing}¶

Python Environment {#python-environment}¶

Testing and Monitoring {#testing-and-monitoring}¶

Production Configuration {#production-configuration}¶

Systemd Service Environment {#systemd-service-environment}¶

The following environment variables are commonly set in production systemd services:

Server Service:
- PYTHONPATH=/opt/bytebiota/src
- SERVER_HOST=0.0.0.0
- SERVER_PORT=8080
- MAX_WORKERS=100
- SEED_BANK_SIZE=2000
- CHECKPOINT_INTERVAL=300

Worker Service:
- PYTHONPATH=/opt/bytebiota/src
- SERVER_URL=http://localhost:8080
- RESOURCE_PRESET=standard
- POLL_INTERVAL=1.0
- HEARTBEAT_INTERVAL=10.0

Usage Examples {#usage-examples}¶

Example 1: Minimal Desktop Worker (Auto-Detection)¶

# Use auto-detection for optimal settings
python -m bytebiota worker --preset auto

# Output will show:
# Auto-detecting optimal resource configuration...
# Detected: 8 CPU cores, 8192MB memory
# Recommended preset: standard
# Optimal settings: CPU=100%, Batch size=4
# Using auto-detected optimal configuration

Example 2: Dedicated Server Worker (Explicit Limits)¶

# Explicitly set resource limits for dedicated server
python -m bytebiota worker \
  --server-url http://server:8080 \
  --cpu-percent 50 \
  --memory-mb 4096 \
  --batch-size 10

Example 3: Background Worker (Preset)¶

# Use background preset for minimal resource usage
python -m bytebiota worker \
  --preset background \
  --server-url http://server:8080

Example 4: Full Power Worker¶

# Use full preset or explicit 100% CPU
python -m bytebiota worker \
  --preset full \
  --server-url http://server:8080

# Or explicitly set 100% CPU
python -m bytebiota worker \
  --cpu-percent 100 \
  --memory-mb 8192 \
  --server-url http://server:8080

Environment Variable Examples¶

# Basic local development
export SOUP_SIZE=100000
export MAX_OCCUPANCY_RATIO=0.8
export COPY_BIT_FLIP_RATE=0.001
python -m bytebiota run

# Distributed deployment
export SERVER_HOST=0.0.0.0
export SERVER_PORT=8080
export MAX_WORKERS=50
export ENABLE_WEB_UI=true
python -m bytebiota server

# Worker with custom settings
export WORKER_ID=worker-001
export SERVER_URL=http://server:8080
export RESOURCE_PRESET=background
export CPU_PERCENT=25
python -m bytebiota worker

Systemd Service Files¶

# /etc/systemd/system/bytebiota-server.service
[Unit]
Description=ByteBiota Distributed Server
After=network.target

[Service]
Type=simple
User=bytebiota
Group=bytebiota
WorkingDirectory=/opt/bytebiota
Environment=PYTHONPATH=/opt/bytebiota/src
EnvironmentFile=/opt/bytebiota/.env
# Pass env vars explicitly to override CLI defaults
ExecStart=/opt/bytebiota/.venv/bin/python -m bytebiota server --host ${SERVER_HOST} --port ${SERVER_PORT} --no-reload
Restart=always

[Install]
WantedBy=multi-user.target

# /etc/systemd/system/bytebiota-worker@.service
[Unit]
Description=ByteBiota Worker %i
After=network.target

[Service]
Type=simple
User=bytebiota
Group=bytebiota
WorkingDirectory=/opt/bytebiota
Environment=PYTHONPATH=/opt/bytebiota/src
EnvironmentFile=/opt/bytebiota/.env
# Override worker ID for this instance
Environment=WORKER_ID=worker-%i
ExecStart=/opt/bytebiota/.venv/bin/python -m bytebiota worker
Restart=always

[Install]
WantedBy=multi-user.target

Configuration Precedence {#configuration-precedence}¶

Environment variables are loaded in the following order (later values override earlier ones):

1. System environment variables
2. .env file in current directory
3. .env file in parent directories (searched upward)
4. Systemd/supervisor environment settings
5. Command-line arguments

Worker Environment Loading {#worker-environment-loading}¶

Both server and worker components now support automatic .env file loading using python-dotenv. The worker searches for .env files in the same manner as the server:

• Current working directory (where the command is run)
• Parent directories (searched upward from the code file location)
• Stops at the first .env file found

Important Notes:
- .env values only fill missing environment variables (existing env vars take precedence)
- Both Config.from_env() (server) and WorkerConfig.from_env() (worker) load .env files
- Use load_dotenv(override=False) to respect existing environment variables

CLI Argument Precedence {#cli-argument-precedence}¶

Server CLI Behavior:
The python -m bytebiota server command has default values that override environment variables:
- --host defaults to 0.0.0.0 and overrides SERVER_HOST from .env
- --port defaults to 8080 and overrides SERVER_PORT from .env

Solution: Pass environment variables explicitly to CLI:

python -m bytebiota server --host ${SERVER_HOST} --port ${SERVER_PORT}

Worker CLI Behavior:
The python -m bytebiota worker command respects environment variables:
- --worker-id has no default, so .env values are used
- --server-url defaults to http://localhost:8080 but only overrides if explicitly provided

Best Practice: Use .env files for worker configuration, pass explicit values for server CLI.

Validation {#validation}¶

All configuration values are validated at startup. Invalid values will cause the system to exit with an error message indicating the problematic parameter.

Troubleshooting {#troubleshooting}¶

.env File Not Loading¶

Symptoms:
- Environment variables from .env file are not being used
- Worker shows auto-generated ID instead of WORKER_ID from .env
- Server binds to 0.0.0.0 instead of 127.0.0.1 from .env

Solutions:

1. Check file location and permissions:
   bash # Ensure .env is in the correct location ls -la .env # Should show: -rw-r--r-- 1 user group size date .env

2. Verify .env file format:
   bash # Check for clean KEY=value format (no spaces around =) cat .env # Should show: WORKER_ID=geckom-macbook # NOT: WORKER_ID = geckom-macbook

3. Test environment loading:
   bash # Test if .env is loaded python -c " from src.bytebiota.worker.config import WorkerConfig config = WorkerConfig.from_env() print('Worker ID:', config.worker_id) "

4. Check working directory:
   bash # Ensure you're running from the correct directory pwd # Should be the directory containing .env

5. For systemd services:
   bash # Ensure WorkingDirectory points to .env location # Use EnvironmentFile=/path/to/.env in service file # Pass explicit values for server: --host ${SERVER_HOST} --port ${SERVER_PORT}

Related Documentation {#related-documentation}¶

• Configuration Management
• Distributed System Setup
• Production Deployment