# Welcome to AMESA ## What is AMESA? AMESA is a platform for building, training, and deploying **multi-agent AI systems** that optimize physical and industrial processes. AMESA connects data, simulations, agents, training infrastructure, and runtime environments into a cohesive system that turns real-world domain knowledge into autonomous behavior. The architecture below provides an overview of how the AMESA platform works:
### Why AMESA? By unifying simulation, orchestration, training, and deployment into a single platform, AMESA enables teams to: * Reduce time to deploy autonomous agents * Leverage existing models, data, and controls * Combine rule-based, learned, and ML-driven behavior * Bridge the gap between operational expertise and AI *** ## From Data to Autonomy AMESA supports a complete process from data to autonomy: first, historical data is used to create a simulator for agent feedback and to identify required agent skills. Then, using expert knowledge, agents are built, configured and trained on the simulation, learning the skills they need to control the system autonomously. Finally, agents are connected to live enterprise systems for closed-loop control or operator support. Read a whitepaper to learn more. {% file src="/files/8mhXQxMzn4UJAjfGEEmo" %}
### 🧩 Data to Autonomy Overview #### 1. **Establish Agent Feedback (Data and Simulation)** AMESA supports two ways to generate simulations: * Upload **historical industrial data** (CSV format) to generate data-driven simulations automatically using the AMESA Simulation Lab. * **Connect an existing simulation**, such as a physics model or emulator, using a Docker container. These simulations provide a safe environment for skill agents to learn and be evaluated. *** #### **2. Identify Agent Skills (Activity Clusters and Scenarios)** Activity clusters represent different operating regions where the team of agents needs to behave differently to succeed. AMESA automatically identifies activity clusters by analyzing the historical data. These activity clusters can then be entered into the platform as different scenarios for agent training. *** #### **3. Build Agents (Knowledge Components)** AMESA allows you to build agents using expert knowledge components with one of three workflows: **A. No-code workflow:** In the **Agent Orchestration Studio**, you can create skill agents without writing code by defining: * **Goals** – what the agent is trying to achieve (e.g., maximize throughput, maintain temperature) * **Constraints** – rules the agent must follow (e.g., stay within safe operating bounds) * **Success Criteria** – signals that the agent is performing correctly This no-code method allows domain experts to encode their expertise directly into agent logic. **B. Publication workflow:** You can bring in your own logic and models using the **Agent Import API**. Examples include: * Python-based MPC and PID controllers * LLM-powered agents * Custom applications and heuristics * Pretrained ML models Once imported, these skill agents are available in the **Agent Orchestration Studio**β€”a no-code interface for assembling agents into a multi-agent system. **C. Coding agent workflow:** Using the **AMESA Autonomy MCP Server**, you can use coding agents to write agent components and publish them to the Agent Orchestration Studio to use in agent designs. Then use the **Agent Orchestration Studio** to visually compose and coordinate skill agents, perceptors, and orchestrators. This is where domain experts design system-level behavior using drag-and-drop components. Once orchestrated, your multi-agent system connects to AMESA’s **Agent Cloud**, which leverages scalable infrastructureβ€”such as Microsoft Azure or AWS Kubernetes clustersβ€”to train and evaluate agents in simulation. Training uses a reward-driven learning process that improves agent performance over time. *** #### 4. **Deploy Agents** Once trained, agents can be exported for deployment and connected to the production system using either: * The **AMESA runtime server** connects to the production environment using MQTT, OPCUA or HTTP, for use with HMI, DCS, SCADA, or any IoT platform * The **AMESA runtime app** connects directly to the PLC This closes the loop from simulation to actionβ€”turning data into autonomy at scale. *** ## Quick Links Find articles on key topics with these links. ### Try Out a Tutorial
Cover image
Material FusionBuild agents for a realistic use case with step by step instructions./files/V6RBFgZ5V5TGVduZWufZ
### Import Agents and ML-Models to AMESA You can use any model, API, or Python algorithm with AMESA for training agent systems, adding perception, analysis, and communication, and making decisions. See how to configure different types of modules in the UI and by publishing them via the data science workflow for agent system design, training, and deployment.
Create a data-driven simulationUse your historical dada and our no-code simulation tool to create a data-driven simulation./pages/ISrGz8Vlq0BBAzke5oMB
Integrate a machine learning modelUse existing ML models for machine vision, prediction, or analysis in AMESA agent systems/pages/ZJ3ZpGZbyPVbvUfaz4p2
Integrate a language modelSet up an LLM call to add communication to your agent system/pages/h2iZg2wO4PXt2YSZzHYr
Integrate a programmed algorithmConfigure any Python algorithm, such as a PID controller or optimization algorithm, as a skill agent in AMESA/pages/2Qg5PgcxVuYfNyRNYTrF
Integrate third-party softwareUse an API call to external software as a skill agent in AMESA/pages/krGlxwbxhFED6KH2cLQL
Integrate a simulatorConfigure your simulator to work with AMESA/pages/Y2xlB5zyF1ZsURn90eVj
Publish to the platformUpload configured models and algorithms to the AMESA UI with one CLI command/pages/YUo2kF8kkHuslmfSV7Xo
### Create Modular Skill Agents AMESA multi-agent systems are built on modular skills that break down a task into separate parts. Learn how to create skill agents to train with deep reinforcement learning.

Create skill agents with goals and constraints

Create skills agents with subject matter expertise by configuring goals and constraints for learning

/pages/V88QQUOECZsoD2OVMUXs
Create skill agents with rewards using the SDKCreate skill agents that learn with rewards and access additional teaching tools in Python/pages/kr39YdlUWJIv7gLOdw1Y
Publish to the platformUpload skill agents configured with the SDK to the AMESA UI for training and deployment/pages/YUo2kF8kkHuslmfSV7Xo
## Deploy Multi-Agent Systems Once AMESA agentic systems are designed and trained, you can export them to the AMESA runtime to connect with your system. Learn how to deploy an agent within the runtime container and how to use AMESA's tools to analyze agent behavior during both training and deployment.
Evaluate the performance of your multi-agent systemEvaluate performance using the AMESA benchmarking feature/pages/QVjLgHMCh34aVmuTPRdh
Deploy a AMESA multi-agent systemExport a multi-agent system and connect to the AMESA runtime container/pages/vxoUPWfg1aVeRm0Tcc1y
Connect the multi-agent system to your production systemConnect the AMESA runtime container to your system for deployment/pages/mhiNGHjS397z9qIECnX0
Audit multi-agent system behavior with the historianUse the AMESA historian to analyze system behavior in detail/pages/ZQOg3cBKoBahrNAkLStf
# Get Started ## Choose the Right Workflow AMESA is a two-part platform with a no-code UI and a Python Agent Training Library. The interplay of these parts gives AMESA its combination of usability and power. The two parts enable teams to work together effectively. People and roles who primarily use code, such as data scientists and controls engineers, use the Agent Training Library to create modules like ML models and deep reinforcement learning skills and publish them to the UI. Then, subject matter experts, process engineers, and others can use the no-code interface to easily create teams of agents from these modular building blocks and train them to succeed. Coding agents can also write agents and other components and publish them to the Agent Orchestration Studio using the AMESA Autonomy MCP server.
We designed the platform this way because, for complex, high-value processes, there will be some tasks that can only be done through coding, and some team members who prefer to work in code, and other tasks that are better performed through a visual interface. Both parts of the platform work together. Most users use both parts of the platform to some extent but spend more time in the no-code UI or the SDK, depending on their expertise and role. How you will use the platform depends on your role and what part of the process you are working on. ## Access AMESA You can access AMESA Via a no-code UI and a Python Agent Training Library. They work together to enable you to build, train, and deploy autonomous agent systems. ### No-Code UI Access the Agent Orchestration Studio at app.amesa.com. Login with your credentials.
### Python CLI Installation **Installing the Agent Training Library** ```bash pip install -U amesa ``` **Development** ```bash pip install -U amesa-dev ``` > Note: The AMESA CLI and Agent Training Library require Python version 3.10.x {% embed url="" %} ### AMESA Autonomy MCP Server AMESA is designed to work with any coding agent of your choice, including Claude, Cursor, Codex, and GitHub Copilot, letting you build autonomy directly from natural language. As your coding agents work, they can access AMESA documentation to help them successfully build skills, perceptors, and selectors. With the MCP server, your agents can also publish directly into your use cases on the AMESA platform, removing the need for a separate manual publishing step. To get started, install the MCP server in your platform of choice β€” all major coding agent platforms are supported, including Claude and Cursor. You can configure the connection either by providing the MCP URL along with its communication method (http), or by supplying an `.mcp.json` file that describes both. {% file src="/files/hUmeZ60jgoqYxqdO3wSz" %} Once installed, invoke the MCP server using the `/mcp` command. If you're using GitHub Copilot specifically, install from [the `amesa-agents` repo](https://github.com/Composabl/amesa-agents/tree/main) instead, which lets you take advantage of AMESA's pre-configured agents out of the box. # About AMESA ## Using AMESA The AMESA platform has multiple access points. To build and train agents, you can use the no-code Agent Builder Studio, designed to make agent building easy and intuitive. To integrate ML models, LLMs, algorithms, and simulations with AMESA, or to create nuanced reinforcement learning algorithms to add to your agents, use the Python software development kit (SDK) to create agent components and simulators and publish them to your projects.
AMESA platform diagram
Agents train in simulations of the real system. AMESA allows multiple ways to train agents, including several cluster compute options for training at scale. As part of training, the AMESA historian allows you to evaluate agent behavior to improve the design and get better performance. Once agents are trained, the AMESA runtime connects to your system for deployment. ## Machine Teaching To get the most out of AMESA, you can use a method called Machine Teaching to design your agents. Machine Teaching breaks down tasks into skills that the agent can acquire piece by piece. This allows intelligent agents to train quickly and efficiently, enables different technologies to control different parts of the process as appropriate, and makes AI systems accessible and explainable. To learn more about machine teaching and how to design and build intelligent agents: * Read the book: [Designing Autonomous AI (O'Reilly, 2022)](https://learning.oreilly.com/library/view/designing-autonomous-ai/9781098110741/) * Take the online course: [Machine Teaching for Autonomous AI](https://www.coursera.org/learn/machine-teaching-ai) # AMESA Workflow ## A Two-Part Platform AMESA is a two-part platform, with a no-code UI and a Python SDK and CLI. The interplay of these parts is what gives AMESA its combination of usability and power. The two parts enable teams to work together effectively. People and roles who primarily use code, such as data scientists and controls engineers, use the SDK to create components of agents like ML models and deep reinforcement learning skills. Then subject matter experts, process engineers, and others can use the no-code interface to easily create agents from these modular building blocks and train them to succeed. We designed the platform this way because for complex, high-value processes, there will be some tasks that can only be done through coding - and some team members who prefer to work in code - and other tasks that are better performed through a visual interface. Both parts of the platform work together Most users use both parts of the platform to some extent, but spend more time in either the no-code UI or the SDK depending on their expertise and role. How you will use the platform depends on your role and what part of the process you are working on. ## Workflow Steps The main workflow for AMESA is: * **Step 1**: **SDK** | Create agent components * Create skill to train with rewards using deep reinforcement learning. * Create or package ML models to import to UI to add advanced perception to agents. * Create or package LLMs to import to UI add natural language to agents. * Create or package controllers and optimization algorithms to import to UI to add programmed decision-making to agents. * Connect simulators to AMESA. * **Step 2:** **SDK** | Publish agent components to the UI with one CLI command * **Step 3: UI** | Orchestrate modular components together to create agents in the UI * **Step 4: UI** |Train agents at scale with one click using the UI * **Step 5: UI and SDK |** Export trained agents and connect them to the AMESA runtime for deployment # Access AMESA You can access AMESA Via a no-code UI and an SDK. They work together to enable you to build, train, and deploy autonomous agents. ### No-Code UI

No-Code UI

### Python CLI and SDK Installation ### Installing the SDK ```bash pip install -U composabl ``` ### Development ```bash pip install -U composabl-dev ``` > Note: The AMESA CLI and SDK require Python version 3.10.x # Release Notes ### Release Notes 03/06/26 ### Summary This release resolves a bug in the sign-up confirmation process that caused an error page after clicking the confirmation link, along with a minor hotfix. ### New Features This release doesn't contain any new feature. ### Improvements to existing features This release doesn’t contain any improvement to a new feature. ### Bug fixes **Sign-Up Confirmation Error (Major Fix):**\ Resolved an issue where the sign-up confirmation link sent to new users redirected to an error page instead of successfully confirming the account. New users can now complete the registration process as expected. **Minor Hotfix:**\ Implemented a small fix to address a minor issue and improve overall system stability. # CLI Reference ## AMESA CLI API Documentation ### Overview The AMESA CLI provides command-line tools for managing AMESA projects, creating components, running simulators, submitting training jobs, and more. It's designed to streamline the development workflow for building autonomous agents. ### Table of Contents 1. [Installation & Setup](#installation-and-setup) 2. [Authentication](#authentication) 3. [Debug & Version Information](#debug-and-version-information) ### Installation & Setup #### Installation The CLI is included with the main AMESA package: ```bash pip install composabl ``` #### Verify Installation ```bash # Check CLI is available composabl --help # Check version composabl version ``` #### Environment Configuration ```bash # Set license key export AMESA_LICENSE="your-license-key" # Accept EULA export AMESA_EULA_AGREED=1 # Set log level (optional) export LOGLEVEL=INFO # DEBUG, INFO, WARNING, ERROR # Set API endpoint (optional, for custom deployments) export AMESA_API_URL="https://api.amesa.com" # Set environment (optional) export AMESA_ENV=prod # prod, stg, dev ``` ### Authentication #### Login Authenticate with AMESA services: ```bash # Interactive login (opens browser) composabl login ``` The login process: 1. Generates a unique session token 2. Opens your web browser for authentication 3. Saves credentials to `~/.composabl/token` #### Token Management ```bash # Token locations by environment ~/.composabl/token # Production ~/.composabl/token_stg # Staging ~/.composabl/token_dev # Development # Manually set token echo "your-token" > ~/.composabl/token ``` ### Debug & Version Information #### Show Version ```bash composabl version # Output: WRAPPER : 0.0.0 API : 0.0.0 CLI : 0.0.0 CORE : 0.0.0 TRAIN : 0.0.0 ``` #### Show Debug Information ```bash composabl debug # Output: Platform : Linux-5.15.0-1234-generic Python Version : 3.11.5 AMESA Version : 0.0.0 AMESA Core Version: 0.0.0 AMESA Train Version: 0.0.0 AMESA CLI Version : 0.0.0 AMESA API Version : 0.0.0 Is Cython Build : True CPU Count : 16 Memory (GB) : 32.0 Ray Version : 2.12.0 ``` ### Project Structure Recommended organization: ``` my-project/ β”œβ”€β”€ agents/ β”‚ β”œβ”€β”€ temperature-controller/ β”‚ β”‚ └── agent.py β”‚ └── navigation-agent/ β”‚ └── agent.py β”œβ”€β”€ skills/ β”‚ β”œβ”€β”€ maintain-temp/ β”‚ β”‚ β”œβ”€β”€ pyproject.toml β”‚ β”‚ └── maintain_temp/ β”‚ β”‚ └── teacher.py β”‚ └── avoid-obstacles/ β”‚ β”œβ”€β”€ pyproject.toml β”‚ └── avoid_obstacles/ β”‚ └── controller.py β”œβ”€β”€ simulators/ β”‚ └── reactor-sim/ β”‚ β”œβ”€β”€ pyproject.toml β”‚ └── reactor_sim/ β”‚ β”œβ”€β”€ sim.py β”‚ └── sim_impl.py β”œβ”€β”€ perceptors/ β”‚ └── sensor-fusion/ β”‚ β”œβ”€β”€ pyproject.toml β”‚ └── sensor_fusion/ β”‚ └── perceptor.py β”œβ”€β”€ selectors/ β”‚ └── mode-selector/ β”‚ β”œβ”€β”€ pyproject.toml β”‚ └── mode_selector/ β”‚ └── controller.py β”œβ”€β”€ configs/ β”‚ β”œβ”€β”€ training.yaml β”‚ └── deployment.yaml └── recordings/ └── training-runs/ ``` #### Environment Variables ```bash # Core settings export AMESA_LICENSE="your-license-key" export AMESA_EULA_AGREED=1 # API configuration export AMESA_API_URL="https://api.amesa.com" export AMESA_ENV=prod # prod, stg, dev # Logging export LOGLEVEL=INFO # DEBUG, INFO, WARNING, ERROR # SSL certificates (enterprise) export SSL_CERT_FILE="/path/to/cert.pem" export SSL_CERT_PATH="/path/to/certs/" # Kubernetes (if using) export KUBECONFIG="/path/to/kubeconfig" # Docker export DOCKER_HOST="unix:///var/run/docker.sock" ``` #### Shell Completion Enable tab completion for bash/zsh: ```bash # Bash composabl --install-completion bash # Zsh composabl --install-completion zsh # Fish composabl --install-completion fish ``` #### Error Handling Common errors and solutions: ```bash # License error Error: License validation failed Solution: export AMESA_LICENSE="your-key" # Connection error Error: Failed to connect to simulator Solution: Check simulator is running with `composabl sim status` # Import error Error: No module named 'composabl' Solution: pip install composabl # Docker error Error: Cannot connect to Docker daemon Solution: Ensure Docker is running and user has permissions # Out of memory Error: Ray out of memory Solution: Reduce num_workers or increase system memory ``` #### Performance Tips 1. **Parallel Operations** ```bash # Run multiple simulators for i in {1..4}; do composabl sim start sim-demo & done wait ``` 2. **Batch Publishing** ```bash # Publish all skills find skills -name "pyproject.toml" -execdir \ composabl skill publish . \; ``` 3. **Resource Optimization** ```bash # Limit Docker resources export DOCKER_MEMORY="4g" export DOCKER_CPUS="2" ``` ### Troubleshooting #### Enable Debug Logging ```bash # Set debug level export LOGLEVEL=DEBUG # Run with verbose output composabl --verbose sim validate # Save debug output composabl sim validate 2>&1 | tee debug.log ``` # Agent Management Create New Agent ```bash # Interactive mode composabl agent new # With all options composabl agent new \ --name my-agent \ --type local \ --location ./agents/ # Agent types: # - local: For local development # - docker: For Docker deployment # - composabl: For cloud deployment ``` #### Generated Agent Structure ```python # agent.py import os from composabl import Agent, Skill, Trainer, MaintainGoal, Sensor # Accept EULA and set license os.environ["COMPOSABL_EULA_AGREED"] = "1" # os.environ["AMESA_LICENSE"] = "YOUR_LICENSE_KEY" class BalanceTeacher(MaintainGoal): def __init__(self, *args, **kwargs): super().__init__( "pole_theta", "Maintain pole upright", target=0, stop_distance=0.418 ) async def compute_action_mask(self, transformed_sensors, action): return None async def transform_sensors(self, sensors, action): return sensors async def transform_action(self, transformed_sensors, action): return action async def filtered_sensor_space(self): return ["cart_pos", "cart_vel", "pole_theta", "pole_alpha"] def main(): # Create agent a = Agent() # Add sensors a.add_sensors([ Sensor("cart_pos", "Cart Position", lambda obs: obs[0]), Sensor("cart_vel", "Cart Velocity", lambda obs: obs[1]), Sensor("pole_theta", "Pole Angle", lambda obs: obs[2]), Sensor("pole_alpha", "Pole Angular Velocity", lambda obs: obs[3]) ]) # Add skill skill = Skill("pole-balance", BalanceTeacher) a.add_skill(skill) # Configure trainer r = Trainer({ "target": {"local": {"address": "localhost:1337"}}, "post_processing": { "record": { "is_enabled": True, "file_path": "/tmp/composabl/recordings" } } }) # Train r.train(a, train_cycles=5) r.close() if __name__ == "__main__": main() ``` #### Train Agent ```bash # Train Python agent file composabl agent train ./agents/my-agent/agent.py # Train with JSON configuration composabl agent train --agent-json ./configs/agent.json ``` #### Visualize Agent ```bash # Visualize agent structure from JSON composabl agent visualize ./agent.json ``` # Skill Management Create New Skill ```bash # Interactive mode composabl skill new # Create teacher skill composabl skill new \ --name temperature-controller \ --type teacher \ --description "Maintains optimal temperature" \ --location ./skills/ # Create controller skill composabl skill new \ --name pid-controller \ --type controller \ --description "PID control implementation" \ --location ./skills/ # Create coordinated skill composabl skill new \ --name swarm-coordinator \ --type "coordinated population" \ --description "Coordinates drone swarm" \ --location ./skills/ ``` #### Skill Types 1. **teacher**: Reinforcement learning based 2. **controller**: Programmatic control logic 3. **coordinated set**: Coordinate specific skills 4. **coordinated population**: Coordinate populations of agents #### Generated Skill Structure ``` my-skill/ β”œβ”€β”€ pyproject.toml β”œβ”€β”€ README.md └── my_skill/ β”œβ”€β”€ __init__.py └── teacher.py # or controller.py, coach.py ``` Example `pyproject.toml`: ```toml [project] name = "temperature-controller" version = "0.1.0" description = "Maintains optimal temperature" dependencies = ["composabl-core"] [composabl] type = "skill-teacher" entrypoint = "temperature_controller.teacher:Teacher" ``` #### Publish Skill ```bash # Publish to AMESA platform composabl skill publish ./skills/temperature-controller/ # Or specify path explicitly composabl skill publish --path ./skills/temperature-controller/ ``` #### List Skills ```bash # List all skills in current project composabl skill list ``` Output: ``` Name Type Version Description UUID temperature-ctrl teacher 2 Maintains temperature abc123... pressure-ctrl controller 1 Controls pressure def456... team-coordinator coach 1 Coordinates team ghi789... ``` #### Delete Skill ```bash # Interactive selection composabl skill delete ``` # Simulator Management Create New Simulator ```bash # Create basic simulator composabl sim new \ --name reactor-sim \ --description "Chemical reactor simulator" \ --location ./simulators/ # Create Docker-ready simulator composabl sim new \ --name reactor-sim \ --description "Chemical reactor simulator" \ --location ./simulators/ \ --docker ``` #### Generated Simulator Structure Basic simulator: ``` reactor-sim/ β”œβ”€β”€ pyproject.toml β”œβ”€β”€ README.md └── reactor_sim/ β”œβ”€β”€ __init__.py β”œβ”€β”€ sim.py # Environment implementation └── sim_impl.py # AMESA interface ``` Docker simulator adds: ``` reactor-sim/ β”œβ”€β”€ Dockerfile β”œβ”€β”€ main.py β”œβ”€β”€ docker/ β”‚ └── sim-start.sh └── module/ └── reactor_sim/ └── ... ``` #### Run Simulator ```bash # Run from current directory composabl sim run # Run specific simulator composabl sim run ./simulators/reactor-sim/ # Run with options composabl sim run \ --path ./simulators/reactor-sim/ \ --host 0.0.0.0 \ --port 8080 \ --protocol grpc \ --env-init '{"difficulty": "medium", "seed": 42}' ``` #### Validate Simulator ```bash # Validate simulator in current directory composabl sim validate # Validate specific path composabl sim validate ./simulators/reactor-sim/ # Validate running simulator composabl sim validate --address localhost:1337 --protocol grpc ``` Validation checks: * Proper interface implementation * Space definitions * Reset/step functionality * Scenario handling * Rendering capabilities #### Benchmark Simulator ```bash # Benchmark current directory composabl sim benchmark # Benchmark with options composabl sim benchmark \ --samples 5000 \ --address localhost:1337 # Output: Samples per second: 15243.7 ``` #### Docker Simulator Management **Start Simulator Container** ```bash # Start simulator composabl sim start sim-cartpole # Start with historian composabl sim start sim-cartpole --enable-historian # Stream logs composabl sim start sim-cartpole --stream # Get container name only composabl sim start sim-cartpole --show-name ``` **List Running Simulators** ```bash # Show all running simulators composabl sim status # Example output: Name Image Address Ports Historian Version sim-cartpole composabl/sim-cartpole localhost:32768 32768->1337 True latest sim-reactor composabl/sim-reactor localhost:32769 32769->1337 False v2.1.0 ``` **Get Simulator Status** ```bash # Specific simulator status composabl sim status sim-cartpole # Get connection string only composabl sim status sim-cartpole --moniker # Output: localhost:32768 ``` **View Logs** ```bash # View logs composabl sim logs sim-cartpole # Stream logs composabl sim logs sim-cartpole --stream ``` **Stop Simulators** ```bash # Stop specific simulator composabl sim stop sim-cartpole # Stop all simulators composabl sim stop ``` #### Get Simulator Information ```bash # Get info from running simulator composabl sim info --address localhost:1337 # Get info from Docker image composabl sim info --name sim-cartpole # Output: Simulator Info - Address: localhost:1337 - Env ID: CartPole-v1 - Is Port Open? True - Is Channel Available? True Space Info - Sensor Space: Box([-4.8, -inf, -0.418, -inf], [4.8, inf, 0.418, inf]) - Action Space: Discrete(2) Sensors Sensor("cart_position", "") Sensor("cart_velocity", "") Sensor("pole_angle", "") Sensor("pole_angular_velocity", "") ``` #### Get Space Mappings ```bash # From running simulator composabl sim mappings --address localhost:1337 # From Docker image composabl sim mappings --image composabl/sim-demo # Output (JSON): { "action_space": { "space": "Discrete(2)", "mappings": [ {"name": "action", "type": "discrete", "values": [0, 1]} ] }, "sensor_space": { "space": "Box(...)", "mappings": [ {"name": "cart_pos", "type": "continuous", "low": -4.8, "high": 4.8}, {"name": "cart_vel", "type": "continuous", "low": -inf, "high": inf}, {"name": "pole_angle", "type": "continuous", "low": -0.418, "high": 0.418}, {"name": "pole_vel", "type": "continuous", "low": -inf, "high": inf} ] } } ``` #### List Available Simulators ```bash # List local Docker images composabl sim list --local # List from platform composabl sim list # Output: Name Version Description Status UUID reactor-sim 2 Chemical reactor simulator READY abc123 navigation-sim 1 Robot navigation simulator BUILDING def456 ``` #### Publish Simulator ```bash # Publish to platform composabl sim publish ./simulators/reactor-sim/ ``` #### Delete Simulator ```bash # Interactive deletion composabl sim delete ``` # Perceptor Management Create New Perceptor ```bash composabl perceptor new \ --name derivative-calc \ --description "Calculates derivative of sensor values" \ --location ./perceptors/ ``` #### Generated Perceptor Structure ```python # perceptor.py import time from composabl_core import PerceptorImpl class DemoPerceptor(PerceptorImpl): def __init__(self, *args, **kwargs): self.previous_value = None self.previous_time = None async def compute(self, obs_spec, obs): current_value = obs["counter"] current_time = time.time() value_derived = 0 if self.previous_value is not None and self.previous_time is not None: time_delta = current_time - self.previous_time if time_delta > 0: value_derived = (current_value - self.previous_value) / time_delta self.previous_value = current_value self.previous_time = current_time return {"counter_derived": value_derived} def filtered_sensor_space(self, obs): return ["counter"] ``` #### Perceptor Commands ```bash # Publish perceptor composabl perceptor publish ./perceptors/derivative-calc/ # List perceptors composabl perceptor list # Delete perceptor composabl perceptor delete ``` # Orchestrator Management Create New Orchestrator ```bash # Interactive mode composabl selector new # Create controller selector composabl selector new \ --name skill-coordinator \ --type controller \ --description "Coordinates multiple skills" \ --location ./selectors/ # Create teacher selector composabl selector new \ --name adaptive-selector \ --type teacher \ --description "Learns skill selection" \ --location ./selectors/ ``` #### Orchestrator Commands ```bash # Publish selector composabl selector publish ./selectors/skill-coordinator/ # List selectors composabl selector list # Delete selector composabl selector delete ``` # Job Management Submit Training Job ```bash # Submit agent training composabl job submit --agent ./agents/my-agent.py # Submit with JSON config composabl job submit --agent-json ./configs/agent.json ``` #### List Jobs ```bash composabl job list # Output: Job Id Type Start Time Status job-abc123 training 2024-01-15 10:30:00 RUNNING job-def456 evaluation 2024-01-15 09:15:00 COMPLETED job-ghi789 training 2024-01-15 08:00:00 FAILED ``` #### Stop Job ```bash composabl job stop job-abc123 ``` #### Start Job (Kubernetes) ```bash # For Kubernetes controller composabl job start \ --job-id job-abc123 \ --k8s-id k8s-abc123 \ --json-path ./config.json \ --resume ``` # Historian Management The historian records training data for analysis and visualization. #### Start Historian ```bash # Start with default output directory composabl historian start # Start with custom output path composabl historian start /path/to/output/ # Output: Created the containers. Service Container Name Status Connection Details Ports EMQX emqx running localhost:1883 1883->1883, 18083->18083 Historian historian running /path/to/output/ ``` #### Check Historian Status ```bash # Full status composabl historian status # Get EMQX connection string composabl historian status --moniker-emqx # Output: localhost:1883 # Get historian output path composabl historian status --moniker-historian # Output: /path/to/output/ ``` #### Stop Historian ```bash composabl historian stop ``` #### Clean Up Resources ```bash # Remove all historian containers and data composabl historian clean ``` #### Historian Data Format The historian saves data in Delta Lake format: ``` output_directory/ └── run_id/ β”œβ”€β”€ _delta_log/ β”‚ └── 00000000000000000000.json └── part-00000-*.parquet ``` Access data with Python: ```python from deltalake import DeltaTable dt = DeltaTable("/path/to/output/run_id") df = dt.to_pandas() print(f"Recorded {len(df)} steps") ``` # Benchmarking Tools Protocol Benchmark Test simulator performance and protocol overhead: ```bash # Default benchmark composabl benchmark protocol # Specific simulator and environment composabl benchmark protocol \ --sim-name sim-mujoco \ --env-name walker2d \ --running-time-s 30 # With custom address composabl benchmark protocol \ --sim-address localhost:1337 \ --running-time-s 60 # Output: 15432 (iterations per second) ``` #### Benchmark Multiple Environments ```python # Example test script using CLI envs = [ ("sim-demo", ""), ("sim-mujoco", "ant"), ("sim-mujoco", "humanoid"), ("sim-lunar-lander", ""), ("sim-cartpole", "") ] for sim_name, env_name in envs: result = subprocess.run([ "composabl", "benchmark", "protocol", "--sim-name", sim_name, "--env-name", env_name, "--running-time-s", "10" ], capture_output=True, text=True) print(f"{sim_name}/{env_name}: {result.stdout.strip()} iter/s") ``` # CI/CD Integration #### CI/CD Integration **Continuous Integration** ```yaml # .github/workflows/ci.yml name: CI on: [push, pull_request] jobs: validate: runs-on: ubuntu-latest steps: - uses: actions/checkout@v3 - name: Setup Python uses: actions/setup-python@v4 with: python-version: '3.11' - name: Install AMESA run: pip install composabl - name: Validate Simulators run: | for sim in simulators/*/; do composabl sim validate "$sim" done - name: Run Tests run: pytest tests/ ``` **Continuous Deployment** ```yaml # .github/workflows/deploy.yml name: Deploy on: push: branches: [main] jobs: deploy: runs-on: ubuntu-latest steps: - uses: actions/checkout@v3 - name: Setup Python uses: actions/setup-python@v4 with: python-version: '3.11' - name: Install AMESA run: pip install composabl - name: Set Token run: echo "${{ secrets.AMESA_TOKEN }}" > ~/.composabl/token - name: Publish Components run: | # Publish simulators composabl sim publish ./simulators/reactor-sim/ # Publish skills composabl skill publish ./skills/temperature-control/ # Publish perceptors composabl perceptor publish ./perceptors/derivative-calc/ ``` #### Scripting with CLI ```python #!/usr/bin/env python3 import subprocess import json import sys def run_command(cmd): """Run CLI command and return output""" result = subprocess.run( cmd, shell=True, capture_output=True, text=True ) if result.returncode != 0: print(f"Error: {result.stderr}") sys.exit(1) return result.stdout.strip() # Get simulator mappings mappings_json = run_command( "composabl sim mappings --address localhost:1337" ) mappings = json.loads(mappings_json) # Extract sensor names sensors = [ s["name"] for s in mappings["sensor_space"]["mappings"] ] print(f"Found sensors: {sensors}") # Create agent dynamically agent_code = f""" from composabl import Agent, Sensor agent = Agent() agent.add_sensors([ {', '.join(f'Sensor("{s}", "", lambda obs: obs[{i}])' for i, s in enumerate(sensors))} ]) """ with open("dynamic_agent.py", "w") as f: f.write(agent_code) # Train agent run_command("composabl agent train dynamic_agent.py") ``` # Plugins The CLI supports custom plugins: ```python # ~/.composabl/plugins/my_plugin.py import typer app = typer.Typer() @app.command() def custom_command(name: str): """My custom command""" print(f"Hello, {name}!") # Register in ~/.composabl/plugins.yaml plugins: - name: my_plugin module: my_plugin command: custom ``` Use custom commands: ```bash composabl custom custom-command World # Output: Hello, World! ``` # SDK Reference ## AMESA SDK Documentation ### Introduction Welcome to the AMESA SDK documentation. AMESA is an enterprise platform for building, training, and deploying autonomous multi-agent systems using reinforcement learning and intelligent control systems. ### What is AMESA? AMESA provides a comprehensive toolkit for creating AI agents that can learn and adapt to complex environments. Whether you're automating industrial processes, optimizing logistics, or building intelligent game agents, AMESA offers the tools and infrastructure you need. #### Key Features * **Modular Agent Design**: Build agent systems using reusable skills, sensors, and perceptors * **Multiple Training Targets**: Train locally, in Docker, on Kubernetes, or in the cloud * **Flexible Skill Types**: Combine learning-based and programmatic control strategies ### SDK Components The AMESA SDK consists of four main packages: #### 1. AMESA (Main Package) The unified interface that combines all SDK components. This is the primary package you'll install and import. **Key Features:** * Single import for all functionality * Simplified API for common tasks * Integrated licensing and configuration #### 2. AMESA Core The foundation layer providing: * Agent and skill definitions * Sensor and perceptor frameworks * Simulator communication protocols * Goal templates (Maintain, Approach, Avoid, Maximize, Minimize) #### 3. AMESA Train The training infrastructure features: * Checkpointing and resume capabilities * Multi-environment training * Recording and visualization tools #### 4. AMESA CLI Command-line tools for: * Project scaffolding * Component management * Simulator operations * Training job submission * Debugging and validation ### Getting Started #### Prerequisites * Python 3.10 or 3.11 * Docker (optional, for containerized simulators) #### Installation ```bash pip install composabl ``` #### License Requirements AMESA requires a valid license key. Request a license -> #### Quick Start Example ```python import os from composabl import Agent, Skill, Trainer, MaintainGoal, Sensor # Configure licensing os.environ["AMESA_LICENSE"] = "your-license-key" os.environ["AMESA_EULA_AGREED"] = "1" # Create an agent agent = Agent() # Define sensors agent.add_sensors([ Sensor("temperature", "Current temperature in Celsius", lambda obs: obs[0]), Sensor("pressure", "Current pressure in bar", lambda obs: obs[1]) ]) # Create a skill with a maintain goal temp_skill = Skill("maintain-temp", MaintainGoal("temperature", "Keep temperature at 25Β°C", target=25.0)) agent.add_skill(temp_skill) # Configure training trainer = Trainer({ "target": { "docker": { "image": "composabl/sim-reactor:latest" } }, "env": { "name": "reactor-optimization" } }) # Train the agent trainer.train(agent, train_cycles=100) ``` ### Architecture Overview ``` β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ User Application β”‚ β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€ β”‚ AMESA (Main) β”‚ β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€ β”‚ AMESA Core β”‚ AMESA Train β”‚ AMESA CLI β”‚ β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€ β”‚ Infrastructure (Ray, Docker, K8s) β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ ``` ### Core Concepts #### Agents The central orchestrator that combines skills, sensors, and perceptors to solve tasks. #### Skills Modular behaviors that can be: * **Teachers**: Learning-based using reinforcement learning * **Controllers**: Programmatic with deterministic logic * **Selectors**: Choose between multiple sub-skills * **Coordinated**: Multi-agent coordination #### Sensors Transform raw observations into meaningful features for skills. #### Perceptors Pre-process sensor data (e.g., calculate derivatives, filter noise). #### Scenarios Define variable initial conditions for robust training. #### Goals Pre-built objectives: * **MaintainGoal**: Keep a value stable * **ApproachGoal**: Move toward a target * **AvoidGoal**: Stay away from a value * **MaximizeGoal**: Increase a value * **MinimizeGoal**: Decrease a value ### Training Workflow 1. **Define the Agent**: Create sensors and skills 2. **Configure Training**: Set target, algorithm, and resources 3. **Train**: Run training cycles with automatic checkpointing 4. **Evaluate**: Test the trained agent's performance 5. **Deploy**: Package and deploy the trained agent ### Development Workflow 1. **Create Components**: Use CLI to scaffold skills, simulators, etc. 2. **Implement Logic**: Define behaviors in Python 3. **Test Locally**: Validate with local simulators 4. **Publish**: Upload to the AMESA platform 5. **Train at Scale**: Leverage AMESA's training platform {% embed url="" %} # Main ## AMESA Main API Documentation ### Overview The AMESA Main API is the primary interface for the AMESA SDK. It provides a unified wrapper that combines functionality from `composabl-core`, `composabl-train`, and `composabl-cli` into a single, convenient package. ### Installation ```bash pip install composabl ``` This single installation provides access to all AMESA SDK components. ### Package Structure The main package re-exports all public APIs from: * **composabl\_core**: Core components and building blocks * **composabl\_train**: Training infrastructure * **composabl\_cli**: Command-line interface (available via `composabl` command) ### Basic Usage #### Importing All functionality is available through the main `composabl` import: ```python from composabl import ( # Core Components Agent, Skill, Sensor, Scenario, Perceptor, # Skill Types SkillTeacher, SkillController, SkillSelector, SkillCoordinatedSet, SkillCoordinatedPopulation, # Goals MaintainGoal, ApproachGoal, AvoidGoal, MaximizeGoal, MinimizeGoal, # Training Trainer, ) ``` #### Environment Setup Before using AMESA, configure your environment: ```python import os # Required: Set your license key os.environ["AMESA_LICENSE"] = "your-license-key" # Required: Accept the EULA os.environ["AMESA_EULA_AGREED"] = "1" # Optional: Set log level os.environ["LOGLEVEL"] = "INFO" # DEBUG, INFO, WARNING, ERROR ``` #### Goal Types ```python # Maintain a value MaintainGoal(sensor_name, description, target, stop_distance) # Approach a target ApproachGoal(sensor_name, description, target) # Avoid a value AvoidGoal(sensor_name, description, target, stop_distance) # Maximize a metric MaximizeGoal(sensor_name, description) # Minimize a metric MinimizeGoal(sensor_name, description) ``` #### Configuration Options ```python config = { "license": "key", "target": { # Choose one: "local": {"address": "host:port"}, "docker": {"image": "name:tag"} }, "env": { "name": "environment-id", "init": {} # Environment parameters }, "resources": { "sim_count": 4, "num_workers": 2, "num_gpus": 0 } } ``` ### Core Classes ```python # Agent - Main orchestrator agent = Agent() agent.add_sensor(sensor) agent.add_sensors([sensor1, sensor2]) agent.add_skill(skill) agent.add_skills([skill1, skill2]) agent.add_perceptor(perceptor) agent.export(path) agent.draw() # Visualize structure # Skill - Behavior module skill = Skill(name, implementation) # Sensor - Data transformer sensor = Sensor(name, description, extractor_fn) # Scenario - Initial conditions scenario = Scenario(variable_dict) # Trainer - Training orchestrator trainer = Trainer(config) trainer.train(agent, train_cycles) trainer.evaluate(agent, num_episodes) trainer.package(agent) trainer.close() ``` ### Migration Guide If migrating from separate imports: ```python # Old way from composabl_core import Agent, Skill from composabl_train import Trainer # New way (recommended) from composabl import Agent, Skill, Trainer ``` #### Debug Mode Enable detailed logging: ```python import logging logging.basicConfig(level=logging.DEBUG) # Or via environment os.environ["LOGLEVEL"] = "DEBUG" ``` # Quick Start Examples Example 1: Simple Temperature Controller ```python from composabl import Agent, Skill, Sensor, MaintainGoal, Trainer # Create an agent agent = Agent() # Add a sensor agent.add_sensors([ Sensor("temperature", "Current temperature reading", lambda obs: obs["temp"]) ]) # Create a skill with a maintain goal skill = Skill("temp-controller", MaintainGoal("temperature", "Maintain temperature at 25Β°C", target=25.0, stop_distance=0.5)) agent.add_skill(skill) # Train locally trainer = Trainer({ "target": { "local": {"address": "localhost:1337"} } }) trainer.train(agent, train_cycles=10) ``` #### Example 2: Multi-Skill Agent ```python from composabl import ( Agent, Skill, Sensor, SkillSelector, ApproachGoal, AvoidGoal, Trainer ) # Create agent with multiple sensors agent = Agent() agent.add_sensors([ Sensor("position_x", "X coordinate", lambda obs: obs[0]), Sensor("position_y", "Y coordinate", lambda obs: obs[1]), Sensor("obstacle_distance", "Distance to nearest obstacle", lambda obs: obs[2]) ]) # Create navigation skills move_to_target = Skill( "move-to-target", ApproachGoal("position_x", "Reach target X", target=10.0) ) avoid_obstacles = Skill( "avoid-obstacles", AvoidGoal("obstacle_distance", "Stay away from obstacles", target=0.0, stop_distance=2.0) ) # Create a selector to coordinate skills class NavigationSelector(SkillSelector): def select_skill(self, observation): if observation["obstacle_distance"] < 3.0: return "avoid-obstacles" return "move-to-target" navigation = SkillSelector( "navigation", NavigationSelector, children=["move-to-target", "avoid-obstacles"] ) # Build the agent agent.add_skill(move_to_target) agent.add_skill(avoid_obstacles) agent.add_skill(navigation) # Train with Docker simulator trainer = Trainer({ "target": { "docker": { "image": "composabl/sim-navigation:latest" } }, "resources": { "sim_count": 4 # Run 4 parallel simulators } }) trainer.train(agent, train_cycles=50) ``` #### Example 3: Custom Teacher Implementation ```python from composabl import Agent, Skill, SkillTeacher, Trainer class CustomTeacher(SkillTeacher): def __init__(self): self.step_count = 0 async def compute_reward(self, obs, action, sim_reward): # Custom reward logic distance_to_goal = abs(obs["position"] - obs["target"]) return -distance_to_goal # Negative distance as reward async def compute_success_criteria(self, obs, action): # Success when close to target return abs(obs["position"] - obs["target"]) < 0.1 async def compute_termination(self, obs, action): # Terminate after 1000 steps or if out of bounds self.step_count += 1 return self.step_count > 1000 or abs(obs["position"]) > 100 async def transform_sensors(self, sensors, action): # Pass through sensors unchanged return sensors async def transform_action(self, obs, action): # Clip actions to valid range return np.clip(action, -1, 1) async def filtered_sensor_space(self): # Specify which sensors this skill needs return ["position", "velocity", "target"] # Create and train agent agent = Agent() agent.add_skill(Skill("custom-skill", CustomTeacher)) trainer = Trainer({"target": {"local": {"address": "localhost:1337"}}}) trainer.train(agent, train_cycles=20) ``` ### Complete Example: Industrial Controller ```python import os from composabl import ( Agent, Skill, Sensor, Scenario, Perceptor, MaintainGoal, MinimizeGoal, SkillController, SkillSelector, Trainer ) # Configure environment os.environ["AMESA_LICENSE"] = "your-license-key" os.environ["AMESA_EULA_AGREED"] = "1" # Create perceptor for derivative calculation class DerivativePerceptor(Perceptor): def __init__(self): super().__init__() self.last_value = None self.last_time = None async def compute(self, obs_spec, obs): import time current_time = time.time() if self.last_value is not None: dt = current_time - self.last_time derivative = (obs["temperature"] - self.last_value) / dt else: derivative = 0 self.last_value = obs["temperature"] self.last_time = current_time return {"temperature_rate": derivative} # Create the agent agent = Agent() # Add sensors agent.add_sensors([ Sensor("temperature", "Current temperature (Β°C)", lambda obs: obs["temp"]), Sensor("pressure", "Current pressure (bar)", lambda obs: obs["pressure"]), Sensor("flow_rate", "Flow rate (L/min)", lambda obs: obs["flow"]), Sensor("energy_consumption", "Energy usage (kW)", lambda obs: obs["energy"]) ]) # Add perceptor agent.add_perceptor(Perceptor("temp-derivative", DerivativePerceptor)) # Create skills # 1. Temperature control temp_control = Skill( "temperature-control", MaintainGoal("temperature", "Maintain reactor temperature", target=75.0, stop_distance=2.0) ) # 2. Energy optimization energy_optimization = Skill( "energy-optimization", MinimizeGoal("energy_consumption", "Minimize energy usage") ) # 3. Emergency shutdown controller class EmergencyShutdown(SkillController): async def compute_action(self, obs, action): if obs["temperature"] > 100 or obs["pressure"] > 10: return [0, 0, 0] # Shutdown action return None # Let other skills handle async def compute_success_criteria(self, obs, action): return obs["temperature"] < 90 and obs["pressure"] < 8 async def filtered_sensor_space(self): return ["temperature", "pressure"] emergency = Skill("emergency-shutdown", EmergencyShutdown) # 4. Coordinator selector class ProcessCoordinator(SkillSelector): def select_skill(self, obs): # Emergency takes priority if obs["temperature"] > 95 or obs["pressure"] > 9: return "emergency-shutdown" # Normal operation elif obs["temperature_rate"] > 5: # Rapid temperature change return "temperature-control" else: return "energy-optimization" coordinator = SkillSelector( "process-coordinator", ProcessCoordinator, children=["temperature-control", "energy-optimization", "emergency-shutdown"] ) # Build agent hierarchy agent.add_skills([temp_control, energy_optimization, emergency]) agent.add_skill(coordinator) # Define training scenarios scenarios = [ Scenario({ "temperature": {"min": 70, "max": 80}, "pressure": {"min": 5, "max": 7}, "flow_rate": 100, "energy": {"min": 10, "max": 50} }), Scenario({ "temperature": 90, # High temp scenario "pressure": 8, "flow_rate": 150, "energy": 75 }) ] # Configure training config = { "target": { "docker": { "image": "composabl/sim-reactor:latest", "environment": { "SCENARIO_MODE": "variable" } } }, "env": { "name": "reactor-control", "init": { "control_frequency": 10, # Hz "simulation_speed": 100 # 100x real-time } }, "algorithm": { "name": "PPO", "config": { "lr": 0.0003, "gamma": 0.99, "lambda": 0.95, "clip_param": 0.2 } }, "resources": { "sim_count": 8, "num_workers": 4 }, "model": { "fcnet_hiddens": [256, 256], "fcnet_activation": "relu" }, "rollout": { "num_rollout_workers": 4, "num_envs_per_worker": 2 }, "scenarios": scenarios, "post_processing": { "record": { "enabled": True, "file_path": "./recordings", "gif_file_name": "reactor_control.gif" } } } # Train the agent trainer = Trainer(config) # Train with callbacks def on_cycle_complete(cycle, metrics): print(f"Cycle {cycle}: Reward = {metrics['episode_reward_mean']:.2f}") trainer.train( agent, train_cycles=100, callbacks={"on_cycle_complete": on_cycle_complete} ) # Evaluate the trained agent print("Evaluating trained agent...") eval_results = trainer.evaluate(agent, num_episodes=10) print(f"Average reward: {eval_results['episode_reward_mean']:.2f}") print(f"Success rate: {eval_results['custom_metrics']['success_rate']:.2%}") # Export the trained agent agent.export("./trained_agents/reactor_controller.json") # Package for deployment deployed_agent = trainer.package(agent) print("Agent ready for deployment!") # Clean up trainer.close() ``` # Core ## AMESA Core API Documentation ### Overview The AMESA Core API provides the fundamental building blocks for creating autonomous agents. It includes components for agents, skills, sensors, scenarios, perceptors, and the networking infrastructure for simulator communication. #### License Management ```python from composabl_core.utils import license_util # Validate license (done automatically) is_valid, error, info = license_util.validate(license_key) if not is_valid: print(f"License error: {error}") else: print(f"License valid for: {info['organization']}") ``` #### Code Organization ```python # Recommended project structure project/ β”œβ”€β”€ agents/ β”‚ β”œβ”€β”€ __init__.py β”‚ └── temperature_controller.py β”œβ”€β”€ skills/ β”‚ β”œβ”€β”€ __init__.py β”‚ β”œβ”€β”€ teachers/ β”‚ β”‚ β”œβ”€β”€ maintain_temp.py β”‚ β”‚ └── optimize_energy.py β”‚ └── controllers/ β”‚ └── emergency_shutdown.py β”œβ”€β”€ perceptors/ β”‚ β”œβ”€β”€ __init__.py β”‚ └── filters.py └── scenarios/ β”œβ”€β”€ __init__.py └── training_scenarios.py ``` # Agent API ### Agent API The `Agent` class is the central orchestrator that combines skills, sensors, and perceptors. #### Creating an Agent ```python from composabl import Agent # Create a new agent agent = Agent() # Create with ID agent = Agent(id="temperature-controller-v1") ``` #### Agent Methods **Adding Components** ```python # Add single sensor agent.add_sensor(sensor) # Add multiple sensors agent.add_sensors([sensor1, sensor2, sensor3]) # Add skill agent.add_skill(skill) # Add multiple skills agent.add_skills([skill1, skill2]) # Add perceptor agent.add_perceptor(perceptor) # Add scenario agent.add_scenario(scenario) ``` **Serialization** ```python # Export to file agent.export("path/to/agent.json") # Load from file agent = Agent.load("path/to/agent.json") ``` **Visualization** ```python # Display agent structure agent.draw() # Get structure as string structure = agent.get_structure() ``` #### Complete Agent Example ```python from composabl import Agent, Sensor, Skill, Scenario, Perceptor from composabl import MaintainGoal # Create agent agent = Agent(id="reactor-controller") # Add sensors agent.add_sensors([ Sensor("temp", "Temperature in Celsius", lambda obs: obs["temperature"]), Sensor("pressure", "Pressure in bar", lambda obs: obs["pressure_reading"]), Sensor("flow", "Flow rate L/min", lambda obs: obs["flow_rate"]) ]) # Add perceptor class RateCalculator(PerceptorImpl): def __init__(self): self.last_temp = None async def compute(self, obs_spec, obs): rate = 0 if self.last_temp is not None: rate = obs["temperature"] - self.last_temp self.last_temp = obs["temperature"] return {"temp_rate": rate} def filtered_sensor_space(self, obs): return ["temperature"] agent.add_perceptor(Perceptor("rate-calc", RateCalculator)) # Add skills temp_skill = Skill("maintain-temp", MaintainGoal("temp", "Keep temperature stable", target=75.0, stop_distance=2.0)) agent.add_skill(temp_skill) # Add scenarios agent.add_scenario(Scenario({ "temperature": {"min": 70, "max": 80}, "pressure": {"min": 5, "max": 7} })) # Visualize agent.draw() ``` # Sensors API ### Sensors API Sensors transform raw observations into meaningful features for skills. #### Creating Sensors ```python from composabl import Sensor # Simple sensor with lambda sensor = Sensor( name="velocity", description="Current velocity in m/s", lambda_str=lambda obs: obs["speed_kmh"] * 0.277778 ) ``` # Skills API ### Skill Configuration ```python from composabl_core.config import SkillConfig skill_config = SkillConfig( name="temperature-control", type="SkillTeacher", config={ "learning_rate": 0.001, "hidden_layers": [128, 128], "activation": "tanh" } ) ``` ### Skills API Skills define agent behaviors through different implementation strategies. #### Skill Types **1. SkillTeacher (Learning-based)** ```python from composabl import SkillTeacher class CustomTeacher(SkillTeacher): def __init__(self, target_position=10.0): self.target = target_position self.episode_steps = 0 async def compute_reward(self, transformed_obs, action, sim_reward): """Calculate reward for reinforcement learning""" distance = abs(transformed_obs["position"] - self.target) # Shaped reward reward = -distance # Negative distance # Bonus for reaching target if distance < 0.1: reward += 100 # Penalty for energy usage reward -= 0.1 * abs(action[0]) return reward async def compute_success_criteria(self, transformed_obs, action): """Define success condition""" return abs(transformed_obs["position"] - self.target) < 0.1 async def compute_termination(self, transformed_obs, action): """Define episode termination""" self.episode_steps += 1 # Terminate on success if await self.compute_success_criteria(transformed_obs, action): return True # Terminate on failure conditions if abs(transformed_obs["position"]) > 100: # Out of bounds return True # Terminate on timeout return self.episode_steps >= 1000 async def transform_sensors(self, sensors, action): """Preprocess sensors if needed""" # Normalize position to [-1, 1] transformed = dict(sensors) if "position" in transformed: transformed["position"] = transformed["position"] / 50.0 return transformed async def transform_action(self, transformed_obs, action): """Transform action to simulator space""" # Clip action to valid range return np.clip(action, -1, 1) async def filtered_sensor_space(self): """Specify which sensors this skill needs""" return ["position", "velocity", "target"] async def compute_action_mask(self, transformed_obs, action): """Optional: Define valid actions""" # Example: Disable reverse if at boundary if transformed_obs["position"] <= -50: return [True, False] # Can only go forward elif transformed_obs["position"] >= 50: return [False, True] # Can only go backward return None # All actions valid # Create skill with teacher skill = Skill("reach-target", CustomTeacher(target_position=25.0)) ``` **2. SkillController (Programmatic)** ```python from composabl import SkillController class PIDController(SkillController): def __init__(self, kp=1.0, ki=0.1, kd=0.01): self.kp = kp self.ki = ki self.kd = kd self.integral = 0 self.last_error = 0 async def compute_action(self, transformed_obs, action): """Compute PID control action""" # Calculate error error = transformed_obs["setpoint"] - transformed_obs["measurement"] # P term p_term = self.kp * error # I term self.integral += error i_term = self.ki * self.integral # D term derivative = error - self.last_error d_term = self.kd * derivative # Combined output output = p_term + i_term + d_term # Update state self.last_error = error return [output] async def compute_success_criteria(self, transformed_obs, action): """Success when error is small""" error = abs(transformed_obs["setpoint"] - transformed_obs["measurement"]) return error < 0.01 async def compute_termination(self, transformed_obs, action): """Never terminate - continuous control""" return False async def transform_sensors(self, sensors, action): """Pass through""" return sensors async def transform_action(self, transformed_obs, action): """Clip to actuator limits""" return np.clip(action, -100, 100) async def filtered_sensor_space(self): """Required sensors""" return ["measurement", "setpoint"] # Create skill with controller pid_skill = Skill("pid-control", PIDController(kp=2.0, ki=0.5, kd=0.1)) ``` **3. SkillSelector** ```python from composabl import SkillSelector class AdaptiveSelector(SkillSelector): """Selects between different control strategies""" async def compute_action(self, transformed_obs, action): """Return selected skill index""" error = abs(transformed_obs["error"]) if error > 10: return [0] # Aggressive control elif error > 1: return [1] # Moderate control else: return [2] # Fine control async def compute_success_criteria(self, transformed_obs, action): """Success when system is stable""" return transformed_obs["error"] < 0.1 and transformed_obs["rate"] < 0.01 async def filtered_sensor_space(self): return ["error", "rate", "mode"] # Create selector with child skills selector = SkillSelector( name="adaptive-control", implementation=AdaptiveSelector, children=["aggressive-pid", "moderate-pid", "fine-pid"] ) ``` **4. Coordinated Skills** ```python from composabl import ( SkillCoordinatedSet, SkillCoordinatedPopulation, SkillPopulation, SkillCoach ) # Coach for coordinated skills class TeamCoach(SkillCoach): async def compute_reward(self, transformed_obs, action, sim_reward): """Reward for team coordination""" # Reward based on team performance team_distance = transformed_obs["team_spread"] target_reached = transformed_obs["targets_reached"] reward = target_reached * 10 # Reward for reaching targets reward -= team_distance * 0.1 # Penalty for spreading too far return reward async def compute_success_criteria(self, transformed_obs, action): return transformed_obs["all_targets_reached"] async def filtered_sensor_space(self): return ["team_spread", "targets_reached", "all_targets_reached"] # Coordinate specific agents team_set = SkillCoordinatedSet( name="team-coordination", implementation=TeamCoach, skills=[ Skill("agent-1", Agent1Controller), Skill("agent-2", Agent2Controller), Skill("agent-3", Agent3Controller) ] ) # Coordinate a population swarm = SkillCoordinatedPopulation( name="swarm-behavior", implementation=SwarmCoach, skills=[ SkillPopulation("drone", DroneController, amount=10), SkillPopulation("scout", ScoutController, amount=2) ] ) ``` #### Skill Composition Patterns ```python # Hierarchical skill structure navigation = SkillSelector("navigation", NavigationSelector, [ Skill("path-planning", PathPlanner), SkillSelector("obstacle-avoidance", ObstacleSelector, [ Skill("go-around", GoAroundObstacle), Skill("go-over", GoOverObstacle) ]), Skill("target-approach", ApproachTarget) ]) # Skill with fallback class FallbackController(SkillSelector): async def compute_action(self, obs, action): # Try primary skill first if obs["system_health"] > 0.8: return [0] # Normal operation else: return [1] # Fallback/safe mode fallback_skill = SkillSelector( "fault-tolerant-control", FallbackController, ["normal-control", "safe-mode-control"] ) ``` ### Per Skill Configuration ### Algorithms #### PPO (Proximal Policy Optimization) ```python config = { "algorithm": { "name": "PPO", "config": { # Learning "lr": 5e-5, "lr_schedule": None, # or [[0, 1e-3], [1000000, 1e-5]] # PPO specific "use_critic": True, "use_gae": True, "lambda": 0.95, "kl_coeff": 0.2, "kl_target": 0.01, "clip_param": 0.3, "vf_clip_param": 10.0, "entropy_coeff": 0.0, "entropy_coeff_schedule": None, # Training "num_sgd_iter": 30, "sgd_minibatch_size": 128, "shuffle_sequences": True, "vf_loss_coeff": 1.0, "model": { "vf_share_layers": True, "free_log_std": False }, # GAE "gamma": 0.99, "normalize_advantages": True, # Batch settings "train_batch_size": 4000, "rollout_fragment_length": 200 } } } ``` #### SAC (Soft Actor-Critic) ```python config = { "algorithm": { "name": "SAC", "config": { # Learning "lr": 3e-4, "lr_schedule": None, # SAC specific "twin_q": True, "q_model_config": { "fcnet_hiddens": [256, 256], "fcnet_activation": "relu" }, "policy_model_config": { "fcnet_hiddens": [256, 256], "fcnet_activation": "relu" }, "tau": 5e-3, "target_network_update_freq": 1, "initial_alpha": 1.0, "target_entropy": "auto", # Replay buffer "replay_buffer_config": { "type": "MultiAgentPrioritizedReplayBuffer", "capacity": 1000000, "prioritized_replay": True, "prioritized_replay_alpha": 0.6, "prioritized_replay_beta": 0.4, "prioritized_replay_eps": 1e-6 }, # Training "train_batch_size": 256, "gamma": 0.99, "n_step": 1, "grad_clip": None, # Exploration "exploration_config": { "type": "StochasticSampling" } } } } ``` #### DQN (Deep Q-Network) ```python config = { "algorithm": { "name": "DQN", "config": { # Learning "lr": 5e-4, "lr_schedule": None, # DQN specific "dueling": True, "double_q": True, "num_atoms": 1, "noisy": False, "sigma0": 0.5, # Replay buffer "replay_buffer_config": { "type": "MultiAgentReplayBuffer", "capacity": 100000 }, # Exploration "exploration_config": { "type": "EpsilonGreedy", "initial_epsilon": 1.0, "final_epsilon": 0.02, "epsilon_timesteps": 10000 }, # Training "train_batch_size": 32, "gamma": 0.99, "n_step": 1, "target_network_update_freq": 500, # Minimum replay size "replay_buffer_replay_ratio": 0.0, "training_intensity": None } } } ``` #### IMPALA ```python config = { "algorithm": { "name": "IMPALA", "config": { # Learning "lr": 0.0005, "lr_schedule": None, # IMPALA specific "vtrace": True, "vtrace_clip_rho_threshold": 1.0, "vtrace_clip_pg_rho_threshold": 1.0, # Architecture "num_workers": 16, "num_gpus": 1, "num_multi_gpu_tower_stacks": 1, "minibatch_buffer_size": 1, "num_sgd_iter": 1, "replay_proportion": 0.0, "replay_buffer_num_slots": 0, # Training "train_batch_size": 500, "rollout_fragment_length": 50, "max_sample_requests_in_flight_per_worker": 2, # Loss "learner_queue_size": 16, "learner_queue_timeout": 300, "grad_clip": 40.0, "opt_type": "adam", "decay": 0.99, "momentum": 0.0, "epsilon": 0.1, "vf_loss_coeff": 0.5, "entropy_coeff": 0.01 } } } ``` #### Custom Algorithm ```python from ray.rllib.algorithms.algorithm import Algorithm class CustomAlgorithm(Algorithm): @classmethod def get_default_config(cls): config = super().get_default_config() config.update({ "custom_param": 1.0, "special_lr": 0.001 }) return config def setup(self, config): super().setup(config) # Custom setup def training_step(self): # Custom training logic result = super().training_step() result["custom_metric"] = self.custom_computation() return result # Use custom algorithm config = { "algorithm": { "name": "Custom", "class": CustomAlgorithm, "config": { "custom_param": 2.0, "special_lr": 0.0001 } } } ``` # Goals API ### Goals API Pre-built goal implementations for common objectives. #### MaintainGoal ```python from composabl import MaintainGoal # Maintain temperature temp_goal = MaintainGoal( sensor_name="temperature", description="Keep temperature at 25Β°C", target=25.0, stop_distance=0.5 # Tolerance band ) # Maintain with specific reward shaping class CustomMaintainGoal(MaintainGoal): def __init__(self): super().__init__("pressure", "Maintain pressure", target=1.0, stop_distance=0.05) async def compute_reward(self, obs, action, sim_reward): base_reward = await super().compute_reward(obs, action, sim_reward) # Add penalty for control effort effort_penalty = -0.1 * np.sum(np.abs(action)) return base_reward + effort_penalty ``` #### ApproachGoal ```python from composabl import ApproachGoal # Approach a position position_goal = ApproachGoal( sensor_name="distance_to_target", description="Reach the target location", target=0.0 # Zero distance ) # Approach with custom success class TimedApproachGoal(ApproachGoal): def __init__(self): super().__init__("distance", "Reach target quickly", target=0.0) self.start_time = None async def compute_reward(self, obs, action, sim_reward): if self.start_time is None: self.start_time = obs.get("time", 0) base_reward = await super().compute_reward(obs, action, sim_reward) # Bonus for speed elapsed = obs.get("time", 0) - self.start_time speed_bonus = max(0, 10 - elapsed) if obs["distance"] < 0.1 else 0 return base_reward + speed_bonus ``` #### AvoidGoal ```python from composabl import AvoidGoal # Avoid obstacles obstacle_goal = AvoidGoal( sensor_name="nearest_obstacle_distance", description="Stay away from obstacles", target=0.0, # Avoid zero distance stop_distance=2.0 # Safe distance ) # Multiple avoidance targets class MultiAvoidGoal(AvoidGoal): def __init__(self, danger_zones): super().__init__("position", "Avoid danger zones", target=None) self.danger_zones = danger_zones async def compute_reward(self, obs, action, sim_reward): position = obs["position"] # Find minimum distance to any danger zone min_distance = float('inf') for zone in self.danger_zones: distance = np.linalg.norm(position - zone) min_distance = min(min_distance, distance) # Reward increases with distance return min_distance ``` #### MaximizeGoal ```python from composabl import MaximizeGoal # Maximize efficiency efficiency_goal = MaximizeGoal( sensor_name="efficiency_metric", description="Maximize system efficiency" ) # Maximize with constraints class ConstrainedMaximizeGoal(MaximizeGoal): def __init__(self): super().__init__("production_rate", "Maximize production") async def compute_reward(self, obs, action, sim_reward): base_reward = await super().compute_reward(obs, action, sim_reward) # Penalty for violating constraints if obs["temperature"] > 100: base_reward -= 10 if obs["pressure"] > 10: base_reward -= 10 return base_reward ``` #### MinimizeGoal ```python from composabl import MinimizeGoal # Minimize energy consumption energy_goal = MinimizeGoal( sensor_name="power_consumption", description="Minimize energy usage" ) # Minimize with multiple objectives class MultiObjectiveMinimizeGoal(MinimizeGoal): def __init__(self, weights): super().__init__("cost", "Minimize total cost") self.weights = weights async def compute_reward(self, obs, action, sim_reward): # Weighted sum of multiple objectives total_cost = sum( weight * obs[metric] for metric, weight in self.weights.items() ) return -total_cost # Negative because we're minimizing ``` # Scenarios API ### Scenarios API Define variable initial conditions for robust training. #### Basic Scenarios ```python from composabl import Scenario # Fixed scenario fixed_scenario = Scenario({ "temperature": 25.0, "pressure": 1.0, "flow_rate": 100.0 }) # Sample from scenario sampled = fixed_scenario.sample() # Returns: {"temperature": 25.0, "pressure": 1.0, "flow_rate": 100.0} # Variable scenario with ranges variable_scenario = Scenario({ "temperature": [20, 30], "pressure": [0.8, 1.2], "flow_rate": [80, 120] }) # Variable scenario with choices variable_scenario_choices = Scenario({ "temperature": [20, 30, 40], "pressure": [0.8, 1.2, 2], "flow_rate": [80, 120, 150] }) # Sample from scenario sampled = variable_scenario_choices.sample() # Returns: {"temperature": 30, "pressure": 2, "flow_rate": 80} ``` # Perceptors API ### Perceptors API Process sensor data before it reaches skills. #### Basic Perceptor ```python from composabl import Perceptor, PerceptorImpl class NoiseFilter(PerceptorImpl): def __init__(self, alpha=0.1): self.alpha = alpha self.filtered_value = None async def compute(self, obs_spec, obs): """Exponential moving average filter""" raw_value = obs["noisy_sensor"] if self.filtered_value is None: self.filtered_value = raw_value else: # EMA filter self.filtered_value = ( self.alpha * raw_value + (1 - self.alpha) * self.filtered_value ) return {"filtered_sensor": self.filtered_value} def filtered_sensor_space(self, obs): """Specify input sensors""" return ["noisy_sensor"] # Create and add to agent filter_perceptor = Perceptor("noise-filter", NoiseFilter(alpha=0.2)) agent.add_perceptor(filter_perceptor) ``` #### Advanced Perceptors ```python # State estimator perceptor class KalmanFilter(PerceptorImpl): def __init__(self): self.x = np.zeros(2) # State: [position, velocity] self.P = np.eye(2) # Covariance self.F = np.array([[1, 0.1], [0, 1]]) # State transition self.H = np.array([[1, 0]]) # Measurement matrix self.R = 0.1 # Measurement noise self.Q = np.array([[0.01, 0], [0, 0.01]]) # Process noise async def compute(self, obs_spec, obs): # Prediction step self.x = self.F @ self.x self.P = self.F @ self.P @ self.F.T + self.Q # Update step z = obs["position_measurement"] y = z - self.H @ self.x S = self.H @ self.P @ self.H.T + self.R K = self.P @ self.H.T / S self.x = self.x + K * y self.P = (np.eye(2) - K @ self.H) @ self.P return { "estimated_position": self.x[0], "estimated_velocity": self.x[1] } def filtered_sensor_space(self, obs): return ["position_measurement"] # Feature engineering perceptor class FeatureEngineer(PerceptorImpl): def __init__(self): self.history = [] self.window_size = 10 async def compute(self, obs_spec, obs): # Add to history self.history.append(obs["value"]) if len(self.history) > self.window_size: self.history.pop(0) # Calculate features if len(self.history) >= 2: features = { "value": obs["value"], "mean": np.mean(self.history), "std": np.std(self.history), "trend": self.history[-1] - self.history[0], "acceleration": 0 } if len(self.history) >= 3: # Second derivative features["acceleration"] = ( self.history[-1] - 2*self.history[-2] + self.history[-3] ) else: features = { "value": obs["value"], "mean": obs["value"], "std": 0, "trend": 0, "acceleration": 0 } return features def filtered_sensor_space(self, obs): return ["value"] ``` # Train ## AMESA Train API Documentation ### Overview The AMESA Train API provides the training infrastructure for agents using Ray for distributed computing. It handles the complete training lifecycle, including algorithm configuration, resource management, checkpointing, and deployment preparation. ### Trainer API The `Trainer` class is the main interface for training agents. #### Basic Usage ```python from composabl import Trainer from composabl_core.config.trainer_config import ( BenchmarkConfig, RecordConfig, PostProcessingConfig, ) # Create trainer with configuration trainer = Trainer({ "target": { "local": {"address": "localhost:1337"} }, "env": { "name": "my-environment" } }) # Train agent trainer.train(agent, train_cycles=100) # Evaluate performance results = trainer.postprocess( agent, postprocess_config=PostProcessingConfig( file_path="model_files/", record=RecordConfig( avi_file_name="output.avi", gif_file_name="output.gif", max_frames=24 * 5, ), benchmark=BenchmarkConfig( num_episodes_per_scenario=2, file_name="benchmark.json", ), ), ) # Package for deployment deployed_agent = trainer.package(agent) # Clean up resources trainer.close() ``` ### Training Configuration #### Complete Configuration Example ```python config = { # License (required) "license": "your-license-key", # Training target (required) "target": { # See Training Targets section for options }, # Environment configuration "env": { "name": "environment-id", "init": { "param1": "value1", "difficulty": "medium" } } } ``` ### Training Targets #### Local Target Train with a simulator running locally: ```python config = { "target": { "local": { "address": "localhost:1337" } } } ``` #### Docker Target Train with simulators in Docker containers: ```python config = { "target": { "docker": { "image": "composabl/sim-reactor:latest" } } } ``` ### Benchmarking #### Benchmarking For every scenario on the top-level skill, this will perform \`num\_episodes\_per\_scenario\` amount of inference episodes. ```python # Evaluate trained agent eval_results = trainer.postprocess( agent, postprocess_config=PostProcessingConfig( file_path="model_files/", benchmark=BenchmarkConfig( num_episodes_per_scenario=2, file_name="benchmark.json", ), ), ) ``` #### Recording ```python # Evaluation with recording eval_results = trainer.postprocess( agent, postprocess_config=PostProcessingConfig( file_path="model_files/", record=RecordConfig( avi_file_name="output.avi", gif_file_name="output.gif", max_frames=24 * 5, ), ), ) ``` # Glossary **Controller**: A skill agent that makes decisions based on programmed algorithms. Also called a programmed skill agent. **Decisions**: The outputs of a AMESA system - its instructions for controlling the system **Design patterns**: Common structures that can be used to quickly create multi-agent systems **Episode**: An entire run through the task **Functional Pattern**: A machine teaching design pattern used to orchestrate skills in sequences **Iteration**: One decision during agent training or performance **Learned skill agent**: A skill agent that uses DRL to make decisions and learn by practicing in simulation **Orchestration**: Arranging agents as modular building blocks to work together to complete tasks **Orchestrator**: A special type of skill agent that assigns decision-making control to the right decision-making skill agent based on current conditions **Perceptor**: An ML model or other algorithm that interprets sensor data **Plan-Execute Pattern**: A machine teaching design pattern used to orchestrate skill agents in pairs that work together to make decisions **Programmed skill agent**: A skill agents that makes decisions based on programmed algorithms. Also called controllers. **Project**: A collection of agents for the same use case that share the same simulator and top-level goal **Scenarios**: Conditions that are associated with specific skills **Sensors**: The part of the system that takes in information from the simulator or real environment - the eyes, ears, and other senses of the system **Simulator**: The virtual environment where agents practice and improve performance **Skill agents**: Modules within a multi-agent system that make decisions to complete all or part of a task **Strategy Pattern**: A machine teaching design pattern used to orchestrate skill agents in hierarchies **Teacher**: An algorithm that creates a skill agent that uses DRL to learn to make decisions # Sample Use Cases The examples and code samples in this documentation refer to our sample use cases. These examples are real-world use cases with complex goals and constraints. In each case, the AMESA team has built agent systems that exceed the benchmark control technology by orders of magnitude. ## Industrial Mixer
### About the Use Case Learn more: [Read the AMESA whitepaper about the production scheduling use case.](https://cdn.prod.website-files.com/65973bba7be64ecd9a0c2ee8/663a811ded2167215bf3b9cf_Industrial%20Mixer%20Whitepaper.pdf) The industrial mixer use case is a realistic case study of a chemical process control agent system controlling a continuous stirred tank chemical reaction. The agent system controls the temperature in a tank where a chemical reaction occurs to create a product. As the chemicals are stirred together in the tank, the reaction produces heat at a nonlinear, unpredictable rate. If the tank isn’t cooled enough, it can reach dangerous temperatures, a condition called thermal runaway. If it’s cooled too much, not enough product will be produced. The agent system needs to balance these two goals, keeping the tank at the right temperature at every moment to optimize production while ensuring safety. ### Explore Agent System Components [Access perceptors, skill agents, and selectors for this use case.](https://github.com/AMESA/industrial-mixer-tutorial) ## Production Scheduling
### About the Use Case Learn more: [Read the AMESA whitepaper about the production scheduling use case](https://cdn.prod.website-files.com/65973bba7be64ecd9a0c2ee8/66d956fccd689331aa3ce1ba_Production%20Scheduling%20Use%20Case.pdf). The production scheduling use case is an complex production planning problem set in an industrial bakery. The agent system must determine the right amount of cookies, cakes, and cupcakes to make each day, directing teams of workers and equipment and responding to fluctuations in costs, pricing, and demand. The case study, developed in partnership with AMESA partner [Rovisys](https://www.rovisys.com/), requires the agent system to make a choice every minute between 24 possible combinations of equipment, task, employee and product, over the course of a 400-decision day, with the ultimate goal of maximizing profit. # Industrial Mixer This tutorial will take you through the process of building agents for a realistic chemical manufacturing use case. 1. Learn about the use case (this page) 2. [Access AMESA and set up your project](/tutorials/industrial-mixer/getting-started) 3. [Build a benchmark agent system with deep reinforcement learning](/tutorials/industrial-mixer/deep-reinforcement-learning) 4. [Build an agent system using the strategy design pattern](/tutorials/industrial-mixer/strategy-pattern) 5. [Add a perception layer to your strategy pattern agent system](/tutorials/industrial-mixer/strategy-pattern-with-a-perception-layer) 6. [Build an agent system using the plan-execute design pattern](/tutorials/industrial-mixer/plan-execute-pattern) 7. Experiment with your own designs
We wrote a[ white paper about this case study](https://cdn.prod.website-files.com/65973bba7be64ecd9a0c2ee8/663a811ded2167215bf3b9cf_Industrial%20Mixer%20Whitepaper.pdf). Read the white paper to learn about the problem, the agent system designs, and agent system performance. Use this tutorial to build and train all the agents for yourself. ## About the Use Case ### Why an industrial mixer? Mixing and blending tasks are great use cases for intelligent agent systems because they are both complex and high-value. In the example in this tutorial, process improvements could lead to millions in ROI. ### Use Case Overview In the industrial mixer use case, raw materials are stirred together inside a tank, undergoing a reaction that produces the desired end product. The goal of the process is to convert as much of the raw material as possible. But as the chemicals mix and the conversion occurs, the tank heats up. If the temperature gets too high, a condition called β€œthermal runaway” occurs, potentially causing explosions and fires. To produce as much chemical as possible, the operator must constantly adjust the temperature in the tank, keeping it high enough to allow productivity but low enough to avoid any thermal runaway risk. As in all machine teaching use cases, this process can be summarized in the form of a goal (maximize yield) and a constraint (avoid thermal runaway) that must be balanced against each other:
The process is controlled by adjusting the mixture's temperature in the tank using a "jacket" filled with coolant. Lowering the coolant temperature in the jacket lowers the temperature in the tank, decreasing the risk of thermal runaway. However, cooling the tank can also reduce yield. By how much? The answer varies unpredictably – temperature changes affect chemical concentration differently at different parts of the reaction. That nonlinear relationship between temperature and yield is why this is a nuanced process that benefits so much from intelligent automation. ## Simulating a Complex Reaction The industrial mixer simulator uses principles of chemistry to model the behavior of the system in real life.
Learn more about the simulator The main reaction equation is: For evaluating agent performance, yield is defined as the percentage of the product that is at or above the concentration defined by the manufacturer as meeting product specifications. The threshold for thermal runaway is set at 400 degrees Kelvin; in the real reaction, there would be some variability of the threshold depending on other factors. The reaction is nuanced because the conversion rate and the temperature have a nonlinear correlation, expressed by the Arrhenius equation: Agents can be tested with different levels of noise, random fluctuations that distort the ability to recognize patterns. All the agent results in this tutorial are tested with a noise level of 5%. Agents generally perform better with less noise, but as the noise level increases, some agent designs handle it better than others. The plan-execute pattern is the best of these agents at handling noise.
## Benchmark The current automation solution is a linear MPC controller. As the current solution, this agent system's performance is the benchmark for the other designs. The image below shows an MPC controller represented in the visual system of agent designs. The agent takes in sensor information about the temperature in the tank and the concentrations of the chemicals. It passes that information to the skills layer of the agent system. The skills layer contains a single programmed skill agent: control reactor. This skill uses a mathematical model to determine the desired temperature set point for the tank. It also determines the control actions to take to achieve that temperature using the cooling jacket, and outputs those actions as decisions.
In simulation, this agent's conversion rate was 82%. That means that 82% of the reagents were turned into product, with 18% waste, but the risk of thermal runaway is high. Like all technologies, model predictive control has a β€œpersonality,” a unique set of strengths and weaknesses that can be seen reflected in this performance. MPC is a rule follower. It is a way of controlling a process using mathematical relationships that can be fully mapped and programmed by human engineers. It works well in situations that are straightforward and linear.
MPC controller performance results
The agent does a good job in the first steady state at the start. However, as the transition phase begins, the agent’s performance starts to fail. The wide shadow shows that this agent can’t adapt to the unpredictable conditions in the transition. Its performance becomes dangerously inconsistent, potentially allowing the temperature to exceed the thermal runaway checkpoint at nearly every point in the reaction. When an MPC controller is used to control this process in the real world, a human operator needs to step in and take over control before the automated system lets the temperature cross the thermal runaway threshold. ## Get Started The following tutorials will walk you through the anatomy of some multi-agent systems that take different approaches to learning how to control the temperature of a mixer more effectively to maximize yield and avoid thermal runaway. # Get Started This getting started guide takes you through the steps you will need to go through before you can start building your teams of agents. ## Prerequisites * A AMESA account ## Create Your Use Case The first thing you will do is [create your use case](https://docs.amesa.com/build-autonomous-agents/create-a-project) in the AMESA UI. Select the simulator you will use for the project and the assigned team. If you haven't created a team yet, you will need to do that before you create your first project. Then, you can use the AI prompt to explain what you are attempting to achieve with your project and have it bootstrap based on your description. I will describe the use case skills, goals, and constraints and call it Chemical Process Control. Click `New use case` and `Create use case`
## Set Use Case Goals To finish setting up your use case, set your top-level project goal. Now, we will set the goal for this example. In this project, we want to: * Maximize yield (make as much product as we can) ### Goal Title and Description Fill out the goal title and description for the top-level goal. **Goal Title**: Chemical Process Control Goals **Description**: The goal for this project is to maximize yield ### Logical Structure We are going to one condition to represent our goal. 1. To maximize yield, the objective is `Maximize` , and the variable is `Eps_Yield`
Now you can save your changes. **Now that you have created your new use case and set the goal, you can start creating agent systems!** ## Optional: Set Up the Data Science Workflow Some of the steps in this tutorial use AMESA's data science workflow, which allows data scientists and controls engineers to publish any Python module to the AMESA platform to use in multi-agent systems. To use this workflow, complete these steps: * [Install the AMESA CLI](https://docs.amesa.com/get-started/access-amesa#installing-the-sdk) * [Clone the Industrial Mixer Repo](https://github.com/AMESA/industrial-mixer-tutorial) **Now that you have your project created and your goals set you're ready to start creating agents!** # Deep Reinforcement Learning The DRL agent system is a simple design with only one skill agent. This agent system does not use machine teaching to decompose the task into skills that can be trained separately. Instead, the entire reaction is controlled by a skill agent trained with deep reinforcement learning.
Diagram of DRL agent
Let's get started! ## 1. Create your first skill agent This agent system has a single-skill agent called `Control Full Reaction`. To create this skill in the UI go to the skill agent page and click `Create new skill agent`
## 2. Set skill agent goals and constraints Configure your agent to set the instructions for its training sessions. This agent has one goal, to maximize yield, and one constraint, to keep the temperature from going above 400 degrees Kelvin. 1. Click `Add goal` In the left drop-down menu, select Maximize, and in the right one, select `Eps_Yield` . This means the agent will train with the goal of maximizing the total product produced by the end of each episode. 2. Click `Add constraint`. In the left drop-down menu, select `Avoid` , and in the right one, select `T` . After you select `T` you're going to see a slider appear for you to set boundaries you want to train the system to avoid. In this case, we want to set the boundaries from 400 to 500. 3. Save your skill agent configuration and return to the Agent Orchestration Studio.
## 3. Create a Scenario Set scenarios to tell each skill agent what specific conditions or phases of the process to practice in. This skill agent controls the full reaction, so it needs to practice with the reaction as a whole. Go to the Scenarios page and select `Add scenario`, then name it `Control full reaction` and click `Save` . We're going to add two criteria to this scenario, and they are a reference temperature and concentration. Control Full reaction: Cref Is 8.57, Tref Is 311 | [Why these numbers?](#user-content-fn-1)[^1]
## 4. Add the Skill Agent to Your Agent Configuration Drag the skill `control_reaction` that you can now see on the left-hand side of your project onto the skills layer. Click on the skill agent once it's in the skill layer and assign the scenario.
## 5. Run Your Training Session Now, we are ready to train your agent and see the results. First, select our built-in training cluster or one you own and have connected to the platform. Then set the number of cycles. For this tutorial, we suggest running 50. You can run multiple simulations in parallel to speed up training time. Under advanced, you can use GPUs instead of CPUs, set a rollout fragment length, and set the number of benchmark runs. Once you have everything configured, click `Allocate training cycles` . This agent system design has only one agent, so all training cycles will be allocated to our DRL agent. In a multi-agent system, you can assign a different number of training cycles to different agents depending on the complexity of the skill.
## 6. View Results When the training has been completed, you can view your results in the training sessions tab in the UI. This will show you information on how well the agent is learning. You will likely see a steep learning curve as the agent experiments with different control strategies and learns from the results. When the learning curve plateaus, that usually means that the skill is trained. ## Analyze the DRL Agent's Performance **Conversion rate**: 90%\ **Thermal runaway risk**: Low We tested this fully trained agent and plotted the results.
The DRL agent system performs well. Its relatively thin shadow means that it performs consistently over different conditions and stays within the safety threshold almost every time. This agent controls the initial steady state well, staying on the benchmark line. But during the transition, the DRL agent goes off the benchmark line quite a bit. It doesn't notice right away when the transition phase begins, staying too long in the lower region of the graph and then overcorrecting. That's because DRL works by experimentation, teaching itself how to get results by exploring every possible way to tackle a problem. It has no prior knowledge or understanding of a situation and relies entirely on trial and error. That means it is potentially well-suited to complex processes, like the transition phase, that can’t be easily represented mathematically. However, its behavior is erratic because it can’t distinguish between the phases. The DRL agent’s skills do better than the traditional automation benchmark, but still leave room for improvement. [^1]: These numbers represent the conditions at the start of the reaction. Full reaction is the default scenario for this simulator. # Strategy Pattern The strategy pattern is one of the key design patterns of machine teaching. When you use the strategy pattern, you break down the task into specific skill agents that each handle one aspect of the process to be controlled. This allows you to "teach" the agent system using subject matter expertise. In the strategy pattern, each skill agent is either trained using deep reinforcement learning or controlled with a programmed algorithm. Then, a special skill agent called an orchestrator decides which skill agent should make the decision based on the current conditions. In the industrial mixer problem, the process is divided into three skill agents based on the phase of the process. All three action skill agents and the orchestrator are trained with DRL: each skill agent practices in the conditions it will face and learns to control its part of the process by experimenting over time.
> *Think of the strategy pattern as like a math class with three students. Student A loves fractions, Student B is great at decimal problems, and Student C thinks in percentages. The* orchestrator *is their teacher. She reads each question, sees what kind of problem it is, and then assigns it to the student who can solve it best, because of their own special math talent.* Let's get started configuring this agent system! ## 1. Create your skill agents This agent system has three skill agents called `Start Process` ,`Control Transition` , and `Produce Product` . To create these skill agents in the UI, go to the skill agent page and click `Add skill agent` . Create all three skill agents and then set the goals and constraints.
## 2. Set Skill Agent Goals and Constraints The goal for these skill agents is to maximize yield, and the constraint is to keep the temperature from going above 400 degrees Kelvin. The goals and constraints are exactly the same in all three skill agents. The agents become specialized during training, as each skill agent trains in a different scenario, corresponding with the three phases of the process. We will create these scenarios later in the tutorial. 1. Click `Add goal` In the left drop-down menu, select Maximize, and in the right one, select `Eps_Yield` . 2. Click `Add constraint`. In the left drop-down menu, select `Avoid` , and in the right one, select `T` . After you select `T` you're going to see a slider appear for you to set boundaries you want to train the system to avoid. In this case, we want to set the boundaries from 400 to 500. 3. Save your skill agent and return to the Agent Orchestration Studio.
## 3. Add the Skill Agents to Your Strategy Pattern Agent System Drag the skill agents `Start Process`, `Control Transition`, and `Produce Product` that you can now see on the left-hand side of your use case onto the skills layer. Drag the skill agents from the side in the order you want them to be used.
## 3. Configure the Orchestrator The green diamond that appears when you place multiple skill agents alongside each other is the orchestrator. This is the "math teacher" agent that decides which specialized skill agent should be chosen to make each decision. The goals of the top-level orchestrator in an agent will typically be the same as the goals of the agent system as a whole. So, we can set it to `Maximize` `Eps_Yield`. A fixed-order sequence is appropriate for a phased process like the industrial mixer reaction. That means the orchestrator has the skill agent apply the skill agents one at a time, rather than switching back and forth between skill agents.
## 4. Configure Scenarios [Scenarios](/build-multi-agent-systems/configure-scenarios) are key to successfully training an agent with the strategy pattern. Scenarios are different possible conditions represented within the simulation. Skill agents are trained to specialize in different scenarios - for example, the `Start Reaction` skill agent specializes in controlling the reaction when the temperature and concentration levels are those found at the beginning of the reaction. This is what allows the skill agents to differentiate from each other. The three specialized skill agents practice only in their designated phase of the process and learn to succeed in their own specific conditions. The orchestrator practices with the whole process so that it knows which skill agent to choose at any point. ### 4.1 Add Scenarios Go to the Scenarios page using the left-hand navigation menu. Click `Add Scenario` to create a new scenario for your agent to use in training.
When building an agent system for your use case, you will define the scenarios based on your knowledge of the task and process. In this case, we provide the values that define the phases of the chemical manufacturing process. Create these scenarios for your agent: * Full reaction: Cref Is 8.57, Tref Is 311 | [Why these numbers?](#user-content-fn-1)[^1] * Startup: Cref Is 8.5698, Tref Is 311.2612 | [Why these numbers?](#user-content-fn-2)[^2] * Transition: Cref Is 8.56, Tref Is 311, count[^3] Is 22 | [Why these numbers?](#user-content-fn-4)[^4] * Production: Cref Is 2, Tref Is 373.1311 | [Why these numbers?](#user-content-fn-5)[^5] ### 4.2 Create Scenario Flows Scenario flows allow you to connect scenarios that have a sequential relationship to ensure that your agent gets practice in navigating the different conditions in the order in which they will occur. For this problem, you do not need to create sequential connections between the scenarios. Drag all the scenarios to the first column to make them available to your skill agents and orchestrators. ### 4.3 Add Scenarios to Skill Agents and Orchestrators Once you have your scenarios set up and connected with scenario flows, you can add them to skill agents and orchestrators to tell the skill agents and orchestrators what conditions they need to practice in. This helps them to develop their specialized expertise. In the Agent Builder Studio, click on each skill agent and the orchestrator in turn. Check the box for each scenario to apply to the skill agent. * Start process: Startup * Control Transition: Transition * Produce product: Production * Orchestrator: Full reaction ## 5. Run Your Training Session We are ready to train your agent system and see the results. Select the cluster you want to use and the number of training cycles. We suggest you run 150 training cycles. You will see the skill agents training one at a time, and you assign the number of cycles you want each skill agent to use. It will automatically assign an equal number of training sessions for each skill agent, but in some agent system designs, some skill agents might require more training than others. For example, in this use case, the transition phase is more difficult to control than the two steady states, so the `Control Transition` skill agent may need more training time than the others to become effective.
## 6. View Results When the training has been completed, you can view your results in the training sessions tab in the UI. This will show you information on how well the agent is learning. You will likely see a steep learning curve as the agent experiments with different control strategies and learns from the results. When the learning curve plateaus, that usually means that the skill agent is trained. ### Analyzing the Strategy Pattern Agent's Performance **Conversion rate**: 92% **Thermal runaway risk**: Low We tested this fully trained agent and plotted the results.
graph of strategy pattern performance
This agent system performance is not perfect, but it stays closer to the benchmark line than either of the two single-skill agent systems. It just needs some help avoiding thermal runaway. We can provide that by [adding a perception layer.](/tutorials/industrial-mixer/strategy-pattern-with-a-perception-layer) [^1]: These numbers represent the conditions at the start of the reaction. Full reaction is the default scenario for this simulator. [^2]: The numbers for startup are very similar to full reaction, because both scenarios represent approximately the same starting point. Startup is slightly different to make the two scenarios distinct from each other in the simulator. [^3]: Count is a key that keeps count of decisions. Set this scenario parameter by [^4]: The start of the transition phase is determined by the count key, which is a time counter. After 22 time periods (30 seconds in this case), the transition phase begins. [^5]: In the production steady state phase of the process, the concentration of raw materials, measured by Cref, has dropped significantly, from 8.57 to 2. That means that a significant amount of product has been created. # Strategy Pattern with a Perception Layer The [strategy pattern agent system](/tutorials/industrial-mixer/strategy-pattern) performs well, but it's not perfect in avoiding thermal runaway. One good way to address that is to add a perception layer. Perceptors[^1][ ](/build-multi-agent-systems/add-a-perception-layer)are special skill agents that process and interpret sensor data before passing it to the rest of the agent system. To improve the strategy pattern's performance on temperature control, you can add a perception layer that uses machine learning to predict thermal runaway. While skill agents can be built within AMESA's no-code studio or created externally and then imported, perceptors are always built outside of AMESA and then published to the platform to use in agents. In this case, the perceptor is a pre-built ML model saved as a pickle file. This ML model is trained to interpret the sensor data and check for conditions that might indicate an elevated risk of thermal runaway, and then pass that information to the selector along with the rest of the sensor data. In this tutorial, you'll use AMESA's [data science workflow](/tutorials/industrial-mixer/getting-started) to to publish the perceptor to your use case so that you can add it to your agent configuration. > *Think of the perception layer as an additional set of senses that helps the agent system predict when something might go wrong, like a teacher monitoring the class for early signs of trouble.*
Let's get started configuring this agent system! ## 1. Add the Perceptor Skill Agent to Your Use Case This agent system has a perceptor skill agent called `thermal_runaway_predictor`. To publish it to your use case, you will need to open up your favorite code editor and terminal. In your terminal, navigate to the perceptors folder and use this command with the [AMESA CLI](/reference/cli-reference). ```bash composabl perceptor publish thermal_runaway_predictor ``` Return to the agent orchestration studio and refresh the page. The skill agent will appear in the skill agents menu on the left side of your page. ### Explore the Code Files All skill agents, perceptors, and selectors have a minimum of two files in them. A Python file contains the code that the agent system will use, and a config file. Perceptors have some more files to load in ML models and other python packages. **File Structure**
### Thermal Runaway Perceptor **pyproject.toml**
See the code ```python [project] name = "Thermal Runaway Predictor - ML 1.2.2" version = "0.1.0" description = "ML thermal runaway predictor" authors = [{ name = "John Doe", email = "john.doe@amesa.com" }] dependencies = [ "composabl-core", "scikit-learn==1.2.2" ] [composabl] type = "perceptor" entrypoint = "thermal_runaway_predictor.perceptor:ThermalRunawayPredict" # Include additional data files [tool.setuptools.packages.find] where = ["thermal_runaway_predictor"] [tool.setuptools.package-data] "*" = ["*.json", "*.pkl"] ```
**thermal\_runaway\_predictor.py**
See the code ```python from composabl_core import PerceptorImpl ####### import os import pickle # Determine the directory where the current script is located path = os.path.dirname(os.path.realpath(__file__)) class ThermalRunawayPredict(PerceptorImpl): def __init__(self, *args, **kwargs): """ Initialize the ThermalRunawayPredict perceptor with default values and load the machine learning model. Args: *args: Variable length argument list. **kwargs: Arbitrary keyword arguments. """ # Initialize the prediction output variable self.y = 0 # Initialize a flag to indicate thermal runaway status self.thermal_run = 0 # Load the pre-trained machine learning model from a pickle file # The model is expected to be located in the 'ml_models' directory relative to the script's path model_path = os.path.join(path, "ml_models", "ml_predict_temperature_122.pkl") try: with open(model_path, 'rb') as model_file: self.ml_model = pickle.load(model_file) except FileNotFoundError: print(f"Machine learning model not found at {model_path}. Please ensure the model file exists.") self.ml_model = None except Exception as e: print(f"An error occurred while loading the ML model: {e}") self.ml_model = None # Initialize a list to store historical ML predictions if needed self.ML_list = [] # Initialize the last recorded 'Tc' value to compute its change (Ξ”Tc) self.last_Tc = 0 async def compute(self, obs_spec, obs): """ Compute the thermal runaway prediction based on current sensor observations. Args: obs_spec: Observation specification (not used in this implementation). obs: Current sensor observations. Can be a list or a dictionary. Returns: dict: A dictionary containing the thermal runaway prediction. Example: {"thermal_runaway_predict": 1} """ # Ensure that 'obs' is a dictionary. If not, convert it using predefined sensor keys. if not isinstance(obs, dict): # Define the expected sensor keys obs_keys = ['T', 'Tc', 'Ca', 'Cref', 'Tref', 'Conc_Error', 'Eps_Yield', 'Cb_Prod'] # Convert the list to a dictionary by zipping it with the sensor keys obs = dict(zip(obs_keys, obs)) print("Converted 'obs' to dictionary format using predefined sensor keys.") # Calculate the change in 'Tc' (Ξ”Tc) since the last observation if self.last_Tc == 0: # If this is the first observation, assume an initial Ξ”Tc of 5 self.Ξ”Tc = 5 else: # Compute Ξ”Tc as the difference between current 'Tc' and the last recorded 'Tc' try: current_Tc = float(obs['Tc']) self.Ξ”Tc = current_Tc - self.last_Tc except (KeyError, ValueError, TypeError) as e: # Handle cases where 'Tc' is missing or cannot be converted to float print(f"Error accessing or converting 'Tc': {e}") self.Ξ”Tc = 0 # Default to 0 if there's an error # Initialize the prediction output y = 0 # Check if the current temperature 'T' exceeds or equals 340 try: current_T = float(obs['T']) except (KeyError, ValueError, TypeError) as e: print(f"Error accessing or converting 'T': {e}") current_T = 0 # Default to 0 if there's an error if current_T >= 340: # Prepare the feature vector for the ML model try: Ca = float(obs['Ca']) Cref = float(obs['Cref']) except (KeyError, ValueError, TypeError) as e: print(f"Error accessing or converting 'Ca' or 'Cref': {e}") Ca = 0 Cref = 0 # Feature vector: [Ca, T, Tc, Ξ”Tc] X = [[Ca, current_T, self.Ξ”Tc]] # If the ML model was loaded successfully, make a prediction if self.ml_model: try: # Predict the probability of thermal runaway y_proba = self.ml_model.predict_proba(X) # Get the predicted class label (e.g., 0 or 1) y = self.ml_model.predict(X)[0] # Optionally, use the probability to adjust prediction confidence # For example, set y=1 only if the probability of class 1 is >= 0.3 if y_proba[0][1] >= 0.3: y = 1 else: y = 0 except Exception as e: print(f"Error during ML model prediction: {e}") y = 0 else: print("ML model is not loaded. Cannot make predictions.") y = 0 # Update the last recorded 'Tc' with the current value for the next computation try: self.last_Tc = float(obs['Tc']) except (KeyError, ValueError, TypeError) as e: print(f"Error accessing or converting 'Tc' for updating last_Tc: {e}") self.last_Tc = self.last_Tc # Keep the previous value if there's an error # Optionally, store the prediction in ML_list for historical tracking self.ML_list.append(y) # Update the prediction output variable self.y = y # Return the prediction as a dictionary return {"thermal_runaway_predict": y} def filtered_sensor_space(self, obs): """ Define which sensors are relevant for this perceptor. Args: obs: Current sensor observations (not used in this implementation). Returns: list: Names of the sensors to be used. """ # Specify the sensors that this perceptor will use return ['T', 'Tc', 'Ca', 'Cref', 'Tref', 'Conc_Error', 'Eps_Yield', 'Cb_Prod'] ```
## 2. Copy the Strategy Pattern Agent System, name it Strategy Pattern with Perceptor, and add the Perceptor Skill Agent to your Strategy Pattern Agent System Drag the Perceptor `thermal_runaway_predictor` that you can now see on the left-hand side of your use case onto the perception layer.
## 3. Run Your Training Session We are ready to train your agent system and see the results. Select the cluster you want to use and the number of training cycles. We suggest you run 150 training cycles. You will see the skill agents training one at a time, and you assign the number of cycles you want each skill agent to use. It will automatically assign an equal number of training sessions for each skill agent, but in some agent system designs, some skill agents might require more training than others.
## 4. View Results When the training has been completed, you can view your results in the training sessions tab in the UI. This will show you information on how well the agent system is learning. The agent system training results will be a little bit different from the strategy pattern alone. That's because the thermal runaway predictor is making a difference in how the agent system performs. ### Analyzing the Strategy Pattern Agent System’s Performance with Perception **Conversion rate**: 92%\ **Thermal runaway risk**: Very low We tested this fully trained agent system and plotted the results.
> Adding perception improves agent system temperature control performance. The red lines on the graph show where the perceptors helped the agent system make adjustments to avoid thermal runaway. This agent system gets the same yield as the strategy pattern agent, but the improved temperature control has reduced the thermal runaway incidents from low to 0. [^1]: An ML model or other algorithm that interprets sensor data # Plan-Execute Pattern Along with the [strategy pattern](/tutorials/industrial-mixer/strategy-pattern), and the [perception pattern](/tutorials/industrial-mixer/strategy-pattern-with-a-perception-layer), the plan-execute pattern is one of the major [design patterns](#user-content-fn-1)[^1] of machine teaching. In this pattern, the skill agents work together in a skill group, with the first skill agent determining what the action should be and the second skill agent determining how to achieve it. What is special about this agent is that it combines DRL and MPC, the technologies from the two single-skilled agent systems β€” the worst performers β€” to create the best performing agent. In this example, the DRL skill agent first uses its powers of learning and experimentation to determine the goal temperature for the cooling jacket β€” the set point. It then passes this information on to the MPC skill agent, which uses its powers of control and execution to direct the agent on what action to take to achieve the desired temperature. This tutorial will show you how to publish the MPC controller to the platform using the [data science workflow](/tutorials/industrial-mixer/getting-started) and then use it to create a multi-agent system using the plan-execute pattern. > *Remember how the strategy pattern is like a math class where each student solves the problems they are best at, as assigned by the teacher? In the plan-execute pattern, the students work in groups to solve problems together. Let’s say Student A is good at translating word problems into equations, while Student B is good at solving equations. Student A works on each problem first, and then passes it over to Student B, who produces the solution. No teacher is needed here, because the students divide each problem the same way.*
Let's get started configuring this agent! ## 1. Publish the MPC Skill Agent to Your Project This agent has two skill agents called `control_full_reaction` and `mpc-skill-group`. We have [already created](/tutorials/industrial-mixer/deep-reinforcement-learning) `control_full_reaction` in our project, so we only need to publish `mpc-skill-group` to build this agent in the Agent Builder UI. To publish `mpc-skill-group` to your use case you will need to open up your favorite code editor and terminal. In your terminal, navigate to the skills folder of the [Industrial Mixer Repo](https://github.com/AMESA/industrial-mixer-tutorial) and use this command with the AMESA CLI. ```bash composabl skill publish mpc-skill-group ``` Return to the agent orchestration studio and refresh the page. The skill agent will appear in the skills menu on the left of your page. ### Explore the Code Files All skill agents, perceptors, and orchestrators have at least two files in them. A Python file contains the code the skill agent will use, and a config file. 1. `pyproject.toml`, a config file with the following information. 2. A Python file. For this skill agent, we use [controller class](https://docs.amesa.com/build-autonomous-agents/define-skills/configure-programmed-algorithms-as-skills#the-python-controller-class) with the following code and explanations in comments inline. **File Structure**
See the Code ### MPC Skill Group Controller Skill Agent **pyproject.toml**
See the code ```python [project] name = "MPC Skill Group" version = "0.1.0" description = "MPC prepared for Skill Group" authors = [{ name = "John Doe", email = "john.doe@amesa.com" }] dependencies = [ "composabl-core", "scipy", "casadi==3.6.6", "do_mpc==4.6.5" ] [composabl] type = "skill-controller" entrypoint = "mpc_skill_group.controller:Controller" ```
**controller.py**
See the code ```python from random import randint from typing import Dict, List from composabl_core import SkillController ###### import os import math import numpy as np import do_mpc import numpy as np from casadi import * from scipy import interpolate from math import exp # time step (seconds) between state updates Ξ”t = 1 Ο€ = math.pi class Controller(SkillController): def __init__(self, *args, **kwargs): """ Initialize the Controller skill with default values. Args: *args: Variable length argument list. **kwargs: Arbitrary keyword arguments. """ # Initialize a counter to track the number of actions computed self.count = 0 async def compute_action(self, obs, action): """ Compute the control action (Ξ”Tc) based on current observations using Model Predictive Control (MPC). Args: obs (list or dict): Current sensor observations. If list, expected order: ['T', 'Tc', 'Ca', 'Cref', 'Tref'] action: The previous action taken (not directly used but considered for Ξ”Tc calculation). Returns: list: A list containing the computed change in Tc (Ξ”Tc). Example: [Ξ”Tc] """ # Convert observations to dictionary if they are provided as a list if type(obs) == list: obs = { 'T': obs[0], 'Tc': obs[1], 'Ca': obs[2], 'Cref': obs[3], 'Tref': obs[4] } # else: # # Uncomment if you need to ensure all values are floats # for key, value in obs.items(): # obs[key] = float(value) # Handle action input: ensure it's a float value if type(action) == list or type(action) == np.ndarray: action = action[0] elif type(action) == dict: assert type(action['action']) == float action = float(action['action']) else: action = float(action) # Initialize noise variable (currently set to 0; can be modified for stochasticity) noise = 0 # Extract and convert sensor readings to float CrSP = float(obs['Cref']) # Reference concentration Ca0 = float(obs['Ca']) # Actual concentration at current step T0 = float(obs['T']) # Temperature at current step Tc0 = float(obs['Tc']) + action # Cooling liquid temperature adjusted by action # Define constants for the CSTR model F = 1 # Volumetric flow rate (mΒ³/h) V = 1 # Reactor volume (mΒ³) k0 = 34930800 # Pre-exponential nonthermal factor (1/h) E = 11843 # Activation energy per mole (kcal/kmol) R = 1.985875 # Boltzmann's ideal gas constant (kcal/(kmolΒ·K)) Ξ”H = -5960 # Heat of reaction per mole (kcal/kmol) phoCp = 500 # Density multiplied by heat capacity (kcal/(mΒ³Β·K)) UA = 150 # Overall heat transfer coefficient multiplied by tank area (kcal/(KΒ·h)) Cafin = 10 # Inlet concentration (kmol/mΒ³) Tf = 298.2 # Feed temperature (K) # --- MPC MODEL SETUP --- model_type = 'continuous' # Define model type: 'discrete' or 'continuous' model = do_mpc.model.Model(model_type) # Define state variables Ca = model.set_variable(var_type='_x', var_name='Ca', shape=(1,1)) # Concentration T = model.set_variable(var_type='_x', var_name='T', shape=(1,1)) # Temperature # Define measurements (if any) with optional measurement noise model.set_meas('Ca', Ca, meas_noise=True) model.set_meas('T', T, meas_noise=True) # Define control input Tc = model.set_variable(var_type='_u', var_name='Tc') # Cooling liquid temperature # Define time-varying parameters (TVPs) model.set_variable(var_type='_tvp', var_name='Caf') # Inlet concentration (kmol/mΒ³) model.set_variable(var_type='_tvp', var_name='Tref') # Reference temperature (K) # Define model equations (right-hand side) model.set_rhs('Ca', (F/V * (Cafin - Ca)) - (k0 * exp(-E/(R*T))*Ca)) model.set_rhs('T', (F/V *(Tf-T)) - ((Ξ”H/phoCp)*(k0 * exp(-E/(R*T))*Ca)) - ((UA /(phoCp*V)) *(T-Tc))) # Finalize model setup model.setup() # --- CONTROLLER SETUP --- mpc = do_mpc.controller.MPC(model) setup_mpc = { 'n_horizon': 20, # Prediction horizon 'n_robust': 1, # Number of robust steps 'open_loop': 0, # Open-loop setting 't_step': Ξ”t, # Time step (seconds) 'store_full_solution': True # Store full solution } mpc.set_param(**setup_mpc) # Suppress IPOPT solver output for cleaner logs surpress_ipopt = {'ipopt.print_level':0, 'ipopt.sb': 'yes', 'print_time':0} mpc.set_param(nlpsol_opts = surpress_ipopt) # Scaling for states and inputs to improve numerical stability mpc.scaling['_x', 'T'] = 100 mpc.scaling['_u', 'Tc'] = 100 # --- OBJECTIVE FUNCTION --- _x = model.x _tvp = model.tvp _u = model.u # Define terminal and stage cost mterm = ((_x['Ca'] - CrSP))**2 # Terminal cost lterm = ((_x['Ca'] - CrSP))**2 # Stage cost mpc.set_objective(mterm=mterm, lterm=lterm) # Define control input penalties to discourage large control actions mpc.set_rterm(Tc=1.5) # Input penalty for Tc # --- CONSTRAINTS --- # Bounds for state variables mpc.bounds['lower', '_x', 'Ca'] = 0.1 # Minimum concentration mpc.bounds['upper', '_x', 'Ca'] = 12 # Maximum concentration mpc.bounds['upper', '_x', 'T'] = 400 # Maximum temperature mpc.bounds['lower', '_x', 'T'] = 100 # Minimum temperature # Bounds for control inputs mpc.bounds['lower', '_u', 'Tc'] = 273 # Minimum cooling temperature (K) mpc.bounds['upper', '_u', 'Tc'] = 322 # Maximum cooling temperature (K) # --- TIME-VARYING PARAMETERS (TVPs) SETUP --- # Define templates for TVPs tvp_temp_1 = mpc.get_tvp_template() tvp_temp_1['_tvp', :] = np.array([8.5698]) tvp_temp_2 = mpc.get_tvp_template() tvp_temp_2['_tvp', :] = np.array([2]) tvp_temp_3 = mpc.get_tvp_template() tvp_temp_3['_tvp', :] = np.array([2]) # Define a function to update TVPs based on current time def tvp_fun(t_now): p1 = 22 # Time step 1 p2 = 74 # Time step 2 time = 90 # Total time # Define concentration and temperature equilibrium points ceq = [8.57, 6.9275, 5.2850, 3.6425, 2] teq = [311.2612, 327.9968, 341.1084, 354.7246, 373.1311] # Interpolate concentration and temperature based on current time C = interpolate.interp1d([0, p1, p2, time], [8.57, 8.57, 2, 2]) T_ = interpolate.interp1d([0, p1, p2, time], [311.2612, 311.2612, 373.1311, 373.1311]) if t_now < p1: return tvp_temp_1 elif p1 <= t_now < p2: y = float(C(t_now)) tvp_temp_3['_tvp', :] = np.array([y]) return tvp_temp_3 else: return tvp_temp_2 mpc.set_tvp_fun(tvp_fun) # Finalize MPC setup mpc.setup() # --- ESTIMATOR SETUP --- estimator = do_mpc.estimator.StateFeedback(model) # --- SIMULATOR SETUP --- simulator = do_mpc.simulator.Simulator(model) params_simulator = { 't_step': Ξ”t # Time step (seconds) } simulator.set_param(**params_simulator) # Define templates for simulator parameters p_num = simulator.get_p_template() tvp_num = simulator.get_tvp_template() # Define functions for TVPs and uncertain parameters in the simulator def tvp_fun_sim(t_now): return tvp_num def p_fun_sim(t_now): return p_num simulator.set_tvp_fun(tvp_fun_sim) simulator.set_p_fun(p_fun_sim) # Finalize simulator setup simulator.setup() # --- INITIAL STATE SETUP --- # Initialize states for MPC, simulator, and estimator x0 = simulator.x0 x0['Ca'] = Ca0 x0['T'] = T0 u0 = simulator.u0 u0['Tc'] = Tc0 mpc.x0 = x0 simulator.x0 = x0 estimator.x0 = x0 mpc.u0 = u0 simulator.u0 = u0 estimator.u0 = u0 # Set initial guess for MPC mpc.set_initial_guess() # --- MPC CONTROL LOOP --- # Simulate N steps (currently set to 1) u0_old = 0 time_steps = 1 for k in range(time_steps): if k > 1: u0_old = u0[0][0] # Make a control step using MPC u0 = mpc.make_step(x0) # Enforce a maximum change of Β±10 on the control input if k > 1: if u0[0][0] - u0_old > 10: u0 = np.array([[u0_old + 10]]) elif u0[0][0] - u0_old < -10: u0 = np.array([[u0_old - 10]]) else: if u0[0][0] - Tc0 >= 10: u0 = np.array([[Tc0 + 10]]) elif u0[0][0] - Tc0 <= -10: u0 = np.array([[Tc0 - 10]]) # Add Gaussian noise to the measurements error_var = noise Οƒ_max1 = error_var * (8.5698 - 2) Οƒ_max2 = error_var * (373.1311 - 311.2612) mu = 0 v0 = np.array([ mu + Οƒ_max1 * np.random.randn(1, 1)[0], mu + Οƒ_max2 * np.random.randn(1, 1)[0] ]) # Simulate the next step with the control input and noise y_next = simulator.make_step(u0, v0=v0) # MPC simulation step # Reshape state values for consistency state_ops = y_next.reshape((1, 2)) # --- BENCHMARK SETUP --- p1 = 22 p2 = 74 ceq = [8.57, 6.9275, 5.2850, 3.6425, 2] teq = [311.2612, 327.9968, 341.1084, 354.7246, 373.1311] # Interpolate reference concentration and temperature C = interpolate.interp1d([0, p1, p2, time_steps], [8.57, 8.57, 2, 2]) T_ = interpolate.interp1d([0, p1, p2, time_steps], [311.2612, 311.2612, 373.1311, 373.1311]) # Update reference concentrations and temperatures based on current step if k < p1: Cref = 8.5698 Tref = 311.2612 elif p1 <= k < p2: y = float(C(k)) y2 = float(T_(k)) Cref = y Tref = y2 else: Cref = 2 Tref = 373.1311 # Update the estimator with the new measurements x0 = estimator.make_step(y_next) # Update state estimates # Increment the action counter self.count += 1 # Compute the change in Tc (Ξ”Tc) based on the new control input newTc = u0[0][0] dTc = float(newTc) - float(obs['Tc']) # Return the computed Ξ”Tc as a list return [dTc] async def transform_sensors(self, obs): """ Process and potentially modify sensor observations before they are used. Args: obs (dict): Current sensor observations. Returns: dict: Transformed sensor observations. Note: - Currently, this method returns the observations unchanged. - This can be customized to apply transformations if needed. """ # Currently, no transformation is applied to sensors return obs async def filtered_sensor_space(self): """ Define which sensors are relevant for this perceptor. Returns: list: Names of the sensors to be used. Note: - Specifies a list of sensor names that this perceptor will utilize. - Helps in focusing the perceptor's operations on relevant data. """ # Specify the sensors that this perceptor will use return ['T', 'Tc', 'Ca', 'Cref', 'Tref', 'Conc_Error', 'Eps_Yield', 'Cb_Prod'] async def compute_success_criteria(self, transformed_obs, action): """ Determine whether the success criteria have been met. Args: transformed_obs (dict): Transformed sensor observations. action: The action taken. Returns: bool: True if success criteria are met, False otherwise. Behavior: - Currently always returns False. - Can be implemented with logic to check if certain conditions are satisfied. """ # Placeholder for success criteria logic return False async def compute_termination(self, transformed_obs, action): """ Determine whether the training episode should terminate. Args: transformed_obs (dict): Transformed sensor observations. action: The action taken. Returns: bool: True if the episode should terminate, False otherwise. Behavior: - Currently always returns False. - Can be implemented with logic to terminate based on certain conditions. """ # Placeholder for termination condition logic return False ```
## 2. Build the Plan Execute Pattern Agent System in the Agent Orchestrator UI First, drag the skill agent `control_full_reaction` from the hand side of the page to the skill layer. Once it's there, drag over the `mpc-skill-group` and make sure that it is dropped below the `control_full_reaction` skill agent and not beside it.
## 3. Run Your Training Session We are ready to train your agent system and see the results. Select the cluster you want to use and the number of training cycles. We suggest you run 50 training cycles. You will see the skill agents training one at a time, and you assign the number of cycles you want each skill agent to use. It will automatically assign an equal number of training sessions for each skill agent, but in some agent system designs, some skill agents might require more training than others.
## 4. View Results When the training has been completed, you can view your results in the training sessions tab in the UI. This will show you information on how well the agent is learning. ### Analyzing the Plan-Execute Pattern Agent’s Performance **Conversion rate**: 95%\ **Thermal runaway risk**: Very low We tested this fully trained agent and plotted the results.
Graph of plan-execute agent performance, showing the agent sticking closely to the reference line
This agent is the best performer of the group. Combining two imperfect technologies together with Machine Teaching produces much better results than either technology achieves alone. [^1]: Common structures that can be used to create agents quickly # Material Fusion This tutorial will take you through the process of building agents for a realistic use case. 1. Learn about the use case (this page) 2. [Create a simulation from data](/tutorials/material-fusion/create-a-simulation) 3. [Generate activity clusters and scenarios](/tutorials/material-fusion/generate-activity-clusters-and-scenarios) 4. [Configure and train your first agent](/tutorials/material-fusion/configure-and-train-your-first-agent) 5. [Build a multi-agent system](/tutorials/material-fusion/build-a-multi-agent-system) 6. Evaluate agent performance Read a white paper about this use case, with detailed end to end steps from data to autonomy. {% file src="/files/8mhXQxMzn4UJAjfGEEmo" %} ## About the Use Case
Material fusion is like welding for plastics. Plastic materials are heated and pressed together, creating a new substance. The critical control actions are temperature and pressure. Heat makes the plastic hot enough to fuse. Pressure presses the materials together. A quality score represents how completely and successfully the materials are fused together. The optimization goal is to maximize the quality metric by controlling the conditions of manufacturing. # Create a Simulation The first step to training a team of agents for a use case is to create or connect a simulation. A simulation is a virtual version of the real system where agents can make decisions, get feedback, and improve their performance over an infinite number of training episodes. This simulation is the training ground for the agents and gives them the feedback necessary to learn and improve. ## Data-Driven Simulation For material fusion, we will use AMESA's simulation lab to create a data-driven simulation. This means starting with a historical data set and then expanding that finite set of real data into an infinite set of representative synthetic data for agents to use for training. There are a few basic steps to prepare data and create a simulation with AMESA: 1. Gather data 2. Merge, clean, and assess and remove outliers 3. Format for simulation 4. Upload the data to AMESA and receive a data score 5. Create simulation Here is the prepared data set for material fusion. {% file src="/files/XwMTA6lZXnUiOTSWAntE" %} ### 1. Gather Data: Identify Variables One of the most important steps in gathering data is to identify the variables that will be used in the dataset and ultimately the simulator. #### Identify Core Variables Not all variables in a dataset need to be included in a simulation. For example, an automated lane changing system for a self-driving car needs to know acceleration, lane position, and the position of other cars. But it does not need to know oil temperature, even if that variable is included in the full set of what is measured. The variables that are necessary to control the system are called **core variables**. Remove unnecessary variables and include only core variables in your dataset for AMESA. There may also be variables in the dataset that are not directly measured, but are calculations based on variables that are directly measured. Do not include these in your dataset. If your agent needs access to this information, include the variables as [perceptors](/build-multi-agent-systems/add-a-perception-layer). #### Sensors and Actions There are two types of variables to use with AMESA: sensors and actions. * **Actions** are control variables, β€œknobs” that are used to control the system * **Sensors** are environment or state variables, β€œgauges” that report on the conditions in the system All variables are either sensors or actions. For the Material Fusion use case, the variables are: * **Action:** temperature set point * **Action:** pressure set point * **Sensor:** material fusion metric - a measure of how well the materials have bonded * **Sensor:** material transformation metric - a measure of how well the separate plastics have physically transformed into a new substance * **Quality rating:** a measure of final product quality ### 2. Merge and Clean Data and Assess and Remove Outliers #### Merge and Clean Data Data for building simulations often lives in multiple systems, but AMESA requires a single data file to create a simulation. Merge the data into a single dataset. AMESA can only accept data that is numeric. Remove any non-numeric data from the dataset. AMESA will also ignore any row that has null values in it. Remove or fill null cells in the dataset. You may fill nulls with fill-forward, fill-backwards, or interpolation, depending on the specifics of the missing data. #### Remove Outliers Outliers are a common issue in industrial datasets, often caused by sensor malfunctions or other disturbances to the process. Outliers are also one of the most common reasons for a low simulation score in AMESA's simulation lab. Outliers are tricky because they often represent simple bad data, in which case they should be removed from the dataset. But at times they represent legitimate instances of unusual behavior that the system needs to be able to learn to control. In those cases, they should remain in the dataset and be included in the simulated behavior used for agent training. Use subject matter expertise to determine whether to remove outliers or include them on a case by case basis. For the Material Fusion use case, the dataset is clean and does not need outliers removed. ### 3. Format the Data for Simulation To create a simulation in AMESA: * The data should be saved in a .CSV file * The columns should be labeled as sensors and actions, with a s\_ prefix before the variable name for sensors and an a\_ prefix for actions (e.g. s\_Material\_Fusion\_Metric, a\_Temperature) * There should be no spaces or special characters in the variable names * Optionally, units can be included in the first line of the dataset ### 4. Upload the Dataset to Receive a Data Score
Upload the .CSV file to the AMEA platform. You will receive a data score that reflects how well the sensors and actions are correlated. Datasets with strong correlations can create high quality simulations.
### 5. Create Simulation Click β€œcreate simulation.” The platform will create the simulation and provide a simulation score.\ The simulation score measures how closely the synthetic data within the simulation matches the real data within the dataset.
You have now created a data-driven simulation and are ready to identify skills and build and train agents. # Generate Activity Clusters and Scenarios After creating a simulation, it's time to generate activity clusters and scenarios, unique operating regions where the agents need to perform differently to succeed. ## Activity Clusters
As part of simulation creation, AMESA automatically generates activity clusters. This process produces three things: a visual display of the clusters, labels for each cluster, and sensor ranges for each cluster.
## Scenarios The sensor ranges for each cluster are used to create scenarios. Scenarios are a way to divide the state space into separate regions for training, so that agents can practice only in the conditions they need to control. This file contains all the sensor ranges for each scenario for the Material Fusion example. {% file src="/files/0MQUiPN8R2rSd2mgaZQH" %} Before you set up scenarios,you must first [set up a use case](/build-multi-agent-systems/create-a-project). Then set up scenarios by entering the sensor range information on the [scenarios page](/build-multi-agent-systems/configure-scenarios) in the Agent Orchestration Studio. # Configure and Train Your First Agent Once you have set up your use case with scenarios, your next step is to configure and train your first agent. ## Configure Agents In the Agent Orchestration Studio, you can configure agents with goals, constraints, and success criteria. ### Configure Agents with the No-Code UI Agents can pursue three goal objectives: maximize, minimize, and maintain.\ Goal statements can apply to any sensor variable. For the Material Fusion example, the overall goal of the process is to maximize the quality rating variable. So your agent should be configured with one goal: maximize quality.
For material fusion, you do not need to use constraints or success criteria. ### Use Coding Agents to Create Agents Alternatively, you can use the coding agent workflow to create and configure agents for the Material Fusion use case. Prompt your coding agents to create an agent with a goal to maximize the variable quality\_rating and publish it to your organization and use case. > Create a teacher agent with a goal to maximize quality\_rating. Include all required files and dependencies. Use the MCP server to publish it to \ and \. After the MCP server publishes the agent, it will be visible in your list of agents in the Agent Orchestration Studio. ## Build a Single-Agent System Eventually, you will build a multi-agent system with a different agent for each scenario. For now, you will build a single-agent system with one agent that learns to specialize in one scenario. Drag your agent into a blank template within the Agent Orchestration Studio. Then select one scenario from the scenarios list on the right.
## Train Agent Click Train Agent to begin training. For this problem, select 50 training cycles. (For your own industrial control use cases, you will likely need more cycles.) Each training cycle is about 1,000 decisions. On the Training Sessions page, you'll see the training in progress. A successful training session shows a learning curve with a steep initial climb followed by a plateau, indiciating that the agent has successfully learned.
# Build a Multi-Agent System After building and training your first agent, you are ready to orchestrate agents together into a multi-agent system. ## Orchestrate Agents For this use case, every agent in your multi-agent system will have the same goal, to maximize quality. What differentiates the agents from each other is that each will practice in and master a separate scenario.
There are six scenarios, so you will need six separate agents. Drag the Maximize Quality goal into the agent system six times. Then select a different scenario for each of the six agents. This is what will allow them to learn to specialize differently and develop the unique skills they need. A supervisor agent called an orchestrator will choose the appropriate agent to take control based on the environmental conditions in the simulator or real system. This orchestration method is called the Strategy Pattern because it allows the system to use different control strategies at different times depending on the need. ## Configure Orchestrators The orchestrator is the green diamond above the purple agents. When you drag more than one skill in parallel into the Agent Orchestration Studio, an orchestrator automatically appears. Orchestrators can be learned or programmed. Programmed orchestrators that use deterministic logic to select the right agent need to be created in code and published to the Agent Orchestration Studio. Learned orchestrators can be configured with goals, just like any other agent. In this case, you should configure the orchestrator with the same goal as the other agents: maximize quality. Orchestrators can also choose agents in either a fixed or a variable order. In this case, the orchestrator should be a variable-order orchestrator that chooses whichever skill is correct in the moment, without regard for sequence. ## Train Your Multi-Agent System Once you have all your agents and your orchestrator configured, click Start Training to train your multi-agent system. You now have more agents, so you need more training cycles to distribute among them. Try training for 140 training cycles, with 20 each for each agent and 20 for the orchestrator. # Evaluate Agent Performance Once your agent has trained, you'll want to know how it performed - how well did it control the system? Was it successful? Did it beat benchmarks? To determine this, after training is finished AMESA runs "benchmark episodes." In these episodes, the agent is controlling the process, with the simulation giving feedback, but unlike in experimental training episodes, the agent is always making what it has learned is the best decision. The default is one benchmark episode; you can set the number of benchmark episodes under Advanced Configuration when you start your training job. ## Benchmark and KPI When you set up your use case, you set the KPI and the benchmark.
In this case, the benchmark is a quality rating of 87 - that is the number that your agent needs to beat to succeed. ## Performance Graphs
On the Benchmarks page, you will see multiple visualizations of your agents' performance. This bar chart compares every agent orchestration within the use case to each other and to the benchmark, represented by a dotted line. You can see that for the Material Fusion use case, both the single agent systems and the multi-agent system beat the benchmark by similar margins. ## Investigate Agent Behavior Further down the Benchmarks page, additional plots allow you to visualize agent behavior in detail.
This plot shows the quality rating in green and the material fusion metric variable in red. The important thing to note is that at the beginning, the material fusion metric is high, leading to a quality rating that is low. But because the agent has learned well, it quickly adjusts its control so that the material fusion metric and quality rating are more favorable. Think back to [activity clusters](/tutorials/material-fusion/generate-activity-clusters-and-scenarios). What you see in this plot is the agent moving from an unsuccessful cluster to a successful one. When you see results like this, you know you have a successful agent and you can begin thinking about testing it for deployment. ## Dig Deeper To dig deeper into agent performance, AMESA allows you to export a benchmark.json file with detailed information about every decision made during the benchmarking episodes. You can analyze this using your preferred data analysis tool. If you have [AMESA's Assist Agents](https://github.com/Composabl/amesa-agents/tree/main), you can also use the Analyst to gain insight about what happened during the benchmarking episodes and how your agent performed. # Solution Design Welcome! In this course, you'll learn to design a team of agents that can perform high-value tasks in the real world. From designing a use case to interviewing experts all the way to configuring and orchestrating agents, this course will take you through the process of designing solutions with AMESA. ## Learning Objectives At the end of this course you will be able to: * Define a use case * Complete and document an expert interview * Design and orchestrate a multi-agent system ## Course Format This is a self-paced course designed for you to complete on your own time. Work through each module by watching the videos and doing the practice exercises and quizzes. This course should take you about 3 hours. ### Course Sequence This course is intended to be completed after the foundational course in this series. In that course, you learned the basics of AMESA and machine teaching, what you can do with a team of agents, and how to identify a use case where a team of agents trained with AMESA will outperform benchmarks. After you complete this course, move on to the third course in the series. In that course, you will learn to build your team of agents in the AMESA platform. ## Course Materials {% file src="/files/kSuo2NJDzkFacjIZFNqj" %} {% file src="/files/pkus1MP7NE0z9cHzC48T" %} * [AMESA documentation](https://docs.amesa.com) * [Glossary](/reference/glossary) ### Case Studies This course uses a number of case studies for realistic practice. Read them here or download a file with all. #### Crude Oil Blending
Oil refineries turn crude oil into everyday products like gasoline, kerosene, and fuel oil. The problem is that not all crude oils are the same. Some are β€œlight” and easy to process, while others are β€œheavy,” full of impurities like sulfur, wax, and metals. Refineries must work with whatever crude oils are available on the market, which may not match their equipment or product quality needs. To solve this, refineries blend different crude oils togetherβ€”like mixing ingredients in a recipeβ€”to get a feedstock that meets product quality standards. For example, heavy crude (cheap but hard to process) can be mixed with light crude (expensive but easier to process) to create a blend that balances cost, quality, and processing efficiency. The refinery has dozens of possible crude oils available, each with different costs and physical and chemical propertiesβ€”density, sulfur, viscosity, wax, and metal contentβ€”that affect how it behaves when heated and separated. Some properties, like sulfur or density, combine linearly and are easy to calculate by weighted average. Others, like viscosity and stability, are nonlinear, meaning the result cannot be directly predicted from the components. Unstable blends can cause asphaltenes (heavy organic molecules) to precipitate, leading to fouling and reduced throughput. Refining the crude oil streams into usable products is a multi-step process. The first step requires deciding how much of each crude oil type should be combined in the initial mixing process in an atmospheric distillation unit. This unit then creates several products. Some of these, like kerosene, diesel, and fuel oil, are sold as-is. Another product, naphtha, can be upgraded further in a different piece of equipment called a fixed-bed reformer, to create different octane levels of gasoline. Additional purchased products, including supplemental naphtha or gasoline, can also be added at this phase to increase higher-grade product quantities. The refining process is continuous, but decisions about crude oil selections and product proportions are made daily to respond to shifting material availability, material cost, and demand for different products. The core automation challenge is to choose inputs, check them against physical and chemical constraints, and find the blending strategy that gives the highest profit while meeting all requirements. Read a white paper about this use case. #### Enzyme Reactor
An enzyme production facility must control conditions within a reactor to ensure that microbes can thrive and produce enzymes efficiently. The system relies on a continuous feed of nutrient β€œfood.” First, the food is pasteurized before entering the reactor. If food is not heated to the proper rangeβ€”170Β°F for at least three minutes or 175Β°F for 2.5 minutesβ€”harmful microbes could contaminate and wipe out the enzyme-producing culture. But heating the food above 190Β°F creates another problem: caramelization, which fouls equipment and strips away essential nutrients. Inside the reactor, conditions must be carefully balanced. Microbes grow optimally at 98Β°F, 17.5% dissolved oxygen, and pH 7.5. Deviations from these values quickly affect growth. For instance, a scarcity of food reduces growth rates and permanently damages some microbes’ ability to reproduce, even if nutrients are restored later. Too much food, however, doesn’t increase growth, since microbial metabolism has a limit. In addition, as microbes multiply, they generate heat, raising the reactor temperature. If left unchecked, this excess heat can push temperatures above 105Β°F, halting reproduction altogether. pH is another delicate factor. Growth byproducts are basic, requiring careful acid addition to maintain balance. Both acid and base are adjusted through manual valves, and errors in timing or dosing risk destabilizing the reactor environment. Oxygen also plays a pivotal role; too little dissolved oxygen slows growth, while too much can stress the microbes. The feed rate itself creates trade-offs. If flow is too fast, microbes are flushed out before reproducing. If it’s too slow, they starve. The β€œresidence time”—how long microbes remain in the reactorβ€”must align with their growth cycle. At startup, flows must remain low to allow the microbial population to build up, but later in continuous mode, flow and level setpoints must be carefully tuned to maintain steady enzyme production. The overarching challenge is balancing all of these competing factors: feed temperature, flow rates, pH control, oxygen levels, and microbial growth. Every adjustment has consequences across the system, and missteps at one stage ripple through the process. A failure to manage these interdependencies can lead to stalled growth, contamination, wasted feed, fouled equipment, or drastically reduced enzyme yields. Read a white paper about this use case. #### Nitrogen Manufacturing
Air separation units (ASUs) are large-scale industrial facilities that extract nitrogen, oxygen, and argon from air using energy-intensive cryogenic distillation. They are critical to industries such as steel, chemicals, and electronics, where steady supplies of high-purity gases are required. The process relies heavily on electricity, particularly for compressors and refrigeration cycles, making it highly exposed to fluctuations in electricity prices. For example, electricity tariffs can vary hourly, with moderate swings on some days and extreme peaks on others. Small variations in energy price can drastically affect profitability. The production system itself is complex. Feed air is compressed, cooled through multi-stream heat exchangers, and then separated in a distillation column. Supporting units include turbines, liquefiers, evaporators, and large storage tanks for liquid nitrogen. These storage systems provide flexibility: the plant can liquefy and store nitrogen when power is cheap and later vaporize it to meet demand when electricity is expensive or production must be reduced. However, relying too much on storage introduces risks of tank depletion, overfilling, or excessive refrigeration costs. The ASU must also adhere to strict product quality and safety requirements. The nitrogen stream must maintain impurity levels below 1500 ppm, temperature differences in critical heat exchange equipment must stay above 2 K, and reboiler inventories must remain within physical limits. Any violation could compromise safety or lead to off-spec product. Complicating matters further are external shocks. Customers may unexpectedly reduce demand, for instance, during unplanned maintenance at downstream plants. Conversely, sudden spikes in demand may stretch capacity. The central problem is therefore one of coordinating production, storage, and energy use under uncertainty and fluctuating market conditions. Decisions occur on different time scales: long-term scheduling of production and storage, medium-term adaptation to electricity pricing patterns, and short-term operational control of compressors, turbines, and distillation units. Poorly timed choicesβ€”such as ramping production too late, overusing storage, or failing to anticipate price shiftsβ€”can lead to financial losses, excessive energy costs, or constraint violations. Read a white paper about this use case. #### Production Scheduling
he manager of an industrial bakery oversees a daily operation that produces cakes, cupcakes, and cookies at scale. Each product line follows the same three-step processβ€”mixing, baking, and decoratingβ€”but with different time requirements and costs. For example, a batch of cookies requires only five minutes to mix, 13 minutes to bake, and 10 minutes to decorate, while a batch of cupcakes takes seven minutes to mix, 30 minutes to bake, and 20 minutes to decorate. Cakes take the longest, with 10 minutes of mixing, 40 minutes of baking, and 30 minutes of decorating. The bakery operates with two mixing units, three ovens, and two decorating stations, all of which must be shared across the three product types. Complicating matters, the bakery employs four bakers, each with different skills and assignments. Once a baker begins mixing or decorating, they remain tied to that task until it is finished, and cannot be shifted to another role midstream. This constraint means the manager must constantly anticipate bottlenecks: an idle oven waiting for batter, or a decorating station delayed because cakes are not yet ready. On the financial side, each product has a sharply different cost-to-revenue ratio. A cake costs about $10 to produce but sells for $42, while cupcakes cost $7 per batch to make and sell for $30. Cookies, though quick to prepare, have the widest rangeβ€”$24 per batch in costs but a $60 selling price. The manager must therefore weigh whether to focus on high-margin cookies, high-demand cupcakes, or large-format cakes, knowing that an imbalance can erode profit or leave customer demand unmet. External pressures add to the complexity. Demand fluctuates from day to day. Predictable fluctuations include higher demand for cakes on weekends and during the summer wedding season, for cupcakes on weekdays during the school year, and for cookies during the holiday season. But demand can also vary unpredictably, and ingredient costs can likewise rise without warning. Overproduction risks waste, spoilage, or costly storage, while underproduction may result in late deliveries, penalties, or lost sales. Transportation schedules and customer expectations compound the stakes. Every day brings 480 individual scheduling decisions across an eight-hour shift, each one shaping the bakery’s overall profitability. Read a white paper about this use case. #### Industrial Mixer
In the industrial mixer use case, raw materials are stirred together inside a tank, undergoing a reaction that produces the desired end product. The goal of the process is to convert as much of the raw material as possible. But as the chemicals mix and the conversion occurs, the tank heats up. If the temperature gets too high, a condition called β€œthermal runaway” occurs, potentially causing explosions and fires. To produce as much chemical as possible, the operator must constantly adjust the temperature in the tank, keeping it high enough to allow productivity but low enough to avoid any thermal runaway risk. As in all machine teaching use cases, this process can be summarized in the form of a goal (maximize yield) and a constraint (avoid thermal runaway) that must be balanced against each other: The process is controlled by adjusting the mixture's temperature in the tank using a "jacket" filled with coolant. Lowering the coolant temperature in the jacket lowers the temperature in the tank, decreasing the risk of thermal runaway. However, cooling the tank can also reduce yield. By how much? The answer varies unpredictably – temperature changes affect chemical concentration differently at different parts of the reaction. That nonlinear relationship between temperature and yield is why this is a nuanced process that benefits so much from intelligent automation. Read a white paper about this use case. # 1. Define a Use Case This module is about getting set up for success by clearly and accurately defining the use case you are trying to solve. Once you've defined the use case, you'll be able to identify the expertise you need to teach the system to learn well. ## Learning Objectives At the end of this module, you will be able to: * Identify the knowledge components that are key for defining a use case * Explain actions, sensors, KPI, benchmarks, and ROI * Identify actions, sensors, and KPI for a real use case *** ## Video Lesson {% embed url="" %} *** ## Practice Activity {% tabs %} {% tab title="Instructions" %} Read the production scheduling case study and think about these questions: * What are some individual control actions? * What are some sensor variables you would need to track? * What is the KPI? Then read on for answers.
{% endtab %} {% tab title="Actions" %} Here are some of the action variables from the case study. Do these match the actions you thought of? #### Actions * Baker A mix cakes - action telling Baker A to start mixing cakes * Baker B mix cupcakes - action telling Baker B to start mixing cupcakes * Baker C bake from Mixer 1 - action telling Baker C to bake the product from Mixer 1 * Baker A decorate from Oven 1 - action telling Baker D to decorate the product from Oven 1 * Wait - action telling the system to do nothing for this action step * And so on for every baker, product, and action combination {% endtab %} {% tab title="Sensors" %} Here are some of the sensor variables from the case study. Do these match the sensors you thought of? #### Sensors * Baker A time remaining - sensor reporting how much time is left on Baker A's task * Mixer 1 recipe - sensor reporting which product is currently in Mixer 1 * Mixer 2 time remaining - sensor reporting how much time is left on Mixer 2's recipe * Completed cakes - sensor reporting how many cakes have been bought so far * Cupcake cost to make - sensor reporting the ingredient cost of making a batch of cupcakes * Cookie demand - sensor reporting how many cookies are predicted to be sold * And so on for every baker, equpiment piece, and product {% endtab %} {% tab title="KPI" %} The KPI for this process is profit. Profit is not directly measured, but it can be calculated by combining the measures of cost to produce the products, the number of products produced, and the demand. {% endtab %} {% endtabs %} *** ## Check Your Understanding Read the oil blending use case and answer the following questions. For questions 1-5, characterize each variable as a sensor or an action 1. Amount of crude oil #1 purchased
Check your answer Action. The amount of each crude purchased per day is one of the key decisions the operator or autonomous system needs to make.
2. Yield of naphtha from distillation
Check your answer Sensor. The amount of naphtha that results from distillation is a variable that is determined by how the system is operating.
3. Yield of kerosene from distillation
Check your answer Sensor. Just like naphtha, the amount of naphtha that results from distillation is a variable that is determined by how the system is operating
4. Amount of naphtha sent to reformer
Check your answer Action. While the amount of naphtha produced in the distillation step is a sensor, the amount of that naphtha that is sent into the next step rather than saved as a finished product is a decision.
5. Total naphtha available
Check your answer Sensor. The total amount of naphtha available is an environment property, not a decision.
6. What would you choose as the KPI for the process? 1. Throughput: maximize total product created 2. Waste: minimize unusable product created 3. Profit: maximize revenue minus costs 4. Demand satisfaction: minimize unsold product
Check your answer The answer is C. According to the use case description, the overall goal of the process is to maximize profit.
7. What is the decision frequency for this process? 1. 1 minute 2. 1 hour 3. 1 day 4. 1 month 5. 1 year
Check your answer The answer is C. According to the use case description, the decision about the blend of oils is made daily.
For questions 8-9: Measurements that define the characteristics of the different crude and finished oils (for example, density, sulfur, viscosity, wax, and metal content) are not included as variables in the system. 8. Why is it possible to control this process effectively without that information? 1. These qualities are not important for making decisions about oil blending 2. That information is assumed to be standard for each crude type and product 3. It is not possible to directly measure the physical characteristics of the oil stocks and products 4. This would provide too much information and slow the system down to the point of uselessness
Check your answer The answer is B. These physical properties are very importnt for oil blending, but they are part of the definition of different crude and product types.
9. The problem set-up assumes that physical properties are standard within the crude and product types. If in the real world there is variation within these categories, what challenging phenomenon would that represent? 1. Changing conditions 2. Different scenarios 3. Incomplete information 4. Noise 5. Funnel states
Check your answer The answer is D. Variation within the data that is not meaningful for system control is noise.
# 2. Codify Expertise Expertise is one of the key factors in creating high performing systems that optimize processes and beat benchmarks. In this module, you'll learn how to gather and document that expertise by working directly with subject matter experts. *** ## Learning Objectives By the end of this module, you will be able to: * Conduct an expert interview to gather information used to define a use case and train a team of agents * Record expertise using an AI Specification Document *** ## Video Lesson {% embed url="" %} *** ## Practice Activity {% tabs %} {% tab title="Instructions" %} Read the [enzyme reactor case study](/tutorials/solution-design). You are conducting an SME interview to capture expertise and fill out the AI specification document for this use case. Choose what questions you would ask and follow the directions to navigate to the appropriate tabs. YOU: Could you please describe the use case? --> Go to Tab 1 {% endtab %} {% tab title="1" %} SME: Well, we're making enzymes in a reactor and we have to keep the conditions as perfect as we can to keep the bugs alive. We call the enzyme producing bacteria bugs. And the bugs are pretty delicate. Some of them tend to die or they don't reproduce as fast if we do something wrong. So it's just a lot of keeping an eye on everything and adjusting everything. Choose one of the two options below. YOU: What would you say is the overall goal of the process? --> Go to Tab 2 YOU: So are you normally the one who operates the reactor? --> Go to Tab 3 {% endtab %} {% tab title="2" %} Good job! The overall goal or KPI of the process is one of the most fundamental pieces of information for defining a use case so it is a good idea to ask this early. SME: Well, we want to produce as much enzyme as possible. There's an outflow rate from the tank and we want to maximize that, so it's overall yield. We just want to keep the conditions ideal, you know the temperature and pH and oxygen in the tank and that kind of thing, so that we're getting as much enzyme produced as possible. There are a lot of different things to balance so it's kind of an art. Choose one of the two options below and follow the instructions. YOU: What are the adjustments that the operators control? --> Go to Tab 4 YOU: That sounds difficult, but a team of agents will be able to control it well. --> Go to Tab 5 {% endtab %} {% tab title="3" %} This wasn't the best next question. It's useful to understand how to process is currently controlled, but the most fundamental pieces of information you need at the beginning of the interview are the KPI and the actions, in order to define the use case. SME: It's sometimes me, but not usually because I'm a supervisor now. We have a few different guys who work on this and they all have their own things that they do. There are PID controllers that are doing the actual adjustments, but my guys are setting the set points. Choose one of the two options below and follow the instructions. YOU: What would you say is the overall goal of the process? --> Go to Tab 2 YOU: What are those adjustments that the operators control? --> Go to Tab 4 {% endtab %} {% tab title="4" %} Good job! It's important to understand the control actions as part of fundamentally defining the use case. SME: There are a few actions that all impact the system in different ways, that's part of why it's tricky to control. We have the flow rate into the tank, and then we can blow in oxygen, which also affects the temperature, and we can send in acid or base which changes the pH but also affects the level in the tank. Did I mention that it's a lot to keep track of? Some operators are a lot better than others and we've never been able to really automate this. Choose one of the two options below and follow the instructions. YOU: That sounds difficult, but balancing multiple actions and goals are something that a team of agents can learn to do well. --> Go to Tab 5 {% endtab %} {% tab title="5" %} Congratulations! You have completed the first questions of the SME interview. If you have asked the questions correctly, you will have learned about the KPI and the actions and be able to fill out the portions of the AI specification document that define the use case. If you didn't get these two pieces of information in your interview, review all 5 tabs to see how you could have structured the interview differently. {% endtab %} {% endtabs %} *** ## Check Your Understanding Read the [oil blending use case ](/tutorials/solution-design)and answer the following questions. 1. Which of the following job roles would you interview to gain expert knowledge about the crude oil blending process? Choose all that apply. 1. The engineer who designed the process 2. The operator who manually determines the quantities of oil types each day 3. The financial analyst who predicts the prices and costs of oil types 4. The engineer who maintains the atmospheric distillation unit
Check your answer The answers are A and B. The engineer and the operator have detailed knowledge of how to control the process.
2. What information would be most relevant to designing an AI control system for crude oil blending? 1. The design of the atmospheric distillation unit 2. The chemical composition of different products 3. The different scenarios that require different control strategies 4. The formulas for nonlinear changes in viscosity and stability as crude types are blended
Check your answer The answer is C.
3. Which of the following information types should be included in the Project Objective section of the AI Specification Document? Choose all that apply. 1. The sensor variables that serve as inputs to the system 2. The design of the atmospheric distillation unit 3. A schematic diagram of the oil blending process flow. 4. An explanation of why improvement is needed over the current automation system.
Check your answer The answers are C and D.
4. The KPI variable for the oil blending process is profit. What best represents the objective for optimizing this variable? 1. Maximize 2. Minimize 3. Maintain 4. Avoid
Check your answer The answer is A.
5. Why is profit a better KPI for this problem than throughput? Choose all that apply. 1. Profit uses dollars as units, which is more understandable than measurements of volume 2. Maximizing throughput might lead to creating too much of lower-value products\* 3. Throughput is fixed and deterministic whereas profit is dynamic and can respond to changing market conditions 4. Profit is the ultimate goal of the business
Check your answer The answer is D.
6. Which of these values is necessary in order to determine the ROI of an AI system? 1. Benchmark 2. Throughput 3. Constraint 4. Sensor
Check your answer The answer is A.
# 3. Analyze Current Methods Current methods of solving the problem or controlling the process provide an important window into both the challenges with the use case and the best way to solve it. In this module, you'll learn to identify promising solution elements from the current control methods and match challenges to key design elements. *** ## Learning Objectives By the end of this module, you will be able to: * Analyze current control methods and their strengths and weaknesses * Identify challenges that make the process difficult to control and their implications for solution design *** ## Video Lesson {% embed url="" %} *** ## Practice Activity {% tabs %} {% tab title="Instructions" %}
Read the enzyme reactor case study. In your SME interview, you learned a few new facts about the process. Consider these facts and then answer the questions. Read on to check your answers. Questions: * What are the current methods for controlling the process? * What challenging phenomena is the SME describing? * What solution design elements could help with those phenomena? SME: "We have a few different guys who work on this and they all have their own things that they do. There are PID controllers that are doing the actual adjustments, but my guys are setting the set points." "There are a few actions that all impact the system in different ways, that's part of why it's tricky to control. We have the flow rate into the tank, and then we can blow in oxygen, which also affects the temperature, and we can send in acid or base which changes the pH but also affects the level in the tank. Did I mention that it's a lot to keep track of? Some operators are a lot better than others and we've never been able to really automate this." "We have a bunch of sensors that we're always watching, especially the temperature, pH, oxygen, and the total level in the tank. One thing we don't know is . . . we know how many microbes are in the tank from the level. But some of them are alive and some are dead at any given point and we don't have a way to know that until we test the batch later in the lab." {% endtab %} {% tab title="Current Methods" %} The current method is a set of PID controllers with human operators determining the set points. This is a combination of control theory and human operation. {% endtab %} {% tab title="Challenges" %} Two challenges that are described are: * Changing conditions, as each action and adjustment causes conditions in the tank to change in intended and unintended ways * Dead time, as the proportion of live and dead microbes is only determined later in a lab {% endtab %} {% tab title="Solutions" %} Solution design element based on current control strategy: PID controllers are working well to execute set points. Keep those in the system, but add AI agents to set the set points. Solution design element based on challenges: both changing conditions and dead time are best addressed by learning. Create a team of agents that use deep reinforcement learning to practice controlling through these challenges to optimize the KPI. {% endtab %} {% endtabs %} *** ## Check Your Understanding Read the production scheduling[ use case ](/tutorials/solution-design)and answer the following questions. For questions 1-2: Currently, the bakery is scheduled by an algorithm that prioritizes first cakes, then cupcakes, and then cookies up to a set number of each product based on the day of the week. 1. What control method does this represent? 1. Control theory 2. Optimization 3. Heuristic rules 4. Human operator
Check your answer The answers is C. The rule β€œmake cakes first, then cupcakes, then cookies” is a classic example of a rules-based or heuristic control system. It encodes human expertise into simple if/then logic.
2. What are likely limitations of this control method as a means to maximize profit? 1. It can’t respond to dynamic changes in demand 2. It doesn’t produce equal numbers of each product 3. It ignores the bakers’ preferences and individual skills 4. It trades off against high stakes events
Check your answer The answer is A. Because heuristic rules are fixed, they can’t adjust to shifts in conditions. For example, if demand for cookies suddenly spikes during the holidays, the algorithm will still produce the same proportions based on its preset rules. This lack of adaptability is one of the key weaknesses of rules-based systems: they are effective for stable environments but fail in dynamic, high-variability systems.
3. Which of these represents the challenging phenomenon of different scenarios in the production scheduling use case? 1. b. A mixer that is slower than the others because of a broken part 2. The design of the atmospheric distillation unit 3. The three phases of the production process (mix, bake, decorate) 4. High demand for cookies during the holiday season
Check your answer The answers is D. Different scenarios occur when conditions fundamentally change, requiring the system to operate under different rules or priorities. In this case, seasonal demand spikes represent distinct operating modes. The bakery’s control system must be able to recognize and adapt to these scenarios, a key motivation for orchestration strategies where different agents specialize in different conditions
4. One of the bakers is known to occasionally eat one of the cookies she is decorating, leading to a smaller batch size than expected. Which challenging phenomenon does this represent? 1. Incomplete information 2. Changing conditions 3. Noise 4. High-stakes events
Check your answer The answer is C. Noise refers to random or unpredictable variations in system signals or performance, such as inaccurate sensor readings or inconsistent human actions. The missing cookies create data noise because the actual output doesn’t match the expected result. Systems designed with learning-based control can filter or adapt to noisy input, recognizing that occasional inconsistencies shouldn’t trigger overreactions
5. Which of these would be a high-stakes event in the context of autonomous control of bakery scheduling? 1. A pipe bursts, flooding the production floor 2. A post about their cupcakes goes viral and everyone has to have one 3. A baker discovers a way to speed up the decorating process 4. The cost of an ingredient doubles
Check your answer The answer is B. A high-stakes event is one with large consequences that require immediate and accurate response that differs from standard control strategy. In this example, a viral trend creates a surge in demand that could make or break the day’s profitability. The system must detect and adapt to such events quickly to avoid missed revenue or reputation damage.
6. What design element could be included to address demand spikes as a high-stakes event? 1. A benchmark that puts fluctuations in a broader context 2. A learning algorithm that optimizes for the KPI 3. An orchestration pattern that includes high-stakes events 4. A perception module that makes predictions based on dynamic data
Check your answer The answer is D. Perception modules trained to interpret dynamic data can help address high-stakes events. In this case, a machine learning model could analyze social media activity, online orders, or sales trends to predict demand surges before they happen. The perception agent would then inform decision-making agents to adjust production scheduling accordingly.
7. One of the challenges facing the bakery is that some human schedulers are much more skilled than others. What deployment method would address this by helping the novice operators to upskill? 1. Decision-support 2. Closed-loop autonomous control 3. Optimization algorithms 4. Funnel states
Check your answer The answer is A. Decision-support deployment keeps humans β€œin the loop” rather than fully automating the process. In this setup, the AI system provides recommendations or insights that guide human schedulers, allowing less experienced operators to learn from expert-like feedback over time.
# 4. Format and Upload Data Current methods of solving the problem or controlling the process provide an important window into both the challenges with the use case and the best way to solve it. In this module, you'll learn to identify promising solution elements from the current control methods and match challenges to key design elements. *** ## Learning Objectives By the end of this module, you will be able to: * Analyze current control methods and their strengths and weaknesses * Identify challenges that make the process difficult to control and their implications for solution design *** ## Video Lesson {% embed url="" %} *** ## Practice Activity {% tabs %} {% tab title="Instructions" %}
Read the enzyme reactor case study. In your SME interview, you learned a few new facts about the process. Consider these facts and then answer the questions. Read on to check your answers. Questions: * What are the current methods for controlling the process? * What challenging phenomena is the SME describing? * What solution design elements could help with those phenomena? SME: "We have a few different guys who work on this and they all have their own things that they do. There are PID controllers that are doing the actual adjustments, but my guys are setting the set points." "There are a few actions that all impact the system in different ways, that's part of why it's tricky to control. We have the flow rate into the tank, and then we can blow in oxygen, which also affects the temperature, and we can send in acid or base which changes the pH but also affects the level in the tank. Did I mention that it's a lot to keep track of? Some operators are a lot better than others and we've never been able to really automate this." "We have a bunch of sensors that we're always watching, especially the temperature, pH, oxygen, and the total level in the tank. One thing we don't know is . . . we know how many microbes are in the tank from the level. But some of them are alive and some are dead at any given point and we don't have a way to know that until we test the batch later in the lab." {% endtab %} {% tab title="Current Methods" %} The current method is a set of PID controllers with human operators determining the set points. This is a combination of control theory and human operation. {% endtab %} {% tab title="Challenges" %} Two challenges that are described are: * Changing conditions, as each action and adjustment causes conditions in the tank to change in intended and unintended ways * Dead time, as the proportion of live and dead microbes is only determined later in a lab {% endtab %} {% tab title="Solutions" %} Solution design element based on current control strategy: PID controllers are working well to execute set points. Keep those in the system, but add AI agents to set the set points. Solution design element based on challenges: both changing conditions and dead time are best addressed by learning. Create a team of agents that use deep reinforcement learning to practice controlling through these challenges to optimize the KPI. {% endtab %} {% endtabs %} *** ## Check Your Understanding Read the production scheduling[ use case ](/tutorials/solution-design)and answer the following questions. For questions 1-2: Currently, the bakery is scheduled by an algorithm that prioritizes first cakes, then cupcakes, and then cookies up to a set number of each product based on the day of the week. 1. What control method does this represent? 1. Control theory 2. Optimization 3. Heuristic rules 4. Human operator
Check your answer The answers is C. The rule β€œmake cakes first, then cupcakes, then cookies” is a classic example of a rules-based or heuristic control system. It encodes human expertise into simple if/then logic.
2. What are likely limitations of this control method as a means to maximize profit? 1. It can’t respond to dynamic changes in demand 2. It doesn’t produce equal numbers of each product 3. It ignores the bakers’ preferences and individual skills 4. It trades off against high stakes events
Check your answer The answer is A. Because heuristic rules are fixed, they can’t adjust to shifts in conditions. For example, if demand for cookies suddenly spikes during the holidays, the algorithm will still produce the same proportions based on its preset rules. This lack of adaptability is one of the key weaknesses of rules-based systems: they are effective for stable environments but fail in dynamic, high-variability systems.
3. Which of these represents the challenging phenomenon of different scenarios in the production scheduling use case? 1. b. A mixer that is slower than the others because of a broken part 2. The design of the atmospheric distillation unit 3. The three phases of the production process (mix, bake, decorate) 4. High demand for cookies during the holiday season
Check your answer The answers is D. Different scenarios occur when conditions fundamentally change, requiring the system to operate under different rules or priorities. In this case, seasonal demand spikes represent distinct operating modes. The bakery’s control system must be able to recognize and adapt to these scenarios, a key motivation for orchestration strategies where different agents specialize in different conditions
4. One of the bakers is known to occasionally eat one of the cookies she is decorating, leading to a smaller batch size than expected. Which challenging phenomenon does this represent? 1. Incomplete information 2. Changing conditions 3. Noise 4. High-stakes events
Check your answer The answer is C. Noise refers to random or unpredictable variations in system signals or performance, such as inaccurate sensor readings or inconsistent human actions. The missing cookies create data noise because the actual output doesn’t match the expected result. Systems designed with learning-based control can filter or adapt to noisy input, recognizing that occasional inconsistencies shouldn’t trigger overreactions
5. Which of these would be a high-stakes event in the context of autonomous control of bakery scheduling? 1. A pipe bursts, flooding the production floor 2. A post about their cupcakes goes viral and everyone has to have one 3. A baker discovers a way to speed up the decorating process 4. The cost of an ingredient doubles
Check your answer The answer is B. A high-stakes event is one with large consequences that require immediate and accurate response that differs from standard control strategy. In this example, a viral trend creates a surge in demand that could make or break the day’s profitability. The system must detect and adapt to such events quickly to avoid missed revenue or reputation damage.
6. What design element could be included to address demand spikes as a high-stakes event? 1. A benchmark that puts fluctuations in a broader context 2. A learning algorithm that optimizes for the KPI 3. An orchestration pattern that includes high-stakes events 4. A perception module that makes predictions based on dynamic data
Check your answer The answer is D. Perception modules trained to interpret dynamic data can help address high-stakes events. In this case, a machine learning model could analyze social media activity, online orders, or sales trends to predict demand surges before they happen. The perception agent would then inform decision-making agents to adjust production scheduling accordingly.
7. One of the challenges facing the bakery is that some human schedulers are much more skilled than others. What deployment method would address this by helping the novice operators to upskill? 1. Decision-support 2. Closed-loop autonomous control 3. Optimization algorithms 4. Funnel states
Check your answer The answer is A. Decision-support deployment keeps humans β€œin the loop” rather than fully automating the process. In this setup, the AI system provides recommendations or insights that guide human schedulers, allowing less experienced operators to learn from expert-like feedback over time.
# 5. Define Scenarios and Strategies Current methods of solving the problem or controlling the process provide an important window into both the challenges with the use case and the best way to solve it. In this module, you'll learn to identify promising solution elements from the current control methods and match challenges to key design elements. *** ## Learning Objectives By the end of this module, you will be able to: * Analyze current control methods and their strengths and weaknesses * Identify challenges that make the process difficult to control and their implications for solution design *** ## Video Lesson {% embed url="" %} *** ## Practice Activity {% tabs %} {% tab title="Instructions" %}
Read the enzyme reactor case study. In your SME interview, you learned a few new facts about the process. Consider these facts and then answer the questions. Read on to check your answers. Questions: * What are the current methods for controlling the process? * What challenging phenomena is the SME describing? * What solution design elements could help with those phenomena? SME: "We have a few different guys who work on this and they all have their own things that they do. There are PID controllers that are doing the actual adjustments, but my guys are setting the set points." "There are a few actions that all impact the system in different ways, that's part of why it's tricky to control. We have the flow rate into the tank, and then we can blow in oxygen, which also affects the temperature, and we can send in acid or base which changes the pH but also affects the level in the tank. Did I mention that it's a lot to keep track of? Some operators are a lot better than others and we've never been able to really automate this." "We have a bunch of sensors that we're always watching, especially the temperature, pH, oxygen, and the total level in the tank. One thing we don't know is . . . we know how many microbes are in the tank from the level. But some of them are alive and some are dead at any given point and we don't have a way to know that until we test the batch later in the lab." {% endtab %} {% tab title="Current Methods" %} The current method is a set of PID controllers with human operators determining the set points. This is a combination of control theory and human operation. {% endtab %} {% tab title="Challenges" %} Two challenges that are described are: * Changing conditions, as each action and adjustment causes conditions in the tank to change in intended and unintended ways * Dead time, as the proportion of live and dead microbes is only determined later in a lab {% endtab %} {% tab title="Solutions" %} Solution design element based on current control strategy: PID controllers are working well to execute set points. Keep those in the system, but add AI agents to set the set points. Solution design element based on challenges: both changing conditions and dead time are best addressed by learning. Create a team of agents that use deep reinforcement learning to practice controlling through these challenges to optimize the KPI. {% endtab %} {% endtabs %} *** ## Check Your Understanding Read the production scheduling[ use case ](/tutorials/solution-design)and answer the following questions. For questions 1-2: Currently, the bakery is scheduled by an algorithm that prioritizes first cakes, then cupcakes, and then cookies up to a set number of each product based on the day of the week. 1. What control method does this represent? 1. Control theory 2. Optimization 3. Heuristic rules 4. Human operator
Check your answer The answers is C. The rule β€œmake cakes first, then cupcakes, then cookies” is a classic example of a rules-based or heuristic control system. It encodes human expertise into simple if/then logic.
2. What are likely limitations of this control method as a means to maximize profit? 1. It can’t respond to dynamic changes in demand 2. It doesn’t produce equal numbers of each product 3. It ignores the bakers’ preferences and individual skills 4. It trades off against high stakes events
Check your answer The answer is A. Because heuristic rules are fixed, they can’t adjust to shifts in conditions. For example, if demand for cookies suddenly spikes during the holidays, the algorithm will still produce the same proportions based on its preset rules. This lack of adaptability is one of the key weaknesses of rules-based systems: they are effective for stable environments but fail in dynamic, high-variability systems.
3. Which of these represents the challenging phenomenon of different scenarios in the production scheduling use case? 1. b. A mixer that is slower than the others because of a broken part 2. The design of the atmospheric distillation unit 3. The three phases of the production process (mix, bake, decorate) 4. High demand for cookies during the holiday season
Check your answer The answers is D. Different scenarios occur when conditions fundamentally change, requiring the system to operate under different rules or priorities. In this case, seasonal demand spikes represent distinct operating modes. The bakery’s control system must be able to recognize and adapt to these scenarios, a key motivation for orchestration strategies where different agents specialize in different conditions
4. One of the bakers is known to occasionally eat one of the cookies she is decorating, leading to a smaller batch size than expected. Which challenging phenomenon does this represent? 1. Incomplete information 2. Changing conditions 3. Noise 4. High-stakes events
Check your answer The answer is C. Noise refers to random or unpredictable variations in system signals or performance, such as inaccurate sensor readings or inconsistent human actions. The missing cookies create data noise because the actual output doesn’t match the expected result. Systems designed with learning-based control can filter or adapt to noisy input, recognizing that occasional inconsistencies shouldn’t trigger overreactions
5. Which of these would be a high-stakes event in the context of autonomous control of bakery scheduling? 1. A pipe bursts, flooding the production floor 2. A post about their cupcakes goes viral and everyone has to have one 3. A baker discovers a way to speed up the decorating process 4. The cost of an ingredient doubles
Check your answer The answer is B. A high-stakes event is one with large consequences that require immediate and accurate response that differs from standard control strategy. In this example, a viral trend creates a surge in demand that could make or break the day’s profitability. The system must detect and adapt to such events quickly to avoid missed revenue or reputation damage.
6. What design element could be included to address demand spikes as a high-stakes event? 1. A benchmark that puts fluctuations in a broader context 2. A learning algorithm that optimizes for the KPI 3. An orchestration pattern that includes high-stakes events 4. A perception module that makes predictions based on dynamic data
Check your answer The answer is D. Perception modules trained to interpret dynamic data can help address high-stakes events. In this case, a machine learning model could analyze social media activity, online orders, or sales trends to predict demand surges before they happen. The perception agent would then inform decision-making agents to adjust production scheduling accordingly.
7. One of the challenges facing the bakery is that some human schedulers are much more skilled than others. What deployment method would address this by helping the novice operators to upskill? 1. Decision-support 2. Closed-loop autonomous control 3. Optimization algorithms 4. Funnel states
Check your answer The answer is A. Decision-support deployment keeps humans β€œin the loop” rather than fully automating the process. In this setup, the AI system provides recommendations or insights that guide human schedulers, allowing less experienced operators to learn from expert-like feedback over time.
# 6. Configure Agents Current methods of solving the problem or controlling the process provide an important window into both the challenges with the use case and the best way to solve it. In this module, you'll learn to identify promising solution elements from the current control methods and match challenges to key design elements. *** ## Learning Objectives By the end of this module, you will be able to: * Analyze current control methods and their strengths and weaknesses * Identify challenges that make the process difficult to control and their implications for solution design *** ## Video Lesson {% embed url="" %} *** ## Practice Activity {% tabs %} {% tab title="Instructions" %}
Read the enzyme reactor case study. In your SME interview, you learned a few new facts about the process. Consider these facts and then answer the questions. Read on to check your answers. Questions: * What are the current methods for controlling the process? * What challenging phenomena is the SME describing? * What solution design elements could help with those phenomena? SME: "We have a few different guys who work on this and they all have their own things that they do. There are PID controllers that are doing the actual adjustments, but my guys are setting the set points." "There are a few actions that all impact the system in different ways, that's part of why it's tricky to control. We have the flow rate into the tank, and then we can blow in oxygen, which also affects the temperature, and we can send in acid or base which changes the pH but also affects the level in the tank. Did I mention that it's a lot to keep track of? Some operators are a lot better than others and we've never been able to really automate this." "We have a bunch of sensors that we're always watching, especially the temperature, pH, oxygen, and the total level in the tank. One thing we don't know is . . . we know how many microbes are in the tank from the level. But some of them are alive and some are dead at any given point and we don't have a way to know that until we test the batch later in the lab." {% endtab %} {% tab title="Current Methods" %} The current method is a set of PID controllers with human operators determining the set points. This is a combination of control theory and human operation. {% endtab %} {% tab title="Challenges" %} Two challenges that are described are: * Changing conditions, as each action and adjustment causes conditions in the tank to change in intended and unintended ways * Dead time, as the proportion of live and dead microbes is only determined later in a lab {% endtab %} {% tab title="Solutions" %} Solution design element based on current control strategy: PID controllers are working well to execute set points. Keep those in the system, but add AI agents to set the set points. Solution design element based on challenges: both changing conditions and dead time are best addressed by learning. Create a team of agents that use deep reinforcement learning to practice controlling through these challenges to optimize the KPI. {% endtab %} {% endtabs %} *** ## Check Your Understanding Read the production scheduling[ use case ](/tutorials/solution-design)and answer the following questions. For questions 1-2: Currently, the bakery is scheduled by an algorithm that prioritizes first cakes, then cupcakes, and then cookies up to a set number of each product based on the day of the week. 1. What control method does this represent? 1. Control theory 2. Optimization 3. Heuristic rules 4. Human operator
Check your answer The answers is C. The rule β€œmake cakes first, then cupcakes, then cookies” is a classic example of a rules-based or heuristic control system. It encodes human expertise into simple if/then logic.
2. What are likely limitations of this control method as a means to maximize profit? 1. It can’t respond to dynamic changes in demand 2. It doesn’t produce equal numbers of each product 3. It ignores the bakers’ preferences and individual skills 4. It trades off against high stakes events
Check your answer The answer is A. Because heuristic rules are fixed, they can’t adjust to shifts in conditions. For example, if demand for cookies suddenly spikes during the holidays, the algorithm will still produce the same proportions based on its preset rules. This lack of adaptability is one of the key weaknesses of rules-based systems: they are effective for stable environments but fail in dynamic, high-variability systems.
3. Which of these represents the challenging phenomenon of different scenarios in the production scheduling use case? 1. b. A mixer that is slower than the others because of a broken part 2. The design of the atmospheric distillation unit 3. The three phases of the production process (mix, bake, decorate) 4. High demand for cookies during the holiday season
Check your answer The answers is D. Different scenarios occur when conditions fundamentally change, requiring the system to operate under different rules or priorities. In this case, seasonal demand spikes represent distinct operating modes. The bakery’s control system must be able to recognize and adapt to these scenarios, a key motivation for orchestration strategies where different agents specialize in different conditions
4. One of the bakers is known to occasionally eat one of the cookies she is decorating, leading to a smaller batch size than expected. Which challenging phenomenon does this represent? 1. Incomplete information 2. Changing conditions 3. Noise 4. High-stakes events
Check your answer The answer is C. Noise refers to random or unpredictable variations in system signals or performance, such as inaccurate sensor readings or inconsistent human actions. The missing cookies create data noise because the actual output doesn’t match the expected result. Systems designed with learning-based control can filter or adapt to noisy input, recognizing that occasional inconsistencies shouldn’t trigger overreactions
5. Which of these would be a high-stakes event in the context of autonomous control of bakery scheduling? 1. A pipe bursts, flooding the production floor 2. A post about their cupcakes goes viral and everyone has to have one 3. A baker discovers a way to speed up the decorating process 4. The cost of an ingredient doubles
Check your answer The answer is B. A high-stakes event is one with large consequences that require immediate and accurate response that differs from standard control strategy. In this example, a viral trend creates a surge in demand that could make or break the day’s profitability. The system must detect and adapt to such events quickly to avoid missed revenue or reputation damage.
6. What design element could be included to address demand spikes as a high-stakes event? 1. A benchmark that puts fluctuations in a broader context 2. A learning algorithm that optimizes for the KPI 3. An orchestration pattern that includes high-stakes events 4. A perception module that makes predictions based on dynamic data
Check your answer The answer is D. Perception modules trained to interpret dynamic data can help address high-stakes events. In this case, a machine learning model could analyze social media activity, online orders, or sales trends to predict demand surges before they happen. The perception agent would then inform decision-making agents to adjust production scheduling accordingly.
7. One of the challenges facing the bakery is that some human schedulers are much more skilled than others. What deployment method would address this by helping the novice operators to upskill? 1. Decision-support 2. Closed-loop autonomous control 3. Optimization algorithms 4. Funnel states
Check your answer The answer is A. Decision-support deployment keeps humans β€œin the loop” rather than fully automating the process. In this setup, the AI system provides recommendations or insights that guide human schedulers, allowing less experienced operators to learn from expert-like feedback over time.
# 7. Orchestrate Agents Current methods of solving the problem or controlling the process provide an important window into both the challenges with the use case and the best way to solve it. In this module, you'll learn to identify promising solution elements from the current control methods and match challenges to key design elements. *** ## Learning Objectives By the end of this module, you will be able to: * Analyze current control methods and their strengths and weaknesses * Identify challenges that make the process difficult to control and their implications for solution design *** ## Video Lesson {% embed url="" %} *** ## Practice Activity {% tabs %} {% tab title="Instructions" %}
Read the enzyme reactor case study. In your SME interview, you learned a few new facts about the process. Consider these facts and then answer the questions. Read on to check your answers. Questions: * What are the current methods for controlling the process? * What challenging phenomena is the SME describing? * What solution design elements could help with those phenomena? SME: "We have a few different guys who work on this and they all have their own things that they do. There are PID controllers that are doing the actual adjustments, but my guys are setting the set points." "There are a few actions that all impact the system in different ways, that's part of why it's tricky to control. We have the flow rate into the tank, and then we can blow in oxygen, which also affects the temperature, and we can send in acid or base which changes the pH but also affects the level in the tank. Did I mention that it's a lot to keep track of? Some operators are a lot better than others and we've never been able to really automate this." "We have a bunch of sensors that we're always watching, especially the temperature, pH, oxygen, and the total level in the tank. One thing we don't know is . . . we know how many microbes are in the tank from the level. But some of them are alive and some are dead at any given point and we don't have a way to know that until we test the batch later in the lab." {% endtab %} {% tab title="Current Methods" %} The current method is a set of PID controllers with human operators determining the set points. This is a combination of control theory and human operation. {% endtab %} {% tab title="Challenges" %} Two challenges that are described are: * Changing conditions, as each action and adjustment causes conditions in the tank to change in intended and unintended ways * Dead time, as the proportion of live and dead microbes is only determined later in a lab {% endtab %} {% tab title="Solutions" %} Solution design element based on current control strategy: PID controllers are working well to execute set points. Keep those in the system, but add AI agents to set the set points. Solution design element based on challenges: both changing conditions and dead time are best addressed by learning. Create a team of agents that use deep reinforcement learning to practice controlling through these challenges to optimize the KPI. {% endtab %} {% endtabs %} *** ## Check Your Understanding Read the production scheduling[ use case ](/tutorials/solution-design)and answer the following questions. For questions 1-2: Currently, the bakery is scheduled by an algorithm that prioritizes first cakes, then cupcakes, and then cookies up to a set number of each product based on the day of the week. 1. What control method does this represent? 1. Control theory 2. Optimization 3. Heuristic rules 4. Human operator
Check your answer The answers is C. The rule β€œmake cakes first, then cupcakes, then cookies” is a classic example of a rules-based or heuristic control system. It encodes human expertise into simple if/then logic.
2. What are likely limitations of this control method as a means to maximize profit? 1. It can’t respond to dynamic changes in demand 2. It doesn’t produce equal numbers of each product 3. It ignores the bakers’ preferences and individual skills 4. It trades off against high stakes events
Check your answer The answer is A. Because heuristic rules are fixed, they can’t adjust to shifts in conditions. For example, if demand for cookies suddenly spikes during the holidays, the algorithm will still produce the same proportions based on its preset rules. This lack of adaptability is one of the key weaknesses of rules-based systems: they are effective for stable environments but fail in dynamic, high-variability systems.
3. Which of these represents the challenging phenomenon of different scenarios in the production scheduling use case? 1. b. A mixer that is slower than the others because of a broken part 2. The design of the atmospheric distillation unit 3. The three phases of the production process (mix, bake, decorate) 4. High demand for cookies during the holiday season
Check your answer The answers is D. Different scenarios occur when conditions fundamentally change, requiring the system to operate under different rules or priorities. In this case, seasonal demand spikes represent distinct operating modes. The bakery’s control system must be able to recognize and adapt to these scenarios, a key motivation for orchestration strategies where different agents specialize in different conditions
4. One of the bakers is known to occasionally eat one of the cookies she is decorating, leading to a smaller batch size than expected. Which challenging phenomenon does this represent? 1. Incomplete information 2. Changing conditions 3. Noise 4. High-stakes events
Check your answer The answer is C. Noise refers to random or unpredictable variations in system signals or performance, such as inaccurate sensor readings or inconsistent human actions. The missing cookies create data noise because the actual output doesn’t match the expected result. Systems designed with learning-based control can filter or adapt to noisy input, recognizing that occasional inconsistencies shouldn’t trigger overreactions
5. Which of these would be a high-stakes event in the context of autonomous control of bakery scheduling? 1. A pipe bursts, flooding the production floor 2. A post about their cupcakes goes viral and everyone has to have one 3. A baker discovers a way to speed up the decorating process 4. The cost of an ingredient doubles
Check your answer The answer is B. A high-stakes event is one with large consequences that require immediate and accurate response that differs from standard control strategy. In this example, a viral trend creates a surge in demand that could make or break the day’s profitability. The system must detect and adapt to such events quickly to avoid missed revenue or reputation damage.
6. What design element could be included to address demand spikes as a high-stakes event? 1. A benchmark that puts fluctuations in a broader context 2. A learning algorithm that optimizes for the KPI 3. An orchestration pattern that includes high-stakes events 4. A perception module that makes predictions based on dynamic data
Check your answer The answer is D. Perception modules trained to interpret dynamic data can help address high-stakes events. In this case, a machine learning model could analyze social media activity, online orders, or sales trends to predict demand surges before they happen. The perception agent would then inform decision-making agents to adjust production scheduling accordingly.
7. One of the challenges facing the bakery is that some human schedulers are much more skilled than others. What deployment method would address this by helping the novice operators to upskill? 1. Decision-support 2. Closed-loop autonomous control 3. Optimization algorithms 4. Funnel states
Check your answer The answer is A. Decision-support deployment keeps humans β€œin the loop” rather than fully automating the process. In this setup, the AI system provides recommendations or insights that guide human schedulers, allowing less experienced operators to learn from expert-like feedback over time.
# 8. Select Technologies Current methods of solving the problem or controlling the process provide an important window into both the challenges with the use case and the best way to solve it. In this module, you'll learn to identify promising solution elements from the current control methods and match challenges to key design elements. *** ## Learning Objectives By the end of this module, you will be able to: * Analyze current control methods and their strengths and weaknesses * Identify challenges that make the process difficult to control and their implications for solution design *** ## Video Lesson {% embed url="" %} *** ## Practice Activity {% tabs %} {% tab title="Instructions" %}
Read the enzyme reactor case study. In your SME interview, you learned a few new facts about the process. Consider these facts and then answer the questions. Read on to check your answers. Questions: * What are the current methods for controlling the process? * What challenging phenomena is the SME describing? * What solution design elements could help with those phenomena? SME: "We have a few different guys who work on this and they all have their own things that they do. There are PID controllers that are doing the actual adjustments, but my guys are setting the set points." "There are a few actions that all impact the system in different ways, that's part of why it's tricky to control. We have the flow rate into the tank, and then we can blow in oxygen, which also affects the temperature, and we can send in acid or base which changes the pH but also affects the level in the tank. Did I mention that it's a lot to keep track of? Some operators are a lot better than others and we've never been able to really automate this." "We have a bunch of sensors that we're always watching, especially the temperature, pH, oxygen, and the total level in the tank. One thing we don't know is . . . we know how many microbes are in the tank from the level. But some of them are alive and some are dead at any given point and we don't have a way to know that until we test the batch later in the lab." {% endtab %} {% tab title="Current Methods" %} The current method is a set of PID controllers with human operators determining the set points. This is a combination of control theory and human operation. {% endtab %} {% tab title="Challenges" %} Two challenges that are described are: * Changing conditions, as each action and adjustment causes conditions in the tank to change in intended and unintended ways * Dead time, as the proportion of live and dead microbes is only determined later in a lab {% endtab %} {% tab title="Solutions" %} Solution design element based on current control strategy: PID controllers are working well to execute set points. Keep those in the system, but add AI agents to set the set points. Solution design element based on challenges: both changing conditions and dead time are best addressed by learning. Create a team of agents that use deep reinforcement learning to practice controlling through these challenges to optimize the KPI. {% endtab %} {% endtabs %} *** ## Check Your Understanding Read the production scheduling[ use case ](/tutorials/solution-design)and answer the following questions. For questions 1-2: Currently, the bakery is scheduled by an algorithm that prioritizes first cakes, then cupcakes, and then cookies up to a set number of each product based on the day of the week. 1. What control method does this represent? 1. Control theory 2. Optimization 3. Heuristic rules 4. Human operator
Check your answer The answers is C. The rule β€œmake cakes first, then cupcakes, then cookies” is a classic example of a rules-based or heuristic control system. It encodes human expertise into simple if/then logic.
2. What are likely limitations of this control method as a means to maximize profit? 1. It can’t respond to dynamic changes in demand 2. It doesn’t produce equal numbers of each product 3. It ignores the bakers’ preferences and individual skills 4. It trades off against high stakes events
Check your answer The answer is A. Because heuristic rules are fixed, they can’t adjust to shifts in conditions. For example, if demand for cookies suddenly spikes during the holidays, the algorithm will still produce the same proportions based on its preset rules. This lack of adaptability is one of the key weaknesses of rules-based systems: they are effective for stable environments but fail in dynamic, high-variability systems.
3. Which of these represents the challenging phenomenon of different scenarios in the production scheduling use case? 1. b. A mixer that is slower than the others because of a broken part 2. The design of the atmospheric distillation unit 3. The three phases of the production process (mix, bake, decorate) 4. High demand for cookies during the holiday season
Check your answer The answers is D. Different scenarios occur when conditions fundamentally change, requiring the system to operate under different rules or priorities. In this case, seasonal demand spikes represent distinct operating modes. The bakery’s control system must be able to recognize and adapt to these scenarios, a key motivation for orchestration strategies where different agents specialize in different conditions
4. One of the bakers is known to occasionally eat one of the cookies she is decorating, leading to a smaller batch size than expected. Which challenging phenomenon does this represent? 1. Incomplete information 2. Changing conditions 3. Noise 4. High-stakes events
Check your answer The answer is C. Noise refers to random or unpredictable variations in system signals or performance, such as inaccurate sensor readings or inconsistent human actions. The missing cookies create data noise because the actual output doesn’t match the expected result. Systems designed with learning-based control can filter or adapt to noisy input, recognizing that occasional inconsistencies shouldn’t trigger overreactions
5. Which of these would be a high-stakes event in the context of autonomous control of bakery scheduling? 1. A pipe bursts, flooding the production floor 2. A post about their cupcakes goes viral and everyone has to have one 3. A baker discovers a way to speed up the decorating process 4. The cost of an ingredient doubles
Check your answer The answer is B. A high-stakes event is one with large consequences that require immediate and accurate response that differs from standard control strategy. In this example, a viral trend creates a surge in demand that could make or break the day’s profitability. The system must detect and adapt to such events quickly to avoid missed revenue or reputation damage.
6. What design element could be included to address demand spikes as a high-stakes event? 1. A benchmark that puts fluctuations in a broader context 2. A learning algorithm that optimizes for the KPI 3. An orchestration pattern that includes high-stakes events 4. A perception module that makes predictions based on dynamic data
Check your answer The answer is D. Perception modules trained to interpret dynamic data can help address high-stakes events. In this case, a machine learning model could analyze social media activity, online orders, or sales trends to predict demand surges before they happen. The perception agent would then inform decision-making agents to adjust production scheduling accordingly.
7. One of the challenges facing the bakery is that some human schedulers are much more skilled than others. What deployment method would address this by helping the novice operators to upskill? 1. Decision-support 2. Closed-loop autonomous control 3. Optimization algorithms 4. Funnel states
Check your answer The answer is A. Decision-support deployment keeps humans β€œin the loop” rather than fully automating the process. In this setup, the AI system provides recommendations or insights that guide human schedulers, allowing less experienced operators to learn from expert-like feedback over time.
# Simulation Overview A simulation is the part of the AMESA agent ecosystem that models the real world. It tells the agent system *what happens when it takes an action*, whether based on historical data, physics, business logic, or other dynamics. The simulator enables agents to learn how to make decisions safely, repeatedly, and at scale. AMESA supports three types of simulations, giving you flexibility depending on your use case and available infrastructure. *** ### 🧠 Simulation Types #### 1. Data-Driven Simulations (No-Code Option) Use historical operational data to automatically generate a simulation in the AMESA UI. * No coding required, just upload a properly formatted CSV * Uses AMESA’s Training-as-a-Service * Ideal for industrial and logistics workflows where historical data reflects real-world constraints *** #### 2. Containerized Simulations Using AMESA’s Training Service Bring your simulation, built in any language or framework, and connect it to AMESA via Docker. * You define the simulation logic * AMESA manages the training cluster and runtime * Ideal for teams with existing physics-based or black-box models This option provides you with complete flexibility while still leveraging AMESA’s infrastructure and tools. *** #### 3. Containerized Simulations Using Your Own Azure Training Cluster For enterprises managing their cloud infrastructure, you can run your simulation and training in your own Azure tenant. * Simulation runs in a Docker container, just like above * AMESA orchestrates training via your own Azure compute resources * Gives you control over scaling, security, and cost management *** ### Explore AMESA’s Example Simulators To understand how a simulator works inside the AMESA platform, you can explore our public Python [simulators hosted on Docker Hub](https://hub.docker.com/u/composabl). These simple environments are perfect for: * Demos and POCs * Self-guided learning * Team education and experimentation Use the CLI to Get Started To list available simulators: ```bash composabl sim list ``` To connect to one of the simulators: ```bash composabl sim run ``` ## Simulation Help ### Simulation Help If you already have a simulator, the documentation will guide you through the process of connecting it to AMESA, whether you’re using our managed training or your infrastructure. If you don’t yet have a simulator, you can: * Start with a data-driven simulation * Explore our example simulations * Or work with a AMESA partner to develop one based on your system [Contact us](mailto:info@composabl.com) for more information about finding a simulation partner. # Data Driven Simulation ## Data-Driven Simulations AMESA enables you to build simulations from historical data. These simulations are used to train and evaluate multi-agent systems in high-fidelity environments that reflect your actual operations. Use cases include modeling equipment behavior, logistics workflows, and process automation. ### Recommended Data Range * **Minimum**: 3 months * **Preferred**: 1 year (especially for systems with seasonal variation) Datasets will likely yield a successful simulation when they reflect a >80% accurate relationship between control actions (operator adjustments) and sensor readings. *** ### 1. CSV Data Format Upload a CSV file where each column follows a naming convention: | Variable Type | Prefix | Example | | ------------- | ------------- | ----------------------- | | Sensor | `s_` | `s_T` (temperature) | | Action | `a_` | `a_dTc` (cooling delta) | | Other Input | *(no prefix)* | `Datetime` | **Sensors** are variables that provide information about the environment or conditions within the process. These may be reported by the machine being controlled or they may come from outside systems. Quality measures such as the results of lab tests are also sensors. **Actions** are variables that describe the adjustments the operator makes to the system controls. > Optionally include a **timestamp** column in UTC. If omitted, AMESA assumes the rows are sequential time steps. > > Not all fields need values for every row. The simulator recognizes when data is collected at different intervals. #### Units Row (Optional) Include a row above your column headers to define units for each variable: ```csv kmol/m3,K,K,,kmol/m3,K s_Ca,s_T,s_Tc,s_Tref,s_Cref,a_dTc …data rows… ``` ### 2. Data Configuration After uploading your CSV file, AMESA guides you through a configuration screen where you confirm the role and type of each column: | **Field** | **Description** | | ------------------ | --------------------------------------------- | | User variable name | Rename your variables for clarity (optional) | | Mode | Sensor / Action / None (for reference values) | | Type | Box or continuous variables | | Value Range | Auto-calculated from your data | | Unit (optional) | Populated from the units row | *** ### 3. Simulation Creation Flow #### πŸ›  Step 1: Upload CSV * Navigate to the Data-Driven Simulator section. * Name your simulation. * Upload your CSV file in the proper format.
#### 🧭 Step 2: Configure Variables * Review detected variables. * Adjust mode, type, unit, and value range if needed.
#### πŸ“Š Step 3: Review Data Score AMESA will scan your data and generate a Data Score from 0 to 100: | **Score Range** | **Meaning** | | --------------- | --------------------------------------------- | | 80–100 | Good quality – Ready to simulate | | Below 80 | Needs improvement – Add data or clean inputs. |
#### πŸš€ Step 4: Create Simulation Click Next, and your simulation will be created. It will now be available in your workspace for training, testing, and analysis.
*** ### 4. Best Practices * Use consistent naming: Prefix sensors and actions properly. * Remove special characters and spaces from variable names. * Include a units row: This improves interpretability and ensures correctness. * Each row should be one decision step. * Handle missing data: Missing values compromise model accuracy. Remove or fill null values with the technique of your choice (forward-fill, backfill, interpolation, etc.). * Remove outliers: Outliers caused by sensor malfunction or other issues are a common cause of low simulation scores. Use subject matter expertise to determine when outliers are not valid data and remove those values from the dataset. * Choose clean batches: For some processes, the best simulation will result from a limited number of batches that are representative of the process. * Timestamps (optional): Include UTC timestamps if events need temporal alignment. *** ### 5. Example File πŸ“ Download sample data: {% file src="/files/MqSFgSIT6ecuOgSqDUj4" %} ple.csv Use this example file, which contains data, to create a [Chemical Process](/tutorials/industrial-mixer) Simulation and see the expected structure, naming conventions, and units in action. *** ### 6. Troubleshooting | **Problem** | **Cause** | **Fix** | | --------------------- | ------------------------------- | ----------------------------------------------- | | Variable not detected | Missing prefix | Use s\_ for sensors and a\_ for actions | | Low data score | Not enough data, missing values | Add more historical data or fill in key columns | | Unit not recognized | Unit row missing or misaligned | Add units row directly above the variable row | # Connect a Simulator to AMESA In this tutorial, we will learn how to upload simulators via the AMESA CLI as well as your custom, adapted simulators to the AMESA UI. ### Upload simulators via AMESA CLI To upload simulators that already follows the AMESA simulation specification all you need to do is the fallowing command from the simulation folder: ```bash composabl sim publish ``` * After that, you can go to the AMESAe editor and connect that sim to any project. ## Upload Third-Party Simulators via Docker ### Prerequisites If your simulator isn't already compatible with the AMESA platofrm you will need to create gRPC bindings and upload it as a Docker image in the AMESA editor. You can follow along with these AMESA API patterns and the following structure: ``` . └── your-simulator-folder/ β”œβ”€β”€ docker/ β”‚ └── entrypoint.sh β”œβ”€β”€ src/ β”‚ β”œβ”€β”€ exceptions/ β”‚ β”‚ └── invalid_usage.py β”‚ β”œβ”€β”€ __init__.py β”‚ β”œβ”€β”€ main.py β”‚ β”œβ”€β”€ server_impl.py β”‚ └── sim.py β”œβ”€β”€ Dockerfile └── requirements.txt ``` Going through the files: * `docker/entrypoint.sh`: This file is the entrypoint of your Docker container. * `src/exceptions/invalid_usage.py`: This file contains the exception class `InvalidUsage` that is used to raise exceptions in the simulator. * `src/__init__.py`: This file is the initialization file of the module. No code is needed here, but for Python to recognize the folder as a module, this file is necessary. * `src/main.py`: This file is the main file of the simulator. It uses the `composabl_core.networking` module to expose the simulator to the AMESA platform. This file is also available zipped along with this tutorial. * `src/server_impl.py`: This file contains the implementation of the server that will be used to run the simulator. * `src/sim.py`: This file contains your implementation of the simulator itself. Usually, a `Env` (inheriting from `gym.Env`) class is implemented here, and it is used to run the simulator. * `Dockerfile`: This file is the Dockerfile that will be used to build the Docker image of your simulator. * `requirements.txt`: This file contains the Python packages that are necessary to run your simulator. It is used to install the necessary packages in the Docker image. #### gRPC Spec for Implementation Under the hood, the AMESA SDK uses gRPC to communicate with the AMESA platform. To create a simulator that works with the AMESA API, you have to implement the following gRPC methods: ```protobuf service AMESA { // Creates the environment with specified configurations. // - MakeRequest: Contains parameters to configure the environment. // - MakeResponse: Returns an identifier for the created environment and possibly other initial setup information. rpc make(MakeRequest) returns (MakeResponse) {} // Advances the environment by one timestep using the action provided in the request. // - StepRequest: Includes the action to be taken in the current state of the environment. // - StepResponse: Returns the new state of the environment, reward received, and a flag indicating if the episode has ended. rpc step(StepRequest) returns (StepResponse) {} // Resets the state of the environment, returning it to its initial conditions. // - ResetRequest: May include parameters for resetting to specific scenarios. // - ResetResponse: Provides the initial observation of the reset environment. rpc reset(ResetRequest) returns (ResetResponse) {} // Performs any necessary cleanup before the environment is closed. // - CloseRequest: May be empty or include specific closing instructions. // - CloseResponse: Acknowledges the environment has been successfully closed. rpc close(CloseRequest) returns (CloseResponse) {} // Generates a sample action from the environment's action space. // - ActionSpaceSampleRequest: May be empty or specify particular sampling criteria. // - ActionSpaceSampleResponse: Provides a sample action from the action space. rpc action_space_sample(ActionSpaceSampleRequest) returns (ActionSpaceSampleResponse) {} // Retrieves information about the environment's action space. // - ActionSpaceInfoRequest: May be empty or include parameters for the information request. // - ActionSpaceInfoResponse: Returns detailed information about the action space. rpc action_space_info(ActionSpaceInfoRequest) returns (ActionSpaceInfoResponse) {} // Retrieves information about the environment's observation space. // - ObservationSpaceInfoRequest: May be empty or include parameters for the information request. // - ObservationSpaceInfoResponse: Returns detailed information about the observation space. rpc observation_space_info(ObservationSpaceInfoRequest) returns (ObservationSpaceInfoResponse) {} // Sets the current scenario for the environment. // - SetScenarioRequest: Includes parameters defining the scenario to set. // - SetScenarioResponse: Acknowledges the scenario has been set. rpc set_scenario(SetScenarioRequest) returns (SetScenarioResponse) {} // Retrieves the current scenario of the environment. // - GetScenarioRequest: May be empty if simply retrieving the current scenario. // - GetScenarioResponse: Returns details of the current scenario. rpc get_scenario(GetScenarioRequest) returns (GetScenarioResponse) {} // Sets the render mode of the environment. // - SetRenderModeRequest: Includes parameters for the desired render mode. // - SetRenderModeResponse: Confirms the render mode has been set. rpc set_render_mode(SetRenderModeRequest) returns (SetRenderModeResponse) {} // Retrieves the current render mode of the environment. // - GetRenderModeRequest: May be empty if simply querying the current mode. // - GetRenderModeResponse: Returns the current render mode. rpc get_render_mode(GetRenderModeRequest) returns (GetRenderModeResponse) {} // Retrieves the current render of the environment. // - GetRenderRequest: May include parameters specifying the render details. // - GetRenderResponse: Provides the current render of the environment. rpc get_render(GetRenderRequest) returns (GetRenderResponse) {} } ``` After making sure that your simulator is compatible with the AMESA platform, you can proceed to the next section. #### Docker *** * Go to the folder where your simulator is located. We can navigate to the simulator folder and see what is inside it. * After that, ensure that you have Docker installed. You can check if Docker is installed by running the following command: ```bash docker --version ``` If Docker is installed, you should see the Docker version. If not, you can install Docker by following the instructions on the [official Docker website](https://docs.docker.com/get-docker/). * Then, before building the image, ensure that you have a Docker Hub account. If you don't have one, you can create one by going to the [Docker Hub website](https://hub.docker.com/). #### DockerHub *** * To log in to Docker Hub, run the following command: ```bash docker login ``` You will be prompted to enter your Docker Hub username and password. After that, you should see a message saying that you are logged in. * Now, we can build the Docker image of the simulator. To do so, run the following command: ```bash docker build -t / . ``` This command will build the Docker image of the simulator. The `-t` flag is used to tag the image with the name `/`. The `.` at the end of the command indicates that the Dockerfile is in the current directory. * After building the image, you can check if it was built successfully by running the following command: ```bash docker images ``` And then push the image to Docker Hub: ```bash docker push / ``` #### AMESA UI *** After that, you can go to the AMESA UI and upload your simulator. To do so, follow the steps below: * Go to the AMESA UI by accessing the following link: . You'll be asked to login and then redirected to your dashboard page. * Then, on the left sidebar, click on the "Simulators" tab. You should see a list of simulators that are already available on the platform. On the top right corner, you should see a button to "New Simulator". Click on it. * A pop-up will appear, asking you to select between "Internal" and "External" simulators. Select "External". * Then, you can fill the Title and Description of the simulator. We suggest smaller names for the Title and a more detailed description for the Description. * After clicking in next step, a brief tutorial will open up. Take care to read it and then click on "Next Step" again. * After that, you can fill the Docker Image field with the name of the Docker image you pushed to Docker Hub `/`. If the image is public, no more fields are needed. If the image is private, you need to fill the Docker Username and Docker Password fields with your Docker Hub username and password, respectively. Then, click on "Validate and next step". ### Conclusion If you've followed all the steps correctly, you should have successfully uploaded your simulator to the AMESA UI. You can now use your simulator to train agents and run simulations on the platform. If you have any questions or need help, feel free to reach out to us. # AMESA Simulation API You can integrate your simulator with the AMESA SDK by using the ServerAMESA class. This class provides the necessary methods for the simulator to interact with the AMESA SDK. The methods of the ServerAMESA class allow the AMESA SDK to automatically take care of serializing and deserializing the different requests and responses. To conform your simulator to the AMESA SDK, you must define a server implementation class that defines methods of how to talk with the AMESA SDK. AMESA’s simulation API extends the [the gymnasium.Env standards.](https://gymnasium.farama.org/api/env/) ## Set Up the Simulation Environment Instance #### Make Make is a request to create a new instance of the environment with the specifications requested. * `string env_Id;` Identifier for the type of environment to create. * `dictionary env_Init;` Initial configuration for the environment, as defined within the runtime configuration (link to section about how to define runtime configuration parameters) ```python async def make(self, env_id: str, env_init: dict): self.env_id = env_id if env_id else self.env_id self.env_init = env_init if env_init else self.env_init self.env = Sim(self.env_init) return { "id": "my_simulator", } ``` #### `Sensor_space_info` `Sensor_space_info` provides details about the environment’s sensor space. ```python async def sensor_space_info(self) -> gym.Space: return self.env.sensor_space ``` #### `Action_space_info` `Action_space_info` defines the agent system's action space. ```python async def action_space_info(self) -> gym.Space: ``` #### `Action_space_sample` The `action_space_sample` function returns an element of the simulator’s action space. ```python async def action_space_sample(self): return self.env.action_space.sample() ``` ### Run the Simulation Environment Instance #### `Reset` `Reset` is a request to reset the environment, and returns the first observation of the newly reset environment. * `observation` Initial observation of the environment. * `Dictionary info` Additional information about the reset environment. ```python async def reset(self): return self.env.reset() ``` #### `Step` `Step` provides the agent system action to be applied to the environment. The return structure is as follows: * `observation`; The observation following the action. * `float reward` The reward received after taking the action. * `bool terminated` Whether the episode has ended. * `bool truncated` Whether the episode was truncated before a natural conclusion. * `Dictionary info` Additional information about the step. ```python async def step(self, action): return self.env.step(action) ``` #### `Close` `Close` denotes the simulator is done being used and may perform any necessary cleanups required. ```python async def close(self): self.env.close() ``` #### `Set_Scenario` `Set_scenario` tells the simulator the current scenario the agent system wishes to train on. [Learn more about scenarios.](/build-multi-agent-systems/configure-scenarios) ```python async def set_scenario(self, scenario): self.env.scenario = scenario ``` #### `Get_Scenario` `Get_scenario` returns the scenario that the simulation is currently running. ```python async def get_scenario(self): if self.env.scenario is None: return None return self.env.scenario ``` ### Create Visualizations #### Get\_Render Get\_render provides the current rendered image of the environment, either as a numpy array or a string. ```python async def get_render(self): return self.env.render() ``` # Generate Activity Clusters ## Identify Activity Clusters Activity clusters are distinct operating regions where the system needs to perform differently in order to succeed. Each cluster represents a single scenario β€” the conditions present in the system, as captured by sensor data β€” paired with a single "skill," meaning the actions an operator or agent must take to succeed under those conditions. A complex process can contain hundreds of activity clusters, each corresponding to a different combination of conditions and required actions.
### Clustering Lens The clustering algorithm relies on a "lens" to determine how activity clusters are formed. This lens defines the axis along which the state space is divided, and can be applied in a few different ways: * **Cluster by time**: For phased processes, clustering by time organizes the state space into regions based on which conditions occur together at the same point in time. * **Cluster by a specific variable**: Choosing a variable β€” such as machines or SKUs β€” organizes the state space into regions based on similarities between groups within the data. * **Cluster in aggregate**: Aggregate clustering considers all variables at once, forming clusters based on the combined similarity across every dimension of the data simultaneously. ### Cluster Labels Each cluster is defined by a specific range of sensor variable values. To make these clusters easier to interpret, an LLM automatically generates a human-readable label for each region β€” for example, naming a cluster "Stable Flow" or "Overheat" based on the sensor ranges it covers. These labels make it possible to quickly identify which regions correspond to successful operation versus which represent strain, stress, or failure conditions. Agents use these labeled regions to optimize the process: they learn to identify the successful regions and steer operation toward them.
## Why Identify Activity Clusters? Identifying activity clusters serves two key purposes. First, activity clusters are foundational to designing a team of agents. Each activity cluster will later be configured as its own scenario, with a dedicated agent assigned to it. That agent then practices within its assigned scenario, receiving feedback until it becomes an expert operator for that one specific skill. Second, activity clusters help build a deeper understanding of the task itself. Domain experts will typically recognize most of the clusters from their own experience, confirming that the analysis reflects real-world operating conditions. At the same time, the clustering process often surfaces additional clusters that weren't previously known to experts β€” revealing patterns or edge cases in the data that hadn't been identified before. # Activity Clusters in AMESA Activity clusters are generated as part of simulation creation. You can see the clusters as one of the steps in creating your simulation. You can also view the clusters later: 1. Go to the Simulators page from the main dashboard 2. Click on your simulator 3. Click on the Activity Clusters tab 4. Choose whether to view the clusters as a list of sensor variable ranges or as a graph.
Activity cluster graphs show a multi-dimensional space represented in two dimensions. For this reason, the X and Y axis are not meaningful in the visualizations. Use the list view to capture the variable ranges to use in populating the [Scenarios](/build-multi-agent-systems/configure-scenarios) page when you are setting up your use case.
# Anatomy of a Multi-Agent System Multi-agent systems have structure, just like our brains. Different parts perform different functions. This modularity is a key to building high-performing agentic systems quickly and effectively. Most complex systems, from factories to rockets to software programs, are broken into modular pieces to separate concerns, reuse design patterns, and integrate pre-built components. Like these systems, decision-making AI also works best when decisions are separated into modular concerns. Modular structure makes intelligent agent systems easier to build, test, and maintain. ## How Agentic Systems Make Decisions Multi-agent systems work through a feedback loop. The system gets information from the simulation or real environment and then makes a decision to adjust one or more of its control variables in some way. This creates a change that is reflected in new sensor information that the agent system can then process and use to decide on the next adjustment.
While agent systems are being built and trained, they are connected to simulators[^1] that give them feedback about how their actions affect the system they are learning to control. Once an agent system is deployed, the simulator is replaced by an equivalent connection to the real system. The simulator and the multi-agent system function in a continuous feedback loop, with the system outputting actions that affect the simulation environment, and the simulator returning new information to the agent system based on every decision. Sensors[^2] are the part of the multi-agent system that take in information from the simulator or the real environment about conditions and about the results of the agent system's actions. They are the equivalent of human senses – the eyes and ears. Sensor readings come from specific variables in the simulation or control system that measure various aspects of the state space. Decisions[^3] are the system's outputs, adjusting one or more of the control variables to control the process dynamically throughout the performance of the task. Each decision during training is known as an iteration[^4], while a series of iterations to complete the task is called an episode[^5]. ## Inside the Multi-Agent System The team of agents is organized in layers, with information and decisions passing through each layer in sequence. ### Agents The agent layer, or decision-making layer, is the heart of a multi-agent system. This is where the system makes its control decisions. When a specific agent is activated, it will determine the control action the system should take. Multi-agent systems contain multiple agents orchestrate[^6]d together according to design[ patterns](#user-content-fn-7)[^7]. Agents can work together in groups, sequences, hierarchies, or coordinated teams. This modularity allows the agent to learn more efficiently and perform better. You can imagine agents being like students on a math team who are working together to complete a set of problems. Each student performs best solving a particular kind of problem: one is good at fractions and one at decimals. Depending on the type of problem, the appropriate student will use their expertise to solve the problem and produce the answer for the team. Sometimes one student might handle the entire problem, and for other problems more than one student might need to work together. Just as different students’ capabilities make them able to solve particular problems, different agents may make use of different technologies. Some types of decisions are best approached through skill agents that can be programmed with mathematical calculations, rules, or optimization algorithms. Others that are more complex and nonlinear can be trained using deep reinforcement learning. #### Examples of Skills For an HVAC system regulating temperature in an office building: * Control the system during the day * Control the system at night For a factory where responses are needed to different types of alarms: * Handle safety critical alarms (programmed with rules) * Handle simple alarms (programmed with calculations) * Handle complex alarms (learned with reinforcement learning) For a drone autopilot: * Stabilize * Get to landing zone * Land * Avoid obstacles For a robotic arm used to grab and stack objects: * Reach (extend the robot arm from the "elbow" and "wrist") * Move (move the arm laterally using the "shoulder) * Orient (turn the "wrist" to position the "hand") * Grasp (Manipulate the "fingers" to clamp down) * Stack (Move laterally while grasping) ### Orchestrators Orchestrator[^8] skills are the supervisors for your agent system. In the math class analogy, the orchestrator would be like the teacher. The teacher assesses the type of problem and assigns the right student. In a multi-agent system, an orchestrator uses information from the sensors[^2] and perceptors[^9] to understand the scenario and then determine which skill agent is needed. Once the skill agent is called into service, it makes the decision. For example, for HVAC control, an orchestrator would determine whether day or night control is needed, and then pass control to the appropriate skill agent. In the safety alarm example, the orchestrator determines the type of alarm and then passes the decision to the right skill agent. In the drone and robotic arm examples, the skills need to be performed in sequence. In these cases, the orchestrator assigns the appropriate skill agent as needed for each step in the process. ### Perceptors A perception layer is an optional enhancement layer. Perceptors[^9] process and translate sensor data into a format that can be used to make decisions. In other words, the perception layer inputs the sensor variables and outputs *new* variables deduced by calculation, machine learning, or other programming. For example, if we design a team of agents as an autopilot for a drone, we might have sensor variables that measure pitch, yaw, roll (position of the drone), velocity in each of those three directions, and acceleration in each of those three directions. But what about stability? Stability is an important characteristic to understand while flying a drone, but there is no sensor variable that describes stability. It is too complex to be captured by a single sensor. The perception layer allows us to create a variable for stability. It can be calculated using dynamics equations or trained with supervised machine learning. The new variable then becomes accessible to the rest of the multi-agent system along with the other sensor variables. #### Examples of Perceptors * Computer Vision: A camera sensor passes image or video feeds into a perceptor module that identifies object types and locations * Auditory Perception: A microphone sensor passes machine sounds to a perceptor module that identifies which state the machine is in based on the sounds that it is making * Prediction: A perceptor module inputs quality measurements and past agent system actions and predicts whether current actions will lead to acceptable quality measurements * Anomaly Detection: A perceptor modules inputs market variables and detects when the market is changing regimes. * Classification and Clustering: A perceptor module inputs machine and process data and classifies which of several conditions a manufacturing line is currently in. [^1]: Virtual environments where agents practice and improve performance [^2]: Part of the agent that takes in information from the simulator or real system [^3]: The outputs of an agent - its instructions for controlling the system [^4]: One decision during agent training or performance [^5]: An entire performance of the task to success or a set stop condition [^6]: Arrange modular building blocks of agents to work together to complete tasks [^7]: Common structures that can be used to create agents quickly [^8]: A special skill that assigns decision-making control to the right skill based on current conditions [^9]: ML models or other algorithms that interpret sensor data # Create a Use Case A use case is a collection of agent orchestrations for the same problem. The best way to use AMESA is to build multiple orchestrations within the same use case using different design patterns and variations. This allows you to iterate and improve for the best possible performance. All orchestrations within a use case share the same simulator and the same overall KPI. ## Create a Use Case in the UI To create a use case, click on New Use Case in the upper right-hand corner of your dashboard.
You'll be prompted to enter the simulator associated with your use case and then choose your team. ### Set KPI and ROI After you select your simulator and team, you'll be prompted to set the KPI and ROI for your use case.
The KPI is the top-level goal, the most important metric you are trying to optimize that you will use to evaluate the performance of your team of agents. Choose the sensor variable that represents this metric to serve as your KPI. For benchmark value, enter the current performance level on the KPI. This is the number that your agents will attempt to outperform. Add the benchmark units where prompted. The ROI section allows you to represent agent performance in terms of financial impact. Enter the value in dollars of a specific percentage improvement in the KPI (either increase or decrease depending on the problem). After agents are trained, the platform will use this information to provide data and visualizations about the performance on the KPI compared to the benchmark and about the financial return on investment on the Benchmarks page. # Configure Scenarios Scenarios[^1] are situations where your agent system needs to behave differently to succeed. Scenarios correspond to [activity clusters](/identify-agent-skills/generate-activity-clusters) - activity clusters identify different operating regions within a dataset and scenarios allow agents to train and specialize in these different regions. Scenarios are created by carving out sections of the simulation space, as defined by specific configurations of variables and variable ranges. This allows you to train agents only in the scenarios where they will need to perform, leading to more efficient training and better performance. Orchestrators learn to recognize scenarios and pass control to the appropriate specialist agent to handle the scenario’s demands. ## Types of Scenarios How you define scenarios depends on the type of sensor variables you are working with. Discrete variables are categories that describe a particular scenario. For each scenario involving discrete variables, there is likely a perceptor in the agent system, such as a machine learning model, that inputs the sensors, processes the sensor values, and outputs the discrete variable category. Continuous variables are numbers. When they are used to define a scenario, one number value determines the section of the space that your agent system will explore for decisions. Sometimes a scenario is better defined by a range of continuous values than by a single continuous value. In that case, the scenario would be defined by a continuous variable range. Here are some examples of how scenarios can be defined in different ways: | Use Case | Discrete Variables | Continuous Variables | Continuous Variable Ranges | | ---------------------------------- | --------------------------------------- | ----------------------------------- | --------------------------------------------------- | | Drone Control | windy, far\_from\_charger, low\_battery | windspeed=20 (knots) | windspeed between 20-40 (knots) | | Autonomous Driving | city, highway, high\_traffic | vehicle\_speed=65 (miles per hour) | vehicle\_speed between 65-85 (miles per hour) | | Cybersecurity Network Optimization | normal, high\_traffic, cyberattack | site\_traffic=0 (clicks per second) | site\_traffic between 100-150 (clicks per second) | | Process Control | startup, steady\_state, shutdown | product\_thickness=50 (milimeters) | product\_thickness between 49.94-50.06 (milimeters) | | Machine Control | break-in, normal, wearing\_out | rpm=280 (revolutions per minute) | rpm between 250-295 (revolutions per minute) | ## Scenarios Example Let’s say that we are creating a team of agents to plan the operations of a restaurant with varying demand for three different recipes. We could create three different scenarios: * Low demand: 30 recipe A, 20 recipe B, and 10 recipe C * Normal demand: 60 recipe A, 45 recipe B, and 10 recipe C * High demand: 100 recipe A, 50 recipe B, and 25 recipe C Once you have defined these scenarios, you could set the agent system to train in the low-demand scenario until the success criteria are reached. Then it would move to normal demand, and then to high demand. We would expect the agent to learn faster as it progressed through the scenarios, with cumulative knowledge building over time. ## Set Up Scenarios ### Add Scenarios to a Use Case Add scenarios to your use case by clicking on Scenarios from the left-hand menu to open the Scenarios page. Click Add scenario to create a new scenario. Give your scenario a name and then click `Add another criteria` to configure it. You will then have the option to choose from any of the available sensor variables and apply conditions to them. * For continuous variables, select Is and enter the exact value. * For continuous variable ranges, select Is between and enter the range. * For discrete variables, select Is element of and enter the possible values.
### Create Scenario Flows In addition to identifying scenarios, AMESA also lets you teach your team of agents about how scenarios relate to each other. To do this, you can build scenario flows to establish sequences of scenarios for the agent system to practice. Without scenario flows, the training platform will connect scenarios at random. But sometimes it is important for agent systems to practice scenarios in a specific order. For example, it might be important to practice flying a drone in high winds and then landing in the same conditions. Or a team of agents managing a production line might need to practice going from a scenario in which raw materials are scarce to one in which material costs rapidly rise. By creating a scenario flow, you can ensure that a given sequence of scenarios will be included in training. Once you have scenarios built, create sequential sections of the task by clicking Add section.
After creating all the sections you need, drag and drop the scenarios into the sections.
Create as many flows as you need to capture the different scenario sequences your agent system will need to navigate. ### Add Scenarios to Agents Scenarios are added as part of configuring agents. Scenarios are added to agents so that they know what specific conditions they need to master. Not all scenarios are relevant to all agents. For example, a drone landing agent doesn’t need to practice taking off in windy weather. The configurations you set up when building the scenario flows will appear in the skill agent configuration modal. For each section of the process, as defined by the flows, tell the orchestrator which scenarios it should apply by checking the boxes next to each scenario.
[^1]: Conditions that are associated with specific skills # Set Goals, Constraints, and Success Criteria The performance goal is the most important KPI or metric you will use to evaluate your agent system's success. This goal directs your agent as it trains. The AI learning technology within the agent will reward the agent system when it gets closer to the goal, helping it to improve. Some use cases will have multiple goals the agent system must meet simultaneously. In fact, the need to balance more than one goal is a sign that a use case is a good fit for Machine Teaching. In the [industrial mixer use case,](/reference/sample-use-cases) the agent system needs to get the right chemical product concentration and keep the tank's temperature from becoming dangerously hot. ## Use the Guidance Library for Reusability AMESA supports a Use Case Guidance Libraryβ€”a reusable set of: * Goals * Constraints * Success Criteria This feature allows you to save commonly used elements and apply them across multiple skill agents. Instead of recreating the same definitions every time, you can quickly load goals, constraints, and success criteria from your shared library. Use this to: * Maintain consistency across agents * Speed up setup for similar workflows * Capture expert-defined logic once and reuse it across Skill agents
## Setting Goals, Constraints, and Success Criteria in the UI To edit a goal created by the copilot or to create a new goal, follow these steps: 1. Navigate to the use case page 2. Click Set up goal and enter a name and description for your goal. 3. Click Add condition to define the goal. Use the dropdown menus to select the variables and then define the parameters for each goal, constraint, or success criterion you want to include.
## Configure Goals Goals define what a skill agent system should do. Goals apply to one of the sensor variables and are defined using one of three possible directives: * **Maximize**: Maximize the value of the variable * **Minimize**: Minimize the value of the variable * **Maintain**: Keep the variable at a specified value For example, for [the industrial mixer](/reference/sample-use-cases#industrial-mixer), we want to maximize the concentration of the product, Ca. ## Configure Constraints Constraints set the boundaries for the skill agent. They define rules that the agent system must follow during operation. Constraints are defined using one of two possible directives: * **Avoid**: The skill agent learns to keep the variable from reaching a specified value or range through withholding rewards * **Terminate**: When the actions of the skill agent lead to certain conditions within a variable, the skill agent has failed and must stop and start a new episode For example, we want to avoid the temperature, T, getting above 400 degrees Kelvin in [the industrial mixer](/reference/sample-use-cases#industrial-mixer) use case. ## Configure Success Criteria Success criteria tell the skill agent when it's doing something right. They are defined using one of two possible directives: * **Approach**: The skill agent learns to get close to a specified value by getting an increased reward * **Succeed**: When the success criteria are achieved, the session ends, and a new one begins so that the skill agent can keep practicing and learn to win every time ## Using Goals, Constraints, and Success Criteria for Multi-Agent AI System Design Goals, constraints, and success criteria transform complex AI agent building into an intuitive process that eliminates the need for programming expertise. You directly apply your domain knowledge without coding by simply defining what you want the agent to achieve and what conditions to respect. This approach reduces development time while ensuring AI agents embody real-world operational wisdom, bridging the gap between technical AI capabilities and practical industrial knowledge in complex physical environments. # Create Agents Agents that perform skills are the foundational building blocks for your intelligent agent system. They take action to achieve goals in key scenarios where your agent orchestration needs to succeed. To build an orchestration with Machine Teaching, you will create multiple skill agents and then orchestrate them together. You can use three different types of agents within AMESA. * Create a teacher[^1] to implement a [learned skill](#user-content-fn-2)[^2] that uses DRL to learn to make decisions. You can set up your teacher to [configure the agent using goals](/build-multi-agent-systems/define-skills/train-with-goals-using-the-ui) or to [train with rewards](/build-multi-agent-systems/define-skills/train-with-goals-or-rewards-using-the-sdk). The skill agent will then train and practice in simulation until it can make decisions successfully. * Create a controller[^3] to implement a [programmed skill agent](#user-content-fn-4)[^4] that makes decisions based on programming rather than learning. A controller can use math, rules, optimization, or any other technology that can be expressed in a Python program. [Configure controllers with the Python agent training library](/build-multi-agent-systems/define-skills/configure-programmed-algorithms-as-skills) and publish them to the UI to use in orchestrations. * Create an orchestrator[^5] to [orchestrate skills together](/build-multi-agent-systems/orchestrate-skills). Orchestrators are specialized agents that direct the orchestration how to choose between different decision-making agents. Orchestrators can be either learned or programmed. You can create agents in two different ways, using the Agent Orchestration Studio or using code. If you use the code workflow, you can then [publish the skills using the CLI](/build-multi-agent-systems/import-agent-components-to-the-ui) to appear in the Agent Orchestration Studio ready be included in agent system designs. Or if you use coding agents, the skills can be published via the AMESA Autonomy MCP Server. For learned skills you can use either the Agent Orchestration Studio or the Agent Training Library to create teachers, but the Agent Training Library includes some fine-tuning options that are not available in the Agent Orchestration Studio. Controllers for programmed skills can be created only through the code workflow. They can then be published to the UI for use in agent systems. You can use either the Agent Orchestration Studio or the code workflow to create orchestrators. [^1]: An algorithm that creates a skill that uses DRL to learn to make decisions [^2]: Skills that use DRL to make decisions and learn by practicing in simulation [^3]: Any algorithm that makes decisions based on programming rather than DRL [^4]: Skills that make decisions based on programmed algorithms [^5]: A special skill that assigns decision-making control to the right skill based on current conditions # Create Agents with the Agent Orchestration Studio You can use AMESA to create agents that learn with deep reinforcement learning. Configure the AMESA teacher by setting goals, constraints, and success criteria for your agent. AMESA then turns these into reward functions and trains the agent. ## Create a New Agent To create or edit an agent, follow these steps: 1. Navigate to the Skill Agents page 2. Click on an agent, or click `+` to create a new agent 3. Configure the agent Use the dropdown menus to select the variables and then define the parameters for each goal, constraint, or success criterion you want to include. ## Choose Implementation Method To create a new skill agent that learns with deep reinforcement learning, select `Teacher` under the `Implementation Method` dropdown. The other option, `Controller`, is used for a programmed agent that is configured with the AMESA Agent Training Library and published to the Agent Orchestration Studio. ## Configure Goals Goals define what a skill agent should do. Goals apply to one of the sensor variables and are defined using one of three possible directives: * **Maximize**: Maximize the value of the variable * **Minimize**: Minimize the value of the variable * **Maintain**: Keep the variable at a specified value or track a specified variable (such as a set point) For example, for [the industrial mixer](/reference/sample-use-cases#industrial-mixer), we want to maximize the yield of the product over the course of an entire episode, `eps_yield`. To create a new goal, follow these steps: 1. Navigate to the use case page 2. Click Set up goal and enter a name and description for your goal. 3. Click Add condition to define the goal. Use the dropdown menus to select the variables and then define the parameters for each goal, constraint, or success criterion you want to include.
## Configure Constraints Constraints set the boundaries for the skill agent. They define rules that the skill agent must follow during operation. Constraints are defined using one of two possible directives: * **Avoid**: The skill agent learns to keep the variable from reaching a specified value or range through withholding rewards * **Terminate**: When the actions of the skill agent lead to certain conditions within a variable, the skill agent has failed and must stop and start a new episode For example, we want to avoid the temperature, T, getting above 400 degrees Kelvin in [the industrial mixer](/reference/sample-use-cases#industrial-mixer) use case. ## Configure Success Criteria Success criteria tell the skill agent when it's doing something right. They are defined using one of two possible directives: * **Approach**: The skill agent learns to get close to a specified value by getting increased reward * **Succeed**: When the success criteria are achieved, the session ends, and a new one begins so that the skill agent can keep practicing and learn to win every time ## Leverage the Guidance Library for Reusability AMESA supports a Use Case Guidance Libraryβ€”a reusable set of: * Goals * Constraints * Success Criteria This feature allows you to save commonly used elements and apply them across multiple agents. Instead of recreating the same definitions every time, you can quickly load goals, constraints, and success criteria from your shared library. Use this to: * Maintain consistency across agents * Speed up setup for similar workflows * Capture expert-defined logic once and reuse it across Skill agents
## Use Goals, Constraints, and Success Criteria for Multi-Agent AI System Design Goals, constraints, and success criteria transform complex AI agent building into an intuitive process that eliminates the need for programming expertise. You directly apply your domain knowledge without coding by simply defining what you want the agent to achieve and what conditions to respect. This approach reduces development time while ensuring AI agents embody real-world operational wisdom, bridging the gap between technical AI capabilities and practical industrial knowledge in complex physical environments. # Create Agents with Rewards Using Python The AMESA Agent Training Library offers a suite of advanced tools to train agents using deep reinforcement learning. Using the Python teacher class, you can fine-tune the rewards for your agents. Once you have configured an agent with code, you can publish it to the Agent Orchestration Studio to use in agent orchestrations. ## Create a New Agent To create an agent in Python, begin by logging in to the Agent Training Library by typing `AMESA login` from the CLI. Then type `amesa agent new`. Give the agent a name and a description in response to the prompts that follow. Choose whether your skill should be a teacher (learned with AI) or a controller (a programmed module like an optimization algorithm or MPC controller). Specify the folder where you’d like to create the agent. The AMESA Agent Training Library will create a folder and Python teacher file from the template. ## The Python Teacher Class The Python teacher class offers several functions that you can use to fine-tune the training of your skills. ### Functions for Training #### Train with Rewards: the `compute_reward` Function The `compute_reward` function provides the bulk of the feedback after each action about how much that action contributed to the success of the agemt. This function returns a number that represents the reward signal the orchestration will receive for its last decision. Reward functions, as they are called in reinforcement learning, can be tricky to craft. [Learn more about how to write good reward functions](https://medium.com/@BonsaiAI/deep-reinforcement-learning-models-tips-tricks-for-writing-reward-functions-a84fe525e8e0). ```python python def compute_reward(self, transformed_sensors, action, sim_reward): self.counter += 1 if self.past_ sensors is None: self.past_ sensors = transformed_ sensors return 0 else: if self.past_ sensors ["state1"] < transformed_ sensors ["state1"]: return 1 else: return -1 ``` #### End Training: the `compute_termination` Function[​](https://autonomy.university/building-agents/adding-skills/teaching-skills.html#compute-termination-function) The `compute_termination` function tells the AMESA platform when to terminate a practice episode and start over with a new practice scenario (episode). From a teaching perspective, it makes most senses to terminate an episode when the agent system succeeds, fails, or is pursuing a course of action that you do not find likely to succeed. This function returns a Boolean flag (`True` or `False`) whether to terminate the episode. You can calculate this criteria however seems best. ```python python def compute_termination(self, transformed_ sensors, action): return False ``` #### Define Success: the `compute_success_criteria` Function[​](https://autonomy.university/building-agents/adding-skills/teaching-skills.html#compute-success-criteria-function) The `success_criteria` function provides a definition of agent success and a proxy for how completely the agent system has learned the skill. The platform uses the output of this function (`True` or `False`) to calculate when to stop training one agent and move on to training the next agent. It is also used to determine when to move to the next agent in a fixed order sequence. The agent system cannot move from one agent in a fixed order sequence to the next, until the success criteria for one agent is reached. ```python python def compute_success_criteria(self, transformed_ sensors, action): return self.counter > 100 ``` Here are some examples of success criteria definition: * A simple but naive success criteria might return `True` if the average reward for an episode or scenario crosses a threshold, but `False` if it does not. * A more complex success criteria might calculate root mean squared error (RMSE) for key variables across the episode and return `True` if the error is less than a customer specified benchmark, but `False` otherwise. * A complex success criteria might compare a benchmark controller or another agent system to the agent system across many key variables and trials. It returns `True` if the agent system beats the benchmark on this criteria, but `False` otherwise. #### Train with Goals Training with goals lets you use a predefined reward structure rather than configuring the rewards individually. When you use a goal, your agent will inherit the compute reward, compute termination, and compute success functions from the goal. (You will still have the option to further customize those functions as needed.) The five goal types you can use are: * `AvoidGoal` * `MaximizeGoal` * `MinimizeGoal` * `ApproachGoal` * `MaintainGoal` These have the same parameters and work the same way as [the goal types in the UI](https://app.gitbook.com/o/TnHRmXbRhAMO4m95Ca9E/s/6X1hFdU8qyyv3sgW7kkX/~/changes/5/build-autonomous-agents/define-skills/set-goals-in-the-ui). Goals are added using specialized teacher classes rather than the general teacher class that you would otherwise use to teach agents. For example, for an agent named Balance that you wanted to train with a goal to maintain a specific orientation, you would use the MaintainGoal teacher class. ```python python class BalanceTeacher(MaintainGoal): def __init__(self, *args, **kwargs): super(),__init__(β€œpole_theta”, β€œMaintain pole to upright”, target=0, stop_distance=0.418) ``` The parameters you can use for goals are:
You can also use more than one goal for a single skill using the `CoordinatedGoal` teacher class. This is useful when your agent system needs to behave in a way that creates a balance between two goals that are both important.
### Functions to Guide Agent System Behavior with Rules Just like rules guide training and behavior for humans, providing rules for the agent system to follow can guide its decision-making more quickly to success. Rules guide the behavior of an agent system based on expertise and constraints. #### Add Rules: the `compute_action_mask` Function The `compute_action_mask` teaching function expresses rules that trainable agents must follow. ```python python # The action mask provides rules at each step about which actions the agent system is allowed to take. def compute_action_mask(self, transformed_ sensors, action): return [0, 1, 1] ``` The `compute_action_mask` teaching function works only for discrete action spaces (where the actions are integers or categories), not for continuous action spaces (where decision actions are decimal numbers). If you specify a mask for a skill whose actions are continuous, the platform will ignore the action mask. The function returns a list of 0 and 1 values. Zero means that the action is forbidden by the rule. One means that the action is allowed by the rule. The function may change the returned value after each decision. This allows complex logic to express nuanced rules. In the example above, the first action is forbidden for the next decision, but the second and third actions are allowed. The logic in the agent itself (whether learned or programmed) will choose between the allowed second and third actions. All orchestrators have a discrete action space (they choose which child skill to activate), so you can always apply the `compute_action_mask` function to teach them. ### **Functions to Manage Information Inside Agent S**ystem**s** As information passes through perceptors, agents, and orchestrators in the agent system, sometimes it needs to change format along the way. You can use three teaching functions to transform sensor and action variables inside agent systems: `transform_` sensors, `transform_action`, and `filtered_` sensor `_space`. #### Transform Sensor Variables: the `transform_sensors` function[​](https://autonomy.university/building-agents/adding-skills/teaching-skills.html#transforming-sensor-variables) To transform sensor variables, use the `transform_sensor` function to calculate changes to specific sensors, then return the complete set of sensor variables (the observation space). ```python python def transform_sensor(self, sensor, action): return sensor ``` Two of the most common reasons for transforming sensor variables are conversion and normalization. For example, if a simulator reports temperature values in Fahrenheit, but the agent system expects temperature values in Celsius, use the `transform_sensor` function to convert between the two. Normalization is when you transform variables into different ranges. For example, one sensor variable in your agent system might have very large values (in the thousands), but another variable might have small values (in the tenths), so you might use the `transform_sensor` function to transform these disparate sensor values to a range from 0 to 1 so that they can be better compared and used in the agent system. #### Transform Decisions within the Agent System[​](https://autonomy.university/building-agents/adding-skills/teaching-skills.html#transforming-decisions-within-the-agent): the `transform_action` function[​](https://autonomy.university/building-agents/adding-skills/teaching-skills.html#transforming-sensor-variables) You may want to transform action variables for the same reasons as sensor variables. ```python python def transform_action(self, transformed_sensor, action): return action ``` #### Filter the Sensor List: the `filtered_sensor_space` function[​](https://autonomy.university/building-agents/adding-skills/teaching-skills.html#transforming-sensor-variables) Use the `filtered_sensor_space` function to pare down the list of sensor variables you need for a particular skill. Pass only the information that a skill or module needs in order to learn or perform well. ```python python def filtered_sensor_space(self): return ["state1"] Return a list of all the sensor variables that you want passed to the skill by this teacher. ``` # Deep Dive: Create and Publish Teachers This guide walks you through everything you need to build an Agent Teacher from scratch and publish it to Amesa Orchestration Studio using the `amesa` CLI. By the end, you will have a working teacher packaged as an artifact and registered in the AMESA registry. ## What Is a Teacher? A **Teacher** is a Python class that defines how an agent learns. When an agent trains with reinforcement learning, it needs to know three things: * **What is a good outcome?** β€” the reward signal * **When has the agent succeeded?** β€” the success condition * **When should training stop early?** β€” the failure/termination condition Your Teacher class answers all three. It also controls which sensors the agent pays attention to and how actions are processed before being sent to the simulation. You write the domain logic. AMESA handles the RL training engine, the simulator connection, and the infrastructure. {% stepper %} {% step %} ## Understand the Teacher Methods A Teacher is a subclass of `AgentTeacher` from `amesa_core`. You must implement four methods. Two more are optional but useful. ### Required Methods | Method | What it does | | -------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------ | | `compute_reward` | Returns a `float` reward for each training step. Higher is better. This is your primary teaching signal. | | `compute_success_criteria` | Returns `True` when the agent has achieved its goal for this episode. | | `transform_action` | Pre-processes the raw action from the RL model before it is sent to the simulator. Return the action unchanged if no processing is needed. | | `filtered_sensor_space` | Returns a list of sensor name strings that this agent observes. Only listed sensors are visible to the RL policy. | ### Optional Methods | Method | What it does | Default | | --------------------- | --------------------------------------------------------------------------------------------------------------------------------- | -------------- | | `compute_termination` | Returns `True` to end the episode early as a failure. Use this to cut off episodes that have gone badly wrong and cannot recover. | `return False` | | `transform_sensors` | Pre-processes the raw sensor dict before it reaches the RL policy. Use for feature engineering or unit conversion. | pass-through | ### Method Signatures ```python from typing import Dict, List from amesa_core import AgentTeacher class MyTeacher(AgentTeacher): # Required β€” called once at startup; result is cached async def filtered_sensor_space(self) -> List[str]: ... # Required β€” called every step; return a float async def compute_reward(self, transformed_sensors: Dict, action, sim_reward: float) -> float: ... # Required β€” called every step; return True when the agent has succeeded async def compute_success_criteria(self, transformed_sensors: Dict, action) -> bool: ... # Required β€” called every step; return the action to send to the simulator async def transform_action(self, transformed_sensors: Dict, action): ... # Optional β€” called every step; return True to end the episode as a failure async def compute_termination(self, transformed_sensors: Dict, action) -> bool: return False # Optional β€” called every step; return a modified sensors dict async def transform_sensors(self, sensors, action) -> Dict: return sensors ``` {% hint style="warning" %} **Important β€” all methods are `async`.** Declare every method with `async def`. Forgetting `async` causes silent failures in the training loop. {% endhint %} {% hint style="warning" %} **Important β€” `__init__` must accept zero arguments.** The framework re-creates your teacher at the start of every training episode by calling `MyTeacher()` with no arguments. Any `__init__` parameter that does not have a default value will cause a `TypeError` on every episode reset. {% endhint %} {% endstep %} {% step %} ## Write Your Teacher There are two approaches: writing a **custom teacher** from scratch, or using a **built-in Goal class** for common objectives. ### Option A: Custom Teacher Use a custom teacher when your reward logic or success criteria are specific to your domain. The example below trains a agent to maintain a process temperature near a target setpoint. The reward decreases the further the temperature drifts from the target. The episode succeeds when temperature is within tolerance and terminates early if it goes too far off. ```python # temperature_teacher/teacher.py from typing import Dict, List from amesa_core import AgentTeacher class TemperatureTeacher(AgentTeacher): """ Trains an agent to maintain process temperature near a setpoint. Reward: negative absolute error from target (closer = higher reward) Success: temperature within TOLERANCE of target Termination: temperature more than FAIL_DISTANCE from target """ TARGET = 80.0 # desired temperature TOLERANCE = 2.0 # success band: Β±2 degrees FAIL_DISTANCE = 20.0 # terminate if this far from target def __init__(self): # __init__ must take no required arguments # self.* state is wiped at every episode reset β€” do not store cross-episode data here pass async def filtered_sensor_space(self) -> List[str]: # Declare which sensors the RL policy observes # Only these sensor names will be visible during training return ["temperature", "heater_output"] async def compute_reward(self, transformed_sensors: Dict, action, sim_reward: float) -> float: error = abs(transformed_sensors["temperature"] - self.TARGET) return -error # reward = 0 at target; decreases with distance async def compute_success_criteria(self, transformed_sensors: Dict, action) -> bool: error = abs(transformed_sensors["temperature"] - self.TARGET) return error <= self.TOLERANCE async def compute_termination(self, transformed_sensors: Dict, action) -> bool: error = abs(transformed_sensors["temperature"] - self.TARGET) return error >= self.FAIL_DISTANCE async def transform_action(self, transformed_sensors: Dict, action): # Clamp heater output to valid range [0.0, 1.0] return max(0.0, min(1.0, action)) async def transform_sensors(self, sensors, action) -> Dict: # No pre-processing needed β€” pass sensors through unchanged # Note: action is always None here; do not write logic that depends on it return sensors ``` ### Option B: Goal-Based Teacher (Less Code) If your objective fits one of the built-in patterns β€” maximize, minimize, maintain, approach, or avoid a sensor value β€” use a **Goal class** instead of writing reward logic by hand. Goal classes are subclasses of `AgentTeacher` and satisfy the full teacher contract automatically. | Goal Class | Use when you want to... | | ----------------- | ----------------------------------------------- | | `MaximizeGoal` | Drive a sensor value as high as possible | | `MinimizeGoal` | Drive a sensor value as low as possible | | `MaintainGoal` | Hold a sensor near a target value within a band | | `ApproachGoal` | Reach a target value as quickly as possible | | `AvoidGoal` | Keep a sensor away from a boundary value | | `CoordinatedGoal` | Combine multiple goals into a single teacher | The example below reproduces the temperature setpoint objective above using `MaintainGoal`, with no manual reward function: ```python # temperature_teacher/teacher.py from typing import Dict from amesa_core.orchestration.agent.goals.coordinated_goal import CoordinatedGoal from amesa_core.orchestration.agent.goals.maintain_goal import MaintainGoal class TemperatureTeacher(CoordinatedGoal): """ Goal-based teacher: maintain temperature near setpoint using MaintainGoal. CoordinatedGoal handles compute_reward, compute_success_criteria, and compute_termination automatically. """ def __init__(self): temperature_goal = MaintainGoal( "temperature", "Maintain process temperature near setpoint", target=80.0, stop_distance=2.0, ) super().__init__([temperature_goal]) async def transform_action(self, transformed_sensors: Dict, action): return max(0.0, min(1.0, action)) async def transform_sensors(self, sensors, action) -> Dict: return sensors ``` Use `CoordinatedGoal` when you have one or more goal objectives. Pass all goal instances as a list to `super().__init__()`. {% endstep %} {% step %} ## Create the Artifact Directory A published teacher must be packaged in a specific directory layout. The `amesa agent new` command scaffolds this for you automatically, or you can create it manually. ### Using the CLI Scaffold (Recommended) ```bash amesa agent new \ --name temperature-teacher \ --type teacher \ --description "Maintains process temperature near setpoint" \ --location ./ ``` This creates the following directory: ``` temperature-teacher/ temperature_teacher/ __init__.py ← empty; required teacher.py ← your AgentTeacher subclass goes here pyproject.toml ← artifact metadata; CLI reads this on publish ``` Replace the placeholder `teacher.py` with your implementation from Step 3. ### Creating Manually If you prefer to create the structure yourself: ```bash mkdir -p temperature-teacher/temperature_teacher touch temperature-teacher/temperature_teacher/__init__.py touch temperature-teacher/temperature_teacher/teacher.py touch temperature-teacher/pyproject.toml ``` Then copy your teacher code into `temperature_teacher/teacher.py` and fill in `pyproject.toml` as described in the next step. ### Naming Rules The directory has two layers with different naming conventions: | Layer | Convention | Example | | ------------------- | ------------- | --------------------- | | Outer directory | kebab-case | `temperature-teacher` | | Inner Python module | snake\_case | `temperature_teacher` | | Source file | by convention | `teacher.py` | | `__init__.py` | always empty | *(no content)* | The outer directory is a **filesystem path used by the CLI only**. The inner snake\_case directory is the **Python module** referenced in your `pyproject.toml` entrypoint. Do not mix these up β€” using the kebab-case outer name as your Python import path will cause a `ModuleNotFoundError` on publish. {% endstep %} {% step %} ## Configure `pyproject.toml` Open `temperature-teacher/pyproject.toml` and fill it in: ```toml [build-system] requires = ["setuptools"] build-backend = "setuptools.backends.legacy:build" [project] name = "temperature-teacher" version = "0.1.0" description = "Maintains process temperature near setpoint" dependencies = [ "amesa-core", ] [amesa] type = "agent-teacher" entrypoint = "temperature_teacher.teacher:TemperatureTeacher" ``` ### Field Reference | Field | Where | Required | Description | | -------------- | ----------- | -------- | ------------------------------------------------------------------------------------- | | `name` | `[project]` | Yes | Artifact name as it will appear in the registry. Kebab-case. | | `version` | `[project]` | Yes | Semantic version string (e.g. `0.1.0`). | | `description` | `[project]` | Yes | Short human-readable description. | | `dependencies` | `[project]` | Yes | Must include `"amesa-core"`. Add any other packages your teacher imports. | | `type` | `[amesa]` | Yes | Must be `"agent-teacher"` for a teacher. | | `entrypoint` | `[amesa]` | Yes | `module_name.file_name:ClassName` β€” resolves your class from inside the inner module. | ### Entrypoint Format The entrypoint string follows the pattern: `inner_module.filename:ClassName` | Part | Value in this example | | -------------- | --------------------------------------------------------------- | | `inner_module` | `temperature_teacher` (the snake\_case inner directory) | | `filename` | `teacher` (the `.py` file, without extension) | | `ClassName` | `TemperatureTeacher` (the class that subclasses `AgentTeacher`) | **Full entrypoint:** `temperature_teacher.teacher:TemperatureTeacher` {% hint style="warning" %} **Common mistake:** using the outer kebab-case directory name (`temperature-teacher`) in the entrypoint. Python cannot import a name with a hyphen. Always use the inner snake\_case module name. {% endhint %} {% endstep %} {% step %} ## Verify Your Directory Before publishing, confirm your structure looks correct: ``` temperature-teacher/ β”œβ”€β”€ temperature_teacher/ β”‚ β”œβ”€β”€ __init__.py ← must exist; must be empty β”‚ └── teacher.py ← contains TemperatureTeacher class └── pyproject.toml ← contains [amesa] section with type and entrypoint ``` Quick checklist: * [ ] `__init__.py` exists and is empty * [ ] `pyproject.toml` is in the **outer** directory (same level as `temperature_teacher/`) * [ ] `type` in `[amesa]` is `"agent-teacher"` * [ ] `entrypoint` uses the **inner snake\_case** module name, not the outer kebab-case name * [ ] The class name in `entrypoint` exactly matches the class name in `teacher.py` * [ ] All imports in `teacher.py` use `from amesa_core import ...` * [ ] `__init__` in your teacher class takes no required arguments {% endstep %} {% step %} ## Publish From the directory that **contains** your artifact folder, run: ```bash amesa agent publish ./temperature-teacher/ ``` You can also use the `--path` flag: ```bash amesa agent publish --path ./temperature-teacher/ ``` The CLI will: 1. Ask you to select a project (if you have more than one) 2. Package the directory as a compressed archive 3. Read `pyproject.toml` to extract the name, description, and type 4. Create the agent entry in the AMESA registry 5. Upload the artifact A successful publish prints the artifact name and confirms the upload. The teacher is now registered and available to training jobs in your selected project. {% endstep %} {% step %} ## Confirm the Publish List your project's agents to confirm the teacher appeared: ```bash amesa agent list ``` This prints a table of all agents registered to the project, including: | Name | Type | Version | Description | UUID | | ------------------- | ------- | ------- | ------------------------------------------- | ---- | | temperature-teacher | teacher | 1 | Maintains process temperature near setpoint | ... | Copy the UUID if you need to reference this teacher in an agent orchestration. {% endstep %} {% endstepper %} ## Updating a Published Teacher To publish a new version of a teacher, increment the `version` field in `pyproject.toml` and run `amesa agent publish` again: ```toml [project] version = "0.2.0" # ← increment before re-publishing ``` ```bash amesa agent publish ./temperature-teacher/ ``` Each publish creates a new version entry in the registry. Prior versions remain available. ## Deleting a Teacher To remove a teacher from the registry: ```bash amesa agent delete ``` The CLI presents an interactive list of your project's agents. Select the one you want to remove and confirm. Deletion is permanent and cannot be undone. ## Troubleshooting
`ModuleNotFoundError` on publish Your `entrypoint` in `pyproject.toml` is using the wrong module path. Check that you are using the **inner snake\_case** directory name, not the outer kebab-case name: ```toml # WRONG β€” hyphens are not valid in Python imports entrypoint = "temperature-teacher.teacher:TemperatureTeacher" # CORRECT β€” use the inner snake_case module entrypoint = "temperature_teacher.teacher:TemperatureTeacher" ```
`TypeError` during episode reset Your `__init__` method has a required argument. The framework calls `MyTeacher()` with no arguments at every episode boundary. Use keyword arguments with default values: ```python # WRONG β€” threshold is required, will raise TypeError on episode reset def __init__(self, threshold: float): self.threshold = threshold # CORRECT β€” provide a default value def __init__(self, threshold: float = 80.0): self.threshold = threshold ```
`ImportError` β€” `composabl` or `composabl_core` not found These are legacy package names. Update all imports to use `amesa_core`: ```python # WRONG from composabl import AgentTeacher from composabl_core import AgentTeacher # CORRECT from amesa_core import AgentTeacher ``` Also update `pyproject.toml`: ```toml # WRONG dependencies = ["composabl-core"] # CORRECT dependencies = ["amesa-core"] ```
Reward is `None` or training crashes immediately `compute_reward` must return a scalar `float`. If you forget to return a value or return `None`, the training loop crashes. Add an explicit `return`: ```python async def compute_reward(self, transformed_sensors, action, sim_reward) -> float: error = abs(transformed_sensors["temperature"] - self.TARGET) return -error # ← must always return a float ```
Both success and termination return `True` on the same step If `compute_success_criteria` and `compute_termination` both return `True` simultaneously, the episode ends as a **failure**, not a success. The success counter is decremented. Review your threshold logic to make sure success and failure conditions are mutually exclusive, or ensure the termination threshold is strictly worse than the success threshold.
# Configure Programmed Algorithms as Skill Agents Controllers[^1] are programmed skill agents used in AMESA agent systems. These may be optimizers, PID or MPC controllers, heuristics, or any other Python algorithms that are useful to control parts of a process. Configure controllers in the SDK and then publish them to the UI to use in agent systems. ## Create a New Skill Agent To create a skill agent in the Python SDK, begin by logging in to the SDK by typing `AMESA login` from the CLI. Then type `amesa agent new`. Give the skill agent a name and a description in response to the prompts that follow. Choose your skill agent type as a controller (a programmed module like an optimization algorithm or MPC controller). Specify the folder where you’d like to create the skill agent. The Composal SDK will create a folder and Python `controller.py()` file from the template. ## The Python Controller Class The Python controller class offers several functions that you can use to build your algorithms or rule-based controllers in AMESA. ### Functions for Training Controllers don't need training, since they are based on programming rather than learning, but they include functions that connect them to the rest of the agent system during training. #### Initialize your algorithm: the `__init__` Function The `__init__` function can be used to initialize your Algorithm and define initial configuration variables, this is called once when the Runtime starts. Let's supose that you want to use a MPC algorithm: ```python def __init__(self, *args, **kwargs): self.counter += 1 self.mpc = LinearMPC() ``` #### Process Observation to Compute Action: the `compute_action` Function The `compute_action` function will process the observation and returns an action. This function returns a number that represents the action that will be sent to the simulation. ```python async def compute_action(self, obs, action): self.counter += 1 action = self.mpc.solve(obs) return action ``` #### End Simulation: the `compute_termination` Function[​](https://autonomy.university/building-agents/adding-skills/teaching-skills.html#compute-termination-function) The `compute_termination` function tells the AMESA platform when to terminate a practice episode and start over with a new practice scenario (episode). From a controller perspective, it makes most senses to terminate an episode when the agent system succeeds, fails, or is pursuing a course of action that you do not find likely to succeed. This function returns a Boolean flag (`True` or `False`) whether to terminate the episode. You can calculate this criteria however seems best. ```python python async def compute_termination(self, transformed_sensors, action): return False ``` #### Define Success: the `compute_success_criteria` Function[​](https://autonomy.university/building-agents/adding-skills/teaching-skills.html#compute-success-criteria-function) The `success_criteria` function provides a definition of the skill agent's success and a proxy for how completely the agent system has learned the skill. The platform uses the output of this function (`True` or `False`) to calculate when to stop training one skill agent and move on to training the next skill agent. It is also used to determine when to move to the next skill agent in a fixed order sequence. The agent system cannot move from one skill agent in a fixed order sequence to the next, until the success criteria for one skill agent is reached. ```python python async def compute_success_criteria(self, transformed_ sensors, action): return self.counter > 100 ``` Here are some examples of success criteria definition: * A simple but naive success criteria might return `True` if a certain sensors or KPI value for an episode or scenario crosses a threshold, but `False` if it does not. * A complex success criteria might compare a benchmark controller or another agent system to the agent system across many key variables and trials. It returns `True` if the agent system beats the benchmark on this criteria, but `False` otherwise. ### **Functions to Manage Information Inside Agent S**ystem**s** As information passes through perceptors, skill agents, and selectors in the agent system, sometimes it needs to change format along the way. You can use three teaching functions to transform sensor and action variables inside agent systems: `transform_` sensors, `transform_action`, and `filtered_` sensor `_space`. #### Transform Sensor Variables: the `transform_sensors` function[​](https://autonomy.university/building-agents/adding-skills/teaching-skills.html#transforming-sensor-variables) To transform sensor variables, use the `transform_sensor` function to calculate changes to specific sensors, then return the complete set of sensor variables (the observation space). ```python python async def transform_sensor(self, sensor, action): return sensor ``` Two of the most common reasons for transforming sensor variables are conversion and normalization. For example, if a simulator reports temperature values in Fahrenheit, but the agent system expects temperature values in Celsius, use the `transform_sensor` function to convert between the two. Normalization is when you transform variables into different ranges. For example, one sensor variable in your agent system might have very large values (in the thousands), but another variable might have small values (in the tenths), so you might use the `transform_sensor` function to transform these disparate sensor values to a range from 0 to 1 so that they can better be compared and used in the agent system. #### Transform Decisions within the Agent System[​](https://autonomy.university/building-agents/adding-skills/teaching-skills.html#transforming-decisions-within-the-agent): the `transform_action` function[​](https://autonomy.university/building-agents/adding-skills/teaching-skills.html#transforming-sensor-variables) You may want to transform action variables for the same reasons as sensor variables. ```python python async def transform_action(self, transformed_sensor, action): return action ``` #### Filter the Sensor List: the `filtered_sensor_space` function[​](https://autonomy.university/building-agents/adding-skills/teaching-skills.html#transforming-sensor-variables) Use the `filtered_sensor_space` function to pare down the list of sensor variables you need for a particular skill agent. Pass only the information that a skill agent or module needs in order to learn or perform well. ```python python def filtered_sensor_space(self): return ["state1"] Return a list of all the sensor variables that you want passed to the skill agent by this teacher. ``` [^1]: Any algorithm that makes decisions based on programming rather than DRL # Deep Dive: Create and Publish Controllers This guide walks you through everything you need to build an Agent Controller from scratch and publish it to Amesa Orchestration Studio using the `amesa` CLI. By the end, you will have a working controller packaged as an artifact and registered in the AMESA registry. ## What Is a Controller? A **Controller** is a deterministic, hand-coded agent. Instead of learning a policy through reinforcement learning, a controller executes rule-based, algorithmic, or expert-system logic that you write directly. Every call to the controller runs your code β€” there is no training involved. Use a controller when: * The correct action can be computed directly from sensor values (PID, proportional, rule-based) * You have an existing algorithm or expert system you want to run as an agent * You want a reliable baseline to compare against a trained RL agent * The action logic is fully specified and does not need to adapt to new conditions Controllers and RL Teachers can coexist in the same orchestration. A common pattern is to use a controller as a stabilizing baseline agent alongside RL teachers that handle more complex sub-tasks. {% stepper %} {% step %} ## Understand the Controller Methods A Controller is a subclass of `AgentController` from `amesa_core`. You must implement four methods. ### Required Methods | Method | What it does | | -------------------------- | ------------------------------------------------------------------------------------------------------------ | | `compute_action` | Returns the action to send to the simulator. This is your core control logic β€” called once per timestep. | | `filtered_sensor_space` | Returns the list of sensor names this controller reads. Only listed sensors are visible in `compute_action`. | | `compute_success_criteria` | Returns `True` when the current episode should be considered a success. | | `compute_termination` | Returns `True` to end the episode early as a failure. | > **All four methods are async.** Declare every method with `async def`. > **`__init__` must accept zero required arguments.** The framework re-creates your controller at the start of every episode by calling `MyController()` with no arguments. Any `__init__` parameter without a default value causes a `TypeError` on every episode reset. ### Method Signatures ```python from typing import Dict, List from amesa_core import AgentController class MyController(AgentController): def __init__(self): # Must accept zero required arguments # All self.* state is wiped at every episode reset pass async def filtered_sensor_space(self, obs_spec) -> List[str]: # Declare which sensors this controller observes # Called once at startup; result is cached ... async def compute_action(self, obs_spec: Dict, action) -> any: # Core control logic β€” called every timestep # obs_spec: filtered sensor dict (keys match filtered_sensor_space) # action: previous action sent to the sim (None on first step) # Returns: the action to send to the simulator ... async def compute_success_criteria(self, obs_spec: Dict, action) -> bool: # Return True when the episode should end as a success ... async def compute_termination(self, obs_spec: Dict, action) -> bool: # Return True to end the episode early as a failure ... ``` ### `compute_action` Parameters | Parameter | Type | Description | | ---------- | ------ | ----------------------------------------------------------------------------------------------------- | | `obs_spec` | `Dict` | Filtered sensor observations for this step. Keys match the names returned by `filtered_sensor_space`. | | `action` | any | The action sent to the simulator on the **previous** step. `None` on the first step of each episode. | **Return value:** The action to send to the simulator. Must match the simulator's expected action format. {% endstep %} {% step %} ## Write Your Controller The example below implements a proportional controller that maintains dissolved oxygen in a fermentation reactor near a target setpoint by adjusting agitation speed. ```python # fermentation_controller/controller.py from typing import Dict, List from amesa_core import AgentController class FermentationController(AgentController): """ Proportional controller for dissolved oxygen in a fermentation reactor. Adjusts agitation speed based on deviation from target DO setpoint. """ TARGET_DO = 0.65 # target dissolved oxygen fraction TOLERANCE = 0.03 # success band: Β±0.03 FAIL_DO_LOW = 0.10 # terminate if DO drops this low KP = 2.5 # proportional gain def __init__(self): # No required arguments β€” called with no args at every episode reset self.prev_error = 0.0 async def filtered_sensor_space(self, obs_spec) -> List[str]: # Expose only the sensors this controller actually reads return ["dissolved_oxygen", "agitation_rpm"] async def compute_action(self, obs_spec: Dict, action) -> List[float]: do_level = obs_spec["dissolved_oxygen"] error = self.TARGET_DO - do_level # Proportional control: positive error β†’ increase agitation delta_rpm = self.KP * error self.prev_error = error return [float(delta_rpm)] async def compute_success_criteria(self, obs_spec: Dict, action) -> bool: do_level = obs_spec["dissolved_oxygen"] return abs(do_level - self.TARGET_DO) <= self.TOLERANCE async def compute_termination(self, obs_spec: Dict, action) -> bool: do_level = obs_spec["dissolved_oxygen"] return do_level < self.FAIL_DO_LOW ``` ### Accessing Full Sensor List To expose all orchestration sensors to the controller rather than a subset: ```python async def filtered_sensor_space(self, obs_spec) -> List[str]: return obs_spec # pass through β€” observe everything ``` {% endstep %} {% step %} ## Create the Artifact Directory ### Using the CLI Scaffold (Recommended) ```bash amesa agent new \ --name fermentation-controller \ --type controller \ --description "Proportional DO controller for fermentation reactor" \ --location ./ ``` This creates: ``` fermentation-controller/ fermentation_controller/ __init__.py ← empty; required controller.py ← your AgentController subclass goes here pyproject.toml ``` Replace the placeholder `controller.py` with your implementation from Step 2. ### Creating Manually ```bash mkdir -p fermentation-controller/fermentation_controller touch fermentation-controller/fermentation_controller/__init__.py touch fermentation-controller/fermentation_controller/controller.py touch fermentation-controller/pyproject.toml ``` ### Naming Rules | Layer | Convention | Example | | ------------------- | ------------- | ------------------------- | | Outer directory | kebab-case | `fermentation-controller` | | Inner Python module | snake\_case | `fermentation_controller` | | Source file | by convention | `controller.py` | | `__init__.py` | always empty | *(no content)* | The outer kebab-case directory is used by the CLI only. The inner snake\_case directory is the importable Python module referenced in `pyproject.toml`. Never use the kebab-case outer name in your entrypoint. {% endstep %} {% step %} ## Configure `pyproject.toml` ```toml [build-system] requires = ["setuptools"] build-backend = "setuptools.backends.legacy:build" [project] name = "fermentation-controller" version = "0.1.0" description = "Proportional DO controller for fermentation reactor" dependencies = [ "amesa-core", ] [amesa] type = "agent-controller" entrypoint = "fermentation_controller.controller:FermentationController" ``` ### Field Reference | Field | Where | Required | Description | | -------------- | ----------- | -------- | ------------------------------------------------------------ | | `name` | `[project]` | Yes | Artifact name as it will appear in the registry. Kebab-case. | | `version` | `[project]` | Yes | Semantic version string. | | `description` | `[project]` | Yes | Short human-readable description. | | `dependencies` | `[project]` | Yes | Must include `"amesa-core"`. | | `type` | `[amesa]` | Yes | Must be `"agent-controller"`. | | `entrypoint` | `[amesa]` | Yes | `inner_module.filename:ClassName` | > **Do not add a `variables` field.** The `variables` field is for perceptors only. Including it in a agent `pyproject.toml` causes a validation error on publish. ### Entrypoint Format | Part | Value in this example | | -------------- | ------------------------------------------------------- | | `inner_module` | `fermentation_controller` (inner snake\_case directory) | | `filename` | `controller` (the `.py` file, without extension) | | `ClassName` | `FermentationController` | **Full entrypoint:** `fermentation_controller.controller:FermentationController` {% endstep %} {% step %} ## Verify Your Directory ``` fermentation-controller/ β”œβ”€β”€ fermentation_controller/ β”‚ β”œβ”€β”€ __init__.py ← must exist; must be empty β”‚ └── controller.py ← contains FermentationController class └── pyproject.toml ← contains [amesa] section with type and entrypoint ``` Checklist: * [ ] `__init__.py` exists and is empty * [ ] `pyproject.toml` is in the outer directory (same level as `fermentation_controller/`) * [ ] `type` in `[amesa]` is `"agent-controller"` * [ ] `entrypoint` uses the inner snake\_case module name * [ ] `entrypoint` class name matches the class in `controller.py` * [ ] All four required methods are implemented with `async def` * [ ] `__init__` takes no required arguments * [ ] No `variables` field in `[amesa]` * [ ] All imports use `from amesa_core import ...` {% endstep %} {% step %} ## Publish ```bash amesa agent publish ./fermentation-controller/ ``` Or using the flag form: ```bash amesa agent publish --path ./fermentation-controller/ ``` The path must point to the **outer kebab-case directory** containing `pyproject.toml`. {% endstep %} {% step %} ## Confirm the Publish ```bash amesa agent list ``` | Name | Type | Version | Description | UUID | | ----------------------- | ---------- | ------- | ----------------------------- | ---- | | fermentation-controller | controller | 1 | Proportional DO controller... | ... | | {% endstep %} | | | | | | {% endstepper %} | | | | | ## Using a Controller in an Orchestration Controllers can be registered in-process the same way as teachers: ```python from amesa_core import Orchestration, Agent, Sensor from fermentation_controller.controller import FermentationController orchestration = Orchestration() orchestration.add_sensors([ Sensor("dissolved_oxygen", "Dissolved oxygen fraction [0..1]"), Sensor("agitation_rpm", "Agitator speed in RPM"), ]) # Pass the CLASS β€” not an instance agent = Agent("fermentation-ctrl", FermentationController) orchestration.add_agent(agent) ``` ## Updating a Published Controller Increment `version` in `pyproject.toml` and republish: ```bash amesa agent publish ./fermentation-controller/ ``` ## Deleting a Controller ```bash amesa agent delete ``` ## Troubleshooting ### `TypeError` during episode reset Your `__init__` has a required argument. The framework calls `MyController()` with no arguments at every episode boundary: ```python # WRONG β€” threshold has no default, raises TypeError on reset def __init__(self, threshold: float): self.threshold = threshold # CORRECT β€” provide a default value def __init__(self, threshold: float = 0.65): self.threshold = threshold ``` ### `ModuleNotFoundError` on publish The entrypoint is using the kebab-case outer directory name: ```toml # WRONG entrypoint = "fermentation-controller.controller:FermentationController" # CORRECT entrypoint = "fermentation_controller.controller:FermentationController" ``` ### Action type mismatch at runtime `compute_action` must return a value in the format the simulator expects. Returning a plain `float` when the simulator expects a list, or vice versa, causes a coercion error. Check your simulator's action space and match the return type accordingly. ### Legacy import error ```python # WRONG from composabl import AgentController # CORRECT from amesa_core import AgentController ``` # Configure API Connections to Third-Party Software as Skill Agents In this tutorial, we will walk through how to set up an agent in AMESA that integrates with a third-party API. This type of integration allows your team of agents to communicate with external systems, such as machine performance APIs, and use the data to make informed decisions. We will create a **programmed agent** that connects to a mock third-party API, process its response, and return an action based on the data received. This tutorial will also touch on orchestrating this skill agent within your agent system. *** ### Step 1: Defining the Programmed Skill Agent A **programmed agent** in AMESA is created by specifying the logic for interacting with the external API and processing the response. In this case, we will create a simple API connection to a fake endpoint that returns data about machine performance. The agent system will act based on the information received. #### 1.1. Creating the API Integration Agent We’ll define a programmed agent for making the API request. Here's an example of how to define the agent using a controller function that calls the API and processes the response. ```python import requests from amesa import AgentController # Define the programmed skill class ThirdPartyAPISkill(AgentController): def __init__(self, *args, **kwargs): self.api_url = "https://api.example.com/machine-status" async def compute_action(self, obs, action): # Send sensor data to the third-party API response = self._call_api(obs) # Process the response and return an action action = self._process_response(response) return action def _call_api(self, observation): try: response = requests.post( self.api_url, json=observation, headers={'Content-Type': 'application/json'} ) response.raise_for_status() return response.json() except requests.RequestException as e: print(f"API call failed: {e}") return None def _process_response(self, response): if not response: # Default action return 0.0 action = float(response.get("action")) reason = response.get("reason", "No reason provided") print(f"Action: {action} - Reason: {reason}") return action async def transform_sensors(self, obs): return obs async def filtered_sensor_space(self): return ['sensor1', 'sensor2', 'sensor3'] async def compute_success_criteria(self, transformed_obs, action): return False async def compute_termination(self, transformed_obs, action): return False ``` In this example: * The `compute_action()` method sends observation data (e.g., from sensors) to a third-party API. * The `_call_api()` function makes the API call and handles any errors that might occur. * The `_process_response()` function processes the response from the third-party API and determines the appropriate action for the agent system to take based on the data. ### Step 2: Adding the Programmed Agent to the Orchestration #### 2.1. Adding the Agent to the AMESA Agent Orchestration Studio Once the agent is defined, you can add it to your orchestration using the methods below: 1. Create a new agent using the AMESA CLI with a given name and description and implementation type, that in this case will be a `controller`. The name will be "third\_party\_api\_skill" ```shell amesa agent new ``` 2. Change the `controller.py` code to use the class that you created: `ThirdPartyAPISkill()`. Change the `pyproject.toml` file to include your class `ThirdPartyAPISkill` in the entrypoint and its name: ```python [project] name = "Third Party API Skill Agent" entrypoint = "third_party_api_skill.controller:ThirdPartyAPISkill" ``` 3. Publish the Skill Agent to the UI ```shell amesa login ``` ```shell amesa agent publish third_party_api_skill ``` Select your organization and project that you want to publish it to. #### 2.2. Adding the Skill Agent to AMESA SDK Once the agent is defined, you can add it to your agent system using the `add_agent()` SDK method. This allows the agent system to execute the API connection skill agent when necessary. Here’s how to add the `ThirdPartyAPISkill` to the skill agent: ```python # Define and add the third-party API skill agent third_party_skill = Agent("third_party_api", ThirdPartyAPISkill) orchestration.add_agent(third_party_skill) ``` By importing and creating the class with `AgentController`, you are indicating that this skill agent is **programmed** and does not require training. It will use predefined logic to interact with the third-party API and make decisions based on the data returned. *** ### Conclusion By following these steps, you’ve successfully defined and integrated a **programmed agent** that communicates with a third-party API into your AMESA orchestration. The orchestration can now take actions based on external data and dynamically respond to scenarios. This approach allows agent systems to interface with a wide range of external systems, from monitoring equipment to adjusting machine settings, all through programmable skill agents. Orchestration of skill agents through orchestrators ensures the agent system executes the correct skill agents at the right time, whether the agents are learned or programmed. # Orchestrate Skill Agents Skill agents can be arranged in sequences or hierarchies, in skill groups, or as coordinated skills that output multiple decisions together. The orchestration structures reflect common design patterns that can be used to accelerate the design and creation of agent systems. ## Orchestrate Skills in Hierarchies and Sequences For some agent system designs, the task will be broken down into different skill agents that each control the system under certain conditions. For these agent systems, a special skill called an orchestrator chooses the right skill agent at the right time to accomplish the task. Orchestrators are the specialized supervisor skill agents that orchestrate the skill agents together, determining which skill agent to activate based on the conditions the system needs to respond to. To add an orchestrator to an agent system, drag the orchestrator into your agent system above the skills layer or drag more than one agent into the template side by side and a green orchestrator diamond will automatically appear.
You will then be prompted to configure the orchestrator. Orchestrators can be trained with DRL just like the other skill agents, and you set goals for them the same way. Click on Configure Orchestrator to be taken to the menu to set your orchestrator’s goals. Note that the goals of the agent’s top-level orchestrator should be the same as the goals for the agent system as a whole. You will also be prompted to choose between an additional set of options that correspond to two separate Machine Teaching design patterns. * **Fixed-order sequence**: perform the skills in a set order. This is used in the [Functional Pattern](#user-content-fn-1)[^1], a design pattern that is useful for tasks that involve fixed sequences of actions. * **Variable order sequence**: perform the skills in any order based on the orchestrator’s determination. This is used in the [Strategy Pattern](#user-content-fn-2)[^2], a design pattern that is useful for tasks that require different control strategies to be used in different situations or conditions. ## Orchestrate Skill Agents in Groups Unlike agent system designs that use an orchestrator to assign control to skills one at a time, agent systems with skill groups use skills working together to make decisions. Skill groups always consist of two or more skill agents. To create a skill group, simply drag the second skill under the first, and a skill group will automatically be created.
Skill groups are used for the [Plan-Execute Pattern](#user-content-fn-3)[^3], where one skill determines what the action should be and a second skill then β€œturns the knobs” to implement the decision. A common plan-execute design is the set point pattern, where a DRL agent skill determines what the set point of a system should be, and then a traditional automation technology (a controller that uses model predictive control or optimization) determines what actions are necessary to get the system to the set point. For example, in [the industrial mixer use case](/reference/sample-use-cases#industrial-mixer), the DRL plan skill agent decides what temperature the liquid in the tank should be. Then the MPC execute skill agent determines what temperature of coolant needs to be flowed around the tank to achieve the desired set point. In the industrial mixer example, the DRL skill agent is able to train effectively because the actions of the MPC controller are predictable. That means that it can practice and learn knowing that variations in performance are due to its own actions. In agent systems with multiple DRL skill agents arranged in plan-execute patterns, AMESA will always train the skills from the bottom to the top. In other words, the execute skill will have to achieve competence before the plan skill will start training. That allows each skill to effectively interpret the feedback from the system without confusion from each other. ## Orchestrate Coordinated Skills Some tasks require multiple skill agents to work together on a single decision, but in parallel rather than in sequence. Skill agents for these tasks use coordinated skills that learn to take action together toward a goal. Also known as Multi-Agent Training, coordinated skills are trained using a coach, rather than a teacher. ### Examples of Coordinated Skill Agents **Traffic Optimization**: Enhancing traffic flow and safety by teaching individual vehicles to navigate optimally and cooperate with each other. **Collaborative Robotics**: Enabling robots to work together on tasks such as assembly in manufacturing or coordination in logistics. **Smart Grids**: Optimizing energy distribution by having agents represent power plants, storage, and consumers to improve efficiency and stability. **Multiplayer Games**: Creating adaptive and intelligent NPCs that can offer dynamic challenges to players in competitive or cooperative game settings. **Communication Networks**: Improving network performance by optimizing resource allocation and traffic routing through agents representing network components. **Environmental Management**: Balancing economic, ecological, and social goals in land use and resource management by simulating stakeholders as agents. **Healthcare Logistics**: Strategizing resource allocation and treatment plans in scenarios like pandemics by considering the actions of hospitals, pharmacies, and patients as agents. **Supply Chain Optimization**: Minimizing costs and delivery times in supply chains by coordinating agents representing various stages of the supply chain process. ### Orchestrate Coordinated Skills with the SDK Coordinated skill agents are not yet available in the UI. In the SDK, we have expanded the API to integrate Coordinated Skills through the `add_coordinated_skill` method on your agent. This method accepts a new class that gets configured, named `CoordinatedSkill`, just as with the `Teacher` or `Controller` classes we implement this class by inheriting from the `Coach` class. The coordinated skill agent will now take the incoming observation and action spaces and pass it to the sub-skills as a shared environment observation and action taking. The sub-skills will then return their observations and actions, which will be passed back to the coordinated skill agent. The coordinated skill agent will then return the combined observations and actions to the agent system. ```python python # #################################################################################################### # Define the Coordinated Coach # #################################################################################################### class CoordinatedCoach(Coach): def __init__(self): self.counter = 0 def compute_reward(self, transformed_ sensors, action, sim_reward): """ Computes the reward for the given transformed observation and action :param transformed_ sensors: The transformed observation :param action: The actions dict :param sim_reward: The reward from the simulation :return: The reward, as a dictionary, with each key the sub-skill name and the value the reward """ self.counter += 1 return 1 def compute_success_criteria(self, transformed_ sensors, action): # keep the episodes short to make testing quicker return self.counter > 100 def compute_termination(self, transformed_ sensors, action): # keep the episodes short to make testing quicker return self.counter > 150 def transform_action(self, composabl_ sensors, action): return action # #################################################################################################### # Construct your Agent # #################################################################################################### s1 = Skill("skill1", IncrementTeacher) s2 = Skill("skill2", IncrementTeacher) a = Agent() a.add_coordinated_skill(CoordinatedSkill( "my-coordinated-skill", CoordinatedCoach, [s1, s2] )) ``` [^1]: Machine Teaching design pattern used to orchestrate skills in sequences [^2]: A Machine Teaching design pattern used to orchestrate skills in hierarchies [^3]: A Machine Teaching design pattern used to orchestrate skills in pairs that work together to make decisions # Deep Dive: Create and Publish Orchestrators This guide walks you through everything you need to build a Orchestrator from scratch and publish it to Amesa Orchestration Studio using the `amesa` CLI. By the end, you will have a working orchestrator packaged as an artifact and registered in the AMESA registry. ## What Is a Orchestrator? A **Orchestrator** is a meta-agent that decides which child agent executes at each timestep. When your orchestration has multiple agents, the orchestrator receives sensor observations and outputs an integer index that determines which child agent runs next. The selected child's action reaches the simulator β€” the orchestrator's index never does. Orchestrators enable agent **orchestrations** that decompose complex behavior into specialized sub-agents and switch between them dynamically. ``` Orchestration └── AgentOrchestrator ("my-orchestrator") ← chooses which agent runs β”œβ”€β”€ Agent-A ("stabilize") ← leaf agent β”œβ”€β”€ Agent-B ("accelerate") ← leaf agent └── Agent-C ("recover") ← leaf agent ``` The orchestrator outputs `0` β†’ Agent-A runs. `1` β†’ Agent-B runs. `2` β†’ Agent-C runs. ### Two Types of Orchestrator | Type | When to use | | --------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------- | | **Controller** (rule-based) | The selection logic is deterministic β€” you can write an explicit rule for which agent to choose given current sensor values. | | **Teacher** (RL-trained) | You want the agent to *learn* when to switch agents. The orchestrator trains as an RL policy with its own reward signal and success criteria. | Orchestrators use the same `AgentController` and `AgentTeacher` interfaces as regular agents. The key difference is what the action means: **for a orchestrator, the action is an integer index**, not a command to the simulator. {% stepper %} {% step %} ## Step 1: Understand the Orchestrator Interface {% tabs %} {% tab title="Controller Orchestrator" %} Implement `AgentController` with `compute_action` returning an integer index: ```python from typing import Dict, List from amesa_core import AgentController class MyOrchestrator(AgentController): def __init__(self): pass async def filtered_sensor_space(self, obs_spec) -> List[str]: # Which sensors does the selection logic need to read? ... async def compute_action(self, obs_spec: Dict, action) -> List[int]: # Return [index] where index selects the child agent to run # 0 β†’ first registered child, 1 β†’ second, etc. ... async def compute_success_criteria(self, obs_spec: Dict, action) -> bool: ... async def compute_termination(self, obs_spec: Dict, action) -> bool: ... ``` {% endtab %} {% tab title="Teacher Orchestrator" %} Implement `AgentTeacher` with `compute_reward` driving the selection policy: ```python from typing import Dict, List from amesa_core import AgentTeacher class MyOrchestrator(AgentTeacher): def __init__(self): pass async def filtered_sensor_space(self) -> List[str]: ... async def compute_reward(self, transformed_sensors: Dict, action, sim_reward: float) -> float: # Reward signal that teaches the orchestrator when to switch agents ... async def compute_success_criteria(self, transformed_sensors: Dict, action) -> bool: ... async def transform_action(self, transformed_sensors: Dict, action): # Return the action unchanged β€” the orchestrator's action is an index return action async def compute_termination(self, transformed_sensors: Dict, action) -> bool: return False ``` {% endtab %} {% endtabs %} {% hint style="info" %} **All methods are async.** Declare every method with `async def`. **`__init__` must accept zero required arguments.** The framework re-creates the orchestrator at the start of every episode. {% endhint %} {% endstep %} {% step %} ## Step 2: Write Your Orchestrator {% tabs %} {% tab title="Controller Orchestrator (Rule-Based)" %} The example below selects between three process control agents based on a `process_state` sensor: a startup agent, a steady-state agent, and a recovery agent. ```python # process_orchestrator/controller.py from typing import Dict, List from amesa_core import AgentController # Agent index mapping (matches registration order in the orchestration) SKILL_STARTUP = 0 SKILL_STEADY_STATE = 1 SKILL_RECOVERY = 2 # process_state thresholds WARMUP_THRESHOLD = 0.2 # below this β†’ still starting up FAULT_THRESHOLD = 0.05 # below this β†’ fault recovery needed class ProcessOrchestrator(AgentController): """ Rule-based orchestrator for a three-agent process control orchestration. Chooses startup, steady-state, or recovery based on process_state. """ def __init__(self): pass async def filtered_sensor_space(self, obs_spec) -> List[str]: return ["process_state", "temperature", "pressure"] async def compute_action(self, obs_spec: Dict, action) -> List[int]: state = obs_spec["process_state"] if state < FAULT_THRESHOLD: return [SKILL_RECOVERY] elif state < WARMUP_THRESHOLD: return [SKILL_STARTUP] else: return [SKILL_STEADY_STATE] async def compute_success_criteria(self, obs_spec: Dict, action) -> bool: return obs_spec["process_state"] >= 0.9 async def compute_termination(self, obs_spec: Dict, action) -> bool: return False ``` {% endtab %} {% tab title="Teacher Orchestrator (RL-Trained)" %} The example below trains the orchestrator with reinforcement learning. The reward encourages it to keep `process_state` high, and the RL policy learns which agent combinations achieve that over time. ```python # process_orchestrator/teacher.py from typing import Dict, List from amesa_core import AgentTeacher class ProcessOrchestrator(AgentTeacher): """ RL-trained orchestrator for a three-agent process control orchestration. Reward drives the policy to maintain high process_state. """ SUCCESS_THRESHOLD = 0.9 FAIL_THRESHOLD = 0.05 def __init__(self): pass async def filtered_sensor_space(self) -> List[str]: return ["process_state", "temperature", "pressure"] async def compute_reward(self, transformed_sensors: Dict, action, sim_reward: float) -> float: # Reward the orchestrator for keeping process_state high return float(transformed_sensors["process_state"]) async def compute_success_criteria(self, transformed_sensors: Dict, action) -> bool: return transformed_sensors["process_state"] >= self.SUCCESS_THRESHOLD async def compute_termination(self, transformed_sensors: Dict, action) -> bool: return transformed_sensors["process_state"] < self.FAIL_THRESHOLD async def transform_action(self, transformed_sensors: Dict, action): # The orchestrator's action is a agent index β€” return it unchanged return action ``` {% endtab %} {% endtabs %} {% endstep %} {% step %} ## Step 3: Create the Artifact Directory {% tabs %} {% tab title="Controller Orchestrator" %} ```bash amesa orchestrator new \ --name process-orchestrator \ --type controller \ --description "Rule-based orchestrator for three-agent process control orchestration" \ --location ./ ``` {% endtab %} {% tab title="Teacher Orchestrator" %} ```bash amesa orchestrator new \ --name process-orchestrator \ --type teacher \ --description "RL-trained orchestrator for three-agent process control orchestration" \ --location ./ ``` {% endtab %} {% endtabs %} Both create: ``` process-orchestrator/ process_orchestrator/ __init__.py ← empty; required controller.py ← or teacher.py, depending on type pyproject.toml ``` ### Creating Manually ```bash mkdir -p process-orchestrator/process_orchestrator touch process-orchestrator/process_orchestrator/__init__.py touch process-orchestrator/process_orchestrator/controller.py # or teacher.py touch process-orchestrator/pyproject.toml ``` ### Naming Rules | Layer | Convention | Example | | ------------------- | ------------- | ------------------------------- | | Outer directory | kebab-case | `process-orchestrator` | | Inner Python module | snake\_case | `process_orchestrator` | | Source file | by convention | `controller.py` or `teacher.py` | | `__init__.py` | always empty | *(no content)* | | {% endstep %} | | | {% step %} ## Step 4: Configure `pyproject.toml` {% tabs %} {% tab title="Controller Orchestrator" %} ```toml [build-system] requires = ["setuptools"] build-backend = "setuptools.backends.legacy:build" [project] name = "process-orchestrator" version = "0.1.0" description = "Rule-based orchestrator for three-agent process control orchestration" dependencies = [ "amesa-core", ] [amesa] type = "orchestrator-controller" entrypoint = "process_orchestrator.controller:ProcessOrchestrator" ``` {% endtab %} {% tab title="Teacher Orchestrator" %} ```toml [build-system] requires = ["setuptools"] build-backend = "setuptools.backends.legacy:build" [project] name = "process-orchestrator" version = "0.1.0" description = "RL-trained orchestrator for three-agent process control orchestration" dependencies = [ "amesa-core", ] [amesa] type = "orchestrator-teacher" entrypoint = "process_orchestrator.teacher:ProcessOrchestrator" ``` {% endtab %} {% endtabs %} ### `type` Values | Orchestrator type | `[amesa] type` value | | ----------------------- | --------------------------- | | Rule-based (controller) | `"orchestrator-controller"` | | RL-trained (teacher) | `"orchestrator-teacher"` | {% hint style="warning" %} **Do not add a `variables` field.** The `variables` field is for perceptors only. {% endhint %} ### Entrypoint Format | Part | Value (controller example) | | -------------- | --------------------------- | | `inner_module` | `process_orchestrator` | | `filename` | `controller` (or `teacher`) | | `ClassName` | `ProcessOrchestrator` | **Full entrypoint:** `process_orchestrator.controller:ProcessOrchestrator` {% endstep %} {% step %} ## Step 5: Verify Your Directory ``` process-orchestrator/ β”œβ”€β”€ process_orchestrator/ β”‚ β”œβ”€β”€ __init__.py ← must exist; must be empty β”‚ └── controller.py ← (or teacher.py) contains ProcessOrchestrator class └── pyproject.toml ← [amesa] type is "orchestrator-controller" or "orchestrator-teacher" ``` Checklist: * [ ] `__init__.py` exists and is empty * [ ] `pyproject.toml` is in the outer directory * [ ] `type` in `[amesa]` is `"orchestrator-controller"` or `"orchestrator-teacher"` * [ ] `entrypoint` uses the inner snake\_case module name * [ ] `compute_action` returns `[index]` β€” a list with a single integer * [ ] Index values match the registration order of child agents in the orchestration * [ ] All methods are declared `async def` * [ ] `__init__` takes no required arguments * [ ] No `variables` field in `[amesa]` * [ ] All imports use `from amesa_core import ...` {% endstep %} {% step %} ## Step 6: Publish ```bash amesa orchestrator publish ./process-orchestrator/ ``` Or using the flag form: ```bash amesa orchestrator publish --path ./process-orchestrator/ ``` {% endstep %} {% step %} ## Step 7: Confirm the Publish ```bash amesa orchestrator list ``` | Name | Type | Version | Description | UUID | | -------------------- | ---------- | ------- | -------------------------- | ---- | | process-orchestrator | controller | 1 | Rule-based orchestrator... | ... | | {% endstep %} | | | | | | {% endstepper %} | | | | | ## Using a Orchestrator in an Orchestration Register the orchestrator and its child agents in the orchestration. The child agents must be registered **before** the orchestrator, and the integer indices in `compute_action` must match registration order. ```python from amesa_core import Orchestration, Agent, Sensor, AgentOrchestrator from startup_agent.teacher import StartupTeacher from steady_state_agent.teacher import SteadyStateTeacher from recovery_agent.teacher import RecoveryTeacher from process_orchestrator.controller import ProcessOrchestrator orchestration = Orchestration() orchestration.add_sensors([ Sensor("process_state", "Overall process health [0..1]"), Sensor("temperature", "Process temperature"), Sensor("pressure", "Process pressure"), ]) # Define child agents β€” order determines the index the orchestrator uses agent_a = Agent("startup", StartupTeacher) # index 0 agent_b = Agent("steady-state", SteadyStateTeacher) # index 1 agent_c = Agent("recovery", RecoveryTeacher) # index 2 # Define the orchestrator with its child agents orchestrator = AgentOrchestrator("process-orchestrator", ProcessOrchestrator, [agent_a, agent_b, agent_c]) orchestration.add_agent(orchestrator) ``` > **Index order matters.** `compute_action` returning `[0]` runs `agent_a`. Returning `[1]` runs `agent_b`. The index is a zero-based position in the child agents list passed to `AgentOrchestrator`. ## Updating a Published Orchestrator Increment `version` in `pyproject.toml` and republish: ```bash amesa orchestrator publish ./process-orchestrator/ ``` ## Deleting a Orchestrator ```bash amesa orchestrator delete ``` ## Troubleshooting
Wrong agent selected at runtime Check that the integer indices returned by `compute_action` match the order in which child agents are passed to `AgentOrchestrator(...)`. The first child is index `0`, the second is `1`, and so on. Returning an out-of-range index causes a runtime error.
`TypeError` during episode reset `__init__` has a required argument. Provide a default value for every parameter: ```python # WRONG def __init__(self, threshold: float): ... # CORRECT def __init__(self, threshold: float = 0.2): ... ```
`ModuleNotFoundError` on publish The entrypoint is using the kebab-case outer directory name instead of the inner snake\_case module name: ```toml # WRONG entrypoint = "process-orchestrator.controller:ProcessOrchestrator" # CORRECT entrypoint = "process_orchestrator.controller:ProcessOrchestrator" ```
Wrong `[amesa] type` Orchestrator types are different from agent types. Use `"orchestrator-controller"` or `"orchestrator-teacher"`, not `"agent-controller"` or `"agent-teacher"`. ```toml # WRONG type = "agent-controller" # CORRECT type = "orchestrator-controller" ```
# Add a Perception Layer Adding perception modules to your agent can provide richer, complex, condensed, and nuanced information to the decision-making parts of the agent system. For example, you might include a computer vision model in your perception layer that inputs images or video from a camera and outputs classifications of objects that it identifies. You can also add large language models as perceptors to take in and interpret information in natural language. Each module in the perception layer for a AMESA agent system inputs the sensor variables, processes those variables in some way, and outputs one or more *new variables* that the platform will automatically add to the list of sensors. Perceptors can use any supported Python function or library to calculate outputs. They can even call machine learning and large language models or their APIs. The next three pages explain how to use the SDK and CLI workflow to create new perceptors or configure existing models as perceptors to use in AMESA agent systems. ## Add Perceptors to Agent Systems Just like skill agents, perceptors can be dragged and dropped into agent systems using the UI. Perceptors will always be situated in the Perception layer that comes before orchestrators and skill agents. That’s because perception needs to be applied to the sensor inputs to create new variables that are then passed to the skills layer for the agent system to use in decision-making.
# Create and Publish a Perceptor This guide walks you through everything you need to build a Perceptor from scratch and publish it to Amesa Orchestration Studio using the `amesa` CLI. By the end, you will have a working perceptor packaged as an artifact and registered in the AMESA registry. ## What Is a Perceptor? A **Perceptor** transforms raw sensor observations into derived, computed features. It sits between the sensor layer and your agents: it receives the raw sensor dictionary, computes new values from it, and injects those new keys into the observation namespace before any agent sees the data. Agents can reference perceptor outputs the same way they reference sensor names β€” by listing the key in `filtered_sensor_space()`. Perceptors are registered on the Orchestration and run for every agent. **Common use cases:** * Rate-of-change / derivative of a sensor value * Running averages and sliding-window statistics * Threshold flags (is this sensor in spec?) * Composite metrics derived from two or more sensors ### Data Flow ``` Simulator β”‚ raw observations β–Ό Sensor mapping β†’ { "temperature": 82.3, "pressure": 1.1, ... } β”‚ β–Ό Perceptor pipeline (registered order; each adds new keys) β”‚ perceptor 1 adds { "efficiency_ratio": 0.91 } β”‚ perceptor 2 adds { "quality_index": 0.87 } β–Ό Agent teachers and controllers (can reference all keys, including perceptor outputs) ``` {% stepper %} {% step %} ## Understand the Perceptor Methods A Perceptor is a subclass of `PerceptorImpl` from `amesa_core`. You must implement exactly two methods. ### Required Methods | Method | Sync/Async | What it does | | ---------------------------- | ---------- | ------------------------------------------------------------------------------------------------------ | | `compute(obs_spec, obs)` | **async** | Called every step. Returns a `dict` of new keys to inject into the observation namespace. | | `filtered_sensor_space(obs)` | **sync** | Returns the list of raw sensor names this perceptor reads. Used for shape inference at initialization. | > **`filtered_sensor_space` must be sync.** Do **not** declare it `async`. Declaring it async will break space construction at training initialization. > **`compute` must be async.** Declare it with `async def`. ### Method Signatures ```python from amesa_core import PerceptorImpl class MyPerceptor(PerceptorImpl): def __init__(self): # Must accept zero required arguments # All self.* state is wiped at every episode reset pass async def compute(self, obs_spec, obs) -> dict: # obs is the full sensor dict: { "sensor_name": value, ... } # Return a dict of NEW keys to add to the observation namespace ... def filtered_sensor_space(self, obs) -> list: # Return the list of sensor names this perceptor reads # Note: sync, not async ... ``` ### The `compute()` Parameters | Parameter | Type | Description | | ---------- | --------------- | -------------------------------------------------------------------------------------------------------------- | | `obs_spec` | `Space \| None` | Gymnasium Space spec. May be `None` in some call paths β€” do not depend on it being a valid Space object. | | `obs` | `dict` | The full named sensor dict after lambda extraction, plus outputs from any perceptors that ran before this one. | **Return value:** a `dict` of new `{ key: value }` pairs. Every key must: 1. **Not** already exist in the observation dict β€” a collision with an existing sensor name raises an error at training initialization 2. **Match** the `variables` list declared in `pyproject.toml` > **Episode state:** `__init__` is called fresh at the start of each episode. All `self.*` state is wiped at every episode reset. Do not rely on state persisting across episodes. > {% endstep %} {% step %} ## Write Your Perceptor The example below computes two derived metrics from raw process sensors: an `efficiency_ratio` (the running fraction of steps where throughput meets a threshold) and a `quality_index` (the ratio of output rate to input rate). ```python # process_monitor/perceptor.py from amesa_core import PerceptorImpl EFFICIENCY_THRESHOLD = 0.75 class ProcessMonitorPerceptor(PerceptorImpl): """ Computes two derived metrics from raw process sensors. Reads: throughput, output_rate, input_rate Outputs: efficiency_ratio, quality_index """ def __init__(self): # __init__ must take no required arguments # State is reset at the start of every episode self._total_steps = 0 self._efficient_steps = 0 async def compute(self, obs_spec, obs) -> dict: throughput = obs["throughput"] output_rate = obs["output_rate"] input_rate = obs["input_rate"] self._total_steps += 1 if throughput >= EFFICIENCY_THRESHOLD: self._efficient_steps += 1 efficiency_ratio = self._efficient_steps / self._total_steps quality_index = min(1.0, output_rate / input_rate) if input_rate > 0 else 0.0 return { "efficiency_ratio": efficiency_ratio, "quality_index": quality_index, } def filtered_sensor_space(self, obs) -> list: # Sync β€” not async return ["throughput", "output_rate", "input_rate"] ``` ### Common `compute()` Patterns **Step-over-step delta (derivative):** ```python async def compute(self, obs_spec, obs) -> dict: current = obs["process_var"] delta = current - self._last if self._last is not None else 0.0 self._last = current return {"process_var_delta": delta} ``` **Running average:** ```python async def compute(self, obs_spec, obs) -> dict: self._window.append(obs["temperature"]) if len(self._window) > self._window_size: self._window.pop(0) return {"temperature_avg": sum(self._window) / len(self._window)} ``` **Threshold flag:** ```python async def compute(self, obs_spec, obs) -> dict: in_spec = 1.0 if 0.08 <= obs["output_metric"] <= 0.09 else 0.0 return {"in_spec_flag": in_spec} ``` {% endstep %} {% step %} ## Create the Artifact Directory ### Using the CLI Scaffold (Recommended) ```bash amesa perceptor new \ --name process-monitor \ --description "Computes efficiency_ratio and quality_index from process sensors" \ --location ./ ``` This creates: ``` process-monitor/ process_monitor/ __init__.py ← empty; required perceptor.py ← your PerceptorImpl subclass goes here pyproject.toml ``` Replace the placeholder `perceptor.py` with your implementation from Step 2. ### Creating Manually ```bash mkdir -p process-monitor/process_monitor touch process-monitor/process_monitor/__init__.py touch process-monitor/process_monitor/perceptor.py touch process-monitor/pyproject.toml ``` ### Naming Rules | Layer | Convention | Example | | ------------------- | ------------- | ----------------- | | Outer directory | kebab-case | `process-monitor` | | Inner Python module | snake\_case | `process_monitor` | | Source file | by convention | `perceptor.py` | | `__init__.py` | always empty | *(no content)* | The outer directory is used by the CLI only. The inner snake\_case directory is the Python module referenced in `pyproject.toml`. Do not use the kebab-case outer name in your entrypoint β€” it is not a valid Python identifier. {% endstep %} {% step %} ## Configure `pyproject.toml` ```toml [build-system] requires = ["setuptools"] build-backend = "setuptools.backends.legacy:build" [project] name = "process-monitor" version = "0.1.0" description = "Computes efficiency_ratio and quality_index from process sensors" dependencies = [ "amesa-core", ] [amesa] type = "perceptor" entrypoint = "process_monitor.perceptor:ProcessMonitorPerceptor" variables = ["efficiency_ratio", "quality_index"] ``` ### Field Reference | Field | Where | Required | Description | | -------------- | ----------- | ------------------------- | ------------------------------------------------------------------------------ | | `name` | `[project]` | Yes | Artifact name as it will appear in the registry. Kebab-case. | | `version` | `[project]` | Yes | Semantic version string. | | `description` | `[project]` | Yes | Short human-readable description. | | `dependencies` | `[project]` | Yes | Must include `"amesa-core"`. | | `type` | `[amesa]` | Yes | Must be `"perceptor"`. | | `entrypoint` | `[amesa]` | Yes | `inner_module.filename:ClassName` | | `variables` | `[amesa]` | **Yes β€” perceptors only** | List of output key names. Must exactly match the keys returned by `compute()`. | > **`variables` is required for perceptors.** This is what the AMESA registry uses to know what named outputs the perceptor exposes. Missing or mismatched `variables` causes publish validation to fail or training initialization to fail. ### Entrypoint Format | Part | Value in this example | | -------------- | ----------------------------------------------- | | `inner_module` | `process_monitor` (inner snake\_case directory) | | `filename` | `perceptor` (the `.py` file, without extension) | | `ClassName` | `ProcessMonitorPerceptor` | **Full entrypoint:** `process_monitor.perceptor:ProcessMonitorPerceptor` {% endstep %} {% step %} ## Verify Your Directory ``` process-monitor/ β”œβ”€β”€ process_monitor/ β”‚ β”œβ”€β”€ __init__.py ← must exist; must be empty β”‚ └── perceptor.py ← contains ProcessMonitorPerceptor class └── pyproject.toml ← contains [amesa] section with type, entrypoint, and variables ``` Checklist: * [ ] `__init__.py` exists and is empty * [ ] `pyproject.toml` is in the outer directory (same level as `process_monitor/`) * [ ] `type` in `[amesa]` is `"perceptor"` * [ ] `variables` list matches the dict keys returned by `compute()` * [ ] `entrypoint` uses the inner snake\_case module name * [ ] `filtered_sensor_space` is declared `def`, **not** `async def` * [ ] `compute` is declared `async def` * [ ] `__init__` takes no required arguments * [ ] All imports use `from amesa_core import ...` {% endstep %} {% step %} ## Publish ```bash amesa perceptor publish ./process-monitor/ ``` Or using the flag form: ```bash amesa perceptor publish --path ./process-monitor/ ``` The path must point to the **outer kebab-case directory** containing `pyproject.toml`. {% endstep %} {% step %} ## Confirm the Publish ```bash amesa perceptor list ``` This prints a table of all perceptors in the selected project: | Name | Version | Description | UUID | | ---------------- | ------- | ------------------------------------------------ | ---- | | process-monitor | 1 | Computes efficiency\_ratio and quality\_index... | ... | | {% endstep %} | | | | | {% endstepper %} | | | | ## Attaching a Perceptor to an Orchestration Once published, a perceptor can also be used in-process (before packaging) while developing: ```python from amesa_core import Orchestration, Sensor, Perceptor from process_monitor.perceptor import ProcessMonitorPerceptor orchestration = Orchestration(id="process-orchestration") orchestration.add_sensors([ Sensor("throughput", "Process throughput [0..1]"), Sensor("output_rate", "Output units per step"), Sensor("input_rate", "Input units per step"), ]) # Pass the CLASS β€” not an instance orchestration.add_perceptor(Perceptor("process-monitor", ProcessMonitorPerceptor)) # Agents can now reference "efficiency_ratio" and "quality_index" # in their filtered_sensor_space() ``` Perceptors run in registration order. Each receives the full observation dict including outputs from perceptors that ran before it. ## Updating a Published Perceptor Increment the `version` in `pyproject.toml` and republish: ```bash amesa perceptor publish ./process-monitor/ ``` ## Deleting a Perceptor ```bash amesa perceptor delete ``` The CLI presents an interactive list. Select the perceptor to remove and confirm. ## Troubleshooting ### Output keys don't match `variables` in `pyproject.toml` The `variables` list in `[amesa]` must exactly match the dict keys returned by `compute()`. A mismatch causes training initialization to fail when the orchestration tries to resolve perceptor outputs by name. ```toml # pyproject.toml declares: variables = ["efficiency_ratio", "quality_index"] ``` ```python # compute() must return exactly these keys: return { "efficiency_ratio": 0.91, # βœ“ matches "quality_index": 0.87, # βœ“ matches } # NOT: return {"efficiency": 0.91} # βœ— wrong key name β€” training init will fail ``` ### `filtered_sensor_space` is async β€” training fails at init `filtered_sensor_space` must be a regular synchronous method. Making it `async` breaks the space construction step that runs before training starts. ```python # WRONG async def filtered_sensor_space(self, obs) -> list: return ["throughput"] # CORRECT def filtered_sensor_space(self, obs) -> list: return ["throughput"] ``` ### Key collision with existing sensor name If `compute()` returns a key that already exists in the sensor dict, AMESA raises an error at training initialization. Choose output key names that are distinct from all sensor names registered on the Orchestration. ### Legacy import error ```python # WRONG from composabl import PerceptorImpl # CORRECT from amesa_core import PerceptorImpl ``` # Create a New Perceptor Perceptors use the SDK and CLI workflow. To access a template for a perceptor, type `composabl perceptor new` into the CLI. AMESA will then generate a perceptor template that you can populate with your information. In this simple perceptor example we calculate the perceptor outputs that will be added as new sensor variables and we create a list of perceptors that comprise the perception layer. ```python python class DeltaCounter(): def __init__(self): self.key = "state1" self.previous_value = None def compute(self, sensors): if self.previous_value is None: self.previous_value = sensors[self.key] return {"delta_counter": 0, "state2": 0} delta = sensors ["state1"] - self.previous_value self.previous_value = sensors["state1"] return {"delta_counter": delta, "state2": 0} def filtered_sensor_space(self, sensors): return ["state1"] delta_counter = Perceptor(["delta_counter", "state2"], DeltaCounter, "the change in the counter from the last two steps") ``` # Configure an ML Model as a Perceptor In this tutorial, we will walk through how to integrate a trained machine learning (ML) model into your AMESA agent system as a **Perceptor**. A perceptor allows your agent system to interpret data from sensors, process it using a machine learning model, and output new variables that will help the agent system make better decisions. The goal is to publish a pre-trained ML model as a perceptor that adds a new layer of perception to your agent system, enabling it to process sensor data in a more advanced way. This could be useful in a variety of scenarios, such as predictive maintenance, anomaly detection, or autonomous decision-making. *** ### Step 1: Understanding the Perceptor A **Perceptor** in AMESA is a module in the perception layer that inputs sensor data, processes it (potentially using an ML model), and outputs new variables that are automatically added to the list of available sensors. For this example, let’s assume we are building a perceptor that uses a trained machine learning model to predict **thermal runaway** in a system. *** ### Step 2: Setting Up the Trained Model We will use a pre-trained ML model stored as a pickle file to predict thermal runaway based on certain temperature and chemical sensor readings. Here’s how to set up the trained ML model for use as a perceptor. 1. **Store the ML Model**: Assume the ML model has been trained and saved as a `.pkl` file. For this example, the model is stored in the path: `ml_models/ml_predict_temperature.pkl`. 2. **Load the ML Model in the Perceptor**: In the perceptor class, we will load the model and define how it processes the sensor data. *** ### Step 3: Creating the Perceptor Now, we’ll create the perceptor using the trained ML model to process the sensor data and predict thermal runaway events. The perceptor will be responsible for calling the model and returning the prediction as a new sensor variable. We can start by creating the preceptor by using the AMESAe CLI with the following command: ```shell amesa preceptor new ``` The new preceptor will have the following file structure: ``` .└── perceptor_name/ β”œβ”€β”€ perceptor_name/ β”‚ β”œβ”€β”€ __init__.py β”‚ └── perceptor.py β”œβ”€β”€ pyproject.toml └── README.md ``` #### 3.1. Configuring your pyproject.toml file ```python [project] name = "perc" # Keep this as the same name as the directory version = "0.1.0" description = "perc" # You can change this to a description of your project authors = [ { name = "John Doe", email = "john.doe@amesa.com" }, ] # Change this to your name and email dependencies = [ "amesa-core", "", # Add any third-party dependencies here ] [composabl]# This metadata should not be changed with exption of the entrypoint # after the : marker if you change the name of the class. Those names # should match. type = "perceptor"entrypoint = "perc.perceptor:DemoPerceptor" ``` #### 3.2. Implementing the Perceptor in the perceptor.py file Here’s the Python code to create the perceptor: ```python import pickle from amesa import Perceptor, PerceptorImpl from sensors import sensors # Assuming sensors are already defined class ThermalRunawayPredict(PerceptorImpl): def __init__(self, *args, **kwargs): self.last_Tc = 0 # Store the last observed temperature self.ml_model = pickle.load(open("ml_models/ml_predict_temperature.pkl", 'rb')) # Load the trained ML model async def compute(self, obs_spec, obs): # Convert the observation to a dictionary using sensor names if type(obs) != dict: obs_keys = [s.name for s in sensors] obs = dict(zip(obs_keys, obs)) # Calculate delta temperature (change in Tc) if self.last_Tc == 0: delta_Tc = 5 # Initial change else: delta_Tc = float(obs['Tc']) - self.last_Tc # Prepare the input for the ML model X = [[float(obs['Ca']), float(obs['T']), float(obs['Tc']), delta_Tc]] # Get the prediction from the ML model prediction = self.ml_model.predict(X)[0] # Update last_Tc for the next iteration self.last_Tc = float(obs['Tc']) # Return the prediction as a new sensor variable return {"thermal_runaway_predict": prediction} def filtered_sensor_space(self, obs): # Define which sensor values the ML model needs return ['T', 'Tc', 'Ca'] ``` In this perceptor: * We load the trained machine learning model from a pickle file. * The `compute()` method takes in sensor data (e.g., temperature, chemical concentrations), processes it, and uses the ML model to predict whether a thermal runaway event will occur. * The perceptor outputs the prediction as a new sensor variable, `thermal_runaway_predict`. #### 3.2. Adding the Perceptor to Your Agent System Once the perceptor is defined, you can login to the AMESA editor and add it to your agent system. *** ### Conclusion In this tutorial, we covered how to publish a trained ML model as a perceptor in AMESA. This allows the agent system to integrate more advanced decision-making by processing raw sensor data through a machine learning model and outputting predictions as new sensor variables. This method can be applied in various domains, such as predictive maintenance, anomaly detection, and control systems. # Configure an LLM Model as a Perceptor You can use an LLM as a perceptor[^1] to add language and communication capabilities to your agent system. This allows you to create human-like assistants or copilots who can contribute natural language capabilities to your agent system. AMESA has several personas for LLM assistants to help structure your agent design. * The **analyst** interprets sensor data and passes it to an interface that the user can access, allowing real-time monitoring of conditions and the agent system's responses. * The **executive** reads external data sources in text and reports information to the agent system, such as trends in the business press that would help to anticipate demand for a product * The **plant manager** allows operators to communicate directly with the agent system and gives it instructions based on information that would not be otherwise available in its sensor space LLM perceptors can either: 1. Output language to the operator about what the agent system is doing (ex. the analyst) 2. Take in inputs in natural language and then transform them into information that the decision-making layer of the agent can use (ex. the executive and plant manager) ## Create an LLM Perceptor ### Step 1: Create the perceptor shell From the CLI, when logged into AMESA, type `composabl perceptor new`. You will be prompted for a location to save your new perceptor, and then a new directory with your perceptor will be created. This will include the `pyproject.toml` file that will allow you to publish the perceptor to the UI once it is created. ### Step 2: Define the perceptor class Within the `perceptor.py` file, create the API call and prompt for the LLM. #### Analyst Perceptor Code Sample The analyst displays information to the human user, but doesn't send information to the decision-making layer of the agent, so the perceptor returns `0.` ```python from fake_llm import llm_client from fake_factory_console import factory_console_client from amesa_core import PerceptorImpl class AnalystPerceptor(PerceptorImpl): """ The analyst type that displays information to the human operators but doesn't send any information to the agent. """ def __init__(self, *args, **kwargs): # Example: self.llm_client = llm_client() self.factory_console_client = factory_console_client() pass async def compute(self, obs_spec, obs): # First, ask the LLM for its thoughts on the current state of the plant llm_response = self.llm_client.ask(f"You are controlling a CSTR plant, the current state of the plant is {obs}. What are your thoughts on the current state of the plant?") # Second, post the LLM's thoughts to the factory console for a human to read self.factory_console_client.post(f"The LLM thoughts on the current state of the plant are: {llm_response}") return {"chemical_engineer_llm": 0} ``` #### Executive Code Sample The executive in this sample related to the industrial mixer use case automatically queries a chemical engineering LLM for advice about control actions to take. The perceptor returns an action that it recommends the decision-making layer of the skill agent to take. This becomes a new sensor variable that the skill agent teacher(s) will take into account when training the agent system in simulation. ```python from fake_llm import llm_client from amesa_core import PerceptorImpl class ChemicalEngineerPerceptor(PerceptorImpl): """ The perceptor for the text agent """ def __init__(self, *args, **kwargs): self.llm_client = llm_client() pass async def compute(self, obs_spec, obs): """ Asks the LLM for its thoughts on the current state of the plant, and returns a recommended action """ llm_response = self.llm_client.ask(f"You are controlling a CSTR plant, the current state of the plant is {obs}. what action do you recommend?") llm_action = llm_response.find("action") return {"chemical_engineer_llm": llm_action} ``` #### Examples and Reference [See full code samples and more examples.](https://github.com/AMESA/examples.composabl.io/tree/main/llm_as_a_perceptor) ### Step 3: Filter the Sensor Space AMESA agent systems can include text fields in perceptors, but they must be transformed or filtered out in the `teacher.py` file before training with DRL. For any text variables that are not transformed into a different data type, use the `filtered_sensor_space` method of the teacher to remove them. ### Step 4: Publish the Perceptor Publish the perceptor to the UI. ```bash amesa login ``` Naviage to the folder above your perceptor. Then publish your perceptor. ```bash amesa perceptor publish foldername ``` Select the organization and project that your perceptor to add your perceptor. The refresh your Agent Orchestration Studio to see the perceptor and add it to agents. [^1]: An ML model or other algorithm that interprets sensor data # Import Agents to the Agent Orchestration Studio ## Import Skill Agents into AMESA AMESA makes it easy to **import existing logic and controllers**, from MPC and PID controllers to custom Python applications and machine learning models. Once imported, these agents can be reused across your use cases and visually orchestrated in the Agent Orchestration Studio. Whether you’ve built your logic using AMESA’s Agent Training Library or you're wrapping existing IP (e.g., LLMs, external APIs, heuristics), you can import it into AMESA and make it available for drag-and-drop integration. You can import agents[^1], orchestrators[^2], and perceptors[^3]. *** ### Why Import Agents? Many organizations already rely on proven control strategies, models, or code. Importing them into AMESA allows you to: * **Re-use your existing IP** in a composable agent system * **Combine logic-based and learning-based agents** (e.g., hybrid PID + DRL setups) * **Standardize deployment workflows** across your team or organization *** ### How to Import Skill Agents and Orchestrators 1. **Log in to AMESA** ```bash amesa login ``` You’ll be redirected to the browser to enter your credentials. 2. Navigate to your agent directory Change to the folder containing the skill agent or orchestrator you want to import. 3. Import the component * For an agent: ``` amesa agent publish agent-name ``` * For an orchestrator: ``` amesa orchestrator publish orchestrator-name ``` 4. Choose where to save it * Select your organization * Select the project or use case 5. Finish importing Once the import is complete: * Open the UI * Go to Agent Orchestration Studio * Refresh your browser Your new skill agent or orchestrator will appear in the sidebar, ready to use. ### Importing Perceptors 1. Navigate to the Perceptors Directory Move to the folder above the individual perceptor you want to import. 2. Import a Perceptor ``` composabl perceptor publish perceptor-name ``` 3. Select Your Organization and Use Case 4. Complete the Import After import: * Go to the UI * Open Agent Orchestration Studio * Refresh the page Your perceptor will appear in the sidebar and can be connected to agents. *** ### What Can You Import? You can import nearly any Python-based logic as a skill agent, including: * PID or MPC controllers * LLM-based agents * Business rule engines * ML models or custom inference code * External API wrappers Once imported, these agents can be composed into intelligent multi-agent systems alongside AMESA-native components. [^1]: Modules within a multi-agent system that make decisions to complete all or part of a task [^2]: A special type of skill agent that assigns decision-making control to the right decision-making skill agent based on current conditions [^3]: An ML model or other algorithm that interprets sensor data # Submit a Training Job through the UI ## About Cluster Training AMESA agent systems use [Kubernetes](https://kubernetes.io/) clusters to train at scale. A cluster is a collection of computers that work on large tasks simultaneously. This provides enough compute to complete large training tasks as efficiently as possible. AMESA offers two options for cluster training: * Use AMESA's Training as a Service offering to train on our clusters * Use your own compute clusters through Azure, AWS, or another provider ## Ensure that Your Agent System is Ready Before you submit your job for training on a cluster, make sure that your agent is fully configured and all the parameters have been set. That means checking all the agent components: * Goals * Perceptors * Orchestrators, including goals for learned selectors and scenarios * Skill agents, including goals for learned skills * Scenarios, including scenario flows
Any component of the agent with a warning sign is not fully configured and not ready for training. Go back to edit that agent component and make sure that all of the fields are filled out. ## Choose the Right Cluster You can train on your own cluster or on AMESA’s clusters using training as a service (TaaS) credits. If you want to use AMESA’s clusters, ensure that you have credits available. To train on your own cluster, make sure that you have [set your cluster up ](/training-infrastructure/creating-a-cluster)and installed AMESA successfully. Click Train and then choose the cluster option in the menu. You will then have the option to configure your training session.
## Configure Your Training Session Training session configuration options are the same whether you’re using TaaS or training on your own cluster. ### Set the Number of Training Cycles A training cycle is a complete pass through the entire task, with the agent system continuing until it reaches success or some other stop criteria. Your agent system will train each skill one at a time for the selected number of training cycles, starting from the bottom of the agent system design. A training cycle involves about 1,000 agent decisions. Depending on the complexity of the task, agent systems may need to complete anywhere between 100 and several thousand training cycles to become proficient. ### Set the Number of Simulators You can run multiple simulators in parallel to speed up training. If you run more than one simulator during a training, the number of training cycles selected will be multiplied by the number of simulators, so 5 training cycles with 3 simulators selected would lead to 15 training cycles total. You can use the Advanced Configuration to choose how powerful each machine running a simulator should be. If you choose Small, each training cycle selected will result in one training cycle completed. If you choose GPU, you will get 4 training cycles for each training cycle. More training cycles running simultaneously will speed up training, but also increase costs. How long your training takes also depends on the complexity of your agent system and your simulator. ## Advanced Configuration Click on `Advanced Configuration` for more options for your training session. * Rollout fragment length: Set this to * **Benchmark No. of Episodes Per Scenario:** Configure this setting to tell AMESA how many benchmark episodes should be run after training is completed * **Number of Steps Per Episode:** Change this setting from the default when you know how many steps should be in a single episode based on subject matter expertise. * **Non-Deterministic Benchmark:** This toggle allows you to control whether all benchmark episodes have the same starting place or whether they vary randomly within each scenario. ## Allocate Training Cycles If you are training an orchestration with multiple agents within it, you have the option to allocate the total number of training cycles among all the agents and orchestrators that need to be trained. Assign more cycles to the agents that have more difficult skills to learn; assign fewer cycles to those that you think will learn more quickly. ## Start Training When you have configured your settings correctly and allocated the training cycles among your agents and orchestrators, click `Start Training`. You will then be taken to the Training Sessions page. There you can follow the agent system training progress by viewing the real-time plots and updates. Note that it will take a few minutes for the visualization to begin. # Understand Training Behavior ## Use the Training Sessions Page The Training Sessions page allows you to view agent systems' training in real time and analyze their performance in training. When you begin a training session, the graphs for each trained skill agent will begin to generate. You can watch your skills learn by viewing the graphs. The shape of the curve can help you understand how your skill agents are learning. When the curve plateaus, that usually means that the skill has been successfully trained and will not learn more. If the curve shows jagged ups and downs, then the skill isn't performing consistently and has more learning to do. Sometimes this is a sign that you should go back and adjust the training settings.
The training sessions page shows a list of all the training sessions for a project in a menu on the left of the screen, allowing you to jump between different training sessions for the same agent system.
During or after training, you can access detailed console logs of the training session by typing /logs at the end of the url in your browser window. This is useful for troubleshooting issue that may occur during training. ## Download and Analyze the Historian File After the training is completed, you can also download a historian file. This is a parquet file that gives you detailed information about the team of agent's decision-making at every step in the training.
Analyze the historian using your favorite data analysis tool or use the [AMESA Assist Agent Analyst](https://github.com/Composabl/amesa-agents/tree/main) to ask questions about the training session in natural language. # Analyze Agent System Behavior Analyzing agent system behavior is a key part of using AMESA. This capability is key for: * Building agent systems that beat performance benchmarks * Monitoring the performance of your agent systems and simulators in real-time * Analyzing your agent systems and simulators to understand what went wrong in case of problems * Creating dashboards and reports to share with your team or customers AMESA offers multiple options for analyzing agent system behavior. For high-level summaries, you can use the benchmark reporting features in the UI. For more in-depth information, you can use the Historian and raw benchmark data in the SDK, which creates a database of all the information gathered during training and deployment. You can also design an LLM module in your agent to provide customizable communications capacity using natural language. # Analyze Data in Detail with the Historian In this tutorial, we will explore how to use the historian to validate the trained AI agent system in AMESA and training logs. The historian stores historical time-series data in an optimized format (parquet) - , which helps in evaluating how the agent is performing during training. ### **Step 1: Accessing the Historian Data** The historian file stores time-series data essential for validating agent system training. There are several ways to access and store the historian data, but the recommended format is as a **delta file** (parquet). 1. **Understanding the Format**: * The historian data is typically large, around 500 megabytes for standard operations. It is stored in a **Delta Lake** file format, optimized for time-series data and supporting efficient queries. 2. **Downloading the Historian File**: * From the AMESA UI, download the historian file. This file will come in a compressed format (e.g., `.gz`). * After extracting it, you should see the delta file containing time-series data. ### **Step 2: Setting Up for Validation** 1. **Unpacking the Historian File**: * If the historian file is compressed (e.g., `.gz`), unpack the file using a tool like `gzip`: ```bash gunzip -k historian_file.gz ``` * Once unzipped, you’ll see a **10 MB+ delta file** with historical time-series data. 2. **Understanding the Delta File**: * The delta file is optimized for fast reads and writes of time-series data. * It supports an append-only structure, which ensures that each new piece of data can be added efficiently without modifying the existing data. ### **Step 3: Querying the Historian Data** 1. **Setting Up a Query Environment**: * To validate your agent system’s training, you’ll need to set up an environment that allows you to query the delta file. Delta Lake integrates well with systems like **Apache Spark**, but for simple querying, you can use tools like **pandas** in Python. 2. **Querying for Agent Training Logs**: * Extract and analyze relevant historical data from the delta file. Here's a simple Python example for querying the delta file using pandas: ```python import pandas as pd # Load the historian delta file df = pd.read_parquet('historian_delta_file.parquet') df = df.sort_values(by=['timestamp']) df_data = df[df['category_sub'].isin(['step', 'skill-training','skill-training-cycle'])] #filter df with composabl_obs on "data" col only df_data = df_data[(df_data['data'].str.contains('composabl_obs')) | (df_data['category_sub'].str.contains('skill-training')) | (df_data['category_sub'].str.contains('skill-training-cycle'))] #df_data['data'] = df_data['data'].apply(lambda x: x if 'composabl_obs' in x else None) def convert_to_dict(x): try: return json.loads(x) except: try: return ast.literal_eval(x) except: return None df_data['data'] = df_data['data'].apply(lambda x: convert_to_dict(x)) df_data['skill_name'] = df_data['data'].apply(lambda x: x['name'] if 'is_done' in x else None) df_data['skill_name'] = df_data['skill_name'].fillna(method='bfill') df_data['reward'] = df_data['data'].apply(lambda x: x['teacher_reward'] if 'composabl_obs' in x else None) df_data['obs'] = df_data['data'].apply(lambda x: x['composabl_obs'] if 'composabl_obs' in x else None) #df_data['done'] = df_data['data'].apply(lambda x: x["teacher_terminated"] if "teacher_terminated" in x else None) df_data['cycle'] = df_data['data'].apply(lambda x: x['cycle'] if 'cycle' in x else None) df_data['cycle'] = df_data['cycle'].fillna(method='bfill') df_data = df_data[df_data['category_sub'] == 'step'] print(df_data) # group by runs df_group = df_data.groupby(['run_id','skill_name','cycle'])['reward'].mean() # Process observation data df_obs = pd.DataFrame(data=[[v[0] for v in list(x.values())] for x in df_data['obs'].values], columns=[list(df_data['obs'][0].keys())]) df_obs['cycle'] = df_data['cycle'] df_obs['run_id'] = df_data['run_id'] df_obs['skill_name'] = df_data['skill_name'] df_obs.columns = [x[0] for x in list(df_obs.columns)] # Episode Reward by Run Id for run_id in list(set([x[0] for x in df_group.index])): for skill in list(set([x[1] for x in df_group.index])): #df_group[run_id].plot(subplots=True, title=run_id) plt.plot(df_group[run_id][skill]) plt.ylabel(f'Mean Episode Reward') plt.xlabel(f'Cycle') plt.title(f'{run_id} - {skill}') plt.show() ``` *** ### Key Benefits of Using the Historian for Validation: * **Optimized Data Handling**: The Delta Lake format is designed for fast querying, making it ideal for time-series data. * **Efficient Storage**: The append-only nature ensures that new data can be added without overwriting or modifying existing data, making it easy to track data over time. * **Continuous Monitoring**: By continuously adding data to the historian, you can validate your agent system's long-term impact on machine performance, uptime, and safety. # Analyze Benchmark Performance Analyzing agent system behavior is a key part of using AMESA. This capability is key for: * Building agent systems that beat performance benchmarks * Monitoring the performance of your agent systems and simulators in real time * Analyzing your agent systems and simulators to understand what went wrong in case of problems * Creating dashboards and reports to share with stakeholders AMESA offers multiple options for analyzing agent system behavior. For high-level summaries, you can use the benchmark reporting features in the UI. For more in-depth information, you can use the historian and benchmark data files, which provide a database of all the information gathered during training and deployment. You can also design an LLM module in your agent to provide customizable communications capacity using natural language. ## The AMESA Benchmarking Page The AMESA benchmarking feature allows you to compare the performance of different multi-agent systems against key performance indicators (KPIs). This tool helps you evaluate your agent system's effectiveness, track its improvements, and calculate potential return on investment (ROI). Benchmarks are generated after training is completed. After all the training episodes are finished, the system will run benchmarking episodes. These are episodes where the team of agents is controlling the process within the simulation, but unlike during training, the agents are no longer experimenting to try to learn new things - they are demonstrating the result of their learning and trying to optimize performance as best as possible. The data from these episodes will then populate the benchmark.json file and the Benchmarks page visualizations. Because benchmark episodes run after training, you won't see benchmark data while training progresses, even for long-running jobs. If a training job fails, no benchmark data will be generated. Note: Benchmarking does not influence agent system training, goals, or rewards. It is purely an analysis tool that helps you answer the question: "Based on how this agent system was trained and how we believe it should perform in real life, how much money is this agent system generating?" ### Define the KPI for Your Project The key performance indicator (KPI) for your project is the top-level objective that you will use to compare and evaluate your multi-agent systems' performance. To define the KPI for your project, choose the sensor variable that represents the metric that best indicates performance success. For most processes, this is a metric with a clear business impact, such as product yield or energy use. ### Setting up the KPI You set up the KPI when you first [create a use case](/build-multi-agent-systems/create-a-project), but you can also change the KPI at any time. To set up the KPI after a use case is created: 1. Navigate to the Benchmark page from the main dashboard 2. Click `Edit KPI and ROI` in the top right corner 3. In the "Set up KPI" section, configure the following: * **Sensor Name**: Select a sensor or preceptor variable to track (e.g., "Eps\_Yield") * **Target Value**: Set the desired target for this KPI (e.g., 0.33) * **Benchmark Unit**: Select the unit of measurement (e.g., "% of Max Theoretical Yield"). Note that this is for UI readability only and doesn't affect calculations.
### Setting up ROI Calculations Return on Investment (ROI) calculations help quantify the financial impact of your agents' performance. The ROI is calculated based on the monetary value of the difference between your target value and the actual value of the KPI generated by that agent system, minus training costs. 1. In the Settings modal, navigate to the "Enter ROI criteria" section 2. Define the conditions for calculating ROI: * **When**: Select the metric to track (e.g., "Eps\_Yield") * **Condition**: Choose how the metric changes (e.g., "increases by") * **Percentage**: Enter the percentage change that matters (e.g., 2%) * **Value**: Set the financial value of this change (e.g., $1,000,000) * **Period**: Select the time period (e.g., "per year") For example: "When Eps\_Yield increases by 1% of max theoretical yield, it is worth $1,000,000 per year." You can change KPI and ROI settings at any time after training completes, and the benchmark results will be automatically recalculated. This allows you to explore different business scenarios without retraining your agent systems. ## Reading the Benchmarking Dashboard
### Understanding the Overview Panel At the top of the benchmarking page, you'll find an Overview panel containing three key metrics: * **Highest performing Agent System**: Displays the outcome of the highest performing agent system * **Lowest performing Agent System**: Displays the outcome of the lowest performing agent system * **ROI**: Shows the calculated financial return based on your KPI settings Each metric displays both the value and the relevant unit (e.g., "% of Max Theoretical Yield"). ### Bar Graph Visualization The bar graph in the KPI Performance Metrics section provides a visual comparison of your agent systems: * Each bar represents an agent system's performance on your selected KPI (all bars correspond to the same KPI) * The height of the bar indicates the specific value of the KPI for that agent system (Y-axis) * Horizontal dashed lines show benchmark averages * The actual value of the KPI for each agent system is shown above the respective bar ### Performance Ranking Table Below the graph is a detailed table showing: * **Agent System Number**: Sequential ID for each agent system * **Agent System Name**: Name or description of the agent system * **ROI**: Calculated financial return based on the agent system's performance * Green arrows (↑) indicate positive ROI with the percentage and absolute value * Red arrows (↓) indicate negative ROI with the percentage and absolute value * **Eps\_Yield (variable) per % of Max Theoretical Yield**: Performance metric with indicators for highest agent system performers ### Agent Formation Performance Plots The last element on the Benchmarks page is a set of plots showing detailed information about performance on the KPI throughout the course of the benchmarking episode(s).
The green shows the value of the KPI at each point throughout the benchmarking episode. The red shows another sensor, which you can choose from a dropdown menu. This allows you to visualize not only the performance on the KPI over time, but also the KPI in relation to another key variable. # Analyze Data ## Benchmark Testing and Data Generation After training a multi-agent system in AMESA, the system automatically runs a series of standardized tests to evaluate its performance. This benchmarking process: 1. Places the system in controlled testing environments 2. Records detailed metrics at each step of operation 3. Aggregates results to provide comprehensive performance statistics The output of this testing process is compiled into a structured benchmark.json file, which contains rich performance data that can be analyzed to assess effectiveness, identify improvement opportunities, and compare different design approaches. This file is a performance record and a valuable analytics resource for optimizing your agentic systems. ## Downloading Benchmark Artifacts To download benchmark data for further analysis: 1. Navigate to the "Training Sessions" page 2. Click the artifacts dropdown in the top right page of a trained system 3. Select "Benchmark" 4. The benchmark.json file will be saved to your local machine
## Understanding the Benchmark.json File The benchmark.json file contains structured data about the performance of a trained agent system. Here's how to interpret this file: ### File Structure ``` { "skill-name": { "scenario-0": { "scenario_data": { ... }, "episode-0": [ ... ], "aggregate": { ... } } } } ``` ### Key Components **Scenario Data**: Contains reference values for the scenario: ``` "scenario_data": { "sensor_one": {"data": 8.57, "type": "is_equal"}, "sensor_two": {"data": 373, "type": "is_equal"} } ``` **Episode Data**: Array of state-action pairs showing how the agent performed in each step: ``` [ { "state": "{'sensor_one': array([311.2639], dtype=float32), ...}", "action": "[-1.253192]", "teacher_reward": 1.0, "teacher_success": false, "teacher_terminal": null }, ... ] ``` **Aggregate Statistics**: Summary statistics for the entire benchmark: ``` "aggregate": { "mean": { ... }, "medians": { ... }, "std_dev": { ... }, "max": { ... }, "min": { ... } } ``` # Deployment Options When you are ready to deploy your team of agents, AMESA offers two options for connecting to your production system.
* The **AMESA Runtime Server** is a broker that uses HTTP, MQTT or OPCUA to connect to a HMI, DCS or other production system. * The **AMESA Runtime App** reads and writes tags directly to the PLC. Both options inference the agents and pass decisions to the deployment system, and then pass sensor readings back to the team of agents. Both options can also be set up for closed-loop control or for decision support for operators. Choose the option that works best for your needs. # Deploy with the AMESA Runtime Server ## Overview The AMESA Runtime allows you to run an AMESA agent on your own hardware. It handles receiving observations (sensor data), processing them through the agent, and computing output actions. The package consists of three components: * Light Touch Runtime β€” the core HTTP inference server that loads and runs the agent. * Client Runtime Proxy β€” the MQTT bridge that routes observations in and actions out. * Packaged Agent β€” the agent model and configuration files (agent.json, etc.). This guide walks through the full setup process: starting the runtime, and verifying the system is working end-to-end. ## Step 1: Start the Light Touch Runtime The Light Touch Runtime is the inference server. It loads the agent and listens for incoming observations over HTTP. Set the required environment variables and start the runtime. Run these commands in sequence in your first terminal: ### Set Environment Variables ``` export AMESA_LICENSE= ---- -- ---- --------- --- export AMESA_DISABLE_INSTALL_PACKAGES=true export AMESA_BASE_DIR=~/packaged_agent ``` ### Launch the Runtime Point the runtime at the agent configuration file : ``` ~/light_touch_runtime/dist/light_touch_runtime ~/packaged_agent/agent.json ``` {% hint style="info" %} Expected result: You know this step was successful when you see "Agent Loaded" printed in the terminal. Keep this terminal open β€” it must stay active for the rest of the process. {% endhint %} ## Step 2: Start the Client Runtime Proxy The Client Runtime Proxy handles the MQTT communication layer. It receives observations over MQTT and forwards them to the Light Touch Runtime's HTTP endpoint, then publishes the computed actions back out. Open a second terminal and run a quick sanity check to confirm the binary is executable: ``` ~/client_runtime_proxy/dist/client_runtime_proxy --agent-id prod-id-agent ``` This will return a message saying PID folder missing β€” that is expected behavior at this stage. ### Run the Full Command ``` ~/client_runtime_proxy/dist/client_runtime_proxy \ --agent-id prod-id-agent \ --license usb-token \ --mqtt-broker localhost:1883 \ --mqtt-input-topic amesa/input \ --mqtt-output-topic amesa/output \ --inference-url http://localhost:8080 ``` {% hint style="info" %} Expected result: No PID file found, proceeding with start, followed by a summary of the arguments you passed. This confirms the proxy is running and connected to the runtime. {% endhint %} ## Step 3: Send a Test Observation With both terminals running, you can now verify the full pipeline by sending a test observation. This simulates what real sensor data will look like when the system is live. In your second terminal (or a third if preferred), send the following curl command: ``` curl -X POST http://localhost:8080/mcp \ -H "Content-Type: application/json" \ -d '{ "jsonrpc": "2.0", "id": 1, "method": "publish", "params": { "channel": "agent.observation", "message": { "observation": [1, 2, 3, 4, 5, 6] } } }' ``` Watch your first terminal (where the Light Touch Runtime is running). A successful result will print: ``` processing observation - computed action ``` {% hint style="info" %} Expected result: If you see both of these messages, the system is fully operational end-to-end. {% endhint %} # Access a Trained Agent System Once you have completed training an agent and are ready to deploy it you can download it from the UI. 1. Go to the `Training` tab in your project. 2. Select the training session for the agent you want to deploy. 3. Click on the `Artifacts` drop down in the upper right of the screen. 4. Click `Inference Model` and you will down load a file called `agent.json`.
# Deploy an Agent System in a Container Based on the notes provided in the image, we'll help you create a tutorial on **accessing the agent system runtime after deploying it to Docker**. This tutorial will explain the steps for building the Docker container, deploying the agent system, and accessing the runtime for inference or interaction. *** ### Tutorial: Accessing the Agent System Runtime After Deploying to Docker Once you have packaged and deployed your agent system inside a Docker container (), the next step is accessing its runtime for operations like model inference. This tutorial will guide you through the process of building and running the Docker container and then connecting to the agent system's runtime for further interactions. *** ### Step 1: Preparing the Dockerfile and Environment To deploy the agent system to Docker, we need to first create an image from the **Dockerfile** (). The Dockerfile will package the necessary runtime, model, and environment for the agent system.
### Step 2: Building the Docker Image 1. **Building the Image**: You can build the Docker image by running the following command in the terminal. This will take the Dockerfile and the associated files (like the pre-trained model) and create an image. ```bash docker build -t composabl_agent_api . ``` * The `-t` flag allows you to tag the image (`composabl_agent_api`), which makes it easier to reference later. * Make sure that the model file (`agent.json`) and all relevant scripts are reachable within the Docker context (i.e., the directory from which you are building). 2. **Checking the Image**: Once the build is complete, you can verify that the image was created successfully by running: ```bash docker images ``` *** ### Step 3: Running the Docker Container Now that the image is built, the next step is to run it in a container. You will run the Docker container in an interactive mode to access the runtime. ```bash docker run -e AMESA_LICENSE="" -it -p 8000:8000 composabl_agent_api ``` * `-it`: Runs the container interactively. * `-p 8000:8000`: Maps port 8000 from the container to port 8000 on your local machine so that you can access the HTTP server for the agent system runtime. * `-e` AMESA\_LICENSE="\" : is exporting the environment variable and linking to your composabl license The HTTP server should now be up and running within the container, ready to handle model inference or other tasks. *** ### Step 4: Accessing the Agent System Runtime With the Docker container running, you can now connect to the agent system's runtime. The runtime will be an HTTP server, as mentioned in your notes. You can access it through a POST request for model inference or other operations. 1. **Sending Requests to the Agent System**: You can send a POST request to the running server using a tool like `curl`, Postman, or any Python HTTP library (such as `requests`). Here’s an example using `curl`: ```bash curl -X POST http://localhost:8000/predict -H "Content-Type: application/json" -d '{"observation": {"T": 311.0, "Tc": 292.0, "Ca": 8.56, "Cref": 8.56, "Tref": 311.0, "Conc_Error": 0.0, "Eps_Yield": 0.0, "Cb_Prod": 0.0}}' ``` This request will: * POST data to the `/predict` endpoint on `localhost:8000`, which is being forwarded from the Docker container. * The agent system will handle the request, infer the model, and return the action as a result. *** ### Conclusion In this tutorial, we walked through the process of: * Building a Docker image with your agent system and its runtime. * Running the Docker container interactively to expose the agent system’s HTTP server. * Accessing the agent system runtime by sending HTTP requests for inference or other tasks. By following these steps, you can deploy and interact with your AMESA agent system in a Dockerized environment. # Deploy an Agent System as an API Deploying an agent system means exporting the trained agent system, loading it into your production environment, and then asking the agent system for decisions. In AMESA, the export is a json file called `agent.json` that contains all you need to deploy your agent system. You can load the agent system file to use in your IT infrastructure with many ways. This document will show how you can deploy your agent system as an API using Python and Flask. ### Step 1: Accessing and Preparing the Files To deploy the agent system as an API, we need to extract the `agent.json` model, get the `agent_api.py` script to start the API, `requirements.txt` to install packages. You can find a sample for these files in our GitHub repo: This is the structure needed for the API:
\*\*\* How to extract the agent.json \*\*\* 1. Log into the AMESA UI (), train your agent system, and navigate to the training sessions section. 2. Check the status of the agent system: * Green status (Done) indicates finished training. 3. Download the agent system essential file: * The agent system file (a `.gz` compressed file). * Extract the agent system file `agent.json` to the `model` folder ### Step 2: Get the API python file ```python import asyncio import os from composabl import Agent, Trainer import numpy as np from flask import Flask, request, jsonify # Initialize Flask app app = Flask(__name__) # Global objects (initialized once) trainer = None trained_agent = None license_key = os.environ["AMESA_LICENSE"] PATH = os.path.dirname(os.path.realpath(__file__)) PATH_CHECKPOINTS = f"{PATH}/model/agent.json" # Initialize the runtime, load the model, and package it when the app starts async def init_runtime(): """ Initializes the trainer and agent before the first request is processed. This sets up the AI model for inference, loading it from checkpoints and preparing the agent. """ global trainer, trained_agent # Assuming 'config' is required to initialize the Trainer config = { "license": license_key, "target": { "local": {"address": "localhost:1337"} }, "env": { "name": "sim-deploy", }, "trainer": { "workers": 1 } } # Initialize the Trainer with the config trainer = Trainer(config) # Load the agent from the given checkpoint path agent = Agent.load(PATH_CHECKPOINTS) # Package the agent for inference using the Trainer's _package function (asynchronously) trained_agent = await trainer._package(agent) # Asynchronous POST route to receive observation and return action @app.route('/predict', methods=['POST']) async def predict(): """ Receives an observation through a POST request, processes it using the pre-trained agent, and returns the corresponding action. """ global trained_agent # Check if the agent has been successfully initialized if not trained_agent: return jsonify({"error": "Agent not initialized"}), 500 # Extract the observation from the request's JSON body obs = request.json.get("observation") obs = dict(obs) obs = np.array( [float(x) for x in list(obs.values())] ) # Validate that the observation was provided in the request if obs is None: return jsonify({"error": "No observation provided"}), 400 # Asynchronously process the observation to generate the action action = await trained_agent._execute(obs) # Return the generated action in the response return jsonify({"action": str(action)}) if __name__ == "__main__": # Run the Flask application with async support on localhost, port 8000 loop = asyncio.get_event_loop() loop.run_until_complete(init_runtime()) app.run(host="0.0.0.0", port=8000, debug=True) ``` ### Step 3: Install the requirement packages Run the following in your terminal: ```bash pip install --no-cache-dir -r requirements.txt ``` It will install these packages: * composabl * flask\[async] * numpy ### Step 4: Export your license and start the API To start AMESA API, you will need to export your license as an environment variable and then use Python to start the Flask API with your agent system. ```bash export AMESA_LICENSE='' python agent_api.py ``` ### Step 5: Test your API After running the API, you can test it by opening the terminal and run the script below: ```bash curl -X POST http://localhost:8000/predict -H "Content-Type: application/json" -d '{"observation": {"T": 311.0, "Tc": 292.0, "Ca": 8.56, "Cref": 8.56, "Tref": 311.0, "Conc_Error": 0.0, "Eps_Yield": 0.0, "Cb_Prod": 0.0}}' ``` In the POST request, we pass the use case "observation" with sensor variables and their values to receive an action from the agent system. The code above is related to the `agent.json` demo for Chemical Process Control. # Connect Runtime Container to Your Operation Based on the notes provided in the image, I'll help you create a tutorial on **accessing the agent system runtime after deploying it to Docker**. This tutorial will explain the steps for building the Docker container, deploying the agent system, and accessing the runtime for inference or interaction. *** ### Tutorial: Accessing the Agent System Runtime After Deploying to Docker Once you have packaged and deployed your agent system inside a Docker container, the next step is accessing its runtime for operations like model inference. This tutorial will guide you through the process of building and running the Docker container and then connecting to the agent system's runtime for further interactions. *** ### Step 1: Preparing the Dockerfile and Environment To deploy the agent system to Docker, we need to first create a **Dockerfile**. The Dockerfile will package the necessary runtime, model, and environment for the agent system. 1. **Dockerfile Setup**: Your Dockerfile should contain the following key components: * **Base Image**: Use a Python base image (or any base that supports the necessary libraries). * **Copy Model Files**: Copy the pre-trained model (e.g., `.gz` file) to the container. * **Install Dependencies**: Install any required Python libraries (like OHTTP or other packages for the agent system). Here’s an example Dockerfile: ```dockerfile # Use an official Python runtime as the base image FROM python:3.10-slim # Set the working directory WORKDIR /usr/src/app # Copy the necessary files into the Docker image COPY . . # Install any dependencies specified in requirements.txt RUN pip install --no-cache-dir -r requirements.txt # Expose port 8000 for the HTTP server EXPOSE 8000 # Command to run the server when the container starts CMD ["python", "agent_inference.py"] ``` *** ### Step 2: Building the Docker Image 1. **Building the Image**: You can build the Docker image by running the following command in the terminal. This will take the Dockerfile and the associated files (like the pre-trained model) and create an image. ```bash docker build -t my-agent-runtime . ``` * The `-t` flag allows you to tag the image (`my-agent-runtime`), which makes it easier to reference later. * Make sure that the model file (`agent.gz`) and all relevant scripts are reachable within the Docker context (i.e., the directory from which you are building). 2. **Checking the Image**: Once the build is complete, you can verify that the image was created successfully by running: ```bash docker images ``` *** ### Step 3: Running the Docker Container Now that the image is built, the next step is to run it in a container. You will run the Docker container in an interactive mode to access the runtime. ```bash docker run -it -p 8000:8000 my-agent-runtime ``` * `-it`: Runs the container interactively. * `-p 8000:8000`: Maps port 8000 from the container to port 8000 on your local machine so that you can access the HTTP server for the agent system runtime. The HTTP server should now be up and running within the container, ready to handle model inference or other tasks. *** ### Step 4: Accessing the Agent System Runtime With the Docker container running, you can now connect to the agent system's runtime. The runtime will be an HTTP server, as mentioned in your notes. You can access it through a POST request for model inference or other operations. 1. **Sending Requests to the Agent System**: You can send a POST request to the running server using a tool like `curl`, Postman, or any Python HTTP library (such as `requests`). Here’s an example using `curl`: ```bash curl -X POST http://localhost:8000/infer -d '{"input_data": "your_input_here"}' ``` This request will: * POST data to the `/infer` endpoint on `localhost:8000`, which is being forwarded from the Docker container. * The agent system will handle the request, infer the model, and return the result. 2. **Interacting with the Agent System**: If you prefer to interact with the agent system directly, you can also enter the container’s interactive mode and run commands. ```bash docker exec -it bash ``` This will open a shell inside the running Docker container, allowing you to execute any runtime commands manually. *** ### Step 5: Automating the Process For convenience, you can automate the entire process of building the image, running the container, and interacting with the agent system by creating a script. Here’s a basic example of an automation script: ```bash #!/bin/bash # Build the Docker image docker build -t my-agent-runtime . # Run the Docker container docker run -it -p 8000:8000 my-agent-runtime ``` Save this as `run_agent.sh`, and then execute it: ```bash bash run_agent.sh ``` This script will: * Build the Docker image. * Run the container, mapping the necessary port and exposing the HTTP server for inference. *** ### Step 6: Troubleshooting and Debugging If the container fails to start, or if the server doesn't respond, you can debug the container by checking the logs: ```bash docker logs ``` This command displays the output of the running container, which can help diagnose issues such as missing dependencies or server errors. *** ### Conclusion In this tutorial, we walked through the process of: * Building a Docker image with your agent system and its runtime. * Running the Docker container interactively to expose the agent’s HTTP server. * Accessing the agent system runtime by sending HTTP requests for inference or other tasks. By following these steps, you can deploy and interact with your AMESA agent system in a Dockerized environment. # Connecting to Agent System Runtime and Plotting Results of Agent System Operations In this tutorial, we will cover how to connect to the agent system runtime, load a pre-trained agent system, run inference, and visualize the results in a production-like environment. The provided script, `agent_inference.py`, is a key component that demonstrates connecting to the AMESA agent system runtime, initializing the environment, and plotting agent system operation results. *** ### Step 1: Understanding `agent_inference.py` The script `agent_inference.py` connects to the runtime, loads a pre-trained agent system, connects to a local simulation, collects sensor data from the sim and plots the results. Here is an outline of the core steps in the process: 1. **Start Runtime and Load Agent System**: The script initializes the trainer and loads a pre-trained agent system from a model folder. 2. **Set Up the Simulation Environment**: It connects to a simulation environment. 3. **Run Inference**: The pre-trained agent system interacts with the simulation to perform inference (decisions), collecting observations and giving actions at each step. 4. **Collect Data and Plot Results**: Sensor data and actions are collected in a Pandas DataFrame, and the results are plotted using Matplotlib to visualize how the agent system is performing over time in a production like environment. *** ### Step 2: Connecting to the Runtime and Loading the Agent System The first task is to connect to the AMESA runtime and load the pre-trained agent system. This is accomplished using the `Trainer` and `Agent` classes. The agent system's model is loaded from the directory where the model was saved during training. ```python async def run_agent(): # Start Runtime trainer = Trainer(config) # Load the pre-trained agent agent = Agent.load(PATH_CHECKPOINTS) # Prepare the agent for inference trained_agent = await trainer._package(agent) ``` Here: * `Trainer(config)` initializes the runtime with a configuration file. * `Agent.load(PATH_CHECKPOINTS)` loads the saved agent from the specified checkpoint directory. * `trainer._package(agent)` prepares the agent for inference by packaging it. *** ### Step 3: Connecting to the Simulation Environment Next, we connect the agent system to the simulation environment. The `make()` function creates a connection to the local simulator, and the environment is initialized. ```python # Inference print("Creating Environment") sim = make( run_id="run-benchmark", sim_id="sim-benchmark", env_id="sim", address="localhost:1337", env_init={}, init_client=False ) print("Initializing Environment") await sim.init() print("Initialized") ``` Here: * The simulator is configured to run locally (localhost:1337) and you have to start it locally and manually before. * The environment is initialized with `sim.init()`, and the agent system is connected to it. *** ### Step 4: Setting the Scenario and Running Inference After connecting to the simulator, you need to set up the specific **scenario** that the agent system will operate in. This scenario determines the environment's initial state. ```python # Set scenario noise = 0.0 await sim.set_scenario(Scenario({ "Cref_signal": "complete", "noise_percentage": noise })) ``` With the environment set, the agent can now run inference for a set number of iterations. At each iteration, the agent observes the environment, takes an action, and collects the results (observations and rewards). This is done in a loop. ```python obs_history = [] df = pd.DataFrame() print("Resetting Environment") obs, info = await sim.reset() obs_history.append(obs) action_history = [] for i in range(90): action = await trained_agent._execute(obs) # Get action from agent obs, reward, done, truncated, info = await sim.step(action) # Step the environment # Create a temporary DataFrame for the current observation df_temp = pd.DataFrame(columns=[s.name for s in sensors] + ['time'], data=[list(obs) + [i]]) # Concatenate the new data to the existing DataFrame df = pd.concat([df, df_temp]) obs_history.append(obs) action_history.append(action) if done: break ``` In each iteration: * The agent system performs an action based on the current observations. * The environment advances one step with `sim.step(action)`, and the agent receives a new observation and reward. * Sensor data and actions are logged into a Pandas DataFrame for later analysis. *** ### Step 5: Saving Data and Plotting Results Once the inference loop is complete, the collected data is saved, and the results are visualized. The results are plotted using **Matplotlib**. ```python # Save the DataFrame to a pickle file for later use df.to_pickle(f"{PATH_HISTORY}/inference_data.pkl") # Plot results plt.figure(figsize=(10, 5)) # Plot Temperature Controller Data (Tc) plt.subplot(3, 1, 1) plt.plot(df.reset_index()['time'], df.reset_index()['Tc']) plt.ylabel('Tc') plt.legend(['reward'], loc='best') plt.title(f'Agent Inference DRL - Noise: {noise}') # Plot Temperature and Reference Temperature (T, Tref) plt.subplot(3, 1, 2) plt.plot(df.reset_index()['time'], df.reset_index()['T']) plt.plot(df.reset_index()['time'], df.reset_index()['Tref'], 'r--') plt.ylabel('Temp') plt.legend(['T', 'Tref'], loc='best') # Plot Concentration and Reference Concentration (Ca, Cref) plt.subplot(3, 1, 3) plt.plot(df.reset_index()['time'], df.reset_index()['Ca']) plt.plot(df.reset_index()['time'], df.reset_index()['Cref'], 'r--') plt.legend(['Ca', 'Cref'], loc='best') plt.ylabel('Concentration') plt.xlabel('Iteration') # Save plot plt.savefig(f"{PATH_BENCHMARKS}/inference_figure.png") ``` This code generates three subplots: 1. **Temperature Controller (Tc)** over time. 2. **Temperature (T)** and **Reference Temperature (Tref)** over time. 3. **Concentration (Ca)** and **Reference Concentration (Cref)** over time. The plots provide a visual representation of the agent system's performance during the simulation. Finally, the figure is saved as `inference_figure.png` in the benchmarks directory. *** ### Step 6: Running the Script To run the script, execute the `agent_inference.py` in your terminal. ```shell python agent_inference.py ``` *** ### Conclusion In this tutorial, we demonstrated how to: * Connect a pre-trained AMESA agent system to a runtime and simulation environment. * Set up a scenario and run inference. * Collect observations and actions, and plot the results using Matplotlib. By following these steps, you can visualize the performance of your agent system and gain insights into how it interacts with the environment over time. # Training Infrastructure Options
There are multiple options for hosting AMESA's training infrastructure. * **Local training:** useful for testing and early stage development. * **Single virtual machine:** * **AMESA-hosted Kubernetes clusters:** training using AMESA's Agent Training Cloud * **Self-hosted Kubernetes clusters:** training using your tenant in Azure, AWS or any other cloud compute provider # Creating a Cluster ## Introduction To train AMESA agent systems on a cluster, you need to install the AMESA controller onto a compatible Kubernetes cluster. This document will guide you through setting up this cluster using various options: * Azure: Set up a cluster through our Azure Marketplace AMESA Training Cluster offer * Automated: Set up a compatible Kubernetes cluster on Azure through the use of Pulumi (IaaS) * Manually: Set up a compatible Kubernetes cluster through any tool and infrastructure of your choice ## Components The entire AMESA controller installation consists of two main components: * Controller: This component connects to the AMESA No-code ui, and allows you to initiate training on your cluster. * Historian: The historian component records detailed statistics regarding the training process of your agent system. After training, these will be available alongside the agent system in the no-code UI ## RBAC The installation of AMESA will introduce 2 namespaces, `composabl-train` and `composabl-sims`, for training workers and simulators respectively. For normal operation, a series of permissions are required by the `ServiceAccount`s of the AMESA installation. ## Cluster Setup We currently have the following guides to setting up your own cluster: * [Azure](https://github.com/Composabl/documentation/blob/main/clusters/creating-a-cluster/azure-offer.md), our recommended way of setting up a training cluster * [Automated](/training-infrastructure/creating-a-cluster/automated), which describes the use of Pulumi for setting up a compatible cluster on Azure * [Manual](/training-infrastructure/creating-a-cluster/manual), if you're experienced in setting up a cluster, or have existing standard tooling for setting up kubernetes clusters # Manual ## Overview Due to the many options available to you for installing Kubernetes clusters, this document will not go into the specifics of setting up the cluster. Rather, it will provide you with guidance and requirements for your cluster. ## Nodes Depending on whether you want to use GPUs or not, you need the following nodes: Nodes that are always required: 1. "main": This means nodes to run the control plane. The AMESA controller does not interact with these nodes, so they should be provisioned as recommended by the Kubernetes distribution you use. 2. "composabl": This node or nodes will be where the AMESA controller and Historian software are scheduled. 3. "envrunners": These nodes will handle training workloads. If you're not using GPUs, all training will be done on these nodes. If you are, these nodes will manage the communication with the simulators, and can be reduced in size 4. "simscpu": These nodes are where the simulators will be scheduled. Sizing depends on the simulator. If you want to use GPU training, you need the following nodepool: 5. "learners": These nodes with GPUs will accelerate the learning step of the training process. If your simulator can be accelerated using GPU, you can add the final node pool: 6. "simsgpu": These will run simulators, assigning a GPU to them. A note on GPUs: Currently, only Nvidia GPUs are supported. The cluster must have the [nvidia-gpu-operator](https://docs.nvidia.com/datacenter/cloud-native/gpu-operator/latest/getting-started.html) installed for training on GPU to be enabled. ### 1. Sizing Whether or not you use autoscaling using [cluster-autoscaler](https://github.com/kubernetes/autoscaler/tree/master/cluster-autoscaler), each node type must be sized accordingly. 1. `main`: As required by your Kubernetes distribution 2. `composabl`: In total, 16GB of memory and 4 CPU - with 1 node at least 8GB of memory. 3. `envrunners`: If not using GPUs, we recommend 8 CPU and 8 or 16 GB of memory. In any case, the number of simulators that can be managed by each envrunner instance depends on the number of CPU 4. `simscpu`: The sizing of these nodes depends on the resource requirements of your simulator 5. `learners`: These nodes should have 1 Nvidia GPU. Other resources can be limited - 2 CPU and 8GB of memory is sufficient 6. `simsgpu`: As with `simscpu`, depends on the simulator requirements. ### 2. Labels All groups of nodes must be labeled accordingly. The name, as given in the sizing guide is the name you should set as the agentpool label. You may be able to define this during your cluster setup, but if not, you can use the following commands: ```bash kubectl label node agentpool=composabl --overwrite kubectl label node agentpool=envrunners --overwrite kubectl label node agentpool=simscpu --overwrite kubectl label node agentpool=learners --overwrite kubectl label node agentpool=simsgpu --overwrite ``` Replace the values in between `<>` with the name of the nodes you'd like to assign to a specific pool. ## Storage The components also need access to (semi)persistent, shared storage. This section will detail the types and amount of storage needed. It needs the following `PersistentVolumeClaim`s in the `composabl-train` namespace: 1. `pvc-controller-data` with a size of Β±`1Gi` and `ReadWriteOnce` (or better) `accessMode` When using Azure, you will need to set the `nobrl` mountOption for this PVC, as this is required for the AMESA controller to function. 2. `pvc-training-results` with a suitable size - this is where your final agent system data will be stored before it is uploaded to the No-code application. It **needs `accessmode` to be `ReadWriteMany` (RWX)**. A good initial size is to match `historian-tmp`. 3. `historian-tmp` is used as temporary storage for historian data. It needs to have an `accessMode` of `ReadWriteOnce` and the size will depend on the length of your training sessions. We recommend starting with `5Gi`. The size of `pvc-training-results` and `historian-tmp` is dependent on the amount and size of training jobs you want to run simultaneously on your cluster. If you plan on running long-lived training sessions with many cycles, you may want to increase the capacity for both, ## Private image registry If you want to use a private registry for simulator images, you will need to set up this private registry yourself, and make sure the cluster is able to pull images from this registry. ## Next steps Once your cluster is running, and you have verified your setup is working, you can continue to [Installing AMESA](/training-infrastructure/creating-a-cluster/manual) # Automated ## Introduction This guide covers setting up your AMESA training cluster using [pulumi](https://www.pulumi.com/), an Infrastructure as Code tool. This example uses Azure Kubernetes Service, but can be adapted to other supported providers. ## Prerequisites 1. An Azure subscription with sufficient permissions to create and update various resources 2. [A working installation of pulumi](https://www.pulumi.com/docs/install/) 3. If you're following along in typescript, a working installation of NodeJS 4. A new pulumi project, as per the pulumi documentation. You can find the documentation for Azure [here](https://www.pulumi.com/docs/clouds/azure/get-started/create-project/) ## Overview We will be deploying the following resources to your Azure subscription: 1. Resource group, containing all resources 2. A container registry, to hold simulator images 3. An AKS cluster ### Resource group The resource group will contain all resources. It is also what determines in what Azure location the resources will be deployed. ```typescript import * as resources from "@pulumi/azure-native/resources/index.js"; const resourceGroup = new resources.ResouceGroup('my-resource-group-', { location: 'eastus' }); export const rgName = pulumi.interpolate`${resourceGroup.name}`; ``` At the end, we export the name of the resource group (which will be randomized by pulumi) for further use in our definition ### Container registry The container registry is where you will be able to privately store your simulator docker images, if any. ```typescript import * as containerregistry from "@pulumi/azure-native/containerregistry/index.js"; const registry = new containerregistry.Registry("registry", { resourceGroupName: resourceGroup.name, sku: { name: "Basic", }, adminUserEnabled: true, }); export const registryName = pulumi.interpolate`${registry.name}`; ``` ### Kubernetes Cluster The cluster is where both the AMESA components and your training will be running. This configuration is more complex, so additional information will be provided as comments in the typescript definition: ```typescript import * as containerservice from "@pulumi/azure-native/containerservice/index.js"; const k8sCluster = new containerservice.ManagedCluster("aks", { resourceGroupName: resourceGroup.name, // Here, we reference the resourceGroup we created earlier location: resourceGroup.location, dnsPrefix: "composabl-aks", kubernetesVersion: "1.31.1", // you can get supported versions using the Azure CLI: az aks get-versions -l -o table - replace with the location you set in your resourcegroup. enableRBAC: true, // Assign a managed identity to the cluster identity: { type: "UserAssigned", userAssignedIdentities: [appMiAKS.id], }, // Configure 3 pools // 1. Main (the kubernetes control plane nodes) // 2. Train (AMESA system components and training workers) // 3. Sims (AMESA simulators) agentPoolProfiles: [ // The Main pool has 3 small nodes to act as a control plane { name: "main", count: 3, vmSize: "Standard_B2s", // (2 core, 4GB RAM, 0.041/hour) osType: "Linux", osSKU: "Ubuntu", mode: "System", } ], sku: { name: "Base", tier: "Standard" }, // This is an optional part, unless using very large clusters with several 100s of nodes. networkProfile: { networkPlugin: "azure", networkPolicy: "calico", } }); // the "AMESA" agent pool will run the composabl system components (Controller, Historian) const composablPool = new containerservice.AgentPool("composabl", { resourceGroupName: resourceGroup.name, resourceName: k8sCluster.name, agentPoolName: "composabl", count: 1, vmSize: "Standard_D4s_v3", // (4 core, 16GB RAM) osType: "Linux", osSKU: "Ubuntu", mode: "System", osDiskSizeGB: 100, osDiskType: "Premium_LRS", }, { replaceOnChanges: ["vmSize"] } ); // the Env Runners will contain the part of the SDK that deals with data gathering from the simulators // If training using GPU is disabled, all training will happen on these nodes as well const envrunnersPool = new containerservice.AgentPool("envrunners", { resourceGroupName: resourceGroup.name, resourceName: k8sCluster.name, agentPoolName: "envrunners", vmSize: "Standard_D8d_v4", count: 1, minCount: 1, maxCount: 10, enableAutoScaling: true, osType: "Linux", osSKU: "Ubuntu", }, { replaceOnChanges: ["vmSize"] } ); // The Sims-CPU pull will run all simulator instances const simsCpuPool = new containerservice.AgentPool("simscpu", { resourceGroupName: resourceGroup.name, resourceName: k8sCluster.name, agentPoolName: "simscpu", vmSize: "Standard_D8d_v4", count: 2, minCount: 2, maxCount: 1000, enableAutoScaling: true, osType: "Linux", osSKU: "Ubuntu", }, { replaceOnChanges: ["vmSize"] } ); ``` #### GPU Training and simulators If you want to enable GPU training and GPU-enhanced simulators, you will also need to add the following pools. In addition, you will also need to install the nvidia-gpu-operator on the cluster. This can be done according to the instructions on the [project website](https://docs.nvidia.com/datacenter/cloud-native/gpu-operator/latest/getting-started.html). Finally, `GPU_ENABLED` must be set to `true` on the AMESA controller deployment, if it hasn't been already. ```typescript // The learners will run the learning part of the training, accelerated by GPU const learnersPool = new containerservice.AgentPool("learners", { resourceGroupName: resourceGroup.name, resourceName: k8sCluster.name, agentPoolName: "learners", vmSize: "Standard_NC4as_T4_v3", // (4vCPU, 28GB RAM, 1GPU (Nvidia Tesla T4), 0.0570$/hour) count: 1, minCount: 1, maxCount: 10, enableAutoScaling: true, osType: "Linux", osSKU: "Ubuntu", osDiskSizeGB: 100, osDiskType: "Premium_LRS", }, { replaceOnChanges: ["vmSize"] } ); // Optional - if you also want to run Simulators on machines with GPUs, provision this pool as well: const simsGpuPool = new containerservice.AgentPool("simsgpu", { resourceGroupName: resourceGroup.name, resourceName: k8sCluster.name, agentPoolName: "simsgpu", vmSize: "Standard_NC4as_T4_v3", // (4vCPU, 28GB RAM, 1GPU (Nvidia Tesla T4), 0.0570$/hour) count: 1, minCount: 1, maxCount: 10, enableAutoScaling: true, osType: "Linux", osSKU: "Ubuntu", }, { replaceOnChanges: ["vmSize"] } ); ``` ### Notes: 1. Autoscaling: * This template enables autoscaling to have the cluster automatically scale to the required size and back down afterward to reduce costs. * You can disable autoscaling by removing the `minCount`, `maxCount` and `enableAutoScaling` properties, but you'll have to set the `count` value accordingly. 2. `vmSize`: The vmSizes used above can be adjusted to instances that adhere more to your needs. # Azure ## Introduction You can find the AMESA Training Cluster Azure application [here](https://portal.azure.com/#view/Microsoft_Azure_Marketplace/GalleryItemDetailsBladeNopdl/id/composabl1715952518637.composabl-cluster-v1) This will set up a fully functional AMESA Training Cluster on your Azure tenant. ## Prerequisites 1. An active Azure subscription 2. Quota for the supported VM sizes (see later during setup) 3. If using GPU training, you'll need quota for GPU-enabled nodes (such as the [NCasT4\_v3 series](https://learn.microsoft.com/en-us/azure/virtual-machines/sizes/gpu-accelerated/ncast4v3-series?tabs=sizebasic)) ## Overview Setting up your cluster consists of the following steps: 1. Registering a new cluster in AMESA 2. Running through the wizard in Azure ## Registering a new cluster in AMESA 1. Head to [AMESA](https://app.composabl.com) and navigate to the clusters page. 2. Choose "Create Cluster" 3. Select "Azure Cluster"
4. Take note of the Cluster ID and License key - you'll need them during the setup wizard on Azure, then click "Create on Azure" ## Creating your cluster on Azure 1. Visit the Microsoft Azure Marketplace page [here](https://portal.azure.com/#view/Microsoft_Azure_Marketplace/GalleryItemDetailsBladeNopdl/id/composabl1715952518637.composabl-cluster-v1) 2. Click "Create" to start the setup wizard
1. On the first step, select a resource group to create the cluster in. We recommend you use a new Resource Group made specifically for this purpose. Give the resource a descriptive name.
2. On the second step, fill in the following:
* Cluster name - this is a technical name and should be unique for all AKS clusters in your Azure subscription * Kubernetes version - select the latest version, unless a policy prevents you from doing so. Only supported versions are listed. * Container registry name - this is the name of the container registry you'll be pushing your simulator images to. As with the Cluster name, this name has to be unique for all Azure Container Registries in your Azure subscription. * GPU Training - Check this if you want to use GPUs to accelerate training. * Various VM sizes - See [VM sizing and types](#VM-sizing-and-types) for additional information on each of these * Scaling settings - Whether to enable autoscaling, and limits for the autoscaling, if enabled. Note that we recommend you leave autoscaling enabled, as this can significantly reduce costs. 3. On the final step, fill in the Cluster ID and License key you received during your [Cluster Registration](#registering-a-new-cluster-in-amesa)
* The advanced settings should not be used unless you require specific features to be enabled for testing or support purposes. Finally, click "Review + Create" to review the settings and create your training cluster. ## VM sizing and types A AMESA cluster consists of up to 6 different node types: 1. Main nodes - these are Kubernetes system nodes, used for running the Kubernetes master processes 2. AMESA nodes - these nodes run the AMESA software - e.g. the AMESA Controller and the Historian processor 3. Environment runner nodes - The AMESA SDK will manage the simulator instances from these nodes. If GPU training is disabled, the training process will run on these nodes as well. 4. Simulator nodes: These nodes are where the simulator(s) will be running If you have enabled GPU training and/or GPU simulators, you will have the following: 5. (GPU Training enabled) Learner nodes - these GPU-enabled nodes will perform the actual training on GPU, after samples are collected from the simulators 6. (GPU Training + GPU Sims enabled) - these nodes will run GPU-enabled simulators. For most cases, we recommend leaving the default sizing. If no quota is available in these for you, you have the option to change to a similar node size for any of these. # Connecting a Cluster ## Introduction This page covers manually installing the AMESA components onto the cluster. ## Requirements To install AMESA on your cluster, you'll need access to the [AMESA No-Code UI](https://app.amesa.com) and a valid license key. Additionally, to run the installation script, you need to have Kubectl set up with access to the cluster. ## Steps 1. Log in to the No-Code app 2. Navigate to the Clusters page 3. Click "Create"
1. In the pop-up, choose a display name for your cluster
1. Select the cloud vendor you're using, or "custom" if none. 2. Copy the command 3. Run the command in your terminal where you have access to a kubectl 4. When the command exits without errors, the installation was complete. 5. Refresh the clusters page. Your custom cluster should now say "Connected" and list its specifications. You can now select the cluster from the dropdown menu when training. # Resolving Certificate Issues for Installing the AMESA SDK on WSL If you use Windows Subsystem for Linux (WSL) on a corporate machine with internal security tools, such as Zscaler, you may encounter issues installing the SDK due to certificate replacement during HTTPS traffic inspection. Below is a method to resolve this issue by importing the appropriate certificates into WSL. ## Step-by-Step Guide to Import Certificates into WSL 1. **Identify the Certificates** * Open in your browser. * Click on the padlock symbol next to the URL to view the site certificates. This method may vary slightly depending on your browser. * Export the certificates in the chain needed to trust the "app.composabl.com" certificate, which may be replaced by Zscaler or similar security tools. The intermediate and root certificates are often issued by your organization's IT department. ![](/files/ST3OqqiRIydBmcAWadwN) 2. **Export the Certificates** * Export the certificates in **Base 64 encoded X.509 (PEM)** format. * Make sure the exported certificate files have a `.crt` extension. 3. **Copy Certificates to WSL** * Copy the exported certificates to `/usr/local/share/ca-certificates` in WSL. You can use any file transfer method between Windows and WSL. 4. **Update the Certificate Store** * Run the following command to update the certificate store in WSL: `sudo update-ca-certificates\` 5. **Verify the Certificate** * You can verify if the certificates are correctly recognized by running: `openssl s_client -connect api.composabl.com:443 -prexit` > This command should connect and print the certificate chain and other relevant information. 6. **Removing Certificates** * If you need to remove the certificates, delete them from `/usr/local/share/ca-certificates` and then run: \`sudo update-ca-certificates --fresh\` > This command will rebuild the certificate bundle from the remaining individual certificates. 7. **Alternative Method: Export from Windows Certificate Store** The intermediate/root certificates may already be available in the Windows certificate store. You can identify and export the required certificates from there if needed. By following these steps, you should be able to configure your WSL environment to trust certificates replaced by corporate security tools, enabling the installation of the SDK without issues.