Maitento Core Concepts Guide

This guide provides a comprehensive introduction to Maitento, covering its architecture, core concepts, and key subsystems. It is intended for developers integrating with or building on the Maitento platform.

Table of Contents

1. What is Maitento?
2. Architecture Overview
3. Core Concepts
4. Authentication
5. Real-Time Notifications
6. RAG Pipeline
7. Build and Deployment

What is Maitento?

Maitento is an AI Operating System - a sophisticated platform that implements traditional operating system concepts for AI workloads. Just as a traditional OS manages processes, memory, and I/O for applications, Maitento provides equivalent abstractions for AI-powered interactions and automation.

The OS Analogy

This design philosophy allows Maitento to provide isolation, resource management, and orchestration for AI workloads in a familiar paradigm.

Architecture Overview

Technology Stack

• Language: C# (.NET 8.0)
• API Framework: ASP.NET Core with OpenAPI/Swagger
• Database: MongoDB
• Message Queue: RabbitMQ
• Logging: Serilog with Elastic APM integration
• Authentication: JWT Bearer + API Key + Custom Authorization Header schemes
• Frontend: Astro with Tailwind CSS
• Serialization: MessagePack (binary), System.Text.Json (APIs)

Component Architecture

+-------------------------------------------------------------+
|                        WebApi (REST)                        |
+-------------------------------------------------------------+
|  ApiServices: Agent, Interaction, Capsule, App, etc.        |
+-------------------------------------------------------------+
|  +-------------+ +-------------+ +-------------------------+ |
|  |   AiModels  | | Connectors  | |  ExternalOperations     | |
|  | (Anthropic, | |  (OpenAPI,  | |   (Tool execution)      | |
|  |   OpenAI)   | |    MCP)     | |                         | |
|  +-------------+ +-------------+ +-------------------------+ |
+-------------------------------------------------------------+
|  Process Runners: App, Interaction, Capsule, CodeGen        |
+-------------------------------------------------------------+
|  +----------------------------------------------------------+|
|  |          Cogniscript Virtual Machine                     ||
|  |  (Compiler -> Assembler -> VM -> Standard/Maitento Lib)  ||
|  +----------------------------------------------------------+|
+-------------------------------------------------------------+
|  Data Layer: MongoDB Repositories + Entity Framework        |
+-------------------------------------------------------------+
|  Infrastructure: MessageBus (RabbitMQ), Storage, VFS        |
+-------------------------------------------------------------+

Major Subsystems

Core Runtime

• CogniscriptCore - VM core implementation
• CogniscriptCompiler - Script compilation
• CogniscriptAssembler - Bytecode assembly
• CogniscriptVirtualMachine - VM execution engine
• CogniscriptStandardLibrary - Standard library functions
• CogniscriptMaitentoLibrary - Maitento-specific syscalls

Process Runners

• AppProcessRunner - Application execution
• InteractionProcessRunner - AI interaction execution
• ConversationRunner - Conversation handling
• CodeGenerationProcessRunner - AI-assisted code generation
• CapsuleProcessRunner - Capsule sandbox execution
• ProcessScheduler - Scheduled task execution

AI and Integration

• AiModels - AI model integrations (Anthropic, OpenAI)
• Connectors - External APIs (OpenAPI, MCP)
• ExternalOperations - External operation execution
• McpProxy - Model Context Protocol proxy

Data and State

• Data - MongoDB repositories (25+ implementations)
• Entities - 373 domain entity classes
• Storage - File storage abstraction
• VirtualFileSystems - VFS implementation

Core Concepts

Tenants

A Tenant represents an isolated organizational unit within Maitento. All resources (users, interactions, files, processes) belong to a tenant, providing multi-tenancy isolation.

• Each user belongs to exactly one tenant
• Resources are scoped to tenants via TenantId
• API authentication includes tenant context via JWT claims

Namespaces

Namespaces provide logical grouping and organization of resources within a tenant. They enable:

• Resource naming without collisions
• Logical separation of projects or environments
• Reference resolution via @ref("namespace.resourceName")

Processes

Maitento uses several types of processes, each serving different purposes:

Each process has a lifecycle managed by its runner, with states tracked and communicated via the message bus.

Interactions

Interactions are the primary way to engage AI agents. They define:

• Which agent(s) participate
• Input parameters and prompts
• Output schemas
• Available external operations (tools)

Interaction types include:

• OneShot - Single-turn interactions
• RoundRobin - Multi-turn conversations with multiple agents
• Managed - Programmatically controlled interactions
• Routed - Dynamic agent routing

Agents

Agents are AI entity configurations that define:

• The underlying AI model (e.g., GPT-4, Claude)
• System prompts and behavior guidelines
• Available functions/tools
• Response format constraints

Capsules

Capsules are containerized sandbox environments that provide:

• Isolated execution environments
• Package management
• File system mounts
• User management
• Startup commands

External Operations

External Operations are callable tools that agents can invoke:

• OpenAPI Operations - REST API endpoints with schema definitions
• MCP Operations - Model Context Protocol tools

Authentication

Maitento supports multiple authentication schemes to accommodate different use cases.

Authentication Methods

1. JWT Bearer Authentication

Standard JWT token authentication using RSA-SHA256 signed tokens.

Token Lifetime: Configurable (default: 24 hours) Clock Skew Tolerance: 2 minutes

JWT Claims:

Login Flow:

1. Client sends email + password to PUT /auth
2. Server validates credentials and creates JWT
3. Token returned with expiration timestamp
4. Client includes token in Authorization: Bearer <token> header

Token Refresh:

• Proactive refresh at 80% of token lifetime
• Call GET /auth/renew with current Bearer token

2. API Key Authentication

For programmatic access without user credentials.

Header Format:

Authorization: ApiKey {ClientId} {ClientSecret}

Features:

• ClientId is a GUID (user ID)
• ClientSecret is SHA256 hashed for storage
• Optional expiration dates
• Named keys for identification
• System-generated keys for internal use

Creating API Keys:

POST /users/self/api-keys

3. Authorization Header Scheme

For WebSocket connections where standard headers are not available.

Usage:

GET /notifications?authorization=<jwt-token>

Role-Based Access Control

SDK Authentication

The Maitento SDK provides convenient factory methods:

// API Key authentication
var client = MaitentoClient.CreateWithApiKey(clientId, clientSecret, baseUrl);

// Username/Password authentication
var client = MaitentoClient.CreateWithCredentials(username, password, baseUrl);

// Pre-obtained JWT token
var client = MaitentoClient.CreateWithJwtToken(jwtToken, baseUrl);

Real-Time Notifications

Maitento provides a WebSocket-based notification system for real-time updates about processes and resources.

Architecture

+-------------------------------------------------------------+
|                     SERVER SIDE                              |
+-------------------------------------------------------------+
|  RabbitMQ Message Bus                                        |
|  +- AppProcessUpdatedMessage                                 |
|  +- CodeGenerationProcessUpdatedMessage                      |
|  +- InteractionProcessStreamMessage                          |
|  +- InteractionProcessUpdatedMessage                         |
|  +- IngestedFileUpdatedMessage                               |
|  +- CapsuleProcessUpdatedMessage                             |
|          |                                                   |
|          v                                                   |
|  UserNotificationRouter (routes by UserId/TenantId)          |
|          |                                                   |
|          v                                                   |
|  NotificationService (per connected client)                  |
+----------+---------------------------------------------------+
           | WebSocket (ws/wss)
+----------v---------------------------------------------------+
|                     CLIENT SIDE (SDK)                        |
+-------------------------------------------------------------+
|  NotificationClient                                          |
|  +- Manages WebSocket connection                             |
|  +- Handles authentication (ApiKey or JWT)                   |
|  +- Auto-reconnection with backoff                           |
|  +- Heartbeat pings every 15s                                |
|          |                                                   |
|          v                                                   |
|  Handler Invocation (async, on thread pool)                  |
|  +- INotificationClientHandlerCodeGenerationProcessUpdated   |
|  +- INotificationClientHandlerAppProcessUpdated              |
|  +- INotificationClientHandlerInteractionProcessUpdated      |
|  +- INotificationClientHandlerInteractionStream              |
|  +- INotificationClientHandlerIngestedFileUpdated            |
|  +- INotificationClientHandlerCapsuleProcessUpdated          |
+-------------------------------------------------------------+

Notification Types

Connection Configuration

URL Construction:

http://example.com/api  -> ws://example.com/api/notifications
https://example.com/api -> wss://example.com/api/notifications

Reconnection Settings:

• Normal disconnect timeout: 30 seconds
• Error disconnect timeout: 2 seconds
• Server ping timeout: 30 seconds
• Client heartbeat interval: 15 seconds

SDK Usage

Connecting:

var client = MaitentoClient.CreateWithApiKey(clientId, secret, baseUrl);
await client.Notifications.StartAsync();

client.Notifications.Connected += (s, e) => Console.WriteLine("Connected");
client.Notifications.Disconnected += (s, e) => Console.WriteLine("Disconnected");

Subscribing to Updates:

// Subscribe to all app process updates
var subscription = client.Notifications.Subscribe(new MyAppHandler());

// Subscribe to specific process only
var subscription = client.Notifications.Subscribe(new MyAppHandler(), processId);

// Unsubscribe
client.Notifications.Unsubscribe(subscription);

Handler Implementation:

public class MyAppHandler : INotificationClientHandlerAppProcessUpdated
{
    public Task OnAppProcessUpdated(AppProcess process)
    {
        Console.WriteLine($"Process {process.Id}: {process.Status}");
        return Task.CompletedTask;
    }
}

Streaming Handler (for real-time AI output):

public class StreamHandler : INotificationClientHandlerInteractionStream
{
    public Task OnInteractionStream(InteractionStreamUserNotification notification)
    {
        Console.Write(notification.Stream);  // Inline streaming output
        return Task.CompletedTask;
    }
}

RAG Pipeline

The Maitento RAG (Retrieval Augmented Generation) pipeline enables semantic search over ingested documents.

Pipeline Overview

File Upload -> Text Extraction -> Chunking -> Vector Generation -> Hybrid Search
     |              |               |              |              |
  WebAPI      MarkItDown/      TextChunker     OpenAI       MongoDB
             Pandoc/etc      (1750 chars)   Embeddings    Vector Search

Supported File Types

File Size Limit: 10 MB

Ingestion Flow

1. Validate file size and type
2. Parse and extract text content
3. Generate checksum for duplicate detection
4. Store file in filesystem
5. Create IngestedFile entity
6. Trigger chunking if document exceeds 750 words

Chunking

Algorithm: Microsoft Semantic Kernel TextChunker.SplitMarkdownParagraphs()

Configuration:

• Maximum tokens per paragraph: 1750 characters
• Overlap tokens: 100 characters

Chunking is automatically triggered for documents with more than 750 words.

Vector Generation

Model: OpenAI text-embedding-3-large

• Dimensions: 3072
• Max tokens: 8192
• Similarity metric: Dot Product

Vector Types:

1. Main Document Vector - Full text embedding (if under 8192 tokens)
2. Chunk Vectors - Per-chunk embeddings for larger documents

Hybrid Search

Endpoint: PUT /files/embeddings/search

Algorithm: Reciprocal Rank Fusion (RRF)

Search Pipeline:

1. Convert query to 3072-dimensional embedding
2. Vector search (70% weight): Top 20 candidates
3. Full-text search (30% weight): Top 20 candidates
4. RRF scoring: vs_rrf = 0.7/(rank+60), fts_rrf = 0.3/(rank+60)
5. Merge and deduplicate results
6. Return top 25 by combined score

Processing Timeline

Total time to searchable:

• Small file (<750 words): 1-3 seconds
• Medium file (750-5000 words): 3-8 seconds
• Large file (20 chunks): 10-20 seconds

Build and Deployment

MTR Resource Files

Maitento uses .mtr (Maitento Resource) files for declarative resource configuration.

Syntax

property_name = value
nested.property = value
array_property = [ item1, item2 ]
object_property = {
  inner_property = value
}

Value Types

Project Structure

project/
+-- agents/                    # AI Agent definitions
|   +-- *.mtr
+-- apps/                      # Applications
|   +-- [app-name]/
|       +-- *.mtr              # Optional metadata
|       +-- *.mtc              # Cogniscript code
+-- capsules/                  # Container templates
|   +-- *.mtr
+-- connectors/
|   +-- open-api/              # REST API connectors
|       +-- *.mtr
+-- external-operations/
|   +-- open-api/              # External functions
|       +-- *.mtr
+-- interactions/
|   +-- one-shot/              # Single-turn flows
|   |   +-- *.mtr
|   +-- round-robin/           # Multi-turn flows
|       +-- *.mtr
+-- json/                      # JSON schema files
|   +-- *.json
+-- prompts/                   # Prompt templates
    +-- *.md

Resource Examples

Agent Definition:

model = @ref("openAi.o4-mini")
prompt = @prompt("developer")
summary = "An expert software developer!"
functions = [
  {
    ref = @ref("doc-search")
    guidance = @prompt("guidance-doc-search")
  }
]

Capsule Definition:

template = "ubuntu:22.04"
packages = ["git", "curl", "python3"]
autoTerminate = false
maxDuration = "01:00:00"

users = [
    { name = "worker", sudoAs = [] }
]

startup = [
    { command = "apt-get", arguments = ["update"], wait = true }
]

OpenAPI Connector:

method = "PUT"
url = "http://127.0.0.1:4201/files/embeddings/search"
description = "RAG search endpoint"
schema.request = @json("doc-search.request")
schema.response = @json("doc-search.response")
auth = "Maitento"

OneShot Interaction:

agent = {
  name = "Developer"
  agent = @ref("developer")
}
task = @prompt("one-shot-task")
inputs.prompts = [
  {
    name = "task"
    description = "The task description"
    required = true
    type = "string"
  }
]
schema.solution = @json("output-schema")

Compilation Pipeline

.mtr file -> Tokenizer -> Parser -> FilePostProcessor -> ResourceGenerator -> Resource

Stages:

1. Tokenization - Lexical analysis to tokens
2. Parsing - Syntax analysis to AST (ResourceNode)
3. File Post-Processing - Resolve @json/@prompt references
4. Resource Generation - Convert AST to typed C# objects

Build and Deploy Commands

Build (Validate Only):

maitento build <project-path>

Deploy (Build + Persist):

maitento deploy <project-path>

Build Phases

PHASE 1: COMPILATION
  +-> ICompiler.Compile() for each resource
  +-> BuildResourceContainer populated

PHASE 2: REFERENCE RESOLUTION
  +-> Resolve all @ref() to GUIDs
  +-> BuildReferenceContainer populated

PHASE 3: DATABASE CREATION
  +-> Start transaction
  +-> Create/Update each resource
  +-> Commit if isDeploy=true

Microservices

Maitento deploys as a collection of microservices:

MCP Proxy

The MCP Proxy bridges Maitento’s external operations with the Model Context Protocol, allowing AI models to invoke tools during interactions.

Architecture:

AI Model (in InteractionProcessRunner)
    |
    v (via HTTP with JWT token)
McpProxy Server (TCP/HTTP port 4202)
    +- ListTools Handler
    +- CallTool Handler
    +- Authentication/Authorization
    |
    v
External Operations & Bindings (Connectors, APIs)

Supported Operations:

• OpenAPI Bound External Operations (REST APIs)
• MCP Bound External Operations (planned)

Summary

Maitento provides a comprehensive platform for building AI-powered applications with:

• Multi-tenant isolation for secure, separated workloads
• Flexible authentication via JWT, API keys, and WebSocket authorization
• Real-time notifications for live process updates
• Semantic search through the RAG pipeline
• Declarative configuration via MTR resource files
• Microservice architecture for scalable deployment

For detailed API references and additional guides, see the accompanying documentation.