SECURITY ENGINEERING / HOME LAB
Deployment & investigation case study · Updated 30 October 2026

RedAmon on Proxmox.
From deployment to agent debugging.

Building a resource-conscious RedAmon v7.4.0 environment, connecting a local Ollama model, completing an initial reconnaissance run against an operator-owned website, and tracing an AI orchestration failure from the UI to its Python validation logic.

Core deployment healthyOllama API connectedInitial reconnaissance completedNeo4j graph populatedAI tool execution unresolvedNo exploitation performed
8 GBVM RAM allocation
4 vCPUVM processor allocation
9 / 9Core containers healthy at validation
5.0 GiBGuest RAM available at idle
01 / Executive summary

Objective and outcome

The objective was to evaluate whether RedAmon could operate in an existing, memory-constrained Proxmox home lab without purchasing additional server hardware or hosting a language model inside the RedAmon VM.

Constraints

Limited compute headroom

The Proxmox host has 32 GB RAM shared among security lab workloads. RedAmon was allocated 8 GB RAM, 4 vCPU and a 200 GB virtual disk. A 4 GB guest swap file provided limited contingency capacity.

Validated outcome

Working core deployment

The nine-container core deployment became healthy, administrator login worked, and RedAmon successfully connected to Ollama. An initial recon pipeline subsequently populated the asset graph; AI Agent tool execution remains unresolved.

Scope of this report

This updated case study includes a completed initial reconnaissance pipeline and graph observations, alongside an unresolved AI Agent tool-selection issue. It does not claim a completed autonomous agent workflow, vulnerability discovery or exploitation.

02 / Technical design

Two-machine architecture

Application services run on Proxmox; model inference remains on a separate Windows GPU machine. The LLM connection uses Ollama's OpenAI-compatible endpoint over the local network.

Proxmox / VM 700
RedAmon v7.4.0Ubuntu Server 24.04.3 LTS
192.168.1.51 · 4 vCPU · 8 GB RAM
Docker Engine / Compose
⇄
Windows GPU server
Ollama192.168.1.54:11434
OpenAI-compatible /v1 endpoint
qwen3:8b
Browser / OperatorRedAmon web UI · TCP 3000
Agent & ServicesDocker containers on VM 700
Local inferenceQwen3:8b on Windows GPU host
ComponentConfiguration
HypervisorProxmox VE, older Xeon E5-2648L host with 32 GB physical RAM
GuestUbuntu Server 24.04.3 LTS; VM ID 700
Guest resources4 vCPU, 8192 MiB RAM, 4 GiB swap, 200 GB virtual disk
Container runtimeDocker Engine 29.9.2; Docker Compose v5.6.1
RedAmon modeCore installation; GVM/OpenVAS and Knowledge Base omitted
LLM providerOpenAI-Compatible; http://192.168.1.54:11434/v1; model qwen3:8b
UI accesshttp://192.168.1.51:3000 on trusted local network
03 / Implementation

Deployment sequence

Provisioned the Ubuntu VM

Created VM 700 on Proxmox, configured VirtIO networking, installed Ubuntu Server, applied package updates, and enabled QEMU Guest Agent.

Installed Docker and Compose

Installed Docker from its official Ubuntu repository and verified execution using the hello-world container. Added the administration account to the Docker group after the installer initially failed to read the daemon version.

Cloned and initialized RedAmon

Cloned samugit83/redamon and invoked ./redamon.sh install. The installer generated application, database, MCP, scanner, and plugin secrets in the local environment configuration.

Built the core container images

Installer detected approximately 7 GB build RAM and 4 CPUs, selected build parallelism of two, and built the webapp separately to reduce memory pressure.

Completed setup and administrator login

Following build retries, nine services reached healthy status. The web interface loaded and an administrator successfully logged in.

Integrated existing Ollama server

Configured an OpenAI-Compatible LLM provider through RedAmon's Global Settings, using the Windows Ollama endpoint and qwen3:8b. The built-in Test Connection succeeded.

# Initial deployment
cd ~/redamon
./redamon.sh install

# Post-install checks
./redamon.sh status
free -h
docker stats --no-stream

# Verify Ollama reachability from the guest
curl http://192.168.1.54:11434/api/tags

Secrets and administrator passwords are intentionally excluded from this portfolio artifact.

04 / Engineering challenges

Problems encountered and resolution

01 · Configuration

Missing database credentials

Running ./redamon.sh status before installation reported missing POSTGRES_PASSWORD and NEO4J_PASSWORD. The .env file and containers did not yet exist.

Resolution: Run the normal installer and let RedAmon generate its own credentials.

02 · Permissions

Docker daemon inaccessible

The installer could not read the Docker Engine version when executed as the normal user, although Docker had worked with sudo.

Resolution: Grant the trusted lab administrator Docker group access, refresh group membership, and rerun installation without sudo.

03 · Session persistence

SSH disconnect during builds

An earlier lengthy build was interrupted around an SSH Connection reset message. A subsequent disconnect also interrupted the interactive administrator prompt.

Resolution: Use tmux for persistent sessions and capture installer output with tee.

04 · Dependency downloads

npm ECONNRESET

A webapp build failed during npm ci after roughly nine minutes with ECONNRESET. DNS, host HTTPS requests, and a Node container's npm API request later returned successfully.

Resolution: Retried the build without deleting Docker cache. The installation subsequently completed. The exact transient network cause was not established.

# Persistent build session and retained log
tmux new -s redamon-install
cd ~/redamon
set -o pipefail
./redamon.sh install 2>&1 | tee ~/redamon-install.log

# After a disconnect
tmux attach -t redamon-install

# Investigate a failed build
grep -iE 'error|failed|ECONNRESET|ETIMEDOUT' \
  ~/redamon-install.log | tail -40
Operational lesson

Long-running Docker builds should be launched inside a persistent session, with durable logs and preserved build cache. A successful HTTP test confirms immediate reachability but does not rule out intermittent connection resets during large dependency installations.

05 / Observed results

Container and resource validation

RedAmon's status command reported nine healthy containers, no OOM kills, no restarts, and no memory-cap drift at the time of validation.

ContainerObserved memoryConfigured limit
Neo4j1.08 GiB1.365 GiB
Kali sandbox599.6 MiB1.25 GiB
Agent288.9 MiB1.96 GiB
Webapp217.4 MiB1.906 GiB
Recon orchestrator78.49 MiB667 MiB
PostgreSQL50.45 MiB350 MiB
Plugin runner45.61 MiB667 MiB
Plugin gateway26.03 MiB128 MiB
Docker broker23.64 MiB240 MiB
Guest RAM

2.8 GiB used / 5.0 GiB available

Guest snapshot: 7.8 GiB total, 2.8 GiB used, 351 MiB free, 4.9 GiB buff/cache, 5.0 GiB available. Linux reclaimable cache is not the same as unavailable RAM.

Guest swap

34 MiB of 4.0 GiB

Very little swap was in use at the measured idle state. These results do not predict peak usage during active scans.

Validated integration

The Ubuntu VM received a model list from Ollama at 192.168.1.54:11434, including qwen3:8b, qwen3:14b and llama3.2. RedAmon's web-based provider connection test then succeeded using qwen3:8b.

06 / Controlled reconnaissance

First project: markcardiff.me

A project named First Test was created for the operator-owned static website markcardiff.me, hosted on Azure Storage static website infrastructure behind Azure Front Door. The initial reconnaissance pipeline ran and populated Neo4j. This establishes that the non-agent reconnaissance path can collect and persist basic asset relationships.

Observed assets

DNS relationship mapping

The graph showed markcardiff.me, www.markcardiff.me and a CNAME relationship to an azurefd.net endpoint. The graph later also displayed an AttackChain node associated with AI workflow activity; this node is not evidence that a scan or exploit ran.

Scope of results

Discovery, not vulnerability validation

The recorded evidence demonstrates domain and DNS relationship discovery. No verified open-port inventory, exposed service, exploitable weakness or successful AI-initiated scan was established from these results.

Project targetmarkcardiff.me
Recon pipelineDiscovery and DNS enrichment
Neo4j graphDomains, CNAME relationship and later AttackChain record

Performance during reconnaissance

With the reconnaissance workload active, the guest showed approximately 3.3 GiB used RAM and 4.5 GiB available. A vmstat 2 5 sample showed zero swap-in and swap-out activity. Proxmox's high displayed memory percentage largely reflected Linux caching rather than evidence of acute memory pressure.

# Useful point-in-time checks during a controlled run
free -h
vmstat 2 5
docker stats --no-stream
./redamon.sh status
Confirmed milestone

The initial reconnaissance workflow executed and produced graph data. This is separate from the AI Agent's ability to invoke tools autonomously, which remained unverified.

07 / AI Agent investigation

When the model can describe a tool but cannot invoke it

The AI Agent repeatedly reported that no reconnaissance data had been gathered, despite the populated project graph. Prompts requesting graph inspection and later Subfinder enumeration produced plausible reasoning in the UI, but no corresponding successful tool execution was observed.

Connectivity worked

Ollama returned successful responses

The agent loaded project settings and sent requests to http://192.168.1.54:11434/v1/chat/completions, receiving HTTP 200 responses. This ruled out a simple provider reachability failure in the observed sessions.

Execution failed

Invalid or incomplete tool decisions

Logs captured a Pydantic validation error for action="execute_subfinder", which is not an allowed action. A subsequent response used the allowed action="use_tool" but omitted tool_name; the orchestrator therefore generated a reply instead of executing a tool.

Reproducible log evidence

Parse attempt 1/3 failed: Validation error for LLMDecision
Input should be 'use_tool', 'plan_tools', 'transition_phase',
'complete', 'ask_user', 'deploy_fireteam' or 'switch_skill'
input_value='execute_subfinder'

Decision: action=use_tool, tool=None
THOUGHT: Starting with subdomain enumeration using subfinder
ACTION: use_tool
No valid action in decision: use_tool, tool: None

The same failure pattern was reproduced in two attempts on 30 October 2026. The AI UI showed a Subfinder plan followed by “Preparing response...” and a message that the scan had not begun.

Source-level trace

SourceObserved behaviourConsequence
agentic/state.pyLLMDecision requires thought and reasoning, but defines tool_name: Optional[str] = None.A decision can be schema-valid without naming an executable tool.
agentic/orchestrator_helpers/parsing.pytry_parse_llm_decision() repairs some malformed JSON and validates using LLMDecision.model_validate(data); no explicit use_tool / missing-name guard was found.An incomplete use_tool decision can pass parsing.
agentic/orchestrator_helpers/nodes/think_node.pyRetries parse failures with corrective feedback, then records the accepted decision.It retries invalid execute_subfinder, but can accept the subsequent incomplete decision.
agentic/orchestrator.pyRoutes to execute_tool only when action == "use_tool" and tool_name is populated.Otherwise logs “No valid action” and routes to response generation.
Important distinction

The log proves a decision-formatting and routing failure, not a confirmed defect in Neo4j or the reconnaissance engine. The precise cause of the model's malformed output—model capacity, prompt/tool registration, or provider configuration—has not yet been isolated. The UI's stated intention to run Subfinder is not proof of tool execution.

Diagnostic commands used

# Find decision schema, parser and routing code
grep -Rnl --include='*.py' 'class LLMDecision' agentic
grep -Rnn --include='*.py' 'def try_parse_llm_decision' agentic
grep -Rnn --include='*.py' -E 'No valid action in decision|Parse attempt .*failed' agentic

# Correlate user-visible agent activity with actual tool decisions
docker logs redamon-agent --since 15m 2>&1 |   grep -E 'Decision:|ACTION:|TOOL:|No valid action|Parse attempt|Executing tool|subfinder' |   tail -60

Settings discovered during follow-up

In a new project, the Enable AI in Pipeline master switch was observed OFF. Its tooltip indicates it controls per-tool AI hooks within the reconnaissance pipeline. This setting is distinct from the AI Agent, which was demonstrably making Ollama calls already. Domain Verification was also OFF, and the Engagement Limits panel indicated no active limits. For future controlled runs, domain verification and rate ceilings warrant deliberate configuration.

Proposed safeguard — not applied

A candidate improvement is to reject action="use_tool" when tool_name is missing, allowing the existing parse-retry mechanism to request a corrected decision. This was discussed but no source-code modification, container rebuild or successful retest was recorded. Comparing qwen3:14b with qwen3:8b is also proposed; no completed 14B comparison is documented.

08 / Next investigation milestones

Resume with controlled validation

Deployment and initial reconnaissance are complete. The next milestone is a single, observable AI Agent tool call, not a broad autonomous assessment.

ActionStatusEvidence required
Confirm active agent model and available registered tool namesOpenProvider settings and agent tool registry
Compare Qwen3 8B and 14B using the same scoped promptProposedParsed action, tool name and actual execution logs
Consider missing-tool-name validation safeguardNot appliedBackup, focused tests, container rebuild plan and rollback
Verify one benign graph query or approved recon tool callOpenTool invocation, output and graph changes
Configure engagement limits and verify scope controlsOpenRate cap, exclusions and domain ownership settings
Responsible test boundary

Only assess assets explicitly controlled or authorised by the operator. The website sits behind third-party Azure infrastructure; discovery of a shared cloud endpoint does not automatically place that infrastructure in scope. Avoid intrusive scanning until limits and scope are documented.

09 / Lessons learned

Engineering lessons from deployment and investigation

Resource-aware deployment

A complex, multi-container security platform can run at idle within a modest VM when optional heavyweight components are excluded and the model is hosted separately.

Operational troubleshooting

Credential initialization, Unix permissions, SSH session persistence and dependency download failures were handled without deleting application state or Docker cache.

Local AI integration

RedAmon's OpenAI-compatible provider can communicate with an existing Ollama instance, enabling reuse of the same GPU infrastructure as other lab automation projects.

Separate pipeline success from agent success

DNS reconnaissance populated Neo4j, while AI Agent requests failed before tool execution. Successful API connectivity and convincing model reasoning are not substitutes for verified actions and outputs.