Local AI for DevOps: OpenCode, Ollama and an AWS EKS Experiment

Can a local coding model help build an infrastructure project—not just explain a command or generate a YAML snippet?

I explored that workflow with OpenCode, Ollama and Qwen2.5-Coder 7B running together inside a Linux VM on Proxmox, using an assignment that covered a Python application, Docker, Terraform, AWS EKS and Argo CD. The AWS identity and CLI credentials were prepared before starting the agent.

The interesting part was the complete path: a written task, repository changes, deployment configuration and verification instructions. The equally important part was separating generated files from demonstrated results.

Project: rizbuk/step on GitHub
Deployment documentation: project README

What runs locally—and what does not

The working environment was a clean Linux installation in a virtual machine on Proxmox. OpenCode and Ollama were installed directly in that guest OS, and the model ran through Ollama there. Neither tool ran in a container. AWS hosted the EKS deployment; Docker was part of the application assignment, not the runtime for the local AI tools.

Proxmox host
└── Linux VM — local working environment
    ├── OpenCode → Ollama API on 127.0.0.1:11434 → Qwen2.5-Coder
    ├── Project files and shell tools
    └── AWS CLI / Terraform / kubectl
                    |
              AWS IAM identity
                    |
                 AWS EKS

Ollama runs model inference. OpenCode provides the coding-agent interface and tools. AWS runs the demonstration infrastructure.

The Proxmox host is not the EKS worker node. The VM is the workstation from which the agent and infrastructure tools operate.

Local inference does not make the entire workflow offline. Package downloads, GitHub, container registries and AWS still involve network access. A VM boundary also does not prevent an agent from using the cloud credentials or network access available inside it.

Install Ollama and download Qwen2.5-Coder

Inside the Linux VM on Proxmox:

curl -fsSL https://ollama.com/install.sh | sh
sudo systemctl enable --now ollama

ollama --version
ollama pull qwen2.5-coder:7b
ollama run qwen2.5-coder:7b

Ollama’s local API normally listens on port 11434. OpenCode can connect through its OpenAI-compatible endpoint:

http://127.0.0.1:11434/v1

Both tools used the same Linux guest OS, so OpenCode reached Ollama through the VM’s loopback address.

Connect OpenCode to Ollama

Install OpenCode:

curl -fsSL https://opencode.ai/install | bash
opencode --version

Create a project directory:

mkdir -p ~/local-ai-devops
cd ~/local-ai-devops

Save the following as opencode.json in that directory:

{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "ollama": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "Ollama (local)",
      "options": {
        "baseURL": "http://127.0.0.1:11434/v1"
      },
      "models": {
        "qwen2.5-coder:7b": {
          "name": "Qwen2.5-Coder 7B"
        }
      }
    }
  }
}

Launch with an explicit model:

opencode --model ollama/qwen2.5-coder:7b

This selects Qwen2.5-Coder 7B, not Qwen3-Coder.

The configuration follows the OpenCode Ollama provider documentation. Configuration and tool-calling behaviour can vary between versions, so record the versions used when reproducing the experiment.

Before providing cloud access, check a small task: reading a file, editing a disposable file and running a harmless validation command. A successful chat response alone does not verify the agent’s tool workflow.

Context length: avoid treating 131k as a switch

Infrastructure projects quickly accumulate context: Terraform modules, Kubernetes manifests, command output and deployment documentation.

Ollama exposes a context-length setting, but requesting a larger window does not prove that a particular model and runtime support it correctly.

The Qwen2.5-Coder-7B-Instruct model card explains that its configuration uses 32,768 tokens and describes additional requirements for longer contexts.

For a bounded starting configuration on a systemd-managed Ollama host:

sudo systemctl edit ollama

[Service]
Environment="OLLAMA_CONTEXT_LENGTH=32768"

Then restart the service and run a model request:

sudo systemctl daemon-reload
sudo systemctl restart ollama
ollama run qwen2.5-coder:7b

While the model is loaded, inspect it from another terminal:

ollama ps

Check allocated context, memory use and CPU/GPU placement. Increase the context only after confirming model support and available memory. See Ollama context-length documentation.

Give the agent a concrete infrastructure assignment

The working directory initially contained only the assignment file, 1.txt. No starter application or infrastructure scaffold was supplied. The model, tools and AWS credentials had been prepared separately; the project files were created during the experiment and later committed to GitHub.

Its scope included:

• A minimal Python HTTP backend and Dockerfile.
• GitHub Actions for building and publishing an image.
• Terraform configuration for AWS EKS.
• Kubernetes Deployment, Service and Ingress resources.
• Argo CD installation and GitOps delivery.

A task file gives the agent a stable specification and gives the reviewer something to compare against the implementation.

For this type of project, useful acceptance checks include application health, manifest rendering, Terraform validation, image availability and the observed GitOps rollout.

AWS IAM user and CLI configuration before OpenCode

Before starting OpenCode, I created a separate IAM user in AWS with the permissions needed for the assignment. Then I configured the AWS CLI inside the same Proxmox VM that would run OpenCode:

aws configure

The interactive setup asks for the access key ID, secret access key, default region and output format. The region for this deployment was eu-central-1. The keys belong in the local credential configuration—not in this article, the task file or a Git commit.

This workflow used the default AWS CLI profile, rather than assuming a named opencode profile had been created. With no environment overrides, check the prepared identity before starting the agent:

aws sts get-caller-identity
aws configure list

AWS CLI configuration and credentials are normally stored under ~/.aws/ for that operating-system user. Environment variables or explicit profile settings can override the default selection. See the AWS CLI configuration documentation.

OpenCode was then launched from the project directory in the prepared VM:

cd ~/local-ai-devops
opencode --model ollama/qwen2.5-coder:7b

The model did not create its own initial AWS access. Its shell tools used credentials that I had configured in advance. Those permissions determined what Terraform and AWS CLI commands could do.

The IAM user was created in a test AWS account on the Free Tier plan for this experiment.

Security boundaries for a local AI DevOps agent

Local model inference changes where the model runs; it does not remove the risks of an agent executing shell commands. In this setup, the boundaries to review are the VM, the operating-system account, its credentials and the AWS permissions.

Security considerations for repeating this setup:

• Limit local access: use a dedicated VM and working account; avoid mounting Proxmox administration credentials, unrelated data or production secrets into the workspace. VM snapshots and backups also need protection if they contain credentials.
• Restrict AWS permissions: scope access to the required work and review IAM role creation and iam:PassRole permissions carefully. A separate IAM user is useful separation, but is not automatically least privilege.
• Protect credentials and state: keep AWS keys, kubeconfig, Terraform state and secret-bearing output out of public Git history. Do not paste credentials into prompts.
• Review execution boundaries: inspect plans and diffs before changes. Task files, repository content and tool output can contain untrusted instructions; cloud permissions remain the enforcement boundary.
• Keep inference private: use Ollama’s loopback endpoint for the co-located tools instead of exposing it publicly.
• Review exposure and activity: inspect security groups, administrative endpoints and CloudTrail records; retire experiment credentials after the required cleanup work is complete.

For ongoing use, follow AWS IAM security best practices: temporary credentials and least-privilege access.

Review the AWS identity and Terraform plan before allowing infrastructure changes. EKS, worker instances and load balancers can generate ongoing charges.

Why access to AWS mattered: test, inspect and correct

The agent worked interactively: it generated configuration, ran commands and corrected problems using the results.

With AWS access available through the prepared CLI credentials, the agent could run tools, inspect resource state and use Terraform or Kubernetes errors to guide the next change. The working approach was a feedback loop:

Read task → generate files → run a check or deployment step
                                      |
                           inspect output and errors
                                      |
                           revise the configuration
                                      |
                                check again

A configuration can look plausible in a chat response and still fail because of permissions, provider behaviour, resource dependencies or deployment conditions. Access to the actual environment lets those assumptions be tested rather than left for the reader to discover.

Without AWS access, the agent could still write the project files and run some local checks, but a person or a separate execution environment would have to perform the cloud validation.

The model used tool feedback within the session to revise its output; its weights were not being retrained.

That feedback loop needs limits: iteration against real AWS resources can create costs or change infrastructure. Keep the task scoped, review plans and permissions, and use a controlled experiment environment rather than treating production as a trial-and-error workspace.

What the repository records

The project README documents an EKS demonstration named 0root-demo, an application deployment and an Argo CD synchronization test.

It also distinguishes the generated Docker Hub workflow from the demonstration deployment: Docker Hub credentials were unavailable for the authenticated push test, and an ECR image was used instead.

That is a useful boundary:

Generated configuration, a successful validation command and a working deployment are different levels of evidence.

The resulting application code, infrastructure configuration and deployment instructions are available in the public repository.

Terraform output from the model-assisted deployment

The following output was captured from the experiment. It records the endpoints and connection commands produced by that deployment; the resource names may differ on a later run.

app_url = "http://app.0root-demo.devops1.0root.com"
argocd_initial_admin_password_command = "kubectl -n argocd get secret argocd-initial-admin-secret -o jsonpath='{.data.password}' | base64 -d"
argocd_url = "http://argocd.0root-demo.devops1.0root.com"
cluster_endpoint = "https://D15505FAB8AD76C102B8A8E874376EFD.gr7.eu-central-1.eks.amazonaws.com"
cluster_name = "0root-demo"
cluster_version = "1.33"
configure_kubectl = "aws eks update-kubeconfig --region eu-central-1 --name 0root-demo"
gitops_repo_url = "https://github.com/rizbuk/step.git"
ingress_load_balancer = "a99956bde1d6d40199a61bd80daf8774-a1d6bacad024ab42.elb.eu-central-1.amazonaws.com"
region = "eu-central-1"

argocd_initial_admin_password_command contains a command, not the password itself. cluster_endpoint is the Kubernetes API endpoint; it is separate from the shared application load balancer.

DNS setup: two CNAME records, one AWS Load Balancer

The Terraform output was not the final DNS configuration. I also needed to add two CNAME records, both pointing to the same AWS load balancer hostname:

Use the hostname returned by your own terraform output ingress_load_balancer, without http:// or a path, as the CNAME target. The value above belongs to the recorded experiment.

How ingress routes the two hostnames

There is one shared AWS Load Balancer for the application and Argo CD—not a separate load balancer for each hostname.

app.0root-demo.devops1.0root.com -----+
                                   +--> one AWS Load Balancer
argocd.0root-demo.devops1.0root.com --+            |
                                           ingress-nginx
                                                |
                                    match HTTP Host / Ingress rule
                                      |                    |
                                app hostname         argocd hostname
                                      |                    |
                              application Service   argocd-server Service
                                      |                    |
                              application Pods        Argo CD Pods

DNS resolves both names to the same load balancer. The AWS NLB forwards connections to ingress-nginx; the ingress controller selects the destination using the HTTP Host header and the matching Ingress rule, then proxies the request to the configured backend. DNS itself does not select the application. This is name-based virtual hosting in Kubernetes Ingress.

For this HTTP demonstration, the same load balancer endpoint can be tested with two different host headers:

LB=$(terraform output -raw ingress_load_balancer)

curl -i -H 'Host: app.0root-demo.devops1.0root.com' "http://$LB/"
curl -i -H 'Host: argocd.0root-demo.devops1.0root.com' "http://$LB/"

These commands run from the deployment’s Terraform directory and test host-based routing independently of the custom DNS records. A response or redirect from the expected backend is the first check; it is not a complete application health test.

After creating the CNAME records, check resolution and the normal public URLs:

dig +short CNAME app.0root-demo.devops1.0root.com
dig +short CNAME argocd.0root-demo.devops1.0root.com

curl -I http://app.0root-demo.devops1.0root.com
curl -I http://argocd.0root-demo.devops1.0root.com

For HTTPS, both hostnames also need an appropriate certificate and TLS configuration. Adding CNAME records alone does not enable HTTPS.

Verify Terraform and Kubernetes separately

In a separate shell inside the prepared VM, use the configured AWS identity and inspect the checked-out configuration before applying it. Confirm that no unintended profile or credential overrides are active:

export AWS_DEFAULT_REGION=eu-central-1
aws sts get-caller-identity

git clone https://github.com/rizbuk/step.git
cd step/terraform

terraform init
terraform fmt -check
terraform validate
terraform plan

Validation does not establish that the architecture is secure or production-ready.

If you proceed with the demonstration, apply only after reviewing the planned resources and account:

terraform apply
terraform output

Configure access to the documented cluster:

aws eks update-kubeconfig \
  --region eu-central-1 \
  --name 0root-demo

Then inspect actual resources:

kubectl get nodes -o wide
kubectl get pods -A
kubectl -n demo get deploy,po,svc,ingress
kubectl -n demo get events --sort-by=.lastTimestamp

Check the GitOps path

Inspect the Argo CD Application:

kubectl -n argocd get applications
kubectl -n argocd describe application 0root-demo-app

The expected delivery path is:

Change deployment configuration
        ↓
Commit and push
        ↓
Argo CD detects the revision
        ↓
Synchronization
        ↓
Kubernetes rollout and health checks

Building an image does not automatically update a Kubernetes image reference. Likewise, a successful synchronization should be followed by checking the rollout and application response.

Once Argo CD owns the application, use the repository as the normal change path. Direct kubectl apply changes can be reverted when self-healing is enabled.

Why this is a demonstration, not a production blueprint

Several details need explicit treatment before reusing the project:

• One worker node: this does not demonstrate worker-level high availability.
• Argo CD over HTTP: the README describes an insecure HTTP configuration. Administrative access needs an appropriate TLS and access-control design.
• ingress-nginx: the assignment uses a controller whose announced maintenance period ended in March 2026. For new deployments, evaluate a maintained alternative. See the Kubernetes retirement announcement.
• Versions and images: inspect current provider constraints, chart versions and image references rather than assuming the demonstration remains reproducible unchanged.

Keeping these limitations visible makes the experiment more useful than presenting generated infrastructure as production-ready.

Clean up the experiment

From the Terraform directory containing the state for this deployment:

aws sts get-caller-identity
terraform workspace show
terraform state list
terraform plan -destroy
terraform destroy

Review the resources before confirming destruction. Check afterward for remaining load balancers, volumes, registry images or other resources outside the managed state. Once cleanup is verified, deactivate and remove experiment access keys that are no longer needed, and remove the experiment IAM user if it is no longer required. Protect any retained VM snapshots or state backups that contain credentials.

What I take from this experiment

A local coding model can be part of a practical infrastructure workflow. The useful pattern is a concrete task, controlled access, reviewable changes and verification against explicit requirements.

The quality of the result should be judged by the evidence: what was generated, what was tested and what actually worked—not simply by whether the agent completed its conversation.

For production work, assessment, implementation planning and controlled delivery remain separate engineering responsibilities.

Explore DevOps & Infrastructure Engineering →
Discuss your infrastructure task →