TL;DR NVIDIA GPU monitoring needs more than a utilization chart. A production design should collect device telemetry with DCGM Exporter, scrape it with Prometheus, visualize fleet and workload behavior in Grafana, and alert on conditions that require action. The runbook must also preserve per-pod context, control metric cardinality, distinguish low utilization from genuine performance problems, … Read more
TL;DR Traditional logs tell operators what individual components recorded. Agent observability must answer a harder question: did an autonomous system remain inside its declared task, authorization boundaries, and approved methods across the complete sequence of actions? The required unit of detection is the trajectory. Prompts, tool calls, shell commands, identities, network destinations, package retrieval, privilege … Read more
TL;DR A coding agent should enter a CI/CD pipeline as an untrusted change producer, not as a privileged developer account. Give it an ephemeral sandbox, read-only repository access, a writable workspace, tightly controlled network egress, no deployment credentials, and no direct path to a protected branch. The agent should produce a patch and evidence bundle. … Read more
TL;DR NVIDIA NIM can be deployed on Kubernetes through Helm or managed declaratively through the NVIDIA NIM Operator. The operator-based path is the better fit when you want Kubernetes-native lifecycle management for model caching, GPU scheduling, health probes, service exposure, scaling, and upgrades. The practical sequence is straightforward, but the dependencies matter. Build a supported … Read more
Table of Contents 1. Introduction Deep learning workloads are transforming enterprise AI. Running them efficiently at scale requires more than just powerful GPUs. Kubernetes has become the go-to platform for orchestrating modern, containerized machine learning environments. With Kubernetes, you get reproducibility, scalability, and deep integration with DevOps workflows. This tutorial and proof-of-concept guide shows how … Read more
In the previous article we covered how to deploy an app using Docker Compose. In this article we will expand upon that and push our files to a GitHub Repo and connect the Repo to Travis CI for continual testing. Pre-reqs:Create a repository in Github – my example is docker-inferno Also, if you have not … Read more
I wasn’t expecting to get so in-depth into namespaces but I had another question come up. What if you have multiple namespaces in your environment and you wanted to see them all. Also, what if you were trying to find a particular pod in an unknown namespace. My previous namespace articles:https://digitalthoughtdisruption.com/2020/10/05/how-to-switch-between-namespaces-in-kubernetes/https://digitalthoughtdisruption.com/2020/10/02/kubernetes-create-namespace-with-default-quotas/https://digitalthoughtdisruption.com/2020/10/01/kubernetes-create-new-namespace-with-quotas/ How to see all … Read more
In my previous two articles I demonstrated how to create new namespaces with quotas. However, someone asked is it possible to switch to the new namespace instead of always being in the default namespace. Yes, yes you can. My previous two articles: As you can see when I issue the get pods commands it returns … Read more
Yesterday I posted a blog article around how to create a new namespace for a new deployment and apply quotas. However, I immediately got several requests around is there an easier way to do this or apply quotas at the namespace level for all future pod creations. Personal best practice; I like creating a yaml … Read more
There are times when projects and team members accessing the projects grow. In order to keep these environments isolated we incorporate namespaces and to ensure these projects don’t exceed their resource limits we can impose quotas. In this article I will provide a couple definitions followed by a two-step process. In step 1, I will … Read more
As we continue to dig deeper and build off each article, I wanted to mimic a Test/Dev workflow leveraging Docker. Below we will create an image and use Docker Compose to deploy the application in our Dev Environment. In part 2. we will take the results of this blog and send to Travis CI for … Read more
In this article as we begin to progress forward with deploying containers I wanted to build on previous articles. In this article we will create a package.json and server.js file, create a Dockerfile and see if our web app works. Now let’s create a directory for our project: $ mkdir <name_of_project>$ cd <DIR> Deploy web … Read more
I think Docker has the best explination of port mapping, “By default, when you create a container, it does not publish any of its ports to the outside world. To make a port available to services outside of Docker, or to Docker containers which are not connected to the container’s network, use the –publish or … Read more
In this article we will be covering building a simple custom Docker image. Following creating this Docker image we will deploy a container. Finally, we will add a couple more dependencies, add a tag, and re-deploy our final custom image. First let’s start with some Dockerfile basics:First word is an instruction that it tells the … Read more
Docker tags relay useful information about a specific image version. Tags often than not look like “image_name:#” the part following the colon is your tag. When you run the docker run command and the image is not local it will go download the latest version. In our example, that would be ubuntu 20.04.1 but what … Read more
As you have seen from previous articles deploying a single container can be easy but what if we need to deploy multiple containers that need to communicate with each other? Docker Compose is the tool you are seeking but first we have to install it. You can find the instructions here: First, let’s make sure … Read more
As I always say, the difference between a good engineer and a SME is repetitions. Already in this back to basics series, we’ve covered Git, GitHub, and YAML files. Now comes time to get familiar with Docker containers. The best way to do this is to build your own lab. There are a few ways … Read more
As we go through several of my DevOps series we will be writing in Python, JSON, & YAML. This is my preferred free tool to used to convert YAML into JSON or Python if the situation calls for it. Just copy & paste your YAML file into the left plane and at the bottom select … Read more
As we go through several of my DevOps series we will be writing and using YAML files. This is my preferred free tool to use to deploy YAML files with confidence. Copy & paste your .yml file into the left plane and click validate. This tool does a great job of identifying the exact line … Read more
As we go through several of my DevOps series we will be writing and using JSON files. This is my preferred free tool to use to deploy JSON files with confidence. Just copy and paste your JSON file and click Validate JSON and it will fix spacing errors or call out bad formatting. If you … Read more
