Kubernetes Made Simple: How GKE Runs and Scales Applications on GCP
Kubernetes can feel confusing when you first learn it.
You hear words like:
- Cluster
- Node
- Pod
- Deployment
- Service
- Replica
- Autoscaling
- Load Balancer
Then someone shows you a huge YAML file.
Suddenly Kubernetes looks much harder than it really is.
The basic idea is actually simple.
Kubernetes is a system that helps you run and manage containers across many machines.
In Google Cloud, the managed Kubernetes service is called Google Kubernetes Engine, usually called GKE. Google describes GKE as a managed Kubernetes service for deploying containerized applications on Google Cloud.
In this article, we will understand Kubernetes using practical examples from Google Cloud.
We will also use an LLM inference API as our main example.
First, What Problem Does Kubernetes Solve?
Imagine you create an API using Python and FastAPI.
The API looks like this:
1User
2 ↓
3FastAPI
4 ↓
5LLM
6 ↓
7Response
You package the application inside a Docker container.
Then you create a Compute Engine virtual machine on Google Cloud and run the container.
1Internet
2 ↓
3Compute Engine VM
4 ↓
5Docker Container
6 ↓
7FastAPI
8 ↓
9LLM
Everything works.
But your application becomes popular.
Now thousands of users are sending requests.
One server is no longer enough.
So you create more machines.
1 Users
2 ↓
3 Load Balancer
4 ↓
5 ┌─────────┼─────────┐
6 ↓ ↓ ↓
7 VM 1 VM 2 VM 3
8 ↓ ↓ ↓
9 Container Container Container
Now you have another problem.
Who manages these machines?
What happens if VM 2 crashes?
Who starts another application instance?
What happens if traffic becomes ten times larger?
Who creates more containers?
What happens when traffic decreases?
Who removes unnecessary resources?
How do you update the application without shutting everything down?
How do containers find each other?
This is the type of problem Kubernetes solves.
Think of Kubernetes as a Manager
Imagine you own a restaurant.
You have:
120 employees
2
3100 tables
4
53 kitchens
6
7Thousands of customers
You need a manager to organize everything.
The manager decides:
1Which employee works where?
2
3How many employees are needed?
4
5Who replaces someone who is unavailable?
6
7How should customers be distributed?
8
9When should more staff be added?
10
11When can staff go home?
Kubernetes plays a similar role for your applications.
You tell Kubernetes what you want.
For example:
1I want 5 copies of my API running.
Kubernetes tries to make sure that 5 copies keep running.
If one crashes, Kubernetes can create another one.
This idea is extremely important.
Desired State
Kubernetes works around something called desired state.
You describe what you want.
For example:
1Desired state
2
3Application: recommendation API
4
5Number of copies: 3
Kubernetes looks at reality.
1Current state
2
3Application copies running: 2
Then Kubernetes notices:
1Desired = 3
2
3Current = 2
Something is wrong.
So Kubernetes creates another copy.
Now:
1Desired = 3
2
3Current = 3
Everything matches again.
This happens automatically.
You usually do not tell Kubernetes every individual action.
You describe the state you want.
Kubernetes tries to maintain it.
Kubernetes on Google Cloud
You can install and manage Kubernetes yourself.
But that requires managing a lot of infrastructure.
Google provides Google Kubernetes Engine.
GKE manages much of the Kubernetes infrastructure for you. GKE currently supports two main operating modes called Standard and Autopilot. With Autopilot, Google manages more of the underlying infrastructure and configuration.
A simplified architecture looks like this:
1 Google Cloud
2 ↓
3 GKE
4 ↓
5 Kubernetes Cluster
6 ↓
7 ┌──────────┼──────────┐
8 ↓ ↓ ↓
9 Node 1 Node 2 Node 3
10 ↓ ↓ ↓
11 Pods Pods Pods
Now we need to understand what these words mean.
What Is a Kubernetes Cluster?
A cluster is a group of machines managed together by Kubernetes.
Imagine:
1GKE Cluster
2
3Node 1
4Node 2
5Node 3
6Node 4
These machines provide:
- CPU
- Memory
- GPU
- Network
- Storage
Your applications run inside this cluster.
In GKE, Pods run on nodes in your cluster.
What Is a Node?
A Node is basically a machine that Kubernetes can use to run applications.
In GKE, a node commonly comes from Google Cloud compute infrastructure.
Imagine you have:
1Node 1
2
3CPU: 8 cores
4Memory: 32 GB
Another node might have:
1Node 2
2
3CPU: 16 cores
4Memory: 64 GB
5GPU: NVIDIA GPU
Kubernetes looks at the available resources and decides where workloads should run.
For example:
1Application A needs
2
32 CPU
44 GB RAM
Kubernetes might place it on Node 1.
Another application might need a GPU.
1LLM Inference Server
2
38 CPU
440 GB RAM
51 GPU
Kubernetes should place that workload on a node that has the required GPU resources.
What Is a Pod?
This is one of the most important Kubernetes concepts.
Your application usually runs inside a Pod.
A Pod is the smallest unit Kubernetes normally manages.
Think about it like this:
1Node
2 ↓
3Pod
4 ↓
5Container
6 ↓
7Your Application
Suppose you have a FastAPI application.
1Pod
2 ↓
3Docker Container
4 ↓
5FastAPI
Or imagine you run vLLM.
1Pod
2 ↓
3Container
4 ↓
5vLLM
6 ↓
7LLM
A node can run many Pods.
1Node 1
2
3┌─────────────┐
4│ Pod A │
5├─────────────┤
6│ Pod B │
7├─────────────┤
8│ Pod C │
9└─────────────┘
One important idea is that Pods should usually be treated as replaceable.
A Pod can disappear.
Another Pod can be created.
Your system should normally not depend on one particular Pod living forever.
Why Not Just Run Containers Directly?
You might ask:
Why do we need Pods? Why not just run Docker containers?
Because Kubernetes needs something it can manage.
A Pod gives Kubernetes a standard unit for things such as:
1Scheduling
2
3Networking
4
5Storage
6
7Health checking
8
9Resource allocation
10
11Restart behavior
For beginners, it is usually enough to remember:
A Pod is where your application container runs.
What Is a Deployment?
Suppose you want your API running in three Pods.
You could manually create:
1Pod 1
2
3Pod 2
4
5Pod 3
But that would create another problem.
What happens when Pod 2 crashes?
You would need to create another Pod yourself.
Instead, Kubernetes gives us something called a Deployment.
You can tell the Deployment:
1Application:
2Recommendation API
3
4Desired replicas:
53
Then Kubernetes tries to maintain three Pods.
1 Deployment
2 ↓
3 Desired replicas = 3
4 ↓
5 ┌─────────┼─────────┐
6 ↓ ↓ ↓
7 Pod 1 Pod 2 Pod 3
Suppose Pod 2 crashes.
Now Kubernetes sees:
1Expected Pods: 3
2
3Running Pods: 2
Kubernetes creates another Pod.
1 ┌─────────┼─────────┐
2 ↓ ↓ ↓
3 Pod 1 Pod 3 Pod 4
We are back to three.
This is one of the biggest benefits of Kubernetes.
What Is a Replica?
A replica is basically another copy of your application.
Imagine your API runs in one Pod.
1Pod 1
2 ↓
3API
You have one replica.
If you run three copies:
1Pod 1 → API
2
3Pod 2 → API
4
5Pod 3 → API
You have three replicas.
Why would you want multiple replicas?
Because one application instance may not handle all traffic.
Multiple replicas can also improve reliability.
If one Pod fails, others may continue serving users.
A GCP Example
Imagine you created an AI application.
Users upload text and your model creates embeddings.
Your architecture might start like this:
1Users
2 ↓
3GKE
4 ↓
5Embedding API Pod
Traffic increases.
Now you run four replicas.
1 Users
2 ↓
3 GKE
4 ↓
5 Deployment
6 ↓
7 ┌───────────┼───────────┐
8 ↓ ↓ ↓
9 Pod 1 Pod 2 Pod 3
10
11 ↓
12 Pod 4
The Pods all run the same application.
But How Do Users Reach the Pods?
Now we have another problem.
Pods can be created and destroyed.
Their network addresses can change.
Your user should not need to know:
1Send this request to Pod 1.
Or:
1Pod 1 disappeared.
2
3Now use Pod 4.
Kubernetes solves this using a Service.
What Is a Kubernetes Service?
A Service provides a stable way to reach a group of Pods.
GKE uses Kubernetes Services to group Pod endpoints and make workloads reachable through stable networking behavior.
Think of a Service like the reception desk of a hotel.
Guests do not need to know which employee is working.
They contact reception.
Reception sends the request to the correct person.
The architecture looks like this:
1 Users
2 ↓
3 Service
4 ↓
5 ┌──────────┼──────────┐
6 ↓ ↓ ↓
7 Pod 1 Pod 2 Pod 3
Users communicate with the Service.
The Service sends traffic toward Pods.
Service Versus Deployment
These two concepts are easy to confuse.
A Deployment answers:
How many copies of the application should exist?
A Service answers:
How do other systems reach those application copies?
So:
1Deployment
2 ↓
3Creates and maintains Pods
While:
1Service
2 ↓
3Provides access to Pods
Together:
1Users
2 ↓
3Service
4 ↓
5Deployment managed Pods
6 ↓
7Application
What About Traffic From the Internet?
Imagine you are building:
1api.example.com
Internet traffic needs to reach your GKE application.
A simplified setup might look like:
1Internet
2 ↓
3Google Cloud networking
4 ↓
5Load Balancer
6 ↓
7Kubernetes Service
8 ↓
9Pods
Google Cloud can integrate GKE workloads with Google Cloud load balancing.
The user does not need to know which Pod actually processes the request.
What Is a Load Balancer?
Imagine 10,000 users arrive at your API.
You have four Pods.
1Pod 1
2
3Pod 2
4
5Pod 3
6
7Pod 4
You do not want all requests going to Pod 1.
A load balancing layer distributes traffic.
Conceptually:
1 Users
2 ↓
3 Load Balancer
4 ↓
5 ┌───────────┼───────────┐
6 ↓ ↓ ↓
7 Pod 1 Pod 2 Pod 3
8 ↓
9 Pod 4
The goal is to spread requests across available application instances.
Kubernetes Self Healing
One of the most useful Kubernetes ideas is that applications can recover automatically from some failures.
Suppose your Deployment requires:
13 replicas
You currently have:
1Pod A
2
3Pod B
4
5Pod C
Then Pod B crashes.
Now:
1Pod A
2
3Pod C
Kubernetes sees that the current state does not match the desired state.
So another Pod is created.
1Pod A
2
3Pod C
4
5Pod D
This does not mean Kubernetes can fix every application bug.
If your application code is broken, Kubernetes cannot magically rewrite your code.
But Kubernetes can recreate failed workload instances and try to maintain the desired number of Pods.
What Happens When a Node Dies?
Now imagine something bigger happens.
The entire machine running your Pod disappears.
1Before
2
3Node 1
4 ↓
5Pod A
6
7
8Node 2
9 ↓
10Pod B
Node 1 fails.
Pod A disappears with it.
Kubernetes can schedule replacement workload capacity onto available infrastructure.
Conceptually:
1Node 1
2Failed
3
4
5Node 2
6 ↓
7Pod B
8
9
10Node 3
11 ↓
12New Pod A
This is one reason Kubernetes is useful for production systems.
Health Checks
How does Kubernetes know whether your application is healthy?
Applications can expose health information.
For example:
1Is the application alive?
2
3Is the application ready to receive traffic?
Kubernetes has health checking mechanisms that can help it make these decisions.
Imagine your API process exists, but the model is still loading.
1Container running
2
3Model loading
4
5API not ready
You probably do not want user traffic going there yet.
Once the model finishes loading:
1Container running
2
3Model loaded
4
5API ready
Now traffic can be sent to it.
This becomes especially useful for LLM inference because large models may take time to load.
Kubernetes Scheduling
Imagine your cluster has three nodes.
1Node A
2
34 CPU
416 GB RAM
5
6
7Node B
8
916 CPU
1064 GB RAM
11
12
13Node C
14
1532 CPU
16128 GB RAM
171 GPU
Now an LLM Pod needs:
18 CPU
2
364 GB RAM
4
51 GPU
Kubernetes needs to decide where that Pod should run.
Node A cannot handle it.
Node B does not have the required GPU.
Node C does.
So the scheduler places the Pod there.
Conceptually:
1LLM Pod
2 ↓
3Kubernetes Scheduler
4 ↓
5Node C
6 ↓
7GPU
This scheduling system becomes extremely useful when you have many workloads.
Resource Requests
How does Kubernetes know what your application needs?
You can describe resource requirements.
For example:
1Recommendation API
2
3CPU needed:
42 cores
5
6Memory needed:
78 GB
Another workload:
1LLM API
2
3CPU needed:
48 cores
5
6Memory needed:
764 GB
8
9GPU needed:
101
Kubernetes uses this information when deciding where workloads should run.
Correct resource configuration is important.
If you request far more resources than your application needs, you can waste capacity.
If you request too little, performance may suffer.
Now We Reach One of the Best Kubernetes Features
Autoscaling
Imagine your application normally receives:
1100 requests per minute
Three Pods are enough.
1Pod 1
2
3Pod 2
4
5Pod 3
Then something happens.
Traffic suddenly becomes:
110,000 requests per minute
Three Pods may no longer be enough.
You could manually increase replicas.
Or Kubernetes can automatically respond.
Horizontal Pod Autoscaling
Horizontal scaling means increasing or decreasing the number of Pods.
Google Kubernetes Engine supports Horizontal Pod Autoscaling, which can adjust workload capacity using metrics.
Imagine:
1Normal traffic
2
3Pod 1
4Pod 2
Traffic increases.
1High traffic
2
3Pod 1
4Pod 2
5Pod 3
6Pod 4
7Pod 5
8Pod 6
Traffic later decreases.
1Low traffic
2
3Pod 1
4Pod 2
This can help performance and cost.
Real Example With an AI API
Imagine you run an embedding model.
Normally:
12 Pods
During business hours, traffic increases.
CPU usage becomes high.
The autoscaler detects increased demand.
Kubernetes increases replicas.
12 Pods
2 ↓
34 Pods
4 ↓
58 Pods
When traffic decreases, it can reduce the number again.
18 Pods
2 ↓
34 Pods
4 ↓
52 Pods
You do not need eight Pods running all night if nobody is using them.
LLM Autoscaling on GKE
This is not only theoretical.
Google provides guidance for autoscaling LLM inference workloads running on GPUs in GKE. One documented example uses Gemma together with GKE Horizontal Pod Autoscaling.
Imagine you serve an LLM using GPUs.
1 Users
2 ↓
3 Service
4 ↓
5 LLM Deployment
6 ↓
7 ┌─────────┼─────────┐
8 ↓ ↓ ↓
9 LLM Pod LLM Pod LLM Pod
10 ↓ ↓ ↓
11 GPU GPU GPU
When traffic grows, you may want more inference replicas.
That is where Kubernetes becomes very useful for AI infrastructure.
But What If We Need More Machines?
This is different from adding Pods.
Suppose every node is full.
1Node 1
2
3FULL
4
5
6Node 2
7
8FULL
9
10
11Node 3
12
13FULL
Kubernetes wants to create another Pod.
But there is nowhere to put it.
Now you need more node capacity.
Cluster Autoscaling
GKE can automatically resize Standard cluster node pools based on workload demand. When workloads require more capacity, the cluster autoscaler can add nodes. When resources are no longer needed, capacity can be reduced.
So there are two different ideas.
Horizontal Pod Autoscaling:
1Need more application capacity
2
32 Pods
4 ↓
56 Pods
Cluster Autoscaling:
1Need more machine capacity
2
33 Nodes
4 ↓
55 Nodes
These can work together.
A Simple Autoscaling Example
Imagine:
13 Nodes
2
36 Pods
Traffic increases.
The Pod autoscaler wants:
112 Pods
But the current nodes only have space for 8.
So:
1Horizontal Pod Autoscaler
2
3Requests more Pods
4 ↓
5Current nodes become full
6 ↓
7Cluster Autoscaler
8 ↓
9Adds nodes
10 ↓
11New Pods can run
That is a powerful idea.
Your infrastructure can respond to changing demand.
Kubernetes for LLM Inference
Now let us connect this to LLM serving.
Suppose you want to serve an open model using vLLM.
A simplified system might look like:
1 Users
2 ↓
3 Cloud Load Balancer
4 ↓
5 Kubernetes Service
6 ↓
7 Deployment
8 ↓
9 ┌────────┼────────┐
10 ↓ ↓ ↓
11 Pod 1 Pod 2 Pod 3
12 ↓ ↓ ↓
13 vLLM vLLM vLLM
14 ↓ ↓ ↓
15 Model Model Model
16 ↓ ↓ ↓
17 GPU GPU GPU
Each Pod can run an inference server.
Kubernetes manages the Pods.
GKE manages much of the Kubernetes infrastructure.
Google Cloud provides the compute resources.
Your application sends requests through the networking layer.
Why Kubernetes Is Useful Here
LLM inference systems can have unpredictable traffic.
For example:
12 AM
2
320 requests per minute
Then:
12 PM
2
32,000 requests per minute
You may need different amounts of capacity.
Kubernetes gives you mechanisms for:
1Running multiple replicas
2
3Replacing failed Pods
4
5Scheduling workloads onto GPUs
6
7Scaling workloads
8
9Managing networking
10
11Updating applications
12
13Managing configuration
This makes it a useful platform for production AI systems.
What Is a Rolling Update?
Imagine you currently run:
1Model API Version 1
You create:
1Model API Version 2
You do not want to shut down every Version 1 Pod and then start Version 2.
Users might experience downtime.
Instead, Kubernetes Deployments can perform rolling updates, gradually replacing old Pods with updated ones. GKE also allows application Deployment updates through the Google Cloud console or Kubernetes configuration.
Conceptually:
1Beginning
2
3V1
4V1
5V1
6V1
Then:
1V1
2V1
3V1
4V2
Then:
1V1
2V1
3V2
4V2
Then:
1V1
2V2
3V2
4V2
Finally:
1V2
2V2
3V2
4V2
The application can remain available while the update happens.
Real AI Example
Suppose your inference server currently runs:
1Model Version 1
You created a better model:
1Model Version 2
Instead of replacing every running model server immediately, you can gradually replace application Pods.
This reduces the risk of suddenly taking the entire API offline.
What Is a Container Image?
We have talked about Pods and Deployments.
But where does your application actually come from?
Usually, you package it as a container image.
For example:
1Python code
2 ↓
3Dependencies
4 ↓
5Application configuration
6 ↓
7Container image
In Google Cloud, you can store container images in Artifact Registry.
A common flow looks like:
1Source Code
2 ↓
3Build Container
4 ↓
5Artifact Registry
6 ↓
7GKE
8 ↓
9Pod
So GKE knows which application image it should run.
A Simple Deployment Flow on GCP
Imagine you build a recommendation API.
Your code:
1Python
2
3FastAPI
4
5Recommendation Model
You create a container.
1Application
2 ↓
3Container Image
You store it in Google Cloud.
1Container Image
2 ↓
3Artifact Registry
Then deploy it to GKE.
1Artifact Registry
2 ↓
3GKE Deployment
4 ↓
5Pods
Users access it through your networking layer.
1Users
2 ↓
3Load Balancer
4 ↓
5Service
6 ↓
7Pods
Now we have a real production architecture.
What Is a ConfigMap?
Applications need configuration.
For example:
1LOG LEVEL
2
3SERVICE NAME
4
5FEATURE FLAG
6
7API URL
You probably do not want to hardcode all of these directly inside your application.
Kubernetes provides ConfigMaps for storing nonsecret configuration.
Conceptually:
1ConfigMap
2
3MODEL NAME = recommendation model
4
5LOG LEVEL = info
6
7ENVIRONMENT = production
Your Pod can use those values.
This allows you to change configuration without rebuilding your entire application image for every small setting.
What About Passwords and API Keys?
You should not treat passwords the same way as normal configuration.
Kubernetes provides Secrets for sensitive configuration.
Examples include:
1Database password
2
3API token
4
5Private credential
In a production GCP system, you may also use Google Cloud services designed for secret management and connect them with your workloads.
The important idea is:
1Normal configuration
2 ↓
3ConfigMap
4
5
6Sensitive configuration
7 ↓
8Secret management
What Are Namespaces?
Imagine your Kubernetes cluster contains many teams.
1Payments Team
2
3Recommendation Team
4
5Search Team
6
7AI Platform Team
Putting everything into one big group becomes messy.
Namespaces allow you to logically separate resources.
For example:
1Namespace: recommendation
2
3Pods
4
5Services
6
7Deployments
Another namespace:
1Namespace: payments
2
3Pods
4
5Services
6
7Deployments
Think of namespaces like folders inside a large workspace.
They help organize Kubernetes resources.
GKE Standard Versus Autopilot
GKE gives you different ways to operate Kubernetes.
Two important modes are:
1Standard
2
3Autopilot
With Standard mode, you have more control over node infrastructure.
With Autopilot, Google manages more of the underlying infrastructure and operational configuration for you.
A simple mental model is:
1Standard
2
3More infrastructure control
4More responsibility
And:
1Autopilot
2
3Less infrastructure management
4More management handled by Google
Which one you choose depends on your workload.
For specialized GPU workloads, networking requirements, or unusual infrastructure needs, you should evaluate both options based on the level of control you need.
Kubernetes Does Not Replace Docker
Another common confusion is:
Kubernetes versus Docker.
They solve different problems.
Docker helps package and run containers.
Kubernetes helps manage many containers.
Think:
1Docker
2
3How do I package and run this application?
Kubernetes:
1How do I manage thousands of these application containers?
They work together.
Kubernetes Does Not Replace GCP Either
GCP gives you infrastructure and cloud services.
Kubernetes manages containerized workloads.
For example:
1Google Cloud
2 ↓
3Provides machines
4networking
5storage
6GPUs
7identity
8monitoring
Then:
1GKE
2 ↓
3Runs Kubernetes
Then:
1Kubernetes
2 ↓
3Manages Pods
4Deployments
5Services
6Scaling
Then:
1Your containers
2 ↓
3Run your application
Putting Everything Together
Imagine you are building a production LLM API.
You use:
1Python
2
3FastAPI
4
5vLLM
6
7Open model
You package everything inside a container.
Then your architecture might look like this:
1 Internet
2 ↓
3 Google Cloud Network
4 ↓
5 Load Balancer
6 ↓
7 Kubernetes Service
8 ↓
9 Deployment
10 ↓
11 ┌────────────┼────────────┐
12 ↓ ↓ ↓
13 Pod 1 Pod 2 Pod 3
14 ↓ ↓ ↓
15 vLLM vLLM vLLM
16 ↓ ↓ ↓
17 LLM LLM LLM
18 ↓ ↓ ↓
19 GPU 1 GPU 2 GPU 3
20 ↓ ↓ ↓
21 └────────────┼────────────┘
22 ↓
23 GKE Cluster
Now imagine traffic increases.
1Traffic increases
2 ↓
3Autoscaler detects demand
4 ↓
5More Pods requested
6 ↓
7More capacity may be needed
8 ↓
9More node capacity becomes available
10 ↓
11Additional inference Pods run
When traffic decreases:
1Traffic decreases
2 ↓
3Fewer replicas needed
4 ↓
5Pods decrease
6 ↓
7Unused capacity can decrease
This is the type of system Kubernetes is designed to manage.
The Most Important Kubernetes Concepts
If Kubernetes still feels confusing, remember these simple definitions.
Cluster
A group of machines managed by Kubernetes.
1Cluster
2
3Node
4Node
5Node
Node
A machine that provides CPU, memory, GPU, and other resources.
1Node
2 ↓
3Pods
Pod
The place where your application container runs.
1Pod
2 ↓
3Container
4 ↓
5Application
Deployment
Maintains the number of application Pods you want.
1Deployment
2 ↓
3Pod
4Pod
5Pod
Replica
One copy of your application.
13 replicas
2=
33 application copies
Service
Provides a stable way to reach a group of Pods.
1Service
2 ↓
3Pod
4Pod
5Pod
Load Balancer
Distributes external traffic toward your application.
1Internet
2 ↓
3Load Balancer
4 ↓
5Application
Horizontal Pod Autoscaler
Changes the number of Pods based on workload demand.
12 Pods
2 ↓
38 Pods
Cluster Autoscaler
Changes available node capacity when workloads require more or fewer resources.
13 Nodes
2 ↓
35 Nodes
A Simple Mental Model
Think about an apartment building.
Cluster
The entire apartment complex.
Node
One apartment building.
Pod
One apartment.
Container
The person living inside the apartment.
Deployment
The manager who makes sure the required number of apartments are occupied.
Service
The reception desk that knows how to reach residents.
Load Balancer
The person distributing incoming visitors.
Autoscaler
The system that adds more capacity when more people arrive.
This is not technically perfect, but it is a useful way to remember the concepts.
What Kubernetes Is Really Doing
Behind all the terminology, Kubernetes repeatedly asks a few simple questions.
1What should be running?
2
3What is currently running?
4
5Where should it run?
6
7Is it healthy?
8
9Do we need more copies?
10
11Do we need fewer copies?
12
13How should traffic reach it?
That is the core idea.
You describe what your system should look like.
Kubernetes continuously tries to make reality match that description.
Final Takeaway
Do not think of Kubernetes as a tool for running one container.
Docker can already run one container.
Kubernetes becomes useful when you have a bigger problem.
For example:
1Many containers
2
3Many machines
4
5Many users
6
7Failures
8
9Traffic changes
10
11Application updates
12
13Different resource requirements
14
15GPU workloads
16
17Production reliability
On Google Cloud, GKE provides a managed way to run Kubernetes.
The easiest way to remember the whole system is:
1Google Cloud
2 ↓
3GKE
4 ↓
5Kubernetes Cluster
6 ↓
7Nodes
8 ↓
9Pods
10 ↓
11Containers
12 ↓
13Your Application
Then add the management pieces:
1Deployment
2 ↓
3Keeps the right number of Pods running
4
5
6Service
7 ↓
8Makes Pods reachable
9
10
11Autoscaling
12 ↓
13Changes capacity when demand changes
14
15
16Load Balancing
17 ↓
18Distributes incoming traffic
Once you understand this structure, Kubernetes stops looking like a collection of strange words.
It becomes a simple idea:
You tell Kubernetes how you want your applications to run, and Kubernetes keeps working to maintain that state.
That is why Kubernetes is so useful for production systems, including modern AI and LLM infrastructure.