GCP || ACE - Certification

Before we start, Some important things to know :-

• To begin with, it is crucial to understand that GCP provides over 200 services, but this exam necessitates expertise in 40+ services.

• The exam requires an in-depth understanding of these services and assesses your decision-making skills, i.e., which service to select in a given situation.
  
  
• The aim of this course is to equip you with comprehensive knowledge and facilitate making difficult choices. 
  
  
• Our ultimate objective is to help you comprehend and apply GCP to your real-world projects!

How do you improve your chances of remembering things?

• Active learning - think and take notes
  
  
• Review this blog or notes every once in a while.
  
  

How should you approach?

• When it comes to approaching the GCP exam, it is best to start with the theory section and discuss it with your friends, colleagues, or classmates.

• (Teaching others is considered one of the most effective ways of learning.)

• sorting out GCP exam multiple-choice questions (MCQs) for the topics you have studied can be helpful (refer to https://www.examtopics.com/exams/google/associate-cloud-engineer/view/).

• Finally, attempting practical exercises relevant to the exam can further improve your preparation.

Before the Cloud :-

Previously, we had on-premises data centers, which were physical data servers located within office premises. For example, we used to manage different-sized data (8gb, 16gb, 32gb, etc.) using memory cards. Nowadays, with the world moving towards cloud computing, we are transitioning to Google Drive, an application of the Google Cloud.

Challenge 1:-

Peak usage during holidays and weekends, Less load during rest of the time. 
e.g- Weekend Sales the crowd usually much then the other weekdays.

During holidays and weekends, peak usage is observed while less load is experienced during the rest of the time. For instance, during weekend sales, the crowd is usually much larger than other weekdays.

Solution without or before cloud:

PEAK LOAD provisioning : Procure (Buy) infrastructure for peak load.

What would the infrastructure be doing during periods of low loads????

Challenge 2:-

Startup suddenly becomes popular.

How to handle the sudden increase in load?

Solution without or before cloud:

Procure (Buy) infrastructure assuming they would be successful.

What if they failed?

Before the Cloud:- 

• High cost of procuring infrastructure
• Needs ahead of time planning (Can you guess the future?)
• Low infrastructure utilization (PEAK LOAD provisioning)
• Dedicated infrastructure maintenance team (Can a startup afford it?)

Silver Lining in the Cloud:-

1. How about provisioning (renting) resources when you want
   them and releasing them back when you do not need them?
2. On-demand resource provisioning
3. Also called Elasticity

Cloud - Advantages

• Trade "capital expense" for "variable expense"
• Benefit from massive economies of scale
• Stop guessing capacity (No assumption needed, reserve resources as required.)
• Stop spending money for running and maintaining data centers even when you are not using them.
• "Go global" in minutes (live the resources in minutes.)

"Trade capital expense for variable expense" refers to the concept of shifting from a traditional IT infrastructure model where companies have to purchase and maintain their own hardware and software, and bear the upfront costs (capital expenses), to a cloud computing model where resources are rented and charged on a pay-per-use basis (variable expenses).

With cloud computing, companies no longer have to invest heavily in IT infrastructure upfront, but can instead provision resources on demand and pay only for what they use. This allows them to avoid the high capital expenses associated with owning and maintaining IT infrastructure, and instead, pay only for the resources they consume, like a utility bill. The variable expenses are based on usage and can be adjusted as needed, providing more flexibility and cost savings.

Google Cloud Platform (GCP)

• One of the Top 3 cloud service providers
• Provides a number of services (200+)
• Reliable, secure and highly-performant:
• Infrastructure that powers 8 services with over 1 Billion Users: Gmail,
  Google Search, YouTube etc.
• One thing I love : "cleanest cloud"
• Net carbon-neutral cloud (electricity used matched 100% with renewable energy)

Best path to learn GCP!

Creating your first GCP Account.

//link will attached soon.

here are the steps for creating your first GCP account:

1. Go to the GCP Console at https://console.cloud.google.com/.
2. Click on the "Get started for free" button on the top right corner of the page.
3. Fill in your details such as name, email, country, and agree to the terms of service and privacy policy.
4. Select a payment method. You can choose to enter your credit card details or use a promotional code if you have one.
5. Verify your account by entering the verification code that was sent to your email address.
6. Set up your billing account by entering your billing details, such as your address and credit card information. You can also set up a budget alert to notify you when your spending exceeds a certain threshold.
7. Create your first project by clicking on the "Create Project" button. Enter a name for your project, select a billing account, and choose a location for your project.
8. Once your project is created, you can start using GCP services such as Compute Engine, Storage, and Networking by creating resources within your project.

Note that some GCP services may have a free tier, while others may require you to pay for usage. Be sure to review the pricing and usage documentation for each service before you start using it.

Regions and Zones :-

• Imagine that your application is deployed in a data center in London
• What would be the challenges?

• Challenge 1 : Slow access for users from other parts of the world (high latency)
• Challenge 2 : What if the data center crashes?
• Your application goes down (low availability)
  
  

Multiple data centers:-

What if we add another data center in a Region?

• CHALANGES:-
  
  
• Challenge 1 : Slow access for users from other parts of the world.
• Challenge 2 (SOLVED) : What if one data center crashes?
• Your application is still available from the other data center.
• Challenge 3 : What if entire region of London is unavailable?
• Application goes down!
  
  
  

Multiple regions
** Let's add a new region : Mumbai **

• CHALANGES:-
  
  
• Challenge 1 (PARTLY SOLVED) : Slow access for users from other parts of the world.
• You can solve this by adding deployments for your applications in other regions.
  
  
• Challenge 2 (SOLVED) : What if one data center crashes?
• Your application is still live from the other data centers.
  
  
• Challenge 3 (SOLVED) : What if entire region of London is unavailable?
• Your application is served from Mumbai.
  
  
  

Regions:-

• Imagine setting up data centers in different regions around the world Would that be easy?
• (Solution) Google provides regions around the world 20+ Expanding every year.
  
  
• Region : Specific geographical location to host your resources.
  
  
• Advantages:
• High Availability
• Low Latency
• Global Footprint
• Adhere to government regulations.

 Zones:-

• How to achieve high availability in the
  same region (or geographic location)?
• Enter Zones.
• Each Region has three or more zones.
• (Advantage) Increased availability and fault tolerance within same region.
• (Remember) Each Zone has one or more
  discrete clusters (distinct physical infrastructure that is housed in a data center.)
• (Remember) Zones in a region are connected through low-latency links.
  
  

Regions and Zones examples:-

 

                            Compute                                   
            
               Compute Engine Fundamentals
          (Provision & Manage Virtual Machines)

Google Compute Engine (GCE):-   

Google Compute Engine is a cloud computing service offered by Google that allows individuals and businesses to run their applications on virtual machines (VMs) hosted on Google's infrastructure.

In simpler terms, it's a way for people to rent computer power from Google to run their programs and store their data, without having to buy and maintain physical hardware themselves. It's like renting a computer in the cloud, where Google takes care of the underlying infrastructure and you only pay for what you use.

• In corporate data centers, applications are deployed to
  physical servers.

• Where do you deploy applications in the cloud?
• Rent virtual servers
• Virtual Machines - Virtual servers in GCP
• Google Compute Engine (GCE) - Provision & Manage Virtual Machines.

In traditional corporate data centers, applications are typically deployed on physical servers. However, in the cloud, applications are deployed on virtual servers, also known as virtual machines. In Google Cloud Platform (GCP), the service that allows you to provision and manage virtual machines is called Google Compute Engine (GCE). With GCE, you can rent virtual machines and deploy your applications on them, without having to manage physical hardware yourself.

GCE - Features:-

                                  

• Create and manage lifecycle of Virtual Machine (VM) instances.

• Load balancing and auto scaling for multiple VM instances

• Attach storage (& network storage) to your VM instances

• Manage network connectivity and configuration for your VM instances.

• Our Goal:
• Setup VM instances as HTTP (Web) Server
• Distribute load with Load Balancers

Load balancing- Load balancing in Google Compute Engine (GCE) refers to the process of distributing incoming network traffic across multiple virtual machines (VMs) to improve the performance, availability, and scalability of applications running in the cloud.

GCE provides various load balancing options, including:

1. Network load balancing: This distributes incoming traffic across multiple VMs based on network protocol data such as IP address and port number.

2. HTTP(S) load balancing: This distributes traffic across multiple VMs based on HTTP(S) protocol data, such as URL path and HTTP headers. It is ideal for web applications.

3. Internal load balancing: This distributes traffic across multiple VMs within a virtual private cloud (VPC) network.

GCE load balancing is highly available and scalable, and it can automatically route traffic to healthy VM instances while detecting and replacing failed ones. It helps improve the performance of applications by spreading the load across multiple VMs, reducing the chances of a single VM becoming overloaded and causing the application to slow down or become unavailable.

Auto Scaling- Auto scaling in Google Cloud Platform (GCP) refers to the ability to automatically adjust the number of virtual machines (VMs) in a managed instance group based on changes in demand for your application. This allows your application to handle sudden traffic spikes without manual intervention, while also minimizing costs during periods of lower demand.

In GCP, you can use managed instance groups to create and manage identical groups of VMs. You can set up auto scaling for a managed instance group to automatically add or remove VMs based on criteria such as CPU utilization, network traffic, and request rate.

When the auto scaling feature is enabled, GCP monitors the metrics you set and automatically adjusts the number of VMs in the managed instance group up or down based on the rules you define. For example, if your web application experiences a sudden spike in traffic, GCP can automatically add more VMs to handle the increased demand. Conversely, if traffic decreases, GCP can remove excess VMs to save costs.

Auto scaling in GCP helps ensure that your application remains available and responsive to users even during periods of high demand, while also allowing you to optimize costs by only paying for the resources you need.

Attach storage (& network storage) In Google Cloud Platform (GCP), you can attach different types of storage to virtual machines (VMs) depending on your needs. Here are two types of storage that you can attach to VMs in GCP:

1. Persistent Disk: Persistent Disk is a durable, high-performance block storage service for GCP. It provides SSD and HDD storage options, and you can choose the storage size and performance level based on your requirements. You can attach Persistent Disks to VMs as additional disks, and they will appear as additional volumes inside the VM. To attach a Persistent Disk to a VM, you need to create the disk first, then attach it to the VM instance.

2. Network File System (NFS): NFS is a distributed file system that allows you to share files and directories among multiple VMs. With NFS, you can create a shared storage solution that can be accessed by multiple VMs at the same time. You can create an NFS server on one VM and mount the NFS share on other VMs. To use NFS in GCP, you can create a Compute Engine instance that serves as an NFS server, configure it to export the shared directory, then mount it on the VMs that need to access the shared data.

Attaching storage in GCP allows you to add additional storage capacity to your VMs, which can help increase the amount of data you can store and process. By using the right type of storage for your needs, you can optimize your application's performance and cost efficiency.

                                                           Compute Engine Hands-on Activity

• Let's create a few VM instances and play with them.

Here are the steps to create VM instances in GCP:

1. Go to the GCP Console and select the project in which you want to create the VM instances.
2. In the left-hand menu, click on "Compute Engine" under the "Compute" section.
3. Click on "Create Instance" to create a new VM instance.
4. Enter a name for your instance, and choose a region and zone in which to create the instance.
5. Select a machine type for your instance, such as "n1-standard-1" or "f1-micro".
6. Choose a boot disk for your instance. You can choose from various operating system images or create your own custom image.
7. Optionally, you can add additional disks to your instance for data storage.
8. Under "Networking," you can choose to assign an internal and/or external IP address to your instance.
9. Configure any additional options for your instance, such as tags or metadata.
10. Click on "Create" to create your instance.

You can repeat the above steps to create multiple VM instances. Once your instances are created, you can connect to them via SSH or RDP, and install software and configure settings as needed.

To play with your instances, you can try different tasks such as:

1. Creating a load balancer and distributing traffic among your instances.
2. Creating a managed instance group to automatically create and manage a group of instances.
3. Creating a firewall rule to control network traffic to and from your instances.
4. Creating snapshots of your instances to back up your data.
5. Creating custom images of your instances to use as a template for creating new instances.

These are just a few examples of the many things you can do with your VM instances in GCP. Have fun experimenting and exploring!

• Let's check out the lifecycle of VM instances.

Here are steps to check out the lifecycle of VM instances can vary depending on the cloud provider and the management tool being used. However, here are some general steps that can be followed:

1. Log in to the cloud provider's management console.
2. Navigate to the section for managing virtual machines or instances.
3. Locate the instance whose lifecycle you want to check.
4. Check the current status of the instance. The status may be "running", "stopped", "suspended", or some other state depending on the provider.
5. Check the creation date of the instance to determine how long it has been running.
6. Look for any recent changes to the instance, such as modifications to its configuration or changes in the network settings.
7. Check for any events or alerts related to the instance that may indicate any issues or maintenance activities that have been performed.
8. Review the billing information for the instance to see how much it has cost during its lifecycle.
9. If the instance is no longer needed, consider terminating it to avoid unnecessary charges.

It is important to regularly monitor the lifecycle of VM instances to ensure they are running efficiently and not incurring unnecessary costs.

• Let's use SSH to connect to VM instances.
  

To use SSH to connect to VM instances in Google Cloud Platform (GCP), you can follow these steps:

1. Open the GCP Console at https://console.cloud.google.com/.
2. In the navigation pane on the left, click on "Compute Engine" to open the Compute Engine dashboard.
3. Find the VM instance that you want to connect to and click on the "SSH" button in the "Connect" column. This will open an SSH session in your browser window.
4. If you prefer to use an SSH client on your local machine, click on the drop-down arrow next to the "SSH" button and select "View gcloud command". This will display the command that you can use to connect to the VM instance using an SSH client.
5. Copy the command and paste it into your terminal or command prompt on your local machine.
6. If prompted, enter your GCP project ID, if it is not already filled in, and then press Enter.
7. If prompted, enter your GCP username, and then press Enter.
8. If prompted, enter your GCP password, and then press Enter.
9. If the credentials are correct, you will be logged in to the VM instance via SSH.

Note: If you are connecting to a Linux-based VM instance, you can also use the standard SSH command from your local machine's terminal or command prompt. To do so, replace username with your GCP username and instance-ip with the external IP address of the VM instance:

java
ssh username@instance-ip

                                                           Compute Engine Machine Family

• What type of hardware do you want to run your workloads on?
  
  
• Different Machine Families for Different Workloads:
• General Purpose (E2, N2, N2D, N1) : Best price-performance ratio
• Web and application servers, Small-medium databases, Dev environments.

• Memory Optimized (M2, M1): Ultra high memory workloads
• Large in-memory databases and In-memory analytics

• Compute Optimized (C2): Compute intensive workloads
• Gaming applications

Compute Engine Machine Family in Google Cloud Platform (GCP) refers to a group of virtual machine (VM) instances that have similar hardware configurations and are optimized for specific workloads. GCP provides several machine families that offer different types of CPUs, memory, and disk options, each designed for different use cases and performance needs. Here are some examples of Compute Engine Machine Families in GCP:

1. General-purpose: This family includes virtual machines that are optimized for a broad range of workloads and provide a balance of CPU, memory, and disk resources.

2. Compute-optimized: This family includes virtual machines that are optimized for CPU-intensive workloads and provide high-performance processors.

3. Memory-optimized: This family includes virtual machines that are optimized for memory-intensive workloads and provide a high ratio of memory to CPU.

4. Accelerator-optimized: This family includes virtual machines that are optimized for machine learning and other high-performance computing workloads that require GPUs.

Each Compute Engine Machine Family has different instance types with varying configurations of CPU, memory, and disk. When you create a new VM instance in GCP, you can choose the machine family and instance type that best meets your needs based on the workload you plan to run.

By offering different Compute Engine Machine Families, GCP allows users to select the appropriate machine type that can deliver the necessary performance and resources for their specific use case.

                                                           

Compute Engine Machine Types

• How much CPU, memory or disk do you want?
• Variety of machine types are available for each machine family.
• Let's take an example : e2-standard-2:
• e2 - Machine Type Family
• standard - Type of workload
• 2 - Number of CPUs
• Memory, disk and networking capabilities increase along with vCPUs.

Compute Engine Machine Types in Google Cloud Platform (GCP) refer to predefined virtual machine (VM) configurations that specify the amount of virtual CPUs (vCPUs), memory, and other resources that a VM instance will have. GCP provides a range of machine types to meet different workload needs, from small, low-cost instances to high-performance instances with multiple vCPUs and large amounts of memory.

Here are some examples of Compute Engine Machine Types in GCP:

1. F1 Micro: This is the smallest and lowest-cost machine type in GCP, with 1 vCPU and 0.6 GB of memory.

2. N1 Standard: This is a general-purpose machine type that provides a balance of CPU and memory resources. It offers a range of sizes, from 1 vCPU and 3.75 GB of memory up to 96 vCPUs and 624 GB of memory.

3. N1 High-Memory: This is a machine type that is optimized for memory-intensive workloads. It offers a range of sizes, from 2 vCPUs and 13 GB of memory up to 96 vCPUs and 624 GB of memory.

4. N1 High-CPU: This is a machine type that is optimized for CPU-intensive workloads. It offers a range of sizes, from 2 vCPUs and 1.8 GB of memory up to 96 vCPUs and 624 GB of memory.

5. C2 High-CPU: This is a machine type that is optimized for high-performance computing and provides up to 60 vCPUs and 240 GB of memory.

Each Compute Engine Machine Type has a different set of resources and price, and users can choose the machine type that best fits their workload needs and budget.

By offering different Compute Engine Machine Types, GCP allows users to select the appropriate virtual machine configuration that can deliver the necessary performance and resources for their specific use case.

Image

  

• What operating system and what software do you want on the instance?
  
  
• Type of Images:
• Public Images: Provided & maintained by Google or Open source communities or third party vendors.
• Custom Images: Created by you for your projects.

In Google Cloud Platform (GCP), an image is a pre-configured virtual machine (VM) instance that can be used to create new VMs. An image contains the operating system, software, and configuration settings needed to run an application or service. Instead of manually installing and configuring software on each new VM, you can use an image to quickly create new VMs that are already set up with the necessary software and settings.

GCP provides several types of images that can be used to create new VMs, including:

1. Public images: These are images provided by GCP that are available to all users. They include images for various operating systems, such as Ubuntu, CentOS, and Windows, as well as images that contain popular software stacks, such as LAMP (Linux, Apache, MySQL, PHP).

2. Custom images: These are images that you create yourself by customizing an existing VM instance, such as by installing software or modifying configuration settings. Once you have created a custom image, you can use it to create new VMs that have the same software and settings as the original VM.

3. Marketplace images: These are images provided by third-party vendors that are available through the GCP Marketplace. They include a wide range of images for different software applications and services, such as databases, web servers, and content management systems.

By using images in GCP, you can save time and effort by quickly creating new VMs that are pre-configured with the necessary software and settings. You can also create and manage your own custom images to ensure consistency and standardization across your VM instances.

Compute Engine Hands-on :
Setting up a HTTP server

#! /bin/bash

sudo su

apt update

apt -y install apache2

sudo service apache2 start

sudo update-rc.d apache2 enable

echo "Hello World" > /var/www/html/index.html

echo "Hello world from $(hostname) $(hostname -I)" > /var/www/html/index.html

Commands:

• sudo su - execute commands as a root user
• apt update - Update package index - pull the latest changes from the APT repositories
• apt -y install apache2 - Install apache 2 web server
• sudo service apache2 start - Start apache 2 web server
• echo "Hello World" > /var/www/html/index.html - Write to index.html
• $(hostname) - Get host name
• $(hostname -I) - Get host internal IP address

                                                           Internal and External IP Addresses

• External (Public) IP addresses are Internet addressable.
• Internal (Private) IP addresses are internal to a corporate network
• You CANNOT have two resources with same public (External) IP address.
• HOWEVER, two different corporate networks CAN have resources with same Internal (private) IP address.
• All VM instances are assigned at least one Internal IP address.
• Creation of External IP addresses can be enabled for VM instances
  (Remember) When you stop an VM instance, External IP address is lost.
  

In Google Cloud Platform (GCP), instances can be assigned both internal and external IP addresses depending on their network configuration and access requirements.

An internal IP address is a private, non-routable IP address used within a virtual private cloud (VPC) network to identify a specific instance. Instances can communicate with each other using their internal IP addresses within the same VPC network, without being exposed to the public internet. Internal IP addresses are assigned automatically when you create a new instance in GCP.

An external IP address, on the other hand, is a public IP address used to communicate with the internet or other resources outside of the VPC network. External IP addresses can be assigned to instances in different ways:

1. Ephemeral external IP: When you create an instance in GCP, it is automatically assigned an ephemeral external IP address. Ephemeral external IP addresses are temporary and are released when the instance is stopped or deleted.

2. Static external IP: You can also assign a static external IP address to an instance, which is a permanent IP address that can be used for a long period of time. Static external IP addresses can be used for instances that require a fixed IP address, such as a web server or a database server.

In addition to internal and external IP addresses, GCP also provides other networking features such as firewall rules, load balancers, and virtual private networks (VPNs) to help you manage and secure your network traffic.

DEMO: VM instances - Internal and External IPs

here are the steps to assign internal and external IPs to a VM instance in GCP:

1. Go to the VM instances page in the GCP Console.
2. Click on the name of the instance to which you want to assign IPs.
3. Click on the "Edit" button at the top of the page.
4. Under the "Network interfaces" section, click on the "Edit" button next to the network interface that you want to configure.
5. In the "Internal IP" section, you can choose to use an ephemeral internal IP address, or specify a custom internal IP address. If you choose to specify a custom internal IP address, enter the IP address that you want to assign to the instance.
6. In the "External IP" section, you can choose to use an ephemeral external IP address, or assign a static external IP address. If you choose to assign a static external IP address, select "Create IP address" and follow the prompts to create a new static external IP address or select an existing one.
7. Click on the "Save" button to save your changes.

Once you have assigned internal and external IPs to your VM instance, you can use them to communicate with other instances within the same VPC network and with resources outside of the VPC network, respectively. You can also configure firewall rules to allow traffic to and from the internal and external IPs as needed.

Static IP Addresses:-

• Scenario : How do you get a constant External IP address for a VM instance?
• Quick and dirty way is to assign an Static IP Address to the VM!
  
  
• Remember:-
• Static IP can be switched to another VM instance in same project.
• Static IP remains attached even if you stop the instance. You have to manually detach it.
  
  
• Remember:- You are billed for an Static IP even when you are NOT using it!
• Make sure that you explicitly release an Static IP when you are not using it

Static external IP: You can also assign a static external IP address to an instance, which is a permanent IP address that can be used for a long period of time. Static external IP addresses can be used for instances that require a fixed IP address, such as a web server or a database server.

DEMO: Using Static IP Address with an VM instance.

here are the steps to assign a static internal IP address to a VM instance in GCP:

1. Go to the VM instances page in the GCP Console.
2. Click on the name of the instance to which you want to assign a static internal IP address.
3. Click on the "Edit" button at the top of the page.
4. Under the "Network interfaces" section, click on the "Edit" button next to the network interface that you want to configure.
5. In the "Internal IP" section, select "Static" and enter the IP address that you want to assign to the instance. Note that the IP address must be within the range of the subnet that the instance is connected to.
6. Click on the "Save" button to save your changes.

Once you have assigned a static internal IP address to your VM instance, you can use it to communicate with other instances in the same VPC network using the internal IP address. You can also configure firewall rules to allow traffic to and from the static internal IP address.

                                                           Simplify VM HTTP server setup:-

• How do we reduce the number of steps in creating an VM instance and setting up a HTTP Server?
  
  
• Let's explore a few options:
• Startup script
• Instance Template
• Custom Image

Bootstrapping with Startup script:-

#!/bin/bash
apt update
apt -y install apache2
echo "Hello world from $(hostname) $(hostname -I)" > /var/www/html/index.html

Bootstrapping: Install OS patches or software when an VM instance is
launched.
In VM, you can configure Startup script to bootstrap.

                          DEMO - Using Startup script.

You can create an instance using a startup script in Google Cloud Platform (GCP) by following these steps:

1. Open the GCP Console at https://console.cloud.google.com/.

2. In the navigation pane on the left, click on "Compute Engine" to open the Compute Engine dashboard.

3. Click on the "Create Instance" button to create a new instance.

4. In the "Create a new instance" page, fill in the necessary details for your instance, such as the instance name, machine type, and boot disk.

5. Under the "Cloud Init" section, click on the "Enable" checkbox to enable the use of startup scripts.

6. In the "Startup script" text box, enter the commands that you want to run when the instance starts up. For example:

   bash
   #!/bin/bash sudo apt-get update sudo apt-get install -y apache2

   This script will update the package index on the instance and install the Apache web server.

7. Click on the "Create" button to create the instance. The instance will be created with the specified startup script, which will be run when the instance starts up.

Note: You can also specify the startup script using the gcloud command-line tool. For example, to create an instance with the above startup script, you can run the following command:

sql
gcloud compute instances create instance-name --image-project=ubuntu-os-cloud --image-family=ubuntu-1804-lts --metadata startup-script='#!/bin/bash sudo apt-get update sudo apt-get install -y apache2'

Replace instance-name with the name of the instance that you want to create. The --image-project and --image-family flags specify the base image for the instance, which in this case is the Ubuntu 18.04 LTS image. The --metadata flag specifies the startup script that should be used for the instance.

Instance templates:-

• Why do you need to specify all the VM instance details (Image, instance type etc) every time you launch an instance?
• How about creating a Instance template?
• Define machine type, image, labels, startup script and other properties.
• Used to create VM instances and managed instance groups
• Provides a convenient way to create similar instances
• CANNOT be updated
• To make a change, copy an existing template and modify it
• (Optional) Image family can be specified (example - debian-9):
• Latest non-deprecated version of the family is used.

DEMO - Launch VM instances using Instance templates.

To create an instance using instance templates in Google Cloud Platform (GCP), follow these steps:

1. Open the GCP Console at https://console.cloud.google.com/.
2. In the navigation pane on the left, click on "Compute Engine" to open the Compute Engine dashboard.
3. Click on the "Create Instance Template" button to create a new instance template.
4. In the "Create a new instance template" page, fill in the necessary details for your instance template, such as the template name, machine type, and boot disk.
5. Under the "Management" section, you can configure optional settings such as auto-delete and automatic restart for your instance.
6. Under the "Networking" section, you can configure networking settings such as the network interface and firewall rules for your instance.
7. Click on the "Create" button to create the instance template.

Once you have created an instance template, you can use it to create instances by following these steps:

1. In the Compute Engine dashboard, click on the "Create Instance" button.
2. In the "Create a new instance" page, click on the "Create from instance template" tab.
3. Select the instance template that you want to use from the drop-down menu.
4. Fill in any additional details for your instance, such as the instance name and zone.
5. Click on the "Create" button to create the instance.

Note: You can also create instances from instance templates using the gcloud command-line tool. For example, to create an instance using an instance template named my-instance-template, you can run the following command:

gcloud compute instances create my-instance --source-instance-template my-instance-template --zone us-central1-a

Replace my-instance with the name that you want to give your instance, and my-instance-template with the name of your instance template. The --zone flag specifies the zone in which to create the instance.

Reducing Launch Time with Custom Image:-

• Installing OS patches and software at launch of VM instances increases boot up time.
• How about creating a custom image with OS patches and software pre-installed?
• Can be created from an instance, a persistent disk, a snapshot, another image, or a file in Cloud Storage.
• Can be shared across projects
• (Recommendation) Deprecate old images (& specify replacement image)
• (Recommendation) Hardening an Image - Customize images to your corporate security standards.
• Prefer using Custom Image to Startup script.

DEMO : Create a Custom Image and using it in an Instance Template.

Steps to create a custom image and use it in an instance template in Google Cloud Platform (GCP), follow these steps:

1. Create a VM instance using an existing image that you want to customize.
2. Customize the VM instance by installing any necessary software or making any other desired changes.
3. Stop the VM instance.
4. In the Compute Engine dashboard, click on the "Disks" link in the left navigation pane.
5. Locate the boot disk for the VM instance that you just customized and click on its name to open the disk details page.
6. Click on the "Create Image" button to create a new custom image.
7. In the "Create a custom image" page, enter a name and description for your image.
8. Click on the "Create" button to create the custom image.

Once you have created a custom image, you can use it in an instance template by following these steps:

1. In the Compute Engine dashboard, click on the "Instance templates" link in the left navigation pane.
2. Click on the "Create instance template" button to create a new instance template.
3. In the "Create a new instance template" page, fill in the necessary details for your instance template, such as the template name, machine type, and boot disk.
4. Under the "Boot disk" section, select the "Custom images" option and select the custom image that you created earlier.
5. Click on the "Create" button to create the instance template.

Once you have created an instance template using your custom image, you can create instances from it by following the steps outlined in the previous answer.

Note: You can also create custom images using the gcloud command-line tool. For example, to create a custom image named my-custom-image from the disk of a VM instance named my-vm, you can run the following command:

css
gcloud compute images create my-custom-image --source-disk my-vm --source-disk-zone us-central1-a

Replace my-custom-image with the name that you want to give your custom image, my-vm with the name of your VM instance, and us-central1-a with the zone in which your VM instance is located.

• Automatic discounts for running VM instances for significant portion of the billing month 
• Example: If you use N1, N2 machine types for more than 25% of a month, you get a 20% to 50% discount on every incremental minute.
• Discount increases with usage (graph)
• No action required on your part!

• Applicable for instances created by Google Kubernetes Engine & Compute Engine.
  
  
• RESTRICTION: 
• Does NOT apply on certain machine types (example: E2 and A2)
•  Does NOT apply to VMs created by App Engine flexible and Dataflow.

                        Source: https://cloud.google.com

                                                           

Committed use discounts:-

• For workloads with predictable resource needs
• Commit for 1 year or 3 years
• Up to 70% discount based on machine type and GPUs
• Applicable for instances created by Google Kubernetes Engine and Compute Engine
• RESTRICTION: Does NOT apply to VMs created by App Engine flexible and Dataflow

                                                           Preemptible VM:-

• Short-lived cheaper (upto 80%) compute instances
• Can be stopped by GCP any time (preempted) within 24 hours
• Instances get 30 second warning (to save anything they want to save)
  
  
• Use Preempt VM's if:
• Your applications are fault tolerant
• You are very cost sensitive
• Your workload is NOT immediate
• Example: Non immediate batch processing jobs
  
  
• RESTRICTIONS:
• NOT always available
• NO SLA and CANNOT be migrated to regular VMs
• NO Automatic Restarts
• Free Tier credits not applicable

                                                           Google Compute Engine - Billing:-

• You are billed by the second (after a minimum of 1 minute)
• You are NOT billed for compute when a compute instance is stopped.
• However, you will be billed for any storage attached with it!
• (RECOMMENDATION) Always create Budget alerts and make use of Budget exports to stay on top of billing!
  
  
• What are the ways you can save money?
• Choose the right machine type and image for your workload
• Be aware of the discounts available:
• Sustained use discounts
• Committed use discounts
• Discounts for preemptible VM instances
  
  

Compute Engine :- 
Live Migration & Availability Policy

• How do you keep your VM instances running when a host system needs to be updated (a software or a hardware update needs to be performed)?

• Live Migration
• Your running instance is migrated to another host in the same zone
• Does NOT change any attributes or properties of the VM
• SUPPORTED for instances with local SSDs
• NOT SUPPORTED for GPUs and preemptible instances
  
  
• Important Configuration - Availability Policy:
• On host maintenance: What should happen during periodic infrastructure maintenance?
• Migrate (default): Migrate VM instance to other hardware
• Terminate: Stop the VM instance
• Automatic restart - Restart VM instances if they are terminated due to non-user- initiated reasons (maintenance event, hardware failure etc.)

                                                           

Compute Engine Features:- Custom Machine Types

What do you do when predefined VM options are NOT appropriate

                for your workload?

• Create a machine type customized to your needs (a Custom Machine Type)

Custom Machine Type: Adjust vCPUs, memory and GPUs

• Choose between E2, N2, or N1 machine types
• Supports a wide variety of Operating Systems: CentOS, CoreOS, Debian, Red Hat,
• Ubuntu, Windows etc
• Billed per vCPUs, memory provisioned to each instance
• Example Hourly Price: $0.033174 / vCPU + $0.004446 / GB
  
  
  

                                                  

Compute Engine Features: GPUs

• How do you accelerate math intensive and graphics-intensive
  workloads for AI/ML etc?

• Add a GPU to your virtual machine:
• High performance for math intensive and graphics-intensive workloads
• Higher Cost
• (REMEMBER) Use images with GPU libraries (Deep Learning) installed
• OTHERWISE, GPU will not be used
• GPU restrictions:
• NOT supported on all machine types (For example, not supported on shared-core or memory-optimized machine types)
• On host maintenance can only have the value "Terminate VM instance"
• Recommended Availability policy for GPUs
• Automatic restart - on

                                                           
Virtual Machine - Remember 

• Associated with a project
• Machine type availability can vary from region to regions
• You can only change the machine type (adjust the number of vCPUs and memory) of a stopped instance
• You CANNOT change the machine type of a running instance
• VM's can be filtered by various properties
• Name, Zone, Machine Type, Internal/External IP, Network, Labels etc
• Instances are Zonal (Run in a specific zone (in a specific region))
• Images are global (You can provide access to other projects - if needed)
• Instance templates are global (Unless you use zonal resources in your templates)
• Automatic Basic Monitoring is enabled
• Default Metrics: CPU utilization, Network Bytes (in/out), Disk Throughput/IOPS
• For Memory Utilization & Disk Space Utilization - Cloud Monitoring agent is needed

                                                           

Virtual Machine - Best Practices

• Choose Zone and Region based on:
• Cost, Regulations, Availability Needs, Latency and Specific Hardware needs
• Distribute instances in multiple zones and regions for high availability
• Choose right machine type for your needs:
• Play with them to find out the right machine type
• Use GPUs for Math and Graphic intensive applications
• Reserve for "committed use discounts" for constant workloads
• Use preemptible instances for fault-tolerant, NON time critical workloads
• Use labels to indicate environment, team, business unit etc

Compute Engine Scenarios

What are the pre-requisites to be able to create a VM instance?

1. Project
2. Billing Account
3. Compute Engines APIs should be enabled

You want dedicated hardware for your compliance, licensing, and management needs
Sole-tenant nodes

I have 1000s of VM and I want to automate OS patch management, OS inventory management and OS configuration management (manag software installed)
Use "VM Manager"

You want to login to your VM instance to install software
You can SSH into it

You do not want to expose a VM to internet
Do NOT assign an external IP Address

You want to allow HTTP traffic to your VM
Configure Firewall Rules

QUICK REVIEW

Image 

• What operating system and what software do you want on the VM instance?
• Reduce boot time and improve security by creating custom hardened Images.
• You can share an Image with other projects

Machine Types

• Optimized combination of compute(CPU, GPU), memory, disk (storage) and networking for specific workloads.
  
  
• You can create your own Custom Machine Types when existing ones don't fit your needs.
  
  
  Remember:-
  
  
• Static IP Addresses: Get a constant IP addresses for VM instances

• Instance Templates: Pre-configured templates simplifying the creation of VM instances
  
  
• Sustained use discounts: Automatic discounts for running VM instances for significant portion of the billing month
  
  
• Committed use discounts: 1 year or 3 year reservations for workloads with predictable resource needs
  
  
• Preemptible VM: Short-lived cheaper (upto 80%) compute instances for non-time-critical fault-tolerant workloads