How to build a custom image for Compute Engine

I’m Kathrin, and I am a software
engineer at Google. In this video,
we’ll show you how to create your own
images that you can then use to launch Google Compute
Engine Virtual Machines. Before we get
started, let’s make sure you have a few things
set up on your machine. We’re going to be
using a laptop here. And you’ll want to
make sure that you will have VirtualBox
installed on it. So I already have that
installed here, as you can see. You will also want to make sure
that you have an ISO downloaded already that you can use
to set up your image. In this case, I used a
Debian testing AMD64, as you can see up here. The last thing you’ll want to
make sure you have installed is a version of Tar that
supports the old style GNU format. So let’s get started. The first thing
we’re going to do is create a new VirtualBox VM. We’re going to name it GCE demo. And what we’ll do is
we’ll select Linux. What we will show
you, although we use Debian testing in
this video, all of this should work roughly the same
for all Linux distributions. We’re going to pick Debian
64 bit, and click Continue. We want to give this
a little more memory. This is optional,
but you can do it if you want to make things
go a little bit faster. After I’ve done
that, we also need to create a virtual hard drive. And we’re going to create this. And what would be good is if
you could choose Copy on Write here. Another thing you’ll
want to make sure is that the disk that you create
is at least 10 gigabytes big. OK, so now we actually have
our first VirtualBox virtual machine. We’ll have to change a
few settings, though. So go ahead and go
to click on Settings. And this will bring
up this dialog here. And the first thing
you might want to do is disable Boot from Floppy. This just makes the setup
process a little bit smoother. And it’s also not
something that we support. So you don’t really need it. Under Storage, when
you click on storage, make sure you click on the
empty under the controller IDE, and then on this little
CD icon that you see next to CD/DVD drive. When you click on that icon,
it will give you the option to choose a virtual
CD or DVD disc file. It will open a dialog,
and just navigate to wherever you have downloaded
that ISO that I talked about at the very beginning. And you select it
and click Open. In the Audio tab, make
sure to disable audio. Again, this is not something
that we need for our purposes here. And finally, in the Network
tab, click on the Advanced menu here. And we’ll want to make sure that
we select the Paravirtualized network– the virtio-net. And this will make sure that the
correct drivers are installed so that everything
will work smoothly with the virtual machine
that you will eventually launch in Compute Engine. OK, ready. Let’s click OK. So this virtual machine
is now ready to go. So the next thing
we’re going to do is we’re going to click Start. And this will actually take us
through the Debian installer now. So you can use your
up and down arrow keys to select Graphical Install. Click Enter, and this will
now start the installer. For this video,
I’m going to just be choosing English and US. So choose these as you
see fit, of course. So I’m going to choose English,
United States, American English. Obviously, these
settings should be adjusted to whatever
makes sense for you. So this takes just a
minute to complete. And it’s now
configuring the network. Excellent. So this has succeeded. The host name we’re
going to leave as is. This is a good default here. And for the domain name,
this is not something that we’re going to be able
to use when this is actually launched as a virtual
machine in Compute Engine. So you can leave that blank. This is just fine. Make sure you do not
set a root password. This will actually
increase the security of your virtual machine. The user account
that we’re going to pick, pick whatever user
account makes sense for you. In my case, I’m just simply
going to pick config. And then you will
want to make sure that you use a very
strong password. This is the password
that you’re going to use to log in to
the virtual machine that you are going to create
later on Google Compute Engine. So I’m going to choose a
password that, hopefully, I will remember later. OK, because I live
in Seattle, I’m going to choose Pacific
Time, although, of course you can change that setting. OK, the next thing we’re
going to need to do is to partition the disks. Really, what we want is
we want one large disk. We don’t really
want any partitions. If you go through
the standard flow– this is not what you
want to do right now, because it will, by default,
install a swap partition. And that’s not really
something you need, and it takes up
space, and you’ll get better performance
if you don’t have it. So let’s select Manual. So we’re going to
click on our hard disk that we have already created. And so it says you have selected
an entire device to partition. Yeah, so this is
exactly what we want. So you click Yes, and
you click Continue. So the next thing you
will do is you will Double click on the free space
and create a new partition. The default setting
is fine here. So we can leave that as is,
and just click Continue. Similarly, primary is fine. So we will click Continue. At this point, we’re
actually already done setting up the disk. So we can Double
click down here, done setting up the partition. And finally, what we will need
to do is, all the settings that we have just
created and selected, we will want to write
those changes to disk. OK. Oh. So now it is complaining
that I did not actually create a swap space. And it allows me to go
back and change my mind. I don’t actually want
to change my mind. So I’m going to say, no,
I don’t want to do that. OK. And finally, yes, this is
simply confirming the choices that we have made. OK, so at this point, the base
system is being installed. And this will take a couple
of minutes, some time. So we have a little bit of time. Excellent, so now
the base system is installed based on
what we have given it. It now gives us the option
to scan another CD or DVD, but we don’t really want
to do that at this point. So we just click Continue. We do want to set
up a network mirror, because the network mirror will
allow us to pick and choose what will get installed
on this virtual machine. So let’s say Yes, and click
Continue, and United States. This network mirror
is a good choice. So we will say Yes. We don’t really need
any proxy information. This may be different
for your network. So you might want to
make sure that you don’t need proxy
information or have the correct information
available here. OK, so the Package Manager
is being configured. And then it will allow us to
pick and choose what we want. In our case, we’ll
want to start light. This will, of course,
depend on your situation, what you want to have installed. In our case, we’ll just
start with an SSH server and some standard
system utilities. At this point, we’re
given the option to actually report
back information. You can pick Yes or No. We’re just going to say No here. Great. We don’t really need the
Debian Desktop Environment. And we don’t need the
print server either. But we do want the
SSH server installed. Great. So now that we’ve installed
everything that we’ve selected, we just need to make
sure that we also have the GRUB bootloader
available to us, so we can actually boot the VM. We’ll click dev SDA here. And we should be good to go. After it finishes
the installation, we can actually click Continue. And the system will
go down and reboot. So now I should be able to log
in with the login and password that I created. So let’s hope I
remember my password. Great. So now I’m in this
virtual machine that we’ve just created,
and configured, and booted. So that’s wonderful, and we
now have this virtual machine. But we haven’t tied this at all
to Google Compute Engine yet. And this will be the next part. So what we’re going
to do now is we’re going to shut down this virtual
machine that we have here. OK, so the system
is now powered off, as you can see here in
the VirtualBox Manager. As we get ready to launch a
Compute Engine virtual machine, we want to make sure that
we exit out of VirtualBox– we quit, so that the
disk is not in use. And then, to get
our disk ready to be used by Compute Engine,
what we’ll want to do is we will want to create a
copy in a different format, in the RAW format. And for this, you can use
the command that I show here. So to create the
disk from RAW format, you tell it to take
the format RAW. VirtualBox, VMs,
and then– so this will take a little bit of time
to actually create that disk, to create the copy of
the disk in RAW format. OK, now we’ve created
a copy of our disk. And so the next thing
we’ll need to do is we will actually need to put
this in a specific format that is required by Google. So we’re going to
create a archive. And the way we do that is,
actually, just– I already have that here in history. So we will create an
archive of the disk that we’ve just created. And we’re going to
call it GCEdemotar.gz. Great. Now we have the archive. The next thing
we’re going to do is we are just simply going to
upload this archive to Google Cloud Storage. And the simplest way to
do this is to use gsutil. Gsutil is our
command line utility for interacting
with cloud storage. So we will simply copy the file. So you can see here
in this command, we use copy, gsutil copy,
the name of the file, and then you give it
the name of your bucket that you want, in
cloud storage, that you want this archive to end up in. In my case, I called this
bucket VirtualBox GCE demo. Make sure this bucket
already exists, otherwise gsutil will complain
and make you create it. So this will be uploading
now for some time. OK, so our archive
has successfully been uploaded to
Google Cloud Storage. What you can see here
is that gsutil actually broke the archive up
into smaller chunks, and uploaded them
separately, and then stitched them back together. So that’s where the separate
uploading lines come from. Now that we actually have
our archive in Google Cloud Storage, we can create a Google
Compute Engine image from it. The way to do that
is gcloud compute, which is our brand
new command line utility for interacting
with Google Compute Engine. And the way you
will do this is you will say gcloud
compute images, create, and then you give it the name
that you want to give it, and the source URI, which is
the bucket and the name of where you had just put it
in cloud storage. So let’s go ahead and do that. And this will also
take a bit of time. Great, so gcloud compute images
came back, and it said yes. Great, I’ve created the image,
and its status is ready. We’re almost finished. The last thing
that we need to do is we actually need to create a
virtual machine from the image that we have just created. And the way to do that, again,
you would use gcloud compute, instance is create, and,
again, you give it a name. You give it the
image that we’ve just created, and then
a machine type, and the zone where you
want this virtual machine to be created in. The first time you create a new
virtual machine from an image, it will take a
little bit longer. The next subsequent instance
creations, or virtual machine creations, will take less time. OK, our instance
has been created. And the status is running,
as you can see here. Now it’s there. It’s running. And what we want to
do is to make sure that we can actually use
it as we can SSH into it. Because we didn’t take
any special steps to make this image GCE
ready, you will have to log in with the
username and password that we set up earlier
in the process. So in my case, the
username was [email protected] And then you give it
the external IP address, which you can see
here as the NAT IP. So I’m going to select this. I’m just going to type it. And then I should be able
to log into this machine. You say Yes, I want to continue. And then I type in my
crazy password again. And here we are. That wraps up our demonstration. I hope you learned from it
how to set up your own images and bring them to
our platform, and use it to set up virtual
machine instances. [MUSIC PLAYING]

19 thoughts on “How to build a custom image for Compute Engine

  1. New DevByte: How to build a custom image for Compute Engine
    #devbytes   #cloudplatform   #cloud   #gce  

    In this video, @Katharina Probst walks you through a process for creating a custom Compute Engine image based on the Debian Testing release.  If you've ever wanted to try your favorite distribution, or run your own highly customized image on Compute Engine, this will be a great tutorial to show you how easy it can be with VirtualBox and a few simple commands.

  2. When you use virtual box and u dedicate it memory when u close virtual boxs is that memory going to be able to be used normaly

  3. Given that VirtualBox is open source project, you could add a button there "Send This to Google Compute Engine" instead of all this. How about that?

  4. I converted my VM to RAW format and use Ubuntu tar to create tarball. 
    After I uploaded with 'gsutil cp' the image was marked as FAILED
    when view with 'gcloud compute images describe'
       sourceType: RAW
       status: FAILED

    I think Linux tar is causing me problems.  How can I get gtar under Ubuntu?

  5. Great video tutorial here.
    Do you have a video explaining the process to create "Windows 7 or 8.1" and "Windows Server" compute Engine Images ?


  6. It should be noted that when you clone the Virtualbox .qcow file into de the .raw file, the name of the cloned file MUST ALWAYS be "disk.raw" otherwise the command "gcloud compute images create <name-of-the-image> –source-uri <bucket/filename>" will fail!

  7. When I add my custom image as a custom "template image" for my instances they seem to crash at boot and therefore not run any startup-script!
    Any ideas?

  8. would it not work if my initial disk format prior to .RAW was .VMDK?
    After I imported my image and created an instance of it I cannot SSH – I get a timeout.

  9. If you already know how to build virtual machines and export a raw disk image the most informative part of the video starts at: 13:30

  10. Cool video. Instead of using virtual box and uploading your vm, can you not create your VM in the Google portal and use that as the source to create your image?

Leave a Reply

Your email address will not be published. Required fields are marked *