Thursday, August 4, 2011

LDOM

                                              LDOM 1.0
It is s/w on Sun Tech. that leverages the hypervisor on sun4v platform to subdivided system and resource.

Virtualization available with the Sun Sparc T1 and T 2 systems (Sun4v). If you want to try it on x86 boxes, please use XEN.

Logical Domains is the server virtualization and partitioning technology from Sun Microsystems released in April 2007. It has been re-branded as Oracle VM Server for SPARC.

Each domain is a full virtual machine with a reconfigurable subset of hardware resources. Operating systems running inside Logical Domains can be started, stopped, and rebooted independently.

Supported hardware:-
The SPARC Hypervisor runs in the Hyper-Privileged execution mode, which was introduced in the sun4v architecture. The sun4v processors released as of September 2010 are the UltraSPARC T1, the UltraSPARC T2, the UltraSPARC T2 Plus and the UltraSPARC T3 [2]. Only systems based on those processors support Logical Domains. These include the UltraSPARC T1-based:
  • Sun / Fujitsu SPARC Enterprise T1000 and T2000 servers
  • Sun Fire T1000 and T2000 servers
  • Netra T2000 Server
  • Netra CP3060 Blade
  • Sun Blade T6300 Server Module
UltraSPARC T2-based:
  • Sun / Fujitsu SPARC Enterprise T5120 and T5220 servers
  • Sun Blade T6320 Server Module
  • Netra CP3260 Blade
  • Netra T5220 Rackmount Server
UltraSPARC T2 Plus systems:
  • Sun / Fujitsu SPARC Enterprise T5140 and T5240 servers (2 sockets)
  • Sun Blade T6340 Server Module (2 sockets)
  • Sun / Fujitsu SPARC Enterprise T5440 (4 sockets)
And SPARC T3 systems:
  • Sun / Fujitsu SPARC T3-1 servers (1 socket)
  • Sun SPARC T3-1B Server Module (1 socket)
  • Sun / Fujitsu SPARC T3-2 servers (2 sockets)
  • Sun / Fujitsu SPARC T3-4 servers (4 sockets)
Key Logical Domain roles and components:-

• The Hypervisor
Software piece works on the top of the hardware. Responsible for maintaining the isolation of domains. Interacts between the hardware and the OS.

• The Control Domain
Creates and manages the other logical domains and service and services
• Service, I/O domains
Provides the virtual network switch or virtual disk

• Multiple Guest Domains
Uses the services from the services and I/O domains and is managed by the control domain. In broad term, can be considered as the additional OS which will get installed on the machine

• Virtualized devices
All the guest domains will be accessing the virtual devices. Like CPU, memory MAU, NIC, Disks, Consoles.

Ldom manager
Controls hypervisor and all domains. It can run in any domain, but only 1 domain at a time. It’s a CLI utility exposes to the administrator. Maps logical domains to physical resources.
Now it’s ready for action: ---
Get a T1000 or T2000 box
Install Solaris 10 U3 or higher
Connect to Service processor or SC to check the firmware version
SC> show host
Sun-Fire-T2000 System Firmware 6.4.2 2007/04/02 18:07
Host flash versions:
Hypervisor 1.4.1 2007/04/02 16:37
OBP 4.26.1 2007/04/02 16:26
POST 4.26.0 2007/03/26 16:45
Upgrade your system firmware if it’s below 6.4.2. If it’s below 6.4.2, please follow the instruction mention at the bottom of this page or refer the firmware upgrade docs.
Check the OS version and patch level
# uname -a
SunOS 5.10 Generic_118833-36 sun4v sparc SUNW,Sun-Fire-T200
# pkginfo -x | grep jass
upgrade the OS level or add the patches/packages
-bash-3.00# patchadd 124921-xx
-bash-3.00# patchadd 125043-xx
-bash-3.00# pkgadd -d SUNWldm
-bash-3.00# pkgadd -d SUNWjass
Ensure that the install packages and services are running
-bas>h-3.00# svcadm enable ldmd
-bash-3.00# svcadm enable vntsd
-bash-3.00# /opt/SUNWldm/bin/ldm list
Name State Flags Cons VCPU Memory Util Uptime
primary active -n-cv SP 32 32G 0.5% 6d 3h 29m
-bash-3.00# psrinfo -vp
The physical processor has 32 virtual processors (0-31)
UltraSPARC-T1 (cpuid 0 clock 1200 MHz)
-bash-3.00# ldm add-vdiskserver primary-vds0 primary
-bash-3.00# ldm add-vconscon port-range=5000-5100 \
primary-vcc0 primary
-bash-3.00# ldm add-vswitch net-dev=e1000g0 primary-vsw0 primary
-bash-3.00# ldm set-vcpu 4 primary
-bash-3.00# ldm set-memory 1g primary
-bash-3.00# ldm add-config initial
-bash-3.00# ldm list-config
factory-default [current]
initial [next]
-bash-3.00# shutdown -y -g0 -i6
-bash-3.00# ldm add-domain myldom1
-bash-3.00# ldm set-vcpu 4 myldom1
-bash-3.00# ldm set-mau 1 myldom1
-bash-3.00# ldm set-memory 1g myldom1
-bash-3.00# ldm add-vnet vnet1 primary-vsw0 myldom1
-bash-3.00# ldm add-vdiskserverdevice /images/ldom1.disk vol1@primary-vds0
-bash-3.00# ldm add-vdisk vdisk1 vol1@primary-vds0 myldom1
-bash-3.00# ldm set-variable autoboot\?=false myldom1
-bash-3.00# ldm set-variable boot-device=/virtual-devices@100/channel-devices@200/disk@0 myldom1
-bash-3.00# ldm bind-domain myldom1
-bash-3.00# ldm start-domain myldom1
Now your ldom is ready .You can follow the below steps to get your virtualized console 
>-bash-3.00# telnet localhost 5000
Trying 127.0.0.1...
Connected to localhost....
Escape character is '^}'.
Connecting to console “myldom1” in group “myldom1” ....
Press ~? for control options ..
{0} ok
>{0} ok show-disks
a) /virtual-devices@100/channel-devices@200/disk@0
q) NO SELECTION
Enter Selection, q to quit:q
{0} ok show-nets
a) /virtual-devices@100/channel-devices@200/network@0
q) NO SELECTION
Enter Selection, q to quit:q
{0} ok banner
Sun Fire T200, No Keyboard
Copyright 2007 Sun Microsystems, Inc. All rights reserved.
OpenBoot 4.26.1, 1024 MB memory available, Serial #66588434.
Ethernet address 0:14:4f:f8:f:12, Host ID: 83f80f12.
To Install Solaris OS using Jumpstart
{0} ok show-nets
a) /virtual-devices@100/channel-devices@200/network@0
q) NO SELECTION
Enter Selection, q to quit: a
/virtual-devices@100/channel-devices@200/network@0 has been selected.
Type ^Y ( Control-Y ) to insert it in the command line.
e.g. ok nvalias mydev ^Y
         for creating devalias mydev for /virtual-devices@100/channel-devices@200/network@0
{0} ok  nvalias vnet0 /virtual-devices@100/channel-devices@200/network@0
{0} ok boot vnet0
This will start the installation of Solaris from the jumpstart server in the network. Make sure you have added  this mac id generated for ldom in the /etc/ethers file of your jumpstart server to start it.
To use file as virtual disk to install  Solaris on Guest domain
#/usr/sbin/mkdir -p /ldoms/disk-images
# /usr/sbin/mkfile 5G /ldoms/disk-images/s10-5g.img
# ldm add-vdsdev /ldoms/disk-images/s10-5g.img vol2@primary-vds0
# ldm add-vdisk vdisk2 vol2@primary-vds0 myldom1

Supported guest operating systems :-
Up to :
> 32 LDom on T2000 (UltraSPARC T1)
> 64 LDom on T5220 (UltraSPARC T2)
> 128 LDom on T5240 (UltraSPARC T2 Plus)
> 128 Ldom on T5440 (UltraSPARC T2 Plus)

Requirements:-
Sun4v, Ultra Spark T1 Processor, Firmware 6.4.x and Solaris 10 11/06 OS,Parch 12492-02 or 125043-01,Logical domain s/w 1.0 (Pkg:-Sunwldmv)
Device Name:- vds, vsw, vdisk, vnet, vcc

Control Domain setup :-
• On the Control Domain do the following ...
• Ensure that the system is properly installed first
• In this case we will combine the Control and Service domains
• Set up the basic services needed..
> pkgadd SUNWldm
> svcadm ldmd start
> ldm add-vdiskserver primary-vds0 primary
> ldm add-vswitch mac-addr=<mac addr of e1000g0> net-dev=e1000g0 primary-vsw0 primary
> ldm add-vconscon port-range=5000-5100 primary-vcc0 primary
> ldm set-vcpu 8 primary
> ldm set-mem 4g primary
> ldm add-config initial
> init 6 (reboot for config to take effect )
> svcadm enable vntsd
Guest Domain setup:-

• On the Control Domain do the following ...
• Ensure that the system is properly installed first
• We will use a disk image on the control domain..
> mkfile 5g /ldom_disk/<ldom_name>/os
• Create our new domain description
> ldm add-domain <ldom_name>
> ldm add-vcpu 2 <ldom_name>
> ldm add-mem 3g <ldom_name>
> ldm add-vnet vnet1 primary-vsw0 <ldom_name>
> ldm add-vdsdev /ldom_disk/<ldom_name>/os zvol_<ldom_name>@primary-vds0
> ldm add-vdisk zvdisk_<ldom_name> zvol_<ldom_name>@primary-vds0 <ldom_name>
> ldm set-variable auto-boot\?=true <ldom_name>
> ldm set-variable boot-device=/virtual-devices@100/channel-devices@200/disk@0
<ldom_name>
> ldm bind <ldom_name>
• Now just need to start the domain...
> ldm start <ldom_name>
• Watch the console of <ldom_name> using ...
> telnet localhost <port no of ldom_name>

LDOM Setup :-

$ zfs clone $poolpath/$source $poolpath/$userldom
$ ldm add-domain $userldom
$ ldm add-vcpu 1 $userldom
$ ldm add-memory 1G $userldom
$ ldm add-vnet vnet1 primary-vsw0 $userldom
$ ldm add-vdsdev /dev/zvol/dsk/$poolpath/$userldom $userldom@primary-vds0
$ ldm add-vdisk vdisk1 $vol@primary-vds0 $userldom
$ ldm set-var auto-boot\?=true $userldom
$ ldm set-var boot-device=vdisk1 $userldom
$ ldm bind-domain $userldom
$ ldm start-domain $userldom
Setup user account via ldom console
Verify user ldom login works

LDOM Deletion :-

$ ldm stop $userldom 
$ ldm unbind $userldom 
$ ldm remove-vdsdev $userldom@primary-vds0
$ ldm destroy $userldom
$ zfs destroy -r $poolpath/$userldom
Extra :- 8 core & 4 thread=32  Virtual CPU’s
·         Virtual N/W Service (Vswitch):Primary-vsw0
·         Virtual N/W Device (Vnet):Vnet1
·         MAC Address:automatically allocated
·         Virtual disk service=Primary-vd0
·         Primary disk:/dev/dsk/c0t1d0s2
·         Virtual disk service name (vdsdev):vol1
·         Virtual disk:vdisk1
·         Virtual Console connector Service=primary-vcc0
·         Myldom1 Virtual console device=5000
·         /opt/SUNwldm/bin/ldm list-domain
   Name    State  Flags cons vcpu Memory util uptime
   Primary Active –t-cv  SP   4    1G     0.9% 16m  
#ldm add-domain test>>>Create new LDOM
#ldm add-vcpu 12 test>>Add virtual CPU
#ldm add-memory 1G test>>>add memory
#ldm add-vnet vnet1 primary-vsw0 test>>Add net
#ldm add-vdsdev /dev/dsk/c0t1d0s2 vil1@Primary-vds0>>Add virtual disk
#ldm add-vdisk vdisk1 vol1@primary-vd0 test>>Connect the physical disk to disk service
#ldm set-variable auto-boot\?=false test
#ldm set-variable boot-device=/virtual-devices@100/channet-device@200/disk0@0 test
#ldm bind-domain test>>finally bind the resource
#ldm start-domain test
#telnet localhost 5000
-----------------------------
-----------------------------
Ok banner/show-devs/dhow-disks/show-nets(to show device/disk/network resource)
Ok nvalias vnet0
Ok boot vnet0:dhcp
Primary#ldm list-domain –l test>>>to see CPU & Memory resource
OR test#psrinfo –p
#ldm ass-spconfig final
#ldm list-spconfig>>>>>>Last two command to finally save the configuration)
Install Logical Domain
1)cd <distribution directory>/LDOM manager 1.0
#pkgadd –d product SUNWldmv>>>Answer ‘Y’ for prompt
2)Enable the logical domain
#svcadm enable svc:/ldoms/ldmd:default
#svcadm enable svc:/ldoms/vntsd:default
Remove logical Domain pkg
Primary#/opt/SUNWldm/bin/ldm set-config factory-default
Primary#ldm list-config
#svcadm disable svc:/ldoms/ldmd:default
#pkgrm SUNWldm
Update Firmware version
Sc>poweroff –fy
Sc>flasupdate –s 127.0.0.1
Upgrading the Logical Domains Manager (1.2 software)
1.Bring down each domain to the ok prompt.
2.Issue the stop-domain subcommand from the control domain for each domain.
primary# ldm stop-domain ldom
3.Issue the unbind-domain subcommand from the control domain for each domain.
primary# ldm unbind-domain ldom

Upgrading to Logical Domains 1.2 Software
Existing LDoms 1.0.1, 1.0.2, 1.0.3, and 1.1 configurations do work with Logical Domains 1.2 software.

Upgrade From LDoms 1.0 Software

Existing Logical Domains 1.0 configurations do not work with Logical Domains 1.2 software,
so you need to save your Logical Domains 1.0 configurations before the upgrade to use them in Logical Domains 1.2 software
1.For each domain, create an XML file containing the domain's constraints.

# ldm list-constraints -x ldom > ldom.xml

2.List all the logical domain configurations stored on the service processor.
# ldm list-config

3.Remove each logical domain configuration stored on the service processor.
# ldm rm-config config-name

4.Disable the Logical Domains Manager daemon (ldmd).
# svcadm disable ldmd

5.Remove the Logical Domains Manager package (SUNWldm).
# pkgrm SUNWldm

6.Remove the Solaris Security Toolkit package (SUNWjass).
# pkgrm SUNWjass

3.Remove each logical domain configuration stored on the service processor.
# ldm rm-config config-name

4.Disable the Logical Domains Manager daemon (ldmd).
# svcadm disable ldmd

5.Remove the Logical Domains Manager package (SUNWldm).
# pkgrm SUNWldm

6.Remove the Solaris Security Toolkit package (SUNWjass).
# pkgrm SUNWjass

3.Remove each logical domain configuration stored on the service processor.
# ldm rm-config config-name

4.Disable the Logical Domains Manager daemon (ldmd).
# svcadm disable ldmd

5.Remove the Logical Domains Manager package (SUNWldm).
# pkgrm SUNWldm

6.Remove the Solaris Security Toolkit package (SUNWjass).
# pkgrm SUNWjass

7.Flash update the system firmware with FTP
# shutdown -i5 -g0 -y
sc> flashupdate -s IP-address -f path/Sun_System_Firmware-
x_x_x_build_nn-server-name.bin>>IP is FTP server
username: your-userid
password: your-password

Reset the service processor.
sc> resetsc -y

Power on and boot the host server.
sc> poweron -c
ok boot disk

Flash update the system firmware with FTP without FTP

 # cd firmware_location
# sysfwdownload system_firmware_file
# shutdown -i5 -g0 -y
sc> poweroff -fy>>3.Power off the system and update the firmware.
sc> flashupdate -s 127.0.0.1
sc> resetsc -y>>4.Reset the service processor and power on the system
sc> poweron


8.Reinstall the Logical Domain Manager and the Solaris Security Toolkit.

9.Reconfigure the primary domain manually.

10.Run the following commands for each guest domain's XML file you created in Step 1.



# ldm add-domain -i ldom.xml
# ldm bind-domain ldom
# ldm start-domain ldom


Upgrade From LDoms 1.0.x or 1.1

1.Flash update the system firmware.

2.Disable the Logical Domains Manager daemon (ldmd).
# svcadm disable ldmd

3.Remove the old SUNWldm package.
# pkgrm SUNWldm

4.Add the new SUNWldm package.

Specifying the -d option assumes that the package is in the current directory.
# pkgadd -Gd . SUNWldm

5.Use the ldm list command to verify that the Logical Domains Manager is running.

The ldm list command should list all domains that are currently defined on the system. In particular, the primary domain should be listed and be in the active state. The following sample output shows that only the primary domain is defined on the system.

# ldm list
NAME             STATE    FLAGS   CONS    VCPU  MEMORY   UTIL  UPTIME
primary          active   ---c-   SP      32    3264M    0.3%  19d 9m




Set Up the Control Domain

1. Determine whether you have cryptographic devices in the control domain.
primary# ldm list -o crypto primary

2. Assign cryptographic resources to the control domain
primary# ldm set-mau 1 primary

If you have any cryptographic devices in the control domain, you cannot dynamically reconfigure CPUs.
So if you are not using cryptographic devices, set-mau to 0.

3. Assign virtual CPUs to the control domain.
primary# ldm set-vcpu 4 primary

Following command would assign 4 virtual CPUs to the control domain, primary.
This leaves the remainder of the virtual CPUs available to a guest domain.

4. Assign memory to the control domain.
primary# ldm add-config initial

The following command would add a configuration called initial.

5. Add a logical domain machine configuration to the service processor (SP).
primary# ldm add-config initial

The following command would add a configuration called initial.

6. Verify that the configuration is ready to be used at the next reboot.
primary# ldm list-config
factory-default
initial [next poweron]

Upgrading the Solaris OS
Saving and Restoring Autosave Configuration Directories
1.Save the autosave directories
# cd /
# tar -cvf autosave.tar var/opt/SUNWldm/autosave-*

2.(Optional) Remove the existing autosave directories to ensure a clean restore operation
# cd /
# rm -rf var/opt/SUNWldm/autosave-*

3.Restore the autosave directories
# cd /
# tar -xvf autosave.tar

Preserving the Logical Domains Constraints Database File When Using Live Upgrade
If you are using live upgrade on the control domain, consider adding the following line to the /etc/lu/synclist file:

/var/opt/SUNWldm/ldom-db.xml     OVERWRITE
Posted by at No comments:
Subscribe to: Posts (Atom)