February 11, 2010 ahydle

Solaris 10 Mirror Repartition

I ran into a problem a while back because I used a manufacturer installed system without installing the OS myself and I overlooked the manufacturer defined partitioning scheme. After the system was in production I realized that the root file system was not using the entire disk and I was wasting about 400GB of disk space on the root mirror. So I had to repartition but I didnt want to reinstall. This documentation requires that the BIOS is able to choose between boot disks and may not work on all systems.

Please use these instructions at your own risk, test thoroughly before attempting, and make sure to backup everything as this could result in loss of data if done incorrectly.


Bring the system to Single user mode and back up to something…. Anything…. Really!

# init S
# ufsdump 0f /mnt/usb_disk/backup.bak /

Examine current disk/mirror configuration

Save the current md configuration and email it to yourself.

# metastat -p > /metanow
# mailx -s meta_save your_email@address.com < /metanow
# metastat -p

d30 -m d31 d32 1
d31 1 1 c0d0s5
d32 1 1 c0d1s5
d20 -m d21 d22 1
d21 1 1 c0d0s1
d22 1 1 c0d1s1
d10 -m d11 d12 1
d11 1 1 c0d0s0
d12 1 1 c0d1s0

Detach submirrors

# metadetach d10 d12
# metadetach d20 d22
# metadetach d30 d32

Clear submirrors

# metaclear d12 d22 d32

Remove metadb’s referencing mirrored disk

Locate the metadb’s on the submirror

# metadb -i
        flags           first blk       block count
     a m  p  luo        16              8192            /dev/dsk/c0d0s7
     a    p  luo        8208            8192            /dev/dsk/c0d0s7
     a    p  luo        16              8192            /dev/dsk/c0d1s7
     a    p  luo        8208            8192            /dev/dsk/c0d1s7
 r - replica does not have device relocation information
 o - replica active prior to last mddb configuration change
 u - replica is up to date
 l - locator for this replica was read successfully
 c - replica's location was in /etc/lvm/mddb.cf
 p - replica's location was patched in kernel
 m - replica is master, this is replica selected as input
 W - replica has device write errors
 a - replica is active, commits are occurring to this replica
 M - replica had problem with master blocks
 D - replica had problem with data blocks
 F - replica had format problems
 S - replica is too small to hold current data base
 R - replica had device read errors

Delete the metadb’s from the submirror (c0d1)

# metadb -d /dev/dsk/c0d1s7

FORMAT Mirrored Disk and setup partitions

# format c0d1s2

NEWFS your new partitions

Slice 1 is swap so we do not want to newfs it

# newfs /dev/rdsk/c0d1s0
# newfs /dev/rdsk/c0d1s5

Backup the vfstab and ready for changes

# cp -p /etc/vfstab /etc/vfstab.orig
# cp -p /etc/vfstab /etc/vfstab.switch

Edit the vfstab.switch and comment out the md mirrors and replace with the secondary disks that we formatted and newfs’ed earlier:

#device        device        mount        FS    fsck    mount    mount
#to mount    to fsck        point        type    pass    at boot    options
fd    -    /dev/fd    fd    -    no    -
/proc    -    /proc    proc    -    no    -
/dev/md/dsk/d20    -    -    swap    -    no    -
#/dev/md/dsk/d10    /dev/md/rdsk/d10    /    ufs    1    no    -
/dev/dsk/c0d1s0    /dev/rdsk/c0d1s0    /    ufs    1    no    -
#/dev/md/dsk/d30    /dev/md/rdsk/d30    /var    ufs    1    no    -
/dev/dsk/c0d1s5    /dev/rdsk/c0d1s5    /var    ufs    1    no    -
/devices    -    /devices    devfs    -    no    -
ctfs    -    /system/contract    ctfs    -    no    -
objfs    -    /system/object    objfs    -    no    -
swap    -    /tmp    tmpfs    -    yes    -
sharefs        -    /etc/dfs/sharetab    sharefs    -    no    -

Backup the system file and ready for changes

# cp -p /etc/system /etc/system.switch
# cp -p /etc/system /etc/system.orig

Edit /etc/system.switch and comment out the following line

* Begin MDD root info (do not edit)
* End MDD root info (do not edit)

Bring system to single user mode

# init S

Backup / Restore File Systems

# mount /dev/dsk/c0d1s5 /mnt
# ufsdump 0f - /dev/md/dsk/d30 | ( cd /mnt; ufsrestore xf - )
# umount /mnt
# mount /dev/dsk/c0d1s0 /mnt
# ufsdump 0f - /dev/md/dsk/d10 | ( cd /mnt; ufsrestore xf - )

Prep system

Prepare new root file system for first mount.

# cp /mnt/etc/vfstab.switch /mnt/etc/vfstab
# cp /mnt/etc/system.switch /mnt/etc/system

Install the boot block on the newly created FS

# /mnt/sbin/installgrub -m /mnt/boot/grub/stage1 /mnt/boot/grub/stage2 /dev/rdsk/c0d1s0

Delete the boot signature file that was copied over during the backup and restore (File name may be different)

# rm /mnt/boot/grub/bootsign/rootfs0

Create a boot signature file of the new root file system to be booted.

# touch /mnt/boot/grub/bootsign/rootfs1

Change the grub menu.lst on the newly created FS

# vi /mnt/boot/grub/menu.lst

Add a new entry to the menu.lst for your disk. Note the findroot entry and make sure it matches the boot signature you created earlier.

title Solaris 10 8/07 NEWBOOT
findroot (rootfs1,0,a)
kernel /platform/i86pc/multiboot
module /platform/i86pc/boot_archive

Locate the physical path to your hardware in order to update the bootenv.rc file.

# ls -l /dev/dsk/c0d1s0
lrwxrwxrwx   1 root     root          50 Feb  8 12:22 /dev/dsk/c0d1s0 -> ../../devices/pci@0,0/pci-ide@7,1/ide@0/cmdk@1,0:a

Next edit the bootenv.rc to add the new path.

vi /mnt/boot/solaris/bootenv.rc

Find the following entry and update it to the physical path of your disk.

setprop bootpath /pci@0,0/pci-ide@7,1/ide@0/cmdk@1,0:a

Unmount root

# cd /
# umount /mnt

Check the new file systems

# fsck -y /dev/rdsk/c0d1s0
# fsck -y /dev/rdsk/c0d1s5

Re-mount root to rebuild boot image upon boot

# mount /dev/dsk/c0d1s0 /mnt
# mount /dev/dsk/c0d1s5 /mnt/var


# init 6

At this point you will need to switch your boot device to the new disk through the BIOS. It may also be possible to change the grub entry in your existing disk but make sure to finish your mirror configuration before rebooting otherwise you should be able to get by using a boot disk

You can verify whether you are booting off of the new disk by looking for the NEWBOOT menu entry.

Check the disk

Verify that we are on the correct disk

Check to make sure there are no md devices mounted except swap and make sure that root is the disk we just brought online (c0d1s0).

# mount | grep md

Rebuild the mirror

Now that we are on the new disk and have verified everything is running correctly we are going to clear the existing root / var partitions and create new ones.

Clear the existing mirrors that are no longer in use

# metaclear -f d10
# metaclear -f d11
# metaclear -f d30
# metaclear -f d31

Setup new mirrors

# metainit -f d11 1 1 c0d1s0
# metainit -f d31 1 1 c0d1s5
# metainit d10 -m d11
# metainit d30 -m d31
# metaroot d10

Do some shuffling around to bring the naming convention back inline on swap

# metainit -f d23 1 1 c0d1s1
# metattach d20 d23

Check to make sure d23 is sync’ed up

 d20: Mirror
    Submirror 0: d21
      State: Okay
    Submirror 1: d23
      State: Okay
    Pass: 1
    Read option: roundrobin (default)
    Write option: parallel (default)
    Size: 2104515 blocks (1.0 GB)

d21: Submirror of d20
    State: Okay
    Size: 2104515 blocks (1.0 GB)
    Stripe 0:
        Device   Start Block  Dbase        State Reloc Hot Spare
        c0d0s1          0     No            Okay   Yes

d23: Submirror of d20
    State: Okay
    Size: 2104515 blocks (1.0 GB)
    Stripe 0:
        Device   Start Block  Dbase        State Reloc Hot Spare
        c0d1s1          0     No            Okay   Yes

Once d23 is sync’ed up we are going to take out d21 and rename it to d22

# metadetach d20 d21
# metaclear d21
# metainit -f d22 1 1 c0d0s1 
# metattach d20 d22

Again make sure that d22 is all sync’ed up (State should read Okay)

# metastat

…and shuffle d23 around making it d21 so it matches the new naming convention

# metadetach d20 d23
# metaclear d23
# metainit -f d21 1 1 c0d1s1 
# metattach d20 d21

Lastly we are going to remove d22 so that we can repartition c0d0 in the next few steps

# metadetach d20 d22
# metaclear d22

Create new metadb’s on c0d1 and delete the old metadb’s on c0d0.

# metadb -afc3 /dev/dsk/c0d1s7
metadb -d /dev/dsk/c0d0s7

Now we can copy the vfstab.orig back over the vfstab or edit /etc/vfstab. There shouldnt be a need to copy the /etc/system.orig back because we did the metaroot d10 command earlier.

# cp /etc/vfstab.orig /etc/vfstab

Reboot the system

# init 6

Copy the new partition scheme to the c0d0

prtvtoc /dev/rdsk/c0d1s2 > /var/tmp/c0d1s2.vtoc
fmthard -s /var/tmp/c0d1s2.vtoc /dev/rdsk/c0d0s2

Finish setting up the mirrors. If you get an error saying c0d0s1 is a dump device see note below.

# metainit d12 1 1 c0d0s0
# metainit d22 1 1 c0d0s1
# metainit d32 1 1 c0d0s5
# metattach d10 d12
# metattach d30 d32
# metattach d20 d22

If you get an “is a dump device” error when adding c0d0s1 to d20 you may need to repoint the dump location to the mirror

# dumpadm -d /dev/md/dsk/d20

Wait for everything to sync back up.


Cleaning up the mess

Now that everything is back up and running with the new disks I went back and edited my Grub menu.lst so that the findroot entries were looking for the right locations and I deleted the NEWBOOT entry I added before.

#---------- ADDED BY BOOTADM - DO NOT EDIT ----------
title Solaris 10 8/07 s10x_u4wos_12b X86
findroot (rootfs1,0,a)
kernel /platform/i86pc/multiboot
module /platform/i86pc/boot_archive
#---------------------END BOOTADM--------------------
#---------- ADDED BY BOOTADM - DO NOT EDIT ----------
title Solaris failsafe
findroot (rootfs1,0,a)
kernel /boot/multiboot kernel/unix -s
module /boot/x86.miniroot-safe
#---------------------END BOOTADM--------------------
, , , , , , ,

Comment (1)

  1. Wow, It seems to be very difficult, but I’m sure I can learn it. Thanks to you I have an example that I can work on.

Leave a Reply

Your email address will not be published. Required fields are marked *