Friday, August 31, 2012

To Create a Live Linux CD/DVD

A live CD is a bootable CD that contains the whole operating system and any applications you want to run from the CD. With the increasing popularity of DVD and Blu-ray drives, there is even a live DVD. A live CD is useful when you want to copy your existing Linux system to a CD and run it on multiple computers. In this post, I show how I normally create a live CD.



Rebuild a kernel with SquashFS and UnionFS patch


A new kernel should be built with SquashFS and UnionFS patches. The live CD will have a huge file that is actually a SquashFS image containing a copy of your existing Linux system. We will use UnionFS so that any changes to the read-only SquashFS will be saved to the volatile memory. Download the patches from the following locations.





After patching the kernel and rebuilding it, install the new kernel.



Make a SquashFS image of an existing Linux system


A SquashFS image needs to be created from an existing Linux system. Copy the contents of the Linux filesystem to an empty directory, assuming the filesystem is mounted at /mnt.



mkdir /tmp/livecd
cd /mnt
tar cf - . | (cd /tmp/livecd; tar xf -)


Empty the contents of fstab so that the live CD will not mount any hard disk partition, including the original root partition.



cp /dev/null /tmp/livecd/etc/fstab


Create a squashfs image.



mksquashfs /tmp/livecd /tmp/linuxfs.sqz -comp xz -b 262144 -Xdict-size 196608 -no-xattrs -e boot lost+found selinux srv


Create a Custom InitRamFS image


This is a tricky part of the job. Basically, the init script of the initramfs image will mount the CD-ROM, then mount the SquashFS image, and then mount a UnionFS. Refer to this post on building an initramfs image.



Create a Live CD structure.


Create an empty directory which will be the contents of live CD. The live CD will contain:




  • SquashFS image (named linuxfs.sqz, for example)
  • isolinux.bin and isolinux.cfg
  • kernel
  • initramfs image


Finally, create an ISO image for the live CD.



genisoimage -V LIVECD -b isolinux.bin -c boot.cat -no-emul-boot -boot-load-size 4 -boot-info-table /tmp/isofs > ~/livecd.iso


Burn the ISO image.



cdrskin -v dev=/dev/sr0 speed=8 driveropts=burnfree ~/livecd.iso

Tuesday, August 28, 2012

Manually Creating initrd / initramfs to Boot Linux

Initrd and initramfs files are used together with a Linux kernel and a bootloader (such as GRUB and syslinux) to start up Linux and boot it in many different ways. Many Linux distributions provide prepackaged kernels and convenient tools for creating initramfs, but I like to build my own kernel and initramfs because it allows me to customize the boot process. For instance, one can choose to boot from the local disk or network-mounted disk with the help of initramfs and some user-defined boot parameters. Here's a list of things that I'd like to achieve with my custom initramfs.




  1. Boot from a live CD
  2. Boot from a read-only filesystem image, compressed with SquashFS
  3. Copy the filesystem image to a RAM disk and run Linux entirely on memory
  4. Boot Linux from a USB flash drive or a Firewire disk
  5. Boot Linux from a local disk partition
  6. Boot Linux from a network drive
  7. Run a rescue shell without booting Linux
  8. Start a kdrive X-server and run gparted, partclone or partimage


In this post, I show how I normally create initrd / initramfs files. For this tutorial, the following Debian/Ubuntu packages are needed:




  • busybox

    provides small essential utilities for booting and rescue shell.
  • cpio

    is used to create the actual initramfs format.
  • dash

    provides a minimalist shell for running the init script and the rescue shell.
  • e2fsprogs, jfsutils, etc.

    provides fsck to check Linux filesystems and replay any stale journal
  • lzma or xz-utils

    compresses the initramfs
  • module-init-tools

    is used to load kernel modules necessary to activate hardware and mount the filesystem
  • pciutils

    is used to detect PCI hardware
  • unionfs-fuse

    makes it possible to use read-only Linux systems, such as live CD and filesystem images
  • unzip

    is used to apply customized settings and changes to the unionfs boot mode
  • v86d

    is used to set the screen resolution for the framebuffer screen


First, create a text file with a list of files to put in initramfs. The following is an example of such file:



bin/busybox
bin/dash
bin/mount
etc/filesystems
etc/fuse.conf
etc/group
etc/modprobe.d
lib/ld-linux.so.2
lib/libacl.so.1
lib/libattr.so.1
lib/libblkid.so.1
lib/libbz2.so.1.0
lib/libcom_err.so.2
lib/libc.so.6
lib/libdl.so.2
lib/libe2p.so.2
lib/libext2fs.so.2
lib/libfuse.so.2
lib/libkmod.so.2
lib/libmount.so.1
lib/libm.so.6
lib/libpci.so.3
lib/libpthread.so.0
lib/libresolv.so.2
lib/librt.so.1
lib/libselinux.so.1
lib/libsepol.so.1
lib/libuuid.so.1
lib/libx86.so.1
lib/libz.so.1
lib/modules/3.5.3/kernel/drivers/ata
lib/modules/3.5.3/kernel/drivers/block/loop.ko
lib/modules/3.5.3/kernel/drivers/cdrom/cdrom.ko
lib/modules/3.5.3/kernel/drivers/connector/cn.ko
lib/modules/3.5.3/kernel/drivers/firewire/firewire-core.ko
lib/modules/3.5.3/kernel/drivers/firewire/firewire-ohci.ko
lib/modules/3.5.3/kernel/drivers/firewire/firewire-sbp2.ko
lib/modules/3.5.3/kernel/drivers/i2c/algos/i2c-algo-bit.ko
lib/modules/3.5.3/kernel/drivers/scsi/sd_mod.ko
lib/modules/3.5.3/kernel/drivers/scsi/sr_mod.ko
lib/modules/3.5.3/kernel/drivers/usb/host/ehci-hcd.ko
lib/modules/3.5.3/kernel/drivers/usb/host/ohci-hcd.ko
lib/modules/3.5.3/kernel/drivers/usb/host/uhci-hcd.ko
lib/modules/3.5.3/kernel/drivers/usb/storage/usb-storage.ko
lib/modules/3.5.3/kernel/drivers/video
lib/modules/3.5.3/kernel/fs/ext3/ext3.ko
lib/modules/3.5.3/kernel/fs/ext4/ext4.ko
lib/modules/3.5.3/kernel/fs/fat/fat.ko
lib/modules/3.5.3/kernel/fs/fat/vfat.ko
lib/modules/3.5.3/kernel/fs/fuse/fuse.ko
lib/modules/3.5.3/kernel/fs/isofs/isofs.ko
lib/modules/3.5.3/kernel/fs/jbd2/jbd2.ko
lib/modules/3.5.3/kernel/fs/jbd/jbd.ko
lib/modules/3.5.3/kernel/fs/jfs/jfs.ko
lib/modules/3.5.3/kernel/fs/nls/nls_cp437.ko
lib/modules/3.5.3/kernel/fs/nls/nls_iso8859-1.ko
lib/modules/3.5.3/kernel/fs/nls/nls_utf8.ko
lib/modules/3.5.3/kernel/fs/reiserfs/reiserfs.ko
lib/modules/3.5.3/kernel/fs/squashfs/squashfs.ko
lib/modules/3.5.3/kernel/fs/xfs/xfs.ko
lib/modules/3.5.3/kernel/lib/crc16.ko
lib/modules/3.5.3/kernel/lib/crc-t10dif.ko
lib/modules/3.5.3/modules.dep
lib/modules/3.5.3/modules.dep.bin
lib/modules/3.5.3/modules.pcimap
sbin/blkid
sbin/e2fsck
sbin/jfs_fsck
sbin/modprobe
sbin/v86d
usr/bin/pcimodules
usr/bin/unionfs-fuse
usr/bin/unzip


Then, create an empty directory, for example, /tmp/initrd and copy the files listed in the above-mentioned text file (called rd354.txt) to the new directory.



mkdir /tmp/initrd
cd /
tar cvhf - -T rd354.txt | (cd /tmp/initrd; tar xf -)


Then, create the necessary directory structure under /tmp/initrd:



cd /tmp/initrd
mkdir -p dev media mnt proc root sys tmp


Since I chose to use busybox, I need to create symbolic links to busybox in /bin. Busybox provides incomplete functionality for modprobe, mount, sh and unzip, so I removed their symbolic links.



cd /tmp/initrd/bin
for f in $(./busybox --list); do [ -e $f ] || ln -s busybox $f ; done
rm modprobe unzip


Then, create essential device nodes in /tmp/initrd/dev:



cd /tmp/initrd/dev
MAKEDEV std fb0 fd0 sda sdb scd
mknod console c 5 1
mknod fuse c 10 229
chmod 666 fuse


Now, create an init script. The contents of this init script is crucial for customization of the boot process. The following is the script I use. This script is capable of booting a live CD and running Linux within memory in addition to booting Linux from hard drives and USB flash.



#!/bin/dash

# Define a function to parse kernel command line options.
get_opt() {
echo $@ | cut -d "=" -f 2
}

# Define a function to load drivers.
loadmod() {
for i in $@ ; do
for j in $(grep $i /tmp/pcimodules.txt); do
modprobe $j
done
done
}

# Define a function to guess the partition type.
gpart() {
for i in $(blkid | grep $1); do
case $i in
*\=*)
eval $i
;;
*)
true
;;
esac
done
}

# Define a function for mounting the root partition.
mountr() {
if [ $uuid ]; then
if [ $# = 2 ]; then
mount -r -U $uuid $2
elif [ $# = 1 ]; then
mount -r -U $uuid $1
else mount -r -U $uuid /mnt
fi
elif [ $label ]; then
if [ $# = 2 ]; then
mount -r -L $label $2
elif [ $# = 1 ]; then
mount -r -L $label $1
else mount -r -L $label /mnt
fi
else
gpart $1
case $TYPE in
ext*)
e2fsck -p $1
[ $# = 2 ] && mount $1 $2 || mount $1 /mnt
;;
jfs)
jfs_fsck $1
if [ $# = 2 ]; then
mount -t jfs -o ro,iocharset=utf8 $1 $2
else mount -t jfs -o ro,iocharset=utf8 $1 /mnt
fi
;;
vfat)
if [ $# = 2 ]; then
mount -t vfat -o ro,gid=100,dmask=2,fmask=113 $1 $2
else mount -t vfat -o ro,gid=100,dmask=2,fmask=113 $1 /mnt
fi
;;
*)
[ $# = 2 ] && mount -r $1 $2 || mount -r $1 /mnt
;;
esac
fi
}

# Create a union filesystem
union() {
mount -t tmpfs none /opt/tmp
modinfo unionfs > /dev/null 2>&1 &&
mount -t unionfs -o dirs=/opt/tmp=rw:/opt=ro none /mnt ||
( mkdir /opt/tmp/.change
modprobe fuse
unionfs-fuse -o allow_other,use_ino,suid,dev,nonempty,kernel_cache \
-o cow,chroot=/opt,max_files=32768 /tmp/.change=RW:/=RO /mnt )
}

# Mount proc and sysfs.
mount -t proc none /proc
mount -t sysfs none /sys

# Find the available PCI hardware
mount -t tmpfs none /tmp
pcimodules > /tmp/pcimodules.txt

# Populate /dev (Needs kernel >= 2.6.32)
mount -t devtmpfs none /dev
mkdir -m 755 /dev/pts
mount -t devpts -o gid=5,mode=620 none /dev/pts

# Set default values
boot=ata
root=/dev/sda6

# Find the root=, label=, uuid= and boot= values on kernel command line.
for i in $(cat /proc/cmdline); do
case $i in
root\=*)
root=$(get_opt $i)
case $root in
/dev/cdr* | /dev/dvd* | /dev/sr* | /dev/scd*)
boot=cdrom
;;
0x200)
root=/dev/fd0
;;
esac
;;
label\=* | uuid\=* | boot\=* | vmode\=* )
eval $i
;;
single)
RUNLEVEL=single
;;
nox)
RUNLEVEL=2
;;
esac
done

# Activate framebuffer display devices.
if [ $vmode ]; then
if [ $boot = cdrom ]; then
modprobe uvesafb scroll=ywrap mode_option=$vmode-16
else for i in $(grep fb /tmp/pcimodules.txt); do
case $i in
atyfb)
modprobe $i mode=$vmode-16
;;
nvidiafb | rivafb)
modprobe nvidiafb mode_option=$vmode bpp=16 hwcur=1
;;
radeonfb | savagefb)
modprobe $i mode_option=$vmode-16
;;
sisfb)
modprobe $i mode=$vmodex16 mem=12288 font=SUN12x22
;;
viafb | vt8623fb)
modprobe viafb viafb_mode=$vmode viafb_bpp=16
;;
*)
modprobe $i
;;
esac
done
if grep -q i915 /tmp/pcimodules.txt; then true
else [ -c /dev/fb0 ] || modprobe uvesafb scroll=ywrap mode_option=$vmode-16
fi
fi
fi

case $boot in
cdrom)
# Boot Linux from a live CD.
loadmod ata_ ahci pdc_adma ^.hci-hcd
modprobe usb-storage &&
modprobe sr_mod &&
sleep 7
modprobe isofs
mount -t iso9660 /dev/sr0 /media
[ -d /media/isolinux -o -d /media/boot/isolinux ] ||
mount -t iso9660 /dev/sr1 /media
if [ -f /media/*.[Ss][Qq]* ]; then
SQF=$(ls -t /media/*.[Ss][Qq]* | head -n 1)
if [ $root = /dev/ram ]; then
echo "Please wait until the RAM disk is ready."
dd if=$SQF of=/dev/ram1 bs=2048 &&
mount -t squashfs /dev/ram1 /opt
else modprobe loop
mount -t squashfs -o loop $SQF /opt
fi
else
mount --move /media /opt
fi
union
;;
loop)
# Boot Linux from an image file.
loadmod ata_ ahci pdc_adma ^.hci-hcd
modprobe usb-storage &&
modprobe sd_mod &&
sleep 7
mountr $root /media
modprobe loop
if [ -f /media/*.[Ss][Qq]* ]; then
SQF=$(ls -t /media/*.[Ss][Qq]* | head -n 1)
mount -t squashfs -o loop $SQF /opt
elif [ -f /media/*.[Ii][Ss][Oo] ]; then
ISO=$(ls -t /media/*.[Ii][Ss][Oo] | head -n 1)
modprobe isofs
mount -t iso9660 -o loop $ISO /opt
fi
union
;;
ram)
# Boot Linux from ramdisk.
loadmod ata_ ahci pdc_adma ^.hci-hcd
modprobe usb-storage &&
modprobe sd_mod &&
sleep 7
mountr $root /media
echo "Please wait until the RAM disk is ready."
if [ -f /media/*.[Ss][Qq]* ]; then
SQF=$(ls -t /media/*.[Ss][Qq]* | head -n 1)
dd if=$SQF of=/dev/ram1 &&
mount -t squashfs /dev/ram1 /opt
elif [ -f /media/*.[Ii][Ss][Oo] ]; then
ISO=$(ls -t /media/*.[Ii][Ss][Oo] | head -n 1)
dd if=$ISO of=/dev/ram1 bs=2048 &&
modprobe isofs
mount -t iso9660 /dev/ram1 /opt
fi
union
;;
usb*)
# Boot Linux from a USB drive.
loadmod ^.hci-hcd
modprobe usb-storage &&
modprobe sd_mod &&
sleep 7
mountr $root
;;
ata*)
loadmod ata_ ahci pdc_adma &&
modprobe sd_mod &&
mountr $root
;;
esac

# Make sure that init exists and is executable.
if [ -x /mnt/sbin/init ]; then
mount --move /dev /mnt/dev
mount --move /proc /mnt/proc
mount --move /sys /mnt/sys
umount /tmp

# Start init from the root filesystem.
cd /mnt
[ -f /media/updates.zip ] && unzip -o /media/updates.zip
case $boot in
cdrom)
[ $root = /dev/ram ] && umount /media
[ $RUNLEVEL ] || RUNLEVEL=3
;;
loop | ram)
umount /media
[ $RUNLEVEL ] || RUNLEVEL=3
;;
*)
[ $RUNLEVEL ] || RUNLEVEL=5
;;
esac
[ -d initrd ] && pivot_root . initrd
exec chroot . /sbin/init $RUNLEVEL
fi

# Start a shell as a last resort.
echo "Error booting from the root filesystem. Starting a shell."
exec /bin/dash


Copy the init script to the initrd folder (Assuming its filename is init.sh).



cp init.sh /tmp/initrd/init
cp init.sh /tmp/initrd/etc
chmod 775 /tmp/initrd/init


Now, use cpio and lzma to create an initramfs file. This assumes that the kernel was compiled with CONFIG_RD_LZMA option enabled.



cd /tmp/initrd
find . | cpio -H newc -o | lzma -c > ../initram.lzm


A file named initram.lzm will be created. If you want to create an old-style initrd file instead, use the mkcramfs command. This assumes your kernel has built-in cramfs support (CONFIG_CRAMFS):



mkcramfs /tmp/initrd /boot/initrd.bin


Copy the kernel and the initramfs file to your boot directory (whereever it is). I use SYSLINUX which I installed at /dev/sda1 partition. Finally, edit the boot configuration file. The following is the contents of a sample syslinux.cfg:



LABEL debian
KERNEL 314.lnx
INITRD /initrd/initram.lzm
APPEND root=/dev/sda7

LABEL sid
KERNEL 314.lnx
INITRD /initrd/initram.lzm
APPEND boot=usb label=SID edd=off vmode=800x600

LABEL bfile
KERNEL 314.lnx
INITRD /initrd/initram.lzm
APPEND boot=loop root=/dev/sda1

LABEL ram
KERNEL 314.lnx
INITRD /initrd/initram.lzm
APPEND boot=ram ramdisk_size=524288 root=/dev/sda1

Friday, August 24, 2012

Getting Eclipse to Work with MinGW C/C++ Compilers

Eclipse is one of my favorite programming environment which is used to develop C, C++ and/or Java software efficiently. Let's set up Eclipse so that we can use it to work with MinGW C/C++ compiler and Java.




  1. First, install MinGW (as shown in this post). You have two choices for the MinGW compiler:




    I prefer the original MinGW compiler to MinGW-w64, but individual tastes may vary. Make sure that you add the folder containing gcc.exe to the PATH environment variable.




  2. Then, install JRE because Eclipse won't run without it. If you want to develop Java, then install JDK (Java SE or Java EE) instead of JRE.



  3. Now, download Eclipse IDE for C/C++ Developers (eclipse-cpp-juno-win32.zip) and unpack it. You need it to develop C/C++ programs.



  4. If you also want to develop Java, then download Eclipse IDE for Java Developers (eclipse-java-juno-win32.zip). Just unpack the Java Developer package into the same folder as Eclipse for C/C++, avoiding to overwrite existing files.



  5. Create a shortcut on your desktop that points to the Eclipse program (eclipse.exe). To start Eclipse, double-click the Eclipse shortcut that you just created.



  6. Let's test our Eclipse installation and create a simple C++ program. Start a new C++ project (File->New->C++ Project). Type in any Project name, and select Empty Project under Executable Project types. Choose MinGW GCC for toolchains. Click Finish.




    Once a new C++ project is open, create a new source file (File->New->Source File).




    Try typing in a simple code like the following:


    #include <iostream>
    using namespace std;

    int main()
    {
    cout << "Hello, boys and girls.\n";
    return 0;
    }


    Save the project (File->Save). Build the project (Project->Build Project). If the build was successful, run the generated program (Run->Run, or Ctrl+F11).


Thursday, August 23, 2012

MinGW: Compiling LZO Compression Library

LZO, short for Lempel-Ziv-Oberhumer, is compression software that is designed for decompression speed. I compiled the LZO library like this.


tar xzvf lzo-2.03.tar.gz

cd lzo-2.03/

./configure --prefix=/mingw

make

make install

I only got a static library liblzo2.a. So I had to convert it to a DLL.



gcc -shared -o liblzo2-2.dll -Wl,--out-implib,liblzo2.dll.a -Wl,--whole-archive /mingw/lib/liblzo2.a -Wl,--no-whole-archive -L/mingw/lib -lwinmm

Compiling gtkmm for Windows with MinGW

gtkmm is a C++ wrapper library that makes it possible to write C++ programs with GTK+. After compiling the GTK+ library, you can build gtkmm as follows.




  1. libsigc++ 2.2.10


    libsigc++ implements a typesafe callback system for standard C++. Download the libsigc++ source code from here and compile it:


    ./configure --prefix=/mingw
    make
    make install

  2. glibmm 2.32.1


    glibmm is a C++ wrapper for Glib. Compile it like this:


    ./configure --prefix=/mingw
    make
    make install

  3. atkmm 2.22.6


    atkmm is a C++ wrapper for ATK. Compile it like this:


    ./configure --prefix=/mingw
    make
    make install

  4. cairomm 1.10.0


    cairomm provides C++ interface to the Cairo library. Compile it like this:


    ./configure --prefix=/mingw
    make
    make install

  5. pangomm 2.28.4


    pangomm is a C++ wrapper for Pango. Compile it like this:


    ./configure --prefix=/mingw
    make
    make install

  6. gtkmm 3.4.0


    gtkmm is a C++ wrapper for GTK+. Build it as follows:


    ./configure --prefix=/mingw
    make
    make install

Friday, August 17, 2012

Building GraphicsMagick

GraphicsMagick is a powerful tool for processing image files. I'm using MinGW to build GraphicsMagick for Windows.




  1. zlib 1.2.7


    Download the zlib source (zlib-1.2.7.tar.gz) and unpack it.


    tar xzvf zlib-1.2.7.tar.gz
    cd zlib-1.2.7

    Compile and install zlib.


    make -f win32/Makefile.gcc
    cp -iv zlib1.dll /mingw/bin
    cp -iv zconf.h zlib.h /mingw/include
    cp -iv libz.a /mingw/lib
    cp -iv libz.dll.a /mingw/lib


  2. bzip2


    Now download bzip2 source from bzip.org and unpack it.


    tar xzvf bzip2-1.0.6.tar.gz
    cd bzip2-1.0.6

    Change line 78 of bzlib.h to read:


    #if defined(_WIN32) && !defined(__MINGW32__)

    Compile and install bzip2.


    make
    cp bzlib.h /mingw/include/
    cp libbz2.a /mingw/lib


  3. liblzma


    Get the XZ Utils source code and unpack it. Go to the src/liblzma folder and compile liblzma like this:


    tar xzvf xz-5.0.4.tar.gz
    cd xz-5.0.4
    ./configure --prefix=/mingw
    cd src/liblzma
    make
    make install


  4. libpng 1.5.12 and libjpeg 8d


    Compile libpng and libjpeg as follows:


    ./configure --prefix=/mingw
    make
    make install


  5. libjasper


    Compile Jasper:


    ./configure --prefix=/mingw
    make
    make install

    If you encounter an error "undefined reference to _sleep", replace sleep() with _sleep with a factor of 1000 in the code:


    _sleep(1000)


  6. JBig


    Get jbigkit from here.


    make
    cp libjbig/*.h /mingw/include
    cp libjbig/*.a /mingw/lib


  7. libtiff 4.0.2


    Compile libtiff like this:


    ./configure --prefix=/mingw
    make
    make install

    If the shared library libtiff-5.dll is not generated and you want to create it manually, use such a command as the following and update libtiff.la accordingly:


    gcc -shared -o libtiff-5.dll -Wl,--out-implib,libtiff.dll.a -Wl,--whole-archive /mingw/lib/libtiff.a -Wl,--no-whole-archive -L/mingw/lib -ljpeg -ljbig -llzma -lz


  8. freetype


    Compile freetype:


    ./configure --prefix=/mingw
    make
    make install


  9. libxml2


    Compile libxml2:


    ./configure --prefix=/mingw
    make
    make install


  10. libwmf 0.2.8.4


    Compile libwmf:


    ./configure --prefix=/mingw
    make
    make install


  11. Little CMS


    Compile liblcms:


    ./configure --prefix=/mingw
    make
    make install


  12. Build GraphicsMagick


    Get GraphicsMagick code and build it like this:


    ./configure --prefix=/mingw --with-gs-font-dir='C:/Windows/Fonts' --with-windows-font-dir='C:/Windows/Fonts' LIBS='-ljbig'
    make
    make install



Creating a PDF document from JPEG files


GraphicsMagick comes in handy when you want to convert scanned JPEG images to a PDF document:


gm convert *.jpg doc.pdf

About This Blog

KBlog logo This blog seeks to provide a collection of interesting pictures found on the Web. Thanks for visiting the blog and posting your comments.

© Contents by KBlog

© Blogger template by Emporium Digital 2008

Followers

Total Pageviews

Powered By Blogger