Using Expect Here Document in Shell Scripts

Sometimes simple interactions are needed for Shell scripts. In this scenario, a here document written in Expect will work well. In following example, the output of Expect here document is assigned to KSH variable rcs_stat.

#!/bin/ksh

autoload formatAPnum

# Set the lab name, COOLLAB
. $COOLXDIR/.netlabs
# Get the RCS cell list
set -A RCSs
typeset -i nx=0
cat $COOLXDIR/.coolcell2dcs | while read cell
do
	if [[ "$cell" = c* ]]
	then
		rcs=${cell%%$COOLLAB*}
		RCSs[$nx]=${rcs#c}
		nx=$nx+1
	fi
done

# Check RCS status
typeset -i loopCount=0
rcs_cnt=${#RCSs[*]}
B_server=$(formatAPnum $BserverAP)
while [ "${#RCSs[*]}" != 0 -a $loopCount -lt 720 ]  # wait at most 2 hours
do
	loopCount=${loopCount}+1

	nx=0
	while [ $nx -lt $rcs_cnt ]
	do
		rcs_stat="OOS"
		rcs_stat=$(
expect - <<!
log_user 0
set timeout 20
spawn $COOL_RSH ap$B_server TICLI
send "op:cell ${RCSs[$nx]}\r"
expect {
	timeout {puts "OOS\n"}
	"*DL(S) DOWN" {puts "OOS\n"}
	"*DL(S) UP" {puts "UP\n"}
}
!
		)

		if [ "$rcs_stat" = "UP" ]
		then
			coolprint - "RCS cell ${RCSs[$nx]} is up."
			unset RCSs[$nx]
		fi

		nx=${nx}+1
	done

	[ -n "${RCSs[*]}" ] && sleep 10 
done

coolprint - "All RCSs are up."
exit 0

 

Shell中使用Expect Here Document

在使用Shell写程序时,有时不得不面对交互的问题——可惜shell往往无法自动完成交互。如果交互的内容很多,自然是直接应用Expect脚本比较方便;如果交互的内容很少,而且考虑到Shell的易用性,在Shell脚本中创建一个Expect 的Here Document更加方便灵活。

如下例所示,Expect Here Document可以直接运行并将运行结果赋给KSH变量rcs_stat。

#!/bin/ksh


            
          

File Descriptors per Process

A file descriptor is a handle created by a process when a file is opened. There is a limit to the amount of file descriptors per process. In order to check the default limit of the system, you can use command:

  1. ulimit -a # all limitations
  2. ulimit -Sn # soft limit
  3. ulimit -Hn # hard limit

If the file descriptor limit is exceeded for a process, you may see the error Too Many Open Files.

And if you want to change the limit per process temporarily, you can:

  1. ulimit -n <desired_#> # both
  2. ulimit -Sn <desired_#> # soft limit
  3. ulimit -Hn <desired_#> # hard limit

To check the open files of a process, you can use command pfiles <PID>.

To check the run time limit of each process in Solaris, you can plimit <PID>(it seems this command is not supported in Linux). If you have the root permission and you want to change the limit of an process without killing or restarting the process, you can plimit -n <desired_#>.

It was said that it may be dangerous to set the soft limit higher than 256 or hard limit greater than 1024 due to limitations with the stdio library.If programs require more file descriptors, they should use setrlimit directly.

-------------
References:

  1. File descriptors
  2. Linux Increase The Maximum Number Of Open Files / File Descriptors (FD)
  3. Linux / UNIX: List Open Files for Process
  4. Princeton University Enterprise Servers and Storage, File Descriptors
  5. plimit

OnLive——在iPad上使用MS Word, Excel, PowerPoint

昨晚睡前看到Walter Mossburg的最新文章Working in Word, Excel, PowerPoint on an iPad:尽管iPad上已经有office软件,可以在上面编辑或创建文档,但这些软件对MS Office的兼容性并不好,而且还是收费的。OnLive
Desktop
即将推出一个免费的应用,该应用可以利用云端的MS Word, Excel, PowerPoint创建并编辑文档,文档保存在云端,可以在iPad和PC上(今后会有Android版)编辑。免费版每人2G云端存储空间,付费版$10每月每人50G空间。 除常用的Office软件,还支持常用的Windows小程序,如计算器,画图板,记事本等。

OnLive对于网络要求较高,低于1M的网络很可能无法连接到服务器。另外,文档输入时的虚拟键盘不是iPad的虚拟键盘而是Windows的键盘,编辑长文档时会很麻烦。

看过视频之后,我认为这个软件就是一个高级版的VNC应用,创新之处就在于云端的Windows服务器集群,能维护Windows环境以及自动为用户分配Windows服务器。一年多以前,在《Splashtop Remote——让你在iPad上也能用Windows下的应用》一文中,我曾预言

"试想如果将来有公司(比如这个DeviceVM)推出一项服务(可以免费,也可收费),为用户提供现成的Windows应用,而用户所需要做的就是在iPad或gPad上安装Splashtop Remote,打开这个应用就可以自动连接到DeviceVM的Windows
PC/Server,利用现成的Windows程序了——省去了自己购买并维护PC的很多麻烦,节省了购买大量Windows软件的开支,何乐而不为呢?而且这也正契合了时下的云计算潮流。
"

果然英雄所见略同,已经有人开始提供这样的服务了。不过OnLive能否流行开来,还有待用户和市场的检验。使用体验是否可以被用户接受?如何解决网速(无论有线或无线)慢的情况? 微软会不会推出iPad版的Office软件?即使该服务的终极目标不仅仅限于Windows或者Office软件,其他只能能在PC运行的程序会不会有替代解决方案在iPad或者Android平板上?以及现存的很多类VNC的软件……凡此种种,都构成了该服务的不确定因素。

无论如何,这都是一个很好的尝试。

Updating Fedora to Fedora16 and Installing Ubuntu 11.04

2.5years ago I installed Fedora11 on my Thinkpad R61,and then updated it to Fedora 12 and then Fedora 14. A bunch of software were installed, and as a result, the system went slower and slower... 固然有我个人懒惰不愿优化的原因,系统性能下降也与两年多来Fedora的复杂程度的增长有关。前天晚上禁不住诱惑升级到了Fedora 16,过程中没遇到问题,只是升级后发现:(1)Gnome 3的界面进不去,因为笔记本配置太低,没有独立显卡;(2)Classic Gnome with compiz也基本无法使用,尽管我已经装好了compiz;(3)KDE界面虽然花哨但速度较慢不甚喜欢;(4)无线网卡经常无法工作。
基于以上缺陷,我终于忍无可忍,决定更换系统到Ubuntu或者FreeBSD了。本来我更倾向于FreeBSD的,但是由于下载速度缓慢,不得不暂时放弃。
Ubuntu很快就下完。首先在Fedora下创建启动U盘,参照 How to create and use Live USB:
Check the size of your USB stick
Many USB sticks indicate the size on the packaging or the outside of the stick.

If you don't know the size of the stick, or want to check it for data, you should be able to auto-mount the USB stick by inserting it into a USB port. You can check the contents and size using the graphical file manager. In Linux, you can also use the command line:

$ df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/VolGroup00-LogVol00
143G 14G 122G 10% /
/dev/sda1 99M 12M 82M 13% /boot
tmpfs 1009M 0 1009M 0% /dev/shm
/dev/sdb1 3.9G 4.0K 3.9G 1% /media/usbdisk
USB drives are usually mounted in /media. In this case, the device is /dev/sdb1, has a 3.9GB capacity and is almost empty.

Take note of "/dev/sdb1" or equivalent; you will be specifying the device name if you use the command line method.

How to Partition

CAUTION
This will erase all data on the USB drive! Please read the instructions below carefully.
If the drive has not been partitioned properly (or if you are unsure), use fdisk to repartition it.

It is also possible to do a non-destructive installation of a LiveUSB image, if you have sufficient empty space. See How to install non-destructively below.

The fdisk command must be run as root. Include only the drive name in the command, not the partition number. Be sure to select the correct disk, or you may erase important data! Check the output of "df -h" if you are unsure. For example, if your partition will be /dev/sdb1, do:

$ /sbin/fdisk /dev/sdb
If you don't have fdisk installed, run

yum install util-linux-ng
as root.
The following session output from fdisk shows the responses to give to the prompts. The line starting Last cylinder ... refers to the size of the flash drive, so may be different than in the example.

Command (m for help): d
Selected partition 1

Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-960, default 1): ↵
Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-960, default 960): ↵
Using default value 960

Command (m for help): t
Selected partition 1
Hex code (type L to list codes): 6
Changed system type of partition 1 to 6 (FAT16)

Command (m for help): a
Partition number (1-4): 1

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.

WARNING: If you have created or modified any DOS 6.x
partitions, please see the fdisk manual page for additional
information.
Syncing disks.
How to Format

CAUTION
This will erase all data on the USB drive! Please read the instructions below carefully.
If your USB media has sufficient free space on a vfat file system already, you do not need to perform this step.

To finish, the partition must be formatted with an actual file system using mkdosfs as the root user. Unmount the device before using mkdosfs. In the below example, /dev/USBPARTITIONNAME might be, for example, /dev/sdb1. Be sure to select the correct partition; formatting destroys all data on it!

$ umount /dev/USBPARTITIONNAME
$ /sbin/mkdosfs -F 32 -n usbdisk /dev/USBPARTITIONNAME
If you don't have mkdosfs installed, run "yum install dosfstools" as root.

通过命令
su -c "yum install liveusb-creator"
安装了Fedora的liveusb-creator,但是发现Fedora的这个程序无法制作Ubuntu的启动盘。无奈之下,试着把Ubuntu的iso文件复制到U盘里:
sudo dd if=F12-Live-i686.iso of=/dev/sdX bs=8M

之后重启电脑,开机之后按c进入grub引导程序。尝试以往的通过kernel指定vmlinuz时失败,才发现Grub2中已经没有了kernel命令,尝试legacy_kernel和xnu_kernel,结果均遭失败。于是我尝试直接指定 initrd (hd1,4)/casper/initrd.gz,然后exit,没成想Ubuntu的安装界面竟然出现了!
安装Ubuntu的过程中竟然就已经支持了WIFI,着实令我欣慰,看来Ubuntu这么多年在用户友好性上改进不少。
安装完成后重启,结果Grub无法正常工作,进入grub rescue模式。用iPad百度之后,发现好文Ubuntu启动问题以及Grub Rescue修复方法,逐步去试,果然灵验。转载该文如下:
Ubuntu启动问题以及Grub Rescue修复方法

问题:
之前系统是Windows7 64bit(C盘) + D,E盘(都是NTFS) + Ubuntu。
今天,在Windows7中删除了E盘,准备向其中安装RedHat 6。 重启Windows7之后发现Ubuntu的grub菜单不能启动。屏幕显示Invalid FileSystem。 (如果没有记错的话,呵呵)
分析
由于在Windows下面更改了分区状态,删掉了Ubuntu之前的一个分区,造成Ubuntu的分区由sda5变为sda4了,这样找不到grub了,开机显示
grub rescue>
解决
1. 先使用ls命令,找到Ubuntu的安装在哪个分区:
grub rescue>ls
会罗列所有的磁盘分区信息,比方说:
(hd0,1),(hd0,5),(hd0,3),(hd0,2)
2. 然后依次调用如下命令: X表示各个分区号码
grub rescue>ls (hd0,X)/boot/grub
如果都找不到的话,需要查一下是否因为Linux版本差异,造成grub的路径不对,例如直接ls(hd0,X)/grub等等。
3. 假设找到(hd0,5)时,显示了文件夹中的文件,则表示Linux安装在这个分区。
4. 调用如下命令:
grub rescue>set root=(hd0,5)
grub rescue>set prefix=(hd0,5)/boot/grub
grub rescue>insmod /boot/grub/normal.mod
5. 然后调用如下命令,就可以显示出丢失的grub菜单了。
grub rescue>normal
6. 不过不要高兴,如果这时重启,问题依旧存在,我们需要进入Linux中,对grub进行修复。
进入Linux之后,在命令行执行:
sudo update-grub
sudo grub-install /dev/sda
(sda是你的硬盘号码,千万不要指定分区号码,例如sda1,sda5等都不对)
7. 重启测试是否已经恢复了grub的启动菜单? 恭喜你恢复成功!

Grub恢复成功后,开始安装常用软件。Ubuntu新的界面不像以前想象中那么难用,而且他们终于部分放弃了以前屎黄色的界面风格,最难得的是在我那破本本上速度还是很快滴!期待Ubuntu12.04早日发布,到时直接升级之。

升级Fedora 16 与安装Ubuntu 11.04

两年半前安装了Fedora11,之后升级到Fedora 12, Fedora 14,期间安装了大量的软件,系统越来越慢……固然有我个人懒惰不愿优化的原因,系统性能下降也与两年多来Fedora的复杂程度的增长有关。前天晚上禁不住诱惑升级到了Fedora 16,过程中没遇到问题,只是升级后发现:(1)Gnome 3的界面进不去,因为笔记本配置太低,没有独立显卡;(2)Classic Gnome with compiz也基本无法使用,尽管我已经装好了compiz;(3)KDE界面虽然花哨但速度较慢不甚喜欢;(4)无线网卡经常无法工作。

基于以上缺陷,我终于忍无可忍,决定更换系统到Ubuntu或者FreeBSD了。本来我更倾向于FreeBSD的,但是由于下载速度缓慢,不得不暂时放弃。

Ubuntu很快就下完。首先在Fedora下创建启动U盘,参照 How to create and use Live USB


  1. Check the size of your USB stick

    Many USB sticks indicate the size on the packaging or the outside of the stick.

    If you don't know the size of the stick, or want to check it for data, you should be able to auto-mount the USB stick by inserting it into a USB port. You can check the contents and size using the graphical file manager. In Linux, you can also use the command
    line:

    $ df -h
    Filesystem            Size  Used Avail Use% Mounted on
    /dev/mapper/VolGroup00-LogVol00
    143G   14G  122G  10% /
    /dev/sda1              99M   12M   82M  13% /boot
    tmpfs                1009M     0 1009M   0% /dev/shm
    /dev/sdb1             3.9G  4.0K  3.9G   1% /media/usbdisk

    USB drives are usually mounted in /media. In this case, the device is /dev/sdb1, has a 3.9GB capacity and is almost empty.

    Take note of "/dev/sdb1" or equivalent; you will be specifying the device name if you use the command line method.

  2. How to Partition

    Warning (medium size).png
    CAUTION 

    This will erase all data on the USB drive! Please read the instructions below carefully.

    If the drive has not been partitioned properly (or if you are unsure), use fdisk to
    repartition it.

    It is also possible to do a non-destructive installation of a LiveUSB image, if you have sufficient empty space. See How
    to install non-destructively
     below.

    The fdisk command must be run as root. Include
    only the drive name in the command, not the partition number. Be
    sure to select the correct disk, or you may erase important data!
     Check the output of "df -h" if you are unsure. For example, if your partition will be /dev/sdb1, do:

    $ /sbin/fdisk /dev/sdb

    If you don't have fdisk installed, run 

    yum install util-linux-ng

     as root.

    The following session output from fdisk shows
    the responses to give to the prompts. The line starting Last
    cylinder ...
     refers to the size of the flash drive, so may be different than in the example.

    Command (m for help): d
    Selected partition 1
    
    
                
              

Backquotes in Ksh here document

When the shell encounters a string between backquotes

     `cmd`

it executes cmd and replaces the backquoted string with the standard output of cmd, with any trailing newlines deleted. Quoting inside backquoted commands is somewhat complicated, mainy because the same token is used to start and to end a backquoted command. As a consequence, to nest backquoted commands, the backquotes of the inner one have to be escaped using backslashes, e.g.

LABNAME=$(echo $labname | tr "[a-z]" "[A-Z]" | sed 's/-/_/g'
cat <<-IOTA >$iota_cfg
path=\`checkpath\`
export ${LABNAME}_COOLXDIR=$COOLXDIR
IOTA

Since the LABNAME is a variable defined out of the here document, it must be quoted by braces {}, or error will occur.

--------------

Reference:

  1.  A Guide to Unix Shell Quoting

Merge/Split PDF files in Linux

First install pdftk (pdf toolkit). Pdftk runs from the command line which means you should unpack the software into a directory that you can access easily. I put the file into c: for testing purposes
along with two pdfs called 123.pdf and 456.pdf.

If you run pdftk without parameters you get a quick output that details the most important parameters. Lets take a look at the possible operations:

Merge
pdf files:

pdftk
123.pdf 456.pdf output 123456.pdf

This tells pdftk to merge the two pdf files 123.pdf and 456.pdf into a new pdf called 123456.pdf. If you want to merge more than two pdf files at once you simply add them between pdftk and output. You could also use wildcards to merge many or all pdfs with
a certain name.

pdftk
*.pdf output 123456.pdf

Split
pages from one or more pdfs:

pdftk
A=123.pdf B=456.pdf cat A1-2 B4-5 output 1245.pdf

Repair
a pdf:

pdftk
123.pdf output 123fixed.pdf

Merge
encrypted pdfs:

pdftk
A=123.pdf 456.pdf input_pw A=password cat output 123456.pdf

Pdftk offers several more possibilities which are worth a look if you want to do something that I did not describe on this website. A version for Linux and Windows is available.

Read
More:

Pdftk
Homepage

原文:http://www.ghacks.net/2007/10/02/merge-pdf-files/

Tk send

send is a Tk command used to execute a command in a different application. This send communication mechanism is limited to applications running on one display. Multiple screens on one workstation still count as the same display on X.

In order to use send in your program, you should

    1. use command winfo interps to find out all the names of Tk applications reachable with send.
    2. to get the real name of a spplicaiton, using tk appname <application>.
    3. run tcl command remotely by send [tk appname <application>] <command>.

------------------
Reference:

  1. send manual page: http://www.tcl.tk/man/tcl8.4/TkCmd/send.htm
  2. Practical Programming in Tcl and Tk(4th edition)

How to change the character case in ksh?

1. To change lower case characters to upper case, you can

echo abc | tr "[a-z]" "[A-Z]"

or 

echo abc | tr '[:lower:]' '[:upper:]'

2. To change upper case characters to lower case, you can

echo ABC | tr "[A-Z]" "[a-z]"

or

echo ABC | tr '[:upper:]' '[:lower:]'