Introduction
This document explains how to set your computer's clock from Linux, how to set your timezone, and other stuff related to Linux and how it does its time-keeping.
Your computer has two timepieces; a battery-backed one that is always running (the ``hardware'', ``BIOS'', or ``CMOS'' clock), and another that is maintained by the operating system currently running on your computer (the ``system'' clock). The hardware clock is generally only used to set the system clock when your operating system boots, and then from that point until you reboot or turn off your system, the system clock is the one used to keep track of time.
On Linux systems, you have a choice of keeping the hardware clock in UTC/GMT time or local time. The preferred option is to keep it in UTC because then daylight savings can be automatically accounted for. The only disadvantage with keeping the hardware clock in UTC is that if you dual boot with an operating system (such as DOS) that expects the hardware clock to be set to local time, the time will always be wrong in that operating system.
Setting your timezone
The timezone under Linux is set by a symbolic link from /etc/localtime[1] to a file in the /usr/share/zoneinfo[2] directory that corresponds with what timezone you are in. For example, since I'm in South Australia, /etc/localtime is a symlink /usr/share/zoneinfo/Australia/South. To set this link, type:
ln -sf ../usr/share/zoneinfo/your/zone /etc/localtime
Replace your/zone with something like Australia/NSW or Australia/Perth. Have a look in the directories under /usr/share/zoneinfo to see what timezones are available.
[1] This assumes that /usr/share/zoneinfo is linked to /etc/localtime as it is under Red Hat Linux.
[2] On older systems, you'll find that /usr/lib/zoneinfo is used instead of /usr/share/zoneinfo. See also the later section ``The time in some applications is wrong''.
Setting UTC or local time
When Linux boots, one of the initialisation scripts will run the /sbin/hwclock program to copy the current hardware clock time to the system clock. hwclock will assume the hardware clock is set to local time unless it is run with the --utc switch. Rather than editing the startup script, under Red Hat Linux you should edit the /etc/sysconfig/clock file and change the ``UTC'' line to either ``UTC=true'' or ``UTC=false'' as appropriate.
Setting the system clock
To set the system clock under Linux, use the date command. As an example, to set the current time and date to July 31, 11:16pm, type ``date 07312316'' (note that the time is given in 24 hour notation). If you wanted to change the year as well, you could type ``date 073123161998''. To set the seconds as well, type ``date 07312316.30'' or ``date 073123161998.30''. To see what Linux thinks the current local time is, run date with no arguments.
Setting the hardware clock
To set the hardware clock, my favourite way is to set the system clock first, and then set the hardware clock to the current system clock by typing ``/sbin/hwclock --systohc'' (or ``/sbin/hwclock --systohc --utc'' if you are keeping the hardware clock in UTC). To see what the hardware clock is currently set to, run hwclock with no arguments. If the hardware clock is in UTC and you want to see the local equivalent, type ``/sbin/hwclock --utc''
The time in some applications is wrong
If some applications (such as date) display the correct time, but others don't, and you are running Red Hat Linux 5.0 or 5.1, you most likely have run into a bug caused by a move of the timezone information from /usr/lib/zoneinfo to /usr/share/zoneinfo. The fix is to create a symbolic link from /usr/lib/zoneinfo to /usr/share/zoneinfo: ``ln -s ../share/zoneinfo /usr/lib/zoneinfo''.
Summary
/etc/sysconfig/clock sets whether the hardware clock is stored as UTC or local time.
Symlink /etc/localtime to /usr/share/zoneinfo/... to set your timezone.
Run ``date MMDDhhmm'' to set the current system date/time.
Type ``/sbin/hwclock --systohc [--utc]'' to set the hardware clock.
Other interesting notes
The Linux kernel always stores and calculates time as the number of seconds since midnight of the 1st of January 1970 UTC regardless of whether your hardware clock is stored as UTC or not. Conversions to your local time are done at run-time. One neat thing about this is that if someone is using your computer from a different timezone, they can set the TZ environment variable and all dates and times will appear correct for their timezone.
If the number of seconds since the 1st of January 1970 UTC is stored as an signed 32-bit integer (as it is on your Linux/Intel system), your clock will stop working sometime on the year 2038. Linux has no inherent Y2K problem, but it does have a year 2038 problem. Hopefully we'll all be running Linux on 64-bit systems by then. 64-bit integers will keep our clocks running quite well until aproximately the year 292271-million.
Other programs worth looking at
rdate - get the current time from a remote machine; can be used to set the system time.
xntpd - like rdate, but it's extremely accurate and you need a permanent 'net connection. xntpd runs continuously and accounts for things like network delay and clock drift, but there's also a program (ntpdate) included that just sets the current time like rdate does.
Saturday, September 29, 2007
Tips And Tricks in Linux Administration
This is a compilation of tips and tricks I got from reading the mailing list from CentOS and PLUG which I will update from time to time.
1. How to get users list in a linux machine.
As local user
# cat /etc/passwd
- Seperated by username, password, UID, GID, home directory, shell
# cat /etc/group
- Use this to see group information.
getent passwd
- This handles everything nsswitch.conf does (NIS, LDAP, xyzzy, whatever).
getent passwd cut -d: -f1 sort -u
- To get just a list of usernames
2. To check what is the IP address of the DHCP server where assigned IP leased from:
For CentOS 5, the command:
# grep dhcp-server-identifier /var/lib/dhclient/dhclient-eth0.leases
or for CentOS 4, the command:
# grep dhcp-server-identifier /var/lib/dhcp/dhclient-eth0.leases
To list all PCI device:
/sbin/lspci
To determine the change log of installed software:
rpm -qi –changelog
yum install yum-changelog
Then …
man yum-changelog
or
repoquery –changelog foo
3. Robert Moskowitz Tips on Private Network IP Addresses:
If you use addresses from the 192.168 range, avoid the nets:
192.168.0.0, 192.168.1.0, 192.168.2.0, and 192.168.100.0 These are commonly used by various ‘appliances’. Choose something ‘wierd’ like 192.168.49.0 (or some other prime number less than 255). Though I tend to use 192.168.64.0, 192.168.96.0, or 192.168.128.0
Also avoid the 172.16 and 10. ranges. These are commonly used in Company internal private networks, and if you want to ‘VPN’ to them, it an cause unwanted results (like it won’t work).
This is general advice to anyone using private addresses for small usages.
4. To Get Memory Usage and Information:
free - for summary information
cat /proc/meminfo - for detailed information
5. To get hardware info:
/usr/sbin/dmidecode
6. To test for disk I/O performance:
Download ‘disktest’ utility from Linux Test Project(LTP) and do the ff:
Sequential reads:
disktest -B 4k -h 1 -I BD -K 4 -p l -P T -T 300 -r /dev/sdX
Sequential writes:
disktest -B 4k -h 1 -I BD -K 4 -p l -P T -T 300 -w /dev/sdX
Random writes:
disktest -B 4k -h 1 -I BD -K 4 -p r -P T -T 300 -r /dev/sdX
Random writes:
disktest -B 4k -h 1 -I BD -K 4 -p r -P T -T 300 -w /dev/sdX
Description of the options used:
-B 4k = 4k block ios
-h 1 = 1 second heartbeat
-I BD = block device, direct io
-K 4 = 4 threads, or 4 outstanding/overlapping ios, typical pattern
(use -K 1 for the raw performance of single drive, aka dd type output)
-p= io type, l=linear, r=random
-P T = output metrics type “Throughput”
-T 300 = duration of test 300 seconds
-r = read
-w = write
These tests will run across the whole disk/partition and the write
tests WILL BE DESTRUCTIVE so be warned!
7. To determine PCI information and replacing LAN card info:
/sbin/lspci
In /etc/modprobe.conf
Put or replace alias entries for LAN card and restart network.
8. To get driver info:
/sbin/modinfo 3w-9xxx
1. How to get users list in a linux machine.
As local user
# cat /etc/passwd
- Seperated by username, password, UID, GID, home directory, shell
# cat /etc/group
- Use this to see group information.
getent passwd
- This handles everything nsswitch.conf does (NIS, LDAP, xyzzy, whatever).
getent passwd cut -d: -f1 sort -u
- To get just a list of usernames
2. To check what is the IP address of the DHCP server where assigned IP leased from:
For CentOS 5, the command:
# grep dhcp-server-identifier /var/lib/dhclient/dhclient-eth0.leases
or for CentOS 4, the command:
# grep dhcp-server-identifier /var/lib/dhcp/dhclient-eth0.leases
To list all PCI device:
/sbin/lspci
To determine the change log of installed software:
rpm -qi –changelog
yum install yum-changelog
Then …
man yum-changelog
or
repoquery –changelog foo
3. Robert Moskowitz Tips on Private Network IP Addresses:
If you use addresses from the 192.168 range, avoid the nets:
192.168.0.0, 192.168.1.0, 192.168.2.0, and 192.168.100.0 These are commonly used by various ‘appliances’. Choose something ‘wierd’ like 192.168.49.0 (or some other prime number less than 255). Though I tend to use 192.168.64.0, 192.168.96.0, or 192.168.128.0
Also avoid the 172.16 and 10. ranges. These are commonly used in Company internal private networks, and if you want to ‘VPN’ to them, it an cause unwanted results (like it won’t work).
This is general advice to anyone using private addresses for small usages.
4. To Get Memory Usage and Information:
free - for summary information
cat /proc/meminfo - for detailed information
5. To get hardware info:
/usr/sbin/dmidecode
6. To test for disk I/O performance:
Download ‘disktest’ utility from Linux Test Project(LTP) and do the ff:
Sequential reads:
disktest -B 4k -h 1 -I BD -K 4 -p l -P T -T 300 -r /dev/sdX
Sequential writes:
disktest -B 4k -h 1 -I BD -K 4 -p l -P T -T 300 -w /dev/sdX
Random writes:
disktest -B 4k -h 1 -I BD -K 4 -p r -P T -T 300 -r /dev/sdX
Random writes:
disktest -B 4k -h 1 -I BD -K 4 -p r -P T -T 300 -w /dev/sdX
Description of the options used:
-B 4k = 4k block ios
-h 1 = 1 second heartbeat
-I BD = block device, direct io
-K 4 = 4 threads, or 4 outstanding/overlapping ios, typical pattern
(use -K 1 for the raw performance of single drive, aka dd type output)
-p
-P T = output metrics type “Throughput”
-T 300 = duration of test 300 seconds
-r = read
-w = write
These tests will run across the whole disk/partition and the write
tests WILL BE DESTRUCTIVE so be warned!
7. To determine PCI information and replacing LAN card info:
/sbin/lspci
In /etc/modprobe.conf
Put or replace alias entries for LAN card and restart network.
8. To get driver info:
/sbin/modinfo 3w-9xxx
Summary of VI Editor commands
This list is a summary of VI commands, categorized by function. There may be other commands available, so check the on-line manual on VI. For easy reference, you can save this file as text and delete any commands you don't think you would use and print out the resulting shorter file.
Cutting and Pasting/Deleting text
"
Specify a buffer to be used any of the commands using buffers. Follow the " with a letter or a number, which corresponds to a buffer.
D
Delete to the end of the line from the current cursor position.
P
Paste the specified buffer before the current cursor position or line. If no buffer is specified (with the " command.) then 'P' uses the general buffer.
X
Delete the character before the cursor.
Y
Yank the current line into the specified buffer. If no buffer is specified, then the general buffer is used.
d
Delete until where. "dd" deletes the current line. A count deletes that many lines. Whatever is deleted is placed into the buffer specified with the " command. If no buffer is specified, then the general buffer is used.
p
Paste the specified buffer after the current cursor position or line. If no buffer is specified (with the " command.) then 'p' uses the general buffer.
x
Delete character under the cursor. A count tells how many characters to delete. The characters will be deleted after the cursor.
y
Yank until , putting the result into a buffer. "yy" yanks the current line. a count yanks that many lines. The buffer can be specified with the " command. If no buffer is specified, then the general buffer is used.
Inserting New Text
A
Append at the end of the current line.
I
Insert from the beginning of a line.
O
(letter oh) Enter insert mode in a new line above the current cursor position.
a
Enter insert mode, the characters typed in will be inserted after the current cursor position. A count inserts all the text that had been inserted that many times.
i
Enter insert mode, the characters typed in will be inserted before the current cursor position. A count inserts all the text that had been inserted that many times.
o
Enter insert mode in a new line below the current cursor position.
Moving the Cursor Within the File
^B
Scroll backwards one page. A count scrolls that many pages.
^D
Scroll forwards half a window. A count scrolls that many lines.
^F
Scroll forwards one page. A count scrolls that many pages.
^H
Move the cursor one space to the left. A count moves that many spaces.
^J
Move the cursor down one line in the same column. A count moves that many lines down.
^M
Move to the first character on the next line.
^N
Move the cursor down one line in the same column. A count moves that many lines down.
^P
Move the cursor up one line in the same column. A count moves that many lines up.
^U
Scroll backwards half a window. A count scrolls that many lines.
$
Move the cursor to the end of the current line. A count moves to the end of the following lines.
%
Move the cursor to the matching parenthesis or brace.
^
Move the cursor to the first non-whitespace character.
(
Move the cursor to the beginning of a sentence.
)
Move the cursor to the beginning of the next sentence.
{
Move the cursor to the preceding paragraph.
}
Move the cursor to the next paragraph.
Move the cursor to the column specified by the count.
+
Move the cursor to the first non-whitespace character in the next line.
-
Move the cursor to the first non-whitespace character in the previous line.
_
Move the cursor to the first non-whitespace character in the current line.
0
(Zero) Move the cursor to the first column of the current line.
B
Move the cursor back one word, skipping over punctuation.
E
Move forward to the end of a word, skipping over punctuation.
G
Go to the line number specified as the count. If no count is given, then go to the end of the file.
H
Move the cursor to the first non-whitespace character on the top of the screen.
L
Move the cursor to the first non-whitespace character on the bottom of the screen.
M
Move the cursor to the first non-whitespace character on the middle of the screen.
W
Move forward to the beginning of a word, skipping over punctuation.
b
Move the cursor back one word. If the cursor is in the middle of a word, move the cursor to the first character of that word.
e
Move the cursor forward one word. If the cursor is in the middle of a word, move the cursor to the last character of that word.
h
Move the cursor to the left one character position.
j
Move the cursor down one line.
k
Move the cursor up one line.
l
Move the cursor to the right one character position.
w
Move the cursor forward one word. If the cursor is in the middle of a word, move the cursor to the first character of the next word.
Moving the Cursor Around the Screen
^E
Scroll forwards one line. A count scrolls that many lines.
^Y
Scroll backwards one line. A count scrolls that many lines.
z
Redraw the screen with the following options. "z" puts the current line on the top of the screen; "z." puts the current line on the center of the screen; and "z-" puts the current line on the bottom of the screen. If you specify a count before the 'z' command, it changes the current line to the line specified. For example, "16z." puts line 16 on the center of the screen.
Replacing Text
C
Change to the end of the line from the current cursor position.
R
Replace characters on the screen with a set of characters entered, ending with the Escape key.
S
Change an entire line.
c
Change until . "cc" changes the current line. A count changes that many lines.
r
Replace one character under the cursor. Specify a count to replace a number of characters.
s
Substitute one character under the cursor, and go into insert mode. Specify a count to substitute a number of characters. A dollar sign ($) will be put at the last character to be substituted.
Searching for Text or Characters
,
Repeat the last f, F, t or T command in the reverse direction.
/
Search the file downwards for the string specified after the /.
;
Repeat the last f, F, t or T command.
?
Search the file upwards for the string specified after the ?.
F
Search the current line backwards for the character specified after the 'F' command. If found, move the cursor to the position.
N
Repeat the last search given by '/' or '?', except in the reverse direction.
T
Search the current line backwards for the character specified after the 'T' command, and move to the column after the if it's found.
f
Search the current line for the character specified after the 'f' command. If found, move the cursor to the position.
n
Repeat last search given by '/' or '?'.
t
Search the current line for the character specified after the 't' command, and move to the column before the character if it's found.
Manipulating Character/Line Formatting
~
Switch the case of the character under the cursor.
<>
Shift the lines up to where to the right by one shiftwidth.
">>"
shifts the current line to the right, and can be specified with a count.
J
Join the current line with the next one. A count joins that many lines.
Saving and Quitting
^\
Quit out of "VI" mode and go into "EX" mode. The EX editor is the line editor VI is build upon. The EX command to get back into VI is ":vi".
Q
Quit out of "VI" mode and go into "EX" mode. The ex editor is a line-by-line editor. The EX command to get back into VI is ":vi".
ZZ
Exit the editor, saving if any changes were made.
Miscellany
^G
Show the current filename and the status.
^L
Clear and redraw the screen.
^R
Redraw the screen removing false lines.
^[
Escape key. Cancels partially formed command.
^^
Go back to the last file edited.
!
Execute a shell. If a is specified, the program which is executed using ! uses the specified line(s) as standard input, and will replace those lines with the standard output of the program executed. "!!" executes a program using the current line as input. For example, "!4jsort" will take five lines from the current cursor position and execute sort. After typing the command, there will be a single exclamation point where you can type the command in.
&
Repeat the previous ":s" command.
.
Repeat the last command that modified the file.
:
Begin typing an EX editor command. The command is executed once the user types return. (See section below.)
@
Type the command stored in the specified buffer.
U
Restore the current line to the state it was in before the cursor entered the line.
m
Mark the current position with the character specified after the 'm' command.
u
Undo the last change to the file. Typing 'u' again will re-do the change.
EX Commands
The VI editor is built upon another editor, called EX. The EX editor only edits by line. From the VI editor you use the : command to start entering an EX command. This list given here is not complete, but the commands given are the more commonly used. If more than one line is to be modified by certain commands (such as ":s" and ":w" ) the range must be specified before the command. For example, to substitute lines 3 through 15, the command is ":3,15s/from/this/g".
:ab string strings
Abbreviation. If a word is typed in VI corresponding to string1, the editor automatically inserts the corresponding words. For example, the abbreviation ":ab usa United States of America" would insert the words, "United States of America" whenever the word "usa" is typed in.
:map keys new_seq
Mapping. This lets you map a key or a sequence of keys to another key or a sequence of keys.
:q
Quit VI. If there have been changes made, the editor will issue a warning message.
:q!
Quit VI without saving changes.
:s/pattern/to_pattern/options
Substitute. This substitutes the specified pattern with the string in the to_pattern. Without options, it only substitutes the first occurence of the pattern. If a 'g' is specified, then all occurences are substituted. For example, the command ":1,$s/Dwayne/Dwight/g" substitutes all occurences of "Dwayne" to "Dwight".
:set [all]
Sets some customizing options to VI and EX. The ":set all" command gives all the possible options. (See the section on customizing VI for some options.)
:una string
Removes the abbreviation previously defined by ":ab".
:unm keys
Removes the remove mapping defined by ":map".
:vi filename
Starts editing a new file. If changes have not been saved, the editor will give you a warning.
:w
Write out the current file.
:w filename
Write the buffer to the filename specified.
:w >> filename
Append the contents of the buffer to the filename.
:wq
Write the buffer and quit.
Cutting and Pasting/Deleting text
"
Specify a buffer to be used any of the commands using buffers. Follow the " with a letter or a number, which corresponds to a buffer.
D
Delete to the end of the line from the current cursor position.
P
Paste the specified buffer before the current cursor position or line. If no buffer is specified (with the " command.) then 'P' uses the general buffer.
X
Delete the character before the cursor.
Y
Yank the current line into the specified buffer. If no buffer is specified, then the general buffer is used.
d
Delete until where. "dd" deletes the current line. A count deletes that many lines. Whatever is deleted is placed into the buffer specified with the " command. If no buffer is specified, then the general buffer is used.
p
Paste the specified buffer after the current cursor position or line. If no buffer is specified (with the " command.) then 'p' uses the general buffer.
x
Delete character under the cursor. A count tells how many characters to delete. The characters will be deleted after the cursor.
y
Yank until , putting the result into a buffer. "yy" yanks the current line. a count yanks that many lines. The buffer can be specified with the " command. If no buffer is specified, then the general buffer is used.
Inserting New Text
A
Append at the end of the current line.
I
Insert from the beginning of a line.
O
(letter oh) Enter insert mode in a new line above the current cursor position.
a
Enter insert mode, the characters typed in will be inserted after the current cursor position. A count inserts all the text that had been inserted that many times.
i
Enter insert mode, the characters typed in will be inserted before the current cursor position. A count inserts all the text that had been inserted that many times.
o
Enter insert mode in a new line below the current cursor position.
Moving the Cursor Within the File
^B
Scroll backwards one page. A count scrolls that many pages.
^D
Scroll forwards half a window. A count scrolls that many lines.
^F
Scroll forwards one page. A count scrolls that many pages.
^H
Move the cursor one space to the left. A count moves that many spaces.
^J
Move the cursor down one line in the same column. A count moves that many lines down.
^M
Move to the first character on the next line.
^N
Move the cursor down one line in the same column. A count moves that many lines down.
^P
Move the cursor up one line in the same column. A count moves that many lines up.
^U
Scroll backwards half a window. A count scrolls that many lines.
$
Move the cursor to the end of the current line. A count moves to the end of the following lines.
%
Move the cursor to the matching parenthesis or brace.
^
Move the cursor to the first non-whitespace character.
(
Move the cursor to the beginning of a sentence.
)
Move the cursor to the beginning of the next sentence.
{
Move the cursor to the preceding paragraph.
}
Move the cursor to the next paragraph.
Move the cursor to the column specified by the count.
+
Move the cursor to the first non-whitespace character in the next line.
-
Move the cursor to the first non-whitespace character in the previous line.
_
Move the cursor to the first non-whitespace character in the current line.
0
(Zero) Move the cursor to the first column of the current line.
B
Move the cursor back one word, skipping over punctuation.
E
Move forward to the end of a word, skipping over punctuation.
G
Go to the line number specified as the count. If no count is given, then go to the end of the file.
H
Move the cursor to the first non-whitespace character on the top of the screen.
L
Move the cursor to the first non-whitespace character on the bottom of the screen.
M
Move the cursor to the first non-whitespace character on the middle of the screen.
W
Move forward to the beginning of a word, skipping over punctuation.
b
Move the cursor back one word. If the cursor is in the middle of a word, move the cursor to the first character of that word.
e
Move the cursor forward one word. If the cursor is in the middle of a word, move the cursor to the last character of that word.
h
Move the cursor to the left one character position.
j
Move the cursor down one line.
k
Move the cursor up one line.
l
Move the cursor to the right one character position.
w
Move the cursor forward one word. If the cursor is in the middle of a word, move the cursor to the first character of the next word.
Moving the Cursor Around the Screen
^E
Scroll forwards one line. A count scrolls that many lines.
^Y
Scroll backwards one line. A count scrolls that many lines.
z
Redraw the screen with the following options. "z
Replacing Text
C
Change to the end of the line from the current cursor position.
R
Replace characters on the screen with a set of characters entered, ending with the Escape key.
S
Change an entire line.
c
Change until . "cc" changes the current line. A count changes that many lines.
r
Replace one character under the cursor. Specify a count to replace a number of characters.
s
Substitute one character under the cursor, and go into insert mode. Specify a count to substitute a number of characters. A dollar sign ($) will be put at the last character to be substituted.
Searching for Text or Characters
,
Repeat the last f, F, t or T command in the reverse direction.
/
Search the file downwards for the string specified after the /.
;
Repeat the last f, F, t or T command.
?
Search the file upwards for the string specified after the ?.
F
Search the current line backwards for the character specified after the 'F' command. If found, move the cursor to the position.
N
Repeat the last search given by '/' or '?', except in the reverse direction.
T
Search the current line backwards for the character specified after the 'T' command, and move to the column after the if it's found.
f
Search the current line for the character specified after the 'f' command. If found, move the cursor to the position.
n
Repeat last search given by '/' or '?'.
t
Search the current line for the character specified after the 't' command, and move to the column before the character if it's found.
Manipulating Character/Line Formatting
~
Switch the case of the character under the cursor.
<>
Shift the lines up to where to the right by one shiftwidth.
">>"
shifts the current line to the right, and can be specified with a count.
J
Join the current line with the next one. A count joins that many lines.
Saving and Quitting
^\
Quit out of "VI" mode and go into "EX" mode. The EX editor is the line editor VI is build upon. The EX command to get back into VI is ":vi".
Q
Quit out of "VI" mode and go into "EX" mode. The ex editor is a line-by-line editor. The EX command to get back into VI is ":vi".
ZZ
Exit the editor, saving if any changes were made.
Miscellany
^G
Show the current filename and the status.
^L
Clear and redraw the screen.
^R
Redraw the screen removing false lines.
^[
Escape key. Cancels partially formed command.
^^
Go back to the last file edited.
!
Execute a shell. If a is specified, the program which is executed using ! uses the specified line(s) as standard input, and will replace those lines with the standard output of the program executed. "!!" executes a program using the current line as input. For example, "!4jsort" will take five lines from the current cursor position and execute sort. After typing the command, there will be a single exclamation point where you can type the command in.
&
Repeat the previous ":s" command.
.
Repeat the last command that modified the file.
:
Begin typing an EX editor command. The command is executed once the user types return. (See section below.)
@
Type the command stored in the specified buffer.
U
Restore the current line to the state it was in before the cursor entered the line.
m
Mark the current position with the character specified after the 'm' command.
u
Undo the last change to the file. Typing 'u' again will re-do the change.
EX Commands
The VI editor is built upon another editor, called EX. The EX editor only edits by line. From the VI editor you use the : command to start entering an EX command. This list given here is not complete, but the commands given are the more commonly used. If more than one line is to be modified by certain commands (such as ":s" and ":w" ) the range must be specified before the command. For example, to substitute lines 3 through 15, the command is ":3,15s/from/this/g".
:ab string strings
Abbreviation. If a word is typed in VI corresponding to string1, the editor automatically inserts the corresponding words. For example, the abbreviation ":ab usa United States of America" would insert the words, "United States of America" whenever the word "usa" is typed in.
:map keys new_seq
Mapping. This lets you map a key or a sequence of keys to another key or a sequence of keys.
:q
Quit VI. If there have been changes made, the editor will issue a warning message.
:q!
Quit VI without saving changes.
:s/pattern/to_pattern/options
Substitute. This substitutes the specified pattern with the string in the to_pattern. Without options, it only substitutes the first occurence of the pattern. If a 'g' is specified, then all occurences are substituted. For example, the command ":1,$s/Dwayne/Dwight/g" substitutes all occurences of "Dwayne" to "Dwight".
:set [all]
Sets some customizing options to VI and EX. The ":set all" command gives all the possible options. (See the section on customizing VI for some options.)
:una string
Removes the abbreviation previously defined by ":ab".
:unm keys
Removes the remove mapping defined by ":map".
:vi filename
Starts editing a new file. If changes have not been saved, the editor will give you a warning.
:w
Write out the current file.
:w filename
Write the buffer to the filename specified.
:w >> filename
Append the contents of the buffer to the filename.
:wq
Write the buffer and quit.
Friday, September 28, 2007
Database Terminalogy
Table
A collection of columns and rows representing a single entity (e.g., customers, orders, employees, etc.).
Column
A single attribute of an entity stored in a table. A column has a name and a datatype. A table may have, and typically does have, more than one column as part of its definition.
Row
A single instance of an entity in a table, including all columns. For example, a student row will store all information about a single student, such as that student’s ID, name, and address.
Cell
The term cell is used to refer to the intersection of a single column in a single row. For example, the CompanyName column for CustomerID 10002 in our example would be a cell holding that data—Bradley Systems.
Constraint
A database object that is used to enforce simple business rules and database integrity. Examples of constraints are PRIMARY KEY, FOREIGN KEY, NOT NULL, and CHECK.
View
A view is a logical projection of data from one or more tables as represented by a SQL statement stored in the database. Views are used to simplify complex and repetitive SQL statements by assigning those statements a name in the database.
Index
An index is a database object that helps speed up retrieval of data by storing logical pointers to specific key values. By scanning the index, which is organized in either ascending or descending order according to the key value, you are able to retrieve a row quicker than by scanning all rows in a table.
Indexorganized table
A table whose physical storage is organized like an index. Unlike a regular table, where rows are inserted in no particular order and querying all rows will retrieve the data in random order, index-organized tables store data organized according to the primary key defined on the table. The difference between a table (referred to as storing data on a heap) and an indexorganized table is like the difference between storing all of your receipts in a shoebox (i.e., in no specific order) and storing it chronologically according to the date the expense was incurred. Taking the receipts out of the shoebox will result in no specific logic in their retrieval, while doing the same when the receipts are organized chronologically will allow you to predict that the June 2 receipt will appear before the August 1 receipt.
Partition
Tables in Oracle 10g can be cut into pieces for more efficient physical storage. A partition (or subpartition) holds a subset of the table’s data, typically on a separate physical disk, so that data retrieval is quicker either by allowing reads from more than one physical disk simultaneously (multipartition parallel reads) or by not reading a partition’s data at all if it is not required to satisfy the query (partition elimination).
Cluster
A storage mechanism object that allows rows from more than one table to be physically stored together for quicker retrieval. For example, if you store the Order information (customer, payment info, delivery details, etc.) in one table and the line items (item, cost, sale price, quantity, etc.) in a different table, you will need to perform at least two reads to retrieve information about an order: one for the order info and the second for line item info. Creating both tables on the cluster organized by the order ID will allow Oracle to place the order and line item data for the same order ID on the same physical block, thereby reducing retrieval of that order’s information to a single read. The downside of clusters is that they force you to preallocate a certain portion or all of the disk space they require when rows are added or the cluster is created.
Sequence
A sequence allows you to creat and increment a counter that can be used to generate numerical values to be used as primary key values for a table.
Synonym
As in the English language, a synonym is another name for an existing object. Synonyms are used in Oracle as shorthand for objects with long names, or to make it easier to remember a specific object.
Stored procedure
A stored procedure is a collection of SQL and PL/SQL statements that perform a specific task, such as to insert a row into a table or to update data.
Trigger
A trigger is a special kind of stored procedure that cannot be invoked manually but rather is automatically invoked whenever an action is performed on a table. Triggers can be associated with a table and a corresponding action such as INSERT, UPDATE, or DELETE as well as system events such as user logon and logoff, or database STARTUP and SHUTDOWN.
Function
A function is a stored program that must return a value. Unlike stored procedures, which can have parameters passed to them and do not need to return any value as output, a function must return a value.
A collection of columns and rows representing a single entity (e.g., customers, orders, employees, etc.).
Column
A single attribute of an entity stored in a table. A column has a name and a datatype. A table may have, and typically does have, more than one column as part of its definition.
Row
A single instance of an entity in a table, including all columns. For example, a student row will store all information about a single student, such as that student’s ID, name, and address.
Cell
The term cell is used to refer to the intersection of a single column in a single row. For example, the CompanyName column for CustomerID 10002 in our example would be a cell holding that data—Bradley Systems.
Constraint
A database object that is used to enforce simple business rules and database integrity. Examples of constraints are PRIMARY KEY, FOREIGN KEY, NOT NULL, and CHECK.
View
A view is a logical projection of data from one or more tables as represented by a SQL statement stored in the database. Views are used to simplify complex and repetitive SQL statements by assigning those statements a name in the database.
Index
An index is a database object that helps speed up retrieval of data by storing logical pointers to specific key values. By scanning the index, which is organized in either ascending or descending order according to the key value, you are able to retrieve a row quicker than by scanning all rows in a table.
Indexorganized table
A table whose physical storage is organized like an index. Unlike a regular table, where rows are inserted in no particular order and querying all rows will retrieve the data in random order, index-organized tables store data organized according to the primary key defined on the table. The difference between a table (referred to as storing data on a heap) and an indexorganized table is like the difference between storing all of your receipts in a shoebox (i.e., in no specific order) and storing it chronologically according to the date the expense was incurred. Taking the receipts out of the shoebox will result in no specific logic in their retrieval, while doing the same when the receipts are organized chronologically will allow you to predict that the June 2 receipt will appear before the August 1 receipt.
Partition
Tables in Oracle 10g can be cut into pieces for more efficient physical storage. A partition (or subpartition) holds a subset of the table’s data, typically on a separate physical disk, so that data retrieval is quicker either by allowing reads from more than one physical disk simultaneously (multipartition parallel reads) or by not reading a partition’s data at all if it is not required to satisfy the query (partition elimination).
Cluster
A storage mechanism object that allows rows from more than one table to be physically stored together for quicker retrieval. For example, if you store the Order information (customer, payment info, delivery details, etc.) in one table and the line items (item, cost, sale price, quantity, etc.) in a different table, you will need to perform at least two reads to retrieve information about an order: one for the order info and the second for line item info. Creating both tables on the cluster organized by the order ID will allow Oracle to place the order and line item data for the same order ID on the same physical block, thereby reducing retrieval of that order’s information to a single read. The downside of clusters is that they force you to preallocate a certain portion or all of the disk space they require when rows are added or the cluster is created.
Sequence
A sequence allows you to creat and increment a counter that can be used to generate numerical values to be used as primary key values for a table.
Synonym
As in the English language, a synonym is another name for an existing object. Synonyms are used in Oracle as shorthand for objects with long names, or to make it easier to remember a specific object.
Stored procedure
A stored procedure is a collection of SQL and PL/SQL statements that perform a specific task, such as to insert a row into a table or to update data.
Trigger
A trigger is a special kind of stored procedure that cannot be invoked manually but rather is automatically invoked whenever an action is performed on a table. Triggers can be associated with a table and a corresponding action such as INSERT, UPDATE, or DELETE as well as system events such as user logon and logoff, or database STARTUP and SHUTDOWN.
Function
A function is a stored program that must return a value. Unlike stored procedures, which can have parameters passed to them and do not need to return any value as output, a function must return a value.
Package
A package is a collection of stored procedures and functions grouped under a common name. This allows you to logically group all program elements for a particular part of the database under a single name for maintenance and performance reasons.
User-defined datatype
A user-defined datatype is a database object that can be used in any table or another object definition. Using user-defined datatypes allows you to ensure consistency between tables and also lets you apply methods (i.e., actions that can be performed by the object) as part of the definition.
BLOB
A BLOB is a binary large object used to store video, images, and large amounts of text. BLOBs are defined as a column in a table and can be one of several datatypes: BLOB, CLOB, NCLOB, or BFILE.
Thursday, September 27, 2007
Crontabrocks
Crontabrocks is a quick and simple guide to crontab usage. The cron daemon provides the ability for a system administrator or any other user (if permitted to do so) to automate the routine running of scripts, tasks or any other server function on a regular basis. The cron daemon is somewhat similar to what the "scheduled tasks" tool on a Windows machine would be. Most Unix/Linux based systems come with the cron daemon installed and configured to start on server boot by default. There is usually not much work, if any, that you have to do to start using the cron daemon. The server itself usually comes out of the box with several systemic cron jobs already set up, so adding your own jobs to the schedule is not big deal. Cron jobs are scheduled by a user using a tool known as "crontab". The crontab is nothing more than a text file written in a syntax that the cron daemon understands.
So first, let's talk about how to handle the crontab itself...
Accessing the Crontab
The following commands would apply to the user which you are logged in as. For example, if you are logged in as "root", these commands would pertain to root's crontab file.
crontab -e - opens the user's crontab file for viewing/editing
crontab -l - simply lists the crontab file's contents for the user. Think of it as a "cat" function for the crontab.
crontab -r - removes the crontab file contents for the user
But what if you want to edit another user's crontab?
The system administrator is usually logged in as "root", but making changes to another user's crontab file or simply looking at another user's crontab file is often necessary. For situations like this, you can append the "-u" flag followed by the desired username.
For example, if logged in as root but you want to edit the crontab for the user "admin", you would do the following:
crontab -e -u admin
The same logic applies to the other crontab commands as well, such as:
crontab -l -u admin - lists the crontab entry for the "admin" user.
crontab -r -u admin - removes the crontab entry for the "admin" user.
Writing to the Crontab
Now that you know how to access the crontab, let's take a look at the syntax of the crontab entry itself.
A typical crontab entry might look like this:
30 0,12 * * * /usr/local/scripts/whatever.script
OK, so what does that mean? Well, there are 2 parts to the entry you see above. In fact, any crontab entry has 2 parts:
Part 1 - The schedule
The schedule, which governs when the task will run, consists of a string of numbers, possible commas and asterisks (*).
So, in the above example, the schedule is:
30 0,12 * * *
What you are seeing is actually split up into 5 sections. The following chart illustrates what each section of the schedule is for:
1. Minute - Minutes after the hour (0-59).
2. Hour - 24-hour format (0-23).
3. Day - Day of the month (1-31).
4. Month - Month of the year (1-12).
5. Weekday - Day of the week. (0-6, where 0 indicates Sunday).
* = An asterisk in a schedule field indicates "every". It means that the task will occur on "every" instance of the given field. So a "*" on the Month field indicates the the task will run "every" month of the year. A * in the Minutes field would indicate that the task would run "every" minute.
, = A comma is used to input multiple values for a field. For example, if you wanted a task to run at hours 12, 15 and 18, you would enter that as "12,15,18".
Let's take a look at how this format fits into the syntax of a crontab entry:
_________________________ 1. Minute - Minutes after the hour (0-59)
______________________ 2. Hour - 24-hour format (0-23).
___________________ 3. Day - Day of the month (1-31)
________________ 4. Month - Month of the year (1-12)
______________5. Weekday - Day of the week. (0-6, where 0 indicates Sunday)
30 0,12 * * * /some/script/or/command
So, when we combine all the schedule elements, we know when and how often this task will run. Going on the above example, this task would run:
At 30 minutes past the hours of 0 (midnight) and 12 (noon), EVERY day of the month, EVERY month of the year and EVERY day of the week.
In other words, the above task would run every single day at 12:30AM and 12:30PM.
Let's play around with the schedule a little bit and try something different. What if we had something like this:
15,45 0,12,6 20 1,2,3 0 /some/script/or/command
Wow! Now this is a pretty complex crontab entry. Let's decipher it...
15,45 - This means that the task will run at 15 and 45 minutes past the hour. But what hours and what days? Well, that's coming up.
0.12.6 - The task will run during the hours of 0 (midnight), 12 (noon) and 6AM.
20 - The task will run on the 20th day of the month. But during what months?
1,2,3 - The task will run only during the months of January, February and March.
0 - The task will only run on a Sunday.
So, when we put all of this information together, here is what it boils down to:
This task is going to run at 0:15, 0:45, 6:15, 6:45, 12:15 and 12:45 on the 20th of January, February and March IF that day falls on a Sunday. Pretty cool, huh?
Part 2: The task itself
This is pretty self-explanatory. In the above example, the "task" to run would be: /some/script/or/command.
Now obviously, that's not a real script or commnad, it's just an example that you would fill in the blanks for. Generally, there are 2 ways to run a task from the crontab:
Method A: Run the desired command directly from the crontab
Here's an example of running a command directly from the crontab:
*/15 * * * * /sbin/ifconfig mail myname@mydomain.com
This is pretty cut and dry. This entry is going to run the "ifconfig" command and then pipe the results out to an e-mail that gets sent to myname@mydomain.com. This task will run EVERY 15 minutes, EVERY day, EVERY month, EVERY day of the week. In other words, this task runs every 15 minutes.
As you can see, the "ifconfig" command is called directly form the crontab entry. This is fine is you are running a simple command. But what if you need to do more than that?
Method B: Run a script from the crontab that runs the commands.
Here's an example of running a script from the crontab. The script that you run, in turn, executes whatever instruction you might want:
15 0* * * /usr/local/scripts/run_webstats.script
As you can see, every night at 0:15, this task runs a script called "/usr/local/scripts/run_webstats.script". That script, as you may guess, generates the webstats for the website on the server. The crontab runs a script and then the script contains the actual instructions on what the task is supposed to do. Get it?
By the way, I'm just making these crontab scripts up out of thin air, so don't expect the above example to really work. You would need to replace my bogus script calls with REAL scripts calls or commands.
So first, let's talk about how to handle the crontab itself...
Accessing the Crontab
The following commands would apply to the user which you are logged in as. For example, if you are logged in as "root", these commands would pertain to root's crontab file.
crontab -e - opens the user's crontab file for viewing/editing
crontab -l - simply lists the crontab file's contents for the user. Think of it as a "cat" function for the crontab.
crontab -r - removes the crontab file contents for the user
But what if you want to edit another user's crontab?
The system administrator is usually logged in as "root", but making changes to another user's crontab file or simply looking at another user's crontab file is often necessary. For situations like this, you can append the "-u" flag followed by the desired username.
For example, if logged in as root but you want to edit the crontab for the user "admin", you would do the following:
crontab -e -u admin
The same logic applies to the other crontab commands as well, such as:
crontab -l -u admin - lists the crontab entry for the "admin" user.
crontab -r -u admin - removes the crontab entry for the "admin" user.
Writing to the Crontab
Now that you know how to access the crontab, let's take a look at the syntax of the crontab entry itself.
A typical crontab entry might look like this:
30 0,12 * * * /usr/local/scripts/whatever.script
OK, so what does that mean? Well, there are 2 parts to the entry you see above. In fact, any crontab entry has 2 parts:
Part 1 - The schedule
The schedule, which governs when the task will run, consists of a string of numbers, possible commas and asterisks (*).
So, in the above example, the schedule is:
30 0,12 * * *
What you are seeing is actually split up into 5 sections. The following chart illustrates what each section of the schedule is for:
1. Minute - Minutes after the hour (0-59).
2. Hour - 24-hour format (0-23).
3. Day - Day of the month (1-31).
4. Month - Month of the year (1-12).
5. Weekday - Day of the week. (0-6, where 0 indicates Sunday).
* = An asterisk in a schedule field indicates "every". It means that the task will occur on "every" instance of the given field. So a "*" on the Month field indicates the the task will run "every" month of the year. A * in the Minutes field would indicate that the task would run "every" minute.
, = A comma is used to input multiple values for a field. For example, if you wanted a task to run at hours 12, 15 and 18, you would enter that as "12,15,18".
Let's take a look at how this format fits into the syntax of a crontab entry:
_________________________ 1. Minute - Minutes after the hour (0-59)
______________________ 2. Hour - 24-hour format (0-23).
___________________ 3. Day - Day of the month (1-31)
________________ 4. Month - Month of the year (1-12)
______________5. Weekday - Day of the week. (0-6, where 0 indicates Sunday)
30 0,12 * * * /some/script/or/command
So, when we combine all the schedule elements, we know when and how often this task will run. Going on the above example, this task would run:
At 30 minutes past the hours of 0 (midnight) and 12 (noon), EVERY day of the month, EVERY month of the year and EVERY day of the week.
In other words, the above task would run every single day at 12:30AM and 12:30PM.
Let's play around with the schedule a little bit and try something different. What if we had something like this:
15,45 0,12,6 20 1,2,3 0 /some/script/or/command
Wow! Now this is a pretty complex crontab entry. Let's decipher it...
15,45 - This means that the task will run at 15 and 45 minutes past the hour. But what hours and what days? Well, that's coming up.
0.12.6 - The task will run during the hours of 0 (midnight), 12 (noon) and 6AM.
20 - The task will run on the 20th day of the month. But during what months?
1,2,3 - The task will run only during the months of January, February and March.
0 - The task will only run on a Sunday.
So, when we put all of this information together, here is what it boils down to:
This task is going to run at 0:15, 0:45, 6:15, 6:45, 12:15 and 12:45 on the 20th of January, February and March IF that day falls on a Sunday. Pretty cool, huh?
Part 2: The task itself
This is pretty self-explanatory. In the above example, the "task" to run would be: /some/script/or/command.
Now obviously, that's not a real script or commnad, it's just an example that you would fill in the blanks for. Generally, there are 2 ways to run a task from the crontab:
Method A: Run the desired command directly from the crontab
Here's an example of running a command directly from the crontab:
*/15 * * * * /sbin/ifconfig mail myname@mydomain.com
This is pretty cut and dry. This entry is going to run the "ifconfig" command and then pipe the results out to an e-mail that gets sent to myname@mydomain.com. This task will run EVERY 15 minutes, EVERY day, EVERY month, EVERY day of the week. In other words, this task runs every 15 minutes.
As you can see, the "ifconfig" command is called directly form the crontab entry. This is fine is you are running a simple command. But what if you need to do more than that?
Method B: Run a script from the crontab that runs the commands.
Here's an example of running a script from the crontab. The script that you run, in turn, executes whatever instruction you might want:
15 0* * * /usr/local/scripts/run_webstats.script
As you can see, every night at 0:15, this task runs a script called "/usr/local/scripts/run_webstats.script". That script, as you may guess, generates the webstats for the website on the server. The crontab runs a script and then the script contains the actual instructions on what the task is supposed to do. Get it?
By the way, I'm just making these crontab scripts up out of thin air, so don't expect the above example to really work. You would need to replace my bogus script calls with REAL scripts calls or commands.
Subscribe to:
Posts (Atom)