Part 1 – Introduction.........................................................................1
1 – Introduction................................................................................................2
Why Use The Command Line?.......................................................................................2
What This Book Is About.................................................................................................3
Who Should Read This Book..........................................................................................3
What's In This Book.........................................................................................................4
How To Read This Book..................................................................................................5
Prerequisites...............................................................................................................5
Why I Don't Call It “GNU/Linux”.............................................................................5
Acknowledgments...........................................................................................................6
Your Feedback Is Needed!..............................................................................................7
Further Reading...............................................................................................................7
Colophon.........................................................................................................................7
Part 2 – Learning The Shell..............................................................9
2 – What Is The Shell?...................................................................................10
Terminal Emulators........................................................................................................10
Your First Keystrokes....................................................................................................10
Command History.....................................................................................................11
Cursor Movement.....................................................................................................11
A Few Words About Mice And Focus...................................................................11
Try Some Simple Commands........................................................................................12
Ending A Terminal Session............................................................................................13
The Console Behind The Curtain........................................................................13
Further Reading.............................................................................................................13
3 – Navigation.................................................................................................14
Understanding The File System Tree............................................................................14
The Current Working Directory......................................................................................14
Listing The Contents Of A Directory..............................................................................15
Changing The Current Working Directory.....................................................................16
Absolute Pathnames................................................................................................16
Relative Pathnames.................................................................................................16
Some Helpful Shortcuts............................................................................................18
Important Facts About Filenames........................................................................18
4 – Exploring The System.............................................................................20
More Fun With ls...........................................................................................................20
Options And Arguments............................................................................................21
A Longer Look At Long Format.................................................................................22
Determining A File's Type With file................................................................................23
Viewing File Contents With less....................................................................................24
What Is “Text”?.....................................................................................................24
Less Is More........................................................................................................26
A Guided Tour................................................................................................................26
Symbolic Links...............................................................................................................29
Hard Links.....................................................................................................................30
Further Reading.............................................................................................................30
5 – Manipulating Files And Directories........................................................31
Wildcards.......................................................................................................................31
Character Ranges................................................................................................33
Wildcards Work In The GUI Too..........................................................................33
mkdir – Create Directories.............................................................................................34
cp – Copy Files And Directories....................................................................................34
Useful Options And Examples..................................................................................35
mv – Move And Rename Files......................................................................................36
Useful Options And Examples..................................................................................36
rm – Remove Files And Directories...............................................................................37
Useful Options And Examples..................................................................................37
Be Careful With rm!.............................................................................................38
ln – Create Links............................................................................................................39
Hard Links................................................................................................................39
Symbolic Links..........................................................................................................40
Let's Build A Playground................................................................................................40
Creating Directories..................................................................................................40
Copying Files............................................................................................................41
Moving And Renaming Files.....................................................................................42
Creating Hard Links..................................................................................................43
Creating Symbolic Links...........................................................................................44
Removing Files And Directories...............................................................................45
Creating Symlinks With The GUI.........................................................................47
Summing Up.............................................................................................................47
6 – Working With Commands........................................................................48
What Exactly Are Commands?......................................................................................48
Identifying Commands...................................................................................................49
type – Display A Command's Type...........................................................................49
which – Display An Executable's Location...............................................................49
Getting A Command's Documentation..........................................................................50
help – Get Help For Shell Builtins............................................................................50
--help – Display Usage Information..........................................................................51
man – Display A Program's Manual Page................................................................51
apropos – Display Appropriate Commands..............................................................53
whatis – Display A Very Brief Description Of A Command.......................................53
The Most Brutal Man Page Of Them All..............................................................53
info – Display A Program's Info Entry.......................................................................54
README And Other Program Documentation Files................................................55
Creating Your Own Commands With alias....................................................................56
Revisiting Old Friends...................................................................................................58
Further Reading.............................................................................................................58
7 – Redirection................................................................................................59
Standard Input, Output, And Error.................................................................................59
Redirecting Standard Output.........................................................................................60
Redirecting Standard Error............................................................................................61
Redirecting Standard Output And Standard Error To One File................................62
Disposing Of Unwanted Output................................................................................63
/dev/null In Unix Culture......................................................................................63
Redirecting Standard Input............................................................................................63
cat – Concatenate Files............................................................................................63
Pipelines........................................................................................................................65
Filters........................................................................................................................66
uniq - Report Or Omit Repeated Lines....................................................................66
wc – Print Line, Word, And Byte Counts..................................................................67
grep – Print Lines Matching A Pattern......................................................................67
head / tail – Print First / Last Part Of Files................................................................68
tee – Read From Stdin And Output To Stdout And Files..........................................69
Summing Up..................................................................................................................70
Linux Is About Imagination..................................................................................70
8 – Seeing The World As The Shell Sees It..................................................72
Expansion......................................................................................................................72
Pathname Expansion...............................................................................................73
Pathname Expansion Of Hidden Files.................................................................74
Tilde Expansion........................................................................................................74
Arithmetic Expansion................................................................................................75
Brace Expansion......................................................................................................76
Parameter Expansion...............................................................................................77
Command Substitution.............................................................................................78
Quoting..........................................................................................................................79
Double Quotes..........................................................................................................79
Single Quotes...........................................................................................................81
Escaping Characters................................................................................................82
Backslash Escape Sequences............................................................................82
Summing Up..................................................................................................................83
Further Reading.............................................................................................................83
9 – Advanced Keyboard Tricks.....................................................................84
Command Line Editing..................................................................................................84
Cursor Movement.....................................................................................................84
Modifying Text...........................................................................................................85
Cutting And Pasting (Killing And Yanking) Text........................................................85
The Meta Key......................................................................................................86
Completion....................................................................................................................86
Programmable Completion..................................................................................88
Using History.................................................................................................................88
Searching History.....................................................................................................89
History Expansion.....................................................................................................91
script....................................................................................................................91
Summing Up..................................................................................................................92
Further Reading.............................................................................................................92
10 – Permissions............................................................................................93
Owners, Group Members, And Everybody Else............................................................94
Reading, Writing, And Executing...................................................................................95
chmod – Change file mode.......................................................................................97
What The Heck Is Octal?.....................................................................................98
Setting File Mode With The GUI.............................................................................100
umask – Set Default Permissions..........................................................................101
Some Special Permissions................................................................................103
Changing Identities......................................................................................................104
su – Run A Shell With Substitute User And Group IDs..........................................105
sudo – Execute A Command As Another User.......................................................106
Ubuntu And sudo...............................................................................................107
chown – Change File Owner And Group................................................................107
chgrp – Change Group Ownership.........................................................................109
Exercising Our Privileges............................................................................................109
Changing Your Password.............................................................................................111
Further Reading...........................................................................................................112
11 – Processes..............................................................................................113
How A Process Works.................................................................................................113
Viewing Processes......................................................................................................114
Viewing Processes Dynamically With top...............................................................116
Controlling Processes..................................................................................................118
Interrupting A Process.............................................................................................119
Putting A Process In The Background....................................................................119
Returning A Process To The Foreground...............................................................120
Stopping (Pausing) A Process................................................................................121
Signals.........................................................................................................................122
Sending Signals To Processes With kill.................................................................122
Sending Signals To Multiple Processes With killall................................................125
More Process Related Commands.............................................................................125
Part 3 – Configuration And The Environment.............................127
12 – The Environment..................................................................................128
What Is Stored In The Environment?..........................................................................128
Examining The Environment..................................................................................128
Some Interesting Variables.....................................................................................130
How Is The Environment Established?.......................................................................131
What's In A Startup File?........................................................................................132
Modifying The Environment.........................................................................................134
Which Files Should We Modify?.............................................................................134
Text Editors.............................................................................................................134
Using A Text Editor.................................................................................................135
Why Comments Are Important..........................................................................138
Activating Our Changes..........................................................................................139
Summing Up................................................................................................................139
Further Reading..........................................................................................................139
13 – A Gentle Introduction To vi.................................................................140
Why We Should Learn vi.............................................................................................140
A Little Background ....................................................................................................141
Starting And Stopping vi..............................................................................................141
Compatibility Mode............................................................................................142
Editing Modes..............................................................................................................143
Entering Insert Mode..............................................................................................144
Saving Our Work....................................................................................................144
Moving The Cursor Around.........................................................................................145
Basic Editing................................................................................................................146
Appending Text.......................................................................................................146
Opening A Line.......................................................................................................147
Deleting Text...........................................................................................................148
Cutting, Copying And Pasting Text.........................................................................149
Joining Lines...........................................................................................................151
Search And Replace....................................................................................................151
Searching Within A Line..........................................................................................151
Searching The Entire File.......................................................................................151
Global Search And Replace...................................................................................152
Editing Multiple Files...................................................................................................154
Switching Between Files.........................................................................................155
Opening Additional Files For Editing......................................................................155
Copying Content From One File Into Another........................................................156
Inserting An Entire File Into Another.......................................................................157
Saving Our Work.........................................................................................................158
Further Reading..........................................................................................................159
14 – Customizing The Prompt.....................................................................160
Anatomy Of A Prompt..................................................................................................160
Trying Some Alternate Prompt Designs......................................................................162
Adding Color................................................................................................................163
Terminal Confusion............................................................................................163
Moving The Cursor......................................................................................................166
Saving The Prompt......................................................................................................167
Summing Up................................................................................................................167
Further Reading..........................................................................................................168
Part 4 – Common Tasks And Essential Tools.............................169
15 – Package Management..........................................................................170
Packaging Systems.....................................................................................................170
How A Package System Works...................................................................................171
Package Files.........................................................................................................171
Repositories............................................................................................................171
Dependencies.........................................................................................................172
High And Low-level Package Tools........................................................................172
Common Package Management Tasks.......................................................................173
Finding A Package In A Repository........................................................................173
Installing A Package From A Repository.................................................................173
Installing A Package From A Package File.............................................................174
Removing A Package.............................................................................................174
Updating Packages From A Repository..................................................................175
Upgrading A Package From A Package File...........................................................175
Listing Installed Packages......................................................................................176
Determining If A Package Is Installed.....................................................................176
Displaying Info About An Installed Package...........................................................177
Finding Which Package Installed A File.................................................................177
Summing Up................................................................................................................177
The Linux Software Installation Myth.................................................................178
Further Reading..........................................................................................................179
16 – Storage Media.......................................................................................180
Mounting And Unmounting Storage Devices..............................................................180
Viewing A List Of Mounted File Systems................................................................182
Why Unmounting Is Important...........................................................................185
Determining Device Names....................................................................................186
Creating New File Systems.........................................................................................189
Manipulating Partitions With fdisk..........................................................................189
Creating A New File System With mkfs..................................................................192
Testing And Repairing File Systems............................................................................193
What The fsck?..................................................................................................193
Formatting Floppy Disks..............................................................................................193
Moving Data Directly To/From Devices.......................................................................194
Creating CD-ROM Images..........................................................................................195
Creating An Image Copy Of A CD-ROM.................................................................195
Creating An Image From A Collection Of Files.......................................................195
A Program By Any Other Name.........................................................................196
Writing CD-ROM Images.............................................................................................196
Mounting An ISO Image Directly............................................................................196
Blanking A Re-Writable CD-ROM...........................................................................197
Writing An Image....................................................................................................197
Further Reading..........................................................................................................197
Extra Credit..................................................................................................................197
17 – Networking............................................................................................199
Examining And Monitoring A Network.........................................................................200
ping.........................................................................................................................200
traceroute...............................................................................................................201
netstat.....................................................................................................................202
Transporting Files Over A Network..............................................................................203
ftp............................................................................................................................204
lftp – A Better ftp.....................................................................................................206
wget........................................................................................................................206
Secure Communication With Remote Hosts...............................................................206
ssh..........................................................................................................................207
Tunneling With SSH..........................................................................................210
scp And sftp............................................................................................................211
An SSH Client For Windows?............................................................................212
Further Reading..........................................................................................................212
18 – Searching For Files..............................................................................213
locate – Find Files The Easy Way...............................................................................213
Where Does The locate Database Come From?..............................................215
find – Find Files The Hard Way...................................................................................215
Tests.......................................................................................................................216
Operators................................................................................................................218
Predefined Actions..................................................................................................221
User Defined Actions..............................................................................................223
Improving Efficiency...............................................................................................224
xargs.......................................................................................................................224
Dealing With Funny Filenames..........................................................................225
A Return To The Playground..................................................................................226
Options...................................................................................................................228
Further Reading..........................................................................................................229
19 – Archiving And Backup.........................................................................230
Compressing Files.......................................................................................................230
gzip.........................................................................................................................231
bzip2.......................................................................................................................233
Don’t Be Compressive Compulsive...................................................................234
Archiving Files.............................................................................................................234
tar............................................................................................................................234
zip...........................................................................................................................240
Synchronizing Files And Directories............................................................................242
Using rsync Over A Network...................................................................................245
Further Reading..........................................................................................................246
20 – Regular Expressions...........................................................................247
What Are Regular Expressions?............................................................................247
grep.............................................................................................................................247
Metacharacters And Literals........................................................................................249
The Any Character......................................................................................................250
Anchors.......................................................................................................................251
A Crossword Puzzle Helper...............................................................................251
Bracket Expressions And Character Classes..............................................................252
Negation.................................................................................................................252
Traditional Character Ranges.................................................................................253
POSIX Character Classes......................................................................................254
Reverting To Traditional Collation Order............................................................257
POSIX Basic Vs. Extended Regular Expressions.......................................................258
POSIX................................................................................................................258
Alternation...................................................................................................................259
Quantifiers...................................................................................................................260
? - Match An Element Zero Or One Time...............................................................260
* - Match An Element Zero Or More Times............................................................261
+ - Match An Element One Or More Times............................................................262
{ } - Match An Element A Specific Number Of Times..............................................262
Putting Regular Expressions To Work.........................................................................263
Validating A Phone List With grep...........................................................................263
Finding Ugly Filenames With find...........................................................................264
Searching For Files With locate.............................................................................265
Searching For Text In less And vim........................................................................265
Summing Up................................................................................................................267
Further Reading..........................................................................................................267
21 – Text Processing....................................................................................268
Applications Of Text.....................................................................................................268
Documents.............................................................................................................269
Web Pages.............................................................................................................269
Email.......................................................................................................................269
Printer Output.........................................................................................................269
Program Source Code............................................................................................269
Revisiting Some Old Friends.......................................................................................269
cat...........................................................................................................................270
MS-DOS Text Vs. Unix Text...............................................................................271
sort..........................................................................................................................271
uniq.........................................................................................................................279
Slicing And Dicing........................................................................................................280
cut...........................................................................................................................280
Expanding Tabs.................................................................................................283
paste.......................................................................................................................284
join..........................................................................................................................285
Comparing Text...........................................................................................................287
comm......................................................................................................................288
diff...........................................................................................................................288
patch.......................................................................................................................291
Editing On The Fly.......................................................................................................292
tr..............................................................................................................................292
ROT13: The Not-So-Secret Decoder Ring........................................................294
sed..........................................................................................................................295
People Who Like sed Also Like.........................................................................303
aspell......................................................................................................................303
Summing Up................................................................................................................307
Further Reading..........................................................................................................307
Extra Credit..................................................................................................................308
22 – Formatting Output................................................................................309
Simple Formatting Tools..............................................................................................309
nl – Number Lines..................................................................................................309
fold – Wrap Each Line To A Specified Length........................................................313
fmt – A Simple Text Formatter................................................................................313
pr – Format Text For Printing..................................................................................317
printf – Format And Print Data................................................................................318
Document Formatting Systems...................................................................................321
groff.........................................................................................................................322
Summing Up................................................................................................................328
Further Reading..........................................................................................................328
23 – Printing..................................................................................................330
A Brief History Of Printing............................................................................................330
Printing In The Dim Times......................................................................................330
Character-based Printers.......................................................................................331
Graphical Printers...................................................................................................332
Printing With Linux......................................................................................................333
Preparing Files For Printing.........................................................................................333
pr – Convert Text Files For Printing........................................................................333
Sending A Print Job To A Printer..................................................................................335
lpr – Print Files (Berkeley Style).............................................................................335
lp – Print Files (System V Style).............................................................................336
Another Option: a2ps..............................................................................................337
Monitoring And Controlling Print Jobs.........................................................................340
lpstat – Display Print System Status......................................................................340
lpq – Display Printer Queue Status........................................................................341
lprm / cancel – Cancel Print Jobs..........................................................................342
Summing Up................................................................................................................342
Further Reading..........................................................................................................342
24 – Compiling Programs............................................................................344
What Is Compiling?.....................................................................................................344
Are All Programs Compiled?..................................................................................345
Compiling A C Program...............................................................................................346
Obtaining The Source Code...................................................................................346
Examining The Source Tree...................................................................................348
Building The Program.............................................................................................350
Installing The Program...........................................................................................354
Summing Up................................................................................................................354
Further Reading..........................................................................................................355
Part 5 – Writing Shell Scripts.......................................................357
25 – Writing Your First Script......................................................................358
What Are Shell Scripts?...............................................................................................358
How To Write A Shell Script.........................................................................................358
Script File Format........................................................................................................359
Executable Permissions..............................................................................................360
Script File Location......................................................................................................360
Good Locations For Scripts....................................................................................362
More Formatting Tricks................................................................................................362
Long Option Names................................................................................................362
Indentation And Line Continuation.........................................................................362
Configuring vim For Script Writing.....................................................................363
Summing Up................................................................................................................364
Further Reading..........................................................................................................364
26 – Starting A Project.................................................................................365
First Stage: Minimal Document...................................................................................365
Second Stage: Adding A Little Data............................................................................367
Variables And Constants.............................................................................................368
Assigning Values To Variables And Constants.......................................................371
Here Documents.........................................................................................................372
Summing Up................................................................................................................375
Further Reading..........................................................................................................375
27 – Top-Down Design.................................................................................376
Shell Functions............................................................................................................377
Local Variables............................................................................................................380
Keep Scripts Running..................................................................................................381
Shell Functions In Your .bashrc File..................................................................384
Summing Up................................................................................................................384
Further Reading..........................................................................................................384
28 – Flow Control: Branching With if.........................................................385
if...................................................................................................................................385
Exit Status...................................................................................................................386
test...............................................................................................................................388
File Expressions.....................................................................................................388
String Expressions..................................................................................................391
Integer Expressions................................................................................................392
A More Modern Version Of test...................................................................................393
(( )) - Designed For Integers........................................................................................395
Combining Expressions...............................................................................................396
Portability Is The Hobgoblin Of Little Minds.......................................................398
Control Operators: Another Way To Branch................................................................399
Summing Up................................................................................................................399
Further Reading..........................................................................................................400
29 – Reading Keyboard Input......................................................................401
read – Read Values From Standard Input...................................................................402
Options...................................................................................................................404
IFS..........................................................................................................................406
You Can’t Pipe read...........................................................................................407
Validating Input............................................................................................................408
Menus..........................................................................................................................409
Summing Up................................................................................................................411
Extra Credit.............................................................................................................411
Further Reading...........................................................................................................411
30 – Flow Control: Looping With while / until...........................................412
Looping........................................................................................................................412
while........................................................................................................................412
Breaking Out Of A Loop...............................................................................................415
until.........................................................................................................................416
Reading Files With Loops...........................................................................................417
Summing Up................................................................................................................418
Further Reading..........................................................................................................418
31 – Troubleshooting...................................................................................419
Syntactic Errors...........................................................................................................419
Missing Quotes.......................................................................................................420
Missing Or Unexpected Tokens..............................................................................420
Unanticipated Expansions......................................................................................421
Logical Errors .............................................................................................................423
Defensive Programming.........................................................................................423
Verifying Input.........................................................................................................425
Design Is A Function Of Time............................................................................425
Testing.........................................................................................................................425
Test Cases..............................................................................................................426
Debugging...................................................................................................................427
Finding The Problem Area......................................................................................427
Tracing....................................................................................................................427
Examining Values During Execution......................................................................430
Summing Up................................................................................................................430
Further Reading..........................................................................................................431
32 – Flow Control: Branching With case...................................................432
case.............................................................................................................................432
Patterns..................................................................................................................434
Summing Up................................................................................................................436
Further Reading..........................................................................................................436
33 – Positional Parameters.........................................................................437
Accessing The Command Line...................................................................................437
Determining The Number of Arguments.................................................................438
shift – Getting Access To Many Arguments............................................................439
Simple Applications................................................................................................440
Using Positional Parameters With Shell Functions................................................441
Handling Positional Parameters En Masse.................................................................442
A More Complete Application......................................................................................444
Summing Up................................................................................................................447
Further Reading..........................................................................................................450
34 – Flow Control: Looping With for..........................................................451
for: Traditional Shell Form...........................................................................................451
Why i?................................................................................................................453
for: C Language Form.................................................................................................454
Summing Up................................................................................................................455
Further Reading..........................................................................................................456
35 – Strings And Numbers..........................................................................457
Parameter Expansion..................................................................................................457
Basic Parameters...................................................................................................457
Expansions To Manage Empty Variables...............................................................458
Expansions That Return Variable Names..............................................................460
String Operations....................................................................................................460
Arithmetic Evaluation And Expansion.........................................................................463
Number Bases........................................................................................................464
Unary Operators.....................................................................................................464
Simple Arithmetic....................................................................................................464
Assignment.............................................................................................................466
Bit Operations.........................................................................................................468
Logic.......................................................................................................................469
bc – An Arbitrary Precision Calculator Language........................................................472
Using bc..................................................................................................................473
An Example Script..................................................................................................473
Summing Up................................................................................................................475
Extra Credit..................................................................................................................475
Further Reading..........................................................................................................475
36 – Arrays....................................................................................................476
What Are Arrays?........................................................................................................476
Creating An Array........................................................................................................476
Assigning Values To An Array......................................................................................477
Accessing Array Elements...........................................................................................478
Array Operations.........................................................................................................480
Outputting The Entire Contents Of An Array..........................................................480
Determining The Number Of Array Elements.........................................................480
Finding The Subscripts Used By An Array.............................................................481
Adding Elements To The End Of An Array.............................................................481
Sorting An Array......................................................................................................482
Deleting An Array....................................................................................................482
Summing Up................................................................................................................483
Further Reading..........................................................................................................484
37 – Exotica...................................................................................................485
Group Commands And Subshells...............................................................................485
Process Substitution...............................................................................................486
Traps............................................................................................................................488
Temporary Files.................................................................................................491
Asynchronous Execution.............................................................................................492
wait.........................................................................................................................492
Named Pipes...............................................................................................................493
Setting Up A Named Pipe.......................................................................................494
Using Named Pipes................................................................................................494
Summing Up................................................................................................................495
Further Reading..........................................................................................................495