In this entry of our Linux Basics series, we will begin to uncover some of the file interactions possible through the command line. We'll touch on things like creating, editing, and removing files and directories, and searching files for a specific phrase. By the end of this post, you will be able to create a new directory in the command line, create a new file in that directory, edit that file, and securely remove that file.
Previous Posts From This Series
The most common way to create a new file in Linux is to touch it. This is a pretty straightforward command.
The above will create
somefile.txt in your present working directory. Alternatively, you can use the absolute path to generate a file as well. To produce multiple files you just add those files to the command.
touch somefile.txt anotherfile.php
This will create both files in your present working directory. There isn't much more to say about this, pretty manageable and easy to start using to create empty files.
What if you want to create a quick note in the command line? There is a straightforward way to do this that many people, and even tutorials, overlook. This method makes use of an output redirect in the command line. Linux has the option to redirect the output of a task to a file. Redirects are seen a lot in crontab entries. The
> operator will redirect the output of a command to a file, overwriting the file if it exists
>> to append to the file. As I mentioned before, a crontab is one of the most prevalent areas you will find a redirect.
00 */1 * * * php /foo/bar/baz.php > output.log 2>&1
That line just tells our cron job to save the output of
output/log. Let's get back to making files now that we have explained redirecting output a little bit. So what is overlooked a lot, using the
cat command to create files and add content to them. When you redirect
cat in the command line to a file you will enter interactive mode, meaning the terminal will wait for you to enter some text.
cat > somefile.txt
When you enter that command try typing Hello World!, press
CTRL + D to save the input to the file. Now you can use a method we covered in the first part of this series to verify that input saved successfully,
cat somefile.txt. Like magic, the file was created with the entered text!
To create a new dir you use the
mkdir command, this is another easy to remember tool. You use
mkdir to make a directory, easy. This command is just about as simple as the touch command.
Instead, like touch, you can use the absolute path.
When it comes to editing files in the command line there is a broad range of command-line based text editors you can install. Linux and Unix come with Vi preinstalled so we will cover the basics of Vi. We will additionally cover what I think is the more user-friendly option for beginners Nano.
Vi can be awkward when you first jump into it, but once you get the hang of it and start to remember all the shortcuts it is a really fast and powerful editor. Most people give up on Vi without even trying, if you type
vi somefile.txt into your command line right now and try to edit the file you can't. You probably think the editor is broken and won't let you do anything.
Command Mode and Input Mode
Vi starts in command mode which is an admin mode. It lets you move around the file, save, and perform other tasks on the file itself. To start editing or adding text we need to enter into input mode. There are a lot of commands you can enter in Vi to move around and interact with files and I won't cover them all here, that is a whole tutorial's worth of information alone.
|k||Move cursor up one line|
|j||Move cursor down one line|
|h||Move cursor left one position|
|l||Move cursor right one position|
|0||Move the cursor to the beginning of the line (Zero, not capital o)|
|$||Move cursor to the end of the line|
|w||Jump to the next word|
|b||Move back to the previous word|
|x||Deletes character under the cursor|
|X||Deletes character before the cursor|
|D||Deletes from the cursor to end of line|
|:w||Write to file (Save)|
|:q||Quit - will not work if the file has been modified|
|:q!||Quit and discard changes|
|:wq||Write and quit (Save and quit)|
|Enter Input Mode|
|i||Insert text before the cursor location|
|I||Insert text at the beginning of the current line (Capital i)|
|r||Replace character under the cursor (Goes back to command mode automatically)|
|a||Insert text after cursor|
|A||Insert at end of current line|
|o||Create a new line below cursor|
|O||Create a new line above cursor (Capital o not Zero)|
|ESC||Go back to Command Mode|
Those are just some of the basic commands that should get you moving around and editing files in Vi, it is a different way of thinking and takes a bit of getting used to, but once you do it's incredibly fast. Some of the more advanced commands will let you move to specific positions, jump half a page, search, etc. For now, these basic commands should be enough to get your feet wet. The big thing to remember is to switch back and forth between command mode and input mode. Hitting the arrow keys while in input mode can cause a headache and trying to type while in command mode can end up with entire deleted lines if you aren't careful. I did a lot of
:q! when I first started using Vi. Even with these basic commands, you can start to get a feeling for how powerful Vi is.
Compared to Vi Nano is a walk in the park. First, check to see if you have Nano installed. If you remember from our first post you can do that by entering
which nano into the command line. If you get back a response that looks like
/bin/nano then Nano is installed, if not you will need to install it.
sudo apt install nano
That should install Nano after you enter your password.
Now type the following to open our file in Nano.
As you can see this is a lot more user friendly. Across the bottom of the screen, you will see some of the more common keyboard commands. To start you can move the cursor with the arrow keys and type as you normally would, that's a lot easier than Vi for sure. Below we'll cover some of the more common commands used within Nano, like Vi, there are a lot more options I won't cover here but this should get you editing in Nano.
|CTRL + S||Save file|
|CTRL + X||Exit|
|CTRL + O||Save As|
|ALT + U||Undo|
|ALT + N||Turn line numbers on or off|
|CTRL + D||Delete character under cursor|
|ALT + BACKSPACE||Delete word to the left|
|CTRL + DEL||Delete word to the right|
|ALT + DEL||Delete current line|
|CTRL + A||To start of line|
|CTRL + E||To end of line|
|CTRL + P||One line up|
|CTRL + N||One line down|
As you can see both Vi and Nano offer a lot of powerful options for editing and interacting with files. Take some time to use both and figure out which one you are more comfortable with. Either tool is a great choice for working with files within the command line.
Back to creating files...
Both Vi and Nano will also allow you to create new files. I wanted to wait until we went over the basic commands to create files first before jumping into creating files with Vi and Nano. Try running the following command in the terminal.
Nano should open up an empty file called
thisfiledoesnotexist.txt. Likewise, Vi will also create a new empty file if you substitute Nano with Vi. In my opinion, this is not the best way to create empty files,
touch is the better command for just creating an empty file. To create and immediately work with a file, if it is not just a quick one-line reminder you can use
cat for, I would go with Vi or Nano.
Park that File Somewhere Else!
Ok, so you made your file and filled it with your family's secret recipes only to realize you created it in the wrong directory. Don't worry, you don't have to delete it and start over. Linux offers a very simple command to mv, I mean move files around.
If you caught my not so subtle typo above then you guessed our next command is
mv command is used to move or rename a file,
mv [SOURCE] [DEST]. Let's go over renaming a file first.
mv somefile.txt somefile2.txt
Running that command in the directory where
somefile.txt is located is telling Linux to move that file to this new location which happens to be the same working directory just to a file with a different name. This is still moving the file but just within the same present working directory so we think of it as more of a renaming of the file. To move the file you would want to enter in the absolute path to the new location, something like this should work.
mv somefile.txt /home
That should move
somefile.txt into the
/home directory. Check if the file was successfully moved by entering the following command
ls /home | grep somefile.txt.
This isn't a huge surprise, but
mv will also move directories. Instead of a filename, you would enter the directory name as the source.
mv /foo/bar /home
The above will move the entire
bar directory to
/home. To move only the contents of
/home you would use the following command.
mv /foo/bar/* /home
This will move all the contents, but leave
foo, instead of moving
bar out of foo to
mv is can you guess what
Can You Make Me a Copy?
cp command works as you probably have already guessed, it duplicates a source file in the destination directory. The syntax is similar to
cp somefile.txt /foo/bar/
You can give it a new name as well by adding a new file name to the end of the destination.
cp somefile.txt /foo/bar/copy.txt
You can also copy a file to a new file in the same directory if needed.
cp somefile.txt somefile_copy.txt
Would make a copy named
somefile_copy.txt in the
To copy an entire directory you have to add the
-r or the recursive option to the command. This just tells Linux to copy everything in the directory.
cp -r test test_back
You could create a basic backup of a directory with this command before trying something risky, we'll get into some better ways to create backups in a later post but for a quick back up
cp works well. Another good use for
cp is to copy project templates to start a new project. I usually have a few project templates set up on my work computer, the base files I need to start a new project. To start a new project with all the files and directories you like you just need to
cp the template to the new project root. Automating this with a shell script to do a few other things (like
git init or
composer install) is a nice little time saver.
DANGER Will Robinson!
Before we get into this next section you need to remember something:
THE COMMAND LINE HAS NO TRASH CAN. DELETES ARE FOR REAL, SERIOUSLY WE'RE PLAYING FOR KEEPS HERE. 100% REAL TALK.
Deleting files from the command line removes the safety net of the trash can.
Deleting files from the command line removes the safety net of the trash can.
Ok, now that we have that covered, this next section is on removing files, for real from the command line. We all learn the hard way that the next command has no safety net, don't be like the rest of us.
Based on the last two sections I bet you are not surprised that
rm is the command to remove a file. Just like
rm is easy to use.
Say goodbye to somefile.txt forever, or until you create it again. You can also remove directories using the
-r recursive option as well, but this will remove everything in the directory as well as the directory itself.
rm -r /foo/bar
Will remove the
bar directory and all of its contents. Now if you're worried about someone taking your computer and trying to recover deleted files you may want to use a different command.
If You're Worried About Security Use Shred Instead.
Linux has another remove command,
shred this is far more secure than using
rm to delete a file or directory. Although
shred won't help on certain systems, for most Linux systems it is a secure way to remove files. What
shred does is overwrite the file and change its contents. Let's go through this step by step. Create a file and add some text to it. Next run the
shred command on that file.
Now print your file out to the command line using
cat testfile.txt, you should see a lot of gibberish, that's exactly what you want to see. There are a few options you can use with
shred that you should know about. First, you can change the number of times the file is overwritten (default is 3 times) by using
-n with a number (
-n 10 to overwrite 10 times). Ensure the file is deleted after it is overwritten using the
-u option. This is another command I use in a shell script to simplify removing sensitive files from my machine. Shell scripts are something we will cover later, but I have a file that is preset with the shred options that I prefer and can use from the command line like
trash.sh somefile.txt. The deeper you get into Linux you'll find yourself making scripts like these to simplify or automate tasks you frequently complete.
Searching for Answers
Like a lot of things in Linux, there are several ways to search for strings in text files. First, we'll go over a familiar command again
grep and a new one
To display a file with numbers lines you use the
nl command. Like
cat this command prints out the contents of the file, but includes line numbers, which is important if you are searching for occurrences of a string in a large file. Using what we already know (hint: piping and
grep) we can filter large files for specific strings and view their line number in the file.
nl somefile.txt | grep 'hello'
Try running that command on a file that contains the word hello, you will be shown the line where it is located. For large files combine that with a redirect to save the output to a file.
nl somefile.txt | grep 'hello' > hellosearch.log
Another powerful command for searching is
sed. This command will even let you search and replace strings in a file, really helpful for changing variables or passwords in a large file. Part of what makes
sed so powerful is the ability to search and replace using regex. For this basic introduction we won't use regex, but as you get familiar with Linux and regex remember
sed. Using the same search we filtered for with
grep our command would look like the following.
sed -n '/hello/p' somefile.txt
'/hello/p' portion of our command can be replaced with any regex expression, or you can replace
hello to search for a different string. Try the following command to see how useful this command is.
sed -n '/root/p' /etc/passwd
That easily finds the root entry into the
/etc/passwd file, you can see how many lines are in that file using the command
wc -l /etc/passwd. You can also use
sed to replace the given string by slightly changing the command you use.
sed s/[SEARCH]/[REPLACE]/g somefile.txt sed s/hello/HELLO/g somefile.txt
sed we are doing a search and replace and the
g flag says this is global, meaning all occurrences in the given file. Try the command above to change "hello" to "HELLO". This command saved me a lot of headache on a project recently when I needed to change hard-coded absolute navigation links to relative links. To save the text change make sure to redirect the output to a file. There are many additional options to use with
sed to specify things like change the nth occurrence of this string or change just the first occurrence.
|s/a/A/1||Change the # occurance (1, 2, 3, etc)|
|s/a/A/5g||Change from # occurance to the end of file|
|sed '1 s/a/A/'||Change on the # line only|
Wrapping Things Up
Well, that covers this installment of our Linux Basics. By this point of the series, you can navigate around in the command line, create new files and directories, edit those files, copy and move files and directories, remove files and directories, and search files for strings. You should feel good, that is a lot in two posts! If you're still here next week we jump into compressing files and directories and the basics of shell scripts, which will start taking some of what you have learned to start automating tasks. Thanks for reading!
A quick list of the commands we have covered in this series.
Welcome To Linux: whoami pwd ls cd locate whereis cat head tail more less cat [FILE] | grep "[TERM]" Touching Files and Making Dirs: touch cat > somefile.txt mkdir vi nano mv cp rm shred nl grep sed