Linux — Shell tricks

Photo by Vino Li on Unsplash

A Shell provides you with an interface to a set of commands and functionalities that are supported by your system. With the shell, you can execute commands and utilities available in Linux and other UNIX-based operating systems that can greatly improve your performance in your daily tasks.

In this post, I going to delve into some tricks that can improve your experience with the shell and can help you with some common problems.

Note. Bash is the default shell on most Linux distributions, for this reason, some tips are specific to this one.

Trick 1 — Stdin as a file argument

Some commands receive a list of filenames as arguments to generate some kind of result, one example of this is the wc command.

Format:

wc file1 file2 …

Note. The wc is a command that receives a list of filenames and by default returns the number of words in the files.

This is fine until you need to pass some short text as if it were the content of a file, a case in which you have two main options:

1. Create a temporal filename manually with the expected content and use the wc command.

cat > /tmp/temporalFile
wc !$

This is the more straightforward way to generate a file with content that you write at that moment, or that you have on the clipboard. The first part (cat > /tmp/temporalFile ) it’s a shortcut for creating a file in the /tmp/temporalFile path, once you have executed the command, everything you type will be taken as the contents of the file. It’s necessary to press CTRL + D to mark the end of the file (also known as \04 or 0x4, END OF TRANSMISSION in Unicode).

Finally, wc !$ is going a execute the wc command with the last argument of the last command executed, in this case, this is the same as writing /tmp/temporalFile.

There is an additional detail to which it is important to make reference. The file it’s located in the /tmp folder, this directory by default is cleaned generally at the reboot of the machine, but this is not strictly necessary. Exists some alternative directories like /var/tmp/ that it’s used for temporal files, but in this case, the files are persisted for more time (after the reboot of the machine in most cases).

2. The second alternative is to use the stdin as a file argument to wc command.

In Linux, each process has a related concept named standard input stream (stdin). This is an abstraction that defines what is the device/source where the input of the command comes from, by default the stdin is the keyboard.

If you replace some argument that expects a filename with the ‘-’ character, this is going to take the stdin as the content of this virtual “temporal file” called “-”.

Example:

wc file1 - file2

Note. After entering this command, you can type many letters as are part of some temporal file. When you finish entering the content, you can press CTRL+Dthat generates the EOF (End Of File) character, this represent the end of the content.

In this example, the wc command line reads file1, then waits while you write to the stdin and press CTRL+D, and finally reads the content of file2. The final output is the sum of the numbers of words in the file1, plus the number of words enters by the stdin, plus the number of words in the file2.

Trick 2 — Use the output of another command as a file argument

Similar to the last example, if you need to put the output of some other command as a file argument, you can use the concept of process substitution.

With the help of process substitution, you can generate a temporary file containing the output of another command that can be passed as if it were a file.

Format:

command_expect_filenames file1 file2 <(command ...)

Where the command_expect_files is a command that receives as arguments a list of filenames, and <(command ...) is a process substitution that wraps a command, gets her output, and returns the path for a temporal file that contains the output.

Example:

wc file1 <(echo “hello world”) file2

In this example, the wc command reads the content of file1, then the output of the echo “hello world" and finally the content of file2.

How works?

When you wrap a command with <(...)bash generates a temporal file in a path like /dev/fd/64, then execute your wrapped command, put the output in this temporal file, and finally replace <(...) with the filename of the temporal file, in this case, /dev/fd/64.

Trick 3 — Avoid conflicts with filenames that start with a dash

Sometimes you have a filename that starts with the ‘-’ character, and you need to print the content of this file or pass the filename as an argument to some other command.

The command options are usually prefixed by ‘-’ and ‘ — ‘, if some argument like the filename starts with ‘-’, this will be taken as an option. To avoid this, you can use ‘ — ‘ to mark the end of the command options.

Example:

cat -- -myFile.txt

In this case, you print the content of the -myFile.txt file.

Trick 4— Re-run commands

Sometimes you run a command requiring administrator permissions but forget to add the sudocommand, resulting in a permissions error. Instead of typing all the commands again, you can use:

sudo !!

This command runs sudo and adds to the end the last command executed (thanks to the expansion of !! ).

You can run commands other than the last one, for example, re-run the command you typed N commands ago.

!-N

Or re-run the N command in your history.

!N

Note. Each shell instance has a related history, this history contains all the commands that are executed and allows make some tricks like re-run old commands, make a reverse search with CTRL+R , etc.

Trick 5— Re-use command arguments

You can reuse arguments of the commands saved in the shell history, for example, if you create a directory, is very probably that you need to go to this directory, so you need to type another time the directory name like:

mkdir very-large-directory-name
cd very-large-directory-name

Instead of duplicating the argument of the mkdir command, you can use !$ to retrieve the last argument of the previous command:

mkdir very-large-directory-name
cd !$

Where !$ is the last argument of the previous command, in this case, the string “very-large-directory-name”.

This is not the only shortcut for getting an argument from the previous commands, exists other alternatives:

• !$ - last argument from the previous command
• !^ - first argument (after the program/built-in/script) from previous command
• !* - all arguments from the previous command

Trick 6— Configure the bash

To customize your shell exists two main files:

• ~/.bash_profile
• ~/.bashrc

Note. The ~ is a shortcut for your home directory.

In some tutorials, some have a preference for one or the other but the principal difference is when these files are executed. In the case of ~/.bash_profile , this file is executed each time you log in, in contrast to ~./bash that is executed every time you open a new shell.

They serve different purposes. Things that are specific to your login session should go inside .bash_profile, and those that are specific to the bash shell itself should go to .bashrc.

When you change these files, you need to force the re-execution of this file in the current shell for the changes you have made to take effect, to do that, you need to use the source command:

source ~./bash

Trick 7 — Ignore the first N lines

In some moments you need to ignore the first N lines of some file, maybe a document header like the cases of a CSV file, or sometimes are only a few empty lines. For both cases, you can quickly remove these lines and show the rest of the content with the support of the tail command.

By default, the tail command will show the last n rows, but if you specify the option -n with a number that starts with the + symbol, like +5 , the first 5 lines are going to be skipped.

Example:

tail -n +10 dataset.csv

In this example, the tail command is going to skip the first 10 lines and print the rest of the file content.

Trick 8 — Track the content of a log file

Show the content of log files can be very straightforward but sometimes you need to see the content of the file in real-time.

To solve the problem, you can use the watch command.

Example:

watch cat log.txt

In this example, the watch command executes the commandcat log.txt each second, in other words, you print the entire log.txtcontent each second.

Although this command does the job, it is not the best option. You can use the tail command with the -f option to track only the new lines that are appended to the file, this reduces the overhead of printing all the file content each second.

Example:

tail -f log.txt

Thanks for reading!

Please share any of your own favorite shells tricks by leaving a response below!