Introduction to Linux

[Audio] Introduction to Linux Tsunglin Liu Department of Biotechnology and Bioindustry Sciences National Cheng Kung University Taiwan Hello, this is Tsunglin from Department of Biotechnology at National Cheng Kung University in Taiwan. Welcome to this online class of Linux, which is a necessary component of the course "Biological Big Data Analysis"..

[Audio] Linux Linux is an operating system like Windows or OS X of Mac. To analyze NGS data, you need to learn Linux because most bioinformatics tools are designed to run on Linux. There are different distributions of Linux and we are going to use Ubuntu. http://www.hobo-geek.com/2013/01/the-best-linux-distribution-2013.html Linux is an operating system like Windows or OS X of Mac. To analyze NGS data, you need to learn Linux because most bioinformatics tools are designed to run on Linux. There are different distributions of Linux as shown in the cute penguins below. We are going to use the left bottom one called Ubuntu..

[Audio] Connecting to Linux You can connect to my Linux server from your personal computer or laptop. If you use Windows, google for MobaXterm, find and install the free portable edition. You do not need to install Linux. Instead, You can connect to my Linux server from your personal computer or laptop. If you use Windows, google for MobaXterm, find and install the free portable edition. Note that you can always stop this video, make sure that you follow, then resume it..

[Audio] Once installed, run MobaXterm, start a new SSH session, and enter the remote host address (i.e., 140.116.48.90; port 1112). You can also specify your username (i.e., your student ID). Once your have installed MobaXterm, run it and start a new SSH session, then enter the remote host address (that is, 140.116.48.90; port 1112). You can also specify your username, which is your student ID. Note that the first letter of your ID is in capital and the case matters. Besides, some contents in the figures of the slides may not fit the descriptions. In that case, stick to the descriptions..

[Audio] Connecting to Linux server After setting the session, a new window will pop up, where the password (i.e., your account name) is requested. Note that when you key in your password, it does not show on the screen. Press enter after entering your password. After setting the session, a new window will pop up, where the password (i.e., your account name) is requested. Note that when you key in your password, it does not show on the screen. Press enter after entering your password..

[Audio] You should then see something as follows if your login is successful. The left window works like a file manager while the right one is the command line interface we will be using and is called a shell. You should then see something as follows if your login is successful. The left window works like a file manager while the right one is the command line interface we will be using and is called a shell..

[Audio] If you use Mac, go to Application -> Utilities and open “Terminal”. Use `ssh -p 1112 account@140.116.48.90` (see below for example, ssh stands for “security shell”) for connecting to the Linux server. Again, the password does not show when you type. If you use Mac, you do not need to install MobaXterm. Simply go to the Application, then Utilities, and open "Terminal". Issue the command as in the example below to connect to my server. Remember to replace my account name with yours. In the command, ssh stands for "security shell". Again, the password does not show when you type..

[Audio] You should now change your password using the command `passwd`. Please come up with a new password containing at least 10 characters of alphabets and digits. You need to enter both the old and new passwords. Again, the password does not show when you type. Once you login to my server, you should change your password using the command `passwd`. Simply type it and press enter and there will be a prompt for the current and new password. Please come up with a new password containing at least 10 characters of alphabets and digits. Again, the password does not show when you type..

[Audio] Basic commands command [options] [arguments] Note that there is always a space between any two items in a command line Now, you are ready to learn some basic Linux commands. Each command can take one or more arguments, and the options can be specified before the arguments. You will first learn the commands through a free Coursera course..

[Audio] Coursera course Hands-on Introduction to Linux Commands and Shell Scripting by IBM @ https://www.coursera.org/learn/hands-on-introduction-to-linux-commands-and-shell-scripting Free registration with your google account or other. The name of the Course course is "Hands-on introduction to Linux commands and shell script", and the course is made by IBM. You can search for the course or simply follow the link to find it. The course is free and you can get into it via clicking on the free registration and enter with your google account or another..

[Audio] Step 1: click on next Step 2: only audit the course instead of purchasing Once you get in the course, scroll down the step one items and click on the next icon. A seven day free trial message will then pop up. But please do not pay anything. Instead, click on the auditing link in the left bottom corner..

[Audio] Go through module 2 Please go through unit 2, which covers several Linux commands about retrieving system information, navigating in the Linux system, and managing files and directories. Stop the video now and come back when you are done..

[Audio] Text editor: vim https://opensource.com/article/19/3/getting-started-vim Before moving on to the next unit, please learn a popular text editor called vim via the link..

[Audio] Coursera course Go through module 3 Please also go through unit 3 to learn some more advanced commands. For the part of creating shell script, you can use the vim editor. Again, stop the video and come back when you are done..

[Audio] Viewing file content `less file`: view the content of file. In the viewing mode, “arrow up/down”: scroll up/down one line, “space or f”: scroll down one window, “b”: scroll up one window, “g/G”: go to the beginning/end of file, “/” + “pattern”: search for the pattern, <= extremely useful “n/N”: during pattern search, find the next/previous hit. Ex: view the content of the files “cpuinfo” and “meminfo” in the directory “/proc”. How many CPUs and how much memory are there in the server? There is a useful command not introduced in the Coursera course. It is called "less". You can less a file, which will bring you to a new viewing mode that shows the content of the file. In the mode, you can use the arrow up and down key to scroll up and down the file. If you like to turn to the next page, use the space bar. On the contrary, use the b key to go back to the previous page. To move to the beginning or ending of the file, use the small g key or capital G key, respectively. There is an extremely useful feature in the viewing mode, that is, you can search for a specific word by using the slash key, and you will see a slash showing in the bottom line. You can then key in the desired word, after which you will see the matches of words highlighted. To move to the next matched word, use the small n key. To move the previous match, use the capital N key. Now, try to get yourself familiar with the key strokes via the following exercise. That is, view the content of the files "cpuinfo" in the directory "proc" in the root. Guess how many CPUs are there in the server. Similarly, view the file meminfo in the same directory and guess how much memory are there in the server?.

[Audio] Documentation A sample run.sc file content It is essential to document whatever you have done during any data analysis. This is essentially the experimental workbook of computer scientist. Please use vi to create a file called "run.sc". From now on, copy whatever commands you issue into the file. For each command, please add a line to describe the function of each command and put a "#" before the line of description. You can check the example shown here. There are several advantages of recording your commands. For example, you will know every single piece of details in your data analysis. Therefore, when the result is skeptical, you can troubleshoot much more efficiently. In fact, you can change some parameters and re-run the subsequent analysis without much efforts. Similarly, if you need to analyze another piece of data, simply replace the file names and you can re-run the whole analysis easily. Specifically, you can make the file executable and run the whole analysis with one single command. When you like to publish, you also need the run script file to describe your analysis procedure. And because you tend to forget the details, the comment will be very useful to recall the original idea of each analysis step. Therefore, please try to add comments to each chunk of codes or even every line of command..

[Audio] Sample run script Here shows the content of another run.sc file, which contains some mistakes. Can you identify the mistakes? It is great if you realize that the command for copying a file is "cp" instead of "copy". This may occur if you manually type in the commands. Therefore, please always use the copy and paste function to record the command. To copy a command, simply highlight the command. To paste it, use the middle or right bottom of your mouse in the vim window. I'd like to remind you again that from now on you should record all the commands to a file run.sc. This will be the homework of this class..

[Audio] Transferring files To submit your homework, you need to first get the file run.sc from the server to your computer. This can be done via the file manager window of MobaXterm. First, you can tick on the "Follow terminal folder" square, which will update the folders and files in a directory in the file manager window when you move into that directory in the shell. To transfer a file from the server to your computer, find the file in the file manager window and drag it to your desktop. You may also drag a file to the file manager window to upload it to the directory in the server..

[Audio] Thanks for your attention! It is great if you have gone through the slides successfully. We will learn some more useful Linux commands in the next class. And please remember to submit the homework..