​## The Shell

​### 常用命令

cd、ls、pwd、echo、mv、rm、mkdir、rmdir

​#### 想做什么

```bash date ```

> Fri May 13 12:15:07 PM CST 2022

```bash echo hello echo hello > hello.txt echo hello >> hello.txt cat < hello.txt cat < hello.txt > hello2.txt ``` > hello

第 2 个命令：如果`hello.txt`不存在，新建后插入`hello`；如果存在用`hello`替换文件内容。

第 3 个命令：与第 2 个命令类似，唯一的区别在于，如果目标文件存在，不会覆盖文件内容，会在原来文本后附上`hello`。

Shell 通过环境变量搜寻可执行程序。`$PATH`、相对绝对路径、`ls`、`pwd`、`cd`。

```bash cd - ```

切换目录（2个）

```bash drwxr-xr-x ```

权限解释：

1. 第一个字符表示是否是目录、软链接。
2. 第二到四个字符表示当前用户的权限（是否可读、可写、可执行）。
3. 第五到七个字符表示当前组的权限（是否可读、可写、可执行）。
4. 最后三个字符表示除了当前用户和组以外的其他用户和组的权限。

Input stream、output stream。

`tee` 能够把输入输出 并把输出导入文件中。

​### 管道 `|`

```bash ls -l | tail -n1 ```

​### root user

```bash sudo su ```

`su`: super user

​## Shell Tools, Scripting

```sh $ foo=foer $ echo "It's $foo" It's foer $ echo 'It is $foo' It is $foo ```

注意彼此的区别。

一个功能更全的 mkdir+cd 函数：

```sh mc () { mkdir -p "$1" cd "$1" } ```

Shell 参数：

• `$0` \- Name of the script
• `$1` to `$9` \- Arguments to the script. `$1` is the first argument and so on.
• `$@` \- All the arguments
• `$#` \- Number of arguments
• `$?` \- Return code of the previous command
• `$$` \- Process identification number (PID) for the current script
• `!!` \- Entire last command, including arguments. A common pattern is to execute a command only for it to fail due to missing permissions; you can quickly re\-execute the command with sudo by doing `sudo !!`
• `$_` \- Last argument from the last command. If you are in an interactive shell, you can also quickly get this value by typing `Esc` followed by `.` or `Alt+.`

Shell globbing:

• 通配符
• `{}`

发现一个图片格式转换利器：`convert`，由 [imagemagick](https://en.wikipedia.org/wiki/ImageMagick) 提供。

运行（`sh script.py`）了以下文件，电脑死机了。

```py #!/usr/bin/env python

import sys for arg in reversed(sys.argv[1:]): print(arg) ```

Shell 脚本分析工具：[shellcheck](https://www.shellcheck.net/)。

[tldr](https://tldr.sh/) 工具。

ffmpeg 视频格式转换工具。

find 更丰富的文件/目录查找工具。[fd](https://github.com/sharkdp/fd): Simple, fast and user-friendly alternative to find.

locate 是哪个软件包的？

tree broot

hstr

take input from both arguments and STDIN: `ls | xargs rm`

​### 练习

1. 阅读 `man ls` 找到满足以下要求的命令：

解答：

```sh ls -ahlt –color=auto ```

1. 在 marco.sh 文件中写两个函数 `marco`, `polo`。`marco` 能够保存当前工作目录，`polo` 能够让用户无论位于哪个文件夹下都能回到 `marco` 所保存的目录下

解答：

```shell #!/usr/bin/env bash

currDir=null marco () { currDir=$(pwd) echo $currDir } polo () { cd $currDir pwd } ```

用 `source marco.sh` 将文件中的函数释放到当前 Shell 中。

1. 调试一个很少失败的命令（UNRESOLVED）

提供了一个脚本，不知道用来干嘛的。

```shell #!/usr/bin/env bash

n=$(( RANDOM % 100 ))

if [* n -eq 42 *]{.spurious-link target=” n -eq 42 “}; then echo "Something went wrong" >&2 echo "The error was using magic numbers" exit 1 fi

echo "Everything went according to plan" ```

提供的关键句子：

• runs the following script until it fails
• captures its standard output and error streams to files and prints everything at the end

1. xargs 将标准输出通过 `|` 作为标准输入，recursively finds all HTML files in the folder and makes a zip with them

Note that your command should work even if the files have spaces (hint: check -d flag for xargs).

```sh find . -name '*.html' -exec zip all.zip {} + ```

1. 找到当前文件夹下最近修改的那个文件

一个脚本：

```sh find . -type f -printf '%T@ %p\n' \

```

​## Editors (Vim)

​### Vim 哲学

> When programming, you spend most of your time reading/editing, not writing. For this reason, Vim is a modal editor: it has different modes for inserting text vs manipulating text. Vim is programmable (with Vimscript and also other languages like Python), and Vim's interface itself is a programming language: keystrokes (with mnemonic names) are commands, and these commands are composable. Vim avoids the use of the mouse, because it's too slow; Vim even avoids using the arrow keys because it requires too much movement. > > The end result is an editor that can match the speed at which you think.

​### Modal editing

Vim 有几种模式：

• normal Esc
• insert i
• replace R

• visual v

  • plain
  • line V
  • block `<C-v>`
• command-line :

`:help :w` 查看 `:w` 的相关内容，与 `:help w` 不同

​### Basics

1. 插入文本

正常模式下，输入 <kbd>i</kbd> 即可。

1. Buffers, tabs, and windows（split panes）
2. Command-line

• :q quit (close window)
• :w save ("write")
• :wq save and quit
• :e {name of file} open file for editing
• :ls show open buffers

• :help {topic} open help

  • :help :w opens help for the :w command
  • :help w opens help for the w movement

​### Vim's interface is a programming language

​### 移动

• find/to forward/backward {character} on the current line
• `,` / `;` for navigating matches

​### 选择

Visual modes:

Can use movement keys to make selection.

​### 编辑

• but for manipulating/deleting text, want to use something more than backspace

• e.g. `dw` is delete word, `d$` is delete to end of line, `d0` is delete to beginning of line

• e.g. `cw` is change word
• like `d{motion}` followed by `i`

• select text, `d` to delete it or `c` to change it

​### 计数

​### Modifiers

You can use modifiers to change the meaning of a noun. Some modifiers are `i`, which means "inner" or "inside", and `a`, which means "around".

​### 练习 Demo

开始：

```py def fizz<sub>buzz</sub>(limit): for i in range(limit): if i % 3 = 0: print('fizz') if i % 5 = 0: print('fizz') if i % 3 and i % 5: print(i)

def main(): fizz<sub>buzz</sub>(10) ```

结束：

```py def fizz<sub>buzz</sub>(limit): result = [] for i in range(0, limit): if i % 3 = 0 and i % 5 = 0: result.append('fizz-\nbuzz') elif i % 3 = 0: result.append('fizz') elif i % 5 = 0: result.append('buzz') else: result.append(str(i)) return result

print(fizz<sub>buzz</sub>(10)) ```

​### 自定义 Vim

```config " Comments in Vimscript start with a `"`.

" If you open this file in Vim, it'll be syntax highlighted for you.

" Vim is based on Vi. Setting `nocompatible` switches from the default " Vi-compatibility mode and enables useful Vim functionality. This " configuration option turns out not to be necessary for the file named " '~/.vimrc', because Vim automatically enters nocompatible mode if that file " is present. But we're including it here just in case this config file is " loaded some other way (e.g. saved as `foo`, and then Vim started with " `vim -u foo`). set nocompatible

" Turn on syntax highlighting. syntax on

" Disable the default Vim startup message. set shortmess+=I

" Show line numbers. set number

" This enables relative line numbering mode. With both number and " relativenumber enabled, the current line shows the true line number, while " all other lines (above and below) are numbered relative to the current line. " This is useful because you can tell, at a glance, what count is needed to " jump up or down to a particular line, by {count}k to go up or {count}j to go " down. set relativenumber

" Always show the status line at the bottom, even if you only have one window open. set laststatus=2

" The backspace key has slightly unintuitive behavior by default. For example, " by default, you can't backspace before the insertion point set with 'i'. " This configuration makes backspace behave more reasonably, in that you can " backspace over anything. set backspace=indent,eol,start

" By default, Vim doesn't let you hide a buffer (i.e. have a buffer that isn't " shown in any window) that has unsaved changes. This is to prevent you from " " forgetting about unsaved changes and then quitting e.g. via `:qa!`. We find " hidden buffers helpful enough to disable this protection. See `:help hidden` " for more information on this. set hidden

" This setting makes search case-insensitive when all characters in the string " being searched are lowercase. However, the search becomes case-sensitive if " it contains any capital letters. This makes searching more convenient. set ignorecase set smartcase

" Enable searching as you type, rather than waiting till you press enter. set incsearch

" Unbind some useless/annoying default key bindings. nmap Q <Nop> " 'Q' in normal mode enters Ex mode. You almost never want this.

" Disable audible bell because it's annoying. set noerrorbells visualbell t<sub>vb</sub>=

" Enable mouse support. You should avoid relying on this too much, but it can " sometimes be convenient. set mouse+=a

" Try to prevent bad habits like using the arrow keys for movement. This is " not the only possible bad habit. For example, holding down the h/j/k/l keys " for movement, rather than using more efficient movement commands, is also a " bad habit. The former is enforceable through a .vimrc, while we don't know " how to prevent the latter. " Do this in normal mode… nnoremap <Left> :echoe "Use h"<CR> nnoremap <Right> :echoe "Use l"<CR> nnoremap <Up> :echoe "Use k"<CR> nnoremap <Down> :echoe "Use j"<CR> " …and in insert mode inoremap <Left> <ESC>:echoe "Use h"<CR> inoremap <Right> <ESC>:echoe "Use l"<CR> inoremap <Up> <ESC>:echoe "Use k"<CR> inoremap <Down> <ESC>:echoe "Use j"<CR> ```

​### 扩展 Vim

​### 高级使用

> A good heuristic: whenever you're using your editor and you think "there must be a better way of doing this", there probably is: look it up online. {.is-success}

1. 查找替换

`:s` (substitute) command ([documentation](https://vim.fandom.com/wiki/Search_and_replace)).

• replace foo with bar globally in file

• replace named Markdown links with plain URLs

1. 分割窗口

`:sp`, `:vsp`

1. 宏 Macros

• first clear the macro with `q{character}q`
• record the macro, with `[cite/t:@character]` to invoke the macro recursively (will be a no\-op until recording is complete)

• Array of objects with keys "name" / "email"
• Use a Python program?

• Use sed / regexes

  • `g/people/d`
  • `%s/<person>/{/g`
  • `%s/<name>\(.*\)<\/name>/"name": "\1",/g`
  • …

• Vim commands / macros

  • `Gdd`, `ggdd` delete first and last lines

  • Macro to format a single element (register `e`)

    • Go to line with `<name>`
    • `qe<sup>r</sup>"f>s": "<ESC>f<C"<ESC>q`

  • Macro to format a person

    • Go to line with `<person>`
    • `qpS{<ESC>j@eA,<ESC>j@ejS},<ESC>q`

  • Macro to format a person and go to the next person

    • Go to line with `<person>`
    • `qq@pjq`

  • Execute macro until end of file

    • `999@q`
  • Manually remove last `,` and add `[` and `]` delimiters

​### 资源

​### 练习

1. 完成 `vimtutor`

Lesson 1.3: TEXT EDITING - DELETION `x` 正常模式下按下，删除光标下的字符

Lesson 1.5: TEXT EDITING - APPENDING `A` 正常模式下按下，自动跳转到句子末尾并转为 insert 模式。

Lesson 2.1: DELETION COMMANDS `dw` 删除一个单词（光标在单词首字母）

Lesson 2.2: MORE DELETION COMMANDS `d$` 删除从光标到一行结尾的部分

Lesson 2.3: ON OPERATORS AND MOTIONS 一个总结：`d motion` motion:

• w 删除到下一个单词的开始，不包括（下一个单词的）首字母
• e 删除到当前单词的结束，包括当前单词的最后一个字符
• $ 删除从光标到一行结尾的部分

Lesson 2.4: USING A COUNT FOR A MOTION `2w` 向前移动2个单词，光标在单词首字母 `3e` 向前移动3个单词，光标在单词尾字母 `0` 移动到一行的开头

Lesson 2.5: USING A COUNT TO DELETE MORE 一个总结：`d number motion` motion:

• d2w 删除光标之后的2个全大写单词

Lesson 2.6: OPERATING ON LINES `dd` 删除光标所在行 `2dd` 删除光标所在行+之后的一行

Lesson 2.7: THE UNDO COMMAND `u` 撤销最后一次更改 `U` 撤销当前行的所有更改

Lesson 3.1: THE PUT COMMAND `p` 粘帖之前删除的内容

Lesson 3.2: THE REPLACE COMMAND `rx` 将光标下的字符替换为 x

Lesson 3.3: THE CHANGE OPERATOR `ce` 删除光标到单词结束的部分，自动转为 insert 模式方便修改当前单词 `cc` 清空光标所在行的内容

Lesson 3.4: MORE CHANGES USING c 一个总结：`c number motion` motion:

• w word
• $ 一行末尾

Lesson 4.1: CURSOR LOCATION AND FILE STATUS CTRL-G 显示光标所在行、文件状态 G 来到文件最后一行 gg 来到文件第一行 `num + G` 跳到目标行 > Type the number of the line you were on and then G. This will return you to the line you were on when you first pressed CTRL-G. {.is-info}

Lesson 4.2: THE SEARCH COMMAND / 搜索 n 再次向前搜索 N 再次向后搜索 ? 向后搜索 CTRL-O takes you back to older positions, CTRL-I to newer positions

Lesson 4.3: MATCHING PARENTHESES SEARCH % 将光标放到括号（`(, [, {`）的前半部分，按下 % 后光标会跳到对应括号的后半部分

Lesson 4.4: THE SUBSTITUTE COMMAND `:s/old/new/g` 替换单词 `:#,#s/old/new/g` # 代表改变的行号范围 `:%s/old/new/g` 改变文件全部内容 `:%s/old/new/gc` 改变文件全部内容，逐个确认

Lesson 5.1: HOW TO EXECUTE AN EXTERNAL COMMAND `:! command` 执行外部命令

Lesson 5.2: MORE ON WRITING FILES `:w TEST` 在当前文件夹下新建一个名为 TEST 的空文件 `:!ls` `:!rm TEST`

Lesson 5.3: SELECTING TEXT TO WRITE `v` -> `:w FILE` 将选中的部分写入 FILE 文件，如果文件已经存在用 `:w! FILE` 可以覆盖

Lesson 5.4: RETRIEVING AND MERGING FILES `:r FILE` 将 FILE 插入当前文件，`:r !ls` 将 ls 的输出插入当前文件

Lesson 6.1: THE OPEN COMMAND `o` open up a line BELOW the cursor and place you in Insert mode `O` open up a line ABOVE the cursor and place you in Insert mode

Lesson 6.2: THE APPEND COMMAND `e` 移动光标到下一个单词的最后一个字符 `a` 移动到单词的下一个字符位，并进入 Insert 模式 `A` 移动到一行的末尾，并进入 Insert 模式

Lesson 6.3: ANOTHER WAY TO REPLACE 光标位于想替换的位置 -> `R` -> 输入替换内容

Lesson 6.4: COPY AND PASTE TEXT `v` -> 选择待复制文本 -> `y` -> 移动光标到待粘贴位置 -> `p` `yy` 复制整行 `yw` 赋值一个单词，光标要放在单词的开头

Lesson 6.5: SET OPTION `/ignore` 搜索 ignore `:set ic` 'ic'(Ignore case) `:set hls is` 'hlsearch' 'incsearch' `:set noic` 关闭 Ignore case `:nohlsearch` `/ignore`̧ 仅一次 Ignore case

Lesson 7.1: GETTING HELP `<HELP>`，`<F1>` 按键，`:help` CTRL-W CTRL-W 跳到另一个窗口 `:q` 退出当前窗口

Lesson 7.2: CREATE A STARTUP SCRIPT `:e ~/.vimrc` -> `:r $VIMRUNTIME/vimrc<sub>example</sub>.vim` -> `:w`

Lesson 7.3: COMPLETION `:set nocp` -> `:!ls` -> `:e` -> CTRL-D

1. Download our [basic vimrc](https://missing.csail.mit.edu/2020/files/vimrc) and save it to `~/.vimrc`. Read through the well\-commented file (using Vim!), and observe how Vim looks and behaves slightly differently with the new config.

2. Install and configure a plugin: [ctrlp.vim](https://github.com/ctrlpvim/ctrlp.vim).

   1. Create the plugins directory with `mkdir -p ~/.vim/pack/vendor/start`
   2. Download the plugin: `cd ~/.vim/pack/vendor/start; git clone https://github.com/ctrlpvim/ctrlp.vim`
   3. Read the [documentation](https://github.com/ctrlpvim/ctrlp.vim/blob/master/readme.md) for the plugin. Try using CtrlP to locate a file by navigating to a project directory, opening Vim, and using the Vim command\-line to start `:CtrlP`.
   4. Customize CtrlP by adding [configuration](https://github.com/ctrlpvim/ctrlp.vim/blob/master/readme.md#basic-options) to your `~/.vimrc` to open CtrlP by pressing Ctrl\-P.
3. To practice using Vim, re\-do the [Demo](https://missing.csail.mit.edu/2020/editors/#demo) from lecture on your own machine.
4. Use Vim for all your text editing for the next month. Whenever something seems inefficient, or when you think "there must be a better way", try Googling it, there probably is. If you get stuck, come to office hours or send us an email.
5. Configure your other tools to use Vim bindings (see instructions above).
6. Further customize your `~/.vimrc` and install more plugins.
7. (Advanced) Convert XML to JSON ([example file](https://missing.csail.mit.edu/2020/files/example-data.xml)) using Vim macros. Try to do this on your own, but you can look at the [macros](https://missing.csail.mit.edu/2020/editors/#macros) section above if you get stuck.

​## Data Wrangling

```sh ssh myserver journalctl

```